PowerFlex FTW: Tales From the Field – August ‘21

“The Fraud Detection System”

We were approached by an existing Vblock customer in spring of 2017. They were looking for a new hyperconverged system to support their new online fraud detection system, which was imperative to their business since they ingested millions of feeds per day from all over the country. They needed to verify the quality and security of this data and had already chosen an application to perform this function. Their ask was simple – what architecture should it sit on? As mentioned, they already had an existing Vblock buildout plus small pockets of legacy storage. They simply wanted to learn more about hyperconverged systems and discuss whether this technology would fit their application needs.

Like most of our customers, they had many questions about (and a slight proclivity towards) hyperconvergence and the sales team was given 30 minutes to explain the differences between legacy storage and a software defined datacenter.

The Dell team first asked questions around the application to understand the architecture. Whatever solution they deployed must be able to process millions of feeds daily as this new fraud detection system was heavily database dependent and must not faulter in relation to storage performance.

Once they understood the application a little better, the Dell team dove into the differences between:

1. A legacy 3-tier architecture vs.

2. An appliance based HCI solution vs.

3. A fully integrated rack solution

The nuances of each solution were quickly laid out on a whiteboard. Not only did the Dell team explain the technical differences from a high level, we also explained the operational benefits, similar to the example below:

Any of the above architectures would have worked just fine based on Dell’s understanding of the application. However, this customer had issues with the cumbersome upgrades on their existing Vblock systems and were looking at a simpler infrastructure to manage. We found out later that this was the impetus for their desire to discuss hyperconvergence.

With traditional 3-tier being out of the mix, we explained what a VxRail solution might look like for their application. They had concerns about how large they may grow their data and therefore had no interested in creating multiple clusters to accommodate for this growth.

We then showed them a rack-based solution running PowerFlex and how that would scale. This fell more in line with their ideas around scalability, plus they were intrigued by the ability to upgrade the entire stack with a few clicks. Remember, they were a current Vblock customer, so the ability to automate the upgrades of just servers, switches and software was far more intriguing than what they were currently doing with their manual Vblock upgrades.

We also walked through the write architecture of a vSAN based system vs a PowerFlex design and showed sample performance statistics. vSAN showed well and would have sufficed, however the PowerFlex stats show 4-6 times performance improvement with seemingly unlimited scalability.

Again, this was an educational session. The account team wasn’t pushing any specific solution, but it became obvious to everyone in the room that a PowerFlex rack solution would be the preferred route for this specific application.

We left the meeting (AFTER JUST 24 MINUTES!) and started working on the new design with the partner. Yes, this meeting was only 24 minutes but shows the power of whiteboarding and the sales team’s deep understanding of Dell and competing technologies.

The customer’s first deployment started with 5 hyperconverged nodes, 2 access switches, a 3-node management cluster and 1 mgmt switch. The partner was adamant about this customer not needing to exceed 1 cabinet worth of nodes, hence the reason for the lack of an aggregation layer. We explained why this might be a concern for future upgrades, but the solution proceeded as designed.

Duplicate solutions were deployed a few months later to 2 separate locations. The customer was exceedingly happy with their initial testing and the amazing performance these 5 HC nodes provided. The real surprise came when the application vendor showed up install their portion.

This specific app ran a proprietary database on the backend that was optimized for the fraud detection application. Every deployment of this app to date had used that built-in database, however once the vendor saw what they were getting for performance, they jokingly told the customer “You can run whatever database you want! This stuff is fast!”. Since the customer was a Microsoft shop, they asked if they could port it to MS SQL….and the vendor did! The application loaded perfectly fine and the very first instance of this application running on MS SQL was deployed to both locations.

The initial results were jaw dropping – millions of transactions per hour were going through their PowerFlex system without a hiccup. This far surpassed their upper limits on throughput, but they weren’t done. Because they were getting such astounding performance, they needed to upgrade almost immediately due to an unforeseen influx of data that was being processed far quicker than previously calculated.

The Dell team facilitated the addition of 4 new HC nodes within 3 months, 10 more HC nodes a year later and finally 4 new storage-only nodes, cabinets and switching 18 months later. This last upgrade taught everyone a lesson – since the partner originally scoffed at the need for an aggregation layer, the customer had to take an outage to deploy the aggregation and new access switches with their new nodes. The PowerFlex engineering and sales teams warned about this but we were at the mercy of the partner in this case. The customer ended up being OK as they were able to failover their prod to DR and vice versa for these upgrades.

A few months after deployment we were talking with this customer to see how things were running. They were still impressed with the performance and wanted to talk about a Vblock replacement running on a new PowerFlex solution. That was an even quicker conversation and a few months later the sales team deployed duplicate 300TB+ systems as a first step towards a Vblock succession plan.

This whole datacenter design was formulated after a quick conversation with the customer about a single application. The value of PowerFlex was quickly ascertained by the customer during this meeting and became their go-to standard for every application in their datacenters. These first 2 PowerFlex rack systems were deployed with only 17 TB of data per side. Today this customer has over 1.2 PB running PowerFlex in their datacenters and are always looking at expanding that footprint. This sort of growth is very common with PowerFlex solutions and the reason it needs to be understood. Never do you have to worry about your solution growing too big or being concerned about PowerFlex not being able to take on a specific workload. This datacenter consolidation concept is one of the big reasons PowerFlex is so popular today.

In the end we accomplished multiple goals for the customer: they had 1 solution running every application in their datacenter, they’d simplified operations dramatically in their day-to-day administration as well as lifecycle management and, most importantly, they could sleep well at night knowing their PowerFlex solution could handle any workload they threw at it.