Hyper-converged, scale-out, black-box, roll your own; on-premises enterprise data storage is a bit more than a hobby of ours here at Symbio. Until recently, most medium scale enterprises bought their storage from one of a few vendors; HP, Dell/Compellent, Nimble, Netapp, EMC, etc. These products arrive as a more-or-less plug and play appliances, and are usually fully supported by their respective vendors. They are also expensive. Symbio got it’s start using a home-brewed Linux based storage appliance we built ourselves because we couldn’t afford anything else.
This was great until I found myself debugging production storage issues during (literally) the birth of my first child. After that experience and at the urging of my wife, we committed to a Compellent investment. The cost was extreme for a company of Symbio’s size (the initial purchase price was nearly 20% of our annual revenue, annual support renewals were almost 10% of revenue the first couple years.) The Compellent served us well, and it’s stability allowed us to reliably quadruple the size of our customer base. However, after the first couple years, we outpaced it’s performance abilities.
As technology decision makers, we’re used to basing a storage platform choice on metrics like cost-per-GB, IOPS, features and support reputation. However, we wanted to take a deeper look at our options this time as it’s clear the very ways we think about storing data are changing. This presents fantastic opportunities for cost-control and adding new capabilities, but also present new risks for businesses to contend with. This series of articles explores a few of the emerging trends in on-premises enterprise storage, ideal applications for each technology, and details our specific experiences of each of the approaches. Symbio is currently running all of these systems in a production capacity.
Hyper-converged – VMware vSAN: Solutions like vSAN or Nutanix place the storage directly inside your processing hosts, then use software on the “back end” to provide for performance and data redundancy. These solutions can offer extreme performance at a moderate cost, but dramatically change failure models and require very careful and experienced planning to implement reliably. Symbio uses vSAN as our primary storage for high performance needs; specifically database and virtual desktops.
Traditional “Black Box” SAN – Nimble Storage: The “usual” enterprise approach with an appliance provided for and supported by a vendor. This approach offers moderate performance, generally very high reliability, and is generally compatible with existing thinking regarding failure modes (storage and compute can be thought of as isolated components of an overall system). Cost is often high compared to alternative approaches, but the “one-ass-to-kick” nature of the support can be of tremendous value to shops that lack deep IT talent. Symbio uses Nimble for our “general purpose” workloads; things that don’t demand extreme performance or capacity but where we derive value from some of the “nice to have” features that aren’t available on our other solutions.
Open Source, Scale Out – RedHat Ceph: Ceph is rapidly emerging as a favorite low-cost, high capacity solution for shops with strong technical capability. Ceph uses a mathematical model to decide where to place data on the underlying disks, and clients talk directly to the disks to request data. This means your controller is no longer a bottleneck or failure point as with a traditional SAN. Ceph can scale to Petabytes simply and without the enormous cost a traditional SAN would require. Ceph is open source, community supported (though enterprise support is available), and will run on commodity hardware. Symbio re-purposed all our old Compellent hardware into Ceph clusters (which, yes, we will write a blog post about) and is used as a low-performance high-capacity storage for backups and our SymbioVault off-site backup product. Ceph is presently limited in some very important ways, however.
We’ll explore each of these technologies in depth in the coming series of articles.