The data explosion

The data explosion

For the last several years, the Big Data momentum has steadily increased as large firms see the value of investing in data analytics. According to Wikibon, the US Big Data market reached $18.8B in 2013, with computer hardware encompassing 38 percent of IT sell-through. For large enterprises, the upfront capital investment of compute and networking clusters provide quick financial returns, as these companies are able to leverage their IT investment to reduce operating expenses or monetize professional services.

Financial firms use data analytics to create an environment for fraud modeling in order to detect fraud in real time across millions of transactions and disparate systems. Healthcare providers enter patient history and symptomology — and while the patient is still in the office — can sift through millions of archived patient records for relevant outcomes. Many large firms go one step further by coupling Big Data with HPC clusters to create a foundation for High-Performance Data Analytics (HPDA). In fact, IDC predicts that by 2015, 7.3 percent of all HPC installations will be focused on HPDA. Insurance companies use this hybrid approach to provide faster means of providing calling/online quotes down to 100ms, while oil and gas firms use similar clusters for analytics and visualizing of field data for internal and external use.

But what about the small- to medium-sized business (SMB)?  Do you see value with an investment in Big Data? Can you afford the upfront cost with ROI uncertainty? The truth is many SMBs are looking at Big Data — as a necessity rather than an option — as their ability to stay competitive is heavily predicated on their ability to use the data they capture. The greater the data volume collected, the greater the need for a firm to manage and capitalize on its storage or compute capacity for accurate and timely decision making. SMBs focused on logistics or shipping can use Big Data solutions to reroute trucks to create efficient routes, alert customers to deliveries and forecast and price services. Boutiques or restaurants can use data analytics to manage leads or inputs through social media for more efficient direct marketing, while smaller healthcare providers can leverage similar solutions to keep in accord with federal guidelines, or to automate patient communications for proactive scheduling and improved relationship management.

On the education front, Lenovo has helped several universities in creating Hadoop clusters to assist with research projects surrounding bioinformatics, molecular simulations, genomics or other data science applications. Below you will see a reference architecture based on a 10Gb Ethernet (GbE) fabric, which is capable of scaling over 1,000 nodes. The cluster utilizes 2U Lenovo ThinkServer RD640 systems for their large internal storage capacity. Through Lenovo’s partnership with Extreme Networks, 24-port 10GbE switches are deployed for a low-latency network of spine, leaf and core switches. This solution offers a cost-effective cluster focused on scalable compute and fabric elements so customers can pay as they grow, versus growing into capacity and increasing their total cost of ownership. This design tenant has been key, as many of the end users have been smaller sub-groups within larger university departments that are just starting to invest in HPDA. These installations are proof cases for larger university grants.

With the exponential rise of mobile data consumption, and the daily ingestion of data from point-of-sale devices, SMBs have proportional access to critical data that can empower the increased efficiency of daily operations, or allow companies to fully optimize their marketing and CRM tools. SMBs can now implement a backend Hadoop cluster as the one above, and tie it to analytics software packages such as QuickBooks, Zendesk, Google Analytics, Tranzlogic or other comparable tools. 

Hopefully you found this blog insightful and will consider Lenovo servers and solutions for your future Big Data deployments. For information on Lenovo servers and Extreme Networks switching products, visit the ThinkServer home page. For further information on Lenovo’s solutions for Big Data or HPC, please contact Edgar Haren at