Blogs & Articles

Monday, Aug 16, 2010

Navigating the Digital Haystack

I recently came across a fascinating research study which cited some incredible statistics relating to the amount of data companies generate. 

According to ESG, “nearly 50% of mid-market companies (100-999 employees) report more than 10 TB of total data volume and nearly 50% of enterprise organizations (1,000 employees or more) cite more than 100 TB.” The study went on to say that data volumes are growing by 20% or more per year, meaning that data volume is doubling every two to five years.

Armed with this information, I dusted off the calculator and started producing our own set of intriguing numbers. 

According to the 2007 Census, there are 639,288 US companies with >20 employees, broken down as follows:

Businesses 20-99 Employees (2007 US Census) 83.3% or 532,391
Businesses 100-499 Employees (2007 US Census) 13.8% or 88,586
Businesses 499+ Employees (2007 US Census) 2.9% or 18,311
Total Market 100% or 639,288

Based on ESG’s study, we assumed an average of 5TB of data resided in the 20-99 segment, 10TB in the middle segment, and 50TB in the 499+ plus market. Here’s the result – which shows a staggering 4,463,365 TB of data volume.

Average TB of Data 20-99 Employees @5TB, 2,661,955TB
Average TB of Data 100-499 Employees @10TB, 885,860TB
Average TB of Data 499+ Employees @50TB, 915,550TB
Total Market - TB of Data Stored 4,463,365TB

With massive amounts of data on hand, companies everywhere are seeking new, better and more economical ways to protect and store their data, and many are finding that the Cloud presents an excellent option. It’s accessible, inexpensive, secure and reliable, and its scales effortlessly. Still, as appealing as the Cloud is today for those seeking data storage solutions, I believe that there is one important element missing in the Cloud as we know it today - Data Storage Assurance.

Today’s challenge is less about how and where companies store their burgeoning amounts of data, and more about how they maintain the integrity of that data and how easily that data can be found and retrieved. Companies need to ensure clean, trusted audit trails when embracing the Cloud for data storage, and once they’ve assured the integrity of their stored data, they need to ensure they can easily and rapidly navigate the massive amount of files that comprise their data – their “digital haystack.” Only then will they be able to find the data they need, when they need it, and with the identifying tags that prove to any and all that their stored files are authentic.

The Digital Haystack isn’t going away, and neither is the Cloud. And if you do put the two together in the right way with strong data audit trails that deliver Cloud storage assurance, the relationship will work now and for years to come.


22nd  - De-risking your DR
27th  - DRaaS becomes mainstream and looks to OpenStack and hybrid deployments for growth
5th  - Nobody Ever Got Fired For Buying … Based on Gartner Magic Quadrants
11th  - We welcome Google and the next evolutionary step in DRaaS
18th  - Microsoft Acquisition Signals Dawn of a New Era for RaaS
3rd  - RaaS, say hello to the Public Cloud
20th  - Recovery as a Service—Earning its Very Own Gartner Magic Quadrant in Only 4 Short Years
3rd  - The Disappearing Data Center
16th  - Lessons from the NFL – The RaaS Opportunity
13th  - The RaaS Secret Sauce: The Channel
23rd  - CA Throws Down The Gauntlet. Has the race for Cloud Data Protection supremacy already been won?
11th  - RaaS – from Evolution to Revolution
16th  - Storage Wars…starring every cloud services provider on the planet
11th  - HPC in the Cloud – Guest Blog: “Telecom Sector Embraces the Cloud”
3rd  - Today’s Forecast for the Telecom Sector: Clouds, Clouds and More Clouds
27th  - The CEO’s (Unused) Secret Weapon
31st  - The Race to Zero. My Cloud Prediction for 2011
30th  - When Brrr…becomes Grrrr!
24th  - Ten gifts from the Cloud this Holiday Season - with a money-back (ROI) guarantee.
7th  - The Tipping Point – or how the Cloud is about to go “main-stream”
20th  - Lessons from History…or how emerging technology can destroy a thriving business – if you let it.
14th  - Today’s Forecast: Lots of Cloud - With 90% probability of Enterprise Engagement
13th  - HPC in the Cloud – Guest Blog: “The Cloud Flattens Technology Landscape…Should Enterprise Worry?”
7th  - Cloud Invaders
25th  - Telcos embrace the Cloud. Next Stop – Mainstream
23rd  - The next-generation data center … aka The Cloud.
16th  - Navigating the Digital Haystack
10th  - Computer Technology Review
3rd  - Get your head OUT of the cloud
28th  - HPC in the Cloud – Guest Blog: “Digital Breadcrumbs: Audit Trails in the Cloud”
19th  - The day the cloud went dark
12th  - Get your head INTO the cloud
27th  - HPC in the Cloud – Guest Blog: “What Recovery as a Service (RaaS) Means for Enterprise”
20th  - The Birth of RaaS or Recovery as a Service