How to Store and Manage BIG Data?

While I mentioned in my previous blog that any size of data is no problem, I often get questioned upon how to store and manage the huge volumes. This is a typical concern of an enterprise faced with increasing data size.

Storage vendors have seen and known this problem as it grew, and have scaled-up or rather scaled-out to help handle this massive growth. Both NAS and SAN vendors have gone beyond the traditional methods of upgrading the storage infrastructure by adding additional shelves and disks. The challenge that the traditional method has is that you end up upgrading capacity with shelves and disks with limited enhancements in processing power. This ends up in performance reduction.

The Scale-out method helps upgrade the storage by adding new nodes which include processing power, memory and capacity, thereby keeping the overall performance consistent with practically no dip in user experience. This is true for both SAN and NAS based storages. These storages can be expanded to PBs on a single storage, or even a single file system, by simply plugging in a new node. It is viable commercially also, as the cost per GB goes down as you keep adding more nodes.

So don’t worry about handling your Big Data as the storage devices are now available to store them more efficiently.

How much Data is good for business?

When you talk of sources of data generation, there is an endless list. Any business stream would have a long list to show how data is getting generated and how much data is being generated. Often businesses get scared with so much of data as they think handling it is a mammoth task. Indeed it is a mammoth task as it needs good investments and infrastructure to handle it. However, if utilized properly, the benefits are much higher. The way businesses are competing, it would soon become inevitable to handle it carefully.

The more data you have, more opportunity you get to see how your products, services and customers behave. There are many examples of business being able to analyse their data patterns and offer more discounts or value added services to give their customer a delightful experience. The new databases handling this Big Data have come up with Multi Parallel Processing technologies and the new applications to handle unstructured data ensure that even if you have PetaBytes of data, you can still do real time analysis and produce results in nano seconds.

Let us enjoy this new revolution on the way technology and businesses are getting shaped up and reap the benefits of these.