Ace Data Corporate Film premieres at CIO Power List event in Lonavla.

https://www.youtube.com/watch?v=4GdVcTCwNBQ

Learn from your mistakes, analytically

As discussed in the previous blog, every business has a risk and needs to find how to decrease the risks. Each negative outcome that occurs presents an opportunity from which to learn new things. In a small scale business, these decisions could be easy and more mind driven being controlled by a set of managers only. As the businesses grow beyond a size, predicting the future course of action and avoiding the past mistakes becomes important. 

The enterprise integrates the predictive model’s scores in order to act upon what has been learned. At each step, the predictive scores foresee where a “blunder” may incur unnecessary risk, thereby guiding the organization to avoid it. In this way, predictive analytics delivers a complete data-driven system for risk management. 

Predictive modeling capabilities are scientifically proven and have benefited from decades of advancement. With a tenured track record of success, predictive analytics boasts mature software solutions that deliver this technology to – and integrate it with – the modern enterprise.

Predictive Analysis used by Insurance Companies

All businesses are run at a risk. Risk is the way business is managed. Every decision an organization takes impacts the risks an enterprise can withstand like the risk of customer defecting, of not responding to an expensive glossy mailer or offering a huge retention discount to a customer who was not leaving even otherwise and in turn missing out on a critical customer who leaves.

The data driven means to compute risk of any type of negative outcome in general is predictive analysis. Insurance companies have used this very well. Insurance companies are augmenting their practices by integrating predictive analysis in order to improve pricing and selection decisions. 

The actuarial methods that enable an insurance company to conduct its core business perform the very same function as predictive models: Rating customers by chance of positive or negative outcomes. Predictive modeling improves on standard actuarial methods by incorporating additional analytical automation, and by generalizing to a broader set

Predictive Analysis and Big Data

Predictive analytics is an enabler of big data: Businesses collect vast amounts of real-time customer data and predictive analytics uses this historical data, combined with customer insight, to predict future events. Predictive analytics enable organizations to use big data (both stored and real-time) to move from a historical view to a forward-looking perspective of the customer.

For example, historical preferences of consumer can be analyzed from their usage patterns and promotional offers can be planned accordingly. The historical data can help predict which promotional offer would be most useful. 

There are a few solutions available that help achieve this. They combine the capabilities of data mining solutions and predictive analysis to provide a single solution for predictive analysis. Vendors may offer proprietary solutions or solutions based on open source technologies. Predictive analytics software can be deployed on-premises for enterprise users or in the cloud for small businesses or for project or team based initiatives.

Predictive Analysis

Organizations are no more asking for just analysis of their data. Traditional Business Intelligence tools have been doing that for all for a long time now.

What we are exploring is getting more useful insights from our data. Organizations are looking for visualization tools and predictive analysis to explore data in new ways and discover new patterns. Big data analysis is not only restricted to processing large volumes of data on a software, it needs equivalent hardware processing capabilities too.

Predictive Analysis is the practice of extracting information from existing data to determine patterns and predict the future outcome & trends. It helps forecasting what might happen in the future with an acceptable level of reliability and includes what-if scenarios and risk assessment. 

Applied to business, predictive analysis models are used to analyze current data and historical facts in order to better understand customers, products and partners and to identify potential risks and opportunities for a company. It uses a number of techniques, including data mining, statistical modeling and machine learning to help analysts make future business forecasts.

Inline Deduplication

Inline deduplication looks as a very impressive term. You are made to believe that the magic would happen on the wire but at the end of the day, there are many caveats.

We have had a very recent experience where someone sized it with assumptions and commitments of reducing the backup & recovery windows tremendously but it did not really go as down as it was expected to be going.

Inline deduplication starts working on the source server itself and is followed by some more processing on the network. The left overs are taken care of by the media device’s memory. So, if you think you have a lot of extra resources on the production servers, go for this. If you are low on resources, you should first upgrade the production servers and then you are expected to have atleast two 10 Gig ports dedicated for the deduplication device and a well sized media server.

Just expecting wonders by replacing the backup device would not help much. It will reduce the backup & recovery windows a bit especially if you move from tape to disk while getting deduplication. However, you should be extremely careful in terms of your upgrade plans and the expectations that you set for yourself.

Source Based Deduplication

Choose the unique content from source itself when you start a backup. It does utilize some processing and memory from the source system so size it well.

Source based deduplication is also very powerful in ensuring that you utilize minimum network bandwidth during the backups. The backup application will create blocks of data on source and then store their hashes there at source and send unique data on the network. This is good for backups only if it is sized well. Catalog created by some applications is large enough to cause trouble for the performance of the source system which could be a production system.

Source based deduplication also gives good results for file system backups. A traditional approach takes long for file system backup that has millions of small files taking days for getting written especially during a full backup cycle. Source based deduplication in this case picks up only the changed content of the changed files reducing the amount of data travelling on the network irrespective of the backup level set.

Global deduplication on the target further reduces the amount of data stored.

Protecting Small Databases

A lot of SME’s get concerned about protecting their databases – typically SQL database. The interesting challenge is the fact that they are really small databases having extremely critical data.

While traditionally a standalone tape based backup solution would be considered ideal, it is not so simple. Since a 50-60 GB database normally compresses down to 10-15 GB, it does not deserve that kind of investment into tapes, each of which is capable of holding terabytes. It ends up holding too small a data and therefore costs more per GB.

With the changing times, better options are available depending on what you want to achieve:

A simple backup could help maintain multiple versions and copies and give you old and new recoveries when required. While this gives flexibility of versioning, the recovery process would take some time based on the kind of resources available.

Alternatively, if you are looking for a quick access to your data even after a disaster, replicating –especially mirroring – is the best option. To keep it in economical range, you can use native mirroring capabilities rather than investing in third party tools.

Ace Data Abhraya offers both: Cloud based backup for option 1 and cloud based infrastructure for option 2 with committed recovery SLA. Infact for SQL databases, you can opt for recovery on cloud infrastructure with option of recovering only database or complete server on cloud while enjoying the flexibility of versioning and compliance, and investing very less based on the backup size usage only.

How to backup Big Data?

The industry has been struggling a lot with the backups of the data they have been having for long now. Traditional tape based backup solutions seem good only for small size environments now. Though they have been growing in individual capacities and number of slots, better disk based options are pushing them more towards being secondary rather than the primary backup mediums.

Big data needs better care anyway for being big, and perhaps a bit more meaningful than the databases with invoice records or product records. The new technologies like deduplication and better compression algorithms like LZOB and ZLIB are making it more cost effective to back them up by bringing down their size.

What is also important is the cost of retaining this large volume of data and the varied sources of this unstructured data.

Ace Data’s Abhraya Cloud based backup offering resolves this challenge for its customers. Its flexible backup policies allow organizations to keep latest data close to them locally, and send the remaining to a cloud based offering. Being cloud based, they pay for what they backup and not invest on large growth assumptions. Furthermore as the backup grows old, it can be automatically archived to low cost disks reducing the cost of long term retention while ensuring data availability for long time.

The solution is capable of backing up smartphones, mobile laptops, large volumes of file servers apart from backing up the large servers and databases thereby ensuring that all sources of data can be backed up through a single solution.

How to Store and Manage BIG Data?

While I mentioned in my previous blog that any size of data is no problem, I often get questioned upon how to store and manage the huge volumes. This is a typical concern of an enterprise faced with increasing data size.

Storage vendors have seen and known this problem as it grew, and have scaled-up or rather scaled-out to help handle this massive growth. Both NAS and SAN vendors have gone beyond the traditional methods of upgrading the storage infrastructure by adding additional shelves and disks. The challenge that the traditional method has is that you end up upgrading capacity with shelves and disks with limited enhancements in processing power. This ends up in performance reduction.

The Scale-out method helps upgrade the storage by adding new nodes which include processing power, memory and capacity, thereby keeping the overall performance consistent with practically no dip in user experience. This is true for both SAN and NAS based storages. These storages can be expanded to PBs on a single storage, or even a single file system, by simply plugging in a new node. It is viable commercially also, as the cost per GB goes down as you keep adding more nodes.

So don’t worry about handling your Big Data as the storage devices are now available to store them more efficiently.

How much Data is good for business?

When you talk of sources of data generation, there is an endless list. Any business stream would have a long list to show how data is getting generated and how much data is being generated. Often businesses get scared with so much of data as they think handling it is a mammoth task. Indeed it is a mammoth task as it needs good investments and infrastructure to handle it. However, if utilized properly, the benefits are much higher. The way businesses are competing, it would soon become inevitable to handle it carefully.

The more data you have, more opportunity you get to see how your products, services and customers behave. There are many examples of business being able to analyse their data patterns and offer more discounts or value added services to give their customer a delightful experience. The new databases handling this Big Data have come up with Multi Parallel Processing technologies and the new applications to handle unstructured data ensure that even if you have PetaBytes of data, you can still do real time analysis and produce results in nano seconds.

Let us enjoy this new revolution on the way technology and businesses are getting shaped up and reap the benefits of these.

Developing an ILM strategy

Information Lifecycle Management or ILM as it is known popularly is perhaps the most important aspect of any organization today. This means that every organization needs to think over how it wants to handle its data right from the time it is created to the time it looses its value.

With the kind of data growth we are witnessing, it is becoming even more important for the organizations to understand how frequently they need to access the data, and how long do they need to retain that data.

Compliance regulations are one of the driving factors to define the overall retention period and business practices help define the criticality of the data.

It is for this reason that I believe that ILM is more of a business function than a pure IT function. In Indian context, you can co-relate this with the VAT authorities. If they have a query of current year data, they call you same day or next day. If it is a couple of years old case, you get 15-20 days to respond to every query and if it goes to 5-6 years old data, sometimes the case goes on for another year. Even they don’t ask for more than 10 years old data.

The only difference is that the business owner stored his sales files earlier at different locations based on their age and now on different disks and storages based on the same factor. The driving factor has always been the criticality and compliance for that data.

By categorizing your data into active and non-active data, and based upon the urgency of its availability, you can store this on tiered storage. This enables you to store the most recent and critical data on your fastest and most accessible devices (or cloud), and retire the rest to archivals, thereby saving both cost and resources.