Learn from your mistakes, analytically

As discussed in the previous blog, every business has a risk and needs to find how to decrease the risks. Each negative outcome that occurs presents an opportunity from which to learn new things. In a small scale business, these decisions could be easy and more mind driven being controlled by a set of managers only. As the businesses grow beyond a size, predicting the future course of action and avoiding the past mistakes becomes important. 

The enterprise integrates the predictive model’s scores in order to act upon what has been learned. At each step, the predictive scores foresee where a “blunder” may incur unnecessary risk, thereby guiding the organization to avoid it. In this way, predictive analytics delivers a complete data-driven system for risk management. 

Predictive modeling capabilities are scientifically proven and have benefited from decades of advancement. With a tenured track record of success, predictive analytics boasts mature software solutions that deliver this technology to – and integrate it with – the modern enterprise.

Predictive Analysis used by Insurance Companies

All businesses are run at a risk. Risk is the way business is managed. Every decision an organization takes impacts the risks an enterprise can withstand like the risk of customer defecting, of not responding to an expensive glossy mailer or offering a huge retention discount to a customer who was not leaving even otherwise and in turn missing out on a critical customer who leaves.

The data driven means to compute risk of any type of negative outcome in general is predictive analysis. Insurance companies have used this very well. Insurance companies are augmenting their practices by integrating predictive analysis in order to improve pricing and selection decisions. 

The actuarial methods that enable an insurance company to conduct its core business perform the very same function as predictive models: Rating customers by chance of positive or negative outcomes. Predictive modeling improves on standard actuarial methods by incorporating additional analytical automation, and by generalizing to a broader set

Predictive Analysis and Big Data

Predictive analytics is an enabler of big data: Businesses collect vast amounts of real-time customer data and predictive analytics uses this historical data, combined with customer insight, to predict future events. Predictive analytics enable organizations to use big data (both stored and real-time) to move from a historical view to a forward-looking perspective of the customer.

For example, historical preferences of consumer can be analyzed from their usage patterns and promotional offers can be planned accordingly. The historical data can help predict which promotional offer would be most useful. 

There are a few solutions available that help achieve this. They combine the capabilities of data mining solutions and predictive analysis to provide a single solution for predictive analysis. Vendors may offer proprietary solutions or solutions based on open source technologies. Predictive analytics software can be deployed on-premises for enterprise users or in the cloud for small businesses or for project or team based initiatives.

Predictive Analysis

Organizations are no more asking for just analysis of their data. Traditional Business Intelligence tools have been doing that for all for a long time now.

What we are exploring is getting more useful insights from our data. Organizations are looking for visualization tools and predictive analysis to explore data in new ways and discover new patterns. Big data analysis is not only restricted to processing large volumes of data on a software, it needs equivalent hardware processing capabilities too.

Predictive Analysis is the practice of extracting information from existing data to determine patterns and predict the future outcome & trends. It helps forecasting what might happen in the future with an acceptable level of reliability and includes what-if scenarios and risk assessment. 

Applied to business, predictive analysis models are used to analyze current data and historical facts in order to better understand customers, products and partners and to identify potential risks and opportunities for a company. It uses a number of techniques, including data mining, statistical modeling and machine learning to help analysts make future business forecasts.

Who is a Data Scientist?

Simple words: A practitioner of data science is a data scientist. They apply their skills to achieve a broad spectrum of end results. They have the ability to find and interpret rich data sources, manage large amounts of data despite hardware, software and bandwidth constraints, merge data sources together, ensure consistency of data-sets, create visualizations to aid in understanding data, build mathematical models using the data, present and communicate the data insights/findings to specialists and scientists in their team and if required to a naive audience.

Data science is emerging to meet the challenges of processing very large data sets i.e. "Big Data" consisting of structured, unstructured or semi-structured data that large enterprises produce. A domain at center stage of data science is the explosion of new data generated from smart devices, web, mobile and social media. Data science requires a versatile skill-set. Many practicing data scientists worldwide specialize in specific domains such as the fields of marketing, medical, security, fraud and finance.

Does Real-Time Analysis Help?

Many people ask the question that is there actually a need for real-time analysis. How would it help them in their business and is it worth the kind of investment it needs to get into real-time analysis?

You can analyse your business to see how you can use it for your business development. We can site a few small examples of different streams that organizations have used to see the benefits. Let us start with healthcare – a small device worn on the waist belt injects desired quantities of Insulin to a  diabetic patient while monitoring the sugar level through the device.

Imagine you enter a hotel and the person at reception knows your name, booking details and your preferences. While the second part is already in control through club memberships, the camera on the door takes your picture, searches for the record and by the time you reach the reception, your details are on the desk.

Someone walking near the store gets a message on additional offers on their favourite product for the next 30 minutes.  If you reject a product from the shelf, you get a message of additional offers on the product. Always get a win-win situation for both the store and the consumer.

From Just Analysis to Real-Time Analysis

Human race has been playing with data for ever. Large volumes of data being generated, and reports created of these, have been the traditional approach the way business have been run and expanded. The future prospects were based on some trends of your own organization, market and competitors and all this was based on the performance.

Analysts have been analysing this data to help organization plan the above.

Times have changed now and businesses cannot only rely on past performances to plan their future. You need to capture the current trends and the needs of the consumer today. Analytics have changed now from analysing past data to perform real-time data. Data generated not only from the structured in-house databases but also from non-structured data generated through social media and consumer behaviour.

Tools are available today to collect data from these sources and put them together through what is called the process of “Data Conditioning” into databases to help analyse them. All this data gets processed real time to produce near instant results helping the businesses serve their consumers better.

What is Data Science?

While most of us believe Data Science is handling large volumes of data effectively and efficiently, Data Science is not restricted to this. Merely using data isn’t really what is meant by “data science.” A data application acquires its value from the data itself, and creates more data as a result. It’s not just an application with data; it’s a data product. Data science enables the creation of data products.

Several organizations have used this well and have given the industry several products that help handle Data Science. Key examples are from Google itself who realized the importance of a search engine and further made it more effective using tracking links. Spell checks with suggestions and speech recognition are other examples of Google’s way of creating products out of data. Facebook and Linkedin have used patterns of friendship relationships to suggest other people you may know or should know, sometimes with great accuracy.

The thread that ties most of these applications together is that data collected from users provides added value. Whether that data is search terms, voice samples, or product reviews, the users are in a feedback loop in which they contribute to the products they use. That’s the beginning of data science.

The Show Must Go On.

When theatre veteran Noel Coward said this, he was talking more about the economics of theatrical world. But very soon many realized that this applies to everything in life.
When we transitioned from DAT drives to DLT, many questioned the fact that the data size would keep growing and DLTs won't help. Technologists did not think that way and they got ready with the LTOs followed by disks and now deduplication.
Technology is always ready for the next level to resolve the next problem you might face. It is important to be adaptable to the new things. Anything can become obsolete and every new thing brings in new and fresh energy to overcome the obsolete.
Never hold yourself to the current technology. It has to become obsolete soon and you have to be ready to accept the new order. At some point, obsolete has to go out and the new stuff would be good for the new situation. So Always Be Ready.
Because the show will go on, with or without your participation.

Unique Deduplication

And there comes the unique one. This classifies on all the three modes of deduplication. This comes using the Abhraya’s unique agentless backup methodology. Let us understand what is so unique here.

Abhraya picks up data from the source production system and brings it to the Abhraya client system. The deduplication happens at the Abhraya client. This ensures that no deduplication process takes any processor or memory of the source system. Of course, it does compression and encryption also here so it keeps it completely free from using production system’s resources. From here the deduplicated data goes to the Abhraya Server for final storage.

How I classify this in all three modes is because it does deduplication before it reaches the final backup destination and does not need any resources or extra disks to store databefore going for target based. For the source production system on the other side, there is no process happening there. Data goes out and gets deduplicated on some other system.

And finally, it does perform inline deduplication also as the target based deduplication happens before writing data on the disk. Unlike a typical target based deduplication which first writes the data and then reads and deletes duplicate blocks. Abhraya does it before writing and offers benefits of all the three types to its users.

Target Based Deduplication

Target based deduplication was one of the initial ways of performing deduplication. This was more on the VTLs when VTLs were launched as a technology, and over time, it seems that this is not much in use now.

All it did was to carry the data to the target backup device – mainly VTL – and store it there. It would then run a deduplication process to match blocks and delete the duplicate data. At a later schedule, system will run a clear garbage type process to finally remove the deleted content and free up the space on the VTL.

When it was launched, it was the only process and was a great process. With newer and better technologies available, target based deduplication has lost its charm since most applications now prefer source based deduplication and reduce data before it travels on the network.

So target based deduplication has its advantages of using minimum source processing and memory cycles in performing deduplication at the target, the hind side being slightly oversizing the target to ensure enough space to accommodate full data before starting the deletion process.

Inline Deduplication

Inline deduplication looks as a very impressive term. You are made to believe that the magic would happen on the wire but at the end of the day, there are many caveats.

We have had a very recent experience where someone sized it with assumptions and commitments of reducing the backup & recovery windows tremendously but it did not really go as down as it was expected to be going.

Inline deduplication starts working on the source server itself and is followed by some more processing on the network. The left overs are taken care of by the media device’s memory. So, if you think you have a lot of extra resources on the production servers, go for this. If you are low on resources, you should first upgrade the production servers and then you are expected to have atleast two 10 Gig ports dedicated for the deduplication device and a well sized media server.

Just expecting wonders by replacing the backup device would not help much. It will reduce the backup & recovery windows a bit especially if you move from tape to disk while getting deduplication. However, you should be extremely careful in terms of your upgrade plans and the expectations that you set for yourself.