Big Data in real time

Hi Guys,

How are you doing today. Just thought of sharing my knowledge on big data. Hope you will like it. I always like to put everything in simple (layman) terms.

Big Data

·         As the name suggests it is big, huge , high volume of data. It starts from Tera byte, Peta byte, Hexa byte, Zeta byte

·         Velocity of Data. The speed of the data is so fast. Example: 1 million likes in 8 hours. How would you process how many of the likes came from Asia-pacific region. It is difficult right ? This is all addressed in Big data

·         Different forms of data.

o   Structured Data – High Volume of organized data. Example: RDBMS System, Web Server

o   Unstructured Data – Images, Facebook, Twitter, Audio files, Video files etc.

o   Semi structured data – XML, Email

Examples

o   Sensor Data

o   Data from electronic devices like ipad, health tracker , smart phones, gps

o   Amazon sales data

o   Facebook, Twitter contents à Customer sentiment analysis

o   Stock market à Almost daily we receive 1 TB data

o   Weather data

o   Google satellite images

 

Traditional system

·         Storage becomes very difficult

·         Data analysis tools are not able to handle this volume

·         Forecasting

·         Unstructured data handling . This is something we have not done in the past

Real time examples

o   Driver less cars

o   In USA, insurance companies like progressive are giving us a device we can fit into car. This will track

§  how you drive ? what speed ? what route you follow ?

§  what is your normal timing etc. All these helps progressive to provide discounts, understand drivers

o   Ebola virus – Google or IBM used to track

o   Amazon Sales /Ebay sales. As soon as you purchase something online, this is captured and if you notice, you will get lot of recommendations regarding that

o   Google ad words -Suppose if you google search something . Once I was looking to purchase new tires but did not buy. From that day, I keep getting tires ads in almost all the websites I am surfing

o   Health data – Just think of finger print given for iphone or to some other system . Government has all

o   Credit Score

 

Advantages

·         Business improvement  à Sales increases as companies are able to target right candidate, right time and right product

·         Predictive analysis à Example election results