Big Data/Analytics Zone is brought to you in partnership with:

Tharindu holds a first class honors degree in computer science and engineering from the University of Moratuwa, Sri Lanka. He also received a professional postgraduate diploma in marketing from the CIM, UK, where he is an associate member. Tharindu currently works at WSO2. He is a Associate Tech Lead and a member of the data technologies management committee, focusing on big data, analytics, and business activity monitoring (BAM). Tharindu is a DZone MVB and is not an employee of DZone and has posted 15 posts at DZone. You can read more from them at their website. View Full User Profile

How to Be Big-Data-Native

11.27.2012
| 5737 views |
  • submit to reddit

Big data has spawned a set of tools that deliver results beyond the buzz. It has started delivering real insights for companies, which result in more effective decisions.

When middleware natively supports big data, big data becomes more than just another option. It becomes the default. Let’s examine this idea:

Big Data Storage

Whenever you think of storage, (almost) everyone thinks of an RDBMS mysql, postgres, mariaDB, Oracle, etc. If you convert to supporting native Big Data, you turn into to NoSQL options that sacrifices SQL for scale. You start storing everything in Cassandra, HBase, mongoDB, redis, etc. You don’t have to dread the day your data volume becomes too big to handle. When it does all you do now is configure a new node and maybe tune the cluster a bit and you are done.

Big Data Analytics

If you are not looking at analytics, you should (Google for innovate or die). All analytics support for big data. If you thought about running some SQL or spread sheet macros, it’s time to move on. Start thinking Hadoop, Big Query, Drill, etc. Natively supporting Big Data analytics allows anyone who sets up the middleware to instantly enable their departments and teams to harvest their data silos, no matter how big they are.

Big Data Speed

There is no point of BigData storage and Big Data Analytics if you can’t collect big data fast enough. Looking at web scale transactions it is not unusual to have millions a second. Before middleware vendors start advocating they need to make sure the middleware is up to par. Computers are definitely fast enough so that even a single node can handle around 10,000 TPS.

The biggest concern I’ve seen is many companies claiming to be able to adapt to Big Data without actually supporting it natively. It is a nightmare to allow to scale for Big Data after choosing incompatible technologies. The concepts differ, the trade offs are different and the effectiveness is very low. Big data has reached the importance level to for middleware evaluators to add another section to their RFPs. And, yes that’s whether the middleware is “Big Data native”.

Published at DZone with permission of Tharindu Mathew, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)