What is Big Data after all?
The term for Big Data is an umbrella term. It can have lots of meanings, just like the term Cloud has. In our opinion, there could be two main categories for it:
1. Big Data as a service
In order to drive better business intelligence solutions from our data, we’ve tried processing the data through services such as GoodData. They take it in, crunch it and generate the results, with graphs and dashboards.
2. Big Data software providers
Big Data software, on the other hand, can run within your organization, on your servers or on rented servers, but under your complete control.
One of the most popular solutions is Hadoop, even if it can be difficult to use. Its map-reduce strategy, although smart, is even harder to implement than complex queries, with the slight note that it can scale a lot better. You will need very good software engineers to make use of this approach. We’ve also tried Tableau Software, which is similar to GoodData, but can be installed onto a server.
Why Big Data as a service is a good idea
As long as prices are affordable enough or proportional to the size of the data, Big Data as a service can be a good idea. But you still need someone to automate the dump and to clean the database, and you still need to get the data out of your systems and into an external data warehouse.
Moving data outside of your organization is not as simple as it may seem. Though, if you can sanitize the data before upload, as well as remove any sensitive information, it might be a solution.
The worst possible case scenario is having the entire data leak on the Internet. If you remove private customer info, such as names, emails, phone numbers etc. you will still have a fair amount of data about how many sales, cancellations, renewals and such you are registering per segment. You could see trends and develop some business intelligence capabilities (such as pattern recognition, classification, clustering or other smart correlations) with little investment.