Technically Speaking

The Official Bigstep Blog

 

4 Things to Look for When Choosing Your Data Lake Vendor

In the beginning was the database, and the database was good. It stored all of the transactional data and powered your users and applications quite nicely. Then the data grew, and the database expanded to the data warehouse. The data warehouse was good, too, and it allowed organizations with larger data sets to power even more users and applications. Then came big data, and the data warehouse just couldn't keep up. Big data involved lots of sloppy unstructured data that didn't get along well with relational databases. Businesses needed more. Hence, the data lake.

In the beginning was the database, and the database was good. It stored all of the transactional data and powered your users and applications quite nicely. Then the data grew, and the database expanded to the data warehouse. The data warehouse was good, too, and it allowed organizations with larger data sets to power even more users and applications. Then came big data, and the data warehouse just couldn’t keep up. Big data involved lots of sloppy unstructured data that didn’t get along well with relational databases. Businesses needed more. Hence, the data lake.

A data lake is an excellent way to collect, house, and process large, unstructured data sets, even when you aren’t sure exactly how that data will be used in the future. But if it’s not done correctly, it becomes a nightmare. Some folks call this nightmare the data swamp - a place where good data goes to hide forever, never to produce meaningful analytics again. So, choosing a vendor to partner with to build and store your data lake is essential. Find someone who knows how to navigate the waters—a ‘been there, done that’ guide who knows the natives and how to avoid the alligators. Here’s how to find that guide.

1. Choose a Data Lake That Easily Integrates with Your Current Architecture

When the data lake is built right, it is a pristine storage solution to hold your data in its original format, so that it’s wide open for exploration anytime.

The first consideration is how the data lake will fit within the rest of your IT architecture. You need a solution that will integrate with the applications and systems you’re already using, and one that won’t tie your hands when it comes time to make changes. Easy integration eliminates data silos that occur when a system isn’t compatible with your data storage and won’t allow for seamless streaming of data into your data lake.

2. Choose a Data Lake That Offers Enterprise-Grade Security

Ah, security. No discussion of big data is complete without it. Big data isn’t just attractive to businesses, it’s a goldmine for the hacker, as well. Look for a data lake solution that offers enterprise-grade security and has the track record to back up their claims.

3. Choose a Data Lake That is Affordable

Cloud data lakes are all the rage, but that doesn’t mean it has to bust your budget for five years out. Look for a storage vendor offering a good deal (think pennies per gigabyte). The cloud is the ideal way to build a data lake without having to hock the office furnishings to pay for hardware and software to house it—not to mention the maintenance and security costs.

4. Choose a Data Lake That Can Accommodate the Types of Data You’re Working With

Without good processes and procedures, the pristine data lake becomes a formidable swamp where data goes in, but never comes out again. Think of your vendor as your guide to the data lake, assuring that it provides benefit to your business.

The beauty of data lakes is the ability to house large quantities of all kinds of data, including data that’s in weird formatting like machine data and logs, data from social media feeds, clickstream data, audio and video files, sensor data, data from proprietary or custom systems like CRMs and ERPs, RDBMS and NoSQL exports, or whatever you happen to have. Make sure your vendor can accommodate these types of data.

Finally, there is a data lake offered as a service (DLaaS) for affordable data storage of all types, that easily fits within your existing IT infrastructure. Discover the first Full Metal Data Lake as a Service in the world. Get 1TB free for life - limited to 100 applicants. Start here.

Got a question? Need advice? We're just one click away.
Sharing is caring:TwitterFacebookLinkedinPinterestEmail

Readers also enjoyed:

5 Best Practices to Assure Your Data Lake is Swimmingly Successful

As big data becomes a mainstay in the business, many organizations are abandoning the data warehouse for data lakes. With a data lake, you don't have…

Is a Data Lake the Better Solution to Your Data Warehousing Issues?

The good old data warehouse has serviced business admirably for decades. Generally structured as a relational database, it is the go-to data resource…

Leave a Reply

Your email address will not be published.

* Required fields to post your comments.
Please review our Privacy Notice in order to understand how we process your personal data and what are your rights in this respect.