Starting a Big Data and Analytics (BDA) project on cloud is not only faster, but also quite cheaper. I have come across many clients who want to start their BDA project, but they don’t proceed or I will say they delay thinking about the upfront cost, required skills and execution time. All these can be well taken care of if they start their project on cloud.
The challenge for them is to decide upon which what cloud services they should go for and which vendor to select. Here is a list of the services which are currently available on cloud and will be a good idea to consider them to start BDA project.
1. Edge Services : This act as an interface between your users, data and Cloud services provider. It serves the following purposes
- DNS resolution
- CDN services
- Load balancers
This is typically available as IaaS from companies like, IBM, AWS, Microsoft etc.
2. Data Streaming : This is primarily for data in motion. You need data streaming for
- Real time analytical processing
- Data Augmentation
Data Streaming tools are available as SaaS on various cloud market places.
3. Data integration: Data from different sources are delivered to the cloud service provider by using Edge services then go through the following to extract insights from it :
- Data Staging
- Data quality checks
- Transformation and loading
These are also available was SaaS on various Cloud Market places.
4. Data Repositories: Data repository consist of both data in motion ( from streaming services) and data at rest (after Data integration process) and then prepares the data for the various Analytical engines. Data repositories are meant for the following functionalities:
- Data warehousing
- Landing, exploration and archive
- Deep Analytics and modelling
- Interactive Analytics and Reporting
Earlier not many SaaS offerings were available for Data repository services. But now there are many SaaS offerings available on different cloud market places.
5. Actionable Insights: Data from Data repository is then fetched in to a variety of tools to extract insights. Typically you need different tools to perform the following:
- Decision Management
- Discovery and Exploration
- Predictive Analytics
- Analysis and Reporting
- Content Analtyics
- Planning and Forecasting
In addition to the above services you do get Data Security an Governance services on cloud. There are multiple vendors providing either all or part of the above services. The market is flooded with services and to select one service or vendor really require a lot of research and many points to be considered.