Big Data Professional Training
We offer training program on Big Data Professional concept. This training program is the combination of training on Big Data Core concept and training of any one of the Big Data product from the product list, such Databricks or Snowflake (or Others). This training program will help individual to be industry ready and to ready to work in the Organizations.
Course Duration: 6 Months
Course Objectives
- Upskill and Reskill.
- Learning about troubleshooting different technical issues in Big Data ecosystem.
- Knowing about the product architecture.
- Knowing about all product features in practical.
- Knowing how to run proof-of-concept (PoC) on the product.
Course Topics
- Installations and Configuration of Hadoop, Hive, Spark, and Airflow.
- Configurations of Hadoop for Hive, Spark, and Airflow.
- Configurations of Hive for Hadoop, Spark, and Airflow.
- Configurations of Spark for Hadoop, Hive, and Airflow.
- Configurations of Aiflow to run query against Hadoop, Hive, and Spark.
- Configuring and Working on custom metastore (MySql or others) in Hive.
- Creating database and tables for real-time data sets (TPC-DS).
- Installation and configuration of DBeaver.
- Loading data and running business queries against Hive.
- Installation and Configuration of Jupyter notebook for PySpark.
- Creating DAG in Airflow for different tasks.
- Deploying DAG on Airflow.
- Running queries from Spark and Airflow against Hive.
- Running queries from Airflow against Hive and Spark.
PRODUCT SYLLABUS
Course Methodology
- Big data maturity model : TDWI or CSC or Others
- CRISP-DM : The CRoss Industry Standard Process for Data Mining (CRISP-DM) is a process model that serves as the base for a data science process.
- SEMMA : SEMMA is a list of sequential steps developed by SAS Institute, one of the largest producers of statistics and business intelligence software.
- OSEMN : OSEMN stands for Obtain, Scrub, Explore, Model, and iNterpret. It is a list of tasks a data scientist should be familiar and comfortable working on.
- TDSP : The Team Data Science Process (TDSP) is a method for developing predictive analytics solutions and intelligent applications in a cost-effective and timely manner.
- TPC-X : Transaction Processing Performance Council benchmark for Hadoop, DS (Decision support), DI (Data Integration), AI (Artificial Intelligence).
Big Data Organizations and Products
- Databricks - Unify all your data, analytics and AI on one platform
- Snowflake - Cloud-based data warehousing platform with separation of compute and storage
- Vertica - Performance analytical database for real-time analytics
- Confluent - Streaming platform for managing and processing high volumes of real-time data
- Starburst - Open-source distributed SQL query engine to query data across disparate sources
- Dremio - Its ability to accelerate data access and analytics on data lakes
- Qubole - Qubole is an open, simple, and secure data lake platform
- Control-M - Transform business with application and data workflow orchestration
- Posit - Deploy all your work, including Shiny, Streamlit, and Dash applications, Models
- Tlmi - Specialists in Machine Learning, AI, Big Data and BI
- Snowplow - Event data collection and analytics platform with flexibility and extensibility
- Cloudera - Enterprise-grade platform, which combines open-source technologies
- Vantage - Provides advanced Big Data capabilities
- Druid - High performance, real-time analytics database
- Aerospike - High-performance, low-latency NoSQL database
- Beam - Unified programming model for batch and streaming data processing
Cloud Big Data Organization and Products
- AWS EMR - Easily run and scale Apache Spark, Hive, Presto, and other big data workloads
- AWS Managed Airflow - Highly available managed workflow orchestration for Apache Airflow
- IBM Big Data - Leverage effective big data technologies
- Azure Big Data - How big data analytics works and why it matters
- Oracle Big Data - Help data professionals manage, catalog, and process raw data
- GCP Big Query - BigQuery is a serverless and cost-effective enterprise data warehouse
