Agenda

Cost to register: $799

Apache Pinot is a high-performance database engineered to serve analytical queries with extremely high concurrency, boasting latencies as low as tens of milliseconds. It excels at ingesting streaming data from sources like Apache Kafka and is optimized for real-time, user-facing analytics applications.

In this full-day training, we will explore the architectures of Apache Kafka, Apache Flink, and Apache Pinot. We will run local clusters of each system, studying the role each plays in a real-time analytics pipeline. We’ll begin by ingesting static data into Pinot and querying it. Not content to stop there, we’ll add a streaming data source in Kafka, and ingest that into Pinot as well, showing how both data sources can work together to enrich an application. We’ll then examine which analytics operations belong in the analytics data store (Pinot) and which ones should be computed before ingestion. We’ll implement those operations in Flink. Having put all three technologies to use on your own in hands-on exercises, you’ll leave prepared to begin exploring the three of them together for your own real-time, user-facing analytics applications.

LEARNING OBJECTIVES

At the successful completion of this training, you will be able to:

• List the essential components of Pinot, Kafka, and Flink
• Explain the architecture of Apache Pinot and its integration with Apache Kafka and Apache Flink
• Form an opinion about the proper role of Kafka, Flink, and Pinot in a real-time analytics stack
• Implement basic stream processing tasks with Apache Flink
• Create a table in Pinot, including schema definition and table configuration
• Ingest batch data into an offline table and streaming data from a Kafka topic into a real-time table
• Use the Pinot UI to monitor and observe your Pinot cluster

PREREQUISITES

To participate in this workshop, you will need the following:

• Docker Desktop: We will use Docker to run Pinot, Kafka, and Flink locally. If you need to install it, please download Docker Desktop and follow the instructions to install it at https://www.docker.com/get-started/
• Resources: Pinot works well in Docker but is not designed as a desktop solution. Running it locally requires a minimum of 8GB of Memory and 10GB of disk space