Administration of Confluent Apache Kafka Training Course

Module 1: Overview of Confluent Apache Kafka Architecture and Configuration

• The role of Kafka in modern data pipelines.
• Distinguishing between Apache Kafka and Confluent Kafka.
• Key components: producers, consumers, brokers, topics, and partitions.
• Deployment models and scaling strategies for Kafka clusters.

Module 2: Configuring Zookeeper Quorums

• An introduction to Zookeeper.
• Zookeeper's function within a Kafka cluster.
• Determining optimal Zookeeper quorum sizes.
• Configuring Zookeeper instances.
• Setting up SSH on target servers.
• Practical exercise: Configuring Zookeeper as both a team and a service.
• Utilizing the Zookeeper Command Line Interface (CLI).
• Practical exercise: Configuring a Zookeeper Quorum.
• The internal file system of Zookeeper.
• Performance considerations specific to Zookeeper.
• Demonstration of management tools, including Zookeeper and Zoonavigator.

Module 3: Configuring the Kafka Cluster

• Foundational Kafka concepts.
• General Kafka configuration practices.
• Practical exercise: Configuring Kafka brokers.
• Practical exercise: Executing Kafka commands.
• Practical exercise: Setting up a multi-broker Kafka cluster.
• Practical exercise: Testing cluster connectivity and performance.
• Verifying cluster accessibility.
• The critical importance of the advertised.listeners setting.
• Topic-specific configurations.
• Settings for downloading and ingesting topic messages.
• Practical exercise: Demonstrating Kafka resilience.
• Performance analysis: I/O operations.
• Performance analysis: Network (RED metrics).
• Performance analysis: RAM utilization.
• Performance analysis: CPU usage.
• Performance analysis: Operating System (OS) interactions.
• Other performance influencing factors.
• Practical exercise: Modifying Kafka broker configurations.

Module 4: Advanced Kafka Configurations

• Configuring the Landoop Kafka topic UI, Confluent REST Proxy, and Confluent Schema Registry.
• Messaging practices using CLI, Java, and the Spring framework.
• Monitoring metrics and leveraging tools such as Confluent Control Center and Elasticsearch.
• Managing log files and offsets.
• Ensuring high availability and disaster recovery.
• Achieving high availability through replication.
• Optimizing producer and consumer performance.
• Developing disaster recovery strategies.
• Controlling failover and managing data recovery.
• Configuring connectors.
• Implementing Kafka Connect.
• Implementing Kafka security features.

Summary and Recommended Next Steps