ALGOTALE - LUMIQ- AWS KAFKA SME

• Design the architecture for Apache Kafka using AWS MSK, including cluster sizing, and integration points with other AWS services

• Provision MSK clusters (Provisioned or Serverless) using AWS Console, CLI, or Infrastructure as Code (CloudFormation, Terraform)

• Select appropriate broker types, instance sizes, storage, and number of brokers per Availability Zone

• Configure Apache Kafka settings, including topics, partitions, and retention policies

• Configure authentication and authorization (IAM, SASL/SCRAM, mTLS, Kafka ACLs)

• Enable encryption at rest and in transit

• Manage secrets and credentials securely

• Deploy clusters across multiple Availability Zones for high availability.

• Set up auto-scaling for storage and brokers as needed

• Plan and implement backup, disaster recovery, and failover strategies (including MSK Replicator for cross-region replication)

• Configure producers and consumers to connect securely to the MSK cluster

• Plan and execute Kafka version upgrades and patching with minimal downtime

Nexthire