Confluent Kafka Migration Engineer

SUMMARY:

Role Overview

Our client is looking for experienced (mid-level Senior) Confluent Kafka Migration Engineers to support the migration of a large-scale, on-premises Confluent Platform environment to Confluent Cloud. The resource will be responsible for migration planning, cloud environment configuration, secure network conn...

POSITION INFO:
Role Overview Our client is looking for experienced (mid-level Senior) Confluent Kafka Migration Engineers to support the migration of a large-scale, on-premises Confluent Platform environment to Confluent Cloud. The resource will be responsible for migration planning, cloud environment configuration, secure network connectivity, Kafka data replication, authentication modernisation, production cutovers, and post-migration stabilisation. The role requires strong hands-on expertise in Apache Kafka, Confluent Platform, Confluent Cloud, and enterprise-scale production migrations, with a focus on data integrity, high availability, security, and minimal service interruption.   Roles and Responsibilities Assess the existing on-premises Kafka and Confluent Platform environment and validate the target cloud architecture. Develop detailed migration strategies, wave-based execution plans, cutover runbooks, rollback procedures, and risk mitigation plans. Provision and configure Confluent Cloud environments for development, pre-production, and production. Configure Kafka topics, partitions, replication, Schema Registry subjects, ACLs, service accounts, and Kafka Connect connectors. Implement and manage Confluent Cluster Linking for replication and migration of Kafka workloads from on-premises to cloud. Validate data consistency, schema compatibility, consumer offsets, application connectivity, and message processing before and after migration. Configure secure private connectivity between on-premises data centres and cloud environments using AWS PrivateLink, GCP Private Service Connect, VPC peering, or equivalent technologies. Modernize Kafka client authentication by migrating from Kerberos-based authentication to supported mechanisms such as API keys, OAuth, or mutual TLS (mTLS), as applicable to the target architecture. Collaborate with application teams to update Kafka client configurations, libraries, and authentication settings. Implement Infrastructure as Code (IaC) using Terraform and automate platform provisioning and configuration through CI/CD pipelines. Configure monitoring, alerting, audit logging, consumer lag tracking, and performance dashboards using enterprise observability tools. Troubleshoot Kafka replication, connectivity, throughput, partitioning, consumer lag, and production performance issues. Execute production cutovers in planned waves, validate workload functionality, and maintain rollback readiness. Support hypercare, operational handover, knowledge transfer, and decommissioning of legacy Kafka infrastructure after successful migration. Required skills: Strong hands-on experience with Apache Kafka and Confluent Platform administration. Proven experience in enterprise-scale Kafka migrations from on-premises environments to cloud-managed platforms. Practical experience with Confluent Cloud architecture, configuration, administration, and operations. Hands-on experience with Confluent Cluster Linking and Kafka replication strategies. Strong understanding of Kafka topics, partitions, replication factors, consumer groups, offsets, and high-availability configurations. Experience with Confluent Schema Registry, schema compatibility, Kafka Connect, connectors, ACLs, and service accounts. Knowledge of Kafka security and authentication mechanisms, including Kerberos, SASL, TLS, API keys, OAuth, or mTLS. Experience with AWS or GCP cloud services, private networking, and secure connectivity. Hands-on experience with Terraform or other Infrastructure as Code tools and CI/CD automation. Experience with production cutovers, rollback planning, disaster recovery, monitoring, and incident troubleshooting. Strong understanding of data integrity, performance optimisation, security controls, and operational readiness. Ability to coordinate with application development, cloud infrastructure, security, networking, and operations teams.   Good-to-Have Skills   Experience with migration of Confluent Platform to Confluent Cloud Enterprise clusters. Knowledge of Apache Flink architecture, stream-processing concepts, and Kafka–Flink integration. Exposure to multi-cloud or cross-region Kafka migration strategies. Experience with AWS PrivateLink, GCP Private Service Connect, VPC peering, VPN, or Direct Connect. Knowledge of ksqlDB, Tableflow, and Confluent Cloud governance capabilities. Experience with Prometheus, Grafana, Instana, or similar observability tools. Knowledge of Kubernetes, Red Hat OpenShift, GitOps, and cloud-native deployment practices. Experience with managed streaming services and enterprise event-driven architectures. Confluent Certified Developer or Administrator for Apache Kafka certification. AWS or GCP cloud certification and HashiCorp Terraform Associate certification.