Principal Engineer - Kafka
JOIN US AS A KAFKA PRINCIPAL ENGINEERSimilar Industry Titles and Keywords: Platform Engineer, Systems Engineer, Developer, Site Reliability EngineerYou should join our team. We're the team that enables Data Movement at Target. A massive part of our technology footprint is Apache Kafka and the ecosystem that has developed around it. It's not every day that you'll get to use some of the technologies we've chosen, do it at the scale we're using them, and interact with engineers at the heart of the Apache Kafka community. Demand for Kafka at Target is high and continues to grow at a rapid pace. We want to level up even more on the platform and we need your help! Here's a smattering of approaches important to us and the technologies we use:
- Everything we do is “as-code” in version control. We don't like clicking buttons or doing things manually.
- All development or infra config changes go through a pull-request process, so you'll always have a say to thumbs up or down things you catch.
- Everything should have test cases and go through a continuous integration process.
- We understand the importance of logs and metrics, so having visibility to things you need to see to do your job isn't an issue. And if you need to add more metrics or see more logs, it's within our control to improve that.
- We try to own as much of the platform as we reasonably can. You don't need to rely on other teams outside our own to improve the stack or change the way we do things.
- Our stack runs across bare metal, OpenStack, and Kubernetes internally. We're also taking advantage of the public cloud in a big way.
- Infrastructure: OpenStack, public cloud, Spinnaker, ZooKeeper, Kafka, Influx, Grafana, ELK, Sensu, Consul, PagerDuty
- Dev: GitHub Enterprise, Jenkins, Artifactory, HipChat, Bots
- Play the Product Owner role and lead a strong engineering team to build out, scale, and operate a large Kafka environment across Target, private, and public cloud data centers.
- Be the voice of the customer, create the vision for the where the Kafka platform needs to be in the future for Target.
- Define and deliver new tooling on top of our Kafka platform that will help us operate Kafka at a larger scale, create additional capabilities (like auditability), help increase automation/self-service for our customers. Look for opportunities to open source what we build.
- Collaborate with other principle engineers across the Target Application Platform (TAP) to ensure Cloud and Compute is delivering a platform that meets the needs of the company.
- Design and implement enhancements to our Kafka platform, this can include implementing new open source solutions that have been developed around the platform, improving our cluster architecture, improving our monitoring capabilities, and developing and possibly open sourcing tooling of our own.
- Troubleshoot and resolve operational issues.
- Help drive adoption of new technologies within the team and across Target
- Build prototypes to prove out concepts
- BA/BS or equivalent experience
- 10+ years total work experience
- Has in-depth knowledge of state-of-the art engineering technical approaches in design, build, testing, debugging problems as required by domain
- Maintains technical knowledge within areas of expertise
- Stays current with new and evolving technologies via formal training and self-directed education
- Experience with Kafka, Git/GitHub, OpenStack, containers, Spinnaker, Linux, InfluxDB, Consul, Zookeeper, working with major public cloud providers
- Personal or professional experience contributing to open source projects
- Innovative mindset - willingness to push new ideas into the company
- 1-5 years of platform/ops experience
- Highly productive, self-starter and self-motivated
- Passionate about staying current with new and evolving technologies
- Ability and desire to mentor engineers on our team, and our partner teams
- Mix of application development and platform engineering skillset