Sr Data Engineer - Merch Data Tools (Hadoop, Hive, Spark, Pig)

Brooklyn Park, Minnesota
Nov 08, 2022
Dec 13, 2022
Employment Status
Full Time

About us:
Target is one of the world's most recognized brands and one of America's leading retailers. But behind the brand our guests love, is a culture of continual innovation - and right now, we are up to big things.

We are looking for highly motivated software engineering professionals who can help us advanced our Data Exploration strategy for Target. You will be able to put your skills, experience and passion to build a platform that is used throughout Target to plan and execute on Merchandising, Price, Promo, Marketing and other key data initiatives. You can be part of a team that is not only utilizing cutting-edge technologies but also driving topline growth for Target.

The Product team that you'd be part of built a solution that allows business users to run complex ad-hoc data queries in real time, across dozens of data domains and 50+ billion records without requiring the need ask a Data Analyst for help. The React UI is designed to be easy to user and catches and prevents users from making logical mistakes in their queries. And the Java based backend is an extremely complex engine that knows how to plan out the execution path of even the most complicated queries that the users can come up with. Your role as the Sr Data Engineer would focus on the data preparation, cleansing and ingestion into the platform via Hadoop, ETL and Spark streaming ingestion techniques with extremely large datasets!

Key Responsibilities:
  • Data Profiling, error detection and working with source teams on resolution
  • Coaching the team and Product Owners on the nuances of specific data sets within Target and designing an index that works within the team's framework
  • Collaborate with engineers and partners to ensure development meets business needs
  • Understanding of data models with sources from different data systems including relational database and conceptual understanding of at least one NoSQL storage
  • Research and proof-of-concept initiatives in new and emerging technology spaces
  • Drive evaluation and learn new tools and technologies to keep technology stack modern as needed for the Product solution

About you:
  • 4 year degree in Computer Science, Applied Mathematics, Physics, Statistics or area of study related to data mining or equivalent experience
  • 5+ years of experience in end to end software development
  • Demonstrates strong domain-specific knowledge regarding Target's technology capabilities and key competitors' products and differentiating features
  • Working knowledge on package-specific configuration and deployment along with ability to build custom solutions
  • Designs new testing methods and resolves routine and non-routine technical issues with minimal assistance
  • Builds strong commitment within the team to support the appropriate team priorities
  • Clearly communicates Agile concepts to partners within Product teams
  • Demonstrates a solid understanding of the impact of own work on the team and/or guests
  • Writes and organizes code using multiple computer languages, including distributed programming and understand different frameworks and paradigm
  • Delivers high-performance, scalable, repeatable, and secure deliverables with broad impact (high throughput and low latency)
  • Influences and applies data standards, policies, and procedures
  • Maintains technical knowledge within areas of expertise
  • Stays current with new and evolving technologies via formal training and self-directed education

Desired Qualifications:
  • Hands on experience with Big Data querying tools, such as Pig, Hive, and Impala
  • Hands on experience with Spark streaming using Python and/or Scala
  • Hands on experience with integration of data from multiple data sources and RDBMS
  • Hands on experience with Kafka messaging systems
  • Good knowledge of building stream-processing systems using solutions such as Storm or Spark-Streaming
  • Proficient understanding of distributed computing principles
  • Proficiency with Hadoop v2, MapReduce, HDFS
  • Management of Hadoop cluster and all included services
  • Ability to solve any ongoing issues with operating the cluster
  • Good understanding of Lambda Architecture along with its advantages and drawbacks
  • Strong problem solving and thought partnership skills
  • Good verbal and written communication skills
  • Working with test-driven development and software test automation

Americans with Disabilities Act (ADA)

Target will provide reasonable accommodations (such as a qualified sign language interpreter or other personal assistance) with the application process upon your request as required to comply with applicable laws. If you have a disability and require assistance in this application process, please visit your nearest Target store or Distribution Center or reach out to Guest Services at 1-800-440-0680 for additional information.


Similar jobs

Similar jobs