Lead Data Engineer

Minneapolis, Minnesota, United States
Oct 09, 2016
Mar 09, 2017
Business, Engineering
Employment Status
Full Time

The Sr. Data Engineer is responsible for the development of high performance, distributed computing tasks using Big Data technologies such as Hadoop, NoSQL, text mining and other distributed environment technologies based on the needs of the organization. Also responsible for analyzing, designing, programing, debugging and modifying software enhancements and/or new products used in distributed, large scale analytics solutions.


1)Designing and Implementation

·Assist in the definition of software architecture to ensure that the online organization's software solutions are built within a consistent framework

·Assist in the decision-making process related to the selection of software architecture solutions

·Implement architectures to handle web-scale data and its organization

·Execute strategies that inform data design and architecture in partnership with enterprise-wide standards

·Assist in creating documents that ensure consistency in development across the online organization.Implements and improves core software infrastructure

·Develop interactive dashboards, reports, and analysis templates

·Implement and support a platform that can provide ad-hoc access to large datasets

2)Scope of Work

·Translate strategic requirements to ensure effective solutions meet business requirements.

·Provide consultation on complex programs

·Review and approve specifications to ensure consistency in approach and use

·Assist partners as a technical and business-savvy resource to software engineering personnel on a range of software design issues

·Perform systems and applications performance characterization and trade-off studies through analysis and simulation

·Research improvements in coding standards



  • 5-7 years' experience in developing software applications including: analysis, design, coding, testing, deploying and supporting of applications
  • BS degree in Computer Science, Applied Mathematics, Physics, Statistics or area of study related to data sciences and data mining
  • Proficient in application/software architecture (Definition, Business Process Modeling, etc.)
  • Understand application/software development and design
  • Understanding of Java - Maven, Gradle
  • Collaborative personality, able to engage in interactive discussions with the rest of the team
  • Inquisitive on Big Data technology; current on new ideas and tools
  • Apache Hadoop Development Experience

-Reading and writing with HDFS APIs

-Apache Storm topology development

-Interacting with various end points like: HDFS, Apache Hive, Apache Kafka, Apache Cassandra, Apache HBase

-Apache Storm core development

-Writing workflows in Apache Oozie

-Writing Java programs to interact with Hive JDBC

-Creativity to write and implement custom metrics for streaming technology

Similar jobs

Similar jobs