- Build end to end data flows from sources to fully curated and enhanced data sets. This can include the effort to locate and analyze source data, create data flows to extract, profile, and store ingested data, define and build data cleansing and imputation, map to a common data model, transform to satisfy business rules and statistical computations, and validate data content.
- Produce data building blocks, data models, and data flows for varying client demands such as dimensional data, data feeds, dashboard reporting, and data science research & exploration
- Create, modify and maintain BDM code and complex SQL for BI/DW data flows
- Learn to program in Spark and Python where BDM Spark co-generation is not adequate Produce automated tests of data flow components
- Use knowledge of the business to automate business-specific tests for data content quality
- Automate code deployment and promotion
- Build automated orchestration with BDM and error handling for use by production operation teams •Provide technical expertise to diagnose errors from production support teams
- Collaborate with team members in an Agile team (e.g., Scrum)
- Participate as both leader and learner in team tasks for architecture, design and analysis
- Coordinate within collocated on-site teams as well as with work plans for off-shore resources
Minimum of two years college with at least ten years involvement in systems analysis and application development, or an equivalent combination of education and experience.
Experience with development of metadata-driven and fully parameterized BDM environments
Advanced SQL coding skills for data transformations, profiling, and query tasks
Unix commands and scripting
Experience in agile environments such as scrum and Kanban
Preference for experience in Hadoop fundamentals and architecture: HDFS, map-reduce, job performance
Preference for open source big data skills in tools such as Hive, HBase, parquet, Spark SQL
Programming in a language such as Python (preferred), Scala, etc.
IDEAL QUALIFICATIONS PREFERREDFive years of ETL tool experience which can be from other tools such as Talend, DataStage, and Informatica Bachelor’s degree in Computer Science or related discipline preferred
At M&T, we understand what's important when you're considering a career change: a company with a solid history of responsible growth and positive earnings. A company committed to the communities it serves, one that rewards performance and provides employees with opportunities to direct their own career paths. M&T is a company with a focus on the future.
Job Posting: Nov 6, 2018, 11:08:31 AM
Unposting Date: Ongoing