Summary:
The role will be responsible for applying technical expertise to deliver complex development projects and high-quality pipelines using Azure Data Factory. The Data Engineer role will be part of the global Software Engineering Development Team and will be responsible for the design, development and maintenance of the Outsourcing NNA Technology Data Platform applications. The role requires a passion for technical design, development, problem solving, optimization of data and data pipeline architecture.
The role will also be required to give inputs to the reporting requirements of the Data Platform using PowerBI. The role will also design and build reports and dashboards, assisting in development of complex ad hoc queries and analyses.
Responsibilities:
- Building and maintaining data pipelines for data integration and processing, ensuring data security and privacy prerequisites are followed. Create, monitor, and maintain data pipeline architecture.
- Understand business requirement and actively provide inputs from Data perspective.
- Understand the underlying data and flow of data.
- Ensure data quality and accuracy through testing and validation
- Optimize data processing performance through tuning and monitoring. Design and implement optimized data sets.
- Performs coding, debugging, testing and troubleshooting for moderately complex issues, acting as a point of escalation to junior team members.
- Maintains and enhances systems by fixing complicated errors, raising risks and escalating issues where necessary
- Designs high-quality solutions in accordance with timelines and specifications to meet user requirements.
- Ensures all activities adhere to the relevant processes, procedures, standards and technical design.
- Oversees and checks the work of junior team members, ensuring that deadlines are met, and outputs are of a high quality.
- Proactively identifies and leads the implementation of continuous improvement items with the support of the Team Lead and Manager.
- Provides feedback and assists with either formal training or mentoring to junior team members to assist their development.
- Adheres to the development coding standards, the operational framework and WTW Excellence guidelines.
- Works closely with QA, Product/Business Analysts, and other Software Engineering functions to ensure high quality on-time delivery
- Demonstrate learning adaptability, understanding of the implications of technical issues on business requirements and / or operations.
- Create reports and visualizations providing insight into data sets.
- Troubleshoot and resolve data pipeline and reporting performance issues.
Requirements:
- 6+ years’ experience as a Data Engineer
- Basic understanding of data modeling- OLTP vs dimensional modeling is preferred
- Knowledge in code quality, performance and security
- Experience on source code control systems such as Git or Bitbucket
- Familiarity on continuous delivery and continuous integration practices
- Willingness to work in a fast-paced collaborative team environment that has tight deadlines.
- Ability to learn and evaluate new tools, concepts, and challenges quickly.
- Client service focus and flexibility in supporting client requests.
- Strong analytical and problem-solving skills.
- Commitment to quality and continuous improvement.
- Strong written and verbal communication skills.
- Proficient in Azure Data Factory. Experience building and optimizing data pipelines, architectures, and data sets
- Experience with Azure data storage solutions such as Azure Synapse, Azure SQL Database, Azure Cosmos DB, and Azure Data Lake Storage
- Experience creating reports using Power BI.
- Experience with relational databases and proficiency using SQL