Data Engineer - 6 months - Ware - Hertfordshire
Global FMCG is looking for an experienced Data Engineer who will play a pivotal role in operationalizing data and analytics initiatives for digital business initiatives. The bulk of the data engineer's work would be in building, managing and optimising data pipelines and then moving these data pipelines effectively into production for key data and analytics consumers.
- Strong ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata and workload management. The ability to work with both IT and business in integrating analytics and data science output into business processes and workflows.
- Strong experience in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional data integration technologies. These should include [ETL/ELT, data replication/CDC, message-oriented data movement, API design and access] and upcoming data ingestion and integration technologies such as [stream data integration, CEP and data virtualization].
- Hands-on development exposure on Microsoft Azure cloud services-Spark structure streaming with Azure,Spark on HD insight, Azure Stream Analytics, Azure Data Factory(v1 & v2), Azure Blob Storage, Azure Data Lake Store(Gen1/Gen2), SQL DW, and Denodo, Talend, others
- Hands on Talend a MUST
- Basic experience working with popular data discovery, analytics and BI software tools like [Tableau, Qlik, PowerBI and others] for semantic-layer-based data discovery.
- Basic experience in working with [data governance/data quality] and [data security] teams and specifically [information stewards] and [privacy and security officers] in moving data pipelines into production with appropriate data quality, governance and security standards and certification.
- Demonstrated ability to work across multiple deployment environments including [cloud, on-premises and hybrid], multiple operating systems and through containerization techniques such as [Docker, Kubernetes, and others].
Spring Technology acts as an employment agency for permanent recruitment and an employment business for the supply of temporary workers. The Spring Group UK is an Equal Opportunities Employer.
By applying for this role your details will be submitted to Spring. Our Candidate Privacy Information Statement explains how we will use your information - please copy and paste the following link in to your browser: www.spring.com/candidate-privacy-information-statement