Latest Vacancies

Senior Data Engineer

Industry: Engineering & Technical

Job Description

JOB PURPOSE AND SCOPE

The Senior Data Engineer will be accountable for and have lead responsibility for designing, building, and maintaining our data infrastructure, ensuring the efficient and reliable definition, processing, storage, and analysis of large datasets.

The role is integral to the Force being able to create value from all the data it collects by leading the data engineering activities required to improve and combine data from first contact with the public through to crime intelligence and the back-office

They will make a significant contribution to making sure our data, and data shared by others is of the highest quality, available, usable and shareable by rigorous standard setting and quality control

The role requires a highly comprehensive level of knowledge, functional understanding of modern data technologies and platforms and as sole accountable data engineer for the Force will need to excel in the application data engineering tools and methodologies.

They will cultivate strong and effective working relationships with individuals, teams and colleagues, both internal and external to the Force. In particular they will work performance, information and data analysts specialising in crime, justice, vulnerable populations, public safety across the Force, from other Forces and public sector partners to build products and services which deliver insight and value to the Force and  its ability to make Lincolnshire a safe place to live, work and visit

Demonstrate the Force values at all times. All staff involved in carrying out functions in this role will do so in accordance with the principles of the Code of Ethics. The aim of the Code of Ethics is to support each member of the policing profession to deliver the highest professional standards in their service to the public.

CORE WORK AREAS

-  Lead the design and implement of scalable, efficient, and robust data pipelines to extract, transform, and load (ETL) data from various sources into data storage and platform solutions.

-  Engage and collaborate with cross-functional teams, to understand their data requirements and translate them into technical solutions.

Build and link data sets so that data scientists and analysts can easily derive insights, including accessing sensitive data held in back-end systems, extracting and loading into the data storage environment (warehouse or lake) or platform (fabric or mesh, whichever strategically appropriate) from which they be provisioned.

-  Write ETL scripts and code to make sure the ETL process performs optimally, leading on extracting data, joining and helping analysts to derive insights

-  Lead the build and be accountable for the optimising the performance of pipelines and APIs that connect operational systems, data for analytics and business intelligence (BI) systems, troubleshooting and resolving issues as they arise, to ensure the timely delivery of high-quality data.

-  Provide leadership to the business in automating its manual data flows and reporting processes, enabling their scaling up and repeatable use, working with leaders across the organisation to identify and elicit the benefits of speeding up and easing access to data.

-  Lead the development and maintain data models, schemas, and structures that optimize data storage and retrieval for analytics and reporting purposes.

-  Work with the Principal DBA and Principal Solutions Developer to actively optimise code within the Force's systems to ensure processes perform optimally

-  Identify ways to improve data reliability, efficiency and quality

-  Engage and work closely with data stakeholders to understand their data needs and provide support and guidance on how to use the data engineering infrastructure effectively.

-  Ensure data engineers, information analysts and BI developers have integral working practises that product optimal and reusable visualisations and reports

-  Work across the Force to extract, clean, combine and store data in a way that allows it to be exploited and as an asset to help service delivery and decision making.

-  Lead major data engineering activities within business-critical live systems, working with services to understand and improve data quality. This may include purging data in accordance with regulation and internal policy.

-  Work with the Information Management Unit (IMU) colleagues implementing data quality checks, data validation, and data cleansing processes to help ensure the accuracy, consistency, and integrity of the data.

-  Uphold data governance policies and standards, ensuring compliance with data privacy and security regulations.

-  Ensure the data is anonymise wherever necessary, working within the principles of GDPR and with suitable data protection and sharing agreements in place.

-  Lead on documenting data engineering processes, workflows, and technical specifications to facilitate knowledge sharing and ensure reproducibility of analyses.

-  Engage with professional networks, actively researching trends and advancements in data engineering, evaluating and adopting new tools, techniques, and best practices where these enhance the data engineering infrastructure, its reliability, security, and scalability.

-  Make significant contributions to data strategy, setting standards and leading the professional practice for data engineering.

-  Engage with and manage commercial relationships with data engineering partners; overseeing any project work undertaken by partners and suppliers, including virtual and remote management; set direction for suppliers, manage supplier risk and communicate our needs clearly.