You will join the Cabinet Office Digital, Data, Insights and AI team as a Software Developer in a multidisciplinary Agile team, delivering services for a large data warehouse and analytics platform (GRID).
As a software engineer, you will initially have a core focus on back-end development.
You will:
Lead development of data engineering requirements from requirement through to live support.
Complete information assurance activities for data transfers containing sensitive data (e.g., Data Protection Impact Assessments, Privacy Notices).
Carry out performance and incident monitoring to ensure the smooth running of data services.
Support the wider team with broader priorities, including front-end design.
Coach and mentor junior developers, sharing good engineering practices.
The role responsibility may slowly change over time.
Your main focus will be:
Python and Extract, Transform and Load (ETL) processes.
You will require knowledge of:
PySpark (AWS Glue) to build scalable data pipelines.
AWS Lake Formation to implement Attribute-Based Access Control (ABAC).
Apache Iceberg to store data and optimise query performance.
Key broader technologies used by the wider team:
Python as the primary programming language for general tasks.
TypeScript to design front-end services using React.
JavaScript/Node.js for backend services.
YAML/JSON (CloudFormation) and Terraform (HCL) for infrastructure as code.
Terraform, CodePipeline, and GitHub Actions for infrastructure deployment.
Amazon Web Services (AWS) for hosting digital services.
Kubernetes and Docker for containerisation.
Apache Iceberg, PostgreSQL, SQL Server, and Redis as databases.
OpenSearch for vector stores used for storing and querying high-dimensional embeddings.
Knowledge graphs, including the use of triplestores for storing and querying RDF data.
Amazon Bedrock for hosting LLMs.
Design System for interface design.
