Data Engineer

The Data Engineer will deploy, operate and evolve data infrastructure and tooling to enable NATO to process data efficiently, securely and reliably across cloud-connected and airgapped environments. The role involves hands-on engineering with autonomy to address complex challenges and drive initiatives in support of NATO's digital transformation.

Responsibilities

• Deploy, configure and maintain data platform components (orchestrators, catalogues, object stores, query engines and lakehouse tooling) on containerized infrastructure using GitOps practices, across cloud-connected and airgapped/disconnected environments including the tactical edge.
• Apply DataOps practices to automate, monitor and continuously improve data pipelines – pipeline observability, data quality automation, CI/CD for data and metadata management – and provide basic MLOps support to enable AI/ML workloads.
• Oversee, coordinate, and provide guidance to industry partners to ensure their work aligns with project objectives, timelines, and quality standards.
• Advise NATO Enterprise and Alliance partners on data architectural frameworks, including Data Mesh, Data Space and federated governance models, contributing to standardisation and capability coherence across the Alliance.
• Take ownership of platform and architectural challenges, proactively identifying opportunities to improve capability and coherence and driving initiatives with a high degree of autonomy.

Requirements

• A Bachelor’s degree from a nationally recognized/certified university in a related discipline (such as Computer Science or Data Science and Engineering), with at least 2 years of related experience.
• Exceptionally, the absence of a university degree may be compensated by the demonstration of particular abilities or experience of interest to NCIA – that is, at least 6 years of extensive and progressive expertise in duties related to the function of the post.
• Strong proficiency in Python for building and integrating data engineering components.
• A strong understanding of data engineering concepts – data warehousing, ETL/ELT processes and data governance – and experience planning and maintaining data lakes and pipelines.
• A solid understanding of DataOps principles: pipeline observability and alerting, data quality automation, CI/CD for data assets, lineage tracking and metadata management.
• Hands-on experience deploying and operating data platform tooling (orchestrators, metadata catalogues, object stores, query engines) in cloud and/or airgapped environments.
• A solid understanding of containerization and orchestration (Docker, Kubernetes), and experience with at least one major cloud platform (AWS, Google Cloud or Azure).
• Sound software development practices (version control, CI/CD, unit/functional testing) and a solid understanding of data security best practices.
• A thorough knowledge of English (written and spoken) is essential.
• Desirable experience includes: basic MLOps familiarity (ML pipeline packaging, model versioning and registry, experiment reproducibility and model serving); practical experience in airgapped, disconnected or operationally isolated environments; exposure to Data Mesh and federated data architectures; knowledge of data standards and governance frameworks (such as DCAT or ODPS); multi-classification security approaches such as attribute-based access control (ABAC) or policy enforcement (OPA); and experience with AI/ML libraries (TensorFlow, PyTorch, Transformers).

Skills

• Python Programming
• Data Warehouse
• ETL
• ELT Processes
• Data Governance
• Data Lake Management
• Data Pipeline Development
• DataOps Principles
• Pipeline Observability
• Alerting Systems
• Data Quality Automation
• CI/CD for Data Assets
• Lineage Tracking
• Metadata Management
• Data Platform Deployment
• Metadata Catalogues
• Object Storage
• Query Engines
• Cloud Platforms
• Docker Containers
• Kubernetes
• Version Control Systems
• Unit testing
• Functional Testing
• Data security practices

Languages

English