Job Summary We are looking for a highly skilled Senior Data Engineer to help build and enhance our Azure Databricks platform that powers economic data for Monetary Analysis, Forecasting and Modelling. This role focuses on designing scalable data pipelines, implementing complex data transformations, and ensuring high data quality and reliability across large datasets in Azure. Key Responsibilities Data Pipeline Development andamp; Optimisation Design, build and maintain scalable pipelines to ingest and process data from APIs, databases and financial data providers. Optimise pipelines for performance, reliability and cost efficiency. Implement automated data quality checks and validation rules. Data Transformation andamp; Processing Develop complex data transformations using Spark (PySpark/Scala). Build data processing logic for cleansing, enrichment and aggregation. Ensure accuracy and consistency across the data life cycle. Azure Databricks andamp; Cloud Engineering Work extensively with Azure Databricks, Unity Catalog and Delta Lake. Optimise Databricks workloads for performance and cost. Develop using SQL, Python, R, YAML and JavaScript. Data Integration Integrate data from relational databases, APIs and streaming sources. Implement best-practice data integration patterns. Collaborate with API teams to ensure seamless data exchange. Data Quality andamp; Governance Implement data governance and quality processes using Azure Purview. Enable data lineage tracking and ..... full job details .....