DATA ENGINEER – MICROSOFT AZURE/FABRIC/DATABRICKS
Overview
We are looking for experienced Data Engineers within the Microsoft Azure ecosystem with skills in Databricks and Fabric. Ideal candidates will have solid experience in ETL processes and should be autonomous in managing large volumes of data. Experience in data ingestion and real-time processing will be considered an advantage.
Specific Skills
- At least 3 years of experience in similar roles, utilizing the MS Azure stack;
- Advanced skills in machine learning, with the ability to independently develop models and integrate complex models on Azure platforms;
- Excellent knowledge of Python and SQL;
- Experience in PySpark;
- Experience in designing and managing complex ETL pipelines with Azure Data Factory;
- Familiarity with big data tools and platforms such as Databricks, Azure Synapse and Azure Fabric;
- Experience with SQL and NoSQL databases;
- Familiarity with managing cloud solutions on Azure.
Responsible For
- Design, build, manage, and optimize ETL data pipelines on Azure Data Factory, Synapse, Fabric, and Databricks;
- Work with large datasets using Apache Spark (preferably PySpark);
- Implement storage solutions on Azure Data Lake, Delta Lake, One Lake leveraging the Data Lakehouse approach;
- Ensure data quality and integrity through advanced data governance practices;
- Respond to requests from various clients using the most suitable technologies available within the Azure ecosystem;
- Develop and maintain technical documentation and best practices for data management;
- Excellent knowledge of the Italian language.
Technological Stack
Experience with data ingestion and real-time processing technologies such as Apache Kafka, Azure Stream Analytics, and Spark Streaming.
Job Location
Full Remote
Job Level
Sr. Position