We are looking for a skilled Python Developer to join our team at SMX. As a remote position, this role requires expertise in ETL (Extract, Transform, Load) processes and Apache Airflow.
Key Responsibilities

The successful candidate will be responsible for:
• Designing and implementing robust and efficient data pipelines using Python and Apache Airflow.
• Collaborating with data analysts and other stakeholders to understand and meet their data requirements.
• Developing and implementing data validation processes to ensure high data quality.
• Troubleshooting and resolving issues related to data pipelines.
• Optimizing data extraction, transformation, and loading processes to improve efficiency and performance.
• Documenting and maintaining the design and details of data processes and schemas.
Requirements

To succeed in this role, you must have:
• Proficiency in Python: A strong understanding of Python programming language is crucial. This includes knowledge of Python libraries and frameworks like Pandas, NumPy, and Django.
• Expertise in Apache Airflow: Experience in designing, building, and maintaining data pipelines using Apache Airflow. Knowledge of Airflow's architecture, including DAGs and Operators, is important.
• ETL Processes: Proficiency in ETL (Extract, Transform, Load) processes, including data extraction from various sources, data transformation (cleaning, validating, aggregating, joining, etc.), and loading data into the end target (database or data warehouse).
• Database Knowledge: Strong understanding of SQL and NoSQL databases, including writing complex queries and database optimization techniques.
• Data Warehousing: Experience with data warehousing solutions like Amazon Redshift, Google BigQuery, or Microsoft Azure SQL Data Warehouse

Related Jobs