Job Description
Skill:
Pyspark Lead
• 810 years of total experience with good knowledge of Knowledge of Pyspark, Big Data experience, AWS EMR, S3, IAM, Lambda, SNS, SQS.
• Good python scripting skills.
• Spark UI/Optimization/debugging techniques.
• Intermediate SQL exposure
Subquery, Joins, CTEs.
• Good understanding of Database technologies.
• Excellent communication skills to liaise with Business & IT stakeholders.
• Expertise in planning execution of a project and efforts estimation.
• Understanding of Data Vault, data mesh and data fabric architecture patterns.
• Exposure to working in Agile ways of working.
Must have:
• Knowledge of Pyspark, Big Data experience, AWS EMR, S3, IAM, Lambda, SNS, SQS.
• Good Spark, Pyspark, Big Data experience.
• Spark UI/Optimization/debugging techniques.
• Good python scripting skills.
• Intermediate SQL exposure
Subquery, Joins, CTEs.
• Database technologies.
• AWS EMR, S3, IAM, Lambda, SNS, SQS.
Salary Range - $125,000-$135,000 a year
#LI-NR3
Responsibilities
- Exposure to working in Agile ways of working
Requirements
- 810 years of total experience with good knowledge of Knowledge of Pyspark, Big Data experience, AWS EMR, S3, IAM, Lambda, SNS, SQS
- Good python scripting skills
- Good understanding of Database technologies
- Excellent communication skills to liaise with Business & IT stakeholders
- Expertise in planning execution of a project and efforts estimation
- Understanding of Data Vault, data mesh and data fabric architecture patterns
- Knowledge of Pyspark, Big Data experience, AWS EMR, S3, IAM, Lambda, SNS, SQS
- Good Spark, Pyspark, Big Data experience
- Spark UI/Optimization/debugging techniques
- Good python scripting skills
- Intermediate SQL exposure