Generate a CV for this Job!
Based on your profile and this job description, you can create a tailored CV to apply directly.
Python Developer with Spark
Inficare • Virginia, US • On-site
Posted on: 27th November, 2024
Employment Type: FULLTIME
Job Description
Job Title: Python Developer with Spark Location: Columbus, OH (Onsite) Mode: Contract Job Description : Develop and maintain data platforms using Python, Spark, and PySpark. Handle migration to PySpark on AWS. Design and implement data pipelines. Work with AWS and Big Data. Produce unit tests for Spark transformations and helper methods. Create Scala/Spark jobs for data transformation and aggregation. Write Scaladoc-style documentation for code. Optimize Spark queries for performance. Integrate with SQL databases (e.g., Microsoft, Oracle, Postgres, MySQL). Understand distributed systems concepts (CAP theorem, partitioning, replication, consistency, and consensus). Skills: Proficiency in Python, Scala (with a focus on functional programming), and Spark. Familiarity with Spark APIs, including RDD, DataFrame, MLlib, GraphX, and Streaming. Experience working with HDFS, S3, Cassandra, and/or DynamoDB. Deep understanding of distributed systems. Experience with building or maintaining cloud-native applications. Familiarity with serverless approaches using AWS Lambda is a plus
Responsibilities
- Handle migration to PySpark on AWS
- Design and implement data pipelines
- Work with AWS and Big Data
- Produce unit tests for Spark transformations and helper methods
- Create Scala/Spark jobs for data transformation and aggregation
- Write Scaladoc-style documentation for code
- Optimize Spark queries for performance
- Integrate with SQL databases (e.g., Microsoft, Oracle, Postgres, MySQL)
Requirements
- Understand distributed systems concepts (CAP theorem, partitioning, replication, consistency, and consensus)
- Skills: Proficiency in Python, Scala (with a focus on functional programming), and Spark
- Familiarity with Spark APIs, including RDD, DataFrame, MLlib, GraphX, and Streaming
- Experience working with HDFS, S3, Cassandra, and/or DynamoDB
- Deep understanding of distributed systems
- Experience with building or maintaining cloud-native applications
Inficare
Technology
Location
Virginia, US
Job Type
FULLTIME
Loading...
Loading...
Generate a Tailored CV Before Applying!
A customized CV will make your application stand out. Use your profile and this job description to create the perfect CV!
Ready to Apply?
Click the button below to start your application process.
Related Jobs
Mindbank Consulting Group
4 days ago
Report Developer QuickSight and SSRS Jobs
Virginia, US
View Job
Capgemini
1 week ago
Python Lead (Philadelphia Locals or who can relocate)
Pennsylvania, US
View Job