NEW JOB OPENING
PYSPARK DEVELOPER WITH KUBERNETES
IN Nyc, NY, USA!

 

Date Posted: 12/02/2024
Hiring Organization: Rose International
Position Number: 475026
Job Title: PySpark Developer with Kubernetes
Job Location: Nyc, NY, USA, 10013
Work Model: Hybrid
Shift: 3 days/week onsite
Employment Type: Temporary
Estimated Duration (In months): 13
Min Hourly Rate($): 90.00
Max Hourly Rate($): 95.00
Must Have Skills/Attributes: Apache Spark, Banking/Financial, Big Data, Cloud, Docker, Hive, Kubernetes, PySpark

 

Job Description
***Only qualified PySpark Developer candidates located near the NYC, NY area to be considered due to the position requiring an onsite presence***



Education:

• Bachelor’s degree in Computer Science, Information Technology, or a related field



Requirements:

• Proven experience as a PySpark Developer, with a solid understanding of Spark architecture and data processing concepts

• Hands-on experience with Kubernetes for container orchestration and deployment

• Strong knowledge of big data technologies, including Hadoop, HDFS, Spark, and Kafka (preferred)

• Experience with cloud platforms (AWS, Google Cloud Platform, or Azure) and their Kubernetes services

• Familiarity with CI/CD tools and automated deployment processes

• Proficient in scripting languages (Python, Shell scripting) and experience with version control (Git)

• Strong analytical and problem-solving skills, with an emphasis on optimizing data workflows

• Excellent communication and collaboration abilities, especially within cross-functional teams



Preferred Skills:

• Experience with Docker and containerization

• Familiarity with monitoring and logging tools for Kubernetes, such as Prometheus and Grafana

• Knowledge of other big data tools and frameworks (Hive, HBase, Flink, etc.)

• Certifications in Apache Spark, Kubernetes, or relevant cloud platforms



We are looking for an experienced PySpark Developer with strong expertise in Kubernetes to join our dynamic team. The ideal candidate will be responsible for designing, developing, and maintaining big data solutions using Apache Spark (PySpark), deploying scalable applications on Kubernetes, and ensuring smooth data pipeline operations. This role requires hands-on experience with data engineering, container orchestration, and cloud technologies.



Responsibilities:

• Develop and maintain data pipelines using PySpark for data transformation, aggregation, and loading

• Design, implement, and manage containerized applications on Kubernetes to ensure scalability, reliability, and efficiency

• Collaborate with data engineering and analytics teams to optimize PySpark jobs for performance and resource utilization

• Automate the deployment and scaling of PySpark jobs using Kubernetes tools and best practices

• Implement and monitor data workflows, ensuring high availability and resilience

• Troubleshoot and resolve issues related to data processing, Kubernetes deployments, and cloud infrastructure

• Work closely with DevOps and infrastructure teams to manage Kubernetes clusters and improve CI/CD pipelines for big data applications

• Create detailed documentation for data workflows, processes, and Kubernetes configuration

• Stay updated on the latest industry trends, particularly within the big data and container orchestration spaces
  • **Only those lawfully authorized to work in the designated country associated with the position will be considered.**

  • **Please note that all Position start dates and duration are estimates and may be reduced or lengthened based upon a client’s business needs and requirements.**


 

Benefits:
For information and details on employment benefits offered with this position, please visit here. Should you have any questions/concerns, please contact our HR Department via our secure website.

California Pay Equity:
For information and details on pay equity laws in California, please visit the State of California Department of Industrial Relations' website here.

Rose International is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, age, sex, sexual orientation, gender (expression or identity), national origin, arrest and conviction records, disability, veteran status or any other characteristic protected by law. Positions located in San Francisco and Los Angeles, California will be administered in accordance with their respective Fair Chance Ordinances.

If you need assistance in completing this application, or during any phase of the application, interview, hiring, or employment process, whether due to a disability or otherwise, please contact our HR Department.

Rose International has an official agreement (ID #132522), effective June 30, 2008, with the U.S. Department of Homeland Security, U.S. Citizenship and Immigration Services, Employment Verification Program (E-Verify). (Posting required by OCGA 13/10-91.).

 

Apply Now

Send me a reminder to complete this application

 

About Rose

  • Founded in 1993
  • Office Locations Across the U.S.
  • 150+ Clients: Corporations and Government Agencies
  • Employee Oriented Company
  • Challenging Assignments Across the U.S.
  • Continuous Professional Development

Working for Rose International was the most pleasant assignment I have ever had. They were always on top of situations when necessary, and very helpful. I was very proud to be an employee of Rose International, and would recommend anyone to try to work with them.

Melvon, Consultant

I had a very positive experience working for Rose. The entire process is very efficient and easy.

Joanne, Consultant

Rose is an assembly of people grounded in honesty, truth and dignity for all of its employees and contractors.

Samba, Consultant

It is a great pleasure being a part of the Rose International Team.

Toni, Consultant

It was great working for Rose International. Everyone was extremely helpful.

Rosann, Consultant

EMPLOYEE COMMENTS

  • We want you to work with us, but don't take our word for it. Take a look at this sampling of employee comments. They speak for themselves.