Data Engineer with Expertise in Groovy and Spark at DataArt
You seem a little late
Most likely, this vacancy is closed because it has already made someone's Mondays happy.
But don't worry - we have something to offer you!
See other open features on the site
Our client is a software company providing client management applications to organizations in the healthcare industry. For the past 15 years, DataArt’s specialists have been helping them create online intake services for hospitals, medical centres, and private practices in the US.
An application is designed to help people schedule appointments, pay medical bills, find test results, and receive notifications about appointments with doctors.
Our team consists of 200+ people, largely focusing on integrations, the platform, and the intake process.
As a Software Engineer – (Data Engineer) on our team, you will contribute to the development of the fastest-growing direct-to-consumer health information platform in the US. Collaborating with a team of seasoned data engineers and architects, you will work with diverse healthcare data from both the US and international sources. This role will provide you with an in-depth understanding of various aspects of the healthcare industry, particularly healthcare data and technology. You will be responsible for building and maintaining complex data pipelines that ingest, process (using our algorithms), and output petabytes of data. Additionally, you will work closely with architects, product managers, and SMEs to develop and maintain algorithms that generate unique insights, aiding patients in finding better care more quickly.
Responsibilities
- Work closely with data engineers, the product team, and other stakeholders to gather data requirements, and design and build efficient data pipelines
- Create and maintain algorithms and data processing code in Java/Groovy
- Implement processes for data validation, cleansing, and transformation to ensure data accuracy and consistency
- Develop Python scripts to automate data extraction from both new and existing sources
- Monitor and troubleshoot the performance of data pipelines in Airflow, proactively addressing any issues or bottlenecks
- Write SQL queries to extract data from BigQuery and develop reports using Google’s Looker Studio
- Participate in daily stand-ups, sprint planning, and retrospective meetings
- Engage in peer code reviews, knowledge sharing, and assist other engineers with their work
- Introduce new technologies and best practices as needed to keep the product up to date
- Assist in troubleshooting and resolving production escalations and issues
Requirements
- Bachelor’s degree or equivalent programming experience
- 4-5 years of overall experience as a backend software developer, with at least 2 years as a Data Engineer using Spark with Java/Groovy and/or Python
- Strong coding skills, and knowledge of data structures, OOP principles, databases, and API design
- Highly proficient in developing programs and data pipelines in Java/Groovy or Python
- 2+ years of professional experience with Apache Spark/Hadoop
Nice to have
- Work experience with AWS (EMR, S3, lambda, EC2, glue, RDS)
- Work experience with SQL (MYSQL is a Plus) and NoSQL Databases
- Experience with Elasticsearch
- Experience with Python
- Experience with Scala (Zeppelin)
- Experience with Airflow or other ETL
- Certification or verified training in one or more of the following technologies/products: AWS, ElasticSearch
We offer
• Professional Development:
— Experienced colleagues who are ready to share knowledge;
— The ability to switch projects, technology stacks, try yourself in different roles;
— More than 150 workplaces for advanced training;
— Study and practice of English: courses and communication with colleagues and clients from different countries;
— Support of speakers who make presentations at conferences and meetings of technology communities.
• The ability to focus on your work: a lack of bureaucracy and micromanagement, and convenient corporate services;
• Friendly atmosphere, concern for the comfort of specialists;
• Flexible schedule (there are core mandatory hours), the ability to work remotely upon agreement with colleagues.
Thanks!
Our editors are notified.