QYW285 - DATA ENGINEER

Jt 1.png

QYW285 - DATA ENGINEER

IT

Remote

Senior

Full-time

Responsibilities

  • Ensures data integrity and quality.

  • Develop, maintain, and optimize data pipelines.

  • Work with stakeholders including the Product, Business, and Analytic teams to assist with data-related issues and support their data transformation requirements.

  • Work directly with management and users to gather requirements, define business needs, provide status updates, and build relationships.

  • Work with stakeholders to identify opportunities for leveraging data to drive business solutions and support their data infrastructure needs.

  • Manage external technical communication with partners and vendors.

  • Contribute to improving data governance policies.

  • Keep current with technical and industry developments

Infomation

Offered Salary

3,000 $ - 4,000 $

Skills

Requirements

  • At least 3+ years of relevant experience in developing scalable secured fault-tolerant, resilient & mission-critical Big Data platforms.

  • Able to maintain and monitor the ecosystem with high availability.

  • Must have a sound understanding of all Big Data components & Administration Fundamentals. Hands-on in building a complete data platform using various open source technologies.

  • Must have good fundamental hands-on knowledge of Linux and building big data stacks on top of AWS infrastructure.

  • Understanding of big data principles and related technologies like HDFS, Spark, Presto, Airflow, Kafka, Apache Atlas, etc.

  • Good knowledge of Complex Event Processing systems like Spark Streaming, Kafka, Apache Flink, Beam, etc.

  • Able to drive DevOps best practices like CI/CD, containerization, blue-green deployments, secrets management, etc in the Data ecosystem.

  • Able to develop an agile platform with auto-scale capability up & down as well vertically and horizontally.

  • Able to develop an observability and monitoring ecosystem for all the components in use in the data ecosystem.

  • Proficiency in at least one of the programming languages Java, Scala, Python or Go.

  • Proficient understanding of distributed computing principles.

  • Familiar or prone to adopt design thinking methods.

  • Openness to experimenting with new techniques and new ways of working.

  • Ability to build internal clients relationships, and work effectively across functions and geographies.

  • Ability to design solutions independently based on high-level architecture.

  • You are a thought leader and are responsible for delivering complex projects successfully.

  • Mentor and guide junior members and contribute to global department expertise, deliverables quality, and skills development.

  • Excellent written and verbal communication skills for coordinating across teams.

Benefits

Salary range: Senior: up to USD 3500 net

Lead: up to USD 4000 net