QYW285 - DATA ENGINEER
QYW285 - DATA ENGINEER
Ensures data integrity and quality.
Develop, maintain, and optimize data pipelines.
Work with stakeholders including the Product, Business, and Analytic teams to assist with data-related issues and support their data transformation requirements.
Work directly with management and users to gather requirements, define business needs, provide status updates, and build relationships.
Work with stakeholders to identify opportunities for leveraging data to drive business solutions and support their data infrastructure needs.
Manage external technical communication with partners and vendors.
Contribute to improving data governance policies.
Keep current with technical and industry developments
At least 3+ years of relevant experience in developing scalable secured fault-tolerant, resilient & mission-critical Big Data platforms.
Able to maintain and monitor the ecosystem with high availability.
Must have a sound understanding of all Big Data components & Administration Fundamentals. Hands-on in building a complete data platform using various open source technologies.
Must have good fundamental hands-on knowledge of Linux and building big data stacks on top of AWS infrastructure.
Understanding of big data principles and related technologies like HDFS, Spark, Presto, Airflow, Kafka, Apache Atlas, etc.
Good knowledge of Complex Event Processing systems like Spark Streaming, Kafka, Apache Flink, Beam, etc.
Able to drive DevOps best practices like CI/CD, containerization, blue-green deployments, secrets management, etc in the Data ecosystem.
Able to develop an agile platform with auto-scale capability up & down as well vertically and horizontally.
Able to develop an observability and monitoring ecosystem for all the components in use in the data ecosystem.
Proficiency in at least one of the programming languages Java, Scala, Python or Go.
Proficient understanding of distributed computing principles.
Familiar or prone to adopt design thinking methods.
Openness to experimenting with new techniques and new ways of working.
Ability to build internal clients relationships, and work effectively across functions and geographies.
Ability to design solutions independently based on high-level architecture.
You are a thought leader and are responsible for delivering complex projects successfully.
Mentor and guide junior members and contribute to global department expertise, deliverables quality, and skills development.
Excellent written and verbal communication skills for coordinating across teams.
Salary range: Senior: up to USD 3500 net
Lead: up to USD 4000 net
3,000 $ - 4,000 $