Job Description
Senior Data Engineer Job Description: Experience: 10+ Years
Job Type: W2 Contract Only
Job Summary We are seeking a Senior Data Engineer with 10+ years of experience in designing, developing, and supporting scalable data solutions. The ideal candidate will have strong experience with Python, SQL, ETL/ELT, cloud platforms, data pipelines, data warehousing, and distributed data processing .
Required Qualifications & Technical Skills: - 10+ years of experience in Data Engineering, ETL, data integration, or related technologies .
- Strong hands-on experience with Python and advanced SQL .
- Experience designing and developing scalable ETL/ELT data pipelines .
- Strong understanding of data warehousing, data lakes, data modeling, and data integration concepts.
- Experience with Apache Spark / PySpark for large-scale data processing.
- Hands-on experience with Databricks or similar modern data platforms.
- Experience working with cloud platforms such as AWS, Azure, or Google Cloud Platform .
- Experience with relational databases such as SQL Server, Oracle, PostgreSQL, MySQL, Snowflake, or similar .
- Experience developing data ingestion and transformation processes from databases, APIs, files, and other sources.
- Strong knowledge of data quality, validation, reconciliation, and error handling .
- Experience with workflow orchestration tools such as Airflow, Azure Data Factory, AWS Glue, or similar .
- Experience with Git, CI/CD, and automated deployment processes .
- Strong troubleshooting and performance optimization skills.
- Experience working in Agile/Scrum environments .
Responsibilities: - Design, develop, and maintain enterprise-scale data pipelines and integration solutions .
- Develop complex data transformations using Python, SQL, Spark, and PySpark .
- Build data ingestion processes from structured and unstructured data sources.
- Design and maintain data warehouse and data lake solutions.
- Optimize SQL queries, Spark jobs, and data pipelines for performance and scalability.
- Implement data validation, quality checks, monitoring, and reconciliation processes.
- Develop and manage data workflows using orchestration tools.
- Work with cloud storage, databases, APIs, and enterprise data platforms.
- Support production data pipelines and troubleshoot data-processing issues.
- Implement CI/CD practices for data engineering code and deployment.
- Collaborate with data architects, analysts, developers, QA teams, and business stakeholders.
- Participate in technical design, code reviews, sprint planning, and other Agile activities.
- Document data flows, technical solutions, data mappings, and operational processes.
- Mentor junior and mid-level data engineers and contribute to data engineering best practices.
Preferred Skills: - Experience with Delta Lake / Lakehouse architecture .
- Knowledge of AWS S3, Azure Data Lake, ADLS, or Google Cloud Storage .
- Experience with Snowflake or other cloud data warehouses.
- Knowledge of Kafka, Event Hubs, or other streaming technologies .
- Experience with Terraform or Infrastructure as Code .
- Knowledge of Docker and Kubernetes.
- Experience with data governance, metadata, lineage, and security.
- Exposure to real-time and near-real-time data processing.
Domain Knowledge: - Strong understanding of enterprise data integration, reporting, analytics, and data warehouse environments .
- Experience working with large-volume transactional and analytical data.
- Understanding of business requirements, data mapping, data transformation, and downstream reporting needs.
- Experience working with cross-functional technical and business teams.
For applications and inquiries, contact:hirings@openkyber.com
Job Tags
Contract work