Job Description

Senior Data Engineer

Technology

Miami, FL (Hybrid)

$150,000 - $170,000 + equity

The Company

Our client is revolutionizing operational safety in critical industries. They specialize in leveraging computer vision to provide real-time insights for the Construction industry. Their mission is to eliminate failures due to inadequate visual inspection, ensuring safety, reliability, and compliance.

Role Description

As a Senior Data Engineer, you will play a pivotal role in designing and optimizing scalable data pipelines that power machine learning and analytics applications. You will be responsible for architecting a robust data infrastructure, collaborating with ML engineers, and ensuring efficient storage and processing of large-scale industrial and vision data. This is an opportunity to make a significant impact on a rapidly growing team that values technical excellence and innovation.

Key Responsibilities

• Architect and build scalable data pipelines using Dagster to transform and move data for analytics and ML.
• Design and optimize storage solutions for large-scale industrial and vision data, ensuring efficient retrieval and processing.
• Develop data ingestion frameworks for real-time and batch processing of images, video, and metadata.
• Work with ML engineers to structure data effectively for model training and experimentation.
• Deploy and manage data pipelines in Kubernetes-based environments.
• Automate CI/CD workflows for data infrastructure using GitLab CI/CD.
• Manage AWS-based data infrastructure, leveraging Terraform for Infrastructure as Code.
• Implement data governance best practices, ensuring data quality, lineage tracking, and metadata management.
• Optimize batch and real-time processing frameworks for scalability and performance.
• Provide technical leadership, establishing best practices and guiding future data scaling strategies.

Requirements

• 5+ years of experience in data engineering with a focus on scalable, production-grade data infrastructure.
• Strong expertise in Python (Pandas, PyArrow, Dask, etc.).
• Hands-on experience with data orchestration tools (Dagster, Prefect, or Airflow).
• Proficiency in Kubernetes for data pipeline orchestration.
• Experience deploying infrastructure via Terraform (or similar IaC tools).
• Cloud expertise, preferably AWS, including S3, EKS, Lambda, Glue, and RDS.
• Experience with streaming and event-driven architectures such as Apache Ray Core, Kafka, Kinesis, Pulsar, or Storm.
• Strong database skills, including SQL, NoSQL, and columnar storage (Postgres, BigQuery, ClickHouse).
• CI/CD expertise with GitLab for managing automated data pipeline deployments.

The Benefits

As a Senior Data Engineer, you will receive a base salary of $150,000 to $170,000 plus benefits like equity, health, dental and vision.

How to Apply

To apply, please submit your resume via the link on this page.

Key Words

Python, Dagster, Kubernetes, Terraform, AWS, Data Pipelines, CI/CD, Streaming Data, Apache Ray, Kafka, Computer Vision, Machine Learning, SQL, Big Data, Data Engineering

Job Tags

Similar Jobs