Azure Databricks Engineer Job at GovServicesHub, Durham, NC

STBhZXZIeXM1bnIyRjVFVjhVY1FwOGxnS2c9PQ==
  • GovServicesHub
  • Durham, NC

Job Description

Job Location: Triangle Area, NC (Hybrid)

Note: Candidate must be local to the Triangle region of North Carolina. Onsite presence is required on the first day to collect equipment and up to 1–2 days/month for in-person meetings.

• Each candidate must submit a government-issued ID (Driver’s License or Passport) and provide three professional references (names, official emails, and phone numbers).

Job Description:

The North Carolina Department of Information Technology – Transportation (NCDOT) seeks an Azure Databricks Engineer to support the Web Systems Team in planning and designing scalable ETL pipelines using Azure Databricks. This position focuses on creating resilient workflows to ingest and process data from various on-premise and cloud-based sources, ensuring robust data architecture and product delivery for NCDOT analysts and stakeholders.

Specialty Areas:

Azure Databricks Engineering – Designing and implementing Delta Lake architecture using Delta Live Tables and Spark.

ETL & Data Pipeline Development – Creating batch and streaming pipelines across cloud and hybrid data platforms.

Data Architecture & Quality – Building secure, well-documented, and validated pipelines aligned with enterprise data standards.

Stakeholder Collaboration – Working closely with development leads and business users to develop high-impact data products.

Responsibilities:

• Engineer robust ETL workflows using Databricks notebooks and Delta Live Tables for batch and stream processing.

• Develop data products aligned with business domain needs and collaborate with DBAs for data ingestion.

• Support secure data development practices and ensure sensitive data protection.

• Maintain documentation for all processes and implement data validation/testing procedures.

• Share architectural insights and collaborate with data teams across the organization.

Skill Matrix:

Skill
Experience (Years)

Azure Databricks & Delta Live Tables

2+ (Required)

Writing clean and efficient Databricks notebooks

2+ (Required)

Data engineering best practices (warehousing, lakes, Delta Lake)

2+ (Required)

Spark & Databricks SQL / PySpark

2+ (Required)

Cloud platforms (Azure, AWS, or GCP)

2+ (Required)

OLTP/OLAP, Azure Service Bus, Apache Spark

2+ (Required)

Object-oriented languages (C#, Java, or Python)

7+ (Desired)

Communication and interpersonal skills

5+ (Required)

Mandatory Requirements:

• Experience in Databricks and Spark development within Azure or similar cloud platforms.

• Proven ability to build and document scalable ETL pipelines.

• Strong communication skills and collaborative mindset.

• Must be a North Carolina local, available for occasional in-person meetings.

Preferred Qualifications:

• Experience with multiple object-oriented programming languages.

• Prior work with public sector data systems or government agencies.

Submission Requirements:

• Updated Résumé

• Government-issued ID (Driver’s License or Passport)

• Three professional references (Names, official emails, phone numbers)

Job Tags

Contract work, Local area,

Similar Jobs

Robert Half

Purchasing Assistant Job at Robert Half

Description We are looking for a detail-oriented and motivated Purchasing Assistant to join our team in Merced, California. In this Contract-to-permanent position, you will play a key role in supporting procurement operations and ensuring smooth collaboration with vendors... 

Factory Direct Marine & RV - Edgewater, FL

Receptionist Job at Factory Direct Marine & RV - Edgewater, FL

Factory Direct Marine & RV is seeking a receptionist. The ideal candidate has working knowledge of computers, social media, phones, and email...  ...any other characteristic protected by federal, state or local laws. At Factory Direct Marine & RV we take security and... 

U.S. Army

Patient Administration Specialist (68G) Job at U.S. Army

 ...Administration Specialist (68G), you'll be entrusted with serving the military community and their Families and providing essential support...  ...records and keep track of patient health. This is not a civilian contractor position. No experience necessary. Position is entry level... 

Domino's Franchise

Delivery Driver - 1088 Boston post rd Job at Domino's Franchise

Job Description Delivery Driver. Deliver pizza with your own car

Stanford University

Research Assistant for the Hoover Technology Policy Accelerator Job at Stanford University

Research Assistant for the Hoover Technology Policy Accelerator**Hoover Institution, Stanford, California, United States**ResearchPost Date Apr 10, 2025Requisition # 105761The Hoover Institution at Stanford University is seeking qualified candidates for the full-time...