Azure data engineer Job at Cognizant, Pittsburgh, PA

NlQ2Tk5HZmE2bE54SzU3SmtDUjA5TXJTd0E9PQ==
  • Cognizant
  • Pittsburgh, PA

Job Description

Job Title - DataBricks Data Engineer

Location - Pittsburg, PA

**Must have 8+ years of real hands on experience**

We are specifically seeking a Data Engineer-Lead with strong expertise in Databricks development.

The role involves:

  • Building and testing data pipelines using Python/Scala on Databricks
  • Hands on experience to develop and lead the offshore team to perform development/testing work in Azure data bricks
  • Architect data platforms using Azure services such as Azure Data Factory (ADF), Azure Databricks (ADB), Azure SQL Database, and PySpark.
  • Collaborate with stakeholders to understand business needs and translate them into technical solutions.
  • Provide technical leadership and guidance to the data engineering team and need to perform development.
  • Familiar with Safe Agile concepts and good to have working experience in agile model.
  • Develop and maintain data pipelines for efficient data movement and transformation.
  • Onsite and offshore team communication and co-ordination.
  • Create and update the documentation to facilitate cross-training and troubleshooting
  • Hands on experience in scheduling tools like BMC control-M and setup jobs and test the schedules.
  • Understand the data models and schemas to support the development work and help in creation of tables in databricks
  • Proficiency in Azure Data Factory (ADF), Azure Databricks (ADB), SQL, NoSQL, PySpark, Power BI and other Azure data tools.
  • Implementing automated data validation frameworks such as Great Expectations or Deequ
  • Reconciling large-scale datasets
  • Ensuring data reliability across both batch and streaming processes

The ideal candidate will have hands-on experience with:

  • PySpark, Scala, Delta Lake, and Unity Catalog
  • Devops CI/CD automation
  • Cloud-native data services
  • Azure databricks/Oracle
  • BMC Control-M

Location: Pittsburgh, PA

Job Tags

Work experience placement,

Similar Jobs

Delta System & Software, Inc.

Estate and Wealth Planning Paralegal Job at Delta System & Software, Inc.

Essential Duties & Responsibilities: Abides by the ethical requirements as set forth by the Rules of Professional Conduct. Drafts and/or prepares documents (e.g., federal estate and gift tax returns, probate documents, estate accountings, estate planning documents...

Haugland Group LLC

Bid Coordinator - Florida Job at Haugland Group LLC

Haugland Group, a privately-owned civil infrastructure and energy construction company, has an immediate opportunity for an experienced Bid Coordinator to join our team. This position is based out of our Central Florida location. At Haugland, we understand that pushing...

Neptune Construction Group LLC

Social Media Content Creator Job at Neptune Construction Group LLC

 ...Content Creator to help build our online presence and support basic marketing tasks. This role focuses on creating simple content, posting...  ...consistency. Preferred Qualifications Previous experience in social media, marketing, or content creation. Basic skills... 

ChaseSource

Data Analyst Intern Job at ChaseSource

 ...is a Texas government state agency. Our internship program aims to provide a rewarding learning...  ...and retirees. The team focuses on data accuracy, reporting, and analysis to ensure...  ...responsibly. ERS is seeking a Data Science and Analytics Intern to work closely with... 

Farm Job Search

Spray Rig Operator Job at Farm Job Search

 ...is a diversified irrigated grain operation that also offers custom farming services such as strip till, planting, spraying, and harvesting. Operator must be able to follow mixing instructions and operate new John Deere equipment. Will be kept busy year around when not...