Job Description
Job Title: Azure Data engineer
Location: Dallas, TX
Duration : Long term
Roles and Responsibilities.
- Develop and manage Snowflake stored procedures, UDFs, and performance tuning techniques.
- Implement access controls, row-level security, and data masking policies for secure data handling.
- Integrate Snowflake with Azure Data Factory, external APIs, and data lakes for hybrid cloud solutions.
- Good understanding on Snowflake RBAC and Roles and User Management.
- Design and orchestrate scalable data pipelines using ADF for batch and near real-time data ingestion.
- Develop ADF pipelines for data integration, transformation, and movement across multiple sources.
- Implement parameterized pipelines, Linked Services, Datasets, and Triggers to support modular data workflows.
- Monitor, troubleshoot, and optimize ADF pipeline performance and cost-efficiency.
- Integrate ADF with Synapse, Snowflake, Azure Key Vault, and Logic Apps as part of end-to-end solutions.
- Develop and optimize Synapse SQL scripts, stored procedures, and data flows for analytics workloads.
- Implement Synapse Pipelines and Spark Notebooks for advanced transformation and machine learning use cases.
- Perform data warehousing activities including partitioning, indexing, and materialized views.
- Integrate Synapse with Power BI, Azure ML full-stack data insights.
- Ensure efficient utilization of Dedicated SQL Pools and Serverless resources.
- Synapse Administration ,RBAC understanding and User management from Azure Entra.
- Set up and maintain CI/CD pipelines for ADF.
- Implement version control, code branching strategies, and peer code reviews for collaborative development.
- Configure release pipelines to handle environment-specific configurations using variable groups and templates.
- Monitor pipeline runs, manage build/release failures, and enforce security and compliance during deployments.
Job Tags