Job Description

Job Description

Job Description

Key Responsibilities
Platform Engineering & Architecture
" Design, implement, and maintain Databricks platform architecture to support enterprise-scale data workloads
" Establish best practices for workspace setup, cluster management, and job orchestration
" Optimize platform performance, scalability, and cost efficiency
" Define and enforce standards for data engineering and analytics workloads within Databricks
Developer Enablement
" Support development teams by providing guidance, documentation, and reusable frameworks
" Enable developers with CI/CD pipelines, automation, and deployment standards
" Troubleshoot and resolve platform-related issues impacting development teams
" Provide technical mentorship and onboarding support to engineers using Databricks
Governance, Security & Compliance
" Implement and manage data governance, access controls, and security models (Unity Catalog, RBAC, etc.)
" Ensure compliance with enterprise data policies and regulatory requirements
" Monitor and audit platform usage, access patterns, and data lineage
Integration & Ecosystem Management
" Integrate Databricks with cloud platforms (AWS/Azure/GCP), data lakes, and enterprise systems
" Manage connectivity with tools such as Power BI, APIs, data ingestion frameworks, and orchestration tools
" Support ingestion and processing frameworks (e.g., batch, streaming, Delta Lake)
Operational Excellence
" Monitor platform health, performance, and usage metrics
" Implement automation for provisioning, monitoring, and scaling
" Drive continuous improvements in platform reliability and developer experience

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)
• 5 years of experience in data engineering or platform engineering roles
• 3 years of hands-on experience with Databricks (architecture, administration, or platform engineering)
• Strong understanding of distributed data processing (Spark)
• Experience with cloud platforms (Azure preferred, or AWS/GCP)
• Knowledge of data lake architecture, Delta Lake, and modern data platforms
• Experience implementing security, governance, and access control frameworks
• Familiarity with CI/CD pipelines, DevOps practices, and infrastructure as code (Terraform, etc.)

Job Tags

Similar Jobs