Role Snapshot
Data Engineer, AI role responsible for building and optimizing data pipelines and transformation processes in Databricks Lakehouse to deliver analytics-ready datasets for Kaplan Australia. Primary contributor driving data engineering across ingestion, transformation, orchestration, and AI/ML enablement.
Job Description
Job Title Data Engineer, AI (Hybrid) Job Description The Data Engineer (India Offshore) is responsible for building, maintaining, and optimising data pipelines and transformation processes that deliver trusted, analytics-ready datasets for Kaplan Australia. This role will be the primary contributor driving data transformation into the Lakehouse (Databricks), ensuring data is modelled, curated, and made available for reporting, analytics, and AI/ML use cases. Reporting to the Senior Solutions Architect, the role contributes hands-on engineering expertise across ingestion, transformation, orchestration, and monitoring, and supports the enablement of AI/ML by providing well-governed features and training datasets. Key Responsibilities Data Engineering Delivery (Pipelines, Transformations, and Data Modelling) • Design, build, and operate reliable ELT/ETL pipelines to ingest, transform, and curate data into analytics-ready datasets. Implement and maintain data models (dimensional and/or lakehouse curated layers) with clear semantic definitions to support reporting, analytics, and AI use cases. Build data quality controls (validation, reconciliation, and automated checks) and implement observability (monitoring, alerting, and SLAs) for pipelines and jobs. Optimise performance and cost through efficient compute usage, incremental processing, partitioning, and tuning of Spark/SQL workloads. Lakehouse / Databricks Transformation Ownership Be the primary engineer responsible for implementing and evolving the Lakehouse transformation layer in Databricks. Build and maintain Databricks workflows, notebooks, and jobs using Spark/SQL/Python, applying reusable patterns and standards. Contribute to Lakehouse design decisions (data layout, medallion architecture, Delta patterns, and access strategies) in collaboration with the Head of AI and platform stakeholders. 3. AI / ML Engineering Enablement Work within the AI team to support delivery of AI/ML initiatives through hands-on engineering (data preparation, pipeline build-out, and integration into production patterns). • Collaborate on GenAI projects, including prompt engineering iteration support (test cases, evaluation datasets, and telemetry/metrics to assess prompt performance). Ability to translate AI, analytics, and business requirements into scalable data engineering solutions in collaboration with AI team, product owners, and domain stakeholders. Help curate and maintain governed feature/training datasets and ensure reproducibility through versioning, documentation, and agreed engineering standards. Information Security, Privacy, and Governance Apply security-by-design principles across ingestion and transformation (least privilege, secure secrets handling, environment separation). Implement governance requirements including data classification, retention, lineage, and access controls in line with enterprise standards. Support audits and assurance activities by providing evidence of controls, data handling practices, and operational procedures. Minimum Qualifications Demonstrated 5+ years’ experience in data engineering, building production-grade data pipelines and transformation layers. Strong hands-on experience with Databricks and Lakehouse patterns (e.g., medallion architecture) including Spark and SQL. Proficiency in Python and SQL for data engineering (transformations, testing, and automation). Experience working with cloud‑based data platforms (AWS) and integrated data storage, orchestration, and analytics services. Strong understanding of data modelling (dimensional modelling, curated layers, semantic definitions) and data quality management. Working knowledge of data governance, privacy, and information security practices (access controls, PII handling, lineage, retention). Strong problem-solving skills, attention to detail, and ability to work autonomously with distributed stakeholders. Experience with Jira, Confluence, and Microsoft 365 tools; comfortable working across Agile delivery practices. Highly Desirable Experience enabling AI/ML use cases (feature engineering datasets, training/serving data considerations, reproducibility). Experience supporting GenAI / LLM solutions, including RAG-style data preparation (document ingestion, chunking, metadata enrichment) and evaluation dataset creation. Experience collaborating on prompt engineering and prompt evaluation cycles, including collecting ground truth, defining test cases, and analysing prompt performance telemetry. • Understanding of vector search / embeddings concepts and data requirements for semantic retrieval. Experience delivering data solutions within higher education or regulated environments. Compliance & Governance Obligations Demonstrate company values and contribute to a kind, safe, supportive and collaborative workplace. Adherence to all Kaplan policies and procedures. Compliance with Workplace Health & Safety legislation and requirements. Completion of all mandatory training as required by Kaplan. Compliance with information security, privacy, data governance and risk management frameworks. Location Bangalore, KA, India Additional Locations Employee Type Employee Job Functional Area IT Development Business Unit 00092 Kaplan
Health Diversity & Inclusion Statement: Kaplan is committed to cultivating an inclusive workplace that values diversity, promotes equity, and integrates inclusivity into all aspects of our operations. We are an equal opportunity employer and all qualified applicants will receive consideration for employment regardless of age, race, creed, color, national origin, ancestry, marital status, sexual orientation, gender identity or expression, disability, veteran status, nationality, or sex. We believe that diversity strengthens our organization, fuels innovation, and improves our ability to serve our students, customers, and communities. Learn more about our culture here. Kaplan considers qualified applicants for employment even if applicants have an arrest or conviction in their background check records. Kaplan complies with related background check regulations, including but not limited to, the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. There are various positions where certain convictions may disqualify applicants, such as those positions requiring interaction with minors, financial records, or other sensitive and/or confidential information. Kaplan is a drug-free workplace and complies with applicable laws. Kaplan, Inc. is a global educational services company, with about 10,000 employees and operations in 26 countries. We serve 1.2 million students each year and partner with about 4,000 educational institutions and 13,000 corporations across the world. Kaplan is the largest subsidiary of Graham Holdings Company (NYSE: GHC). Across its 85-year history, first as a pioneer of the test prep industry, then as an early online education leader, and now as a global education provider, Kaplan has been recognized for expanding educational access and improving student outcomes through innovative uses of technology, instructional design, and learning science. We’ve been recognized as a Top Workplace for Innovators and Brands That Matter (Fast Company); and as one of America’s Top 100 employers (Forbes). Kaplan’s US-based businesses provide individuals, educational institutions, businesses and governments a broad array of services, supporting our students and partners to meet their diverse and evolving needs throughout their educational and professional journeys. Among the services provided are test preparation, coaching and advising, performance training, industry credentialing, and university support services, online enablement, analytics, and marketing. The test preparation and professional products are offered in the market under our brand names such as Kaplan, Kaplan Financial Education, Kaplan Schweser, PPI, College for Financial Planning, and Manhattan Prep. Headquartered in Fort Lauderdale, FL, employees who support Kaplan’s North America division work primarily remotely across the US and in our corporate campus in Bengaluru, India. California Worker Privacy Statement Questions? Please contact us at knarecruiting@kaplan.com.
More Jobs at Kaplan



