Integration Guides
End-to-end patterns and step-by-step guides for real-world data integrations.
Mainframe COBOL → Azure Databricks (PySpark)
End-to-end migration pattern: ingestion from mainframe flat files, COBOL-to-PySpark conversion approach, metadata-driven framework, and CI/CD deployment.
On-Premises Hadoop → Azure ADLS & Synapse
Framework for data movement from on-premises Hadoop/Hive to ADLS Gen2 and Synapse Analytics using ADF and Databricks.
Metadata-Driven Ingestion Pipeline
ForEach + Lookup pattern for config-driven ingestion: one pipeline handles multiple sources via a control table.
Databricks to Snowflake — Spark Connector Setup
Configuring the Snowflake Spark connector in Databricks: authentication, staging, and write modes.
Semantic Layer Sync — Executor Agent Output to Snowflake
How the Executor Agent deploys metric view YAML to both Databricks and Snowflake targets in the same pipeline run.
VS Code + GitHub Copilot — Agent Development Environment
Setting up VS Code with Copilot, LangGraph, and Databricks SDK for multi-agent development.
Agent-to-GitHub PR Integration — Deployment Agent
How the Deployment Agent authenticates to GitHub and raises pull requests programmatically.
Informatica Mapping Conversion Pattern
Approach and patterns for converting Informatica PowerCenter mappings and workflows to PySpark using LLM-assisted transformation.