Description
Lakeflow takes you from raw source data to a production-ready, orchestrated pipeline on one platform. Watch Tomasz Bacewicz build a Salesforce pipeline end to end: incremental ingestion with Lakeflow Connect, transformation and data quality checks with Spark Declarative Pipelines generated by Genie Code, no-code data prep in Lakeflow Designer, and orchestration with Lakeflow Jobs.
🎯 What You'll Learn:
• How Lakeflow Connect ingests Salesforce data incrementally in a few clicks
• How Genie Code generates Spark Declarative Pipelines with built-in data quality checks
• How Lakeflow Designer handles last-mile data prep in a no-code interface
• How Lakeflow Jobs orchestrates ingestion, transformation, and prep in sequence
⏱️ CHAPTERS:
00:00 - Introduction and platform overview
02:10 - Ingesting Salesforce data with Lakeflow Connect
05:46 - Building pipelines with Genie Code and Spark Declarative Pipelines
11:29 - No-code data prep with Lakeflow Designer
15:03 - Orchestrating everything with Lakeflow Jobs
Resources:
• Get Started with Data Engineering training (free): https://www.databricks.com/training/catalog/get-started-with-databricks-for-data-engineering-1511
• Lakeflow: https://www.databricks.com/product/data-engineering
🤝 Stay connected:
Website: https://databricks.com
Twitter: https://x.com/databricks/
LinkedIn: https://www.linkedin.com/company/databricks/posts/
Instagram: https://www.instagram.com/databricksinc/?hl=en
Facebook: https://www.facebook.com/databricksinc/
Unify your pipelines and build an agentic data foundation for AI with the Big Book of Data Engineering: https://www.databricks.com/resources/ebook/big-book-of-data-engineering