# Migrating from dex ## Migration Alternatives Guide As dex transitions, we're committed to helping you continue your data engineering journey with minimal disruption. This guide outlines two recommended migration paths, each designed to match different needs and preferences. It's important to note that both alternatives will require adaptation and are not exactly the same as dex. In this guide you will find the most similar feature which replaces dex features in each tool, but remember they are not the same. {% hint style="info" %} **No data or infrastructure migration needed.** Your data stays in your cloud storage, and your dbt code stays in your Git repository. Simply point your new tools (dbt Cloud, Fivetran, or Databricks) to these existing resources. Minor configuration adjustments may be required, but your core infrastructure remains unchanged. {% endhint %} *** ### Alternative 1: dbt Cloud + Fivetran **Best for:** Teams that prefer best-of-breed tools and want flexibility in their stack composition. This alternative is also less complex in terms of infrastructure. **Your existing infrastructure can be reused.** Since dex already uses dbt under the hood, your models, tests, and Git repository can be connected directly to dbt Cloud with minor configuration adjustments. Your data remains in your cloud storage—no migration required. #### What This Alternative Offers This combination replaces dex with two specialized platforms working together: * **Fivetran** handles data ingestion (replacing dex Connectors) * **dbt Cloud** manages transformation, orchestration, testing, and documentation (replacing dex Models, Flows, Tests, and Documentation) #### Comparison to dex | dex Capability | Equivalent Solution | | ------------------------------------------- | ------------------------------------------ | | **Connectors** (one-click data ingestion) | Fivetran connectors (500+ sources) | | **Models** (SQL/Python transformations) | dbt models with SQL | | **Flows** (orchestration and scheduling) | dbt Cloud Job Scheduler | | **Tests** (data quality checks) | dbt tests (schema, freshness, custom) | | **Documentation** (lineage and column docs) | dbt Cloud Catalog with auto-generated docs | | **Version Control** (Git integration) | dbt Cloud native Git integration | **Key Difference:** Unlike dex's unified platform, this requires managing two separate tools. However, you gain access to the largest ecosystem in modern data engineering, with extensive community support and third-party integrations. #### Pricing Model and Recommendations **dbt Cloud Pricing** dbt Cloud uses a **hybrid pricing model** based on developer seats and successful model builds. **Developer Plan** (Free) * 1 developer seat * Up to 3,000 successful models built per month * Best for: Individual users or small trial projects **Team Plan** ($100/developer/month) * Up to 8 developer seats + 5 read-only seats * 15,000 successful models built per month included * $0.01 per additional successful model built * Best for: Small to medium teams {% hint style="info" %} **What Counts as a "Successful Model Built"?** Every time a model successfully builds in production (via scheduler, CI/CD, or API) counts toward your quota. Development runs in the IDE don't count. {% endhint %} **Fivetran Pricing** Fivetran uses **Monthly Active Rows (MAR)** pricing calculated **per connection**: * **MAR** = rows inserted, updated, or deleted each month * Pricing is tiered: the more data you move, the lower your per-row cost **Pricing Plans:** * **Free Plan:** Trial period with limited usage * **Standard Plan:** Pay-as-you-go based on MAR per connection For most teams migrating from dex, the **Team and Standard Plans** will be the most suitable option, offering the right balance of features, capacity, and cost. Start with the free plans to test the platform, then upgrade to Team/Standard when you're ready to onboard your full team. #### Getting Started **Resources:** * [Fivetran Quickstart Guide](https://fivetran.com/docs/getting-started/quickstart) * [Fivetran Connector Directory](https://fivetran.com/docs/connectors) * [dbt Cloud Setup Documentation](https://docs.getdbt.com/docs/cloud/about-cloud-setup) * [dbt Fundamentals Course](https://courses.getdbt.com/courses/fundamentals) (free) You can also use the [Fivetran + dbt Cloud integration](https://fivetran.com/docs/transformations/dbt-cloud) to automatically trigger dbt runs after connector syncs. *** ### Alternative 2: Databricks **Best for:** Teams that want an all-in-one platform with native support for data science, machine learning, and advanced analytics. **Your existing infrastructure can be reused.** Your data remains in your cloud storage—no migration required. While dex uses dbt for transformations, Databricks offers multiple approaches including SQL queries, notebooks, and Delta Live Tables. You will need to adapt your existing dbt models to work with Databricks' tooling, which may require more extensive refactoring compared to migrating to dbt Cloud. #### What This Alternative Offers Databricks provides a unified lakehouse platform that combines all capabilities: * Data ingestion and ETL * SQL and Python transformations * Workflow orchestration * Notebooks for collaborative development * Built-in governance and security * Native support for ML/AI workloads #### Comparison to dex | dex Capability | Databricks Equivalent | | ------------------- | ----------------------------------------------------------------------- | | **Connectors** | Databricks connectors + Partner Connect (Fivetran, Airbyte integration) | | **Models** | SQL queries, notebooks, Delta Live Tables | | **Flows** | Databricks Workflows (orchestration) | | **Tests** | Expectations in Delta Live Tables + custom validation | | **Documentation** | Unity Catalog with automated lineage | | **Version Control** | Native Git integration (Repos) | | **Python Scripts** | Databricks notebooks with full Python support | **Key Difference:** Databricks is a more comprehensive platform that includes dex's capabilities plus advanced features for data science, machine learning, and real-time streaming. It requires more infrastructure knowledge but offers greater power and flexibility. #### Pricing Model and Recommendations Databricks uses a **consumption-based pricing model** with two cost components: **1. Databricks Platform Cost (DBUs)** **Databricks Units (DBUs)** measure processing power. Different workload types consume DBUs at different rates: * **Jobs Compute:** $0.07 - $0.15 per DBU (automated/scheduled workloads) * **All-Purpose Compute:** $0.40 - $0.65 per DBU (interactive development) * **SQL Warehouses:** $0.22 - $0.55 per DBU (SQL analytics) * **Delta Live Tables:** $0.20 - $0.36 per DBU (declarative ETL) **2. Cloud Infrastructure Cost** You also pay your cloud provider (AWS, Azure, or GCP) for: * Virtual machine compute time * Storage (S3, ADLS, GCS) * Data transfer **Important:** Infrastructure costs are often 50-200% of DBU costs, so budget accordingly. For most teams migrating from dex, the **Standard Tier** will be the most suitable option. #### Getting Started **Resources:** * [Databricks Getting Started Guide](https://docs.databricks.com/getting-started/index.html) * [Repository connection](https://docs.databricks.com/aws/pt/repos) * [Partner Connect](https://docs.databricks.com/integrations/partner-connect/index.html) *** ### Choosing the Right Alternative #### Choose dbt Cloud + Fivetran if you: * Prefer specialized, best-of-breed tools * Have a SQL-focused data team * Want the largest community and ecosystem * Need predictable, seat-based pricing for development * Primarily focus on analytics and BI use cases #### Choose Databricks if you: * Want an all-in-one platform * Need advanced data science and ML capabilities * Have Python-proficient team members * Require real-time streaming and complex transformations * Want unified governance across all data workflows *** ### Important Note: dex Connectors **dex's connectors are proprietary technology and will not be provided as part of the migration.** These connectors were developed internally and remain the intellectual property of dex. Some connectors may not be available in other tools. If you have unique data sources or prefer to maintain control, you can build custom extraction scripts. This approach works well when combined with a simple orchestration tool. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.dexlabs.io/migration/migrating-from-dex.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.