Skip to content

VDE_MODE only derived from project config, not from state #5887

Description

@davem-bis

Steps to Reproduce

  1. The prod environment is initially planned with vde_mode = All
  2. vde_mode is set to dev_only
  3. A plan of a dev environment will assume that prod is set as physical models, and perform table copy operations as if this is the case, despite prod still being in vde_mode = All.

For Databricks, this can cause errors as SQLMesh attempts to clone the prod environment assuming it is a table, and fails with [DELTA_CLONE_UNSUPPORTED_SOURCE] as the prod model is still a view at this point.

Location in Code

The VDE mode is set in the self.config object of the loader, which is set as self.context.configs[self.config_path] in sqlmesh/core/loader.py::Loader::__init__.

Proposed Solution

When in a plan or run, VDE mode should be derived from both the config, and the current state of the prod environment in the target gateway. Potentially a warning should be raised when they do not agree.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions