Schema Projects

Audience: Data Owners, Data Users, and Data Governors

Content Summary: This overview describes concepts related to schema projects, schema monitoring, and column detection.

Introduction

Schema projects are different from user created projects in several ways, but mainly in that they are automatically created and managed by Immuta. They group all the data sources of the schema, and when new data sources are created, manually or with schema detection, they are automatically added to the schema project.

Schema projects are created with any table-backed data source; when you create the data source, you choose the project name at that time. The user creating the data source does not need the CREATE_PROJECT permission to have the project auto-create because no data sources can be added by the owner. Instead, new data sources are managed by Immuta. The user can manage Subscription policies for schema projects, but they cannot apply Data policies or purposes to them.

The schema settings, such as schema evolution and connection information, can be edited from the project overview tab. Note: Deleting the project will delete all of the data sources within it as well.

Schema Monitoring

Schema Monitoring is enabled while creating a data source or when editing a data source. It monitors servers for schema and table changes, including when schemas and tables are added or removed, and notifies Data Owners when any changes are made.

When this feature is enabled by a Data Owner, Immuta detects when a new table has been added and automatically creates a new data source. Correspondingly, if a remote table is removed, that data source will be disabled in the console. Data Owners or Governors can edit which user will monitor schema changes from the schema project overview tab.

See Create Query-backed Data Source for instructions on enabling it or Manage Schema Projects for instructions on editing the schema detection owner.

Column Detection

Data Owners can also enable Column Detection, which monitors when columns are added or removed and when column types are changed.

When new columns are added to the remote table, Immuta automatically applies the New tag to these columns in the data source, and, since these new columns could contain sensitive data, a seeded New Column Added Global Policy masks them. The New Column Added Global Policy is active by default.

See Clone, Activate, or Stage a Global Policy to stage this seeded Global Policy if you do not want new columns automatically masked.

Data Owners can then review and approve these changes from the Requests tab of their profile page. Approving column changes removes the New tags from the data source.

See Create a Query-backed Data Source for instructions on enabling column detection.