When a dataset is first added to Luzmo, Luzmo will save the metadata of that dataset, i.e. what are the column names and data type of these columns. Depending on the data type of column, they can be used differently within Luzmo (e.g. you can sum a numeric column but not a hierarchy).
Luzmo will determine the metadata based on information provided by the datasource. Most data sources provide an information schema or similar which Luzmo can query to retrieve the information. For other data sources that don't provide this information (e.g. MongoDB), Luzmo will infer the columns and data types of each column based on a sample of rows of the data. Be mindful that in some cases, the sample may contain data that leads to the misidentification of the column's type (e.g. mostly null values).
Since schema metadata doesn’t change often, it isn’t fetched with every query. Instead, Luzmo stores it and updates it based on the "Schema updates" setting for the dataset. By default, a dataset inherits this setting from its data connection, which has a default update frequency of once per hour. If needed, you can manually trigger a schema update by clicking “Update schema now” in the dataset settings or using the the API.
The available settings for schema updates on a dataset look like this:
Important:
Here are some expected behaviors to be aware of when schema changes happen:
- Deleted or renamed tables/views: If a table or view is deleted or renamed in the datasource, Luzmo won’t be able to query it. Charts using those columns - either directly or through formulas or derived columns - will display a "Query Failed" error indicating a connection issue.
- Recreated tables/views with the same name: If a table or view is deleted and then recreated with the same name in the datasource, queries will continue to work as long as the columns used retained the same names and data types.
- Deleted or renamed columns: If a column is removed or renamed, queries will fail. Once the schema is updated in Luzmo, the old column is removed and, in case of a rename, a new one is added with the new name.
- Recreated columns with the same name:
- If a column is deleted and then recreated with the same name in the data source, queries will continue to work as long as the column retains the same data type. If the data type changed, see bullet point Changed column types.
- If a column is deleted and then recreated with the same name in a SQL dataset, Luzmo will consider the column as a new column with it's own column id. As such, charts, formulas and derived columns referincing the old column will display query failed.
- Changed column types: If a column’s type is changed in the data source or SQL dataset, it will cause errors when querying until the schema is updated. After updating, the column will reflect the new data type. The updated column can be used to build new charts, derived columns, formulas, ... However, previously build charts, derived columns and formulas may continue to fail if the new data type is incompatible with how the column is used in Luzmo.