[
https://issues.apache.org/jira/browse/IGNITE-28067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-28067:
--------------------------------------
Description:
Currently, schema compatibility checking (see SchemaCompatibilityValidator)
relies on the fact that the table version has changed (see
CatalogTableSchemaVersions).
This works fine until we need to check for catalog changes that don't increment
the table's schema version.
For example, indexes should not increment the table version, but some changes
involving indexes may need to be validated.
Thus, it is proposed to add an additional counter to the catalog table
descriptor, which will be incremented when table schema or any dependency
changes and rework validator to use this counter instead of table version.
In addition, it is necessary to think about more optimal caching of
intermediate table schemas.
Currently catalogVersionSpansCache has the following key:
{noformat}
int tableId
int fromCatalogVersion
int toCatalogVersion{noformat}
Thus we may have overlapping catalog version ranges in cache keys for the same
table.
For example: \{1 -> 3}, \{1 -> 2}, \{2 -> 3}
We should think about how to de-duplicate cache keys so that cache doesn't
grow too much.
Another cache is a diffCache currently has the following key
{noformat}
int tableId
int fromSchemaVersion
int toSchemaVersion
{noformat}
But toSchemaVersion = fromSchemaVersion + 1 always, so one of these fields can
be safely removed,
was:
Currently, schema compatibility checking (see SchemaCompatibilityValidator)
relies on the fact that the table version has changed (see
CatalogTableSchemaVersions).
This works fine until we need to check for catalog changes that don't increment
the table's schema version.
For example, indexes should not increment the table version, but some changes
involving indexes may need to be validated.
Thus, it is proposed to add an additional counter to the catalog table
descriptor, which will be incremented when table schema or any dependency
changes and rework validator to use this counter instead of table version.
In addition, it is necessary to think about more optimal caching of
intermediate table schemas.
Currently catalogVersionSpansCache has the following key:
{noformat}
tableId
fromCatalogVersion
toCatalogVersion.
{noformat}
Thus we may have overlapping catalog version ranges in cache keys for the same
table.
For example: {1 -> 3}, {1 -> 2}, {2 -> 3}
We should probably think about how to de-duplicate cache keys so that cache
doesn't grow too much.
Another cache is a diffCache currently has the following key
{noformat}
tableId
fromSchemaVersion
toSchemaVersion
{noformat}
But toSchemaVersion = fromSchemaVersion + 1 always, so one of these fields can
be safely removed,
> Table schema compatibility validation extension
> -----------------------------------------------
>
> Key: IGNITE-28067
> URL: https://issues.apache.org/jira/browse/IGNITE-28067
> Project: Ignite
> Issue Type: Improvement
> Components: sql ai3, tables ai3
> Reporter: Pavel Pereslegin
> Priority: Major
>
> Currently, schema compatibility checking (see SchemaCompatibilityValidator)
> relies on the fact that the table version has changed (see
> CatalogTableSchemaVersions).
> This works fine until we need to check for catalog changes that don't
> increment the table's schema version.
> For example, indexes should not increment the table version, but some changes
> involving indexes may need to be validated.
> Thus, it is proposed to add an additional counter to the catalog table
> descriptor, which will be incremented when table schema or any dependency
> changes and rework validator to use this counter instead of table version.
> In addition, it is necessary to think about more optimal caching of
> intermediate table schemas.
> Currently catalogVersionSpansCache has the following key:
> {noformat}
> int tableId
> int fromCatalogVersion
> int toCatalogVersion{noformat}
> Thus we may have overlapping catalog version ranges in cache keys for the
> same table.
> For example: \{1 -> 3}, \{1 -> 2}, \{2 -> 3}
> We should think about how to de-duplicate cache keys so that cache doesn't
> grow too much.
> Another cache is a diffCache currently has the following key
> {noformat}
> int tableId
> int fromSchemaVersion
> int toSchemaVersion
> {noformat}
> But toSchemaVersion = fromSchemaVersion + 1 always, so one of these fields
> can be safely removed,
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
