[
https://issues.apache.org/jira/browse/IGNITE-15404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-15404:
-------------------------------------
Description:
Based on prototyping carried out within IGNITE-15374 it's requited to rework
distributed configuration flow processing:
# Use configuration registry instead of meta storage manager.
## Use configuration manager for internal configuration processing. Blocked by
IGNITE-15047
## Use configuration manager in order to retrieve distributed up-to-date
values.
## Use configuration listeners instead of meta storage watches.
# Favor one-step table creation (calculate assignments and inner schema
together with initial table configuration update).
Please pay attention that during prototyping plenty of bugs were found that
should be fixed during refactoring. To name some:
# Columns validation is skipped if column was added or removed during
alterTable() See onConfigurationChanged() for more details.
{code:java}
if (!newTbl.columns().namedListKeys().equals(oldTbl.columns().namedListKeys()))
return true;
{code}
# table identified ignores update counter
{code:java}
UUID tblId = new UUID(ctx.storageRevision(), 0L);
{code}
Of course in addition to fixing known issues we will introduce new bugs during
refactoring.
*Following items were implemented:*
1. Unnecessary phases of assignment calculation and schema preparation as a
reaction on distributed event were simplified to in-place calculation within
table creation:
{code:java}
clusterCfgMgr
.configurationRegistry()
.getConfiguration(TablesConfiguration.KEY)
.tables()
.change(
change -> change.create(
name,
(ch) -> {
tableInitChange.accept(ch);
((ExtendedTableChange)ch).
// Table id specification.
changeId(tblId.toString()).
// Affinity assignments calculation.
changeAssignments(
ByteUtils.toBytes(
AffinityService.calculateAssignments(
baselineMgr.nodes(),
ch.partitions(),
ch.replicas()
)
)
).
// Table schema preparation.
changeSchemas(
schemasCh -> schemasCh.create(
String.valueOf(INITIAL_SCHEMA_VERSION),
schemaCh -> schemaCh.changeSchema(
ByteUtils.toBytes(
SchemaService.prepareSchemaDescriptor(
((ExtendedTableView)ch).schemas().size(),
ch
)
)
)
)
);
}
)
)
{code}
Besides obvious that allowed to simplify AffinityManager and SchemaManager that
were converted to simple stateless services.
2. Same is about table id: instead of using meta storage revision and update
counter (this wasn't implemented though) explicit table id generation was used.
This is the main reason for the large number of changed files.
3. Inner local events were substituted by common Futures within
create/alter/dropTable.
Few more tickets were created for further improvements:
[IGNITE-15414 Schema validation refactoring with configuration
validators.|https://issues.apache.org/jira/browse/IGNITE-15414]
[IGNITE-15480 Refactor index column mapping based on column id automatically
generated on column creation with the help of internal configuration
extensions.|https://issues.apache.org/jira/browse/IGNITE-15480]
[IGNITE-15485 Support table rename
operation.|https://issues.apache.org/jira/browse/IGNITE-15485]
Some tickets for further refactoring are still awaited:
[IGNITE-15409 Listener with any
placeholders|https://issues.apache.org/jira/browse/IGNITE-15409]
[IGNITE-15412 Configuration registry: add ability to retrieve distributed
value|https://issues.apache.org/jira/browse/IGNITE-15412] That will allow to
remove metastorage from table manager completely and get rid of current verison
of TableManager#tableNamesConfigured
was:
Based on prototyping carried out within IGNITE-15374 it's requited to rework
distributed configuration flow processing:
# Use configuration registry instead of meta storage manager.
## Use configuration manager for internal configuration processing. Blocked by
IGNITE-15047
## Use configuration manager in order to retrieve distributed up-to-date
values.
## Use configuration listeners instead of meta storage watches.
# Favor one-step table creation (calculate assignments and inner schema
together with initial table configuration update).
Please pay attention that during prototyping plenty of bugs were found that
should be fixed during refactoring. To name some:
# Columns validation is skipped if column was added or removed during
alterTable() See onConfigurationChanged() for more details.
{code:java}
if (!newTbl.columns().namedListKeys().equals(oldTbl.columns().namedListKeys()))
return true;
{code}
# table identified ignores update counter
{code:java}
UUID tblId = new UUID(ctx.storageRevision(), 0L);
{code}
Of course in addition to fixing known issues we will introduce new bugs during
refactoring.
> Rework disctributed configuration flow
> --------------------------------------
>
> Key: IGNITE-15404
> URL: https://issues.apache.org/jira/browse/IGNITE-15404
> Project: Ignite
> Issue Type: Improvement
> Reporter: Alexander Lapin
> Assignee: Alexander Lapin
> Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Based on prototyping carried out within IGNITE-15374 it's requited to rework
> distributed configuration flow processing:
> # Use configuration registry instead of meta storage manager.
> ## Use configuration manager for internal configuration processing. Blocked
> by IGNITE-15047
> ## Use configuration manager in order to retrieve distributed up-to-date
> values.
> ## Use configuration listeners instead of meta storage watches.
> # Favor one-step table creation (calculate assignments and inner schema
> together with initial table configuration update).
> Please pay attention that during prototyping plenty of bugs were found that
> should be fixed during refactoring. To name some:
> # Columns validation is skipped if column was added or removed during
> alterTable() See onConfigurationChanged() for more details.
> {code:java}
> if
> (!newTbl.columns().namedListKeys().equals(oldTbl.columns().namedListKeys()))
> return true;
> {code}
> # table identified ignores update counter
> {code:java}
> UUID tblId = new UUID(ctx.storageRevision(), 0L);
> {code}
> Of course in addition to fixing known issues we will introduce new bugs
> during refactoring.
> *Following items were implemented:*
> 1. Unnecessary phases of assignment calculation and schema preparation as a
> reaction on distributed event were simplified to in-place calculation within
> table creation:
>
> {code:java}
> clusterCfgMgr
> .configurationRegistry()
> .getConfiguration(TablesConfiguration.KEY)
> .tables()
> .change(
> change -> change.create(
> name,
> (ch) -> {
> tableInitChange.accept(ch);
> ((ExtendedTableChange)ch).
> // Table id specification.
> changeId(tblId.toString()).
> // Affinity assignments calculation.
> changeAssignments(
> ByteUtils.toBytes(
> AffinityService.calculateAssignments(
> baselineMgr.nodes(),
> ch.partitions(),
> ch.replicas()
> )
> )
> ).
> // Table schema preparation.
> changeSchemas(
> schemasCh -> schemasCh.create(
> String.valueOf(INITIAL_SCHEMA_VERSION),
> schemaCh -> schemaCh.changeSchema(
> ByteUtils.toBytes(
> SchemaService.prepareSchemaDescriptor(
>
> ((ExtendedTableView)ch).schemas().size(),
> ch
> )
> )
> )
> )
> );
> }
> )
> )
> {code}
>
> Besides obvious that allowed to simplify AffinityManager and SchemaManager
> that were converted to simple stateless services.
> 2. Same is about table id: instead of using meta storage revision and update
> counter (this wasn't implemented though) explicit table id generation was
> used. This is the main reason for the large number of changed files.
> 3. Inner local events were substituted by common Futures within
> create/alter/dropTable.
> Few more tickets were created for further improvements:
> [IGNITE-15414 Schema validation refactoring with configuration
> validators.|https://issues.apache.org/jira/browse/IGNITE-15414]
> [IGNITE-15480 Refactor index column mapping based on column id automatically
> generated on column creation with the help of internal configuration
> extensions.|https://issues.apache.org/jira/browse/IGNITE-15480]
> [IGNITE-15485 Support table rename
> operation.|https://issues.apache.org/jira/browse/IGNITE-15485]
> Some tickets for further refactoring are still awaited:
> [IGNITE-15409 Listener with any
> placeholders|https://issues.apache.org/jira/browse/IGNITE-15409]
> [IGNITE-15412 Configuration registry: add ability to retrieve distributed
> value|https://issues.apache.org/jira/browse/IGNITE-15412] That will allow to
> remove metastorage from table manager completely and get rid of current
> verison of TableManager#tableNamesConfigured
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
