[
https://issues.apache.org/jira/browse/IGNITE-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-15374:
-------------------------------------
Description:
In general there are three core points of distributed flow refactoring:
1. Using only one interface instead of two in order to process distributed
configuration and corresponding events. Currently both ConfigurationRegistry
and MetastorageManager are used. Besides obvious point that it leads to some
mess it also means that metastorage keys should be exposed from configuration
registry.
In order to get read of MetastorageManager as a direct interface that provides
access to configuration it's required to
* Support internal/private properties in configuration framework
* Add ability to retrieve distributed up-to-date value with configuration
registry. Something like ConfigurationProperty.distirbuteValue() is expected.
As a result it will allow to change
{code:java}
//TODO This is an egregious violation of encapsulation. Current approach
has to be revisited.
private Set<String> tableNamesConfigured() {
IgniteBiTuple<ByteArray, ByteArray> range = toRange(new
ByteArray(PUBLIC_PREFIX)); Set<String> tableNames = new HashSet<>();
try (Cursor<Entry> cursor = metaStorageMgr.range(range.get1(),
range.get2())) {
while (cursor.hasNext()) {
Entry entry = cursor.next(); List<String>
keySplit = ConfigurationUtil.split(entry.key().toString()); if
(keySplit.size() == 5 && NamedListNode.NAME.equals(keySplit.get(4)))
tableNames.add(ByteUtils.fromBytes(entry.value()).toString());
}
}
catch (Exception e) {
LOG.error("Can't get table names.", e);
} return tableNames;
}
{code}
to something similar to
{code:java}
private Set<String> tableNamesConfigured() {
return new
HashSet<>(clusterCfgMgr.configurationRegistry().getConfiguration(TablesConfiguration.KEY).
tables().distirbuteValue().namedListKeys());
}
{code}
2. Using 1 step flow instead of 2 step one:
Currently, creation of a table configuration triggers corresponding listeners
that calculate assignments and schema descriptor and persists it to meta
storage. After implementing internal configuration extensions it'll be possible
to combine given steps in most cases, so as a result something like following
might be used:
{code:java}
clusterCfgMgr
.configurationRegistry()
.getConfiguration(TablesConfiguration.KEY)
.tables()
.change(
change -> change.create(
name,
(ch) -> {
tableInitChange.accept(ch);
((ExtendedTableChange)ch).changeAssignments(ByteUtils.toBytes(affMgr.calculateAssignments(ch.partitions(),
ch.replicas())));
((SchemaExtendedTableChange)ch).changeSchemaDescriptor(ByteUtils.toBytes(schemaMgr.prepareSchemaDescriptor(tableInitChange)));
}
)
)
.get();
{code}
Open issue here is table id generation. In case of using rev and update counter
from meta storage we still need two phase creation process with second step
that will persist table id to the configuration.
3. Using appropriate listeners instead of wide ones: instead of listening
tableUpdates and manual routing to particular schema update it's possible to
listen to every column type change, column rename, etc directly.
> Prototype event flow refactoring
> --------------------------------
>
> Key: IGNITE-15374
> URL: https://issues.apache.org/jira/browse/IGNITE-15374
> Project: Ignite
> Issue Type: Improvement
> Reporter: Alexander Lapin
> Assignee: Alexander Lapin
> Priority: Major
> Labels: ignite-3
>
> In general there are three core points of distributed flow refactoring:
> 1. Using only one interface instead of two in order to process distributed
> configuration and corresponding events. Currently both ConfigurationRegistry
> and MetastorageManager are used. Besides obvious point that it leads to some
> mess it also means that metastorage keys should be exposed from
> configuration registry.
> In order to get read of MetastorageManager as a direct interface that
> provides access to configuration it's required to
> * Support internal/private properties in configuration framework
> * Add ability to retrieve distributed up-to-date value with configuration
> registry. Something like ConfigurationProperty.distirbuteValue() is expected.
> As a result it will allow to change
> {code:java}
> //TODO This is an egregious violation of encapsulation. Current approach
> has to be revisited.
> private Set<String> tableNamesConfigured() {
> IgniteBiTuple<ByteArray, ByteArray> range = toRange(new
> ByteArray(PUBLIC_PREFIX)); Set<String> tableNames = new HashSet<>();
> try (Cursor<Entry> cursor = metaStorageMgr.range(range.get1(),
> range.get2())) {
> while (cursor.hasNext()) {
> Entry entry = cursor.next(); List<String>
> keySplit = ConfigurationUtil.split(entry.key().toString()); if
> (keySplit.size() == 5 && NamedListNode.NAME.equals(keySplit.get(4)))
>
> tableNames.add(ByteUtils.fromBytes(entry.value()).toString());
> }
> }
> catch (Exception e) {
> LOG.error("Can't get table names.", e);
> } return tableNames;
> }
> {code}
> to something similar to
> {code:java}
> private Set<String> tableNamesConfigured() {
> return new
> HashSet<>(clusterCfgMgr.configurationRegistry().getConfiguration(TablesConfiguration.KEY).
> tables().distirbuteValue().namedListKeys());
> }
> {code}
>
> 2. Using 1 step flow instead of 2 step one:
> Currently, creation of a table configuration triggers corresponding listeners
> that calculate assignments and schema descriptor and persists it to meta
> storage. After implementing internal configuration extensions it'll be
> possible to combine given steps in most cases, so as a result something like
> following might be used:
> {code:java}
> clusterCfgMgr
> .configurationRegistry()
> .getConfiguration(TablesConfiguration.KEY)
> .tables()
> .change(
> change -> change.create(
> name,
> (ch) -> {
> tableInitChange.accept(ch);
>
> ((ExtendedTableChange)ch).changeAssignments(ByteUtils.toBytes(affMgr.calculateAssignments(ch.partitions(),
> ch.replicas())));
>
> ((SchemaExtendedTableChange)ch).changeSchemaDescriptor(ByteUtils.toBytes(schemaMgr.prepareSchemaDescriptor(tableInitChange)));
> }
> )
> )
> .get();
> {code}
> Open issue here is table id generation. In case of using rev and update
> counter from meta storage we still need two phase creation process with
> second step that will persist table id to the configuration.
> 3. Using appropriate listeners instead of wide ones: instead of listening
> tableUpdates and manual routing to particular schema update it's possible to
> listen to every column type change, column rename, etc directly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
