[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17833527#comment-17833527 ] Jeremiah Jordan commented on CASSANDRA-17044: - Yes it was part of CEP-18 https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=95652201#content/view/191335397 > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1-alpha1, 4.1 > > Time Spent: 20m > Remaining Estimate: 0h > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17833303#comment-17833303 ] Jeff Jirsa commented on CASSANDRA-17044: [~jjordan] - you mentioned "A CEP around this is in process, should be ready to propose in the next week" - did you folks ever publish that? > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1-alpha1, 4.1 > > Time Spent: 20m > Remaining Estimate: 0h > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696941#comment-17696941 ] Jacek Lewandowski commented on CASSANDRA-17044: --- [~benedict] thanks for pointing that. In CASSANDRA-18291 I'm fixing a different minor issue with schema management and I attached also a commit to fix that perf regression. If you have a moment, you can review, just a couple of lines. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1-alpha1, 4.1 > > Time Spent: 20m > Remaining Estimate: 0h > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696082#comment-17696082
]
Benedict Elliott Smith commented on CASSANDRA-17044:
This patch [introduced a minor performance regression to schema
lookups|https://github.com/apache/cassandra/blob/2e680a33c03ce66d4b1358e1a1cc11cf4ee0189f/src/java/org/apache/cassandra/schema/Schema.java#L323],
which affects query parsing, by invoking {{distributedAndLocalKeyspaces}} in a
lookup method, which involves building a new map before finding an element.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.1-alpha1, 4.1
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506543#comment-17506543 ] Jacek Lewandowski commented on CASSANDRA-17044: --- https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/196/workflows/4423feec-655a-4964-b836-5e727a90fceb > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506429#comment-17506429 ] Jacek Lewandowski commented on CASSANDRA-17044: --- My apologize for not linking the latest test results. I've run the tests before merging on freshly rebased branch and there were one failure also present on trunk and one flaky in jvm dtest afair. I'll link it later today (afk atm) and look into the tests results you mentioned > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17506404#comment-17506404 ] Josh McKenzie commented on CASSANDRA-17044: --- So the big caveat to the following: ASF Jenkins infra has had some disk space issues and builds not even completing, so it's quite possible the below failures are due to the environment and not this patch. There's one test that looks like it didn't have a record before this commit and is failing now: https://ci-cassandra.apache.org/job/Cassandra-trunk/1005/testReport/junit.framework/TestSuite/ We've had failures on trunk steadily creeping up over the course of 4 or so commits but it's also right after a stretch of 4 or so failed builds. [CI Results] Branch: trunk, build number: 1005 jenkins url: [https://ci-cassandra.apache.org/job/Cassandra-trunk/1005/] JIRA: CASSANDRA-17044 commit url: [https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=2b2c6decfafc6235ad537e72073fab2fd4467e2f] affected paths: * src/java/org/apache/cassandra/cql3/QueryProcessor.java * test/unit/org/apache/cassandra/db/compaction/CompactionsTest.java * test/unit/org/apache/cassandra/SchemaLoader.java * src/java/org/apache/cassandra/tools/SSTableLevelResetter.java * test/unit/org/apache/cassandra/locator/AssureSufficientLiveNodesTest.java * test/unit/org/apache/cassandra/utils/CassandraGenerators.java * src/java/org/apache/cassandra/schema/Diff.java * src/java/org/apache/cassandra/schema/MigrationCoordinator.java * test/unit/org/apache/cassandra/db/SystemKeyspaceTest.java * src/java/org/apache/cassandra/service/PendingRangeCalculatorService.java * src/java/org/apache/cassandra/repair/consistent/admin/SchemaArgsParser.java * src/java/org/apache/cassandra/schema/SchemaMigrationDiagnostics.java * test/unit/org/apache/cassandra/db/RowCacheTest.java * src/java/org/apache/cassandra/schema/SchemaTransformation.java * test/unit/org/apache/cassandra/service/paxos/uncommitted/PaxosRowsTest.java * test/unit/org/apache/cassandra/db/commitlog/CommitLogTest.java * src/java/org/apache/cassandra/schema/Keyspaces.java * test/unit/org/apache/cassandra/db/compaction/CompactionTaskTest.java * src/java/org/apache/cassandra/schema/SchemaTransformations.java * src/java/org/apache/cassandra/schema/OfflineSchemaUpdateHandler.java * src/java/org/apache/cassandra/db/guardrails/Guardrails.java * src/java/org/apache/cassandra/dht/BootStrapper.java * src/java/org/apache/cassandra/net/MessagingService.java * test/unit/org/apache/cassandra/db/view/ViewUtilsTest.java * src/java/org/apache/cassandra/tools/StandaloneSplitter.java * test/unit/org/apache/cassandra/db/streaming/EntireSSTableStreamConcurrentComponentMutationTest.java * test/unit/org/apache/cassandra/index/sasi/disk/PerSSTableIndexWriterTest.java * src/java/org/apache/cassandra/schema/DefaultSchemaUpdateHandler.java * src/java/org/apache/cassandra/utils/FBUtilities.java * test/unit/org/apache/cassandra/db/SecondaryIndexTest.java * test/unit/org/apache/cassandra/db/ReadCommandTest.java * test/unit/org/apache/cassandra/db/SchemaCQLHelperTest.java * src/java/org/apache/cassandra/gms/Gossiper.java * test/unit/org/apache/cassandra/db/compaction/CompactionsBytemanTest.java * src/java/org/apache/cassandra/db/commitlog/CommitLog.java * src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java * src/java/org/apache/cassandra/service/CassandraDaemon.java * test/unit/org/apache/cassandra/hints/HintTest.java * src/java/org/apache/cassandra/schema/SchemaChangeNotifier.java * test/unit/org/apache/cassandra/cql3/ViewSchemaTest.java * test/unit/org/apache/cassandra/locator/SimpleStrategyTest.java * src/java/org/apache/cassandra/tools/SSTableOfflineRelevel.java * test/simulator/main/org/apache/cassandra/simulator/cluster/OnInstanceSyncSchemaForBootstrap.java * src/java/org/apache/cassandra/schema/Types.java * tools/stress/src/org/apache/cassandra/stress/CompactionStress.java * src/java/org/apache/cassandra/batchlog/BatchlogManager.java * test/distributed/org/apache/cassandra/distributed/impl/Instance.java * test/unit/org/apache/cassandra/service/LeaveAndBootstrapTest.java * src/java/org/apache/cassandra/cache/AutoSavingCache.java * test/unit/org/apache/cassandra/service/StorageServiceServerTest.java * src/java/org/apache/cassandra/net/MessagingServiceMBeanImpl.java * src/java/org/apache/cassandra/schema/DefaultSchemaUpdateHandlerFactory.java * test/distributed/org/apache/cassandra/distributed/test/metric/TableMetricTest.java * test/unit/org/apache/cassandra/schema/SchemaTestUtil.java * test/unit/org/apache/cassandra/db/commitlog/CommitLogUpgradeTest.java * src/java/org/apache/cassandra/cql3/statements/schema/CreateTableStatement.java * src/java/org/apache/cassandra/schema/KeyspaceMetadata.java * test/unit/org/apache/cassandra/service/OptionalTasksTest.java *
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458375#comment-17458375 ] Jacek Lewandowski commented on CASSANDRA-17044: --- ||j11|| |[(/)|https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/151/workflows/1805e301-cf48-48f6-bc24-6452f9f58137]| > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455208#comment-17455208 ] Jacek Lewandowski commented on CASSANDRA-17044: --- ||j11|| |[(!)|https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/149/workflows/e7293b1d-b546-4019-9b35-04e0be49b5be/jobs/828] > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453898#comment-17453898 ] Jacek Lewandowski commented on CASSANDRA-17044: --- i've rebased it, I'll provide test links once they are worth showing ;-) > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440533#comment-17440533 ] Alex Petrov commented on CASSANDRA-17044: - [~jlewandowski] thank you for quick turnaround and for being so facilitating and moving issues to their separate patches. Feel free to pick me up as a reviewer on some of them if you wish. I've glanced over the patch and it mostly looks good to me. Would mind to submit a circleci before I give a final +1? > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438941#comment-17438941 ] Jacek Lewandowski commented on CASSANDRA-17044: --- btw. I'm updating just the branch against trunk > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438940#comment-17438940
]
Jacek Lewandowski commented on CASSANDRA-17044:
---
I reverted {{SchemaManager}} back to {{Schema}} as it seems there is no
consensus for that rename. As I said, it is not crucial for this ticket and I
can be made any time later if needed.
bq. Even though I generally like that SchemaChangeListener is now an interface
rather than abstract class, I’m concerned about potential users who might have
relied on it being implemented the way it was. Given this was a useful
interface, I think we should be more careful with it. I think it’s possible to
bring backwards compatibility back by creating a wrapper class with the old
interface. Would still require recompilation for the users, but could will make
it painless to migrate.
According to the later discussion I assumed we can keep just an interface
bq. Is there any reason we wouldn’t want to make the maps in
TableMetadataRefCache immutable? I realize they can be sizeable, but in this
case we could also hide them behind the interface, make default implementation
immutable and let other implementations be more efficient if necessary. This
way we can atomically swap/publish to all three maps.
Ok, I've implemented it so that it is immutable. However, I have some concerns
in that case because the {{TableMetadataRef}} objects are purposely mutable and
thus we probably should keep the synchronization (although I removed it in the
commit)
bq. I’m not sure about the name SharedSchema class and nomenclature:
bq. Class itself, if the whole point was to add a version to Keyspaces, maybe
we should just add the version to Keyspaces, and make sure keyspaces are being
updated synchronously with the version. Moreover, I think having version here
and version left in SchemaManager is making it more confusing. I realise that
you’ve tried to hide the version for SharedSchema in the Default handler, it’s
unclear whose responsibility the version actually is.
As far as I understand, version not always makes sense with {{Keyspaces}} - for
example, we only care about the version calculated for non-local keyspaces. So
I'd prefer the approach with removing that class entirely and keep version
separate. On the other hand, in a couple of places keyspaces and version are
passed together to a method and it is quite convenient to keep them in a single
immutable object.
bq. Nomenclature: I don’t think this schema is “shared” in any way: all
keyspaces that aren’t local are just nonlocal, and a subset of them is just
“userDefined”. Maybe we could use “replicated” or “distributed” instead of
“nonlocal”.
I really had a problem with that. I like {{DistributedSchema}} name and I
committed the renaming.
bq. Is there anything that justifies the existing of LocalKeyspaces class and
prevents it from being just an instance of Keyspaces which is more functional
but seems to be at least equivalent for its current usages?
I have no explanation, perhaps I was working on this for too long and another
pair of eyes was needed to catch this. I committed removal of
{{LocalKeyspaces}} in favor of using just {{Keyspaces}}.
bq. reloadSchemaAndAnnounceVersion seems to pick all keyspaces and then filter
out all system/local ones. Maybe we can just use “shared” (in nomenclature of
your patch)
It gets modified in later commits.
bq. in SchemaDiagnostics, “schemata” doesn’t seem to be a typo, but rather a
plural of “schema”. There are a few other mentions of schemata in this file, so
I’d check it again.
I checked - my understanding is that the schema is a set of all keyspaces
metadata rather than metadata of a single keyspace. That's why we have
{{Schema}} class to represent those keyspaces. We only deal with multiple
schemata during the transformation, in particular we have the old schema and
the new schema but this is unrelated to the fixed typo.
bq. Should we make a separate implementation of IEndpointStateChangeSubscriber
within SchemaUpdateHandler, since
I'm not sure if I get what do you mean by that - do you think I should create a
kinda inner class implementing that interface instead of implementing it on the
handler level?
bq. There seems to be an unintentional use of DSE here. I do not mind the
presence of this method though even though it seems to be unused.
bq. Nit: here probably you have meant “local keyspace definitions” or something
similar.
Hopefully fixed.
bq. Unfortunately, I could not find a full set of passing tests accompanying
the patch. Could you, when posting the updates, make sure to include a link to
CircleCI?
Indeed, I haven't run CI yet because I expected (possibly significant) changes
to be made as a result of the reviews. However, it is not an unproven solution
- it was first implemented on a fork in DS and all the tests
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435282#comment-17435282 ] Jacek Lewandowski commented on CASSANDRA-17044: --- For things which were mentioned as unrelated and suggested to be addressed in separate tickets, I've created: - CASSANDRA-17071 - CASSANDRA-17072 - CASSANDRA-17073 - CASSANDRA-17074 > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433720#comment-17433720 ] Alex Petrov commented on CASSANDRA-17044: - I'd say, for the APIs where we expect significant changes, I'd say we should not (and can not, really) make any guarantees whatsoever. For some of the APIs (for which seems to be some discussion on ML), we should have at least some stability. That said, say, if we have a 2i API, since our main goal as Apache project is to maintain 2i implementations in-tree (be it SASI or, potentially, SAI), if someone were to come and make a change, but at the same time adjust the in-tree SASI and SAI implementations accordingly, I'd say we should allow such a change. Maintainers of external indexes will have to adjust. What I haven't considered when writing my previous comment is if we'll allow external code to dictate what's good for the project, we may not be able to deliver the changes in the best possible way. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433719#comment-17433719 ] Benedict Elliott Smith commented on CASSANDRA-17044: I think APIs fall into two buckets: internal and external. I can't imagine a reason why internal APIs should need annotating, as they should offer no guarantees at all to any user that exploits them, they simply provide utility for structuring or testing the code. AFAICT this refactor falls into this bucket. External APIs should have some explicit consideration for compatibility, even if it is minimal it should have a well defined and documented approach with what the user can expect. External APIs obviously have to clear much greater hurdles for introduction to the codebase given the restrictions they place on future development, and I'd be pretty opposed to that in this instance. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433716#comment-17433716
]
Jacek Lewandowski commented on CASSANDRA-17044:
---
Thank you [~ifesdjeen], I really appreciate such a fast review. I'll be
responding to your points shortly.
Just regarding the APIs - perhaps some idea for the future would be to start
explicitly marking which interfaces/classes are {{@DeveloperAPI}}? I saw this
approach in Spark project. Such classes had some guarantees about not being
changed for certain versions range.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433708#comment-17433708 ] Alex Petrov commented on CASSANDRA-17044: - [~benedict] you're right. My thinking was that since it's a Listener, there might be implementations outside the codebase, but I agree we should always move towards what's most useful for Cassandra today. This, then, also applies to the API introduced in this patch: if changes introducing consistent schema require changes in this API, we will have to make changes to it. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433683#comment-17433683 ] Benedict Elliott Smith commented on CASSANDRA-17044: bq. Even though I generally like that SchemaChangeListener is now an interface rather than abstract class, I’m concerned about potential users who might have relied on it being implemented the way it was My personal view on internal APIs like this is that we shouldn't offer any guarantees about backwards compatibility. In this instance if it's easy to maintain then great, but we should not set an expectation of this. The main goal should always be what is useful for Cassandra today. Just my 2¢. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433642#comment-17433642
]
Alex Petrov commented on CASSANDRA-17044:
-
Thank you for the patch and clarifications. I like the overall direction
towards modularisation and interfaces. My comments are following.
* Even though I generally like that {{SchemaChangeListener}} is now an
interface rather than abstract class, I’m concerned about potential users who
might have relied on it being implemented the way it was. Given this was a
useful interface, I think we should be more careful with it. I think it’s
possible to bring backwards compatibility back by creating a wrapper class with
the old interface. Would still require recompilation for the users, but could
will make it painless to migrate.
* Is there any reason we wouldn’t want to make the maps in
{{TableMetadataRefCache}} immutable? I realise they can be sizeable, but in
this case we could also hide them behind the interface, make default
implementation immutable and let other implementations be more efficient if
necessary. This way we can atomically swap/publish to all three maps.
* I’m not sure about the name {{SharedSchema}} class and nomenclature:
** Class itself, if the whole point was to add a version to Keyspaces, maybe
we should just add the version to Keyspaces, and make sure keyspaces are being
updated synchronously with the version. Moreover, I think having version here
and version left in {{SchemaManager}} is making it more confusing. I realise
that you’ve tried to hide the version for {{SharedSchema}} in the Default
handler, it’s unclear whose responsibility the version actually is.
** Nomenclature: I don’t think this schema is “shared” in any way: all
keyspaces that aren’t local are just nonlocal, and a subset of them is just
“userDefined”. Maybe we could use “replicated” or “distributed” instead of
“nonlocal”.
* Is there anything that justifies the existing of {{LocalKeyspaces}} class
and prevents it from being just an instance of {{Keyspaces}} which is more
functional but seems to be at least equivalent for its current usages?
* {{reloadSchemaAndAnnounceVersion}} seems to pick all keyspaces and then
filter out all system/local ones. Maybe we can just use “shared” (in
nomenclature of your patch)
* in {{SchemaDiagnostics}}, “schemata” doesn’t seem to be a typo, but rather a
plural of “schema”. There are a few other mentions of schemata in this file, so
I’d check it again.
* Should we make a separate implementation of
{{IEndpointStateChangeSubscriber}} within {{SchemaUpdateHandler}}, since
* There seems to be an unintentional use of DSE
[here|[https://github.com/apache/cassandra/pull/1270/commits/42a825a98fb4d2564508463911f43fd2754051d6#diff-fcad6f463c8c500997d27cceced47b8209088e21df328f8bc8f75507bc92b3eaR97]].
I do not mind the presence of this method though even though it seems to be
unused.
* Nit: [CASSANDRA-17044: Refactor schema management by jacek-lewandowski ·
Pull Request #1270 · apache/cassandra · GitHub|#1270 · apache/cassandra ·
GitHub]([https://github.com/apache/cassandra/pull/1270/commits/42a825a98fb4d2564508463911f43fd2754051d6#diff-67531e8854562a763e98074cadaeee6d7c7c288a30ebef9fa465e249acab9fb2R115])
here probably you have meant “local keyspace definitions” or something similar.
* Unfortunately, I could not find a full set of passing tests accompanying the
patch. Could you, when posting the updates, make sure to include a link to
CircleCI?
Lastly, I feel like for a 4k lines of code patch, this one has surprisingly few
tests. I realise that schema was always a generally undertested area of code,
but I think our current quality standards do not allow us to do massive
refactoring without substantial testing. Therefore, I suggest we create fuzz
test that verifies schema eventual propagation, reads/writes during schema
changes, and cross-node consistency of schema operations themselves, including
concurrent ones. What seems to be particularly important to test is the gossip
propagation of the version ID, which would lead to schema pulls, alongside with
active schema pushes that is done when there’s a non-local schema change, and
picking up schema during bootstrap.
Also, two comments about the scope of the patch, since the patch seems to have
included several changes that seem to be unrelated to refactoring, and are
rather bugfixes or performance improvements:
* 631fa2939708dd3cbd90f26442a7e167a3386cd7 is touching things that are used in
almost every subsystem in the code. {{Keyspace.open()}} is a super common call.
I think changes to such method deserves extensive testing, especially if
previous code was inefficient but not problematic. If this is a performance
improvement, I suggest to extract it and put it in a different patch, add tests
and demonstrate the improvement. If it’s a bug -
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432312#comment-17432312
]
Jacek Lewandowski commented on CASSANDRA-17044:
---
Note there is no more references to gossiper, messaging or to schema keyspace
in {{Schema}} - that stuff is in the update handler
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432310#comment-17432310
]
Jacek Lewandowski commented on CASSANDRA-17044:
---
[~ifesdjeen] the current {{Schema}} implementation does several things - it
maintains the local copy of keyspaces metadata, the keyspace instances, table
metadata references cache and schema change listeners. Performing an update on
a schema, say dropping a table requires several things to happen, in particular:
- we need to remove it from keyspace metadata
- we need to tear down related CFS and remove it from the keyspace instance
- we need to remove the table metadata reference
- we need to send the notifications
- we need to store the updates and sync with other nodes
- we need to update the version
Some of those operations are unrelated to the way how we store the schema. In
particular, managing CFSs, handling notifications or table metadata references
is common for whichever schema storage mechanism we use.
Therefore I've extracted {{SchemaUpdateHandler}} interface which serves just
for storage and sync of the schema. That is, the current {{Schema}} (or
{{SchemaManager}}) only needs to call a method on that interface to request the
schema change. On the other hand, the factory of update handler takes the
callback function as an argument. This callback is called whenever the schema
gets changed, regardless which party initiated the change. In other words, the
caller ({{Schema}}) can only request schema changes and react to the changes
that were applied on the schema, wherever that schema is.
There are obviously some issues which remains unfixed as I was not aiming to
fix them - there can be a failure when applying changes locally, so the changes
can be applied only partially and there is no explicit recovery mechanism (say
we fail to tear down CFS when dropping a table). But this does not affect
changes to the schema itself as we handle local application as a followup
action.
Does it clarify things a bit?
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431286#comment-17431286 ] Alex Petrov commented on CASSANDRA-17044: - bq. Sorry for the noise and thank you both for taking the time to clarify this. No worries: I do not see discussing things as noise, especially when it's productive. I appreciate your input and collaboration. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431280#comment-17431280 ] Paulo Motta commented on CASSANDRA-17044: - bq. I agree with what Benedict has mentioned above with regard the fact that XManager is well used with countable nouns, where it makes it clear that it manager multiple instances of X. Fair enough - after thinking more about it and checking the code this makes sense. Sorry for the noise and thank you both for taking the time to clarify this. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431259#comment-17431259 ] Alex Petrov commented on CASSANDRA-17044: - [~jlewandowski] could you give some background on how you're planning to use / plug in etcd? Some changes are just not fully obvious without this information. I really like strides towards encapsulation, just want to understand where some of the things coming from. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431258#comment-17431258
]
Benedict Elliott Smith commented on CASSANDRA-17044:
bq. HintsManager
This is a {{HintsService}, or perhaps a {{HintsStore}}
Additionally, we have {{CommitLog}}, {{SystemKeyspace}},
{{SystemDistributedKeyspace}}, {{Tracing}}, {{PaxosState}},
{{CassandraMetricsRegistry}}, {{MessagingService}}, {{ActiveRepairService}},
{{VirtualKeyspaceRegistry}}, {{FullQueryLogger}}, {{DiagnosticEventService}},
{{PendingRangeCalculatorService}}, {{CacheService}}, and no doubt many more.
There are many {{XManager}} objects as well, but as far as I can tell these are
all countable nouns as I previously posited. {{Schema}} is not a countable
noun, and there is no specific set of {{Schema}} object to be managed, so
consistency is not a valid justification for this refactor in my view. Brevity
remains important.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431229#comment-17431229
]
Paulo Motta commented on CASSANDRA-17044:
-
{quote}"Manager" is one of those words that often means nothing by itself
{quote}
We already adopt the "Manager" suffix in many instances to distinguish the
object manager class from the actual (generally immutable) object being managed
(ie. HintsManager (manages Hints), SnapshotManager (manages Snapshots),
CompactionStrategyManager (manages Compaction Strategies), etc), so it would be
at the very least inconsistent with other parts of the codebase to have a
Schema object which *is not* a schema, but manages the schema, so I'm +1 for
making our codebase more consistent and meaningful by renaming {{Schema}} to
{{SchemaManager}} (perhaps in a separate ticket to reduce review noise on this
ticket).
{quote}I'd prefer to remove Duration class and leave just the utility methods
which implement the conversions we need (and those conversions should convert
to/from Java's Duration I think).
{quote}
sounds good to me
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431223#comment-17431223 ] Alex Petrov commented on CASSANDRA-17044: - bq. I tried hard to make individual commits to include distinct changes, I hope you find some value in that I do see a lot of value in it, and really appreciate you doing this! > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431220#comment-17431220
]
Jacek Lewandowski commented on CASSANDRA-17044:
---
Ok, I understand you point guys. This is not super important in the context in
this ticket.
[~ifesdjeen] - I tried hard to make individual commits to include distinct
changes, I hope you find some value in that
[~paulo] I'd prefer to remove {{Duration}} class and leave just the utility
methods which implement the conversions we need (and those conversions should
convert to/from Java's {{Duration}} I think).
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431206#comment-17431206 ] Benedict Elliott Smith commented on CASSANDRA-17044: bq. Overall, I'm -0 on the change, I just do not see addition of "manager" in the name as an improvement. I agree here, and in general we should always prefer shorter names if there is no improved meaning from longer names. Manager is one of those words that often means nothing by itself. It's usually added only to countable nouns that have concrete implementations themselves, such as CommitLogSegmentManager (which manages CommitLogSegments). In most other situation it is a bit like "Data" - it offers no additional context for the reader, only pollutes their screen making each line of code involving it harder to parse. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431203#comment-17431203
]
Alex Petrov commented on CASSANDRA-17044:
-
bq. It is used to modify the schema and query the schema by accessing various
parts of the system.
Right, by that logic we may want to rename {{CommitLog}} and {{Memtable}}. I
think this class is responsible for access to Schema, and many methods such as
{{loadFromDisk}} sort of make sense if the class is named {{Schema}} but not
really if it's called {{SchemaManager}}. Overall, I'm -0 on the change, I just
do not see addition of "manager" in the name as an improvement.
Renaming or leaving the name as-is seems like a matter of personal preference
more than anything, so if you strictly insist on renaming it, it's not a
problem but I'd ask to do it outside the scope of this ticket. I've renamed it
back in my review branch for now.
bw. For the noisiness - I purposely did that in a distinct commit where no
other changes were made for the review convenience.
I appreciate you doing that. I (personally) prefer to review patches in their
entirety after reviewing individual commits, since some changes are not
transitively obvious.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431201#comment-17431201 ] Paulo Motta commented on CASSANDRA-17044: - > For Duration I would really remove it as it does not seem to bring any value > and is not compatible with native java.time classes. I don't have any objection to remove it, as long as we keep the parsing capability used by the snapshot ttl feature. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431193#comment-17431193
]
Paulo Motta commented on CASSANDRA-17044:
-
bq. I'd ask Paulo Motta and Stefan Miklosovic about Duration.
We used the {{Duration}} class from CASSANDRA-15234 (cc [~e.dimitrova]) on
CASSANDRA-16789 mostly because we needed a way to parse {{Duration}} from
string which is not provided by the standard {{Duration}} class but that was a
specific use and didn't intend to replace java {{Duration}} globally but
perhaps this would be a good time to discuss how to standardize its use.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431167#comment-17431167 ] Jacek Lewandowski commented on CASSANDRA-17044: --- Thank you for explanation. I certainly forgot to do my homework ;) > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431165#comment-17431165
]
Benedict Elliott Smith commented on CASSANDRA-17044:
bq. I don't have any opinion on the new Future implementation yet apart from
that it should be renamed so that it is easily distinguishable from the native
Java's Future and Netty's Future.
As you mention, there are already two interfaces called {{Future}}, and these
are generally forbidden* from being used directly so there should be no
confusion. This {{Future}} implements both of these forbidden interfaces, as
well as Guava's {{ListenableFuture}}.
The purpose of this new {{Future}}, besides supporting simulation, was
specifically to standardise the very divergent approaches across the codebase,
and this consistency is now enforced by a linter to ensure we do not diverge
again.
\* I think Netty's isn't globally forbidden, due to needing it in places for
Netty integration, but I will file a patch to restrict its use to the minimal
set of classes to further restrict any potential for confusion.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431164#comment-17431164
]
Jacek Lewandowski commented on CASSANDRA-17044:
---
Thanks, I'll need to refactor the usages of futures then.
bq. I do not see anything obviously wrong with Schema as a name/concept that
would grant renaming it
>From my point of view, {{Schema}} is not a schema. It is used to modify the
>schema and query the schema by accessing various parts of the system. For the
>noisiness - I purposely did that in a distinct commit where no other changes
>were made for the review convenience.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431150#comment-17431150
]
Alex Petrov commented on CASSANDRA-17044:
-
> I was really surprised that there are new Cassandra specific Future and
> Duration implementations.
If you check the context, the change was made in the context of
[CEP-10|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-10%3A+Cluster+and+Code+Simulations],
and right now your branch throws {{Illegal import -
java.util.concurrent.CompletableFuture. [IllegalImport]}}. In short, you'll
have to use the Cassandra-specific implementation. I'd ask [~paulo] and
[~stefan.miklosovic] about Duration.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431092#comment-17431092
]
Jacek Lewandowski commented on CASSANDRA-17044:
---
btw. I was really surprised that there are new Cassandra specific {{Future}}
and {{Duration}} implementations. There can be some inconsistency around those
classes in the PR for trunk because my primary PR is against 4.0 and I used
{{CompletableFuture}} and {{Duration}} from Java.
I don't have any opinion on the new {{Future}} implementation yet apart from
that it should be renamed so that it is easily distinguishable from the native
Java's {{Future}} and Netty's {{Future}}.
For {{Duration}} I would really remove it as it does not seem to bring any
value and is not compatible with native {{java.time}} classes.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431089#comment-17431089 ] Jacek Lewandowski commented on CASSANDRA-17044: --- Sure [~ifesdjeen], thank you for volunteering! > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430606#comment-17430606 ] Alex Petrov commented on CASSANDRA-17044: - [~jlewandowski] may I volunteer myself as a reviewer for this one? > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429857#comment-17429857 ] Berenguer Blasi commented on CASSANDRA-17044: - Yeah, it's just that 16996 issues will be sthg to take into account here as well but that's about it without knowing the details here. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429827#comment-17429827
]
Jacek Lewandowski commented on CASSANDRA-17044:
---
Looking at [~Bereng] PR - there is a small overlap between our changes, but in
general those PRs address different things. My one is not going to fix the
synchronization issues, though some concepts, like the refactoring to not
access the schema directly through {{SchemaKeyspace}}, are common.
Whoever is going to review that, I suggest to go commit by commit, because
although some commits introduce many diff lines, they do not need much
attention.
> Refactor schema management to allow for schema source pluggability
> --
>
> Key: CASSANDRA-17044
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17044
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> The idea is decompose `Schema` into separate entities responsible for
> different things. In particular extract what is related to schema storage and
> synchronization into a separate class so that it is possible to create an
> extension point there and store schema in a different way than
> `system_schema` keyspace, for example in etcd.
> This would also simplify the logic and reduce the number of special cases,
> make all the things more testable and the logic of internal classes
> encapsulated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429366#comment-17429366 ] Caleb Rackliffe commented on CASSANDRA-17044: - Having not even read the patch, it feels like there might be some overlap w/ CASSANDRA-16996, which is probably going to land shortly. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429355#comment-17429355 ] Jeremiah Jordan commented on CASSANDRA-17044: - A CEP around this is in process, should be ready to propose in the next week. > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429340#comment-17429340 ] Jacek Lewandowski commented on CASSANDRA-17044: --- https://github.com/apache/cassandra/pull/1270 (cassandra-4.0 - perhaps not needed, but...) https://github.com/apache/cassandra/pull/1271 (trunk) > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
