[
https://issues.apache.org/jira/browse/CASSANDRA-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick McFadin updated CASSANDRA-21659:
----------------------------------------
Description:
Every schema change allocates in proportion to the whole schema rather than to
what changed.
h2. What
Keyspaces.diff and Tables.diff build created/dropped with filter(), which
removes every non-matching entry from a copy one at a time. On a CREATE TABLE
no keyspace is created or dropped, so Keyspaces.filter falls into
withoutKsTablesViews and removes every table in the cluster individually –
twice per diff. Tables.diff then calls TableMetadata.compare for every
surviving table, and compare has no identity fast path.
Several diffs run per DDL: AlterSchemaStatement:194 and :200, AlterSchema:160,
DistributedSchema:213, :260 and :325.
h2. Change
* Collect created/dropped directly instead of filtering whole collections.
* Skip comparing entries carried over by reference. Untouched TableMetadata
are reference-identical across a schema change (Tables.Builder.add stores the
instance verbatim), and x.compare(x) is empty by construction, so identity is
exact here, not an approximation.
No signature, format or API changes.
h2. Result
New KeyspacesDiffScalingTest measures allocation for adding one table at 400
and 3200 existing tables and asserts the growth ratio. Allocation rather than
elapsed time: counted exactly rather than sampled, independent of GC timing,
and it is what produces the failure (a GC wall, not gradual slowdown).
|| ||400 tables||3200 tables||growth||
|before|292,552 B|3,774,864 B|12.9x|
|after|3,944 B|3,944 B|1.0x|
Before is superlinear because the BTreeMap teardown is O(N log N).
h2. Scope – read before benchmarking
This removes the allocation term, not the scan. Time stays O(N) per diff and
bulk creation stays quadratic in wall-clock. What it removes is the allocation
that drives the GC wall. Wall clock at small N will barely move.
h2. Tests
New KeyspacesDiffScalingTest: 2 scaling assertions, 3 correctness guards.
Verified red before and green after, and re-verified the final test fails with
the fix stashed. Regression: schema 27 suites/130 tests, tcm 13/55. checkstyle
and checkstyle-test clean.
was:
Every schema change allocates in proportion to the whole schema rather than to
what changed.
h2. What
Keyspaces.diff and Tables.diff build created/dropped with filter(), which
removes
every non-matching entry from a copy one at a time. On a CREATE TABLE no
keyspace
is created or dropped, so Keyspaces.filter falls into withoutKsTablesViews and
removes every table in the cluster individually – twice per diff. Tables.diff
then calls TableMetadata.compare for every surviving table, and compare has no
identity fast path.
Several diffs run per DDL: AlterSchemaStatement:194 and :200, AlterSchema:160,
DistributedSchema:213, :260 and :325.
h2. Change
* Collect created/dropped directly instead of filtering whole collections.
* Skip comparing entries carried over by reference. Untouched TableMetadata are
reference-identical across a schema change (Tables.Builder.add stores the
instance verbatim), and x.compare(x) is empty by construction, so identity is
exact here, not an approximation.
No signature, format or API changes.
h2. Result
New KeyspacesDiffScalingTest measures allocation for adding one table at 400 and
3200 existing tables and asserts the growth ratio. Allocation rather than
elapsed
time: counted exactly rather than sampled, independent of GC timing, and it is
what produces the failure (a GC wall, not gradual slowdown).
|| ||400 tables||3200 tables||growth||
|before|292,552 B|3,774,864 B|12.9x|
|after|3,944 B|3,944 B|1.0x|
Before is superlinear because the BTreeMap teardown is O(N log N).
h2. Scope – read before benchmarking
This removes the allocation term, not the scan. Time stays O(N) per diff and
bulk
creation stays quadratic in wall-clock. What it removes is the allocation that
drives the GC wall. Wall-clock at small N will barely move.
h2. Tests
New KeyspacesDiffScalingTest: 2 scaling assertions, 3 correctness guards.
Verified
red before and green after, and re-verified the final test fails with the fix
stashed. Regression: schema 27 suites/130 tests, tcm 13/55. checkstyle and
checkstyle-test clean.
> Adding a large amount of schema allocates excessively
> -----------------------------------------------------
>
> Key: CASSANDRA-21659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21659
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
> Reporter: Patrick McFadin
> Assignee: Patrick McFadin
> Priority: Normal
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Every schema change allocates in proportion to the whole schema rather than
> to what changed.
> h2. What
> Keyspaces.diff and Tables.diff build created/dropped with filter(), which
> removes every non-matching entry from a copy one at a time. On a CREATE TABLE
> no keyspace is created or dropped, so Keyspaces.filter falls into
> withoutKsTablesViews and removes every table in the cluster individually –
> twice per diff. Tables.diff then calls TableMetadata.compare for every
> surviving table, and compare has no identity fast path.
> Several diffs run per DDL: AlterSchemaStatement:194 and :200,
> AlterSchema:160, DistributedSchema:213, :260 and :325.
> h2. Change
> * Collect created/dropped directly instead of filtering whole collections.
> * Skip comparing entries carried over by reference. Untouched TableMetadata
> are reference-identical across a schema change (Tables.Builder.add stores the
> instance verbatim), and x.compare(x) is empty by construction, so identity is
> exact here, not an approximation.
> No signature, format or API changes.
> h2. Result
> New KeyspacesDiffScalingTest measures allocation for adding one table at 400
> and 3200 existing tables and asserts the growth ratio. Allocation rather than
> elapsed time: counted exactly rather than sampled, independent of GC timing,
> and it is what produces the failure (a GC wall, not gradual slowdown).
> || ||400 tables||3200 tables||growth||
> |before|292,552 B|3,774,864 B|12.9x|
> |after|3,944 B|3,944 B|1.0x|
> Before is superlinear because the BTreeMap teardown is O(N log N).
> h2. Scope – read before benchmarking
> This removes the allocation term, not the scan. Time stays O(N) per diff and
> bulk creation stays quadratic in wall-clock. What it removes is the
> allocation that drives the GC wall. Wall clock at small N will barely move.
> h2. Tests
> New KeyspacesDiffScalingTest: 2 scaling assertions, 3 correctness guards.
> Verified red before and green after, and re-verified the final test fails
> with the fix stashed. Regression: schema 27 suites/130 tests, tcm 13/55.
> checkstyle and checkstyle-test clean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
