[
https://issues.apache.org/jira/browse/CASSANDRA-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick McFadin updated CASSANDRA-21659:
----------------------------------------
Resolution: Duplicate
Status: Resolved (was: Triage Needed)
New Jira superceeds this one
> Adding a large amount of schema allocates excessively
> -----------------------------------------------------
>
> Key: CASSANDRA-21659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21659
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema
> Reporter: Patrick McFadin
> Assignee: Patrick McFadin
> Priority: Normal
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Every schema change allocates in proportion to the whole schema rather than
> to what changed.
> h2. What
> Keyspaces.diff and Tables.diff build created/dropped with filter(), which
> removes every non-matching entry from a copy one at a time. On a CREATE TABLE
> no keyspace is created or dropped, so Keyspaces.filter falls into
> withoutKsTablesViews and removes every table in the cluster individually –
> twice per diff. Tables.diff then calls TableMetadata.compare for every
> surviving table, and compare has no identity fast path.
> Several diffs run per DDL: AlterSchemaStatement:194 and :200,
> AlterSchema:160, DistributedSchema:213, :260 and :325.
> h2. Change
> * Collect created/dropped directly instead of filtering whole collections.
> * Skip comparing entries carried over by reference. Untouched TableMetadata
> are reference-identical across a schema change (Tables.Builder.add stores the
> instance verbatim), and x.compare(x) is empty by construction, so identity is
> exact here, not an approximation.
> No signature, format or API changes.
> h2. Result
> New KeyspacesDiffScalingTest measures allocation for adding one table at 400
> and 3200 existing tables and asserts the growth ratio. Allocation rather than
> elapsed time: counted exactly rather than sampled, independent of GC timing,
> and it is what produces the failure (a GC wall, not gradual slowdown).
> || ||400 tables||3200 tables||growth||
> |before|292,552 B|3,774,864 B|12.9x|
> |after|3,944 B|3,944 B|1.0x|
> Before is superlinear because the BTreeMap teardown is O(N log N).
> h2. Scope – read before benchmarking
> This removes the allocation term, not the scan. Time stays O(N) per diff and
> bulk creation stays quadratic in wall-clock. What it removes is the
> allocation that drives the GC wall. Wall clock at small N will barely move.
> h2. Tests
> New KeyspacesDiffScalingTest: 2 scaling assertions, 3 correctness guards.
> Verified red before and green after, and re-verified the final test fails
> with the fix stashed. Regression: schema 27 suites/130 tests, tcm 13/55.
> checkstyle and checkstyle-test clean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
