Caideyipi opened a new pull request, #18211:
URL: https://github.com/apache/iotdb/pull/18211
## Description
This PR reduces short-lived allocation on wide tablet ingestion paths:
- Avoid creating an identity deduplication set for single InsertNode leaves
and single-child composite nodes in `InsertNodeMemoryEstimator`.
- Create `TabletStringInternPool` for Batch V2 raw tablet deserialization
only when there are multiple raw tablets.
- Preserve a null string intern pool through `PipeTransferTabletRawReqV2` so
the single raw tablet path does not recreate the pool internally.
- Add opt-in manual performance UTs for the 100,000-column paths.
## Tests
- `mvn spotless:apply -pl iotdb-core/datanode`
- `mvn -o -pl iotdb-core/datanode -am -Dcheckstyle.skip=true
-Dspotless.skip=true -DskipTests test-compile`
- BUILD SUCCESS
- `mvn -o -pl iotdb-core/datanode -Dcheckstyle.skip=true
-Dspotless.skip=true
-Dtest=InsertNodeMemoryEstimatorTest,PipeDataNodeThriftRequestTest test`
- Tests run: 52, Failures: 0, Errors: 0, Skipped: 0
- `mvn -o -pl iotdb-core/datanode -Dcheckstyle.skip=true
-Dspotless.skip=true
-Dtest=InsertNodeMemoryEstimatorPerformanceTest,PipeTransferTabletBatchReqV2PerformanceTest
-Diotdb.pipe.insert.node.memory.estimator.perf.enabled=true
-Diotdb.pipe.insert.node.memory.estimator.perf.iterations=3
-Diotdb.pipe.insert.node.memory.estimator.perf.warmup.iterations=1
-Diotdb.pipe.tablet.batch.v2.deserialize.perf.enabled=true
-Diotdb.pipe.tablet.batch.v2.deserialize.perf.iterations=3
-Diotdb.pipe.tablet.batch.v2.deserialize.perf.warmup.iterations=1 test`
- Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
- InsertNode memory estimate benchmark: columns=100000, warmups=1,
iterations=3, 1.572 ms/op
- Batch V2 single raw tablet deserialization benchmark: columns=100000,
warmups=1, iterations=3, median=7.362 ms/op
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
