CalvinKirs commented on PR #66717: URL: https://github.com/apache/doris/pull/66717#issuecomment-5356909679
Reworked the estimators per the architecture follow-up (implemented in 66a94dd487b): - Production sizing now uses the suggested coarse cardinality formulas: a rounded-up constant per stable logical dimension plus the loader-materialized string payload. Per-node constants absorb post-admission lazy growth (schema indexes, fieldsBySourceId, RowType lookup maps, store-derived copies) instead of modeling those objects. - All SDK-private layout fingerprints and exact JVM/Guava layout modeling are removed. Unknown SDK subtypes (content files, partition containers, logical types) now receive generic conservative weights instead of disabling weighted caching. The only remaining SDK-private access is the TableMetadata snapshotsLoaded probe that keeps weighing IO-free. - The GPL-licensed jol-core test dependency is dropped (also resolves the dependency-review check); calibration tests now use a dependency-free reflective probe with an order-of-magnitude acceptance band (observed ratios 0.49-4.1) plus monotonicity and no-under-estimation-of-materialized-growth checks, instead of the previous 10% JOL bound. - The admission/lifecycle framework is unchanged: no IO or lazy materialization during weighing, bounded publication work, saturated arithmetic, O(1) hits/removals, softValues as the safety net. max-weight is documented as an estimated admission budget. Net effect on the sizing code: IcebergCacheSizeEstimator 1230 -> ~490 lines, PaimonCacheSizeEstimator 592 -> ~330, MetaCacheWeightUtils 368 -> ~160. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
