xiangfu0 commented on PR #19303: URL: https://github.com/apache/pinot/pull/19303#issuecomment-5491260890
Rebased and updated on `7c3954f8b75808fc17faa6d09fa184aea3a34ca3`, a single commit directly on current `master` (`88ea694bae5f98760fb8c61f737fe6d3bc713427`). I re-audited all eight review threads plus the non-threaded follow-ups; all threads are resolved and no newer human feedback was left unaddressed. The final patch now: - coalesces mutable-HNSW refreshes by exact writer generation, short-circuits empty scopes, and keeps uncommitted rows visible; - uses sparse exact scan when the required scope and forward index make that cheaper, while dense/no-forward cases retain filtered ANN and never degrade to a post-filter; - shares `BasePinotDocIdBitmapFilterQuery`, resets crash-left temp indexes with `OpenMode.CREATE`, fails fast on missing doc-ID values, and cleans up after checked or unchecked close failures; - warms benchmark readers before timing, defaults to 200 aggregate samples, proves a measured add/query overlap, observes a real refresh waiter, and verifies exactly one refresh for a frozen reader generation. Validation on the published tree: - 80 targeted vector tests: 64 `pinot-segment-local` + 16 `pinot-core`; - 62 planner/strategy/operator tests; - `MutableVectorIndexTest`: 18/18; - `VectorUpsertTableTest`: 2/2 across both query engines (full 63-module reactor); - reduced and 25,000-vector performance workloads, including `timedIngestQueryOverlaps=1`, a real waiter, and refresh delta `1`; - Spotless, license format, Checkstyle, and license check across all five affected modules. Fresh CI has been triggered for this exact head. @Jackie-Jiang, please re-review when convenient; the earlier change request remains the outstanding review gate until then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
