xiangfu0 commented on PR #18722: URL: https://github.com/apache/pinot/pull/18722#issuecomment-4665331371
Closing — this isn't the right tradeoff. The explicit `jsonExtractObject` parse-once intermediate (#18711) already solves the repeated-extraction case cleanly and opt-in. An automatic per-record parse cache adds a per-call check plus a per-record scratch allocation to the hot ingestion path, and new SPI surface on `GenericRow`, but only helps the narrow pattern of many `jsonPath*` extractions on the same column: - at high ingestion throughput, that broad per-record/per-call overhead (and the extra allocation/GC across all partition-consumer threads) isn't a clean win; - at low ingestion throughput, there's no re-parse perf problem to solve. Users who extract many fields from one JSON column should use `jsonExtractObject(col)` with an intermediate column (now also valid at config time via #18721). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
