abstractdog commented on PR #6793: URL: https://github.com/apache/hive/pull/6793#issuecomment-6300071305
### TPC-DS validation: flag off / on / off / on I ran all 99 TPC-DS queries against `tpcds_partitioned_parquet_10` on a k8s LLAP cluster. Each query ran four times in sequence, alternating `hive.llap.io.parquet.native.enabled` false / true / false / true, starting from a cold daemon. The query results cache was off. | Run | Flag | Cache state | Summed `Run DAG` | |---|---|---|---| | r1 | false | cold | 1502.2 s | | r2 | true | warm (filled by r1) | 764.4 s | | r3 | false | warm | 916.9 s | | r4 | true | warm | 757.7 s | **Results** - **Same answers:** 96/99 queries returned identical output in all four runs. The other three (q31, q65, q71) differed only in row order, and none of them has a total `ORDER BY`. - **Shared cache works:** r2 ran at almost a 100% hit rate on bytes that the stock reader cached in r1, so both readers cut and key files the same way. Only q34 and q93 had a few misses. - **Faster when warm:** with the same bytes served from the same cache, the native path is about 17% faster than the stock path (r3 vs r4: 916.9 s → 757.7 s). r2 and r4 are within 1% of each other, so the native path needed no warm-up of its own. The gain comes from decoding in place over the cache on the IO threads, with no copy into parquet's buffers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
