thomasrebele opened a new pull request, #6825: URL: https://github.com/apache/hive/pull/6825
### What changes were proposed in this pull request? Update hive-postgres-tpcds-metastore to use a TPC-DS 30TB dump with histogram column statistics. I've spent quite a lot of time to find a way to reduce the image size while keeping the performance of TestTezTPCDS30TBPerfCliDriver. I opted for compressing the raw database files from Postgres and setting some Postgres config options. See [the scripts](https://github.com/thomasrebele/hive-postgres-metastore/tree/d4409d7e4a22980c93e1ed6b647077f0914ea256) that I used to cleanup and generate the raw dump. On my machine, TestTezTPCDS30TBPerfCliDriver using the proposed image runs in about 270s, while TestTezTPCDS30TBPerfCliDriver with the currently used image takes 350s, assuming the images have already been downloaded. So while the image with the histogram column statistics has a much larger database, the tests are executed faster. The download time is not taken into account, as here the image without the histogram statistics would clearly win. ### Why are the changes needed? The dump includes histogram statistics, making it possible to use them in future tests. ([HIVE-26830](https://issues.apache.org/jira/browse/HIVE-26830)). ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? I build the image locally and executed TestTezTPCDS30TBPerfCliDriver. See [the corresponding q.out file changes](https://github.com/thomasrebele/hive/commit/8039009c9000229879064ee59aaa9cfdb1711c0b). See [this comment](https://github.com/apache/hive/pull/6163#issuecomment-3492582155) for an explanation for the q.out file changes (the histograms are not yet used in the planning phase). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
