zabetak commented on code in PR #6825: URL: https://github.com/apache/hive/pull/6825#discussion_r4216653592
########## standalone-metastore/metastore-server/docker/hive-postgres-tpcds-metastore/Dockerfile: ########## @@ -14,9 +14,19 @@ # See the License for the specific language governing permissions and # limitations under the License. # -FROM postgres:12.3 +FROM postgres:18-alpine + +RUN wget -O /tmp/metastore_db.zstd https://github.com/thomasrebele/hive-postgres-metastore/releases/download/tpcds-30tb-histogram-1.1/metastore_tpcds30tb_with_histograms.raw_db.zstd \ + && echo 'bc9e206d5f963a14451dad6f499ab069800be9ce /tmp/metastore_db.zstd' > /tmp/metastore_db.sha1 \ + && sha1sum -c /tmp/metastore_db.sha1 \ + && chown postgres:postgres /tmp/metastore_db.zstd + +COPY entrypoint.sh /usr/local/bin/entrypoint.sh +RUN chmod u+x /usr/local/bin/entrypoint.sh +COPY postgresql.conf /etc/postgresql/postgresql.conf +RUN chown postgres:postgres /etc/postgresql/postgresql.conf + +ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] + +CMD ["postgres", "-c", "config_file=/etc/postgresql/postgresql.conf"] Review Comment: The `PostgreSQLContainer` used by `PostgresTPCDS.java` defines another command: ```java this.setCommand(new String[]{"postgres", "-c", "fsync=off"}); ``` Are we using the expected configuration when running the tests? ########## standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/dbinstall/rules/PostgresTPCDS.java: ########## @@ -57,6 +70,12 @@ public void install() { String hiveSchemaVer = MetaStoreSchemaInfoFactory.get(new Configuration()).getHiveSchemaVersion(); try (InputStream script = PostgresTPCDS.class.getClassLoader() .getResourceAsStream("sql/postgres/upgrade-3.1.3000-to-" + hiveSchemaVer + ".postgres.sql")) { + // The dump was created from a metastore where some columns had the wrong types. + // Fix those that some tests verify. + String fixColumnTypes = + "UPDATE \"COLUMNS_V2\" c SET \"TYPE_NAME\" = 'char(16)' WHERE \"COLUMN_NAME\" in ('c_customer_id', 'ca_address_id')" + "AND EXISTS (SELECT * FROM \"SDS\" s WHERE s.\"CD_ID\" = c.\"CD_ID\");\n"; + InputStream script2 = + new SequenceInputStream(new ByteArrayInputStream(fixColumnTypes.getBytes(StandardCharsets.UTF_8)), script); Review Comment: We can incorporate this UPDATE statement along with the explanatory comment directly inside the `upgrade-3.1.3000-to-4.3.0.postgres.sql`. In any case it is unlikely to be able to use the same script with different dumps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
