pvary commented on pull request #1789: URL: https://github.com/apache/iceberg/pull/1789#issuecomment-731365732
> @pvary, looks great! Thanks for looking into the archive options. I think we may be able to do this more simply by archiving reports instead of creating new log files. The reports I've used have had stderr/stdout. Thanks for reviewing @rdblue! I have tried archiving reports as a first option. You can see the results here in the artifact: https://github.com/apache/iceberg/runs/1429922302 I found them lacking because of 2 reasons: 1. Hive often does not propagate the exception to the client (the theory is that infra errors should not be handled by the users), so Hive based tests produce the following exceptions without enough details to investigate (INFO logs contain the real Exception): ``` org.apache.iceberg.mr.hive.TestHiveIcebergStorageHandlerWithCustomCatalog > testScanTable[fileFormat=PARQUET, engine=tez] FAILED java.lang.IllegalArgumentException: Failed to execute Hive query 'SELECT * FROM default.customers ORDER BY customer_id DESC': Error while processing statement: FAILED : Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask Caused by: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask ``` 2. Having separate directories for listing the tests for every subproject does not help in identifying the failing test at glance For the 2nd option I have tried the [official](https://github.com/gradle/gradle/blob/v5.4.1/subprojects/docs/src/samples/testing/testReport/groovy/build.gradle) solution for aggregating the results, but it only does the aggregation if there is no test failure which is less than ideal 😄 So that is why I have decided to backtrack to the "have a file for a subproject output and keep the original test output" solution. After my trip to the test output aggregation I feel that if we want to have aggregate test results then it should be done in a different PR and here we should concentrate on the archiving the logs ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
