nastra edited a comment on issue #3157: URL: https://github.com/apache/iceberg/issues/3157#issuecomment-928928939
> @nastra, can we avoid using Spark 2.4 for this and run the benchmarks in 3.1? I have no idea what might have changed in 2.4.6 that caused this. @rdblue the goal is definitely to run those benchmarks for Spark3 but I was rather trying to figure out when `VectorizedReadDictionaryEncodedFlatParquetDataBenchmark` started to take so long for setup (which was 2020-08-06 with 996ed97 when it wasn't possible to run that benchmark with Spark3). The same benchmark for Spark3 was only recently (2021-06-21) introduced with ed0c70284e1f6506331a60d8c9e9706df2e32ec9, meaning that `VectorizedReadDictionaryEncodedFlatParquetDataBenchmark` actually never worked with Spark3 in the first place and setup always took so long. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
