One note, when running TPC-DS I noticed a log message about decimal pushdown that wasn't present in 1.3 and earlier, e.g. "Failed to check if i_current_price >= 0.99 can be pushed down: Cannot create expression literal from org.apache.spark.sql.types.Decimal: 0.99", though not sure if that is a regression or just a new warning.
> On Sep 28, 2023, at 1:46 PM, Bryan Keller <brya...@gmail.com> wrote: > > +1 (non-binding) > > I ran the TPC-DS 3 TB benchmark with AWS EMR 6.13/Spark 3.4.1, Iceberg 1.4.0 > RC1, and a REST catalog. The results were slightly slower than 1.3.1 but > within the margin of error. > > 1.3.1 load: 1671 sec > 1.4.0 load: 1820 sec > 1.3.1 query: 4961 sec > 1.4.0 query: 5210 sec > > On Thu, Sep 28, 2023 at 10:09 AM Anton Okolnychyi <aokolnyc...@apache.org > <mailto:aokolnyc...@apache.org>> wrote: >> I'll start with +1 (binding). >> >> Signature: OK >> Checksum: OK >> RAT checks: OK >> >> I also tested our new write defaults in Spark 3.5 on a cluster. No local >> sort, no ridiculously tiny files. We still prefer parallelism by default so >> folks would have to manually tune the advisory partition size for the final >> write to get closer to the target file size (if that's the goal). We will >> offer some knobs to do that automatically in the future. >> >> - Anton >> >> On 2023/09/28 02:02:50 Anton Okolnychyi wrote: >> > Hi Everyone, >> > >> > I propose that we release the following RC as the official Apache Iceberg >> > 1.4.0 release. >> > >> > The commit ID is 8f37faa6a21e863551b17992370edc0f8706465d >> > * This corresponds to the tag: apache-iceberg-1.4.0-rc1 >> > * https://github.com/apache/iceberg/commits/apache-iceberg-1.4.0-rc1 >> > * >> > https://github.com/apache/iceberg/tree/8f37faa6a21e863551b17992370edc0f8706465d >> > >> > The release tarball, signature, and checksums are here: >> > * https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.4.0-rc1 >> > >> > You can find the KEYS file here: >> > * https://dist.apache.org/repos/dist/dev/iceberg/KEYS >> > >> > Convenience binary artifacts are staged on Nexus. The Maven repository URL >> > is: >> > * https://repository.apache.org/content/repositories/orgapacheiceberg-1145/ >> > >> > Please download, verify, and test. >> > >> > Please vote in the next 72 hours. (Weekends excluded) >> > >> > [ ] +1 Release this as Apache Iceberg 1.4.0 >> > [ ] +0 >> > [ ] -1 Do not release this because... >> > >> > Only PMC members have binding votes, but other community members are >> > encouraged to cast non-binding votes. This vote will pass if there are 3 >> > binding +1 votes and more binding +1 votes than -1 votes. >> > >> > - Anton
