It was not correctly backported, I do think we want to add this since it fixes a CVE as mentioned earlier. I've created a PR: https://github.com/apache/iceberg/pull/10988
Kind regards, Fokko Op do 22 aug 2024 om 11:35 schreef Jean-Baptiste Onofré <j...@nanthrax.net>: > Hi guys, > > FYI, the reason I mentioned ORC update is because the PR is "flagged" > with milestone 1.6.1. > So it's a bit surprising to not have it in 1.6.1. > > We should at least update the PR/issue removing the 1.6.1 milestone, > else it would not be "accurate". > > Regards > JB > > On Thu, Aug 22, 2024 at 12:04 AM Piotr Findeisen > <piotr.findei...@gmail.com> wrote: > > > > Hi Eduard, > > > > JB wrote > > > >> For the record (maybe it helps users/reviewers), this release includes: > >> - ORC 1.9.4 update > >> - introduce memory limit on ParallelIterable > > > > > > I can confirm ParallelIterable change, but i am not sure whether ORC > update was part of the release. > > > > > > Best > > Piotr > > > > > > > > > > On Wed, 21 Aug 2024 at 09:45, Fokko Driesprong <fo...@apache.org> wrote: > >> > >> Hey Eduard, > >> > >> I think it relates to this PR. It contains a CVE and would be good to > be backported. We wanted to include it in 1.6.1 if we needed another RC, > but that didn't happen, so I think we didn't cherry-pick it to 1.6.x branch. > >> > >> Kind regards, > >> Fokko > >> > >> Op wo 21 aug 2024 om 09:34 schreef Eduard Tudenhöfner < > etudenhoef...@apache.org>: > >>> > >>> @Piotr can you please elaborate which ORC update you are referring to? > Or did you mean the Avro update (which I think we were planning for 1.6.2)? > >>> > >>> On Tue, Aug 20, 2024 at 7:05 PM Piotr Findeisen < > piotr.findei...@gmail.com> wrote: > >>>> > >>>> Hi > >>>> > >>>> -1 (non-binding) > >>>> > >>>> I verified source tarball matches the git tag (except it lacks > jitpack.yml, docs/ and 'examples/Convert table to Iceberg.ipynb'). > >>>> However, i noted that source tarball verification is not part of > https://iceberg.apache.org/how-to-release/#validating-a-source-release-candidate > . > >>>> I started a separate dev list thread about this ( > https://lists.apache.org/thread/24c0xhfbb2680nrqyd2jrngxtg6qoz8c). > >>>> > >>>> as to the changes, it looks like it contains the ParallelIterable > change, but I don't see ORC update > >>>> > >>>> $ git diff apache-iceberg-1.6.0..apache-iceberg-1.6.1-rc1 --numstat > >>>> 167 55 > core/src/main/java/org/apache/iceberg/util/ParallelIterable.java > >>>> 48 0 > core/src/test/java/org/apache/iceberg/util/TestParallelIterable.java > >>>> > >>>> I tested with Trino https://github.com/trinodb/trino/pull/23083 > >>>> The parallel change iterable caused a regression in Trino when > planning queries with LIMIT. > >>>> Now the query scheduler will open more manifests than it used to > (test io.trino.plugin.iceberg.TestIcebergFileOperations#testSelectWithLimit > in Trino) > >>>> Reverting the change around queue low water mark [1][2] solved the > test for me locally. > >>>> > >>>> Best, > >>>> Piotr > >>>> > >>>> [1] https://github.com/apache/iceberg/pull/10978 > >>>> [2] https://github.com/apache/iceberg/pull/10979 > >>>> > >>>> > >>>> > >>>> On Tue, 20 Aug 2024 at 15:31, Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >>>>> > >>>>> +1 (non binding) > >>>>> > >>>>> I checked: > >>>>> - download links are OK (both on dist and Maven Staging repo) > >>>>> - build passed on the tag using JDK11, including the tests (I'm not > >>>>> able to reproduce Renjie's issue) > >>>>> - checksum and signature are good > >>>>> - ASF header present in expected files > >>>>> - no unexpected binary files found in the source distribution > >>>>> > >>>>> For the record (maybe it helps users/reviewers), this release > includes: > >>>>> - ORC 1.9.4 update > >>>>> - introduce memory limit on ParallelIterable > >>>>> > >>>>> Regards > >>>>> JB > >>>>> > >>>>> > >>>>> On Tue, Aug 20, 2024 at 4:53 AM Carl Steinbach <c...@apache.org> > wrote: > >>>>> > > >>>>> > Hi Everyone, > >>>>> > > >>>>> > I propose that we release the following RC as the official Apache > Iceberg 1.6.1 release. > >>>>> > > >>>>> > The commit ID is e18a2fe10214f5f3ffa0a317a28af8b2a619817a > >>>>> > * This corresponds to the tag: apache-iceberg-1.6.1-rc1 > >>>>> > * > https://github.com/apache/iceberg/commits/apache-iceberg-1.6.1-rc1 > >>>>> > * > https://github.com/apache/iceberg/tree/e18a2fe10214f5f3ffa0a317a28af8b2a619817a > >>>>> > > >>>>> > The release tarball, signature, and checksums are here: > >>>>> > * > https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.6.1-rc1 > >>>>> > > >>>>> > You can find the KEYS file here: > >>>>> > * https://dist.apache.org/repos/dist/dev/iceberg/KEYS > >>>>> > > >>>>> > Convenience binary artifacts are staged on Nexus. The Maven > repository URL is: > >>>>> > * > https://repository.apache.org/content/repositories/orgapacheiceberg-1170/ > >>>>> > > >>>>> > Please download, verify, and test. > >>>>> > > >>>>> > Please vote in the next 72 hours. > >>>>> > > >>>>> > [ ] +1 Release this as Apache Iceberg 1.6.1 > >>>>> > [ ] +0 > >>>>> > [ ] -1 Do not release this because... > >>>>> > > >>>>> > Only PMC members have binding votes, but other community members > are encouraged to cast > >>>>> > non-binding votes. This vote will pass if there are 3 binding +1 > votes and more binding > >>>>> > +1 votes than -1 votes. >
