+1 (non-binding) - Checked signature/SHA512 - Ran RAT license check - Ran tests on JDK17
Thanks, Steve Zhang > On Feb 27, 2025, at 11:45 AM, Daniel Weeks <dwe...@apache.org> wrote: > > +1 (binding) > > Verified sigs/sums/license/build/test (Java 17) > > -Dan > > On Thu, Feb 27, 2025 at 11:30 AM Aihua Xu <aihu...@gmail.com > <mailto:aihu...@gmail.com>> wrote: >> Thanks Alex. >> >> On Thu, Feb 27, 2025 at 10:31 AM Alex Dutra <alex.du...@dremio.com.invalid> >> wrote: >>> Hi Aihua, >>> >>> I was indeed suspecting that you had a custom RESTClient :-) Thanks for >>> digging further and clarifying! Let me know if you need my help in adapting >>> your code. >>> >>> And with that: +1 (non-binding) from me as well. >>> >>> Thanks, >>> >>> Alex >>> >>> On Thu, Feb 27, 2025 at 6:45 PM Aihua Xu <aihu...@gmail.com >>> <mailto:aihu...@gmail.com>> wrote: >>>> Hi Alex, >>>> >>>> I checked our code further. We have an internal implementation for >>>> RESTClient which needs an update after your change to place the token in >>>> DefaultAuthSession and not pass through initHeaders anymore. The existing >>>> code assumes the token coming from initHeaders and we need to make the >>>> change to get the token from the headers of DefaultAuthSession. >>>> >>>> With that, +1 (non-binding) >>>> >>>> Thanks, >>>> Aihua >>>> >>>> >>>> >>>> On Thu, Feb 27, 2025 at 9:26 AM Alex Dutra <alex.du...@dremio.com.invalid> >>>> wrote: >>>>> Hi Aihua Xu, >>>>> >>>>> I reviewed your PR but without further details I do not agree with your >>>>> change, and I am unable to reproduce the issue. Besides, we have unit >>>>> tests that cover this extensively. >>>>> >>>>> Could you please provide a simple reproducer or a test case? >>>>> >>>>> Thanks! >>>>> >>>>> Alex >>>>> >>>>> On Thu, Feb 27, 2025 at 5:06 PM Aihua Xu <aihu...@gmail.com >>>>> <mailto:aihu...@gmail.com>> wrote: >>>>>> I'm running within our integration tests. I'm able to trace and find the >>>>>> breaking change: https://github.com/apache/iceberg/pull/11992. >>>>>> >>>>>> Seems we need to make the following changes to pass the token: >>>>>> https://github.com/apache/iceberg/pull/12415/files. Alex, can you help >>>>>> take a look at this? >>>>>> >>>>>> if (hasCredential) { >>>>>> authResponse = >>>>>> OAuth2Util.fetchToken( >>>>>> initClient, initHeaders, credential, scope, oauth2ServerUri, >>>>>> optionalOAuthParams); >>>>>> Map<String, String> authHeaders = >>>>>> RESTUtil.merge(initHeaders, >>>>>> OAuth2Util.authHeaders(authResponse.token())); >>>>>> config = fetchConfig(initClient, authHeaders, props); >>>>>> } else { >>>>>> authResponse = null; >>>>>> Map<String, String> authHeaders = >>>>>> RESTUtil.merge(initHeaders, OAuth2Util.authHeaders(initToken)); >>>>>> config = fetchConfig(initClient, authHeaders, props); >>>>>> } >>>>>> >>>>>> On Thu, Feb 27, 2025 at 12:46 AM Alex Dutra >>>>>> <alex.du...@dremio.com.invalid> wrote: >>>>>>> Hi Aihua, >>>>>>> >>>>>>> I just tested 1.8.1 with Polaris OSS and I am not seeing anything >>>>>>> different. Can you share your setup? >>>>>>> >>>>>>> Below is my Spark setup. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Alex >>>>>>> >>>>>>> ./gradlew run >>>>>>> >>>>>>> token=$(curl -s http://localhost:8181/api/catalog/v1/oauth/tokens \ >>>>>>> --user root:<secret> \ >>>>>>> -d grant_type=client_credentials \ >>>>>>> -d scope=PRINCIPAL_ROLE:ALL | sed -n >>>>>>> 's/.*"access_token":"\([^"]*\)".*/\1/p') >>>>>>> >>>>>>> curl -s -H "Authorization: Bearer ${token}" \ >>>>>>> -H 'Accept: application/json' \ >>>>>>> -H 'Content-Type: application/json' \ >>>>>>> http://localhost:8181/api/management/v1/catalogs \ >>>>>>> -d '{ >>>>>>> "catalog": { >>>>>>> "name": "polaris_demo", >>>>>>> "type": "INTERNAL", >>>>>>> "readOnly": false, >>>>>>> "properties": { >>>>>>> "default-base-location": "file:///tmp/polaris/" >>>>>>> }, >>>>>>> "storageConfigInfo": { >>>>>>> "storageType": "FILE", >>>>>>> "allowedLocations": [ >>>>>>> "file:///tmp" >>>>>>> ] >>>>>>> } >>>>>>> } >>>>>>> }' >>>>>>> >>>>>>> >>>>>>> spark-sql \ >>>>>>> --packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.8.1 >>>>>>> \ >>>>>>> --conf >>>>>>> spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions >>>>>>> \ >>>>>>> --conf >>>>>>> spark.sql.catalog.polaris=org.apache.iceberg.spark.SparkCatalog \ >>>>>>> --conf >>>>>>> spark.sql.catalog.polaris.catalog-impl=org.apache.iceberg.rest.RESTCatalog >>>>>>> \ >>>>>>> --conf >>>>>>> spark.sql.catalog.polaris.uri=http://127.0.0.1:8181/api/catalog \ >>>>>>> --conf spark.sql.catalog.polaris.credential=root:<secret> \ >>>>>>> --conf spark.sql.catalog.polaris.scope=PRINCIPAL_ROLE:ALL \ >>>>>>> --conf spark.sql.catalog.polaris.warehouse=polaris_demo >>>>>>> >>>>>>> spark-sql (default)> create namespace polaris.foo; >>>>>>> 25/02/27 09:43:24 WARN RESTSessionCatalog: Iceberg REST client is >>>>>>> missing the OAuth2 server URI configuration and defaults to >>>>>>> http://127.0.0.1:8181/api/catalog/v1/oauth/tokens. This automatic >>>>>>> fallback will be removed in a future Iceberg release.It is recommended >>>>>>> to configure the OAuth2 endpoint using the 'oauth2-server-uri' property >>>>>>> to be prepared. This warning will disappear if the OAuth2 endpoint is >>>>>>> explicitly configured. See >>>>>>> https://github.com/apache/iceberg/issues/10537 >>>>>>> Time taken: 0.688 seconds >>>>>>> spark-sql (default)> >>>>>>> >>>>>>> >>>>>>> On Thu, Feb 27, 2025 at 12:22 AM Aihua Xu <aihu...@gmail.com >>>>>>> <mailto:aihu...@gmail.com>> wrote: >>>>>>>> I tested 1.8.1 RC with Snowflake build. I'm seeing the following (I'm >>>>>>>> not seeing that in 1.7.x). >>>>>>>> >>>>>>>> "exception": "java.io.IOException: Authorization header is >>>>>>>> missing\n\tat >>>>>>>> org.apache.polaris.service.dropwizard.auth.PolarisPrincipalAuthenticator.filter(PolarisPrincipalAuthenticator.java:43)\n\tat >>>>>>>> >>>>>>>> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:108)\n\tat >>>>>>>> >>>>>>>> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:44)\n\tat >>>>>>>> >>>>>>>> org.glassfish.jersey.process.internal.Stages.process(Stages.java:173)\n\tat >>>>>>>> >>>>>>>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:248). >>>>>>>> >>>>>>>> Does anyone know which change could introduce this? >>>>>>>> >>>>>>>> On Wed, Feb 26, 2025 at 5:16 AM Péter Váry >>>>>>>> <peter.vary.apa...@gmail.com <mailto:peter.vary.apa...@gmail.com>> >>>>>>>> wrote: >>>>>>>>> +1 >>>>>>>>> >>>>>>>>> checked the signatures, checksums >>>>>>>>> build and run some tests >>>>>>>>> >>>>>>>>> Amogh Jahagirdar <2am...@gmail.com <mailto:2am...@gmail.com>> ezt >>>>>>>>> írta (időpont: 2025. febr. 26., Sze, 6:11): >>>>>>>>>> +1 (binding) >>>>>>>>>> >>>>>>>>>> Verified signatures, checksum, RAT checks. >>>>>>>>>> Ran build and test with JDK17 >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Amogh Jahagirdar >>>>>>>>>> >>>>>>>>>> On Wed, Feb 26, 2025 at 2:30 AM Honah J. <hon...@apache.org >>>>>>>>>> <mailto:hon...@apache.org>> wrote: >>>>>>>>>>> +1 (binding) >>>>>>>>>>> >>>>>>>>>>> - Checked signatures and checksum >>>>>>>>>>> - Checked license >>>>>>>>>>> - Full Build and Test >>>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Honah >>>>>>>>>>> >>>>>>>>>>> On Tue, Feb 25, 2025 at 10:52 AM Russell Spitzer >>>>>>>>>>> <russell.spit...@gmail.com <mailto:russell.spit...@gmail.com>> >>>>>>>>>>> wrote: >>>>>>>>>>>> +1 >>>>>>>>>>>> Checked Sigs and Checksum >>>>>>>>>>>> Ran Rat >>>>>>>>>>>> Ran full build/test >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Feb 25, 2025 at 11:30 AM Driesprong, Fokko >>>>>>>>>>>> <fo...@driesprong.frl> wrote: >>>>>>>>>>>>> +1 (binding) >>>>>>>>>>>>> Checked signatures and checksum >>>>>>>>>>>>> Checked licenses >>>>>>>>>>>>> Spotchecked NOTICE/LICENSE >>>>>>>>>>>>> Kind regards, >>>>>>>>>>>>> Fokko >>>>>>>>>>>>> >>>>>>>>>>>>> Op di 25 feb 2025 om 16:56 schreef Kevin Liu >>>>>>>>>>>>> <kevinjq...@apache.org <mailto:kevinjq...@apache.org>>: >>>>>>>>>>>>>> +1 (non-binding) >>>>>>>>>>>>>> >>>>>>>>>>>>>> I followed "How to Verify a Release" >>>>>>>>>>>>>> <https://iceberg.apache.org/how-to-release/#how-to-verify-a-release>. >>>>>>>>>>>>>> Checked out artifact from SVN, >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> svn checkout >>>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1/ >>>>>>>>>>>>>> . >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> >>>>>>>>>>>>>> Verified >>>>>>>>>>>>>> * Signature Good >>>>>>>>>>>>>> * Checksum Ok >>>>>>>>>>>>>> * RAT check passed. 1 unrelated error message >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> ERROR: Ignored 0 lines in your exclusion files as comments or >>>>>>>>>>>>>> empty lines. >>>>>>>>>>>>>> ``` >>>>>>>>>>>>>> * Build + test passed, running on Java 17.0.6 (openjdk 17.0.6 >>>>>>>>>>>>>> 2023-01-17 LTS) on M1 >>>>>>>>>>>>>> * Ran a few examples on Spark >>>>>>>>>>>>>> * Ran pyiceberg integration tests, >>>>>>>>>>>>>> https://github.com/kevinjqliu/iceberg-python/pull/11 >>>>>>>>>>>>>> >>>>>>>>>>>>>> I ran the tests both with and without the docker daemon. Without >>>>>>>>>>>>>> docker, a few tests failed in `iceberg-aws`, `iceberg-azure`, >>>>>>>>>>>>>> and `iceberg-kafka-connect`. There's already an issue to track >>>>>>>>>>>>>> this at https://github.com/apache/iceberg/issues/12236. >>>>>>>>>>>>>> I'm also continuing to see the flakey test for `iceberg-core`'s >>>>>>>>>>>>>> `testConcurrentFastAppends` test. I believe this is a local >>>>>>>>>>>>>> issue with my machine. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for running the release, Eduard! >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best, >>>>>>>>>>>>>> Kevin Liu >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Feb 25, 2025 at 4:23 AM Jean-Baptiste Onofré >>>>>>>>>>>>>> <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: >>>>>>>>>>>>>>> +1 (non binding) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - Hash and checksum are good >>>>>>>>>>>>>>> - LICENSE and NOTICE are OK in different distributed artifacts >>>>>>>>>>>>>>> (source >>>>>>>>>>>>>>> distribution, aws bundle, etc) >>>>>>>>>>>>>>> - ASF header present in all expected files >>>>>>>>>>>>>>> - No binary file found in the source distribution >>>>>>>>>>>>>>> - Did quick smoke tests >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Regards >>>>>>>>>>>>>>> JB >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Feb 24, 2025 at 1:46 PM Eduard Tudenhoefner >>>>>>>>>>>>>>> <etudenhoef...@gmail.com <mailto:etudenhoef...@gmail.com>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > Hi Everyone, >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > I propose that we release the following RC as the official >>>>>>>>>>>>>>> > Apache Iceberg 1.8.1 release. >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > The commit ID is 9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>>>>>>>> > * This corresponds to the tag: apache-iceberg-1.8.1-rc1 >>>>>>>>>>>>>>> > * >>>>>>>>>>>>>>> > https://github.com/apache/iceberg/commits/apache-iceberg-1.8.1-rc1 >>>>>>>>>>>>>>> > * >>>>>>>>>>>>>>> > https://github.com/apache/iceberg/tree/9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > The release tarball, signature, and checksums are here: >>>>>>>>>>>>>>> > * >>>>>>>>>>>>>>> > https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1 >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > You can find the KEYS file here: >>>>>>>>>>>>>>> > * https://downloads.apache.org/iceberg/KEYS >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > Convenience binary artifacts are staged on Nexus. The Maven >>>>>>>>>>>>>>> > repository URL is: >>>>>>>>>>>>>>> > * >>>>>>>>>>>>>>> > https://repository.apache.org/content/repositories/orgapacheiceberg-1184/ >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > Please download, verify, and test. >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > Please vote in the next 72 hours. >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > [ ] +1 Release this as Apache Iceberg 1.8.1 >>>>>>>>>>>>>>> > [ ] +0 >>>>>>>>>>>>>>> > [ ] -1 Do not release this because... >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > Only PMC members have binding votes, but other community >>>>>>>>>>>>>>> > members are encouraged to cast >>>>>>>>>>>>>>> > non-binding votes. This vote will pass if there are 3 binding >>>>>>>>>>>>>>> > +1 votes and more binding >>>>>>>>>>>>>>> > +1 votes than -1 votes. >>>>>>>>>>>>>>> >
