Re: [DISCUSS] Supporting region attribute in vended credentials for AWS

[rdb...@gmail.com]

Thanks for finding this, Aniket. It sounds like a good thing to fix in the
spec to me.

This also brings up a question for the Polaris community. JB said he would
“draft a proposal to update the Iceberg REST Spec *as well*“. Does the
Polaris community intend to maintain a separate REST protocol and only
sometimes upstream changes? I think it would be good to have clear guidance
on this so that it is clear when an update should be taken upstream first
vs decided here and later standardized.

Ryan

On Wed, Sep 11, 2024 at 7:34 AM Jean-Baptiste Onofré 
wrote:

> Hi Aniket
>
> It looks good to me. I think AwsStorageConfigurationInfo is not
> enough, we also need the client.region in the Iceberg REST Spec (for
> consistency between engines).
>
> If there's no objection, I would draft a proposal to update the
> Iceberg REST Spec as well.
> As a workaround, we can store the client.region as property in the
> Polaris entity (e.g. table).
>
> Thoughts ?
>
> Regards
> JB
>
> On Wed, Sep 11, 2024 at 4:31 AM Aniket Kulkarni
>  wrote:
> >
> > For iceberg tables stored in AWS S3 buckets, knowing the region of the
> bucket is critical for engines using vended credentials (when configured)
> to access a table.
> >
> > E.g - the vended credentials for AWS look like this
> >
> > { "s3.access-key-id": "ASI”,
> >   "s3.secret-access-key": "gbVT9PpFBY...”,
> >   "s3.session-token": "IQoJb3JpZ2luX2VjEN3//...”,
> >"expiration-time": “1725572949000” }
> >
> > An engine consuming this, would need to either infer (s3api
> get-bucket-location) the region or ask the end user to provide the region
> separately which misses the point of vended credentials.
> >
> > A engine engine cannot use get-bucket-location, because the credential
> generation explicitly allows only s3:GetObject, s3:GetObjectVersion,
> s3:PutObject, s3:DeletObject, s3:ListBucket for the table location prefix.
> Refer -
> org.apache.polaris.core.storage.aws.AwsCredentialsStorageIntegration#policyString
> >
> > I propose that
> >
> > - the storage setup for S3 should have parameter for the bucket region
> (org.apache.polaris.core.storage.aws.AwsStorageConfigurationInfo)
> > - if the parameter is not specified, then Polaris attempts to look up
> (get-bucket-location) the region.
> > - the information is returned in vended credentials (if enabled) as
> "s3.region”:…
> >
> > Note - another option could be to allow ’s3:GetBucketLocation’ in the
> policyString when generating vended credentials’ iam role, but that is sub
> optimal and therefore I am not proposing it. It would engines to make
> multiple get-bucket-location calls - one per table being looked up.
> >
> > --
> > aniket
> >
>

Re: [DISCUSS] Supporting region attribute in vended credentials for AWS

[rdb...@gmail.com]

Polaris is not intended to be a fork of the Iceberg REST spec, but an
implementation.

About your point Ryan, my “view” on that is that Polaris should follow the
REST spec, and the REST spec is at Iceberg.

Thanks! I thought that was the case, but I wanted to make sure early on. I
think we all want to avoid drift in the specs and also ensure the client
maintained in the Iceberg project is fully compatible.

On Wed, Sep 11, 2024 at 10:41 AM Yufei Gu  wrote:

> >
> > - the storage setup for S3 should have parameter for the bucket region
> > (org.apache.polaris.core.storage.aws.AwsStorageConfigurationInfo)
> > - if the parameter is not specified, then Polaris attempts to look up
> > (get-bucket-location) the region.
>
>
> I'd always prefer the second one. It provides more flexibility. For
> example, it's likely that the regions are different for two namespaces
> under the same catalog. Polaris may not support namespace-level or
> table-level storage location well at this point, but it makes sense to do
> so. This seems like a big topic. I will not expand here, we can discuss
> separately.
>
> Plus, the assume-role configured for catalog should have permission to
> infer the region.
>
> Yufei
>
>
> On Wed, Sep 11, 2024 at 10:03 AM Yufei Gu  wrote:
>
> > Sorry, just realized that Edward has a new proposal to standardize
> > credentials, https://github.com/apache/iceberg/pull/10722. In that case,
> > we should add client.region as a field of AWS credential schema. I will
> > comment on it in the PR.
> >
> > > Does the Polaris community intend to maintain a separate REST protocol
> > and only sometimes upstream changes?
> >
> > I'm with JB for not diverging from the Iceberg REST spec. This is mainly
> > for consistency, and to ensure interoperability across engines.
> >
> > Yufei
> >
> >
> > On Wed, Sep 11, 2024 at 9:00 AM Jean-Baptiste Onofré 
> > wrote:
> >
> >> About your point Ryan, my "view" on that is that Polaris should follow
> >> the REST spec, and the REST spec is at Iceberg.
> >>
> >> I don't think it would be a good idea to have a separate REST protocol
> >> in Polaris, especially for engine interoperability about Iceberg
> >> (engines should know only the Iceberg REST Spec/Client).
> >> So, I'm more in favor of proposing directly REST spec changes at
> Iceberg.
> >>
> >> Thoughts ?
> >>
> >> Regards
> >> JB
> >>
> >> On Wed, Sep 11, 2024 at 5:29 PM rdb...@gmail.com 
> >> wrote:
> >> >
> >> > Thanks for finding this, Aniket. It sounds like a good thing to fix in
> >> the
> >> > spec to me.
> >> >
> >> > This also brings up a question for the Polaris community. JB said he
> >> would
> >> > “draft a proposal to update the Iceberg REST Spec *as well*“. Does the
> >> > Polaris community intend to maintain a separate REST protocol and only
> >> > sometimes upstream changes? I think it would be good to have clear
> >> guidance
> >> > on this so that it is clear when an update should be taken upstream
> >> first
> >> > vs decided here and later standardized.
> >> >
> >> > Ryan
> >> >
> >> > On Wed, Sep 11, 2024 at 7:34 AM Jean-Baptiste Onofré  >
> >> > wrote:
> >> >
> >> > > Hi Aniket
> >> > >
> >> > > It looks good to me. I think AwsStorageConfigurationInfo is not
> >> > > enough, we also need the client.region in the Iceberg REST Spec (for
> >> > > consistency between engines).
> >> > >
> >> > > If there's no objection, I would draft a proposal to update the
> >> > > Iceberg REST Spec as well.
> >> > > As a workaround, we can store the client.region as property in the
> >> > > Polaris entity (e.g. table).
> >> > >
> >> > > Thoughts ?
> >> > >
> >> > > Regards
> >> > > JB
> >> > >
> >> > > On Wed, Sep 11, 2024 at 4:31 AM Aniket Kulkarni
> >> > >  wrote:
> >> > > >
> >> > > > For iceberg tables stored in AWS S3 buckets, knowing the region of
> >> the
> >> > > bucket is critical for engines using vended credentials (when
> >> configured)
> >> > > to access a table.
> >> > > >
> >> > > > E.g - the vended credentials for AWS look like

Re: Podling Polaris Report Reminder - October 2024

[rdb...@gmail.com]

JB, there's one linked in the wiki here:
https://cwiki.apache.org/confluence/display/INCUBATOR/September24#polaris

I think the link that went out was bad. I also had trouble accessing it.

On Mon, Oct 7, 2024 at 2:17 PM Jean-Baptiste Onofré  wrote:

> Hi folks,
>
> I already prepared the report using last month page:
> https://cwiki.apache.org/confluence/display/INCUBATOR/September24
>
> As soon as the October24 page will be created, I will complete the
> Polaris report.
>
> Can you please take a look and complete if I missed anything?
>
> Thanks !
> Regards
> JB
>
> On Mon, Oct 7, 2024 at 12:07 PM  wrote:
> >
> > Dear podling,
> >
> > This email was sent by an automated system on behalf of the Apache
> > Incubator PMC. It is an initial reminder to give you plenty of time to
> > prepare your quarterly board report.
> >
> > The board meeting is scheduled for Wed, 16 October 2024.
> > The report for your podling will form a part of the Incubator PMC
> > report. The Incubator PMC requires your report to be submitted 2 weeks
> > before the board meeting, to allow sufficient time for review and
> > submission (Wed, October 02).
> >
> > Please submit your report with sufficient time to allow the Incubator
> > PMC, and subsequently board members to review and digest. Again, the
> > very latest you should submit your report is 2 weeks prior to the board
> > meeting.
> >
> > Candidate names should not be made public before people are actually
> > elected, so please do not include the names of potential committers or
> > PPMC members in your report.
> >
> > Thanks,
> >
> > The Apache Incubator PMC
> >
> > Submitting your Report
> >
> > --
> >
> > Your report should contain the following:
> >
> > *   Your project name
> > *   A brief description of your project, which assumes no knowledge of
> > the project or necessarily of its field
> > *   A list of the three most important issues to address in the move
> > towards graduation.
> > *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> > aware of
> > *   How has the community developed since the last report
> > *   How has the project developed since the last report.
> > *   How does the podling rate their own maturity.
> >
> > This should be appended to the Incubator Wiki page at:
> >
> > https://cwiki.apache.org/confluence/display/INCUBATOR/October2024
> >
> > Note: This is manually populated. You may need to wait a little before
> > this page is created from a template.
> >
> > Note: The format of the report has changed to use markdown.
> >
> > Mentors
> > ---
> >
> > Mentors should review reports for their project(s) and sign them off on
> > the Incubator wiki page. Signing off reports shows that you are
> > following the project - projects that are not signed may raise alarms
> > for the Incubator PMC.
> >
> > Incubator PMC
>

Re: Podling Polaris Report Reminder - February 2025

[rdb...@gmail.com]

I looked through the report and left a few comments. The report says that
this is the first report, but there is one from September/October last
year:
https://cwiki.apache.org/confluence/display/INCUBATOR/September2024#polaris

My main comment is that I'd like to see these reports focus on community
development and growth over the project updates. Project updates are great
and it's nice to see them, but the purpose of incubation is to help grow
and mature the community.

The report isn't very specific about community growth. It mentions "more
interest on Slack", "good growth", and "more contributors" but those are
interpretations. It would be nice to include the underlying data when this
is what you're reporting on. That said, community growth is not just
contributor stats. Growth and maturity includes how decisions are made and
how the community builds consensus around new ideas. I think one of the
most positive things I've seen is that the community is effectively using
the dev list for proposals and circulating design docs. I would encourage
you all to highlight this in this report or the next.

On Wed, Feb 5, 2025 at 11:20 AM Dmitri Bourlatchkov 
wrote:

> LGTM. Thanks, JB!
>
> Cheers,
> Dmitri.
>
> On Wed, Feb 5, 2025 at 11:58 AM Jean-Baptiste Onofré 
> wrote:
>
> > Hi folks,
> >
> > I drafted a first incubator report:
> >
> https://cwiki.apache.org/confluence/display/INCUBATOR/February2025#polaris
> >
> > Can you please take a look and let me know if you wanna add content ?
> > You can directly update the report (if you have the permissions).
> >
> > Thanks !
> > Regards
> > JB
> >
> > On Tue, Feb 4, 2025 at 6:27 AM Jean-Baptiste Onofré 
> > wrote:
> > >
> > > Hi folks
> > >
> > > I will draft a report and share with you for review.
> > >
> > > Thanks
> > > Regards
> > > JB
> > >
> > > Le mar. 4 févr. 2025 à 00:25,  a écrit :
> > >>
> > >> Dear podling,
> > >>
> > >> This email was sent by an automated system on behalf of the Apache
> > >> Incubator PMC. It is an initial reminder to give you plenty of time to
> > >> prepare your quarterly board report.
> > >>
> > >> The board meeting is scheduled for Wed, 19 February 2025.
> > >> The report for your podling will form a part of the Incubator PMC
> > >> report. The Incubator PMC requires your report to be submitted 2 weeks
> > >> before the board meeting, to allow sufficient time for review and
> > >> submission (Wed, February 05).
> > >>
> > >> Please submit your report with sufficient time to allow the Incubator
> > >> PMC, and subsequently board members to review and digest. Again, the
> > >> very latest you should submit your report is 2 weeks prior to the
> board
> > >> meeting.
> > >>
> > >> Candidate names should not be made public before people are actually
> > >> elected, so please do not include the names of potential committers or
> > >> PPMC members in your report.
> > >>
> > >> Thanks,
> > >>
> > >> The Apache Incubator PMC
> > >>
> > >> Submitting your Report
> > >>
> > >> --
> > >>
> > >> Your report should contain the following:
> > >>
> > >> *   Your project name
> > >> *   A brief description of your project, which assumes no knowledge of
> > >> the project or necessarily of its field
> > >> *   A list of the three most important issues to address in the move
> > >> towards graduation.
> > >> *   Any issues that the Incubator PMC or ASF Board might wish/need to
> be
> > >> aware of
> > >> *   How has the community developed since the last report
> > >> *   How has the project developed since the last report.
> > >> *   How does the podling rate their own maturity.
> > >>
> > >> This should be appended to the Incubator Wiki page at:
> > >>
> > >> https://cwiki.apache.org/confluence/display/INCUBATOR/February2025
> > >>
> > >> Note: This is manually populated. You may need to wait a little before
> > >> this page is created from a template.
> > >>
> > >> Note: The format of the report has changed to use markdown.
> > >>
> > >> Mentors
> > >> ---
> > >>
> > >> Mentors should review reports for their project(s) and sign them off
> on
> > >> the Incubator wiki page. Signing off reports shows that you are
> > >> following the project - projects that are not signed may raise alarms
> > >> for the Incubator PMC.
> > >>
> > >> Incubator PMC
> >
>

Re: [VOTE] Release Apache Polaris 0.9.0-incubating (rc5)

[rdb...@gmail.com]

+1

It’s great to see the progress!

I did find a couple of minor things:

   - RAT checks exclude all **.md files, which I don’t think is correct.
   Luckily, commenting out that line and running RAT shows that all of the
   markdown files have good license headers. Because the markdown has headers,
   I don’t think this is a blocker.
   - The NOTICE file includes the entire Netty notice via the Nessie
   NOTICE, but the only content from Nessie are build files so this almost
   certainly doesn’t apply. It would be nice to clarify this and remove it so
   that downstream consumers don’t need to worry about it.

Other checks:

   - Validated signature and checksum
   - Ran RAT check and validated exclusions
   - Checked LICENSE and NOTICE changes


On Thu, Feb 6, 2025 at 8:55 AM Jean-Baptiste Onofré  wrote:

> Hi folks,
>
> We did new fixes on NOTICE and LICENSE fix in the source distribution.
>
> This is the vote for Apache Polaris 0.9.0-incubating rc5.
>
> * This corresponds to the tag: apache-polaris-0.9.0-incubating-rc5
> *
> https://github.com/apache/polaris/tree/apache-polaris-0.9.0-incubating-rc5
> *
> https://github.com/apache/polaris/tree/829dff6ec4728bf70e8a772c42bccb519679574e
>
> The release source tarball, signature, and checksums are here:
> *
> https://dist.apache.org/repos/dist/dev/incubator/polaris/0.9.0-incubating/
>
> You can find the KEYS file here:
> * https://dist.apache.org/repos/dist/release/incubator/polaris/KEYS
>
> NB: this release doesn't provide any binary distribution, it provides
> source distribution only.
>
> Please download, verify, and test.
>
> Please vote in the next 72 hours.
> [ ] +1 Release this as Apache polaris 0.9.0-incubating
> [ ] +0
> [ ] -1 Do not release this because...
>
> Only PPMC members and mentors have binding votes, but other community
> members are encouraged to cast non-binding votes. This vote will pass
> if there are
> 3 binding +1 votes and more binding +1 votes than -1 votes.
>
> NB: if this vote passes, a new vote will be started on the Incubator
> general mailing list.
>
> Thanks
> Regards
> JB
>

Re: [VOTE] Release Apache Polaris 0.9.0-incubating

[rdb...@gmail.com]

-1 (binding)

I don't think that the license documentation is sufficient. It looks like
the LICENSE file for the source tarball was auto-generated from
dependencies and doesn't document the difference between dependencies and
sources that are included. There is at least one library, jersey-json, that
is dual licensed with a category B license (CDDL 1.1) and a category X
license (GPL v2). I don't _think_ that code from that project is included,
but nothing tells me that and if there were code included in the source
tarball, this would not be releasable under [ASF policy](
https://www.apache.org/legal/resolved.html).

The NOTICE file is also incomplete, most likely because the documentation
is auto-generated. There are clear notices in the text of LICENSE, which is
what the NOTICE file is intended for (so downstream projects have one place
to look for legally required notices). That file must also be minimal, so I
think it needs to be researched and curated rather than generated.

Ryan


On Thu, Nov 21, 2024 at 5:22 AM Jean-Baptiste Onofré 
wrote:

> Good call Dmitri.
>
> I agree with this plan.
>
> Is everyone happy with that ?
>
> Regards
> JB
>
> On Thu, Nov 21, 2024 at 12:46 AM Dmitri Bourlatchkov
>  wrote:
> >
> > I believe the 0.9.0 release was meant to be a trial release to test the
> process. With that in mind, in-memory polaris still works fine in 0.9.0. I
> suppose we can finish the release and fix issues on main for 0.9.1. This
> way we get to test how smoothly patch releases work too.
> >
> > ... just my 2 cents :)
> >
> > Cheers,
> > Dmitri.
> >
> > On Wed, Nov 20, 2024 at 4:53 PM Jean-Baptiste Onofré 
> wrote:
> >>
> >> Just to be clear, my preference would be to revert #438 on 0.9 branch.
> >>
> >> Regards
> >> JB
> >>
> >> On Wed, Nov 20, 2024 at 10:51 PM Jean-Baptiste Onofré 
> wrote:
> >> >
> >> > Hi Dmitri
> >> >
> >> > Good catch.
> >> >
> >> > I think it's severe enough to cancel rc1 and prepare a new one
> >> > including eclipselink fix.
> >> >
> >> > Thoughts ?
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On Wed, Nov 20, 2024 at 9:21 PM Dmitri Bourlatchkov
> >> >  wrote:
> >> > >
> >> > > I've just noticed that RC1 includes
> >> > > https://github.com/apache/polaris/pull/438, which, I think, makes
> >> > > bootstrapping on EclispeLink effectively impossible, because
> there's no
> >> > > user-level way to discover the generated root secret.
> >> > >
> >> > > From my personal point of view it is not a release blocker, so I'm
> keeping
> >> > > my previous +1 vote, but I wanted to raise this concern in case it
> is
> >> > > important for other people.
> >> > >
> >> > > Cheers,
> >> > > Dmitri.
> >> > >
> >> > >
> >> > > On Tue, Nov 19, 2024 at 11:37 AM Jean-Baptiste Onofré <
> j...@nanthrax.net>
> >> > > wrote:
> >> > >
> >> > > > Actually, reverting my vote to +1 (binding)
> >> > > >
> >> > > > NB
> >> > > > - .keep and .typed are actually empty files (even if flagged as
> >> > > > binary), so not a problem
> >> > > > - I will fix all gradle related issue during release process and
> >> > > > documentation (already working on fix about digest sourceTarball
> and
> >> > > > Maven publication)
> >> > > >
> >> > > > Regards
> >> > > > JB
> >> > > >
> >> > > > On Tue, Nov 19, 2024 at 6:04 AM Jean-Baptiste Onofré <
> j...@nanthrax.net>
> >> > > > wrote:
> >> > > > >
> >> > > > > -1 (binding)
> >> > > > >
> >> > > > > I found that the source distribution includes binaries,
> especially
> >> > > > > gradlew, .keep, *.typed, it has to excluded from the source
> >> > > > > distribution
> >> > > > >
> >> > > > > I will cancel this vote to fix that in rc2.
> >> > > > >
> >> > > > > Regards
> >> > > > > JB
> >> > > > >
> >> > > > > On Sun, Nov 17, 2024 at 7:00 AM Jean-Baptiste Onofré <
> j...@nanthrax.net>
> >> > > > wrote:
> >> > > > > >
> >> > > > > > Hi everyone,
> >> > > > > >
> >> > > > > > I propose that we release the following RC as the official
> Apache
> >> > > > > > Polaris 0.9.0-incubating release.
> >> > > > > >
> >> > > > > > * This corresponds to the tag:
> apache-polaris-0.9.0-incubating-rc1
> >> > > > > > *
> >> > > >
> https://github.com/apache/polaris/commits/apache-polaris-0.9.0-incubating-rc1
> >> > > > > > *
> >> > > >
> https://github.com/apache/polaris/tree/445c42768d1e3148c912ac5c45ee53036b9ef318
> >> > > > > >
> >> > > > > > The release tarball, signature, and checksums are here:
> >> > > > > > *
> >> > > >
> https://dist.apache.org/repos/dist/dev/incubator/polaris/0.9.0-incubating/
> >> > > > > >
> >> > > > > > You can find the KEYS file here:
> >> > > > > > *
> https://dist.apache.org/repos/dist/release/incubator/polaris/KEYS
> >> > > > > >
> >> > > > > > Convenience binary artifacts are staged on Nexus. The Maven
> repository
> >> > > > URL is:
> >> > > > > > *
> >> > > >
> https://repository.apache.org/content/repositories/orgapachepolaris-1003/
> >> > > > > >
> >> > > > > > Please download, verify, and test.
> >> > > > > >
> >> > > > > > Please vote in the next 72 ho

Re: [VOTE] Release Apache Polaris 0.9.0-incubating (rc4)

[rdb...@gmail.com]

-1 because I think I’ve found copied source code that is not documented in
the LICENSE file. I grepped for [Cc]opied and found that there are several
mustache files that were copied from openapi-generator that are not
documented in LICENSE. The openapi-generator project has an ALv2 LICENSE
file with the following copyright:

Copyright 2018 OpenAPI-Generator Contributors (
https://openapi-generator.tech) Copyright 2018 SmartBear Software

I didn’t find any other copied code, but I would encourage others to grep
for phrases like “copied” to help us catch and document these.

I also ran across the site folder that contains the Hugo site. I don’t
think it is a problem to include the site, but I would generally recommend
preventing other content in the repo from being included in the tarball.
That makes the source tarball smaller (i.e. not sending blog posts and
images in it) and also avoids potential problems in the future, where
unrelated content (like build directories with third-party code) is
included in the ASF release. In Iceberg, we limit what is in the tarball
using git archive --worktree-attributes and a .gitattributes file that
ignores certain patterns.

Otherwise the release looks good. I checked:

   - Validated signature and checksum
   - Ran RAT checks
   - Checked the disclaimer
   - Checked that the binary license issues are no longer present (thanks,
   JB!)
   - Checked NOTICE content against entries in LICENSE (I don’t see
   anything to carry over from gradle)
   - Checked LICENSE updates about gradle (looks good)


On Mon, Jan 27, 2025 at 2:07 PM Russell Spitzer 
wrote:

> +1 (binding)
>
> Verified
> Checksum
> Signatures - Note that me and JB need to verify each other's keys next time
> we meet in person :)
> Build and Test
> Used runApp and poked to make sure nothing was broken
>
> On Thu, Jan 23, 2025 at 6:41 PM Dmitri Bourlatchkov 
> wrote:
>
> > +1 (ns)
> >
> > Verified:
> > * Checksum
> > * Signarures
> > * ./gradlew clean assemble
> > * ./gradlew test
> > * Start with in-memory MetaStore
> > * Create catalog via ./polaris CLI with FILE storage
> > * Basic  create table / select in Spark
> >
> > Cheers,
> > Dmitri.
> >
> > On Wed, Jan 22, 2025 at 3:51 AM Jean-Baptiste Onofré 
> > wrote:
> >
> > > Hi folks,
> > >
> > > Following the NOTICE/LICENSE fix and the issue found on
> > > TestPolarisVersion in rc3, we fixed the test compilation issue.
> > >
> > > This is the vote for Apache Polaris 0.9.0-incubating rc4.
> > >
> > > * This corresponds to the tag: apache-polaris-0.9.0-incubating-rc4
> > > *
> > >
> >
> https://github.com/apache/polaris/tree/apache-polaris-0.9.0-incubating-rc4
> > > *
> > >
> >
> https://github.com/apache/polaris/tree/37936f5157b37b8b7db2b8b1ebb86d9d067a353e
> > >
> > > The release source tarball, signature, and checksums are here:
> > > *
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/polaris/0.9.0-incubating/
> > >
> > > You can find the KEYS file here:
> > > * https://dist.apache.org/repos/dist/release/incubator/polaris/KEYS
> > >
> > > NB: this release doesn't provide any binary distribution, it provides
> > > source distribution only.
> > >
> > > Please download, verify, and test.
> > >
> > > Please vote in the next 72 hours.
> > > [ ] +1 Release this as Apache polaris 0.9.0-incubating
> > > [ ] +0
> > > [ ] -1 Do not release this because...
> > >
> > > Only PPMC members and mentors have binding votes, but other community
> > > members are encouraged to cast non-binding votes. This vote will pass
> > > if there are
> > > 3 binding +1 votes and more binding +1 votes than -1 votes.
> > >
> > > NB: if this vote passes, a new vote will be started on the Incubator
> > > general mailing list.
> > >
> > > Thanks
> > > Regards
> > > JB
> > >
> >
>

Re: [VOTE] Release Apache Polaris 0.9.0-incubating (rc2)

[rdb...@gmail.com]

I’m still -1 on this release due to licensing concerns.

The LICENSE file includes a blanket statement that there are third-party
components that are licensed under the Apache Software License 2.0, but
doesn’t list what they are. I think this needs to be specific.

The NOTICE file has a copyright notice for DropWizard that I would expect
to be in LICENSE. The third-party policy states that for third-party
notices:

Apache releases should contain a copy of each license, usually contained in
the LICENSE document. For many licenses this is a sufficient notice. Some
licenses require some additional notice. In many cases, you can include
this notice within the dependent artifact.

For the DropWizard content, I’d also expect to see documentation of what
was copied into the Polaris source tree. There are similar notices for ASF
projects, which would be nice to document in the LICENSE file, but aren’t
strictly necessary.

The binary license file includes this:

Apache Polaris distributions contain some or all of the following
dependencies

I don’t think this is adequate. Each binary artifact should document the
third-party code that it includes, the license under which it is included,
and no other license text (see “How should I handle a work when there is a
choice of license?”
). As it is
right now, there are copies of the GPL and that can create a lot of concern
— doing the work to show that all components use a Category A or B license
is super important for downstream consumers. In addition, it isn’t
sufficient to say that a third-party Category B project might be included.
It needs to be clear for each artifact what exactly is included; this will
also help with the issues below which may not actually apply to artifacts
because dependencies are provided at runtime rather than bundled.

There should also be a NOTICE for each binary artifact. And given the other
issues with the binary license (see below), I’m not confident that there is
not additional work to be done to compile the NOTICE.

It’s also a good practice to link to the license text rather than include
it when it is generic, like the CDDL. When the license embeds authorship
information (such as “Neither the name of Company Inc. nor the names of its
contributors …”) I think it’s fine to include.

I recommend a bit more formatting to make the text more clear. For example,
the jakarta.activation section has confusing sub-sections that state that
the license identifier is BSD-3-clause but just above it says it is
EDLv1.0. It would be better to show that this entire section was copied
from the other project. (This looks like a common problem.)

The binary license also includes a few issues:

Sax (0.2)

   - License: SAX-PD
   - Project: http://www.megginson.com/downloads/SAX/
   - Source: http://sourceforge.net/project/showfiles.php?group_id=29449

I’m not sure what the SAX-PD license is and what category it falls under.

wagon-http-lightweight (3.0.0)

   - License: Pending
   - Project: https://maven.apache.org/wagon/
   - Source:

   
https://mvnrepository.com/artifact/org.apache.maven.wagon/wagon-http-lightweight/3.0.0

This needs to be clarified.

dom4j (1.6.1)

   - License: Custom license based on Apache 1.1

Is this custom license compatible?

jakarta.xml.bind-api has this in its third-party section:

JTHarness (5.0)

   - License: (GPL-2.0 OR GPL-2.0 WITH Classpath-exception-2.0)
   - Project: https://wiki.openjdk.java.net/display/CodeTools/JT+Harness
   - Source: http://hg.openjdk.java.net/code-tools/jtharness/

Neither GPL-2.0 or GPL-2.0 WITH Classpath-exception-2.0 is Category B so I
think this is Category X and cannot be included. SigTest has the same issue
in this section.


   -

   Service Data Objects (SDO) (2.1)
   -

   License: OSOA SDO License

What is this license and what category does it fall under?

JPA (2.0)

   - License: Negotiated agreement between Sun and Eclipse (supercedes spec
   terms)
   - Project: http://jcp.org/en/jsr/detail?id=317

org.apache.felix.framework (6.0.3)

   - License: Pending

pax-exam (n/a)

   - License: Pending

pax-exam-container-forked (4.13.1)

   - License: Pending

pax-exam-junit4 (4.13.1)

   - License: Pending

pax-exam-link-mvn (4.13.1)

   - License: Pending

There are a lot more “Pending” that I won’t list.

org.jline:jline

JLine is distributed under the BSD License, meaning that you are completely
free to redistribute, modify, or sell it with almost no restrictions.

This should include the license and not a third-party interpretation of
what the license means.

On Fri, Jan 10, 2025 at 9:51 PM Jean-Baptiste Onofré 
wrote:

> Hi Dmitri
>
> That's right: https://github.com/apache/polaris/issues/648
>
> I will open a PR soon.
>
> Regards
> JB
>
> On Sat, Jan 11, 2025 at 12:44 AM Dmitri Bourlatchkov 
> wrote:
> >
> > +1 (nb)
> >
> > Verified signature, checksum.
> >
> > JB: I believe you mentioned in the community sync call that you w

Re: [VOTE] Release Apache Polaris 0.9.0-incubating (rc2)

[rdb...@gmail.com]

Hey JB, I just sent a reply to the list with additional details. I don't
think that this release applies license policy correctly even in the other
files, so my vote is still -1.

On Tue, Jan 14, 2025 at 11:05 AM Jean-Baptiste Onofré 
wrote:

> Hi Ryan
>
> As you can see in my previous email, I totally agree with you about the
> issues on the LICENSE_BINARY_DIST.
> As this release only includes source distribution (no jar files, no binary
> packages), and I checked LICENSE/NOTICE for this distribution (see my vote
> email for details), I think we are good.
>
> Do you maintain your -1 vote ?
>
> Thanks
> Regards
> JB
>
> Le lun. 13 janv. 2025 à 19:07, rdb...@gmail.com  a
> écrit :
>
> > I’m still -1 on this release due to licensing concerns.
> >
> > The LICENSE file includes a blanket statement that there are third-party
> > components that are licensed under the Apache Software License 2.0, but
> > doesn’t list what they are. I think this needs to be specific.
> >
> > The NOTICE file has a copyright notice for DropWizard that I would expect
> > to be in LICENSE. The third-party policy states that for third-party
> > notices:
> >
> > Apache releases should contain a copy of each license, usually contained
> in
> > the LICENSE document. For many licenses this is a sufficient notice. Some
> > licenses require some additional notice. In many cases, you can include
> > this notice within the dependent artifact.
> >
> > For the DropWizard content, I’d also expect to see documentation of what
> > was copied into the Polaris source tree. There are similar notices for
> ASF
> > projects, which would be nice to document in the LICENSE file, but aren’t
> > strictly necessary.
> >
> > The binary license file includes this:
> >
> > Apache Polaris distributions contain some or all of the following
> > dependencies
> >
> > I don’t think this is adequate. Each binary artifact should document the
> > third-party code that it includes, the license under which it is
> included,
> > and no other license text (see “How should I handle a work when there is
> a
> > choice of license?”
> > <https://www.apache.org/legal/resolved.html#mutually-exclusive>). As it
> is
> > right now, there are copies of the GPL and that can create a lot of
> concern
> > — doing the work to show that all components use a Category A or B
> license
> > is super important for downstream consumers. In addition, it isn’t
> > sufficient to say that a third-party Category B project might be
> included.
> > It needs to be clear for each artifact what exactly is included; this
> will
> > also help with the issues below which may not actually apply to artifacts
> > because dependencies are provided at runtime rather than bundled.
> >
> > There should also be a NOTICE for each binary artifact. And given the
> other
> > issues with the binary license (see below), I’m not confident that there
> is
> > not additional work to be done to compile the NOTICE.
> >
> > It’s also a good practice to link to the license text rather than include
> > it when it is generic, like the CDDL. When the license embeds authorship
> > information (such as “Neither the name of Company Inc. nor the names of
> its
> > contributors …”) I think it’s fine to include.
> >
> > I recommend a bit more formatting to make the text more clear. For
> example,
> > the jakarta.activation section has confusing sub-sections that state that
> > the license identifier is BSD-3-clause but just above it says it is
> > EDLv1.0. It would be better to show that this entire section was copied
> > from the other project. (This looks like a common problem.)
> >
> > The binary license also includes a few issues:
> >
> > Sax (0.2)
> >
> >- License: SAX-PD
> >- Project: http://www.megginson.com/downloads/SAX/
> >- Source: http://sourceforge.net/project/showfiles.php?group_id=29449
> >
> > I’m not sure what the SAX-PD license is and what category it falls under.
> >
> > wagon-http-lightweight (3.0.0)
> >
> >- License: Pending
> >- Project: https://maven.apache.org/wagon/
> >- Source:
> >
> >
> >
> https://mvnrepository.com/artifact/org.apache.maven.wagon/wagon-http-lightweight/3.0.0
> >
> > This needs to be clarified.
> >
> > dom4j (1.6.1)
> >
> >- License: Custom license based on Apache 1.1
> >
> > Is this custom license compatible?
> >
> > jakarta.xml.bind-api has this in its third-party section:
> >

Re: [VOTE] Release Apache Polaris 0.9.0-incubating (rc2)

[rdb...@gmail.com]

The “blanket” is not a blanket: it’s just the note about gradle.

I do read it as a blanket statement because it is not clearly associated
with the next lines about gradle and refers to “various third-party
components”. Here’s the text:

This product bundles various third-party components also under the Apache
Software License 2.0.

Third-party components need to be individually documented so that people
using the library know what is included in the source tree and the license
under which those components are available. Saying there are “various
third-party components” doesn’t meet the requirement. If it is only gradle,
then I think it should state that the Gradle wrapper is included and that
it is under the ALv2 license (See this for how Iceberg does it).

For DropWizard LICENSE is unmodified LICENSE file (
https://www.apache.org/licenses/LICENSE-2.0) without change, so no need to
copy into the LICENSE file

DropWizard NOTICE is actually included in Polaris NOTICE file

There’s no code copying from Dropwizard, but as we use Discoverable and
DropWizard Test Extension, we mention in the NOTICE

If there is no code copied into the release tarball, then there is no need
to note it in LICENSE or NOTICE. From the Assembling LICENSE and NOTICE
files page <https://infra.apache.org/licensing-howto.html>:

The LICENSE and NOTICE files must *exactly* represent the contents of the
distribution they reside in. Only components and resources that are
actually included in a distribution have any bearing on the content of that
distribution’s NOTICE and LICENSE.

And:

You must customize LICENSE and NOTICE files according to the content of the
specific distribution they reside within. Do not add to LICENSE and NOTICE
dependencies which are not in the distribution. *Only bundled bits matter.*

When a dependency is downloaded directly and is not bundled, it should not
affect the license documentation. From what you wrote, it sounds like
that’s the case with the DropWizard components.

as we use Discoverable and DropWizard Test Extension, we mention in the
NOTICE (which is totally acceptable in NOTICE, some Apache projects use
NOTICE to mention used and included dependencies for instance)

It’s not true to say that this is “totally acceptable”. The section about
Modifications to NOTICE is very clear:

*Do not* add anything to NOTICE which is not legally required.

The reason for this is that additions to NOTICE create obligations for
downstream consumers. NOTICE is not the right place to document bundled
code or bundled dependencies. The place for that is in LICENSE, even if
other projects have not done this correctly. From the same section:

The NOTICE file is reserved for a certain subset of legally required
notifications which are not satisfied by either the text of LICENSE or the
presence of licensing information embedded within the bundled dependency.

Last, thanks for pointing out that there are no binaries. That means that
the license problems there aren’t as critical, but still, I would not vote
to approve this release with the issues and with that file present in the
RC. Since the issues above need to be handled, I’d recommend removing the
file from the release tarball to avoid possible confusion.

On Mon, Jan 13, 2025 at 11:52 AM Jean-Baptiste Onofré 
wrote:

> Hi Ryan
>
> Thanks for the review. Here's my comment:
>
> 1. The "blanket" is not a blanket: it's just the note about gradle.
> 2. For DropWizard LICENSE is unmodified LICENSE file
> (https://www.apache.org/licenses/LICENSE-2.0) without change, so no
> need to copy into the LICENSE file
> 3. DropWizard NOTICE is actually included in Polaris NOTICE file,
> that's this section:
> "Dropwizard
> Copyright 2010-2013 Coda Hale and Yammer, Inc., 2014-2020 Dropwizard Team
>
> This product includes software developed by Coda Hale and Yammer, Inc.
> "
> 4. There's no code copying from Dropwizard, but as we use Discoverable
> and DropWizard Test Extension, we mention in the NOTICE (which is
> totally acceptable in NOTICE, some Apache projects use NOTICE to
> mention used and included dependencies for instance).
> 5. This release only includes source distribution, so everything in
> LICENSE-BINARY-DIST is unrelated to the release and will be fixed with
> the first release including binary distributions (the script
> generating that should be changed but as we are changing the runtime
> framework, it will be revisited)
>
> Regards
> JB
>
> On Mon, Jan 13, 2025 at 7:07 PM rdb...@gmail.com  wrote:
> >
> > I’m still -1 on this release due to licensing concerns.
> >
> > The LICENSE file includes a blanket statement that there are third-party
> > components that are licensed under the Apache Software License 2.0, but
> > doesn’t list what they are. I think this needs to be specific.
> >
> > Th

Re: [VOTE] Release Apache Polaris (incubating) 0.9.0-rc6

[rdb...@gmail.com]

Carrying over my +1 from the dev list.

Because there is no intent to publish convenience binaries, I don't think
there are requirements for the Jars that are produced. If that
understanding is incorrect, please let me know and I'll start checking the
Jars even if they aren't going to be published. Otherwise, I'm planning to
validate the LICENSE and NOTICE of the Jars in the next release, when there
are convenience binaries.

On Fri, Feb 21, 2025 at 5:51 AM Jean-Baptiste Onofré 
wrote:

> Hi PJ
>
> You are right: this first Polaris release only includes source
> distribution (which is the minimum required).
> There are two reasons:
> - we did/do a change in the runtime framework and the binary
> artifacts, completely changing the LICENSE/NOTICE and layout (that's
> why we don't have LICENSE/NOTICE, see
> https://github.com/apache/polaris/issues/1031). So, technically
> speaking, we can do binary artifacts in this release, but we preferred
> to focus on the 1.0.0 release for binary artifacts.
> - we wanted to already do a pass on the source distribution in this
> release, and focus on the binary artifacts for the next release
>
> The next release (1.0.0) will include binary distributions (and binary
> artifacts).
>
> Regards
> JB
>
> On Fri, Feb 21, 2025 at 2:32 PM PJ Fanning  wrote:
> >
> > +0 binding
> >
> > * incubating in name
> > * LICENSE, NOTICE, DISCLAIMER seem ok
> > * no unexpected binaries
> > * headers in files
> > * gradle build seems to work
> >
> > The reason that I am not voting +1 (yet) is that I don't know about
> > the plans (or otherwise) for releasing jars. There are no polaris jars
> > in repository.apache.org staging or snapshots areas.
> >
> > The jars that are built in the gradle build contain not META-INF
> > LICENSE or NOTICE (or DISCLAIMER). I think they should.
> >
> > On Fri, 21 Feb 2025 at 11:08, Jean-Baptiste Onofré 
> wrote:
> > >
> > > Ack:
> > > https://downloads.apache.org/incubator/polaris/KEYS
> > >
> > > Regards
> > > JB
> > >
> > > Le ven. 21 févr. 2025 à 10:39, sebb  a écrit :
> > >
> > > > On Fri, 21 Feb 2025 at 07:00, Jean-Baptiste Onofré 
> > > > wrote:
> > > > >
> > > > > Hi folks,
> > > > >
> > > > > This is a call for the vote to release Apache Polaris (incubating)
> > > > 0.9.0-rc6.
> > > > >
> > > > > The Polaris community has voted and approved Apache Polaris
> > > > > (incubating) 0.9.0-rc6.
> > > > > We are now asking the IPMC to review and vote for this release.
> > > > >
> > > > > Polaris community vote thread:
> > > > > https://lists.apache.org/thread/7jxoswynzpnhsmbt77t4klc7rp983o66
> > > > >
> > > > > Polaris community vote result:
> > > > > https://lists.apache.org/thread/3ymqrtlvtcq1pgtd14ghn2n158y6yl5t
> > > > >
> > > > > * This corresponds to the tag: apache-polaris-0.9.0-incubating-rc6
> > > > > *
> > > >
> https://github.com/apache/polaris/tree/apache-polaris-0.9.0-incubating-rc6
> > > > > *
> > > >
> https://github.com/apache/polaris/tree/4b18ec065ff16f74b11bc85fdc6ea9036eca7274
> > > > >
> > > > > The release source tarball, signature, and checksums are here:
> > > > > *
> > > >
> https://dist.apache.org/repos/dist/dev/incubator/polaris/0.9.0-incubating/
> > > > >
> > > > > You can find the KEYS file here:
> > > > > *
> https://dist.apache.org/repos/dist/release/incubator/polaris/KEYS
> > > >
> > > > Please use the public download URL for he KEYS file!
> > > >
> > > > > The vote will be open for at least 72 hours or until the necessary
> > > > > number of votes are reached.
> > > > >
> > > > > Please vote accordingly:
> > > > >
> > > > > [ ] +1 Release this as Apache Polaris (incubating) 0.9.0
> > > > > [ ] +0
> > > > > [ ] -1 Do not release this because...
> > > > >
> > > > > This vote will pass if there are 3 binding +1 votes and more
> binding
> > > > > +1 votes than -1 votes.
> > > > >
> > > > > Thanks,
> > > > > Regards
> > > > > JB
> > > > >
> > > > >
> -
> > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > > > > For additional commands, e-mail: general-h...@incubator.apache.org
> > > > >
> > > >
> >
> > -
> > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> > For additional commands, e-mail: general-h...@incubator.apache.org
> >
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: [VOTE] Release Apache Polaris 0.9.0-incubating (rc6)

[rdb...@gmail.com]

+1

The update to DISCLAIMER and NOTICE look okay to me, although I don't think
that the Nessie NOTICE content is necessary. That information, which is
just the copyright, is already in LICENSE to document the parts that are
licensed.

On Thu, Feb 20, 2025 at 6:58 AM Russell Spitzer 
wrote:

> +1
>
> On Thu, Feb 20, 2025 at 6:06 AM Jean-Baptiste Onofré 
> wrote:
>
> > +1 (binding)
> >
> > Regards
> > JB
> >
> > Le lun. 17 févr. 2025 à 21:27, Jean-Baptiste Onofré  a
> > écrit :
> >
> > > Hi folks,
> > >
> > > After the vote on the incubator general mailing list, we fixed the
> > > DISCLAIMER content and cleaned up the NOTICE file.
> > >
> > > This is the vote for Apache Polaris 0.9.0-incubating rc6.
> > >
> > > * This corresponds to the tag: apache-polaris-0.9.0-incubating-rc6
> > > *
> > >
> >
> https://github.com/apache/polaris/tree/apache-polaris-0.9.0-incubating-rc6
> > > *
> > >
> >
> https://github.com/apache/polaris/tree/4b18ec065ff16f74b11bc85fdc6ea9036eca7274
> > >
> > > The release source tarball, signature, and checksums are here:
> > > *
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/polaris/0.9.0-incubating/
> > >
> > > You can find the KEYS file here:
> > > * https://dist.apache.org/repos/dist/release/incubator/polaris/KEYS
> > >
> > > NB: this release doesn't provide any binary distribution, it provides
> > > source distribution only.
> > >
> > > Please download, verify, and test.
> > >
> > > Please vote in the next 72 hours.
> > > [ ] +1 Release this as Apache polaris 0.9.0-incubating
> > > [ ] +0
> > > [ ] -1 Do not release this because...
> > >
> > > Only PPMC members and mentors have binding votes, but other community
> > > members are encouraged to cast non-binding votes. This vote will pass
> > > if there are
> > > 3 binding +1 votes and more binding +1 votes than -1 votes.
> > >
> > > NB: if this vote passes, a new vote will be started on the Incubator
> > > general mailing list.
> > >
> > > Thanks
> > > Regards
> > > JB
> > >
> >
>