I did a release check for license issues - all in all, we need a new RC.
The only blocker I found was the missing jquery license file.
Another somewhat critical thing is that "statefun-flink-distribution"
bundles many unwanted dependencies.
- Because the shading merges the notice files, this is not a legal
issue.
- Because Flinks inverted classloading still uses "parent-first" for
all
"org.apache.flink.*" classes, this does not break the system
But it is unwanted behavior and makes the artifacts unnecessarily large.
I opened FLINK-16891 - FLINK-16897 for the issues I found.
All issues are fixed in this PR:
https://github.com/apache/flink-statefun/pull/85
On Tue, Mar 31, 2020 at 7:17 PM Stephan Ewen <se...@apache.org> wrote:
I have found a few things, am preparing a joint PR to fix them.
So far, only the missing jquery license would have been a release
blocker.
On Tue, Mar 31, 2020 at 6:24 PM Chesnay Schepler <ches...@apache.org>
wrote:
The jquery license is in fact missing from the master/release-1.10
branches. https://issues.apache.org/jira/browse/FLINK-16888
On 31/03/2020 12:18, Chesnay Schepler wrote:
For Kafka we traditionally exclude the NOTICE file since as far as
we
can tell it is misleading anyway, see the flink-sql-connector-kafka
modules.
@Robert for the Flink project the jquery license is in the source at
licenses/LICENSE.jquery
I'm a bit concerned just how many licensing issues are showing up in
these RCs. I would suggest to do a proper scan of the licensing
before
opening another RC.
And yes, the missing MIT license is grounds for cancellation,
hence, -1.
On 31/03/2020 11:56, Robert Metzger wrote:
Thanks a lot Gordon!
Checked:
- files in the staging repository seem to be ok (no unexpected
files,
versions set correctly, quickstart archetype looks ok)
- statefun-ridesharing-example-simulator-2.0.0.jar (and
/org/apache/flink/statefun-flink-distribution/2.0.0/statefun-flink-distribution-2.0.0.jar)
contains a NOTICE file in the root which seems to come from Apache
Kafka.
The file states
This distribution has a binary dependency on jersey, which is
available
under the CDDL
License. The source code of jersey can be found at
https://github.com/jersey/jersey/.
This text is not mentioned in our NOTICE file (which is located in
META-INF/NOTICE).
I'm not a lawyer, but the NOTICE file situation might be confusing
in
that
jar. The first NOTICE file you see is from Kafka. If we argue that
this is
not the right file,
because that one is located in META-INF/NOTICE, then we might be at
risk of
not having properly forwarded Kafka's NOTICE file.
I believe this is okay, as we somehow include all the necessary
information, but we should address this in the next release (or if
this RC
gets cancelled again).
I'm also curious to hear the opinion of others on this.
- The source release contains "docs/page/js/jquery.min.js", which
is
MIT
licensed. The MIT license requires us to ship a copy of the license
with
each copy of the source.
apache/flink also has this file:
https://github.com/apache/flink/blob/master/docs/page/js/jquery.min.js,
but
it ships the jquery license in the "licenses/" folder (even though
this
file is not in git, I guess it's added during release generation?!)
I believe we have to cancel this RC because of the missing license
file in
the source distribution? I'm not voting on this RC, in case I have
overlooked something and we can continue.
On Tue, Mar 31, 2020 at 9:31 AM Tzu-Li (Gordon) Tai
<tzuli...@apache.org>
wrote:
======= NOTICE =======
For your testing, please continue to use this staging area for the
Maven
artifacts:
https://repository.apache.org/content/repositories/orgapacheflink-1344/
The only difference between this staging repo and the original
repo
posted
in this thread (
https://repository.apache.org/content/repositories/orgapacheflink-1343/)
is that a few unintended source release distributions have been
removed
from the Maven repo staging area.
Those should not be built and published by Maven, since we use
our own
tools to build the source distributions (staged at
https://dist.apache.org/repos/dist/dev/flink/flink-statefun-2.0.0-rc4/).
Since this does not affect any code in the project, and the staged
Maven
artifacts are still built with the same commit hash as the source
distribution,
this RC vote will continue to run until the original vote end
time.
All previous votes in this thread will still be accounted for.
On Tue, Mar 31, 2020 at 2:57 PM Tzu-Li (Gordon) Tai
<tzuli...@apache.org>
wrote:
Sounds good, I'll post a new link to this vote thread, which will
have
the
problem fixed in a new maven staging repository.
On Tue, Mar 31, 2020 at 2:51 PM Robert Metzger <
rmetz...@apache.org>
wrote:
Thank you for looking into this.
I'm fine with keeping this RC open, but re-vote on a new maven
staging
repository.
On Tue, Mar 31, 2020 at 8:42 AM Tzu-Li (Gordon) Tai <
tzuli...@apache.org>
wrote:
Found the culprit:
The Stateful Functions project uses the Apache POM as the
parent
POM,
and
uses the `apache-release` build profile to build the staging
jars.
The problem arises because the `apache-release` build profile
itself
bundles a source release distribution to be released to Maven.
This should be disabled specifically for us, because we use
our own
tooling
(tools/releasing/create_source_release.sh) to create the source
tarballs
which does correctly exclude all those unexpected files Robert
found.
Will rebuild the RC. I think in this case, it's completely
fine to
keep
with the original voting end time, since nothing is really
touched,
only
excluding some files from the staging Maven repository.
On Tue, Mar 31, 2020 at 2:29 PM Tzu-Li (Gordon) Tai <
tzuli...@apache.org>
wrote:
Hi Robert,
I think you're right. There should be no tarballs / jars
packaged
for
statefun-parent actually, only the pom file since that's the
parent
module
which only has pom packaging.
I'm looking into it.
On Tue, Mar 31, 2020 at 2:23 PM Robert Metzger <
rmetz...@apache.org
wrote:
While checking the release, I found a 77
MB statefun-parent-2.0.0-source-release.zip file in the maven
staging
repo:
https://repository.apache.org/content/repositories/orgapacheflink-1343/org/apache/flink/statefun-parent/2.0.0/
It seems that the file contains all ruby dependencies in
docs/
from
jekyll
for the docs (in
"statefun-parent-2.0.0/docs/.rubydeps/ruby/2.5.0").
I
don't think we want to publish these files as part of the
release
to
maven
central?
(It also contains python venv files in
"statefun-python-sdk/venv")
I guess this is a reason to cancel the RC?
On Tue, Mar 31, 2020 at 6:10 AM Tzu-Li (Gordon) Tai <
tzuli...@apache.org>
wrote:
+1 (binding)
** Legal **
- checksums and GPG files match corresponding release files
- Source distribution does not contain binaries, contents
are
sane
(no
.git* / .travis* / generated html content files)
- Bundled source LICENSEs and NOTICE looks good. Mentions
bundled
font-awesome dependency in docs and copied sources from
fastutil
(
http://fastutil.di.unimi.it/)
- Bundled LICENSEs and NOTICE files for Maven artifacts
looks
good.
Artifacts that do bundle dependencies are:
statefun-flink-distribution,
statefun-ridesharing-example-simulator, statefun-flink-core
(copied
sources).
- Python SDK distributions (source and wheel) contain ASLv2
LICENSE
and
NOTICE files (no bundled dependencies)
- All POMs / README / Python SDK setup.py / Dockerfiles /
doc
configs
point
to same version “2.0.0”
- README looks good
** Functional **
- Building from source dist with end-to-end tests enabled
(mvn
clean
verify
-Prun-e2e-tests) passes (JDK 8)
- Generated quickstart from archetype looks good (correct
POM /
Dockerfile
/ service file)
- Examples run: Java Greeter / Java Ridesharing / Python
Greeter
/
Python
SDK Walkthrough
- Flink Harness works in IDE
- Test remote functions deployment mode with AWS ecosystem:
remote
Python
functions running in AWS Lambda behind AWS API Gateway, Java
embedded
functions running in AWS ECS
On Tue, Mar 31, 2020 at 12:09 PM Tzu-Li (Gordon) Tai <
tzuli...@apache.org>
wrote:
FYI - I've also updated the website Downloads page to
include
this
release.
Please also consider that for your reviews:
https://github.com/apache/flink-web/pull/318
On Tue, Mar 31, 2020 at 3:42 AM Konstantin Knauf <
konstan...@ververica.com>
wrote:
Hi Gordon,
+1 (non-binding)
* Maven build from source...check
* Python build from source...check
* Went through Walkthrough based on local builds...check
Cheers,
Konstantin
On Mon, Mar 30, 2020 at 5:52 AM Tzu-Li (Gordon) Tai <
tzuli...@apache.org>
wrote:
Hi everyone,
Please review and vote on the *release candidate #4* for
the
version
2.0.0
of Apache Flink Stateful Functions,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide
specific
comments)
**Testing Guideline**
You can find here [1] a doc that we can use for
collaborating
testing
efforts.
The listed testing tasks in the doc also serve as a
guideline
in
what
to
test for this release.
If you wish to take ownership of a testing task, simply
put
your
name
down
in the "Checked by" field of the task.
**Release Overview**
As an overview, the release consists of the following:
a) Stateful Functions canonical source distribution, to
be
deployed to
the
release repository at dist.apache.org
b) Stateful Functions Python SDK distributions to be
deployed
to
PyPI
c) Maven artifacts to be deployed to the Maven Central
Repository
**Staging Areas to Review**
The staging areas containing the above mentioned
artifacts
are as
follows,
for your review:
* All artifacts for a) and b) can be found in the
corresponding
dev
repository at dist.apache.org [2]
* All artifacts for c) can be found at the Apache Nexus
Repository
[3]
All artifacts are singed with the
key 1C1E2394D3194E1944613488F320986D35C33D6A [4]
Other links for your review:
* JIRA release notes [5]
* source code tag "release-2.0.0-rc4" [6] [7]
**Extra Remarks**
* Part of the release is also official Docker images for
Stateful
Functions. This can be a separate process, since the
creation
of
those
relies on the fact that we have distribution jars already
deployed
to
Maven. I will follow-up with this after these artifacts
are
officially
released.
In the meantime, there is this discussion [8] ongoing
about
where
to
host
the StateFun Dockerfiles.
* The Flink Website and blog post is also being worked on
(by
Marta)
as
part of the release, to incorporate the new Stateful
Functions
project.
We
can follow up with a link to those changes afterwards in
this
vote
thread,
but that would not block you to test and cast your votes
already.
* Since the Flink website changes are still being worked
on,
you
will
not
yet be able to find the Stateful Functions docs from
there.
Here
are
the
links [9] [10].
**Vote Duration**
Since this RC only fixes licensing issues from previous
RCs,
and the code itself has not been touched,
I'd like to stick with the original vote ending time.
The vote will be open for at least 72 hours starting
Monday
*(target end date is Wednesday, April 1st).*
It is adopted by majority approval, with at least 3 PMC
affirmative
votes.
Thanks,
Gordon
[1]
https://docs.google.com/document/d/1P9yjwSbPQtul0z2AXMnVolWQbzhxs68suJvzR6xMjcs/edit?usp=sharing
[2]
https://dist.apache.org/repos/dist/dev/flink/flink-statefun-2.0.0-rc4/
[3]
https://repository.apache.org/content/repositories/orgapacheflink-1343/
[4]
https://dist.apache.org/repos/dist/release/flink/KEYS
[5]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12346878
[6]
https://gitbox.apache.org/repos/asf?p=flink-statefun.git;a=commit;h=5d5d62fca2dbe3c75e8157b7ce67d4d4ce12ffd9
[7]
https://github.com/apache/flink-statefun/tree/release-2.0.0-rc4
[8]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Creating-a-new-repo-to-host-Stateful-Functions-Dockerfiles-td39342.html
[9]
https://ci.apache.org/projects/flink/flink-statefun-docs-master/
[10]
https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.0/
TIP: You can create a `settings.xml` file with these
contents:
"""
<settings>
<activeProfiles>
<activeProfile>flink-statefun-2.0.0</activeProfile>
</activeProfiles>
<profiles>
<profile>
<id>flink-statefun-2.0.0</id>
<repositories>
<repository>
<id>flink-statefun-2.0.0</id>
<url>
https://repository.apache.org/content/repositories/orgapacheflink-1343/
</url>
</repository>
<repository>
<id>archetype</id>
<url>
https://repository.apache.org/content/repositories/orgapacheflink-1343/
</url>
</repository>
</repositories>
</profile>
</profiles>
</settings>
"""
And reference that in you maven commands via `--settings
path/to/settings.xml`.
This is useful for creating a quickstart based on the
staged
release
and
for building against the staged jars.
--
Konstantin Knauf | Head of Product
+49 160 91394525
Follow us @VervericaData Ververica <
https://www.ververica.com/>
--
Join Flink Forward <https://flink-forward.org/> - The
Apache
Flink
Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin,
Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park
Tung
Jason,
Ji
(Tony) Cheng
