Long term plan _is_ to move flink-shaded-hadoop to flink-shaded, I
believe there's even a JIRA for that.
Until that is in place they _must_ have retain the flink version as
otherwise we'd be unable to change them in follow-up releases without
changing the version scheme again.
And even after the move they will retain the flink-shaded version like
all other flink-shaded modules, for the above reason.
On 18.03.2019 12:10, jincheng sun wrote:
Hi Chesnay,
The artifacts to be released do not have a SNAPSHOT suffix:
https://repository.apache.org/content/repositories/orgapacheflink-1213/org/apache/flink/flink-shaded-hadoop2-uber/
Thank you for providing this link. It's very useful for contributors
who want to check the RC on YARN.
My suggestion may not describe being clear, let me explain:
1. Since 1.8.0, Flink's release package will not contain the
corresponding Hadoop dependency, then the user has two ways to get the
required hadoop dependency:
1). Download the existing Hadoop version on the Flink download page.
2). Generate the version required by the user from the source code
(see
https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/building.html#hadoop-versions)
For example, version 2.6.1 is required: `mvn clean install -DskipTests
-Dhadoop.version=2.6.1`.
2. About how to manage the JARs release of Hadoop dependencies:
1). The name of Hadoop shaded version should not include Flink
version, take your link as an example:
`.../flink-shaded-hadoop2-uber/2.4.1-1.8.0/xx.jar`
`.../flink-shaded-hadoop2-uber/2.6.5-1.8.0/xx.jar`
`.../flink-shaded-hadoop2-uber/2.7.5-1.8.0/xx.jar`
`.../flink-shaded-hadoop2-uber/2.8.3-1.8.0/xx.jar`
The above version name I think it is possible to change `2.4.1-1.8.0`
to `2.4.1`. That is, the same version of `Hadoop` shade can be used in
many Flink versions, such as 2.8.3 Hadoop is not only available for
Flink-1.8.0, it can be used by Flink-1.8.x or it can be used by
Flink-1.9.x. etc.
2). Release the shaded-Hadoop independently:
For a long-term, we can release the shaded JARs independently and
move `flink-shaded-hadoop` into
`https://github.com/apache/flink-shaded`
<https://github.com/apache/flink-shaded%60>, So I suggest that we can
publish Hadoop versions independently, and share them in multiple
Flink versions.
What do you think?
Best,
Jincheng
Chesnay Schepler <ches...@apache.org <mailto:ches...@apache.org>>
于2019年3月18日周一 下午4:15写道：
We release SNAPSHOT artifacts for all module, see
https://repository.apache.org/content/groups/public/org/apache/flink/flink-core/
.
The artifacts to be released do not have a SNAPSHOT suffix:
https://repository.apache.org/content/repositories/orgapacheflink-1213/org/apache/flink/flink-shaded-hadoop2-uber/
Finally, we are already adding flink-shaded-hadoop to the optional
components section in this PR:
https://github.com/apache/flink-web/pull/180
On 18.03.2019 08:55, jincheng sun wrote:
> -1
>
> Currently, we have released the Hadoop-related JRA as a snapshot
> version(such as flink-shaded-hadoop2-uber/2.4.1-1.8-SNAPSHOT
>
<https://repository.apache.org/content/groups/public/org/apache/flink/flink-shaded-hadoop2-uber/>),
> I think we should release a stable version.
> When testing the release code on YARN, currently user cannot
find out the
> Hadoop dependency. Although there is a download explanation for
Hadoop in
> PR [`Update Downloads page for Flink 1.8
> <https://github.com/apache/flink-web/pull/180/files>`], a 404
error occurs
> when you click Download ( I had left detail comments in the PR).
>
> So, I suggest as follows:
>
> 1. It would be better to add the changes for
> `downloads.html#optional-components`, add the Hadoop relation
JARs download
> link first.
> 2. Then add instructions on how to get the dependencies of
the Hadoop or
> add the correct download link directly in the next VOTE mail,
due to we do
> not include Hadoop in `flink-dist`.
> 3. Release a stable version Hadoop-related JRAs.
>
> Then, contributors can test it more easily on YARN. What do you
think?
>
> Best,
> Jincheng
>
>
> Chesnay Schepler <ches...@apache.org
<mailto:ches...@apache.org>> 于2019年3月15日周五 下午10:35写道：
>
>> -1
>>
>> Missing dependencies in NOTICE file of flink-dist (and by
extension the
>> binary distribution).
>> * com.data-artisans:frocksdbjni:jar:5.17.2-artisans-1.0
>>
>> On 14.03.2019 13:42, Aljoscha Krettek wrote:
>>> Hi everyone,
>>> Please review and vote on the release candidate 2 for Flink
1.8.0, as
>> follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific
comments)
>>>
>>>
>>> The complete staging area is available for your review, which
includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release and binary convenience
releases to
>> be deployed to dist.apache.org <http://dist.apache.org>
<http://dist.apache.org/> [2], which are
>> signed with the key with fingerprint
>> F2A67A8047499BBB3908D17AA8F4FD97121D7293 [3],
>>> * all artifacts to be deployed to the Maven Central Repository
[4],
>>> * source code tag "release-1.8.0-rc2" [5],
>>> * website pull request listing the new release [6]
>>> * website pull request adding announcement blog post [7].
>>>
>>> The vote will be open for at least 72 hours. It is adopted by
majority
>> approval, with at least 3 PMC affirmative votes.
>>> Thanks,
>>> Aljoscha
>>>
>>> [1]
>>
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344274
>> <
>>
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344274
>>> [2]
https://dist.apache.org/repos/dist/dev/flink/flink-1.8.0-rc2/ <
>> https://dist.apache.org/repos/dist/dev/flink/flink-1.8.0-rc2/>
>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS <
>> https://dist.apache.org/repos/dist/release/flink/KEYS>
>>> [4]
>>
https://repository.apache.org/content/repositories/orgapacheflink-1213
<
>>
https://repository.apache.org/content/repositories/orgapacheflink-1210/>
>>> [5]
>>
https://gitbox.apache.org/repos/asf?p=flink.git;a=tag;h=c77a329b71e3068bfde965ae91921ad5c47246dd
>> <
>>
https://gitbox.apache.org/repos/asf?p=flink.git;a=tag;h=2d00b1c26d7b4554707063ab0d1d6cc236cfe8a5
>>> [6] https://github.com/apache/flink-web/pull/180 <
>> https://github.com/apache/flink-web/pull/180>
>>> [7] https://github.com/apache/flink-web/pull/179 <
>> https://github.com/apache/flink-web/pull/179>
>>> P.S. The difference to the previous RC1 is very small, you can
fetch the
>> two tags and do a "git log
release-1.8.0-rc1..release-1.8.0-rc2” to see the
>> difference in commits. Its fixes for the issues that led to the
>> cancellation of the previous RC plus smaller fixes. Most
>> verification/testing that was carried out should apply as is to
this RC.
>>
>>
>>
