[
https://issues.apache.org/jira/browse/FLINK-29634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653483#comment-17653483
]
Jiale Tan edited comment on FLINK-29634 at 1/2/23 3:27 AM:
-----------------------------------------------------------
[~gyfora] [~thw] Thanks for the guidance!
I am trying to bump {{flink.version}} up to 1.17 (SNAPSHOT) locally to take
advantage of the checkpoint rest API classes in 1.17 and start coding. However
I found it a little bit tricky to deal with the dependencies, since in flink
1.17 {{io.fabric8}} classes are shaded and relocated in the
{{flink-kubernetes}} in
[this|https://github.com/apache/flink/commit/17d7c39bb2a9fcbaac1ead42073c099a52171d7d]
commit.
This will cause some {{io.fabric8}} classes in the imports not found (which
means these classes used to be only provided by {{{}flink-kubernetes{}}}),
fixing their package is comparatively easy, just to change import
{{io.fabric8}} classes packages to
{{org.apache.flink.kubernetes.shaded.io.fabric8}}
Things get trickier when {{io.fabric8}} classes used to be provided by
{{flink-kubernetes-operator}} (like
[these|https://github.com/apache/flink-kubernetes-operator/blob/a1842d4c0170feb008293963ec51c0343f42771d/flink-kubernetes-standalone/pom.xml#L74-L79]
) and also available in {{flink-kubernetes}} . The new relocation change in
flink 1.17 will cause {{flink-kubernetes-operator}} functions (like
[this|https://github.com/apache/flink-kubernetes-operator/blob/a1842d4c0170feb008293963ec51c0343f42771d/flink-kubernetes-standalone/src/test/java/org/apache/flink/kubernetes/operator/kubeclient/parameters/ParametersTestBase.java#L92])
with arguments of {{io.fabric8}} classes now requiring
{{org.apache.flink.kubernetes.shaded.io.fabric8}} classes arguments. Meanwhile
it seems {{flink-kubernetes}} vs {{flink-kubernetes-operator}} are using
different {{io.fabric8}} versions (5.12.3 vs 6.2.0 respectively), this means
changing import class packages like previous case will cause a version
downgrade for {{io.fabric8}} classes.
And also there are certain classes which are not available in
{{flink-kubernetes}} like
[these|https://github.com/apache/flink-kubernetes-operator/blob/7ced741f51a99f2093ce8a45c8c92879a247f836/flink-kubernetes-standalone/src/test/java/org/apache/flink/kubernetes/operator/kubeclient/Fabric8FlinkStandaloneKubeClientTest.java#L33-L34].
Calling functions from those classes will just create {{io.fabric8}} class
objects which do not work with rest of code, which only accept
{{org.apache.flink.kubernetes.shaded.io.fabric8}} class objects
2 work arounds I can think of
# to have a dedicated subproject just to relocate back the {{io.fabric8}}
classes in {{{}flink-kubernetes{}}}, (or other way around relocate things to
{{{}org.apache.flink.kubernetes.shaded.io.fabric8{}}}) both seem a little bit
ugly.
# ignore the {{io.fabric8}} classes from {{flink-kubernetes}} and add all
required {{io.fabric8}} dependencies to pom.xml, not sure if it is ok to
upgrade all dependencies version to 6.2.0.
I am wondering if you folks have some good advices how to deal with this.
was (Author: JIRAUSER290356):
[~gyfora] [~thw] Thanks for the guidance!
I am trying to bump {{flink.version}} up to 1.17 (SNAPSHOT) locally to take
advantage of the checkpoint rest API classes in 1.17 and start coding. However
I found it a little bit tricky to deal with the dependencies, since in flink
1.17 {{io.fabric8}} classes are shaded and relocated in the
{{flink-kubernetes}} in
[this|https://github.com/apache/flink/commit/17d7c39bb2a9fcbaac1ead42073c099a52171d7d]
commit.
This will cause some {{io.fabric8}} classes in the imports not found (which
means these classes used to be only provided by {{{}flink-kubernetes{}}}),
fixing their package is comparatively easy, just to change import
{{io.fabric8}} classes packages to
{{org.apache.flink.kubernetes.shaded.io.fabric8}}
Things get trickier when {{io.fabric8}} classes used to be provided by
{{flink-kubernetes-operator}} (like
[these|https://github.com/apache/flink-kubernetes-operator/blob/a1842d4c0170feb008293963ec51c0343f42771d/flink-kubernetes-standalone/pom.xml#L74-L79]
). The new relocation change in flink 1.17 will cause
{{flink-kubernetes-operator}} functions (like
[this|https://github.com/apache/flink-kubernetes-operator/blob/a1842d4c0170feb008293963ec51c0343f42771d/flink-kubernetes-standalone/src/test/java/org/apache/flink/kubernetes/operator/kubeclient/parameters/ParametersTestBase.java#L92])
with arguments of {{io.fabric8}} classes now requiring
{{org.apache.flink.kubernetes.shaded.io.fabric8}} classes arguments. Meanwhile
it seems {{flink-kubernetes}} vs {{flink-kubernetes-operator}} are using
different {{io.fabric8}} versions (5.12.3 vs 6.2.0 respectively), this means
changing import class packages like previous case will cause a version
downgrade for {{io.fabric8}} classes.
2 work arounds I can think of
# to have a dedicated subproject just to relocate back the {{io.fabric8}}
classes in {{{}flink-kubernetes{}}}, (or other way around relocate things to
{{{}org.apache.flink.kubernetes.shaded.io.fabric8{}}}) both seem a little bit
ugly.
# ignore the {{io.fabric8}} classes from {{flink-kubernetes}} and add all
required {{io.fabric8}} dependencies to pom.xml, not sure if it is ok to
upgrade all dependencies version to 6.2.0.
I am wondering if you folks have some good advices how to deal with this.
> Support periodic checkpoint triggering
> --------------------------------------
>
> Key: FLINK-29634
> URL: https://issues.apache.org/jira/browse/FLINK-29634
> Project: Flink
> Issue Type: New Feature
> Components: Kubernetes Operator
> Reporter: Thomas Weise
> Assignee: Jiale Tan
> Priority: Major
>
> Similar to the support for periodic savepoints, the operator should support
> triggering periodic checkpoints to break the incremental checkpoint chain.
> Support for external triggering will come with 1.17:
> https://issues.apache.org/jira/browse/FLINK-27101
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
