Re:[VOTE] FLIP-301: Hybrid Shuffle supports Remote Storage

[Wencong Liu]

+1 (non-binding)

Best regards,

Wencong Liu















At 2023-03-20 12:05:47, "Yuxin Tan"  wrote:
>Hi, everyone,
>
>Thanks for all your feedback for FLIP-301: Hybrid Shuffle
>supports Remote Storage[1] on the discussion thread[2].
>
>I'd like to start a vote for it. The vote will be open for at
>least 72 hours (03/23, 13:00 UTC+8) unless there is an
>objection or not enough votes.
>
>[1]
>https://cwiki.apache.org/confluence/display/FLINK/FLIP-301%3A+Hybrid+Shuffle+supports+Remote+Storage
>[2] https://lists.apache.org/thread/nwrqd5jtqwks89tbxpcrgto6r2bhdhno
>
>Best,
>Yuxin

[VOTE] FLIP-301: Hybrid Shuffle supports Remote Storage

[Yuxin Tan]

Hi, everyone,

Thanks for all your feedback for FLIP-301: Hybrid Shuffle
supports Remote Storage[1] on the discussion thread[2].

I'd like to start a vote for it. The vote will be open for at
least 72 hours (03/23, 13:00 UTC+8) unless there is an
objection or not enough votes.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-301%3A+Hybrid+Shuffle+supports+Remote+Storage
[2] https://lists.apache.org/thread/nwrqd5jtqwks89tbxpcrgto6r2bhdhno

Best,
Yuxin

[jira] [Created] (FLINK-31520) Move SqlCreateTable conversion logic to SqlNodeConverter

[xuzhiwen (Jira)]

xuzhiwen created FLINK-31520:


 Summary: Move SqlCreateTable conversion logic to SqlNodeConverter
 Key: FLINK-31520
 URL: https://issues.apache.org/jira/browse/FLINK-31520
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: xuzhiwen


Introduce SqlNodeConverter for SqlModule-related conversion logic



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release 1.17.0, release candidate #3

[Leonard Xu]

Thanks Jing for the validation, I’ve created FLINK-31519[1] to fix the outdated 
doc,
Sure that it’s a minor doc issue and won’t block the release.

Best,
Leonard

[1]https://issues.apache.org/jira/browse/FLINK-31519

> On Mar 20, 2023, at 7:45 AM, Jing Ge  wrote:
> 
> Hi Qingsheng,
> 
> While reviewing the announcement PR[1], I found that the related Flink doc
> for Watermark Alignment Enhancement has not been updated yet. Afaiac, It is
> not a blocking issue but would be great if it contains up-to-date
> information and is consistent with the announcement.
> 
> Best regards,
> Jing
> 
> [1] https://github.com/apache/flink-web/pull/618
> [2]
> https://github.com/apache/flink/blob/release-1.17/docs/content/docs/dev/datastream/event-time/generating_watermarks.md?plain=1#L308
> 
> On Sat, Mar 18, 2023 at 10:10 PM Jing Ge  wrote:
> 
>> +1
>> 
>> - Verified the signature
>> - Verified hashes
>> - Verify that the source archives do not contain any binaries
>> - Built from the source code
>> - Verified web PR
>> - Checked jars in the staging repo
>> 
>> Best regards,
>> Jing
>> 
>> On Sat, Mar 18, 2023 at 9:11 AM Lijie Wang 
>> wrote:
>> 
>>> +1 (binding)
>>> 
>>> -  Built from the source code
>>> -  Verified the signature and checksum
>>> -  Ran both streaming/batch jobs on yarn cluster
>>> -  Checked the website PR
>>> 
>>> Best,
>>> Lijie
>>> 
>>> Qingsheng Ren  于2023年3月17日周五 22:02写道：
>>> 
 Hi everyone,
 
 Please review and vote on the release candidate #3 for the version
>>> 1.17.0,
 as follows:
 [ ] +1, Approve the release
 [ ] -1, Do not approve the release (please provide specific comments)
 
 The complete staging area is available for your review, which includes:
 
 * JIRA release notes [1], and the pull request adding release note for
 users [2]
 * the official Apache source release and binary convenience releases to
>>> be
 deployed to dist.apache.org [3], which are signed with the key with
 fingerprint A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B [4],
 * all artifacts to be deployed to the Maven Central Repository [5],
 * source code tag "release-1.17.0-rc3" [6],
 * website pull request listing the new release and adding announcement
>>> blog
 post [7].
 
 The vote will be open for at least 72 hours. It is adopted by majority
 approval, with at least 3 PMC affirmative votes.
 
 [1]
 
 
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351585
 [2] https://github.com/apache/flink/pull/22146
 [3] https://dist.apache.org/repos/dist/dev/flink/flink-1.17.0-rc3/
 [4] https://dist.apache.org/repos/dist/release/flink/KEYS
 [5]
>>> https://repository.apache.org/content/repositories/orgapacheflink-1600
 [6] https://github.com/apache/flink/releases/tag/release-1.17.0-rc3
 [7] https://github.com/apache/flink-web/pull/618
 
 Thanks,
 Martijn and Matthias, Leonard and Qingsheng
 
>>> 
>> 

Re: [VOTE] Release 1.17.0, release candidate #3

[Qingsheng Ren]

Thanks Jing for the validation!

The document doesn't block the releasing process. A GitHub action picks up
the latest commit on release branches and builds docs nightly.

Also cc'ing the author of FLINK-28853 for updating the doc.

Best,
Qingsheng

On Mon, Mar 20, 2023 at 7:46 AM Jing Ge  wrote:

> Hi Qingsheng,
>
> While reviewing the announcement PR[1], I found that the related Flink doc
> for Watermark Alignment Enhancement has not been updated yet. Afaiac, It is
> not a blocking issue but would be great if it contains up-to-date
> information and is consistent with the announcement.
>
> Best regards,
> Jing
>
> [1] https://github.com/apache/flink-web/pull/618
> [2]
>
> https://github.com/apache/flink/blob/release-1.17/docs/content/docs/dev/datastream/event-time/generating_watermarks.md?plain=1#L308
>
> On Sat, Mar 18, 2023 at 10:10 PM Jing Ge  wrote:
>
> > +1
> >
> > - Verified the signature
> > - Verified hashes
> > - Verify that the source archives do not contain any binaries
> > - Built from the source code
> > - Verified web PR
> > - Checked jars in the staging repo
> >
> > Best regards,
> > Jing
> >
> > On Sat, Mar 18, 2023 at 9:11 AM Lijie Wang 
> > wrote:
> >
> >> +1 (binding)
> >>
> >>  -  Built from the source code
> >>  -  Verified the signature and checksum
> >>  -  Ran both streaming/batch jobs on yarn cluster
> >>  -  Checked the website PR
> >>
> >> Best,
> >> Lijie
> >>
> >> Qingsheng Ren  于2023年3月17日周五 22:02写道：
> >>
> >> > Hi everyone,
> >> >
> >> > Please review and vote on the release candidate #3 for the version
> >> 1.17.0,
> >> > as follows:
> >> > [ ] +1, Approve the release
> >> > [ ] -1, Do not approve the release (please provide specific comments)
> >> >
> >> > The complete staging area is available for your review, which
> includes:
> >> >
> >> > * JIRA release notes [1], and the pull request adding release note for
> >> > users [2]
> >> > * the official Apache source release and binary convenience releases
> to
> >> be
> >> > deployed to dist.apache.org [3], which are signed with the key with
> >> > fingerprint A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B [4],
> >> > * all artifacts to be deployed to the Maven Central Repository [5],
> >> > * source code tag "release-1.17.0-rc3" [6],
> >> > * website pull request listing the new release and adding announcement
> >> blog
> >> > post [7].
> >> >
> >> > The vote will be open for at least 72 hours. It is adopted by majority
> >> > approval, with at least 3 PMC affirmative votes.
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351585
> >> > [2] https://github.com/apache/flink/pull/22146
> >> > [3] https://dist.apache.org/repos/dist/dev/flink/flink-1.17.0-rc3/
> >> > [4] https://dist.apache.org/repos/dist/release/flink/KEYS
> >> > [5]
> >> https://repository.apache.org/content/repositories/orgapacheflink-1600
> >> > [6] https://github.com/apache/flink/releases/tag/release-1.17.0-rc3
> >> > [7] https://github.com/apache/flink-web/pull/618
> >> >
> >> > Thanks,
> >> > Martijn and Matthias, Leonard and Qingsheng
> >> >
> >>
> >
>

[jira] [Created] (FLINK-31519) The watermark alignment docs is outdated after FLIP-217 finished

[Leonard Xu (Jira)]

Leonard Xu created FLINK-31519:
--

 Summary: The watermark alignment docs is outdated after FLIP-217 
finished
 Key: FLINK-31519
 URL: https://issues.apache.org/jira/browse/FLINK-31519
 Project: Flink
  Issue Type: Bug
  Components: Documentation
Affects Versions: 1.17.0
Reporter: Leonard Xu
 Fix For: 1.18.0, 1.17.1


With FLIP-217 finished, the watermark alignment limitation has been resolved, 
the *beta* tag and *warning note* can be safely removed.

[1] 
https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/dev/datastream/event-time/generating_watermarks/#watermark-alignment-_beta_



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] FLIP-301: Hybrid Shuffle supports Remote Storage

[Yuxin Tan]

Hi all,

Thanks for all the feedback and suggestions so far.

The discussion has been going on for some time. If there are no
further comments, we will start voting today.

Best,
Yuxin


Yuxin Tan  于2023年3月17日周五 12:52写道：

> Hi,
> Thanks for joining the discussion and giving the ideas.
>
> @ron
> > can the Hybrid Shuffle replace the RSS in the future?
>
> The hybrid shuffle and RSS offer distinct solutions to address
> the shuffle operation challenge. To optimize performance, we
> store shuffle data in different tiers of memory and disk, enabling
> greater flexibility and ease of use. Specifically, we cache
> intermediate data in memory to reduce disk I/O overhead.
> In contrast, RSS is a standalone service that can operate across
> multiple servers within a cluster, parallelizing shuffle operations
> to enhance performance. However, this introduces additional
> deployment and maintenance costs. Each approach has its own
> benefits and drawbacks, and users should be able to select the
> method that best suits their needs. So I think we cannot replace
> RSS in the future.
>
> @ConradJam
> > Should we define a data acceleration layer like Alluxio in remote
> storage?
>
> I'm not entirely clear on the detailed plan you've proposed, but I
> understand that you want to use Alluxio to serve as a cache layer for
> the remote stoarge tier. It's designed to provide low-latency data
> access to applications through a distributed caching layer. However,
> implementing Alluxio introduces additional dependencies and
> deployment/maintenance costs for users. While our design approach
> is to supplement local storage with remote storage, as local storage
> is generally sufficient. Given the limited usage scenarios, introducing
> such costs for optimization may not be worthwhile or meaningful.
> Additionally, for users, added dependencies imply increased complexity.
>
> Best,
> Yuxin
>
>
> ConradJam  于2023年3月17日周五 11:11写道：
>
>> Thanks for your start this discuss
>>
>>
>> Here I am a bit confused about the memory layer definition. This refers to
>> local memory. Should we define a data acceleration layer like Alluxio [1]
>> in remote storage?
>>
>>
>> Let me cite a scenario: If I use Fluid [2] to mount an AlluxioRuntime [3]
>> on K8S, it looks like a local disk (but it is actually a remote memory
>> storage), Have we specified this behavior or optimized it for this
>> scenario?
>>
>>
>> [1]  What is alluxio :
>> https://docs.alluxio.io/os/user/stable/en/Overview.html
>>
>> [2]  Fluid: https://fluid-cloudnative.github.io/
>>
>> [3]  Fluid Alluxio Runtime:
>>
>> https://fluid-cloudnative.github.io/samples/tieredstore_config.html#prerequisites
>>
>> liu ron  于2023年3月17日周五 10:39写道：
>>
>> > Hi, Yuxin,
>> >
>> > Thanks for creating this FLIP. Adding remote storage capability to
>> Flink's
>> > Hybrid Shuffle is a significant improvement that addresses the issue of
>> > local disk storage limitations, this also can improve the stability of
>> > Flink Batch Job.
>> > I just have one question: can the Hybrid Shuffle replace the RSS in the
>> > future? Due to the Hybrid Shuffle having remote storage ability, I think
>> > maybe we don't need to maintain a standalone RSS, it will simplify our
>> > operation work.
>> >
>>
>>
>> --
>> Best
>>
>> ConradJam
>>
>

Re: [VOTE] Release 1.17.0, release candidate #3

[Jing Ge]

Hi Qingsheng,

While reviewing the announcement PR[1], I found that the related Flink doc
for Watermark Alignment Enhancement has not been updated yet. Afaiac, It is
not a blocking issue but would be great if it contains up-to-date
information and is consistent with the announcement.

Best regards,
Jing

[1] https://github.com/apache/flink-web/pull/618
[2]
https://github.com/apache/flink/blob/release-1.17/docs/content/docs/dev/datastream/event-time/generating_watermarks.md?plain=1#L308

On Sat, Mar 18, 2023 at 10:10 PM Jing Ge  wrote:

> +1
>
> - Verified the signature
> - Verified hashes
> - Verify that the source archives do not contain any binaries
> - Built from the source code
> - Verified web PR
> - Checked jars in the staging repo
>
> Best regards,
> Jing
>
> On Sat, Mar 18, 2023 at 9:11 AM Lijie Wang 
> wrote:
>
>> +1 (binding)
>>
>>  -  Built from the source code
>>  -  Verified the signature and checksum
>>  -  Ran both streaming/batch jobs on yarn cluster
>>  -  Checked the website PR
>>
>> Best,
>> Lijie
>>
>> Qingsheng Ren  于2023年3月17日周五 22:02写道：
>>
>> > Hi everyone,
>> >
>> > Please review and vote on the release candidate #3 for the version
>> 1.17.0,
>> > as follows:
>> > [ ] +1, Approve the release
>> > [ ] -1, Do not approve the release (please provide specific comments)
>> >
>> > The complete staging area is available for your review, which includes:
>> >
>> > * JIRA release notes [1], and the pull request adding release note for
>> > users [2]
>> > * the official Apache source release and binary convenience releases to
>> be
>> > deployed to dist.apache.org [3], which are signed with the key with
>> > fingerprint A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B [4],
>> > * all artifacts to be deployed to the Maven Central Repository [5],
>> > * source code tag "release-1.17.0-rc3" [6],
>> > * website pull request listing the new release and adding announcement
>> blog
>> > post [7].
>> >
>> > The vote will be open for at least 72 hours. It is adopted by majority
>> > approval, with at least 3 PMC affirmative votes.
>> >
>> > [1]
>> >
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12351585
>> > [2] https://github.com/apache/flink/pull/22146
>> > [3] https://dist.apache.org/repos/dist/dev/flink/flink-1.17.0-rc3/
>> > [4] https://dist.apache.org/repos/dist/release/flink/KEYS
>> > [5]
>> https://repository.apache.org/content/repositories/orgapacheflink-1600
>> > [6] https://github.com/apache/flink/releases/tag/release-1.17.0-rc3
>> > [7] https://github.com/apache/flink-web/pull/618
>> >
>> > Thanks,
>> > Martijn and Matthias, Leonard and Qingsheng
>> >
>>
>

Re: [DISCUSS] FLIP-304: Pluggable failure handling for Apache Flink

[David Morávek]

Hi Panagiotis,

This is an excellent proposal and something everyone trying to provide
"Flink as a service" needs to solve at some point. I have a couple of
questions:

If I understand the proposal correctly, this is just about adding tags to
the Throwable by running a tuple of (Throwable, FailureContext) through a
user-defined model.

1) Should we also consider adding labels? The combination of tags and
labels seems to be what most systems offer; sometimes, they offer labels
only (key=value pairs) because tags can be implemented using those, but not
the other way around.

2) Since we can not predict how heavy user-defined models ("listeners") are
going to be, it would be great to keep the interfaces/data structures
immutable so we can push things over to the I/O threads. Also, it sounds
off to call the main interface a Listener since it's supposed to enhance
the original throwable with additional metadata.

I'd propose something along the lines of (we should have better names, this
is just to outline the idea):

interface FailureEnricher {

  ThrowableWithTagsAndLabels enrichFailure(Throwable cause,
ImmutableContextualMetadataAboutTheThrowable context);
}

The names should change; this is just to outline the idea.

3) You're proposing to support a set of listeners. Since you're passing the
mutable context around, which includes tags set by the previous listener,
do you expect users to make any assumptions about the order in which
listeners are executed?

*@Shammon*

Users may want to save results in listener, and then they can get the
> historical results even jabmanager failover. Can we provide a unified
> implementation for data storage requirements?


I think we should explicitly state that all "listeners" are treated as
stateless. I don't see any strong reason for snapshotting them.

Best,
D.

On Sat, Mar 18, 2023 at 1:00 AM Shammon FY  wrote:

> Hi Panagiotis
>
> Thank you for starting this discussion. I think this FLIP is valuable and
> can help user to analyze the causes of job failover better!
>
> I have two comments as follows
>
> 1. How about adding more job information in FailureListenerContext? For
> example, job vertext, subtask, taskmanager location. And then user can do
> more statistics according to different dimensions.
>
> 2. Users may want to save results in listener, and then they can get the
> historical results even jabmanager failover. Can we provide a unified
> implementation for data storage requirements?
>
>
> Best,
> shammon FY
>
>
> On Saturday, March 18, 2023, Panagiotis Garefalakis 
> wrote:
>
> > Hi everyone,
> >
> > This FLIP [1] proposes a pluggable interface for failure handling
> allowing
> > users to implement custom failure logic using the plugin framework.
> > Motivated by existing proposals [2] and tickets [3], this enables
> use-cases
> > like: assigning particular types to failures (e.g., User or System),
> > emitting custom metrics per type (e.g., application or platform), even
> > exposing errors to downstream consumers (e.g., notification systems).
> >
> > Thanks to Piotr and Anton for the initial reviews and discussions!
> >
> > For anyone interested, the starting point would be the FLIP [1] that I
> > created,
> > describing the motivation and the proposed changes (part of the core,
> > runtime and web).
> >
> > The intuition behind this FLIP is being able to execute custom logic on
> > failures by exposing a FailureListener interface. Implementation by users
> > can be simply loaded to the system as Jar files. FailureListeners may
> also
> > decide to assign failure tags to errors (expressed as strings),
> > that will then be exposed as metadata by the UI/Rest interfaces.
> >
> > Feedback is always appreciated! Looking forward to your thoughts!
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-304%
> > 3A+Pluggable+failure+handling+for+Apache+Flink
> > [2]
> > https://docs.google.com/document/d/1pcHg9F3GoDDeVD5GIIo2wO67
> > Hmjgy0-hRDeuFnrMgT4
> > [3] https://issues.apache.org/jira/browse/FLINK-20833
> >
> > Cheers,
> > Panagiotis
> >
>

[jira] [Created] (FLINK-31518) HighAvailabilityServiceUtils resolves wrong webMonitorAddress in StandaloneHA

[Vineeth Naroju (Jira)]

Vineeth Naroju created FLINK-31518:
--

 Summary: HighAvailabilityServiceUtils resolves wrong 
webMonitorAddress in StandaloneHA
 Key: FLINK-31518
 URL: https://issues.apache.org/jira/browse/FLINK-31518
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Configuration, Runtime / REST
Affects Versions: 1.16.1, 1.16.0, 1.17.0, 1.17.1
Reporter: Vineeth Naroju


{{HighAvailabilityServiceUtils#createHighAvailabilityServices()}}  in 
{{HighAvailabilityMode.NONE}} mode uses 
{{HighAvailabilityServiceUtils#getWebMonitorAddress()}} to get web monitor 
address.

{{HighAvailabilityServiceUtils#getWebMonitorAddress()}} reads only from 
{{rest.port}} in {{flink-conf.yaml}} . If {{rest.port}} is not enabled, then it 
returns {{0}} port number. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31517) Move execution logic of AddJarOperation out from TableEnvironmentImpl

[Jark Wu (Jira)]

Jark Wu created FLINK-31517:
---

 Summary: Move execution logic of AddJarOperation out from 
TableEnvironmentImpl
 Key: FLINK-31517
 URL: https://issues.apache.org/jira/browse/FLINK-31517
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: Jark Wu


This should implement ExecutableOperation for the {{AddJarOperation}} to move 
the execution logic out from {{TableEnvironmentImpl#executeInternal()}}.





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31516) Move execution logic of LoadModule and UnloadModule operations out from TableEnvironmentImpl

[Jark Wu (Jira)]

Jark Wu created FLINK-31516:
---

 Summary: Move execution logic of LoadModule and UnloadModule 
operations out from TableEnvironmentImpl
 Key: FLINK-31516
 URL: https://issues.apache.org/jira/browse/FLINK-31516
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: Jark Wu


This should implement ExecutableOperation for LoadModule and UnloadModule 
operations to move the execution logic out from 
{{TableEnvironmentImpl#executeInternal().





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31515) Move execution logic of AnalyzeTableOperation out from TableEnvironmentImpl

[Jark Wu (Jira)]

Jark Wu created FLINK-31515:
---

 Summary: Move execution logic of AnalyzeTableOperation out from 
TableEnvironmentImpl
 Key: FLINK-31515
 URL: https://issues.apache.org/jira/browse/FLINK-31515
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: Jark Wu


This should implement ExecutableOperation for the AnalyzeTableOperation to move 
the execution logic out from {{TableEnvironmentImpl#executeInternal().





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31514) Move query SqlNode conversion logic to SqlQueryConverter

[Jark Wu (Jira)]

Jark Wu created FLINK-31514:
---

 Summary: Move query SqlNode conversion logic to SqlQueryConverter
 Key: FLINK-31514
 URL: https://issues.apache.org/jira/browse/FLINK-31514
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: Jark Wu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-31513) [DOC] update a path in README

[jack Lei (Jira)]

jack Lei created FLINK-31513:


 Summary: [DOC] update a path in README
 Key: FLINK-31513
 URL: https://issues.apache.org/jira/browse/FLINK-31513
 Project: Flink
  Issue Type: Improvement
Reporter: jack Lei
 Attachments: 截屏2023-03-19 15.58.45.png

when run the flink-tpcds-test, find a path in readme need to update



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [Discussion] - Release major Flink version to support JDK 17 (LTS)

[Tamir Sagi]

I agree, there are several options to mitigate the migration from v2 to v5.
yet, Oracle roadmap is to end JDK 11 support in September this year.




From: ConradJam 
Sent: Thursday, March 16, 2023 4:36 AM
To: dev@flink.apache.org 
Subject: Re: [Discussion] - Release major Flink version to support JDK 17 (LTS)

EXTERNAL EMAIL



Thanks for your start this discuss


I have been tracking this problem for a long time, until I saw a
conversation in ISSUSE a few days ago and learned that the Kryo version
problem will affect the JDK17 compilation of snapshots [1] FLINK-24998 ,

As @cherry said it ruined our whole effort towards JDK17

I am in favor of providing an external tool to migrate from Kryo old
version checkpoint to the new Kryo new checkpoint at one time (Maybe this
tool start in flink 2.0 ?), does this tool currently have any plans or
ideas worth discuss


I think it should not be difficult to be compatible with JDK11 and JDK17.
We should indeed abandon JDK8 in 2.0.0. It is also mentioned in the doc
that it is marked as Deprecated [2]


Here I add that we need to pay attention to the version of Scala and the
version of JDK17


[1] FLINK-24998  IGSEGV in Kryo / C2 CompilerThread on Java 17
https://issues.apache.org/jira/browse/FLINK-24998

[2] FLINK-30501 Update Flink build instruction to deprecate Java 8 instead
of requiring Java 11  https://issues.apache.org/jira/browse/FLINK-30501

Tamir Sagi  于2023年3月16日周四 00:54写道：

> Hey dev community,
>
> I'm writing this email to kick off a discussion following this epic:
> FLINK-15736.
>
> We are moving towards JDK 17 (LTS) , the only blocker now is Flink which
> currently remains on JDK 11 (LTS). Flink does not support JDK 17 yet, with
> no timeline,  the reason, based on the aforementioned ticket is the
> following tickets
>
>   1.  FLINK-24998 - SIGSEGV in Kryo / C2 CompilerThread on Java 17<
> https://issues.apache.org/jira/browse/FLINK-24998>.
>   2.  FLINK-3154 - Update Kryo version from 2.24.0 to latest Kryo LTS
> version
>
> My question is whether it is possible to release a major version (Flink
> 2.0.0) using the latest Kryo version for those who don't need to restore
> old savepoints/checkpoints in newer format.
>
>   1.  Leverage JDK 17 features within JVM
>   2.  Moving from the old format to the newer one will be handled only
> once - a mitigation can be achieved by a conversion tool or external
> serializers, both can be provided later on.
>
> I'd like to emphasize that the next JDK LTS (21) will be released this
> September.  furthermore, Flink already supports JDK 12-15, which is very
> close to JDK 17 (LTS) - that was released in September 2021.  JDK 11 will
> become a legacy soon, as more frameworks moving towards JDK 17 and are less
> likely to support JDK 11 in the near future. (For example, Spring Boot 3
> requires JDK 17 already).
>
> Thank you for your consideration of my request.
>
> Tamir.
>
>
>
>
> Confidentiality: This communication and any attachments are intended for
> the above-named persons only and may be confidential and/or legally
> privileged. Any opinions expressed in this communication are not
> necessarily those of NICE Actimize. If this communication has come to you
> in error you must take no action based on it, nor must you copy or show it
> to anyone; please delete/destroy and inform the sender by e-mail
> immediately.
> Monitoring: NICE Actimize may monitor incoming and outgoing e-mails.
> Viruses: Although we have taken steps toward ensuring that this e-mail and
> attachments are free from any virus, we advise that in keeping with good
> computing practice the recipient should ensure they are actually virus free.
>


--
Best

ConradJam

Confidentiality: This communication and any attachments are intended for the 
above-named persons only and may be confidential and/or legally privileged. Any 
opinions expressed in this communication are not necessarily those of NICE 
Actimize. If this communication has come to you in error you must take no 
action based on it, nor must you copy or show it to anyone; please 
delete/destroy and inform the sender by e-mail immediately.
Monitoring: NICE Actimize may monitor incoming and outgoing e-mails.
Viruses: Although we have taken steps toward ensuring that this e-mail and 
attachments are free from any virus, we advise that in keeping with good 
computing practice the recipient should ensure they are actually virus free.

[jira] [Created] (FLINK-31512) Move SqlAlterView conversion logic to SqlAlterViewConverter

[Jark Wu (Jira)]

Jark Wu created FLINK-31512:
---

 Summary: Move SqlAlterView conversion logic to 
SqlAlterViewConverter
 Key: FLINK-31512
 URL: https://issues.apache.org/jira/browse/FLINK-31512
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: Jark Wu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)