[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of these protobufs messages are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket [https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0] All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 and a path for snapshot migration https://issues.apache.org/jira/browse/FLINK-3154. was: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket [https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0] All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 and a path for snapshot migration https://issues.apache.org/jira/browse/FLINK-3154. > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > *Background:* > Flink supports protobuf dataformat to exchange messages between different > operators and the serialization and deserialization of these protobufs > messages are performed by library called "Kryo". In order to move away from > experimental support of Java 17 released as part of Flink 1.18.1, the Kryo > library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo > 2.24.0 does not support Java
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket [https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0] All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 and a path for snapshot migration https://issues.apache.org/jira/browse/FLINK-3154. was: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. Migration Plan is documented here: [https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0] *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 and a path for snapshot migration https://issues.apache.org/jira/browse/FLINK-3154. > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > *Background:* > Flink supports protobuf dataformat to exchange messages between different > operators and the serialization and deserialization of those protobufs are > performed by library called "Kryo". In order to move away from experimental > support of Java 17 released as part of Flink 1.18.1, the Kryo library in > Flink 1.18.1 needs to be
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. Migration Plan is documented here: [https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0] *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 and a path for snapshot migration https://issues.apache.org/jira/browse/FLINK-3154. was: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. Migration Plan is documented here: [https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0] *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 and snapshot migration https://issues.apache.org/jira/browse/FLINK-3154. > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > *Background:* > Flink supports protobuf dataformat to exchange messages between different > operators and the serialization and deserialization of those protobufs are > performed by library called "Kryo". In order to move away from experimental > support of Java 17 released
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. Migration Plan is documented here: [https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0] *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 and snapshot migration https://issues.apache.org/jira/browse/FLINK-3154. was: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. Migration Plan is documented here: https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0 *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 https://issues.apache.org/jira/browse/FLINK-3154. > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > *Background:* > Flink supports protobuf dataformat to exchange messages between different > operators and the serialization and deserialization of those protobufs are > performed by library called "Kryo". In order to move away from experimental > support of Java 17 released as part of Flink 1.18.1, the Kryo
[jira] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491 ]
Dhruv Patel deleted comment on FLINK-34491:
-
was (Author: JIRAUSER289387):
Following issue has been observed in after enabling SSL in flink. Since after
migration flink uses tls1.3 as default
|Change |Description | |
|SSL / TLS v1.3|the handshake between the flink components now uses TLS v1.3
with Cipher:
TLS_AES_256_GCM_SHA384
which is causing SSL handshake failures.
{code:java}
SSL3 alert read:fatal:handshake failure
SSL_connect:error in error
409B7454F87F:error:0A000410:SSL routines:ssl3_read_bytes:sslv3 alert
handshake failure:ssl/record/rec_layer_s3.c:1586:SSL alert number 40
–
Server Temp Key: ECDH, prime256v1, 256 bits
—
SSL handshake has read 470 bytes and written 730 bytes
Verification: OK
—
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
This TLS version forbids renegotiation.| {code}| |
> Move from experimental support to production support for Java 17
>
>
> Key: FLINK-34491
> URL: https://issues.apache.org/jira/browse/FLINK-34491
> Project: Flink
> Issue Type: New Feature
>Affects Versions: 1.18.1
>Reporter: Dhruv Patel
>Priority: Major
>
> This task is to move away from experimental support for Java 17 to production
> support so that teams running Flink in production can migrate to Java 17
> successfully
> *Background:*
> Flink supports protobuf dataformat to exchange messages between different
> operators and the serialization and deserialization of those protobufs are
> performed by library called "Kryo". In order to move away from experimental
> support of Java 17 released as part of Flink 1.18.1, the Kryo library in
> Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0
> does not support Java 17. This improvement plan is tracked as part of this
> ticket https://issues.apache.org/jira/browse/FLINK-3154.
> All Flink applications using protobuf currently generate state with Kryo v2.
> Once the above improvement plan is complete all Flink applications will fully
> support reading that state and write newer state with Kryo v5. However,
> latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This
> will prevent applications which are using snapshot mechanism to deploy their
> jobs to latest Flink version with Kryo v5 support without a bridge version
> running on Java 11. Applications will have to run on a bridge release version
> that will read their state with Kryo v2 data and write it with Kryo v5 data
> before upgrading to a future version of Flink that completely drops support
> for Kryo v2.
> Basically, Flink applications using protobuf dataformat cannot move directly
> from Java 8 to Java 17 without downtime after the kryo v5 release in Flink.
> Applications will need to first move to Java 11 (bridging version) and then
> move to Java 17 to have a safe deployment.
> Migration Plan is documented here:
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0
> *Blocker for this task:*
> Upgrade to Kryo 5.5.0 which supports Java 17
> https://issues.apache.org/jira/browse/FLINK-3154.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime after the kryo v5 release in Flink. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. Migration Plan is documented here: https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0 *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 https://issues.apache.org/jira/browse/FLINK-3154. was: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 https://issues.apache.org/jira/browse/FLINK-3154. > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > *Background:* > Flink supports protobuf dataformat to exchange messages between different > operators and the serialization and deserialization of those protobufs are > performed by library called "Kryo". In order to move away from experimental > support of Java 17 released as part of Flink 1.18.1, the Kryo library in > Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 > does not support Java 17. This improvement plan is tracked as part of this > ticket
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. Basically, Flink applications using protobuf dataformat cannot move directly from Java 8 to Java 17 without downtime. Applications will need to first move to Java 11 (bridging version) and then move to Java 17 to have a safe deployment. *Blocker for this task:* Upgrade to Kryo 5.5.0 which supports Java 17 https://issues.apache.org/jira/browse/FLINK-3154. was: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. *Blocker for this task:* Below are some of the blocker task which prevent certain Flink applications which use protobuf from upgrading to Java 17 *Blocker 1:* Upgrade to Kryo 5.5.0 which supports Java 17 https://issues.apache.org/jira/browse/FLINK-3154. *Blocker 2:* Validate whether a snapshot with Java 8 is compatible with Java 17 without using a bridge version running on Java 11. https://issues.apache.org/jira/browse/FLINK-33707 > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > *Background:* > Flink supports protobuf dataformat to exchange messages between different > operators and the serialization and deserialization of those protobufs are > performed by library called "Kryo". In order to move away from experimental > support of Java 17 released as part of Flink 1.18.1, the Kryo library in > Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 > does not support Java 17. This improvement plan is tracked as part of this > ticket https://issues.apache.org/jira/browse/FLINK-3154. > All Flink applications using protobuf currently generate
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Background:* Flink supports protobuf dataformat to exchange messages between different operators and the serialization and deserialization of those protobufs are performed by library called "Kryo". In order to move away from experimental support of Java 17 released as part of Flink 1.18.1, the Kryo library in Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 does not support Java 17. This improvement plan is tracked as part of this ticket https://issues.apache.org/jira/browse/FLINK-3154. All Flink applications using protobuf currently generate state with Kryo v2. Once the above improvement plan is complete all Flink applications will fully support reading that state and write newer state with Kryo v5. However, latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This will prevent applications which are using snapshot mechanism to deploy their jobs to latest Flink version with Kryo v5 support without a bridge version running on Java 11. Applications will have to run on a bridge release version that will read their state with Kryo v2 data and write it with Kryo v5 data before upgrading to a future version of Flink that completely drops support for Kryo v2. *Blocker for this task:* Below are some of the blocker task which prevent certain Flink applications which use protobuf from upgrading to Java 17 *Blocker 1:* Upgrade to Kryo 5.5.0 which supports Java 17 https://issues.apache.org/jira/browse/FLINK-3154. *Blocker 2:* Validate whether a snapshot with Java 8 is compatible with Java 17 without using a bridge version running on Java 11. https://issues.apache.org/jira/browse/FLINK-33707 was: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Blocker for this task:* Savepoint migration is not supported with Java 17 and Flink 1.18.1 as mentioned in this ticket https://issues.apache.org/jira/browse/FLINK-33707 > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > *Background:* > Flink supports protobuf dataformat to exchange messages between different > operators and the serialization and deserialization of those protobufs are > performed by library called "Kryo". In order to move away from experimental > support of Java 17 released as part of Flink 1.18.1, the Kryo library in > Flink 1.18.1 needs to be updated from 2.24.0 to 5.5.0 because Kryo 2.24.0 > does not support Java 17. This improvement plan is tracked as part of this > ticket https://issues.apache.org/jira/browse/FLINK-3154. > All Flink applications using protobuf currently generate state with Kryo v2. > Once the above improvement plan is complete all Flink applications will fully > support reading that state and write newer state with Kryo v5. However, > latest Kryo v5 doesn't support snapshots made by a previous Kryo v2. This > will prevent applications which are using snapshot mechanism to deploy their > jobs to latest Flink version with Kryo v5 support without a bridge version > running on Java 11. Applications will have to run on a bridge release version > that will read their state with Kryo v2 data and write it with Kryo v5 data > before upgrading to a future version of Flink that completely drops support > for Kryo v2. > *Blocker for this task:* > Below are some of the blocker task which prevent certain Flink applications > which use protobuf from upgrading to Java 17 > *Blocker 1:* Upgrade to Kryo 5.5.0 which supports Java 17 > https://issues.apache.org/jira/browse/FLINK-3154. > *Blocker 2:* Validate whether a snapshot with Java 8 is compatible with Java > 17 without using a bridge version running on Java 11. > https://issues.apache.org/jira/browse/FLINK-33707 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34491) Move from experimental support to production support for Java 17
[
https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819860#comment-17819860
]
Dhruv Patel commented on FLINK-34491:
-
Following issue has been observed in after enabling SSL in flink. Since after
migration flink uses tls1.3 as default
|Change |Description | |
|SSL / TLS v1.3|the handshake between the flink components now uses TLS v1.3
with Cipher:
TLS_AES_256_GCM_SHA384
which is causing SSL handshake failures.
{code:java}
SSL3 alert read:fatal:handshake failure
SSL_connect:error in error
409B7454F87F:error:0A000410:SSL routines:ssl3_read_bytes:sslv3 alert
handshake failure:ssl/record/rec_layer_s3.c:1586:SSL alert number 40
–
Server Temp Key: ECDH, prime256v1, 256 bits
—
SSL handshake has read 470 bytes and written 730 bytes
Verification: OK
—
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
This TLS version forbids renegotiation.| {code}| |
> Move from experimental support to production support for Java 17
>
>
> Key: FLINK-34491
> URL: https://issues.apache.org/jira/browse/FLINK-34491
> Project: Flink
> Issue Type: New Feature
>Affects Versions: 1.18.1
>Reporter: Dhruv Patel
>Priority: Major
>
> This task is to move away from experimental support for Java 17 to production
> support so that teams running Flink in production can migrate to Java 17
> successfully
>
> *Blocker for this task:*
> Savepoint migration is not supported with Java 17 and Flink 1.18.1 as
> mentioned in this ticket https://issues.apache.org/jira/browse/FLINK-33707
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully *Blocker for this task:* Savepoint migration is not supported with Java 17 and Flink 1.18.1 as mentioned in this ticket https://issues.apache.org/jira/browse/FLINK-33707 was:This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > > *Blocker for this task:* > Savepoint migration is not supported with Java 17 and Flink 1.18.1 as > mentioned in this ticket https://issues.apache.org/jira/browse/FLINK-33707 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully (was: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/java_compatibility/#untested-flink-features-1) > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/java_compatibility/#untested-flink-features-1 was:This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully > > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/java_compatibility/#untested-flink-features-1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34491) Move from experimental support to production support for Java 17
[ https://issues.apache.org/jira/browse/FLINK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-34491: Description: This task is to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully (was: Tracking task to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully) > Move from experimental support to production support for Java 17 > > > Key: FLINK-34491 > URL: https://issues.apache.org/jira/browse/FLINK-34491 > Project: Flink > Issue Type: New Feature >Affects Versions: 1.18.1 >Reporter: Dhruv Patel >Priority: Major > > This task is to move away from experimental support for Java 17 to production > support so that teams running Flink in production can migrate to Java 17 > successfully -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34491) Move from experimental support to production support for Java 17
Dhruv Patel created FLINK-34491: --- Summary: Move from experimental support to production support for Java 17 Key: FLINK-34491 URL: https://issues.apache.org/jira/browse/FLINK-34491 Project: Flink Issue Type: New Feature Affects Versions: 1.18.1 Reporter: Dhruv Patel Tracking task to move away from experimental support for Java 17 to production support so that teams running Flink in production can migrate to Java 17 successfully -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-32015) Flink does not fail SSL Handshake when expired cert is used
[ https://issues.apache.org/jira/browse/FLINK-32015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Patel updated FLINK-32015: Description: We recently observed that Flink is able to perform SSL communication without any issues even if certs in the trust store are expired As per this issue in Netty library, it says that Default TrustManagerFactory does not come with that check in place and we need to implement our own TrustManagerFactory for the desired behavior [https://github.com/netty/netty/issues/8461] It would be good to perform the cert validity check to prevent the risk of using expired certs was: We recently observed that Flink is able to perform SSL communication without any issues even if certs in the trust store are expired As per this issue in Netty library, it says that Default TrustManagerFactory does not come with that check in place and we need to implement our own TrustManagerFactory for the desired behavior https://github.com/netty/netty/issues/8461 > Flink does not fail SSL Handshake when expired cert is used > --- > > Key: FLINK-32015 > URL: https://issues.apache.org/jira/browse/FLINK-32015 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Reporter: Dhruv Patel >Priority: Major > > We recently observed that Flink is able to perform SSL communication without > any issues even if certs in the trust store are expired > As per this issue in Netty library, it says that Default TrustManagerFactory > does not come with that check in place and we need to implement our own > TrustManagerFactory for the desired behavior > [https://github.com/netty/netty/issues/8461] > It would be good to perform the cert validity check to prevent the risk of > using expired certs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32015) Flink does not fail SSL Handshake when expired cert is used
Dhruv Patel created FLINK-32015: --- Summary: Flink does not fail SSL Handshake when expired cert is used Key: FLINK-32015 URL: https://issues.apache.org/jira/browse/FLINK-32015 Project: Flink Issue Type: Bug Components: Runtime / Network Reporter: Dhruv Patel We recently observed that Flink is able to perform SSL communication without any issues even if certs in the trust store are expired As per this issue in Netty library, it says that Default TrustManagerFactory does not come with that check in place and we need to implement our own TrustManagerFactory for the desired behavior https://github.com/netty/netty/issues/8461 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29400) Default Value of env.log.max in documentation is incorrect
Dhruv Patel created FLINK-29400: --- Summary: Default Value of env.log.max in documentation is incorrect Key: FLINK-29400 URL: https://issues.apache.org/jira/browse/FLINK-29400 Project: Flink Issue Type: Improvement Components: Documentation Reporter: Dhruv Patel The default value of env.log.max is 10 as per the code in master ([https://github.com/apache/flink/blob/master/flink-dist/src/main/flink-bin/bin/config.sh#L137).] However the Flink Documentation says the default value is 5 (https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#env-log-max) which is incorrect -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-27590) Provide endpoint with information about leader Jobmanager in HA setups via Zookeeper
Dhruv Patel created FLINK-27590: --- Summary: Provide endpoint with information about leader Jobmanager in HA setups via Zookeeper Key: FLINK-27590 URL: https://issues.apache.org/jira/browse/FLINK-27590 Project: Flink Issue Type: Improvement Components: Runtime / REST Affects Versions: 1.15.0, 1.14.0 Reporter: Dhruv Patel Attachments: error.log In High Availability setups via Zookeeper, it would be nice to provide information about the leader jobmanager. Although, the job submission and all the operations can be performed via any jobmanager. It would be nice to expose data about leader jobmanager due to following reasons * Since there are multiple job managers in HA setup, each of them will expose a web ui and a rest endpoint. It is difficult to know which of the endpoint is backed by active jobmanager. Moreover, we cannot access the taskmanager logs from the flink ui exposed by the standby jobmanager. [^error.log] * If the flink UI/REST endpoint is fronted by a Load Balancer, it would be nice to detect active jobmanager so that all the UI requests can be routed to the active jobmanager to avoid having issues in accessing logs -- This message was sent by Atlassian Jira (v8.20.7#820007)
