[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[ https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789027#comment-17789027 ] Ryan Skraba commented on FLINK-33149: - Hello! Thanks for bringing this back. I've checked all of the connectors (as well as the Hive connector currently being externalized) and the versions of Snappy are 1.1.10.4 or 1.1.10.5. I created FLINK-33627 for flink-statefun, however, so this JIRA can be closed for Flink core and connectors. > Bump snappy-java to 1.1.10.4 > > > Key: FLINK-33149 > URL: https://issues.apache.org/jira/browse/FLINK-33149 > Project: Flink > Issue Type: Bug > Components: API / Core, Connectors / AWS, Connectors / HBase, > Connectors / Kafka, Stateful Functions >Affects Versions: 1.18.0, 1.16.3, 1.17.2 >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0, kafka-4.0.0, 1.16.3, 1.17.2 > > > Xerial published a security alert for a Denial of Service attack that [exists > on > 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv]. > This is included in flink-dist, but also in flink-statefun, and several > connectors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[ https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789014#comment-17789014 ] Matthias Pohl commented on FLINK-33149: --- [~rskraba] wanted to work on bringing this update to the connectors as well. Ryan: Could you create dedicated tickets for that so that we could close this one? I guess that would make sense because 1.18.0 was already released. > Bump snappy-java to 1.1.10.4 > > > Key: FLINK-33149 > URL: https://issues.apache.org/jira/browse/FLINK-33149 > Project: Flink > Issue Type: Bug > Components: API / Core, Connectors / AWS, Connectors / HBase, > Connectors / Kafka, Stateful Functions >Affects Versions: 1.18.0, 1.16.3, 1.17.2 >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0, kafka-4.0.0, 1.16.3, 1.17.2 > > > Xerial published a security alert for a Denial of Service attack that [exists > on > 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv]. > This is included in flink-dist, but also in flink-statefun, and several > connectors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[ https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788969#comment-17788969 ] Yun Tang commented on FLINK-33149: -- [~mapohl] When can we close this ticket? > Bump snappy-java to 1.1.10.4 > > > Key: FLINK-33149 > URL: https://issues.apache.org/jira/browse/FLINK-33149 > Project: Flink > Issue Type: Bug > Components: API / Core, Connectors / AWS, Connectors / HBase, > Connectors / Kafka, Stateful Functions >Affects Versions: 1.18.0, 1.16.3, 1.17.2 >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0, kafka-4.0.0, 1.16.3, 1.17.2 > > > Xerial published a security alert for a Denial of Service attack that [exists > on > 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv]. > This is included in flink-dist, but also in flink-statefun, and several > connectors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[ https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776130#comment-17776130 ] Matthias Pohl commented on FLINK-33149: --- Reverted the fixVersion change back to 1.18.0 (and added the other fix version). The issue is resolved in 1.18.0 already. This Jira issues isn't closed, yet, due to the open work on the connectors as far as I understand. > Bump snappy-java to 1.1.10.4 > > > Key: FLINK-33149 > URL: https://issues.apache.org/jira/browse/FLINK-33149 > Project: Flink > Issue Type: Bug > Components: API / Core, Connectors / AWS, Connectors / HBase, > Connectors / Kafka, Stateful Functions >Affects Versions: 1.18.0, 1.16.3, 1.17.2 >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0, kafka-4.0.0, 1.16.3, 1.17.2 > > > Xerial published a security alert for a Denial of Service attack that [exists > on > 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv]. > This is included in flink-dist, but also in flink-statefun, and several > connectors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[ https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773686#comment-17773686 ] Martijn Visser commented on FLINK-33149: Merged in: apache/flink-connector-kafka:main 73f761fa73d4200d18f628eef7c79cf91dd1a0bc > Bump snappy-java to 1.1.10.4 > > > Key: FLINK-33149 > URL: https://issues.apache.org/jira/browse/FLINK-33149 > Project: Flink > Issue Type: Bug > Components: API / Core, Connectors / AWS, Connectors / HBase, > Connectors / Kafka, Stateful Functions >Affects Versions: 1.18.0, 1.16.3, 1.17.2 >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0, kafka-3.1.0, 1.16.3, 1.17.2, 1.19.0 > > > Xerial published a security alert for a Denial of Service attack that [exists > on > 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv]. > This is included in flink-dist, but also in flink-statefun, and several > connectors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[ https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769933#comment-17769933 ] Ryan Skraba commented on FLINK-33149: - Thanks for taking a look! I'm going through the Flink connector repos as well (I also did the flink-statefun repo, but so did Dependabot :D ) Kafka is currently bumping their implementation and it looks like a safe "drop-in" replacement (with some minor testing issues). Avro *just missed* the upgrade with their last 1.11.3 release, but I've confirmed locally that it's safe to override the snappy dependency there as well. It will be present in their next release. I'll include a summary here about the connectors when I've finished taking a look. > Bump snappy-java to 1.1.10.4 > > > Key: FLINK-33149 > URL: https://issues.apache.org/jira/browse/FLINK-33149 > Project: Flink > Issue Type: Bug > Components: API / Core, Connectors / AWS, Connectors / HBase, > Connectors / Kafka, Stateful Functions >Affects Versions: 1.18.0, 1.16.3, 1.17.2 >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Labels: pull-request-available > Fix For: 1.18.0, 1.16.3, 1.17.2, 1.19.0 > > > Xerial published a security alert for a Denial of Service attack that [exists > on > 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv]. > This is included in flink-dist, but also in flink-statefun, and several > connectors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[ https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769678#comment-17769678 ] Sergey Nuyanzin commented on FLINK-33149: - Merged to flink main repo master as [ec6ebe2d22d15883f7236895387a45a533cfefe0|https://github.com/apache/flink/commit/ec6ebe2d22d15883f7236895387a45a533cfefe0] > Bump snappy-java to 1.1.10.4 > > > Key: FLINK-33149 > URL: https://issues.apache.org/jira/browse/FLINK-33149 > Project: Flink > Issue Type: Bug > Components: API / Core, Connectors / AWS, Connectors / HBase, > Connectors / Kafka, Stateful Functions >Affects Versions: 1.18.0, 1.16.3, 1.17.2 >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Labels: pull-request-available > > Xerial published a security alert for a Denial of Service attack that [exists > on > 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv]. > This is included in flink-dist, but also in flink-statefun, and several > connectors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[
https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768878#comment-17768878
]
Matthias Pohl commented on FLINK-33149:
---
I think my previous investigation is not enough: It seems to be used by
{{flink-avro}}, {{flink-parquet}} and {{flink-presto}}, as well.
> Bump snappy-java to 1.1.10.4
>
>
> Key: FLINK-33149
> URL: https://issues.apache.org/jira/browse/FLINK-33149
> Project: Flink
> Issue Type: Bug
> Components: API / Core, Connectors / AWS, Connectors / HBase,
> Connectors / Kafka, Stateful Functions
>Affects Versions: 1.18.0, 1.16.3, 1.17.2
>Reporter: Ryan Skraba
>Assignee: Ryan Skraba
>Priority: Major
> Labels: pull-request-available
>
> Xerial published a security alert for a Denial of Service attack that [exists
> on
> 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv].
> This is included in flink-dist, but also in flink-statefun, and several
> connectors.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-33149) Bump snappy-java to 1.1.10.4
[
https://issues.apache.org/jira/browse/FLINK-33149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768867#comment-17768867
]
Matthias Pohl commented on FLINK-33149:
---
Thanks for looking into it. I did a code investigation to see where we use
snappy in flink core.
Snappy was introduced for the state backend and used in
[SnappyStreamCompressionDecorator.java:25-26|https://github.com/apache/flink/blob/116f297478f2d443178510565b1cd5a2f387e241/flink-runtime/src/main/java/org/apache/flink/runtime/state/SnappyStreamCompressionDecorator.java#L25].
The classes that are affected by this vulnerability ({{SnappyInputStream}} and
{{SnappyOutputStream}}) are not used. Flink uses {{SnappyFramedInputStream}}
and {{SnappyFramedOutputStream}}. Therefore, it's not critical and priority
Major makes sense. But it's still good to have this fixed considering the
alerts that might pop up in security scanners.
I also did a brief analysis of a few connector implementations:
{code}
➜ workspace for c in $(ls -d flink-connector*); do echo $c; grep
--include=pom.xml -Hirn snappy $c; done
flink-connector-aws
flink-connector-aws/pom.xml:254:
org.xerial.snappy
flink-connector-aws/pom.xml:255:
snappy-java
flink-connector-cassandra
flink-connector-elasticsearch
flink-connector-gcp-pubsub
flink-connector-hbase
flink-connector-hbase/pom.xml:245:
org.xerial.snappy
flink-connector-hbase/pom.xml:246:
snappy-java
flink-connector-hive
flink-connector-jdbc
flink-connector-kafka
flink-connector-kafka/pom.xml:70:
1.1.8.3
flink-connector-kafka/pom.xml:231:
org.xerial.snappy
flink-connector-kafka/pom.xml:232:
snappy-java
flink-connector-kafka/pom.xml:233:
${snappy-java.version}
flink-connector-mongodb
flink-connector-opensearch
flink-connector-pulsar
flink-connector-rabbitmq
flink-connector-redis-streams
{code}
Only {{flink-connector-kafka}} and {{flink-connector-aws}} have this dependency
listed. None of them actually uses any classes from within the {{xerial}}
package:
{code}
for c in $(ls -d flink-connector*); do echo $c; grep --include="*java" -Hirn
xerial $c; done
flink-connector-aws
flink-connector-cassandra
flink-connector-elasticsearch
flink-connector-gcp-pubsub
flink-connector-hbase
flink-connector-hive
flink-connector-jdbc
flink-connector-kafka
flink-connector-mongodb
flink-connector-opensearch
flink-connector-pulsar
flink-connector-rabbitmq
flink-connector-redis-streams
{code}
Would it be worth removing the dependency from the connectors entirely? WDYT?
> Bump snappy-java to 1.1.10.4
>
>
> Key: FLINK-33149
> URL: https://issues.apache.org/jira/browse/FLINK-33149
> Project: Flink
> Issue Type: Bug
> Components: API / Core, Connectors / AWS, Connectors / HBase,
> Connectors / Kafka, Stateful Functions
>Affects Versions: 1.18.0, 1.16.3, 1.17.2
>Reporter: Ryan Skraba
>Assignee: Ryan Skraba
>Priority: Major
> Labels: pull-request-available
>
> Xerial published a security alert for a Denial of Service attack that [exists
> on
> 1.1.10.1|https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv].
> This is included in flink-dist, but also in flink-statefun, and several
> connectors.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
