[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.7.0

2021-01-17 Thread Apache Spark (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17267028#comment-17267028
 ] 

Apache Spark commented on SPARK-33913:
--

User 'dongjoon-hyun' has created a pull request for this issue:
https://github.com/apache/spark/pull/31223

> Upgrade Kafka to 2.7.0
> --
>
> Key: SPARK-33913
> URL: https://issues.apache.org/jira/browse/SPARK-33913
> Project: Spark
>  Issue Type: Improvement
>  Components: Build, DStreams
>Affects Versions: 3.2.0
>Reporter: dengziming
>Priority: Major
>
>  
> The Apache Kafka community has released for Apache Kafka 2.7.0, some features 
> are useful for example the KAFKA-9893
>  configurable TCP connection timeout, more details : 
> https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.7.0

2021-01-17 Thread Apache Spark (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17267027#comment-17267027
 ] 

Apache Spark commented on SPARK-33913:
--

User 'dongjoon-hyun' has created a pull request for this issue:
https://github.com/apache/spark/pull/31223

> Upgrade Kafka to 2.7.0
> --
>
> Key: SPARK-33913
> URL: https://issues.apache.org/jira/browse/SPARK-33913
> Project: Spark
>  Issue Type: Improvement
>  Components: Build, DStreams
>Affects Versions: 3.2.0
>Reporter: dengziming
>Priority: Major
>
>  
> The Apache Kafka community has released for Apache Kafka 2.7.0, some features 
> are useful for example the KAFKA-9893
>  configurable TCP connection timeout, more details : 
> https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.7.0

2020-12-27 Thread L. C. Hsieh (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255395#comment-17255395
 ] 

L. C. Hsieh commented on SPARK-33913:
-

Seems we cannot upgrade to Kafka 2.7.0. Because Kafka core inlines the Scala 
library, you can not use a different Scala patch version than what Kafka used 
to compile its jars: https://github.com/embeddedkafka/embedded-kafka/issues/202

Kafka 2.7.0 uses Scala 2.12.12 and currently Spark uses Scala 2.12.10, so there 
will be {{java.lang.NoClassDefFoundError: scala/math/Ordering$$anon$7}} errors.

Due to an issue in Scala 2.12.12, Spark won't upgrade to Scala 2.12.12: 
https://github.com/scala/bug/issues/12096, and waits for Scala 2.12.13.

So, seems for Kafka, Spark needs to wait for next Kafka version which uses 
Scala 2.12.13 too.

> Upgrade Kafka to 2.7.0
> --
>
> Key: SPARK-33913
> URL: https://issues.apache.org/jira/browse/SPARK-33913
> Project: Spark
>  Issue Type: Improvement
>  Components: Build, DStreams
>Affects Versions: 3.2.0
>Reporter: dengziming
>Priority: Major
>
>  
> The Apache Kafka community has released for Apache Kafka 2.7.0, some features 
> are useful for example the KAFKA-9893
>  configurable TCP connection timeout, more details : 
> https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-33913) Upgrade Kafka to 2.7.0

2020-12-27 Thread Apache Spark (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-33913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255324#comment-17255324
 ] 

Apache Spark commented on SPARK-33913:
--

User 'viirya' has created a pull request for this issue:
https://github.com/apache/spark/pull/30939

> Upgrade Kafka to 2.7.0
> --
>
> Key: SPARK-33913
> URL: https://issues.apache.org/jira/browse/SPARK-33913
> Project: Spark
>  Issue Type: Improvement
>  Components: Build, DStreams
>Affects Versions: 3.2.0
>Reporter: dengziming
>Priority: Major
>
>  
> The Apache Kafka community has released for Apache Kafka 2.7.0, some features 
> are useful for example the KAFKA-9893
>  configurable TCP connection timeout, more details : 
> https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org