[jira] [Commented] (SPARK-13120) Shade protobuf-java
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136320#comment-15136320 ] Ted Yu commented on SPARK-13120: I do see the advantage of shading protobuf-java which would make user experience better where wire-compatibility across different protobuf versions is provided. > Shade protobuf-java > --- > > Key: SPARK-13120 > URL: https://issues.apache.org/jira/browse/SPARK-13120 > Project: Spark > Issue Type: Improvement > Components: Build >Reporter: Ted Yu > > See this thread for background information: > http://search-hadoop.com/m/q3RTtdkUFK11xQhP1/Spark+not+able+to+fetch+events+from+Amazon+Kinesis > This issue shades com.google.protobuf:protobuf-java as > org.spark-project.protobuf -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13120) Shade protobuf-java
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136383#comment-15136383 ] Ted Yu commented on SPARK-13120: I don't think this JIRA is limited to the scenario from the thread initially cited. I have dropped that from description. > Shade protobuf-java > --- > > Key: SPARK-13120 > URL: https://issues.apache.org/jira/browse/SPARK-13120 > Project: Spark > Issue Type: Improvement > Components: Build >Reporter: Ted Yu > > https://groups.google.com/forum/#!topic/protobuf/wAqvtPLBsE8 > PB2 and PB3 are wire compatible, but, protobuf-java is not compatible so > dependency will be a problem. > Shading protobuf-java would provide better experience for downstream projects. > This issue shades com.google.protobuf:protobuf-java as > org.spark-project.protobuf -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13120) Shade protobuf-java
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136360#comment-15136360 ] Sean Owen commented on SPARK-13120: --- What problem does that solve then? shading really helps when the API has changed across versions. It doesn't fix what third-party libs might provide at runtime though. I like shading in general as a defensive mechanism, but would like to do so only when we have some plausible scenario it fixes. The thread at the top doesn't show a protobuf problem of this form. > Shade protobuf-java > --- > > Key: SPARK-13120 > URL: https://issues.apache.org/jira/browse/SPARK-13120 > Project: Spark > Issue Type: Improvement > Components: Build >Reporter: Ted Yu > > See this thread for background information: > http://search-hadoop.com/m/q3RTtdkUFK11xQhP1/Spark+not+able+to+fetch+events+from+Amazon+Kinesis > This issue shades com.google.protobuf:protobuf-java as > org.spark-project.protobuf -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13120) Shade protobuf-java
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128759#comment-15128759 ] Ted Yu commented on SPARK-13120: https://groups.google.com/forum/#!topic/protobuf/wAqvtPLBsE8 PB2 and PB3 are wire compatible, but, protobuf-java is not compatible so dependency will be a problem. Shading protobuf-java would provide better experience for downstream projects. > Shade protobuf-java > --- > > Key: SPARK-13120 > URL: https://issues.apache.org/jira/browse/SPARK-13120 > Project: Spark > Issue Type: Improvement > Components: Build >Reporter: Ted Yu > > See this thread for background information: > http://search-hadoop.com/m/q3RTtdkUFK11xQhP1/Spark+not+able+to+fetch+events+from+Amazon+Kinesis > This issue shades com.google.protobuf:protobuf-java as > org.spark-project.protobuf -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13120) Shade protobuf-java
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15129820#comment-15129820 ] Sean Owen commented on SPARK-13120: --- Yeah, aren't you also saying this wouldn't help? And it is still not something identified as a problem in the thread you linked > Shade protobuf-java > --- > > Key: SPARK-13120 > URL: https://issues.apache.org/jira/browse/SPARK-13120 > Project: Spark > Issue Type: Improvement > Components: Build >Reporter: Ted Yu > > See this thread for background information: > http://search-hadoop.com/m/q3RTtdkUFK11xQhP1/Spark+not+able+to+fetch+events+from+Amazon+Kinesis > This issue shades com.google.protobuf:protobuf-java as > org.spark-project.protobuf -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13120) Shade protobuf-java
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127106#comment-15127106 ] Apache Spark commented on SPARK-13120: -- User 'tedyu' has created a pull request for this issue: https://github.com/apache/spark/pull/10995 > Shade protobuf-java > --- > > Key: SPARK-13120 > URL: https://issues.apache.org/jira/browse/SPARK-13120 > Project: Spark > Issue Type: Improvement > Components: Build >Reporter: Ted Yu > > See this thread for background information: > http://search-hadoop.com/m/q3RTtdkUFK11xQhP1/Spark+not+able+to+fetch+events+from+Amazon+Kinesis > This issue shades com.google.protobuf:protobuf-java as > org.spark-project.protobuf -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13120) Shade protobuf-java
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127555#comment-15127555 ] Sean Owen commented on SPARK-13120: --- Why does shading help there? I don't see any info on the cause. In the past protobuf problems have been wire protocol incompatibilities which shading doesn't help. They also arise from third party deps which shading may not help. I'd close this if its only a guess. > Shade protobuf-java > --- > > Key: SPARK-13120 > URL: https://issues.apache.org/jira/browse/SPARK-13120 > Project: Spark > Issue Type: Improvement > Components: Build >Reporter: Ted Yu > > See this thread for background information: > http://search-hadoop.com/m/q3RTtdkUFK11xQhP1/Spark+not+able+to+fetch+events+from+Amazon+Kinesis > This issue shades com.google.protobuf:protobuf-java as > org.spark-project.protobuf -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
