[
https://issues.apache.org/jira/browse/STORM-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197229#comment-15197229
]
ASF GitHub Bot commented on STORM-1537:
---------------------------------------
Github user abhishekagarwal87 commented on the pull request:
https://github.com/apache/storm/pull/1223#issuecomment-197279568
This is not to be merged yet. Right now, I only wanted to showcase all the
changes that are to be made for the upgrade. Kryo 3 apparently has some
performance improvements. Since the kryo serialization is used only for intra
topology transfer, there should be no compatibility issues. Let me know if this
change has other side effects.
> Upgrade to Kryo 3
> -----------------
>
> Key: STORM-1537
> URL: https://issues.apache.org/jira/browse/STORM-1537
> Project: Apache Storm
> Issue Type: Improvement
> Affects Versions: 1.0.0
> Reporter: Oscar Boykin
> Assignee: P. Taylor Goetz
>
> In storm, Kryo (2.21) is used for serialization:
> https://github.com/apache/storm/blob/02a44c7fc1b7b3a1571b326fde7bcae13e1b5c8d/pom.xml#L231
> The user must use the same version storm does, or there will be a java class
> error at runtime.
> Storm depends on a quasi-abandoned library: carbonite:
> https://github.com/apache/storm/blob/02a44c7fc1b7b3a1571b326fde7bcae13e1b5c8d/pom.xml#L210
> which depends on Kryo 2.21 and Twitter chill 0.3.6:
> https://github.com/sritchie/carbonite/blob/master/project.clj#L8
> Chill, currently on 0.7.3, would like to upgrade to Kryo 3.0.3:
> https://github.com/twitter/chill/pull/245
> because Spark, also depending on chill, would like to upgrade for performance
> improvements and bugfixes.
> https://issues.apache.org/jira/browse/SPARK-11416
> Unfortunately, summingbird depends on storm:
> https://github.com/twitter/summingbird/blob/develop/build.sbt#L34
> so, if chill is upgraded, and that gets on the classpath, summingbird will
> break at runtime.
> I propose:
> 1) copy the carbonite code into storm. It is likely the only consumer.
> 2) bump the storm kryo dependency after chill upgrades: recall that storm
> actually depends on chill-java. A dependency that could possibly be removed
> after you pull carbonite in.
> 3) once a new version of storm is published, summingbird (and scalding) can
> upgrade to the latest chill.
> Also, I hope for:
> 4) we as a JVM community get better about classpath isolation and versioning.
> Diamonds like this in one big classpath make large codebases very fragile.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
