[ https://issues.apache.org/jira/browse/METRON-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595345#comment-15595345 ]
ASF GitHub Bot commented on METRON-495: --------------------------------------- Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/318 A couple of points of context around what changed here: * Kafka shifted some of their integration testing infrastructure, so the in memory component for Kafka that we use for integration testing had to change * Ansible blueprint was changed to use the HDP 2.5 stack * There was a bug around the PCap backend where our use of `MultiScheme` was assuming that the `ByteBuffer`s passed are filled with our data as opposed to wrapping an existing larger buffer. * HBase dependencies brought in various Hadoop 2.5.1 dependencies. This was a long-standing issue, but when the classpath shuffled around, it bit us all. * Apache Curator 2.10.0 is now a dependency of the storm-kafka component. The service discovery piece of this version seems to have a bug that prevents it from working with Model as a Service. We are maintaining the dependency on 2.7.1 for the time being, but should look further into it in a follow-on JIRA. One thing to note, is that when this PR is merged, we will need to regenerate the image used in vagrant for `quick-dev`. > Upgrade Storm to 1.0.x > ---------------------- > > Key: METRON-495 > URL: https://issues.apache.org/jira/browse/METRON-495 > Project: Metron > Issue Type: Improvement > Reporter: Justin Leet > Assignee: Justin Leet > > As listed at https://storm.apache.org/2016/04/12/storm100-released.html, > there's a variety of improvements with Storm 1.0. The obvious and likely > most important improvement is to performance, but a variety of other > improvements are noted on that link. > There are several changes that have to occur in order to make this upgrade. > As noted in http://storm.apache.org/releases/current/index.html, Storm's code > packages moved from backtype.storm to org.apache.storm, meaning all > topologies have to be recompiled if that change. There is a runtime > converter to run things in place with backtype.storm, but this doesn't appear > to be enough for our case because a couple interfaces change from byte[] to > ByteBuffer (somewhat, but not entirely, related to > https://issues.apache.org/jira/browse/STORM-1449). Even without this issue, > the long term solution is to use the new package naming. > In addition, our dev instances right now spin up an HDP 2.4 instance, which > matches our current version of Storm. HDP 2.5 uses Storm 1.0.1, so to migrate > to Storm 1.0.x, I'd prefer to match that version, rather than going to 1.0.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)