GitHub user justinleet opened a pull request:

    METRON-495: Upgrade Storm to 1.0.x

    ***** INCOMPLETE DO NOT MERGE ***** 
    Putting the PR out, so people can take a look and potentially fix/improve 
    Initial start at upgrading the Storm version to 1.0.1, and I'm going to hop 
off of this for right now.  Provided in case anyone wants to take a stab at 
fixing / cleaning up the various issues with it, and so feedback can be 
provided. If you do make improvements, feel free to grab the branch and submit 
a new PR.
    What changes occurred
    - The move over to org.apache.storm from backtype.storm
    - API changes from byte[] to ByteArray are handled
    - Kafka also updated to 0.10 out of necessity, storm-kafka is still used 
(storm-kafka-client seemed like significantly more work given the amount of 
changes to the underpinnings).  I would not expect this to cause issues, given 
backwards compatibility, but since tests still fail this hasn't been validated.
    - Some of the various classpath issues have been taken care of (things move 
between jars, and a couple test items now need a test jar which has been pulled 
in and scoped appropriately on various modules)
    - A couple simple Scala wrappers about some of the Storm Kafka handling in 
regards to Zookeeper has been added (including to the pom) to avoid having to 
call direct Scala only testing tools that use pretty Scala specific stuff.  The 
wrappers just take in the args we were using and use defaults for the rest.
    - Changes to the full dev platform to account for using HDP 2.5 to match 
the new version.  This is entirely untested given test failures.
    What works
    - Most unit tests.  Interestingly, metron-platform/metron-management fails 
on CLI, but succeeds in IntelliJ.  I'm guessing classpath issue as a result of 
attempting to fix integration tests broke this on the CLI as it used to work, 
but unsure.
    - Most integration tests. This involved moving.
    What doesn't work
    - Both some unit and some integration tests
    - Dev platforms are untested.
    What's left
    - Fixing tests
    - Testing full dev
    - Building quick dev image and testing it
    - End to End testing.  Given the scope of the changes, this needs to be 
pretty thorough.
    - Ambari MPack testing.

You can merge this pull request into a Git repository by running:

    $ git pull storm_1.0

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #310
commit dbf7ff19072cb44df66928920a70e0f1f1f8e4e8
Author: justinjleet <>
Date:   2016-10-11T13:58:39Z

    Initial move to Storm 1.0.1

commit 9796bf36fc4a19fbed5276166c305cb8fb815e00
Author: justinjleet <>
Date:   2016-10-12T15:13:45Z

    fixing more imports

commit 044b9d7113f3b7d2d367f2f58e40872ea372d75a
Author: justinjleet <>
Date:   2016-10-13T19:56:27Z

    changes to kakfa.  WIP

commit ec157af245cfcac3a8eda390d145328d62dfb0ab
Author: justinjleet <>
Date:   2016-10-13T19:56:43Z

    Missing changes from last commit

commit 90321655055e9a290f64724c8e3bd8a013deebfc
Author: justinjleet <>
Date:   2016-10-14T12:51:17Z

    WIP changes with scala wrapper

commit f418f50958d9826c9307858914253f304a18d066
Author: justinjleet <>
Date:   2016-10-14T16:22:59Z

    More integration tests working

commit f445bfb9fef0583bbce444406227f96a4c506706
Author: justinjleet <>
Date:   2016-10-14T19:24:41Z



If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

Reply via email to