[
https://issues.apache.org/jira/browse/PIRK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428638#comment-15428638
]
ASF GitHub Bot commented on PIRK-4:
-----------------------------------
GitHub user clharris opened a pull request:
https://github.com/apache/incubator-pirk/pull/74
PIRK-4 [WIP] Add Streaming Implementation for Apache Storm
This is an initial implementation for a streaming version of Pirk to run on
Apache Storm. I am leaving it temporarily as WIP so people have a chance to
look over it and add feedback. Right now there is only one integration test
which runs the Storm topology 4 times with each of the different significant
configuration possibilities. I wanted to unit test the bolts, but it seems not
very straightforward with the way that the Pirk processing works. I'll try to
add some documentation (at the minimum a diagram of the Pirk Storm topology) by
early next week.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/clharris/incubator-pirk pirk-4
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-pirk/pull/74.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #74
----
commit 577fb33d04ce9061d2e88a27595737f30b9c446d
Author: charris <[email protected]>
Date: 2016-07-19T14:33:34Z
Temporarily using Cloudera pom.
commit 41f9042b81a5b683ecbd2c718f98712283acacf9
Author: charris <[email protected]>
Date: 2016-07-19T14:37:36Z
PIRK-4 Initial commit.
commit 0c147c36b377c730c7d6489a82b71958e59cdb2d
Author: charris <[email protected]>
Date: 2016-07-19T14:38:27Z
Adding toMap method and constructor.
commit a30f1a9de6e02dbf55bdc4e771d673b429ac331f
Author: charris <[email protected]>
Date: 2016-07-19T14:39:14Z
Adding functionality for streaming processing.
commit 8d27e5d3f2ff1fec9f2fffd29052189f567d51f0
Author: charris <[email protected]>
Date: 2016-07-19T14:40:17Z
Broke out method to checkDNSHostNameQueryResults.
commit 4df0e31836ec26da00662d92c5eac081c61419b2
Author: charris <[email protected]>
Date: 2016-07-19T15:04:07Z
Added Storm parameters.
commit 126361c31426d8285b4202a1134a653dd210dd4b
Author: eawilliams <[email protected]>
Date: 2016-07-22T14:55:28Z
initial modifications to storm code
commit 9f78a94dec84c7fcb5c09f13ae62598a7700adf4
Author: eawilliams <[email protected]>
Date: 2016-07-22T14:55:57Z
initial modifications to storm code - removing logs
commit 4bdd0c1f2461fdf0a1254bd6369db70b05b3e12a
Author: eawilliams <[email protected]>
Date: 2016-07-22T15:52:06Z
splitPartitions update to storm implementation
commit 180c56161ceffbe7c2d24719f8f9e5071ca632cb
Author: eawilliams <[email protected]>
Date: 2016-07-23T16:44:24Z
changed splitPartitions to false in test
commit 2520fbe8e0e6eb6a9ee8d259e0a5f10f965e916e
Author: charris <[email protected]>
Date: 2016-07-25T15:02:00Z
splitPartitions allows data element partitions to be sent individually.
commit e4e69e0119df4c0659a55bee7d6a5c55a016c9ff
Author: charris <[email protected]>
Date: 2016-07-30T19:41:11Z
Moved hashing into the spout.
commit 3262f0fdd52d62c050c6dc7fb6c6b41c34e99173
Author: charris <[email protected]>
Date: 2016-07-30T19:41:55Z
Added some functionality useful for Storm.
commit 290404531eaaa4442ef611252f4d24b2be7a6371
Author: charris <[email protected]>
Date: 2016-07-30T19:42:12Z
Fixed based on refactoring.
commit ba81c6814d0f0e01974fb934a0625c44676a316e
Author: charris <[email protected]>
Date: 2016-07-31T02:21:55Z
Added Apache headers.
commit 987d2f5254e359aa54ec5969935dda124b4d85a1
Author: charris <[email protected]>
Date: 2016-07-31T20:34:34Z
Changed Scala version of Spark and added curator-test.
commit 3b617c1b6c5ccdc560a2e32e58815b8732f18e58
Author: charris <[email protected]>
Date: 2016-07-31T23:17:34Z
Merging.
commit 22b946cbb361f96f4ecb67fb317d44be1c1b1498
Author: charris <[email protected]>
Date: 2016-07-31T23:18:24Z
Merge branch 'master' of https://github.com/apache/incubator-pirk into storm
commit 204495f8a75d38fd0b51a0970352e2204367e6ff
Author: eawilliams <[email protected]>
Date: 2016-08-01T00:34:32Z
fixed logging and scala version issues via pom modifications; refactored
storm test package
commit 8f8113727be5ae7de97adf82d8953abb2efbde63
Author: eawilliams <[email protected]>
Date: 2016-08-01T01:57:04Z
Merging - closes clharris/incubator-pirk#1
commit 1f8b42fdbf2326114f0cbce5fdfb7bd9c53306c3
Author: Chris Harris <[email protected]>
Date: 2016-08-01T12:22:36Z
Merge pull request #1 from ellisonanne/chris-storm
fixed logging and scala version issues via pom modifications; refacto…
commit a61c9765bff4018041201d3adc7f81038f5499c6
Author: charris <[email protected]>
Date: 2016-08-01T12:39:00Z
Merging.
commit 21c2a3c0166574da7f05576853733a93d4826a65
Author: charris <[email protected]>
Date: 2016-08-16T15:31:35Z
Debugging.
commit bec45e7f39eb6fb73f54c0744382a725897bd3ae
Author: charris <[email protected]>
Date: 2016-08-17T18:49:15Z
Merging. Storm works when jar not minimized.
commit 3bbc4d8e4c4b7d80b6c478b96e7947e673580995
Author: charris <[email protected]>
Date: 2016-08-18T12:32:40Z
Fixing logging issues.
commit 6a9ebcbacd0c0a4354db4cc309309ca7fba228ce
Author: charris <[email protected]>
Date: 2016-08-19T12:42:40Z
Merge branch 'master' of https://github.com/apache/incubator-pirk into storm
commit 78a7d3a8fff29781d3c336d1f97fc1d0ae1b5389
Author: charris <[email protected]>
Date: 2016-08-19T13:17:45Z
Merging pom with unminimized jar.
commit 275b1db125c0c7055f4b5dfb42e675e7fc0a2bd1
Author: charris <[email protected]>
Date: 2016-08-19T15:13:48Z
Topology runs from ResponderDriver
commit 2d2abe1b90891737204bbded9d9d6f7b3fa2f1ff
Author: charris <[email protected]>
Date: 2016-08-19T18:44:25Z
Streamlined integration test to run all 4 configurations.
----
> Add Streaming Implementation for Apache Storm
> ---------------------------------------------
>
> Key: PIRK-4
> URL: https://issues.apache.org/jira/browse/PIRK-4
> Project: PIRK
> Issue Type: Task
> Components: Responder
> Reporter: Chris Harris
> Assignee: Chris Harris
>
> Per the Pirk Roadmap, this is a feature to add support for Apache Storm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
