GitHub user SreeramGarlapati opened a pull request:
https://github.com/apache/storm/pull/2588
Microsoft Azure EventHubs Storm Spout and Bolt improvements
This is continuation of work done by @raviperi
- update to the latest version of eventhubs java client
-Introduce config params to use latest EH client, control request prefetch
size, batch size of events received per call.
-Refactor the code to group classes more appropriately
-Remove redundant types
-Javadoc comments where applicable
-Preftch config parameter to dictate EH prefetch count
-config parameter to introduce sleep between spout's nexttuple calls
-config parameter to retrieve a batched number of events per call to EH
(opposed to single event)
-New data scheme to group event payload and audit params into a single
type, and expose the single type as the only tuple field to downstream
bolts.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SreeramGarlapati/storm master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2588.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2588
----
commit e9fc45d3d604b65ab3e99895b4a7711bf1b7ab30
Author: Ravi Peri <raviperi@...>
Date: 2017-09-11T21:35:48Z
Introduce config params to dictate EH request prefetch size, max number
of events received per call.
-Refactor the code to group classes more appropriately
-Remove redundant types
-Javadoc comments where applicable
-Preftch config parameter to dictate EH prefetch count
-config parameter to introduce sleep between spout's nexttuple calls
-config parameter to retrieve a batched number of events per call to EH
(opposed to single event)
-New data scheme to group event payload and audit params into a single
type, and expose the single type as the only tuple field to downstream
bolts.
commit 3f8f37f65551414afdaaf48935b662e9e42836be
Author: Ravi Peri <raviperi@...>
Date: 2017-09-13T18:07:18Z
Remove unncessary file.
commit 9a18f9a8d1629f34fac648e05a3ae61d388e0670
Author: Ravi Peri <raviperi@...>
Date: 2017-09-14T17:09:35Z
Add license header, remove next tuple sleep time as it is redundant.
commit 5a7eb2e1f5d5f1a3f65e1530b284f364e618f99f
Author: Ravi Peri <raviperi@...>
Date: 2017-09-14T17:14:08Z
Remove unused nextTuple sleep interval field
commit 6ba9d11a9343c47add523e55415238b6df010de5
Author: Ravi Peri <raviperi@...>
Date: 2017-09-19T19:27:32Z
Add javadoc for EventHubConfig. Remove unused fields. Improve logging when
returning empty/null values
commit 53569ac4ccdca5e0978ac3b8eae2423c97657197
Author: Ravi Peri <raviperi@...>
Date: 2017-09-19T20:33:20Z
Organize usings. Remove extra constructor and fields in EventHubSpoutConfig
commit b1bde9b592024dcf1eaa70365c010cc56cd99313
Author: Ravi Peri <raviperi@...>
Date: 2017-09-20T18:30:46Z
Remove obselete Binary and EventDataSchemes, and add unit tests for
supported schemes. Enable scheme based serialization in spout
commit 4516d01487e4dcb2211ea713fb160f6a45bd34fd
Author: Ravi Peri <raviperi@...>
Date: 2017-09-20T18:52:50Z
Mark all fields final in EventHUbMessage
commit 2063f4547939dfde459d8241b80e3b262e5e1816
Author: Ravi Peri <raviperi@...>
Date: 2017-09-20T19:15:40Z
Update batch size log message to debug level
commit 480e2fcdaa4e14f503cde1552a11d53830a21cda
Author: Sreeram Garlapati <sreeramgarlapati@...>
Date: 2018-03-07T03:18:56Z
Merge branch 'master' of https://github.com/raviperi/storm
# Conflicts:
#
external/storm-eventhubs/src/main/java/org/apache/storm/eventhubs/bolt/EventHubBolt.java
#
external/storm-eventhubs/src/main/java/org/apache/storm/eventhubs/core/FieldConstants.java
#
external/storm-eventhubs/src/main/java/org/apache/storm/eventhubs/format/StringEventDataScheme.java
#
external/storm-eventhubs/src/main/java/org/apache/storm/eventhubs/spout/EventDataScheme.java
#
external/storm-eventhubs/src/main/java/org/apache/storm/eventhubs/spout/EventHubException.java
#
external/storm-eventhubs/src/main/java/org/apache/storm/eventhubs/spout/EventHubSpout.java
#
external/storm-eventhubs/src/main/java/org/apache/storm/eventhubs/trident/TransactionalTridentEventHubSpout.java
#
external/storm-eventhubs/src/test/java/org/apache/storm/eventhubs/spout/SpoutOutputCollectorMock.java
# pom.xml
commit a61466d4f4cfda92aea3157abbeec4b3646cdd5d
Author: Sreeram Garlapati <sreeramgarlapati@...>
Date: 2018-03-07T05:32:14Z
remove files as continuation to the merge commit
commit 19bd1c9ddd13f73cc8e6ddfc2d987c7c8ea58efd
Author: Sreeram Garlapati <sreeramgarlapati@...>
Date: 2018-03-07T06:09:12Z
move the library to 1.0.0
----
---
