[
https://issues.apache.org/jira/browse/IGNITE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866082#comment-15866082
]
ASF GitHub Bot commented on IGNITE-533:
---------------------------------------
GitHub user dream-x opened a pull request:
https://github.com/apache/ignite/pull/1532
IGNITE-533: Implement IgniteZeromqStreamer to stream data from ZeroMQ
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dream-x/ignite ignite-533
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/1532.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1532
----
commit 1bbbe9771ce1869ce996a5c8ddd774fff45fe9a2
Author: Max Kozlov <[email protected]>
Date: 2017-02-06T11:33:07Z
Init ZeroMQ module.
commit 7a3f534809ce77c368aa1a1baa510bae91f1ceae
Author: Max Kozlov <[email protected]>
Date: 2017-02-06T16:12:05Z
Add IgniteZeroMqStreamer class.
commit 13c7d90f9e371ee92e2c7693100b5aa535d8935c
Author: Max Kozlov <[email protected]>
Date: 2017-02-06T16:12:45Z
Add ZeroMqSettings settings class.
commit 054955abc17761abf808454ff2bc14e2e3a3fd8b
Author: Max Kozlov <[email protected]>
Date: 2017-02-06T16:13:49Z
Add converter package and first implementation converter class.
commit f65c0f6a58b231b2eeba4f6338b4072840606230
Author: Max Kozlov <[email protected]>
Date: 2017-02-06T16:14:33Z
Add IgniteZeroMqStreamerImpl implementation ZeroMQ streamer.
commit d2443d501d961825891e9b9fc1ee63d62e487d9b
Author: Max Kozlov <[email protected]>
Date: 2017-02-07T11:51:03Z
Update converter.
commit 3893f34c9cd8ec9e18b7dff974d6fb2ba1a4f6e3
Author: Max Kozlov <[email protected]>
Date: 2017-02-07T11:52:04Z
wip
commit 277f119966ab278fe578d30ba202ca5334c50dc5
Author: Max Kozlov <[email protected]>
Date: 2017-02-07T11:52:39Z
Add test for ZeroMQ streamer.
commit f43765a5c22c4aac0cfd5c0013ef198e32ae7235
Author: Max Kozlov <[email protected]>
Date: 2017-02-08T08:45:18Z
Cleaning code and writing comments.
commit e25a25cd64cbc9ae8a1e5676ed2adb01132be9e0
Author: Max Kozlov <[email protected]>
Date: 2017-02-08T12:56:20Z
wip
commit f508e0b5eeecf8657e467c1e45e24e564bc65e6c
Author: Max Kozlov <[email protected]>
Date: 2017-02-08T16:46:13Z
wip
commit ae513f900cfc5271a22e2eb7a4e25123fd995e29
Author: Max Kozlov <[email protected]>
Date: 2017-02-09T11:04:49Z
Change type isStart variable.
commit 4311f7e6030f5549bc03f3ec30489ff74c18dc20
Author: Max Kozlov <[email protected]>
Date: 2017-02-09T11:05:49Z
Change type variable in ZeroMqTypeSocket.
commit 12bdc5866b197f2b873cf2395838b5ab6c3a1c7a
Author: Max Kozlov <[email protected]>
Date: 2017-02-09T14:08:16Z
Delete converter.
commit d806a55ad57d07abe778b0640d7c0aff77e0d56f
Author: Max Kozlov <[email protected]>
Date: 2017-02-09T14:09:54Z
Delete IgniteZeroMqStreamerImpl.java and add StreamSingleTupleExtractor in
IgniteZeroMqStreamer by default.
commit 1fc16cb6e98671f408fb746cdd4c42155e8d17fc
Author: Max Kozlov <[email protected]>
Date: 2017-02-10T10:57:41Z
Delete ZeroMqStreamerException.
commit f6f35482313790cd7fd5401c92e522a9da335473
Author: Max Kozlov <[email protected]>
Date: 2017-02-10T11:06:58Z
Add comment in ZeroMqTypeSocket.
commit da1cbad35a42aec0d255e1ae30e2e64bca944ff2
Author: Max Kozlov <[email protected]>
Date: 2017-02-10T11:22:57Z
Fix for help run all test in test case.
commit 533add6446c3cc009e148135e7966f437e45c351
Author: Max Kozlov <[email protected]>
Date: 2017-02-10T12:35:08Z
Fix run tests in test case.
commit b8f25f786bb9b446d427c85a487d1a77b251d089
Author: Max Kozlov <[email protected]>
Date: 2017-02-13T08:34:08Z
Change name artifact.
commit c2db765fa34deaa8e0329f81dd75bad47e37a942
Author: Max Kozlov <[email protected]>
Date: 2017-02-13T08:36:16Z
IgniteZeroMqStreamer impl AutoCloseable, repeat call start() write log.info
message.
commit 460659bc0bb345943f3808a86604b0e60e00eaf7
Author: Max Kozlov <[email protected]>
Date: 2017-02-13T08:48:55Z
DefaultSingleTupleExtractorImpl class take out for IgniteZeroMqStreamer,
create new ZeroMqStringSingleTupleExtractor class. Change tests.
commit 4b96810f33e44bbb7bfe928e2b5db8bf8b749c32
Author: Max Kozlov <[email protected]>
Date: 2017-02-13T09:07:09Z
Delete ZeroMqSettings class, 0mq params in constructor IgniteZeroMqStreamer
and fix tests.
commit f1e7d7c5c48b4fe11c5ab463d9a005c3a1e0d686
Author: Max Kozlov <[email protected]>
Date: 2017-02-13T09:12:21Z
Code clean.
commit 90a73c5a1df6ec75050ad72729d5240f2773defc
Author: Max Kozlov <[email protected]>
Date: 2017-02-13T12:54:53Z
Change test timeout.
commit eccd570db649047fc68bf58d0b6794eb8fc35dd9
Author: Max Kozlov <[email protected]>
Date: 2017-02-14T13:06:39Z
Add check params.
commit fa0aa666b5d57b05d981dfccb5e1d7b39e5f0482
Author: Max Kozlov <[email protected]>
Date: 2017-02-14T13:09:16Z
Change method from beforeTest() to beforeTestsStarted() for single call
this test. Optimize import.
commit e37f72518d07a9c8b389134b76bf0d8411201255
Author: Max Kozlov <[email protected]>
Date: 2017-02-14T13:16:49Z
Add full description.
commit 8654ecb5ccf2b81cef31a6e137754d32895462d5
Author: Max Kozlov <[email protected]>
Date: 2017-02-14T14:09:32Z
Code clean.
----
> Implement IgniteZeromqStreamer to stream data from ZeroMQ
> ---------------------------------------------------------
>
> Key: IGNITE-533
> URL: https://issues.apache.org/jira/browse/IGNITE-533
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
> Reporter: Dmitriy Setrakyan
> Assignee: Maksim Kozlov
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [ZeroMQ|http://zeromq.org/] for more info.
> We should create {{IgniteZeroMqStreamer}} which will consume messages from
> Twitter and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert ZeroMQ messages to Ignite data using an optional pluggable
> converter. If not provided, we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
