[
https://issues.apache.org/jira/browse/SAMZA-235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987056#comment-13987056
]
Yan Fang commented on SAMZA-235:
--------------------------------
Tested this idea.
1) Wrote a simple script and put it under /bin directory. Because the command
line [~criccomini] provides does not work if there is not topic in Kafka, put
some lines to check the existence of the topic and create one if not exist. RB:
https://reviews.apache.org/r/20988/
2) Put a json file containing 1000 wikipedia-edit records under the root
directory. (Maybe we should put it into a seperate folder?)
Let me know if I need to make changes. Will modify the hello-samza tutorial to
demonstrate this approache after commiting.
Thank you.
> Add internal input stream for hello-samza
> -----------------------------------------
>
> Key: SAMZA-235
> URL: https://issues.apache.org/jira/browse/SAMZA-235
> Project: Samza
> Issue Type: Improvement
> Components: hello-samza
> Reporter: Yan Fang
> Assignee: Yan Fang
> Attachments: SAMZA-235.patch
>
>
> As reported by Sonali and Yan Fang, some corporations blocks IRC
> service/port. So they will not be able to run the hello-samza successfully.
> http://mail-archives.apache.org/mod_mbox/samza-dev/201403.mbox/%3cb84b01583bebbc45ad442b3f9045b8ac0ed46...@048-ch1mpn3-331.048d.mgd.msft.net%3E
> As suggested by [~jghoman] and [~criccomini] , we should add internal input
> stream for hello-samza as an alternative. There are two ways:
> 1. use simulate/fake data.
> 2. use local environment related data.
> I lean to the first approach. We can simulate wikimedia data (though it is a
> little boring). Because it can reuse the WikipediaParserStreamTask and
> WikipediaStatsStreamTask. Another reason is, since we use simulate data, the
> output is very predictable, that will help bring hello-samza to integration
> test stated in SAMZA-205 .
> In addition, if we use FS reader in SAMZA-138 , that will also be a good
> example for writing SystemFactory (besides the out-of-box KafkaSystemFactory).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
