[
https://issues.apache.org/jira/browse/STORM-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290334#comment-16290334
]
Jungtaek Lim edited comment on STORM-2854 at 12/14/17 4:44 AM:
---------------------------------------------------------------
[~ptgoetz]
We are planning to write some sampled events to file and let log feeder ingests
the file and parse the lines, and publish to Solr which we can query the
indexed sampled events later. Currently the event is written to file as a form
of hard-to-parse: it's written as csv but no escape on each part, and date is
written as locale dependent. We know the structure of our events, so can make a
better shape of log lines via having custom event logger implementation.
We are leveraging search engine and log feeder for now, but later we may want
to store the sampled events to external storage directly, with some
transformations.
In case of confusion, the use case is not for Storm, but for other project I'm
working on.
was (Author: kabhwan):
[~ptgoetz]
We are planning to write some sampled events to file and let log feeder ingests
the file and parse the lines, and publish to Solr which we can query the
indexed sampled events later. Currently the event is written to file as a form
of hard-to-parse: it's written as csv but no escape on each part, and date is
written as locale dependent. We know the structure of our events, so can make a
better shape of log lines via having custom event logger implementation.
In case of confusion, the use case is not for Storm, but for other project I'm
working on.
> Expose IEventLogger to make event logging pluggable
> ---------------------------------------------------
>
> Key: STORM-2854
> URL: https://issues.apache.org/jira/browse/STORM-2854
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-client
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> For the first time, "Event Logger" feature is designed to make implementation
> pluggable, so that's why IEventLogger exists, but we didn't have actual use
> case other than just writing them to the file at that time, so we just
> simplified the case.
> Now we have use case which also write events to file, but with awareness of
> structure of event so that it can be easily parseable from log feeder. We
> would want to have custom IEventLogger to represent event as our own format
> in this case.
> There's another issue as well: EventInfo has `ts` which stores epoch but it's
> defined as String, not long.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
