[
https://issues.apache.org/jira/browse/METRON-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529806#comment-15529806
]
Nick Allen commented on METRON-470:
-----------------------------------
+1
In general, the `metron-writer` code is too tightly coupled to specific
concepts in the rest of Metron which make the code less reusable. For example,
the BulkMessageWriter assumes it is used in the Enrichment Topology via
subclassing ConfiguredEnrichmentBolt. The writers also use the concept of
'sensorType' which means I can't use this code outside of a topology that
doesn't have the concept of 'sensors'.
This is why I could not reuse any of the `metron-writer` logic in implementing
the Profiler. For an understanding of how writers could be implemented, see
the HBaseBolt and HBaseMapper in `metron-hbase`. No Metron concept like
'sensorType' or 'Enrichment Topology' are used in that code which makes it more
generic and flexible. I think we should move `metron-writers` in that
direction.
> Create a separate zookeeper config for the indexing topology
> ------------------------------------------------------------
>
> Key: METRON-470
> URL: https://issues.apache.org/jira/browse/METRON-470
> Project: Metron
> Issue Type: Improvement
> Reporter: Jon Zeolla
> Priority: Minor
>
> Currently the writer bolts in the indexing topology use the batchSize from
> the enrichment topology's config. It may be useful to split this out in the
> future to allow control of the indexing topology inside zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
