[
https://issues.apache.org/jira/browse/METRON-326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536042#comment-15536042
]
ASF GitHub Bot commented on METRON-326:
---------------------------------------
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/286#discussion_r81340830
--- Diff:
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java
---
@@ -69,14 +66,19 @@ public void init(Map stormConfig, WriterConfiguration
configurations) {
@Override
- public void write(String sourceType
+ public BulkWriterResponse write(String sourceType
, WriterConfiguration configurations
, Iterable<Tuple> tuples
, List<JSONObject> messages
) throws Exception
{
SourceHandler handler = getSourceHandler(sourceType);
handler.handle(messages);
+
+ // HDFS SourceHanlder manages writing and rotating files. Will throw
its own Exceptions.
--- End diff --
Will fix typo. Same deal as previous comment where it's pretty easy to
change and manage that directly.
> Error Handling in ElasticsearchWriter
> -------------------------------------
>
> Key: METRON-326
> URL: https://issues.apache.org/jira/browse/METRON-326
> Project: Metron
> Issue Type: Bug
> Reporter: Ajay Yadav
> Assignee: Justin Leet
>
> In Elasticsearch writer we raise a exception if BulkResponse object has
> failures and that results in failing the whole batch even if some objects
> failed in it. This has spiral effect specially when there is continuous
> stream of bad messages and errorStream is tied to indexingBolt.
> If possible we should iterate through items in BulkResponse object and send
> only failed messages to errorStream.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
