nickwallen commented on a change in pull request #1338: METRON-2005: Batch
Writer writes 0-byte files to HDFS on rotation
URL: https://github.com/apache/metron/pull/1338#discussion_r257324590
##########
File path:
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java
##########
@@ -109,10 +119,12 @@ public BulkWriterResponse write(String sourceType
(String)configurations.getSensorConfig(sourceType).getOrDefault(IndexingConfigurations.OUTPUT_PATH_FUNCTION_CONF,
""),
message
);
+ LOG.trace("Writing message {} to path: {}", message.toJSONString(),
path);
SourceHandler handler = getSourceHandler(sourceType, path,
configurations);
handler.handle(message, sourceType, configurations, syncPolicyCreator);
}
} catch (Exception e) {
+ LOG.error("HdfsWriter encountered error writing", e);
Review comment:
This log statement is useful. Can you add the sourceType, number of
messages, and output path to the log statement? That context is going to be
helpful when debugging an issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
