Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/1239#discussion_r226386542
--- Diff:
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/BulkWriterComponent.java
---
@@ -118,12 +118,15 @@ public void commit(BulkWriterResponse response) {
public void error(String sensorType, Throwable e, Iterable<Tuple>
tuples, MessageGetStrategy messageGetStrategy) {
LOG.error(format("Failing %d tuple(s); sensorType=%s",
Iterables.size(tuples), sensorType), e);
- MetronError error = new MetronError()
- .withSensorType(Collections.singleton(sensorType))
- .withErrorType(Constants.ErrorType.INDEXING_ERROR)
- .withThrowable(e);
- tuples.forEach(t -> error.addRawMessage(messageGetStrategy.get(t)));
- handleError(tuples, error);
+ tuples.forEach(t -> {
--- End diff --
> You're right! My mistake. I changed it to pass in just the current tuple
to handleError
Your latest commit was my first thought of a fix too; it is much simpler.
But I don't think it solves the second problem that I mentioned.
> There is also the nuisance that we report the error to the collector for
each and every failed message, instead of just once for the batch. There is
only one Throwable error to report, so we should just report it once.
I think reporting the error once is both a usability and performance
improvement.
I had to unravel the code base a bit more in my suggested code snippet so
that it only reports the error once.
---
