[GitHub] metron issue #741: METRON-1153 HDFS HdfsWriter never recovers from exception...

2017-09-08 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/741 I absolutely agree, this does not address the higher level issue or severity. This is merely a stopgap to fix something that we are seeing in the field and on the list. I made a JIRA to capture

[GitHub] metron issue #741: METRON-1153 HDFS HdfsWriter never recovers from exception...

2017-09-08 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/741 @justinleet My question with regards to the necessity for a refactor came from wanting to handle the exception in the HDFSWriter, where the exceptions where being caught at the time.

[GitHub] metron issue #741: METRON-1153 HDFS HdfsWriter never recovers from exception...

2017-09-08 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/741 This looks good to me as a stopgap. I'd like to understand eventually why this happens, but it appears very intermittent, so this isn't a bad solution at least as a stopgap. +1 by

[GitHub] metron issue #741: METRON-1153 HDFS HdfsWriter never recovers from exception...

2017-09-08 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/741 Adjusted to only catch the "Stream Closed" exception, not the general IOException. If it's stream closed, attempt the rotate and write, otherwise rethrow ---

[GitHub] metron issue #741: METRON-1153 HDFS HdfsWriter never recovers from exception...

2017-09-08 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/741 @ottobackwards From the ticket description, it looks like you thought there was a more involved refactoring for doing something like this. Are you good with this sort of solution (at least as an