mattcasters opened a new pull request, #8674: URL: https://github.com/apache/hop/pull/8674
Text File Output kept each write in a 5000-byte buffer, so slow input did not show up in the file until that buffer filled or the transform closed the file. `HOP_FILE_OUTPUT_MAX_STREAM_LIFE` was already documented as the flush interval, but it defaulted to 0 and the elapsed-time check subtracted the clock in the wrong order, so a configured interval never flushed. The comparison is now the time since the last flush. The variable defaults to 5000 milliseconds, and a value of 0 (what existing Hop configurations already store) uses that same default. A positive value still sets the interval in milliseconds. Rows written after a flush mark the file dirty again, so the next interval is not skipped, and closing a file flushes the buffer even when that flag is clear. Compression output streams pass `flush()` through to the underlying stream. `TextFileOutputFlushTest` and `TextFileOutputTest` passed in `hop-transform-textfile`. `CompressionOutputStreamTest` passed in `hop-engine`. A full reactor build and the Hop GUI were not run. fixes #2406 ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Run `mvn clean install apache-rat:check` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If you have a group of commits related to the same change, please squash your commits into one and force push your branch using `git rebase -i`. - [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. To make clear that you license your contribution under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) you have to acknowledge this by using the following check-box. - [x] I hereby declare this contribution to be licensed under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) - [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
