hansva opened a new issue, #9344: URL: https://github.com/apache/hop/issues/9344
### Apache Hop version? 2.20.0 ### Java version? 21 ### Operating system Other ### What happened? With **Pass all rows** enabled, the Blocking transform is much slower than it should be, both when it keeps the rows in memory and when it writes them to temporary files. **Kept in memory:** each time a row is sent on, all the rows still waiting are moved up one place in the list. The more rows the cache holds, the slower every row gets. 500,000 rows reach about 42,000 rows/s, and almost all of the time goes into moving the list. **Written to temporary files:** every value is compressed on its own instead of in larger blocks. With 1,000,000 rows and the default cache this reaches about 104,000 rows/s, and more than half of the time goes into compression. **Fix** - Read the rows in memory in order, without moving the list around. - Write and read the temporary files in blocks. ### Issue Priority Priority: 2 ### Issue Component Component: Transforms -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
