hansva commented on issue #4212: URL: https://github.com/apache/hop/issues/4212#issuecomment-2294360116
As I said before rowset_size does not mean you can only process that amount of rows... you can process a billion rows with rowset_size value = 10K. Apache Hop is a micro-batch/streaming application. The row set size is the amount of rows each transform can have in its waiting buffer. We have a pushback mechanism, when a buffer is full it will tell the upstream transform to stop providing new rows temporarily. In contrast to other tools all transforms are running in parallel we do not do sequential processing so our data does not have to fit in memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
