bamaer commented on issue #5819: URL: https://github.com/apache/hop/issues/5819#issuecomment-3400157573
What is the file size of your CSV files? The blocking transforms require all of your data to be buffered in memory before you can write it out to your Excel sheets. You have 32GB of memory in your machine, but Hop only grabs 2GB by default. Try changing the `-Xmx` parameter in `hop-gui.bat`, set it to for example 8GB. If that doesn't work, splitting up your pipeline to load on CSV file into one Excel sheet at a time will eliminate the need for the blocking transforms. You could do this in a single pipeline with a parameter with CSV file to read, there probably isn't any need to create multiple pipelines. You would need 4 different runs instead of a single one, but won't necessarily had a huge impact on your total throughput time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
