Baunsgaard commented on PR #2154: URL: https://github.com/apache/systemds/pull/2154#issuecomment-2564081194
> So there is some more constant time, the building of the frameblock a few lines before the `.convert` calls for example is around 400ms. We could put the allocation into the parallel transfer call for each column? > I looked at the profiling a bit more and it seems like most time is spent on socket communication between Java and Python. My assumption would be that this adds quite a bit of overhead and doesn't parallelize well. I was afraid this was the case, Maybe there is a way to set the thread pool to use on the java side to make it better at receiving requests in parallel. However, that is probably just a dream. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@systemds.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
