ajinkya-k commented on issue #43404: URL: https://github.com/apache/arrow/issues/43404#issuecomment-2254199926
Thanks for the update @thisisnic . I do a join and a few filters that drop less than 1% of the rows and then collect, but it's still a huge dataset after that, which I plug into a Bayesian model. The Bayesian model does work, it's just that due to DUA constraints I have to keep the file on a network drive and pull from there. And therefore it's hard to figure out if the file is even being loaded at all, i.e. a progress bar will help me figure out if the read is even progressing at all, or if the network throttling means the process is hung up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
