Perhaps if its a precompling issue, it would be faster if I precomplied with a smaller distributed array. I'll try that and report back in the next day or two.
On Tuesday, January 26, 2016 at 12:28:21 PM UTC-5, Christopher Fisher wrote: > > Thanks for the idea. It may not apply in my particular case. I forgot to > mention that my memory is 32GB on the machine I am using. So swapping with > the hard drive does not seem likely. > > Given that the second execution of the function is much faster (1 second > vs 5 min), my best guess is that the long initial run might be due to > compiling. > > On Tuesday, January 26, 2016 at 12:12:41 PM UTC-5, [email protected] > wrote: >> >> Hello Christopher, >> >> I saw you code and you noticed you are reading all the data(3GB) locally, >> maybe it's causing your machine to use swap memory, which leads to the >> performance issue. >> You can check that by >> 1: open htop in one terminal (apt-get install -y htop) >> 2: run your application >> 3: check the memory usage on htop, if it use 100% of memory, then the >> machine will start to swap >> >> Please, correct me if I'm wrong. >> >
