Hi Christopher, Regarding to the limitations of your machine, you could consider using CloudArray.jl (https://github.com/gsd-ufal/CloudArray.jl) and their cloud platform CloudArrayBox (http://cloudarraybox.cloudapp.net/)
Hope it helps, Charles On 26 January 2016 at 18:41, Christopher Fisher <fishe...@miamioh.edu> wrote: > Perhaps if its a precompling issue, it would be faster if I precomplied > with a smaller distributed array. I'll try that and report back in the next > day or two. > > > On Tuesday, January 26, 2016 at 12:28:21 PM UTC-5, Christopher Fisher > wrote: >> >> Thanks for the idea. It may not apply in my particular case. I forgot to >> mention that my memory is 32GB on the machine I am using. So swapping with >> the hard drive does not seem likely. >> >> Given that the second execution of the function is much faster (1 second >> vs 5 min), my best guess is that the long initial run might be due to >> compiling. >> >> On Tuesday, January 26, 2016 at 12:12:41 PM UTC-5, nae...@ic.ufal.br >> wrote: >>> >>> Hello Christopher, >>> >>> I saw you code and you noticed you are reading all the data(3GB) >>> locally, maybe it's causing your machine to use swap memory, which leads to >>> the performance issue. >>> You can check that by >>> 1: open htop in one terminal (apt-get install -y htop) >>> 2: run your application >>> 3: check the memory usage on htop, if it use 100% of memory, then the >>> machine will start to swap >>> >>> Please, correct me if I'm wrong. >>> >> -- Um axé! :) -- Charles Novaes de Santana, PhD https://github.com/cndesantana
