It's likely that much of the memory was for temporary allocations that were reused.
Cheers, Kevin On Sun, Jul 6, 2014 at 10:32 AM, Sid <[email protected]> wrote: > Regarding memory allocation -- it seems like it might be a bug in the > @allocated macro rather than an actual indication of memory usage. > > When I run > > a = rand(100000) > @allocated sort(a, by=x->x) > > I get 768403668 which is 768 MB. > > However, I was also recording the process's memory usage using %mem. For > my 16 GB machine, the usage initially is 0.5% and jumps to 1.3% after > running the above two commands which comes to around ~200MB rather than > 768MB. > > While that's still a high jump compared to the array size (which is only > ~8MB), that might be because of the way VM allocates more memory. Perhaps > someone who's more familiar with the internals can comment. > > On Sunday, July 6, 2014 7:30:47 AM UTC-4, gentlebeldin wrote: >> >> No, that wasn't the reason, in this case. I had another, hard look at my >> own (rough and ready) implementation of factoring, found a rather stupid >> mistake, and now, it runs in 2.2 seconds. >> Using List<Integer> or List<Double> instead of int[] or double[] for >> numerical computations would be a silly idea, indeed, and not just because >> of poor performance. It would also eat up lots of memory. In this case, I >> had to sort a big array of integers (100,000,000 elements), with a custom >> comparator. That isn't possible with Java's int[], and Integer[] with so >> many elements is a bit harsh for a netbook with 2 GB of RAM. Then, I >> thought "hey, I can implement the List<Integer> interface with an int[]!" >> No such luck, the implementation of Collections.sort calls toArray, >> first, ->Integer[] :-( >> Then, they introduced streams in Java 8, including IntStream, a stream of >> ints, for better performance, and defined the corresponding functional >> interface for a custom comparator. That sounds promising, but as we know, >> most promises aren't kept. Believe it or not, the implementation of >> IntStream.sort collects the whole stream in an Integer[], first. :-( >> That's why I put some hope into Julia. Unfortunately, it may allocate >> huge amounts of memory in an unpredictable way (for me, at least), too. >> >> Am Sonntag, 6. Juli 2014 07:12:26 UTC+2 schrieb Sid: >>> >>> Are you using Integer/Double objects by any chance in Java? I've found >>> using them to be a surefire way of destroying Java's performance... >>> >>> On Saturday, July 5, 2014 3:11:47 AM UTC-4, gentlebeldin wrote: >>>> >>>> It's hard to tell for sure, I wouldn't know how to check that. I prefer >>>> to port a few more solutions. My curiosity was much increased by this one: >>>> >>>> julia> include("julia/jl/e443gcd.jl") >>>> elapsed time: 3.255950529 seconds (929628 bytes allocated) >>>> >>>> >>>> The Java original "runs" much longer, 33 seconds. >>>> >>>> >>>>>
