Hi Alan, David and Judy friends:
<snip>
> In 2014 two things happened that brought understanding to what was the
> "green eyed monster". The recent discussion on malloc is related to one
> of thoes things. A modification to malloc in concert with an
> enhancement to the OS alone will improve the performance of Judy (1.0.5)
> by 30-40% with large data sets (1GB+). The OS enhancement was made to
> the Ubuntu Kernel (i think it is just an #ifdef in Linux) in spring
> 2014. I have not found another OS with the same enhancement yet (Mac
> OS-X, BSD, Windows). It deserves a lot of discussion and I will be
> happy to participate, but not in this thread. The other thing that
> happened was the release of the Intel Haswell processor. This also
> deserves a lot of discussion and I will again be happy to participate.
> "green eyed monster". The recent discussion on malloc is related to one
> of thoes things. A modification to malloc in concert with an
> enhancement to the OS alone will improve the performance of Judy (1.0.5)
> by 30-40% with large data sets (1GB+). The OS enhancement was made to
> the Ubuntu Kernel (i think it is just an #ifdef in Linux) in spring
> 2014. I have not found another OS with the same enhancement yet (Mac
> OS-X, BSD, Windows). It deserves a lot of discussion and I will be
> happy to participate, but not in this thread. The other thing that
> happened was the release of the Intel Haswell processor. This also
> deserves a lot of discussion and I will again be happy to participate.
You mentioned two, but I only see one ;)
Do you have any links to this info it sounds interesting.
> To get to the initial question asked by David, I have some thoughts.
> First I do not use swapping anymore. I find it too slow, while RAM is
> so cheap that swapping is not cost effective anymore.
That depends on if you feel like buying a new computer, often times,
sadely, some hardware is not avalible for a newer computer that was
for an older one, for example, the serial port.
Also, I intend the software that I write to be used by others, they
may not have the money and I don't what to push if I can help it :)
And, if you use madvise() you can tell the kernel that you're going to
do sequentail access, whihc is what I was talking about, and that speeds
up the access to a large extent form what I understand (though I've
done no benckmarks.)
> For example, the
> last notebook I bought a year ago at WalMart was $250 and had 4GB of
> RAM. I don't think anybody sells a new computer with less that 4GB of
> RAM. The "small" test machine I use for testing Judy performance is a
> Haswell (G3258) with 32GB of RAM.
> last notebook I bought a year ago at WalMart was $250 and had 4GB of
> RAM. I don't think anybody sells a new computer with less that 4GB of
> RAM. The "small" test machine I use for testing Judy performance is a
> Haswell (G3258) with 32GB of RAM.
Small????? The biggest compter that anyone in my family owns has
8GiB and has an AMD 1090T processor.
> The current Judy design has awesome
> performance. Perhaps 3-4X of the performance of released Judy with
> large data sets (4GB+). Second, the real elephant in the room regarding
> malloc is the small change to malloc and the enhanced OS (kernel) that
> need to be done by every operating system.
> performance. Perhaps 3-4X of the performance of released Judy with
> large data sets (4GB+). Second, the real elephant in the room regarding
> malloc is the small change to malloc and the enhanced OS (kernel) that
> need to be done by every operating system.
What do you mean by current? Is that the development version?
> As far as I know, all modern malloc's use anonymous mmap to get memory
> from the OS and if they do not, then they are not modern. I believe
> sbrk is deprecated in Mac OS-X and is not used in Linux malloc. Mmap is
> preferred because of the control over the logical address of the
> de/allocated memory which allows logical "holes" without wasting
> physical RAM.
I disagree, glibc still used sbrk a year ago, at least acording to the docs...
> P.S I am building a house in Thailand now and have little time to work
> on Judy right now.
Understood. I helped my dad (a lot) build a house. 5 years ago it was finished
and it's still in good shape. If you need a pointer you can drop me a line,
so many things go into a house that you would not think of. For example,
(my dad got this wrong), using two p drain pipes in a row will defeat the
caching purpose of the one.
David
------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________ Judy-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/judy-devel
