> > Just thinking, but it seesm to me that you might be able to use mmap > > with your own version of malloc, which allocates a chunk inside of the > > mmaped area, if your careful you might be able to keep a rough > > ordering of the elements, keep the most common one(s) in memory, and > > also allow sequentail access. But don't take this to seriously, I'm > > an outsider looking in. > > Sure, you are talking about optimizing malloc(). For a long time we had > an intermediate layer within libJudy to get only large chunks from > malloc() (think sort-of mmap) and then do our own app-specific memory > allocation for reasons like you are saying. But Doug ditched that when > he explored around and found wide variations in malloc() libraries, with > Ian Lea (I think) malloc() being the best. > > Typical problem, you can often do a lot more optimization if you know > specifics that an app needs, versus a cleaner but dumber approach > through standard interfaces. >
This is a tad rantish, but I can't think of a better way so... I agree, the problem is telling the system which version of malloc to use. For example: On my gentoo system I have two choices for malloc (there might be more, I just did a quick search in my package manager for "malloc") if I installed libc, and jemalloc which one would judy use? Typically, to use a different library for malloc instead of the traditional one you'd need to include that libraries header. If that header was not found then you'd need to include the next best implimentations header.... This can easily become a painstaking and time consuming task, becuase you'd have to ensure that all the programs on the system use the "supported" version of the malloc libraries. And then you'd need to consider the possibility that there is a malloc out there that you're unaware of, perhaps it's better then the other choices, perhaps it's worse and that a person out there only uses that package for thier whole system. You'd also have to explain to everyone why there is the dependency on some wierd malloc. And there would be complaints as to why you dare to use a wierd version of malloc, why the current one is bad, bug reports because x version of judy runs slower then y.... I could go on, but as you can see, the better the program, the harder it is to impliment it. Peolpe like the easiest choice these days. It is really quite sad, because those of us who are willing to put forth the necessary effort are normally jaded into something dumber because the best choice is nowhere to be found... David ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Judy-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/judy-devel
