Hi Anders! How're you doing ?
Thanks for the suggestion. We actually have a solution in production right now (www.besport.com) that relies on netshm. Unfortunately, the serialization / deserialization is killing the performance : about 200ms for a full request against 7ms for a simple Hashtbl lookup on our hardware. ocaml-everlasting did solve this performance bottleneck but we get unacceptable segmentation faults, and before debugging this library I'd rather learn about other approaches :) Best regards, William On Thu, Dec 8, 2011 at 12:49 AM, Anders Fugmann <[email protected]> wrote: > Hi William, > > Have you tried netshm from ocamlnet library? It seems that it would fit > your requirements by placing memory outside the reach of the garbage > collector. > > Using netshm would require a lot of serialization / deserilization and > memory copying, which might not be that efficient if you access the cache a > lot. > > Regards > Anders Fugmann > > > > > On 12/08/2011 05:35 AM, William Le Ferrand wrote: > >> Dear list, >> >> We are building a cache in ocaml and we're wondering if it would make >> sense to store ocaml values outside the reach of the gc. (gc on a 20GB >> cache hangs the process for a second or so). >> >> To run some experiments, we wrote a small library >> (https://github.com/besport/**ocaml-everlasting<https://github.com/besport/ocaml-everlasting>) >> that exposes two >> functions, get and set. >> >> When inserting a value, we copy recursively the blocs outside of the >> reach of the gc (and put the resulting value in some C array). When >> getting the value, we simply pass the pointer to the copied value to the >> ocaml code (the structure is still coherent and the value is directly >> usable). We also wrote an "update" function that compare a new value >> with the existing value in cache, to avoid unnecessary memory >> allocation/deallocation. >> >> It does not seems very stable though, but I don't know if it is a bug in >> the update function or simply because this approach is not reasonable. >> Do you have any thoughts? Is there any clever way to build a large cache >> in an ocaml app ? >> >> Thanks in advance for any tips! >> >> Best >> >> William >> >> >> -- >> William Le Ferrand >> >> Mobile : (+1) (415) 683-1484 >> Web : >> http://williamleferrand.**github.com/<http://williamleferrand.github.com/> >> <http://www.linkedin.com/in/**williamleferrand<http://www.linkedin.com/in/williamleferrand> >> > >> >> > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa-roc.inria.fr/**wws/info/caml-list<https://sympa-roc.inria.fr/wws/info/caml-list> > Beginner's list: > http://groups.yahoo.com/group/**ocaml_beginners<http://groups.yahoo.com/group/ocaml_beginners> > Bug reports: > http://caml.inria.fr/bin/caml-**bugs<http://caml.inria.fr/bin/caml-bugs> > > -- William Le Ferrand Mobile : (+1) (415) 683-1484 Web : http://williamleferrand.github.com/ <http://www.linkedin.com/in/williamleferrand> -- Caml-list mailing list. Subscription management and archives: https://sympa-roc.inria.fr/wws/info/caml-list Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
