Thank you for the details
I intend to use JudyL and JudySL where expanse would be close to 125Million
and Population around 10Million. Values are consumed *as-is* without further
dereferencing. After verifying several other options - JudyL and Key Less
Caches worked out fine for me.
A simple build from file using inserts(10M) took 6 seconds(excluding file
read time) and its within the acceptable limits. On the opposite direction,
a similar approach *for all keys, get(key,value); persist" wouldn't fit in
because I don't have keys outside the array. Is there a way to *read out all
key-values* in one go?
Thanks Again,
Pramod K P
On Tue, Sep 29, 2009 at 7:58 AM, Alan Silverstein <[email protected]> wrote:
> > How do we persist <key, value> pairs from Judy array into a file?
>
> There's never been a supported solution, although it's something we
> certainly studied back in 1999-2001. Basically what you really want is
> a "batch load/unload" capability. And, if I recall right, in the source
> software package there IS some experimental, unsupported code to do just
> this -- I know because I wrote it -- and it's "not known not to work."
> I'm just not sure it's on SourceForge. If not, I might be able to dig
> it up.
>
> Doug asserted that libJudy set/insert (and readout) was so fast there
> was little point to batch operations, and he was mostly correct, but if
> I recall right years later, there was SOME speedup from having code
> knowing how to build completed internal tree nodes all at once, not
> incrementally.
>
> Beware, any batch in/out of libJudy arrays is only useful if the data is
> fully in the array; that is, Judy1, or JudyL with the values being the
> real data, not just a pointer to other, malloc'd memory. In that case
> you just gotta write your own load/unload that traverses the array to
> unload, and traverses the file data to reload.
>
> Cheers,
> Alan Silverstein
>
------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Judy-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/judy-devel
