On Jan 4, 2008 2:15 AM, Leslie P. Polzer <[EMAIL PROTECTED]> wrote: > > > I've decided to put elephant 0.9.1 under some heavy load test, and > > play with Netflix data set a little bit. The attached program that > > tries to import everything in BerkeleyDB fails when trying to import > > movie file number 8 with the following traceback. Do you have any idea > > what could be the problem? > > No, but we might if you tell us > > - OS > - RAM > - Swap limits > - Kernel limits on memory (e.g. /etc/security/limits.conf for GNU/Linux)
Of course :). # uname -a Linux esculap 2.6.22-14-generic #1 SMP Tue Dec 18 05:28:27 UTC 2007 x86_64 GNU/Linux (Ubuntu 7.10) RAM: Physical 2GB, SWAP 4GB Lisp: SBCL 1.0.13 /etc/security/limits.conf contains only commented out strings. > - size of the data you're trying to pull Well, I'm trying to import Netflix data set. It consists of ratings (from 1 to 5) for 17770 movies given by 480189 different users; total number of ratings is 100480507. Honestly, I don't know how to map that nicely into the Elephant world, so I'm using the following approach; each user (or movies) is a class with 2 slots: id of type (integer 32), and ratings, which is btree mapping movies (or users) ids into respective ratings. I *suspect* that data mapping above is horribly inefficient (too many btrees?), but my intuition may be wrong. Anyway, first 8 files contain exactly 20008 ratings, so at the point of failure we should have created: No more then 8 movie objects No more then 20008 user objects No more then 20016 (= 20008 + 8) btrees (slot elements of the above two classes). I'd be happy to provide more details. Regards, Victor. _______________________________________________ elephant-devel site list elephant-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel
