[ZODB-Dev] LargeFS

2006-02-11 Thread Tamas Hegedus
Hi, I beg for your patience :-) I run into a problem that I can not solve. It could be something very trivial, but ZODB seems to be so simple that I can not figure out what could be the problem. I have to text files: 1. 800M, approx 200,000 records 2. 6G, approx 2,000,000 records I parse the

[ZODB-Dev] Re: LargeFS

2006-02-11 Thread Tamas Hegedus
Hi! It seems that the savepoint() does not work (ZODB3-3.5.1). Now I am using commits after every 50,000 objects. (Originally I had to be wrong about the differences populating using savepoints or commits: it should have never worked.) I let you know the final results. T -- Tamas Hegedus, P

[ZODB-Dev] LargeFS3

2006-02-11 Thread Tamas Hegedus
Hi, I just would like to summarize my experience: Two files: 1. 800M; approx 200,000 record 2. 6G; approx 2,400,000 record --- POPULATION: a, it was pretty linear (see details below); - average values: 50,000 object in 2-4 minutes; 600M RAM; 120M swap - after the