changing that number did the trick. I tried 50006 first and it still split, so I tried 10006 and that worked great.
I tested the new version, and each time it went through the flushing/splitting process I saw this:
...
SCANNING movies: Hu xi (2005) (movieID: 190001)
* FLUSHING MoviesCache...
* TOO MANY DATA (100000 items), SPLITTING (run #1)...
* FLUSHING MoviesCache...
* TOO MANY DATA (50000 items), SPLITTING (run #2)...
* FLUSHING MoviesCache...
* TOO MANY DATA (25000 items), SPLITTING (run #3)...
* FLUSHING MoviesCache...
* FLUSHING MoviesCache...
* FLUSHING MoviesCache...
* TOO MANY DATA (25000 items), SPLITTING (run #4)...
WARNING recursion level exceded trying to flush data
WARNING this batch of data is lost.
WARNING recursion level exceded trying to flush data
WARNING this batch of data is lost.
SCANNING movies: Ink (2007) (movieID: 200001)
Sometimes the "recursion level" messages appeared twice, sometimes once, always at least once. Searching for some TV shows, it appears that there are no duplicate records--one record for each show, and the episodes correctly pointing back to it in episode_of_id. However there are definitely some data missing, as the messages indicated there would be. Let me know if you come up with another version or change, I'll be glad to test it out. Thanks again for the help in getting it working.
-Jay
On 10/19/06, Davide Alberani <[EMAIL PROTECTED]> wrote:
On Oct 18, Davide Alberani <[EMAIL PROTECTED]> wrote:
> Yes; the bug must be there. :-/
Attached (and committed to the CVS), there is a new version of
the imdbpy2sql.py script, with some changes to the code used to
split too large data sets.
With very limited tests, it seems to work correctly, but...
who knows? :-)
If you can test it, it would be great!
--
Davide Alberani <[EMAIL PROTECTED]> [PGP KeyID: 0x465BFD47]
http://erlug.linux.it/~da/
------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________ Imdbpy-devel mailing list Imdbpy-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-devel
