Re: [computer-go] Serializing a very large object in Java
Peter Drake wrote: out.writeObject(root); Storing the root node of the Monte Carlo search will get storeObject() called on all the siblings and children (assuming you are using an implementation similar to the one on http://senseis.xmp.net/?UCT. With 10.000 nodes, the depth of the tree is a little less than with 2.000.000 nodes. Since for each sibling (important) and each child (not so important), the writeObject() method is called, you get that stack overflow. Since there are as many siblings per move as valid moves, the recursion depth grows quite quickly. You can avoid that by not calling writeObject(this.sibling), but doing that in a while (...) loop. Roland ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Serializing a very large object in Java
Peter Drake wrote (3 times): Exception in thread main ... . . . and 91.449 bytes later . . . ... (ObjectOutputStream.java:1369) I studied the log file in depth and the problem is . . . . . . (you guessed) using Java ;-) Jacques. BTW. I store this list. If you need your log file in the future, contact me cos I have 3 copies. ;-) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Serializing a very large object in Java
Peter, java serialization is not a good way to do persistent storage of any kind, especially large data structures. It has some pretty severe drawbacks: - It is slow - It breaks easily (ie, becomes unable to load older data sets) when you make even small changes in your code. - It makes inefficient use of space - The format is difficult to decode or manipulate in any way other than reading or writing with the exact .class file used to generate the serialization code. Java serialization is excellent for RMI, but is pretty poor for any other use. I've used serialization myself several times and I regretted it every time (except for when I used it for RMI). My advice would be to come up with your own data format and use that. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Serializing a very large object in Java
Is this opening book database used for the UCT portion, or the playout portion of Orego? In the UCT portion, speed of access may not be that important; a database would probably be ideal. If used during the playout, then speed of access is more crucial. Terry McIntyre UNIX for hire software development / systems administration / security [EMAIL PROTECTED] - Original Message From: Nick Apperson [EMAIL PROTECTED] What type of data are you trying to serialize or rather store to disk? Do you have pointers in the data. Don't tell me you don't have pointers just because it is java. Java has pointers, it just preteneds it doesn't. Show us the datastructure and we can probably help you more. If each entry has no pointers than your task should be pretty straightforward. If you do have them, you are going to need to find a way to store those so that they are able to be redetermined. In a language like java, where you can't do pointer arithmetic or anything of the sort this can get rather difficult. Also, have you considered using a database instead of a flat file to store this data? - Nick On 2/9/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Peter, java serialization is not a good way to do persistent storage of any kind, especially large data structures. It has some pretty severe drawbacks: - It is slow - It breaks easily (ie, becomes unable to load older data sets) when you make even small changes in your code. - It makes inefficient use of space - The format is difficult to decode or manipulate in any way other than reading or writing with the exact .class file used to generate the serialization code. Java serialization is excellent for RMI, but is pretty poor for any other use. I've used serialization myself several times and I regretted it every time (except for when I used it for RMI). My advice would be to come up with your own data format and use that. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ Any questions? Get answers on any topic at www.Answers.yahoo.com. Try it now.___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Serializing a very large object in Java
On Feb 9, 2007, at 10:48 AM, Nick Apperson wrote: What type of data are you trying to serialize or rather store to disk? Do you have pointers in the data. Don't tell me you don't have pointers just because it is java. I wouldn't dream of it. Java has pointers and I use the hell out of 'em. Java has pointers, it just preteneds it doesn't. Show us the datastructure and we can probably help you more. If each entry has no pointers than your task should be pretty straightforward. If you do have them, you are going to need to find a way to store those so that they are able to be redetermined. Java's serialization mechanism does this automagically. In a language like java, where you can't do pointer arithmetic or anything of the sort this can get rather difficult. Also, have you considered using a database instead of a flat file to store this data? I'm going to try Terry's idea of storing individual Nodes instead of storing the entire Node pool. Stay tuned... Peter Drake Assistant Professor of Computer Science Lewis Clark College http://www.lclark.edu/~drake/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Serializing a very large object in Java
The UCT portion. I'm storing/loading a pre-built UCT tree once at startup; the disk is not accessed during the game. Peter Drake Assistant Professor of Computer Science Lewis Clark College http://www.lclark.edu/~drake/ On Feb 9, 2007, at 11:08 AM, terry mcintyre wrote: Is this opening book database used for the UCT portion, or the playout portion of Orego? In the UCT portion, speed of access may not be that important; a database would probably be ideal. If used during the playout, then speed of access is more crucial. Terry McIntyre UNIX for hire software development / systems administration / security ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
RE: [computer-go] Serializing a very large object in Java
Three alternative options to Java's native serialisation: * Object database db4o: http://db4o.com * WOX (Web Objects in XML) (my own) http://algoval.essex.ac.uk/wox/serial/readme.html * JSON (JavaScript Object Notation) - also has Java libraries. Simon Lucas -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: 09 February 2007 18:40 To: computer-go@computer-go.org Subject: Re: [computer-go] Serializing a very large object in Java Peter, java serialization is not a good way to do persistent storage of any kind, especially large data structures. It has some pretty severe drawbacks: - It is slow - It breaks easily (ie, becomes unable to load older data sets) when you make even small changes in your code. - It makes inefficient use of space - The format is difficult to decode or manipulate in any way other than reading or writing with the exact .class file used to generate the serialization code. Java serialization is excellent for RMI, but is pretty poor for any other use. I've used serialization myself several times and I regretted it every time (except for when I used it for RMI). My advice would be to come up with your own data format and use that. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Serializing a very large object in Java
At 09:55 AM 2/9/2007, you wrote: ..., Java has a stack overflow error. i assume you have tried the java -Xss to set the stack size (type java -X for help on these)? thanks --- vice-chair http://ocjug.org/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
