Correction: "Each file compiles in* around 14 seconds*, reducing the compile time to around 1.5 hours. " should be 70 seconds each (see the table in my post). 70*61=4270 seconds = 1.2 hours.
On Sunday, May 28, 2017 at 7:50:27 PM UTC-4, rocketpwr.com wrote: > > Dear OpenCog Development Community, > > First, thank you for your hard work on this project. > > I found a few bugs that I have been able to work around that you might > find helpful. Hopefully the below description of the problems and my > work-arounds will help you or someone else who comes across the same > problems. > > *1) Really really long compile times on loading up the conceptnet4.scm > test dataset in test-datasets/conceptnet/conceptnet4.scm.* I left it to > run overnight, and it still had not finished. Actually, I believe it was > going to take 70+ hours to load and compile this 14.5 MB file. This seemed > strange to me as smaller files loaded much faster. What I determined by > experiment is that, for some unknown reason, the Guile Scheme compiler had > a compile time that is proportional approximately n^2 of the number of > rules. Here are some results on my test rig (a 4 core 4 GB Ubuntu 16.04 > setup): > > Rules loadtime rules/sec > > 259 7 37.00 > 634 31 20.45 > 1000 70 14.29 > 1249 110 11.35 > > > Because the Conceptnet4.scm file is about 60.5K I estimate that the > compile time would be around 71 hours or 3 days. I am guessing this is a > bug/feature in Guile Scheme which was not really meant as a database. > > My work-around: a) split the 60K rule file into 61 files of 1K rules > each, and then compile them. Each file compiles in around 14 seconds, > reducing the compile time to around 1.5 hours. Once compiled, the rules do > not need to be compiled again. b) I wrote a small loader that loads all 61 > files into Guile in a single command. > > *2) Really long execution time for (cog-bind deduction-inheritance-rule):* > I am experimenting with PLN, using the how-to tutorial: PLN by hand ( > http://wiki.opencog.org/w/PLN_by_Hand). After loading up conceptnet4, I > tried to run the inheritance rule as per the tutorial: > > (cog-bind deduction-inheritance-rule) > > Again, I left the computer to run overnight, and by morning it wasn't > complete. I searched the machine and found the errors in the opencog.log > file which has grown to 1 GB. It was filled with a repeat of this message > (with some other hex-addresses changed): > > In unknown file: > ?: 7 [opencog-extension cog-bind-first-n (# -1)] > In ice-9/boot-9.scm: > 157: 6 [catch #t #<catch-closure 2b750a0> ...] > In unknown file: > ?: 5 [apply-smob/1 #<catch-closure 2b750a0>] > In ice-9/boot-9.scm: > 157: 4 [catch #t #<catch-closure 2b77be0> ...] > In unknown file: > ?: 3 [apply-smob/1 #<catch-closure 2b77be0>] > In opencog.scm: > 89: 2 [cog-undefined-handle] > In ice-9/boot-9.scm: > 102: 1 [#<procedure 95b1940 at ice-9/boot-9.scm:97:6 (thrown-k . args)> > wrong-number-of-args ...] > In unknown file: > ?: 0 [apply-smob/1 #<catch-closure 2b77ba0> wrong-number-of-args ...] > > ERROR: In procedure apply-smob/1: > ERROR: Wrong number of arguments to #<procedure cog-undefined-handle (X)> > ABORT: wrong-number-of-args > > > > I have been researching this for a few hours and I was about to give up. > I then decided to experiment with smaller numbers of atoms (rules). I > used the Guile profiler and it showed that the majority of the time was > being wasted in error "Catch" and formatting functions. After setting debug > breakpoints, I decided that the problem was actually in this > cog-undefined-handle function, and not the cog-bind-first-n function, or > whatever apply-smob/1 is. (I can't find any documentation on apply-smob/1.) > The cog-undefined-handle function was in > /usr/local/share/opencog/scm/opencog.scm: > > I changed this line: > > (define-public (cog-undefined-handle X) '()) > > to > > (define-public (cog-undefined-handle . X) '()) > > Note: I am not expert in Guile, Scheme, or Lisp. A poster on stack > exchange said that by inserted the dot character in the argument list, it > allows a Scheme function to take any number of arguments. This seems to > work -- now the opencog.log is clean and the (cog-bind > deduction-inheritance-rule) runs on the entire data set in only a few > minutes. > > *3) Hard coded gcc/g++ 4.8 compiler version in octool prevents > installation on Ubuntu 16.04 LTS*. I understand that the base Linux > configuration for Opencog is still 14.04 LTS. However, that version is > already 3 years old and I had just installed a 16.04 LTS image, and I did > not want to reinstall after investing the time to set it up properly. To > get octool to finish the compilation and subsequent installation, I did the > following change: > > $ diff octool octool~ > 843,844c843 > < #CXX=g++-4.8 cmake .. -DCMAKE_BUILD_TYPE=Release > < CXX=g++ cmake .. -DCMAKE_BUILD_TYPE=Release > --- > > CXX=g++-4.8 cmake .. -DCMAKE_BUILD_TYPE=Release > > > A more clever programmer can ensure that the a version of g++ <= 4.8 is > used by octool script. > > *4) Old python conceptnet converter.py doesn't match newer 5.5.0 version > of conceptnet. * [NOTE: The following python code is likely deprecated in > favor of newer c++ converter found here: > https://github.com/opencog/external-tools/tree/master/cnet_importer ] > > Earlier in the week, I tried to convert an English only > conceptnet-assertions-5.5.0.csv file prepared by grep filtering out the > non-English assertions. Unfortunately the Python converter.py that was > downloaded (I believe by octool) does not work on this file. Note: this > version of converter.py was installed in my > ~/opencog/opencog/opencog/python/conceptnet/ directory. I further modified > the conceptnet-assertions.5.5.0.csv file work by changing the column format > to the same as the conceptnet4.csv file. This allowed the python converter > script to run to the end of the file without terminating prematurely. > > ----------- > > Overall I am still working to understand Opencog and its capabilities. > > Finally, if anyone has uploaded a recent dbpedia ruleset to Opencog, I > would like to experiment with it as well. Please drop a line. > > Thank you again. > > > > -- You received this message because you are subscribed to the Google Groups "opencog" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/opencog. To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/3b87c087-fe03-4ed3-a3b2-80d9b10dc994%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
