I'm just checking in to see if anyone has been able to reproduce this
bug. It's a pretty nasty one and my solution does not scale
well--requires a lot of time when reopening a database to construct
the indexes.
-Red
On Tue, Dec 29, 2009 at 9:51 PM, Ian Eslick <esl...@media.mit.edu> wrote:
> I'll give it a try with that configuration...
>
> Thanks,
> Ian
>
> On Dec 29, 2009, at 9:42 PM, Red Daly wrote:
>
>> On Tue, Dec 29, 2009 at 5:53 PM, Ian Eslick <esl...@media.mit.edu> wrote:
>>> Hmmm....I can't reproduce this locally with my copy of latest.
>>>
>>> Your test code passes on a fresh DB as well. It's possible you're catching
>>> an error having to do with opening non 1.0 databases.
>>>
>>> A few quick sanity checks:
>>> 1) Did you clean out the test repository before running?
>>> (tests/delscript.sh)
>>> 2) Also, are you using elephant-1.0 or the defunct elephant-dev darcs
>>> repository?
>>
>>
>> I ran the test in a different, non-elephant directory with a fresh
>> darcs repository pulled from the elephant-1.0 tree. Here's a copy of
>> the my-config.sexp:
>>
>> ((:compiler . :gcc)
>> (:berkeley-db-version . "4.7")
>> (:berkeley-db-include-dir . "/usr/local/BerkeleyDB.4.7/include/")
>> (:berkeley-db-lib-dir . "/usr/local/BerkeleyDB.4.7/lib/")
>> (:berkeley-db-lib . "/usr/local/BerkeleyDB.4.7/lib/libdb-4.7.so")
>> (:berkeley-db-deadlock . "/usr/local/BerkeleyDB.4.7/bin/db_deadlock")
>> (:berkeley-db-cachesize . 20971520)
>> (:berkeley-db-max-locks . 2000)
>> (:berkeley-db-max-objects . 2000)
>> (:berkeley-db-max-transactions . 1000)
>> (:berkeley-db-map-degree2 . t)
>> (:berkeley-db-mvcc . nil)
>> (:clsql-lib-paths . nil)
>> (:prebuilt-libraries . nil))
>>
>> Did you try to reproduce this using Linux/SBCL? I don't know why it
>> would make a difference.
>>
>>>
>>> Thank you,
>>> Ian
>>
>> Best,
>> Red
>>
>>>
>>> On Dec 29, 2009, at 12:52 PM, Red Daly wrote:
>>>
>>>> This bug is easy to reproduce with the latest version of elephant.
>>>> Here is some code I used to reproduce the bug:
>>>>
>>>> (require :elephant)
>>>>
>>>> (defpackage :ele-bug
>>>> (:use :ele :cl))
>>>>
>>>> (in-package :ele-bug)
>>>>
>>>> (defclass a ()
>>>> ((x :initarg :x :index t :initform 0 :accessor x))
>>>> (:metaclass ele:persistent-metaclass))
>>>>
>>>> (defun create-instances (n &optional (class 'a))
>>>> (dotimes (i n)
>>>> (make-instance class :x i)))
>>>>
>>>> (defun stats ()
>>>> (dolist (class-sym '(a b))
>>>> (format t "~A: ~A instances; ~A with value 10~%"
>>>> class-sym
>>>> (length (get-instances-by-class class-sym))
>>>> (length (get-instances-by-value class-sym 'x 10)))))
>>>>
>>>> (defun open-close-test ()
>>>> (declare (optimize (debug 3)))
>>>> (macrolet ((is (form)
>>>> `(assert ,form)))
>>>> (flet ((open-it ()
>>>> (ele:open-store '(:bdb "db2/")))
>>>> (test-class-indexes (n-class n-10)
>>>> (is (= n-class (length (get-instances-by-class 'a))))
>>>> (is (= n-10 (length (get-instances-by-value 'a 'x 10))))))
>>>> (with-open-store ('(:bdb "db2/"))
>>>> (create-instances 100 'a)
>>>> (test-class-indexes 100 1))
>>>> (with-open-store ('(:bdb "db2/"))
>>>> (test-class-indexes 100 1)))))
>>>>
>>>> (open-close-test)
>>>>
>>>> I was able to fix the problem with a minor patch, but I doubt the
>>>> correctness of my solution. Here is the diff:
>>>>
>>>> diff -rN old-elephant-1.0/src/elephant/controller.lisp
>>>> new-elephant-1.0/src/elephant/controller.lisp
>>>> 711c711,712
>>>> < :key-form 'schema-classname-keyform))
>>>> ---
>>>>> :key-form 'schema-classname-keyform
>>>>> :populate t))
>>>> 716c717,718
>>>> < :key-form 'instance-cidx-keyform))))
>>>> ---
>>>>> :key-form 'instance-cidx-keyform
>>>>> :populate t))))
>>>>
>>>>
>>>> I essentially just added a :populate t argument to the open-controller
>>>> method. This seems to indicate that the 'BY-NAME secondary indices
>>>> that map a class indices to OIDs, and class names to class indices, do
>>>> not persist. Is this expected behavior? I'm not familiar enough with
>>>> the internals to tell.
>>>>
>>>> Thanks,
>>>> Red
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Sep 23, 2009 at 3:27 PM, Red Daly <redd...@gmail.com> wrote:
>>>>> This is the 1.0 repository. I will see if I can reproduce the problem in
>>>>> a
>>>>> test case.
>>>>> Red
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sep 23, 2009, at 9:50 AM, Ian Eslick <esl...@media.mit.edu> wrote:
>>>>>
>>>>> Which darcs repository? There is the old 0.91 release and the new
>>>>> elephant-1.0 branch...
>>>>> Ian
>>>>> On Sep 22, 2009, at 8:41 PM, Red Daly wrote:
>>>>>
>>>>> I should also note that I am using the version of elephant from darcs.
>>>>> And
>>>>> I discovered that when the program went down it was due to a segfault (I
>>>>> believe in emacs). However, all instances had been created under many
>>>>> different transactions that did not abort.
>>>>>
>>>>> Here is some more fiddling around:
>>>>>
>>>>> CL-USER> (lisp-implementation-version)
>>>>> "1.0.29.54.rc1"
>>>>> BOOKS> (ele::get-db-schemas *ele-db* 'material)
>>>>> NIL
>>>>> BOOKS> (ele::controller-instance-class-index *ele-db*)
>>>>> #<BDB-BTREE-INDEX
>>>>> oid:13701>
>>>>> BOOKS> (ele:map-btree #'(lambda (key
>>>>> val)
>>>>> (format t "~S => ~S~%" key
>>>>> val))
>>>>> (ele::controller-instance-class-index *ele-db*))
>>>>> 307 =>
>>>>> 307
>>>>> NIL
>>>>>
>>>>>
>>>>> Red
>>>>>
>>>>> On Tue, Sep 22, 2009 at 8:08 PM, Red Daly <redd...@gmail.com> wrote:
>>>>>>
>>>>>> I noticed there was a disparity between get-instances-by-class and
>>>>>> map-inverted-index. I will let the code speak for itself:
>>>>>>
>>>>>> BOOKS> (length (ele:map-inverted-index #'(lambda (isbn obj) obj)
>>>>>> 'material
>>>>>> 'isbn :collect
>>>>>> t))
>>>>>>
>>>>>> 1945
>>>>>> BOOKS> (length (ele:get-instances-by-class 'material))
>>>>>> 0
>>>>>> BOOKS> (defparameter f (make-instance 'material :title "Fake Book"))
>>>>>>
>>>>>> F
>>>>>> BOOKS> (length (ele:get-instances-by-class 'material))
>>>>>>
>>>>>> 1
>>>>>> BOOKS> (length (ele:map-inverted-index #'(lambda (isbn obj) obj)
>>>>>> 'material
>>>>>> 'isbn :collect
>>>>>> t))
>>>>>> 1946
>>>>>>
>>>>>> There is a similar problem with the university class.
>>>>>>
>>>>>> BOOKS> (length (ele:get-instances-by-class 'university))
>>>>>> 0
>>>>>> BOOKS> *stanford-university*
>>>>>> #<UNIVERSITY
>>>>>> oid:204>
>>>>>> BOOKS> (official-name *stanford-university*)
>>>>>> "Stanford University"
>>>>>>
>>>>>> All of the instances in the database were created in a prior session.
>>>>>> Then I shut down the DB and restarted Lisp. Now MAP-CLASS and
>>>>>> GET-INSTANCES-BY-CLASS return nil for all classes. It seems like new
>>>>>> instances get added to the "class index," but older instances are not
>>>>>> found. However, these classes persist in the inverted slot indexes. Can
>>>>>> anybody diagnose what's going on?
>>>>>>
>>>>>> here are the definitions of the classes:
>>>>>>
>>>>>> (defclass department ()
>>>>>> ((university :initarg :university :initform nil :accessor
>>>>>> department-university
>>>>>> :index t)
>>>>>> (official-name :initarg :official-name :initform nil :accessor
>>>>>> official-name)
>>>>>> (common-name :initarg :common-name :initform nil :accessor
>>>>>> common-name)
>>>>>> (abbreviation :initarg :abbreviation :initform nil :accessor
>>>>>> abbreviation
>>>>>> :index t))
>>>>>> (:metaclass ele:persistent-metaclass)
>>>>>> (:documentation "A department has a bunch of course listings."))
>>>>>>
>>>>>> (defclass material ()
>>>>>> ((title :initarg :title :initform nil :accessor material-title
>>>>>> :index t)
>>>>>> (authors :initarg :authors :initform nil :accessor material-authors)
>>>>>> ;;
>>>>>> a list of
>>>>>> authors
>>>>>> (edition :initarg :edition :initform nil :accessor material-edition)
>>>>>> (isbn :initarg :isbn :initform nil :accessor material-isbn
>>>>>> :index t)
>>>>>> (asin :initarg :asin :initform nil :accessor material-asin
>>>>>> :index t
>>>>>> :documentation "An Amazon.com identifier that uniquely
>>>>>> identifies
>>>>>> the material")
>>>>>> (image-uri :initarg :image-uri :initform nil :accessor
>>>>>> material-image-uri))
>>>>>> (:metaclass ele:persistent-metaclass))
>>>>>>
>>>>>> Thanks,
>>>>>> Red
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> elephant-devel site list
>>>>> elephant-devel@common-lisp.net
>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>>>>
>>>>> _______________________________________________
>>>>> elephant-devel site list
>>>>> elephant-devel@common-lisp.net
>>>>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>>>
>>>> _______________________________________________
>>>> elephant-devel site list
>>>> elephant-devel@common-lisp.net
>>>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>>
>>>
>>> _______________________________________________
>>> elephant-devel site list
>>> elephant-devel@common-lisp.net
>>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>>
>>
>> _______________________________________________
>> elephant-devel site list
>> elephant-devel@common-lisp.net
>> http://common-lisp.net/mailman/listinfo/elephant-devel
>
>
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
