Re: 32 to 64 database conversion problem
On Mon, Oct 03, 2011 at 12:56:31PM +0700, Henrik Sarvell wrote:
> Seems like I need to pass some arguments, rebuild (X Var Cls Hook)
>
> Can you give me an example, what could X and Var be, Cls is probably
> something like '+Class
Yes. 'rebuild' needs to know _which_ objects in the DB should be put
into the index. For that, you have several options:
1. Supply a list of objects in the first argument ('X'). This makes
sense if the number of involved objects is not too big (a few
million perhaps). You must somehow obtain the list of objects. If
such a list is not known, it is typically built from some other
(non-defect) index:
(rebuild (collect 'foo '+Cls) 'bar '+Cls)
2. Pass the number of the DB file. In that case, 'rebuild' will go
through that file, and put all objects of type '+Cls' it finds
into the index. This is the easiest way, and also not limited by
any memory constraints like (1). But you don't have control about
which objects are put into the index, they'll be all used:
(rebuild 7 'bar '+Cls)
Here, '7' is assumed to be the DB file. You do normally obtain it
with 'db:'
(rebuild (db: +Cls) 'bar '+Cls)
3. If 'X' is NIL, 'rebuild' tries to do the 'collect' by itself. This
makes only sense for a non-damanged index, and is used when e.g.
the DB model of that index changed.
> but what is Hook?
'Hook' is another object, being the base of an object-local index. If
you haven't heard of it, you probably haven't used it ;-) See the
reference of '+Hook'.
Cheers,
- Alex
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: 32 to 64 database conversion problem
Seems like I need to pass some arguments, rebuild (X Var Cls Hook) Can you give me an example, what could X and Var be, Cls is probably something like '+Class but what is Hook? On Mon, Oct 3, 2011 at 12:42 PM, Alexander Burger wrote: > Hi Henrik, > > > I haven't done anything yet, all 64bit testing has been done locally to > see > > first what happens. > > > > So I try and rebuild live then and we see what happens, I load with pool > and > > then just call rebuild, is that it? > > As I understood it, you started out with an empty DB on 64-bits, right? > So for that one no rebuilding is required. > > If you want to port the old DB from 32-bits to 64-bits, you should first > try to rebuild the index which gave an error, then run 'dbCheck' again, > and only run the routines in "lib/db32-64.l" after everything looks > good. > > Yes, 'rebuild' should be called in a single-user situation, after > calling 'pool' (or simply 'main' without 'go'). 'rebuild' does call > 'commit' by itself, so changes will be immediately manifest. > > Cheers, > - Alex > -- > UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe >
Re: 32 to 64 database conversion problem
Hi Henrik, > I haven't done anything yet, all 64bit testing has been done locally to see > first what happens. > > So I try and rebuild live then and we see what happens, I load with pool and > then just call rebuild, is that it? As I understood it, you started out with an empty DB on 64-bits, right? So for that one no rebuilding is required. If you want to port the old DB from 32-bits to 64-bits, you should first try to rebuild the index which gave an error, then run 'dbCheck' again, and only run the routines in "lib/db32-64.l" after everything looks good. Yes, 'rebuild' should be called in a single-user situation, after calling 'pool' (or simply 'main' without 'go'). 'rebuild' does call 'commit' by itself, so changes will be immediately manifest. Cheers, - Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: 32 to 64 database conversion problem
"If you can live with the newly set-up DB, it would be the easiest."
I haven't done anything yet, all 64bit testing has been done locally to see
first what happens.
So I try and rebuild live then and we see what happens, I load with pool and
then just call rebuild, is that it?
On Sun, Oct 2, 2011 at 10:44 PM, Alexander Burger wrote:
> On Sun, Oct 02, 2011 at 08:33:26PM +0700, Henrik Sarvell wrote:
> > Unfortunately I managed to accidentally delete the local copy 32bit I had
> of
> > the DB after the conversion was finished :(
> >
> > However, when I run it on the live database which the copy is an older
> > snapshot of I get this:
> >
> > {2L} (302 . {P-D}) tag +ArticleTag {1}
> > {2L} (302 . {P-;}) article +ArticleTag {1}
> > {2L} (302 . {P-H}) user +ArticleTag {1}
> > {5} (306 . {N-4}) tag +Tag {1}
> > [vr_db_check.l:3] {N-4K} -- Bad sequence
> > ? (show '{N-4K})
> > {N-4K} (NIL ("haskell" NIL . {8-2y}) ("iPhone" {N-23} . {8-u}) ("ibm" NIL
> .
> > {8-2F}) ("im" NIL . {8-2B}))
> > -> {N-4K}
> > ?
> >
> > Could this be the culprit?
>
> Indeed. This indicates that an DB index is out of order, for whatever
> reason. Don't know how this could happen, perhaps by some non-synced
> access. I haven't seen this error in the wild before, but it is issued
> by 'chkTree'. It could be repaired by 'rebuild'.
>
> If you can live with the newly set-up DB, it would be the easiest.
>
> Cheers,
> - Alex
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
>
Re: 32 to 64 database conversion problem
On Sun, Oct 02, 2011 at 08:33:26PM +0700, Henrik Sarvell wrote:
> Unfortunately I managed to accidentally delete the local copy 32bit I had of
> the DB after the conversion was finished :(
>
> However, when I run it on the live database which the copy is an older
> snapshot of I get this:
>
> {2L} (302 . {P-D}) tag +ArticleTag {1}
> {2L} (302 . {P-;}) article +ArticleTag {1}
> {2L} (302 . {P-H}) user +ArticleTag {1}
> {5} (306 . {N-4}) tag +Tag {1}
> [vr_db_check.l:3] {N-4K} -- Bad sequence
> ? (show '{N-4K})
> {N-4K} (NIL ("haskell" NIL . {8-2y}) ("iPhone" {N-23} . {8-u}) ("ibm" NIL .
> {8-2F}) ("im" NIL . {8-2B}))
> -> {N-4K}
> ?
>
> Could this be the culprit?
Indeed. This indicates that an DB index is out of order, for whatever
reason. Don't know how this could happen, perhaps by some non-synced
access. I haven't seen this error in the wild before, but it is issued
by 'chkTree'. It could be repaired by 'rebuild'.
If you can live with the newly set-up DB, it would be the easiest.
Cheers,
- Alex
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
Re: 32 to 64 database conversion problem
Unfortunately I managed to accidentally delete the local copy 32bit I had of
the DB after the conversion was finished :(
However, when I run it on the live database which the copy is an older
snapshot of I get this:
{2L} (302 . {P-D}) tag +ArticleTag {1}
{2L} (302 . {P-;}) article +ArticleTag {1}
{2L} (302 . {P-H}) user +ArticleTag {1}
{5} (306 . {N-4}) tag +Tag {1}
[vr_db_check.l:3] {N-4K} -- Bad sequence
? (show '{N-4K})
{N-4K} (NIL ("haskell" NIL . {8-2y}) ("iPhone" {N-23} . {8-u}) ("ibm" NIL .
{8-2F}) ("im" NIL . {8-2B}))
-> {N-4K}
?
Could this be the culprit?
On Sun, Oct 2, 2011 at 3:21 PM, Alexander Burger wrote:
> Hi Henrik,
>
> > I just tried to run vizreader (from scratch) with the 64bit version and I
> > did not experience any problems (as opposed to with early versions of the
> > 64bit version).
>
> Great. That's good news :)
>
>
> > So I thought I should try and run with a complete converted 32 -> 64bit
> > database.
> > ...
> > [/opt/resources/rss-reader/convert.l:8] DB read: Success
> > ?
> >
> > Good or not?
>
> This doesn't look good :(
>
> It is an I/O (read) error, despite the misleading error message. It
> seems it tries to access a non-existing object, or out-of-range database
> file.
>
>
> The reason is hard to tell without a direct inspection of the DB. I
> don't think it has to do with concrete E/R issues, because the
> conversion 32->64 runs on a lower level, directly following the links
> between objects, and observing only a few E/R rules heuristically.
>
>
> Could it be that already the 32-bit DB was not consistent? I remember we
> talked about several problems in the beginning. What happens if you run
>
> : (load "lib/too.l")
>
> : (dbCheck)
>
> on the 32-bit system? It should print diagnostics of the form
>
> 1 4 (5282 . 5282)
> 2 1 (18591 . 18591)
> 3 1 (272 . 269)
> 4 1 (28254 . 28254)
> 5 1 (14243 . 8815)
> ...
>
> iterating all DB files in the first column, followed by lines like
>
> {obj1} (1 . {obj2}) var +Cls {1}
> ...
>
> for each index. If something looks wrong, a hard error ("!? ..") is
> issued, or a warning like (dbfCheck Class) for suspicious DB-models, or
> (dangling {obj} ..) for missing indexes is printed.
>
> Cheers,
> - Alex
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
>
Re: 32 to 64 database conversion problem
Hi Henrik,
> I just tried to run vizreader (from scratch) with the 64bit version and I
> did not experience any problems (as opposed to with early versions of the
> 64bit version).
Great. That's good news :)
> So I thought I should try and run with a complete converted 32 -> 64bit
> database.
> ...
> [/opt/resources/rss-reader/convert.l:8] DB read: Success
> ?
>
> Good or not?
This doesn't look good :(
It is an I/O (read) error, despite the misleading error message. It
seems it tries to access a non-existing object, or out-of-range database
file.
The reason is hard to tell without a direct inspection of the DB. I
don't think it has to do with concrete E/R issues, because the
conversion 32->64 runs on a lower level, directly following the links
between objects, and observing only a few E/R rules heuristically.
Could it be that already the 32-bit DB was not consistent? I remember we
talked about several problems in the beginning. What happens if you run
: (load "lib/too.l")
: (dbCheck)
on the 32-bit system? It should print diagnostics of the form
1 4 (5282 . 5282)
2 1 (18591 . 18591)
3 1 (272 . 269)
4 1 (28254 . 28254)
5 1 (14243 . 8815)
...
iterating all DB files in the first column, followed by lines like
{obj1} (1 . {obj2}) var +Cls {1}
...
for each index. If something looks wrong, a hard error ("!? ..") is
issued, or a warning like (dbfCheck Class) for suspicious DB-models, or
(dangling {obj} ..) for missing indexes is printed.
Cheers,
- Alex
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
32 to 64 database conversion problem
Hi Alex, everyone.
I just tried to run vizreader (from scratch) with the 64bit version and I
did not experience any problems (as opposed to with early versions of the
64bit version).
So I thought I should try and run with a complete converted 32 -> 64bit
database.
The first step in db32-64.l went just fine.
So I then copy and load the database and run (import32) as per the
instructions, here is my actual code:
(setq *BP "/opt/picolisp/projects/rss-reader/")
(setq *Rdir "/opt/resources/")
(load (pack *BP "er.l"))
(pool (pack *Rdir "rss-reader/db/") *Dbs)
(load "lib/too.l" "lib/db32-64.l" "lib/debug.l")
(import32)
It shows me this:
[/opt/resources/rss-reader/convert.l:8] DB read: Success
?
Good or not?
Because there are problems with the following E/R:
(class +FeedTag +Entity)
(rel user (+Ref +Link) NIL (+User))
(rel tag(+Aux +Ref +Link) (user) NIL (+Tag))
(rel feed (+Aux +Ref +Link) (user) NIL (+Feed))
I'm testing with the following lines:
(setq Usr (db 'uname '+User "henrik"))
(setq Tag (db 'tag '+Tag "Sweden"))
(mapc show (list Usr Tag))
(mapc show (collect 'tag '+FeedTag (list Tag Usr)))
I see that both Usr and Tag are not nil (ie works) but the collect statement
doesn't return anything.
For some reason the problem is the links:
(mapc show (collect 'user '+FeedTag Usr Usr))
and
(mapc show (collect 'tag '+FeedTag Tag Tag))
return nothing.
However,
(mapc show (collect 'tag '+FeedTag NIL T))
returns all the objects just fine.
The last lines of the output:
{H3274} (+FeedTag)
user {62}
tag {G77}
feed {A3264}
{H3275} (+FeedTag)
user {62}
tag {G77}
feed {A3265}
Doing (show '{G77}) gives me:
{G77} (+Tag)
tag "News"
Trying a collect with G77 doesn't return anything either:
(setq Tag (db 'tag '+Tag "News"))
(mapc show (collect 'tag '+FeedTag Tag Tag))
Any thoughts?
