Re: Pilog faster than DB access?

[cle]

Hello Henrik,

Henrik Sarvell wrote:

 I don't know exactly the circumstances of what you're trying to
 achieve,

ok, I will try to answer this in the next mail to Alex ...

 but if I were you and needed to access by unique id I would
 first use +Key in the relations:

 (rel id (+Key +String))

Ah, that makes sense! But it will only work for my id relation ... but I
will also access them by 'src' and 'tgt'.

 Then I would use (db) to access (like you do but without the Pilog):

 (db 'id '+Test 1)

Ok, I will try this too for sure ... thanks! :-)

 I think that's the fastest way, see also (collect) and (aux) for
 more utility functions that will let you access the DB without using
 Pilog.

But in the end, I *want* to use Pilog, I explain in the later mail. :-)

(...)

Thank you for your non-ending effort to answer my noob questions :-)


Ciao,
cle.

-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[Henrik Sarvell]

Sure, Pilog works fine if you have something more complex, but you get
quite far with (db), (aux) and (collect). As far as src and tgt goes
if they are not unique you can do for instance:

(collect 'src '+Test D D)

Then you get all objects with D as src, I think you need to come up
with something more complex in order to actually get to a reasonable
real-world Pilog example.

My tutorial series has a few fairly simple Pilog examples here:
http://www.prodevtips.com/2008/04/28/advanced-oodb-in-pico-lisp/

/Henrik

On Fri, Dec 11, 2009 at 9:54 AM, cle cle-picol...@qiao.in-berlin.de wrote:

 Hello Henrik,

 Henrik Sarvell wrote:

 I don't know exactly the circumstances of what you're trying to
 achieve,

 ok, I will try to answer this in the next mail to Alex ...

 but if I were you and needed to access by unique id I would
 first use +Key in the relations:

 (rel id (+Key +String))

 Ah, that makes sense! But it will only work for my id relation ... but I
 will also access them by 'src' and 'tgt'.

 Then I would use (db) to access (like you do but without the Pilog):

 (db 'id '+Test 1)

 Ok, I will try this too for sure ... thanks! :-)

 I think that's the fastest way, see also (collect) and (aux) for
 more utility functions that will let you access the DB without using
 Pilog.

 But in the end, I *want* to use Pilog, I explain in the later mail. :-)

 (...)

 Thank you for your non-ending effort to answer my noob questions :-)


 Ciao,
 cle.

 --
 UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[cle]

Alexander Burger wrote:

 Hi Cle,

Hi Alex,

 let me first correct some errors:

 (pool test)

 This is a funny one. As the value of 'test' is a function
 definition, you get a rather strange database file ;-)

Ahhh ... this was a copy/paste error from the console into the mail. My
DB file was named test :-)

 The name test, on the other hand, will give a conflict as there is
 a directory with that name. I take x.db for now:

Ahhh not for me ... as I am working within another directory parallel to
picoLisp ;-)

 (pool x.db)

Ok, I agree :-)

 (class +Test +Entity) (rel id (+Ref +String)) (rel src (+Ref
 +String)) (rel tgt (+Ref +String))

 (new '(+Test) 'id 1  'src A  'tgt B) (new '(+Test) 'id 11  'src
 B  'tgt C) (new '(+Test) 'id 111  'src C  'tgt D)

 If 'new' is called this way, it will not create database objects but
 memory-resident symbols. You should call either

:-O But ... but ... but ... URGHS! :-( I had looked especially into the
x.db file and found my keys 1, 11 and 111 there. So I thought
.. now I feel stupid again :-(

No! I am stupid ... in my real program I used 'new!' and *not* 'new'. So
in my real slower program the DB was filled but only by accident :-}

(...)

 This would actually return no result at all, as you pass a _number_
 '1' while the key is a _string_. I suppose the double quotes got lost
 when copy/pasting the expression from the console?

Yes you are right! Those missing  were not the problem. Only the
missing 'T' was a real problem in this made-up example so far! Thanks
for pointing this out!

(...)

 Yes, the Pilog queries 'select' and its small brother 'db' are
 intended to return result _sets_, useful in GUI search components.
 For simple direct DB accesses Pilog is rather overkill. You typically
 use the Lisp function 'db' for that:

 : (db 'id '+Test 1) - {2}

 The Pilog 'db' is actually a misnomer. It behaves more like the Lisp
 'collect' function.

Ok, perhaps I could use the Lisp 'db' function when I have a direct
access :-) But anyway I will also need the Pilog 'db' clause. But I
would need it with a functionality like SQL

   select ID, SRC, TGT
 from +Test
   where ID = 1'

an *not*

   select ID, SRC, TGT
  from +Test
where ID like 1%

is there any way to achieve my goal?

(...)

 True. Pilog is a powerful search mechanism, but you have to
 understand that for a simple indexed access the overhead of setting
 up the query and communicating environments from Lisp to Pilog and
 back is much larger than the actual db access.

 So it is not recommended, and was never intended, to use Pilog to
 retrieve a single value from the DB.

Ok! So perhaps I cannot use the DB for my purposes :-(

(...)

 Perhaps you should try with the corrected version (see above). Also,
 when using DB _plus_ Pilog, you get the overhead from both worlds.
 You could also try a DB version without Pilog. Is this possible?

I do not know for sure. I originally want to use Java + Prolog for my
application. Then I detected your picoLisp ...

(...)

 There are many ways to tune the performance. Perhaps you can
 elaborate a little more on the details?

Ok, here comes my story! Currently there is an application/tool written
in  C++, which is very hard to maintain. The task is, to read-in some
XML formatted description of a rail infrastructure. To let our main
application deal with the original rail infrastructure (points, tracks,
signals ...) a team has to take the original planning sketches and
describe them into XML. There are a lot of rules to obey to describe the
infrastructure properly. To help the team with that task, a check tool
was written, that read in said XML description and check them according
the rules of our projection handbook/guide.

My idea now is that I can write a new check tool that will have the
rules formulated in Pilog (formerly Prolog). All in all I would think,
that the resulting tool will be much more friendlier to maintain as the
said C++ one.

For that I need to formulate the edges of the tracknet somehow (for
instance) as Pilog facts like this:

   #  (Edge id fromNode toNode)
   (be edge (ts1 end#1 points#1))
   (be edge (ts2 points#1 points#2))
   (be edge (ts3 points#1 points#2))
   (be edge (ts4 points#2 end#2))

Would describe a graph like this (better look with a fixed size font):

  ts2
  ___
   ts1   /   \   ts4
End#1  |---o Points#1   Points#2 o---| End#2
 \___/

  ts3

Then I have a Pilog clause like this:

   (be walk (@Begin @Way)
 (edge @Begin @From @To)
 (walk @To (@Begin) @Way)
   )

   (be walk (@Node @Way @Way))

   (be walk (@FromNode @AuxWay @Way)
 (edge @Edge @FromNode @ToNode)
 (walk @ToNode (@Edge . @AuxWay) @Way)
   )

Which would allow me to iterate from a certain edge thru all reachable
edges until no other way can be found anymore. This clause would form a
base of my future 

Re: Pilog faster than DB access?

[Alexander Burger]

Hi Cle,

  but if I were you and needed to access by unique id I would
  first use +Key in the relations:
 
  (rel id (+Key +String))
 
 Ah, that makes sense! But it will only work for my id relation ... but I
 will also access them by 'src' and 'tgt'.

Please note that '+Key' will not change the 'collect'-like behavior of
the Pilog 'db' predicate in that regard. It only creates a unique index
(while '+Ref' maintains a non-unique index).

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[Alexander Burger]

Hi Henrik,

 Sure, Pilog works fine if you have something more complex, but you get
 quite far with (db), (aux) and (collect). As far as src and tgt goes
 if they are not unique you can do for instance:
 
 (collect 'src '+Test D D)

 Then you get all objects with D as src, I think you need to come up

Correct, though in this case also just

   (collect 'src '+Test D)

would do.

'collect' with a single key argument will return a list of all hits
having that key. The list will have of course only zero or one elements
if the key is unique.

Otherwise

   (collect 'src '+Test from to)

will return a list of all hits between (and including) these keys.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[cle]

Henrik Sarvell wrote:

 Sure, Pilog works fine if you have something more complex, but you
 get quite far with (db), (aux) and (collect).

(...)

 Then you get all objects with D as src, I think you need to come up
 with something more complex in order to actually get to a reasonable
 real-world Pilog example.

My Pilog program will be complex. As I have written in my last mail to
Alex, I will have to traverse directed graphs egde by edge and ensure
rules are fulfilled. An easier rule could be formulated like this:

   Ensure, that for every path thru a train station beginning from every
entry signal up to the exit signal, there is no path longer than 500 m
without any intervening signal or auxiliary signal :-)

This could be nicely solved with Pilog :-)))

 My tutorial series has a few fairly simple Pilog examples here:
 http://www.prodevtips.com/2008/04/28/advanced-oodb-in-pico-lisp/

I had already a fly-over over your site, it is really nice done! I will
look at it again, though. But I think, I will have no problem with
Pilog, but more with the database thingy :-)

(...)


Ciao,
cle.

-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[cle]

Alexander Burger wrote:

 Hi Cle,

Hi Alex,

(...)

 If 'new' is called this way, it will not create database objects
 but

 ... :-O But ... but ... but ... URGHS! :-( I had looked especially
 into the x.db file and found my keys 1, 11 and 111 there.
 So I thought

 Right. I never tried it this way, but the reason is that though
 there were memory-only objects created, the relation-handling
 side-effects of '+Ref' etc. were executed, so that the indexes were
 in the file but not the objects.

ahhh ... now I understand, what I've seen :-)

(...)

 The way I would use is the already-discussed 'collect'

(...)

 If you really prefer Pilog also for direct accesses, then you can
 still call it the direct way:

Nice idea! Now I reformulated my Pilog rules for usage with database
like this:

   (be graph:walk (@Begin @Way)
 (@E db 'id '+DB:Edge (- @Begin))
 (@To get (- @E) 'tgt)
 (graph:walk @To (@Begin) @Way)
   )

   (be graph:walk (@Node @Way @Way))

   (be graph:walk (@FromNode @AuxWay @Way)
 (@Edges collect 'src '+DB:Edge (- @FromNode))
 (lst @Edge @Edges)
 (@Id get (- @Edge) 'id)
 (@ToNode get (- @Edge) 'tgt)
 (graph:walk @ToNode (@Id . @AuxWay) @Way)
   )

This is an excerp from my actual code. You will find, that I use a
module: prefix in my clauses to modularize the Pilog clauses and
facts. A slight misuse I know, as those prefix should be used for
function in shared libraries ... :-}

With those changes the traversion becomes a bit faster; it needed 87
seconds for 1000 pathes, where Pilog facts would take 3 seconds ;-)

Any other hint? :-

 I'll study your project now ... Looks interesting :-)

Thank you for your interest :-)


 Cheers, - Alex

Ciao,
cle.

-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[Alexander Burger]

Hi Cle,

 Ok, here comes my story! Currently there is an application/tool written
 ...
#  (Edge id fromNode toNode)
(be edge (ts1 end#1 points#1))
(be edge (ts2 points#1 points#2))
(be edge (ts3 points#1 points#2))
(be edge (ts4 points#2 end#2))
 ...
(be walk (@Begin @Way)
  (edge @Begin @From @To)
  (walk @To (@Begin) @Way)
)
 
(be walk (@Node @Way @Way))
 
(be walk (@FromNode @AuxWay @Way)
  (edge @Edge @FromNode @ToNode)
  (walk @ToNode (@Edge . @AuxWay) @Way)
)


OK, so my modification of your original DB program would be

(pool x.db)

(class +Test +Entity)
(rel id (+Ref +String))
(rel src (+Ref +String))
(rel tgt (+Ref +String))

(new T '(+Test) 'id ts1  'src end#1  'tgt points#1)
(new T '(+Test) 'id ts2  'src points#1  'tgt points#2)
(new T '(+Test) 'id ts3  'src points#1  'tgt points#2)
(new T '(+Test) 'id ts4  'src points#2  'tgt end#2)

(commit)

(be walk (@Begin @Way)
   (db id +Test @Begin @Obj)
   (val @From @Obj  src)
   (val @To @Obj tgt)
   (walk @To (@Begin) @Way) )

(be walk (@Node @Way @Way))

(be walk (@FromNode @AuxWay @Way)
   (db src +Test @FromNode @Obj)
   (val @Edge @Obj id)
   (val @ToNode @Obj tgt)
   (walk @ToNode (@Edge . @AuxWay) @Way) )


Now, I must confess that I use the 'db' Pilog predicate here, instead of
the suggested Lisp 'db' function. In the latter case, the line

   (db id +Test @Begin @Obj)

would need to be replaced with

   (@Lst collect 'id '+Test (- @Begin))
   (lst @Obj @Lst)

and

   (db src +Test @FromNode @Obj)

would be

   (@Lst collect 'src '+Test (- @FromNode))
   (lst @Obj @Lst)

No big thing, but not so beautiful ;-)


But I remembered a fact which would allow us to stay with the beautiful
solution: The 'db' predicate matches val% only for string types (for
some obscure practical and historical reasons). If 'id' were a numeric
type, then only exact matches are returned. Would this be an option,
i.e.

   (rel id (+Ref +Number))

or probably even

   (rel id (+Key +Number))

If so, then we could safely stay with the 'db' predicate.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[Alexander Burger]

Hi Cle,

 Nice idea! Now I reformulated my Pilog rules for usage with database
 like this:
 
(be graph:walk (@Begin @Way)
  (@E db 'id '+DB:Edge (- @Begin))
  (@To get (- @E) 'tgt)
  (graph:walk @To (@Begin) @Way)
)
 
(be graph:walk (@Node @Way @Way))
 
(be graph:walk (@FromNode @AuxWay @Way)
  (@Edges collect 'src '+DB:Edge (- @FromNode))
  (lst @Edge @Edges)
  (@Id get (- @Edge) 'id)
  (@ToNode get (- @Edge) 'tgt)
  (graph:walk @ToNode (@Id . @AuxWay) @Way)
)

Hehe, nice. Our mails crossed-over, without paying attention to a point
or signal ;-) The solutions look pretty similar. As you see, the
explicit 'get' can be substituted by a more concise 'val' predicate.

I also notice that the '@From' in my walk/2 is not used and thus
unnecessary. So I would propose (especially if you use a numeric 'id'):

   (be walk (@Begin @Way)
  (db id +Test @Begin @Obj)
  (val @To @Obj tgt)
  (walk @To (@Begin) @Way) )

   (be walk (@Node @Way @Way))

   (be walk (@FromNode @AuxWay @Way)
  (db src +Test @FromNode @Obj)
  (val @Edge @Obj id)
  (val @ToNode @Obj tgt)
  (walk @ToNode (@Edge . @AuxWay) @Way) )


 This is an excerp from my actual code. You will find, that I use a
 module: prefix in my clauses to modularize the Pilog clauses and
 facts. A slight misuse I know, as those prefix should be used for
 function in shared libraries ... :-}

I would not worry about that. It is just the other way round: If the
interpreter finds an _undefined_function_ with a colon in the name, then
it searches for a library. This doesn't inhibit us, though, to define
such functions ourselves. And here we are not dealing with functions
anyway.


 With those changes the traversion becomes a bit faster; it needed 87
 seconds for 1000 pathes, where Pilog facts would take 3 seconds ;-)

This large difference is rather strange. I would expect _some_ slowdown,
as the DB involves I/O operations while Pilog only runs in memory, but
not so much.

The question is what happens if you have millions of edges. Then the
linear lookup in Pilog versus the index accesses in the DB might show
effect.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[cle]

Alexander Burger wrote:
 Hi Cle,

   
 Nice idea! Now I reformulated my Pilog rules for usage with database
 like this:

(be graph:walk (@Begin @Way)
  (@E db 'id '+DB:Edge (- @Begin))
  (@To get (- @E) 'tgt)
  (graph:walk @To (@Begin) @Way)
)

(be graph:walk (@Node @Way @Way))

(be graph:walk (@FromNode @AuxWay @Way)
  (@Edges collect 'src '+DB:Edge (- @FromNode))
  (lst @Edge @Edges)
  (@Id get (- @Edge) 'id)
  (@ToNode get (- @Edge) 'tgt)
  (graph:walk @ToNode (@Id . @AuxWay) @Way)
)
 

 Hehe, nice. Our mails crossed-over, without paying attention to a point
 or signal ;-)

Yeah, but also without any accident ;-)))

 The solutions look pretty similar.

That should be so! As they say in China: 英雄所见略同 (Heros have the
same mind/opinion) ;-)

 As you see, the explicit 'get' can be substituted by a more concise
 'val' predicate.

To rephrase the chinese sentence once more: your solution was exactly
the same as I came up with my first attempt. But due to the fact, I use
'+String' for all fields, the Pilog 'db' clause did find the wrong
solutions. To handle that, I did introduce a 'equal' check then. But
now, if I can use '+Number' for the id's the 'equal' check could vanish,
I assume. :-)

But I would guess -- if I understood your and Henriks explanation
correctly -- that my solution should be faster, as I use the Lisp access
functions, shouldn't it?

 I also notice that the '@From' in my walk/2 is not used and thus
 unnecessary. So I would propose (especially if you use a numeric 'id'):
   

I would think, *only* if I use numeric 'id', 'src' and 'tgt'. Otherwise
the 'db' clause of Pilog would find a connection 'pt11' in e.g. the
'src' field although it was requested only to search for a connection
from 'src' = 'pt1' ... :-(

But I will have a look, if I can use numeric ids for all properties of
an edge. I would guess that unification in my Pilog solution should also
be faster, if those arguments were numeric instead of strings. :-) And
perhaps the DB behaves better then.

But if I do this, I will need another facts, to mark, which node ('src'
or 'tgt') is a start or end point of a tracknet. At the moment those are
named like End#12 or such ...

(...)

 I would not worry about that. It is just the other way round: If the
 interpreter finds an _undefined_function_ with a colon in the name, then
 it searches for a library. This doesn't inhibit us, though, to define
 such functions ourselves. And here we are not dealing with functions
 anyway.
   

A good to know! So I can go forth with that procedure :-)

 With those changes the traversion becomes a bit faster; it needed 87
 seconds for 1000 pathes, where Pilog facts would take 3 seconds ;-)
 

 This large difference is rather strange. I would expect _some_ slowdown,
 as the DB involves I/O operations while Pilog only runs in memory, but
 not so much.
   

That was, what made me crazy too ... I had thought, that direct access
via an index should be faster than sequential search of 360 edges again
and again and again ...

So I am really eager to understand, why this is not the case, even with
my Lisp database access in Pilog? Otherwise I feel I did something wrong
perhaps ...

 The question is what happens if you have millions of edges. Then the
 linear lookup in Pilog versus the index accesses in the DB might show
 effect.
   

If it really need millions of edges to show effect, I can safely reside
on my Pilog facts, as my tracknet will never be so large. The 360 edges
are a not so small tracknet already :-D


Ciao,
cle.

-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[Alexander Burger]

Hi Cle,

 To rephrase the chinese sentence once more: your solution was exactly
 the same as I came up with my first attempt. But due to the fact, I use
 '+String' for all fields, the Pilog 'db' clause did find the wrong
 solutions. To handle that, I did introduce a 'equal' check then. But

Right. BTW, the 'equal' check, from a procedural point of view, is not
so wrong. Because if instead 'db' would search for the exact solution
itself, it would have to do the same processing, i.e. try subsequent
tree entries, and throw away the first hits which are not equal. This
is due to the nature of tree searches that the smallest hits are found
first.


 now, if I can use '+Number' for the id's the 'equal' check could vanish,
 I assume. :-)

Unfortunately, as occurs to me now, this would only help for the access
to 'id'. Later we have also 'src' and there we probably will need
strings anyway.


 But I would guess -- if I understood your and Henriks explanation
 correctly -- that my solution should be faster, as I use the Lisp access
 functions, shouldn't it?

Yes, it avoids some overhead, but I think this is minimal. The overhead
is some Pilog layer, but as we saw the Pilog interpreter is not the
bottleneck.


 I would think, *only* if I use numeric 'id', 'src' and 'tgt'. Otherwise
 the 'db' clause of Pilog would find a connection 'pt11' in e.g. the
 'src' field although it was requested only to search for a connection
 from 'src' = 'pt1' ... :-(

Yep.


 But I will have a look, if I can use numeric ids for all properties of
 an edge. I would guess that unification in my Pilog solution should also
 be faster, if those arguments were numeric instead of strings. :-) And
 perhaps the DB behaves better then.

The more I think about it, the more I believe that a plain in-memory
solution is better. The I/O overhead and index tree traversals might
still be heavy in comparison the the rather sequential traversal of
Pilog rules in memory.

If you really had millions of edges, I would consider replacing the two
'db' instances with something handmade out of an in-memory index with
the 'idx' function.


 That was, what made me crazy too ... I had thought, that direct access
 via an index should be faster than sequential search of 360 edges again
 and again and again ...

Yes, but

- An index is also not really a direct access, but involves searching
  btree nodes. In-memory, linear lookup for a few dozens or hundreds of
  items is usually faster.

- I'm not even sure if Pilog does much searching here, of if it just
  iterates the lists of clauses?


 If it really need millions of edges to show effect, I can safely reside
 on my Pilog facts, as my tracknet will never be so large. The 360 edges
 are a not so small tracknet already :-D

Yeah, so let's better stay with plain Pilog. Then the database is just
the set of rules read into memory once, and then processing can start
without further I/O.

A database really makes only sense when

- there are more data than fit into memory

- only a small subset needs to be loaded into memory for a certain
  operation

- the data are loaded/modified/written in arbitrary ways, and need to be
  persistent

- multiple users operate on the data and need to be synchronized

Did I forget something?

So a constant, small set of rules can simply be read at program start.

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[Henrik Sarvell]

I don't know exactly the circumstances of what you're trying to
achieve, but if I were you and needed to access by unique id I would
first use +Key in the relations:

(rel id (+Key +String))

Then I would use (db) to access (like you do but without the Pilog):

(db 'id '+Test 1)

I think that's the fastest way, see also (collect) and (aux) for more
utility functions that will let you access the DB without using Pilog.

/Henrik


On Fri, Dec 11, 2009 at 5:25 AM, cle cle-picol...@qiao.in-berlin.de wrote=
:
 Hello,

 today I tried to verify if using a database instead of Pilog facts would
 speed up my processing. While trying this, I stumbled over the
 following. Suppose:

 =A0(pool test)

 =A0(class +Test +Entity)
 =A0(rel id (+Ref +String))
 =A0(rel src (+Ref +String))
 =A0(rel tgt (+Ref +String))

 =A0(new '(+Test) 'id 1 =A0'src A =A0'tgt B)
 =A0(new '(+Test) 'id 11 =A0'src B =A0'tgt C)
 =A0(new '(+Test) 'id 111 =A0'src C =A0'tgt D)

 =A0(commit)

 =A0(? (db id +Test 1 @E) (show @E))

 The last Pilog query will successively return three answers where I
 thought to get only one. I understand, why 'db' will deliver three
 results. But the question comes up, how I could formulate the query
 using only exact matches!

 Without this, the 'db' query would return similar matches that I have to
 explicitely guarding against with e.g. 'equal' like this:

 =A0(? (db id +Test 1 @E) (val @Id @E id) (equal @Id 1) (show @E))

 But this would defeat the reason, why I tried to use a DB at all. I
 hoped that accessing an Edge via an index would be faster than asking
 Pilog to match against all its facts until it found a corresponding it
 could unify against.

 So could this be the reason that my DB solution of traversing all ways
 thru a graph is a lot slower than using Pilog facts? Both are trying to
 iterate over the same graph built by 362 edges. Whereas the Pilog facts
 will allow for finding ca. 1000 ways per 1-2 seconds, the database
 solution was canceled after some time, as it did not reach the first
 1000 solutions before I lost my patience ;-)

 Or is there any way to improve the performance of the DB? For instance
 by only matching exactly in the db query for a certain edge?


 Thanks and Ciao,
 cle.

 --
 UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=3dunsubscribe

-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe

Re: Pilog faster than DB access?

[Alexander Burger]

Hi Cle,

let me first correct some errors:

   (pool test)

This is a funny one. As the value of 'test' is a function definition,
you get a rather strange database file ;-)

The name test, on the other hand, will give a conflict as there
is a directory with that name. I take x.db for now:

   (pool x.db)

   (class +Test +Entity)
   (rel id (+Ref +String))
   (rel src (+Ref +String))
   (rel tgt (+Ref +String))
 
   (new '(+Test) 'id 1  'src A  'tgt B) 
   (new '(+Test) 'id 11  'src B  'tgt C)
   (new '(+Test) 'id 111  'src C  'tgt D)

If 'new' is called this way, it will not create database objects but
memory-resident symbols. You should call either

   (new T '(+Test) 'id 1  'src A  'tgt B) 

if there is a single db file, or

   (new (db: +Test) '(+Test) 'id 1  'src A  'tgt B) 

in the general case.

The values passed for 'src' and 'tgt' are random, depending on the
bindings of 'A' and 'B'. Probably NIL.


   (commit)
 
   (? (db id +Test 1 @E) (show @E))

This would actually return no result at all, as you pass a _number_ '1'
while the key is a _string_. I suppose the double quotes got lost when
copy/pasting the expression from the console?


However, you are right in your observation:

 The last Pilog query will successively return three answers where I
 thought to get only one. I understand, why 'db' will deliver three
 results. But the question comes up, how I could formulate the query
 using only exact matches!

Yes, the Pilog queries 'select' and its small brother 'db' are intended
to return result _sets_, useful in GUI search components. For simple
direct DB accesses Pilog is rather overkill. You typically use the Lisp
function 'db' for that:

   : (db 'id '+Test 1) 
   - {2}

The Pilog 'db' is actually a misnomer. It behaves more like the Lisp
'collect' function.


   (? (db id +Test 1 @E) (val @Id @E id) (equal @Id 1) (show @E))
 
 But this would defeat the reason, why I tried to use a DB at all. I
 hoped that accessing an Edge via an index would be faster than asking
 Pilog to match against all its facts until it found a corresponding it
 could unify against.

True. Pilog is a powerful search mechanism, but you have to understand
that for a simple indexed access the overhead of setting up the query
and communicating environments from Lisp to Pilog and back is much
larger than the actual db access.

So it is not recommended, and was never intended, to use Pilog to
retrieve a single value from the DB.


 So could this be the reason that my DB solution of traversing all ways
 thru a graph is a lot slower than using Pilog facts? Both are trying to

For such a small database this may well be possible. In general, if all
data fit into memory, a non-DB version might well be faster. But I
wouln't expect a dramatic difference.

 iterate over the same graph built by 362 edges. Whereas the Pilog facts
 will allow for finding ca. 1000 ways per 1-2 seconds, the database
 solution was canceled after some time, as it did not reach the first
 1000 solutions before I lost my patience ;-)

Perhaps you should try with the corrected version (see above). Also,
when using DB _plus_ Pilog, you get the overhead from both worlds. You
could also try a DB version without Pilog. Is this possible?


 Or is there any way to improve the performance of the DB? For instance
 by only matching exactly in the db query for a certain edge?

There are many ways to tune the performance. Perhaps you can elaborate a
little more on the details?

Cheers,
- Alex
-- 
UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe