Re: Pilog faster than DB access?
Hi Cle, > To rephrase the chinese sentence once more: your solution was exactly > the same as I came up with my first attempt. But due to the fact, I use > '+String' for all fields, the Pilog 'db' clause did find the wrong > solutions. To handle that, I did introduce a 'equal' check then. But Right. BTW, the 'equal' check, from a procedural point of view, is not so wrong. Because if instead 'db' would search for the exact solution itself, it would have to do the same processing, i.e. try subsequent tree entries, and throw away the first hits which are not "equal". This is due to the nature of tree searches that the smallest hits are found first. > now, if I can use '+Number' for the id's the 'equal' check could vanish, > I assume. :-) Unfortunately, as occurs to me now, this would only help for the access to 'id'. Later we have also 'src' and there we probably will need strings anyway. > But I would guess -- if I understood your and Henriks explanation > correctly -- that my solution should be faster, as I use the Lisp access > functions, shouldn't it? Yes, it avoids some overhead, but I think this is minimal. The overhead is some Pilog layer, but as we saw the Pilog interpreter is not the bottleneck. > I would think, *only* if I use numeric 'id', 'src' and 'tgt'. Otherwise > the 'db' clause of Pilog would find a connection 'pt11' in e.g. the > 'src' field although it was requested only to search for a connection > from 'src' = 'pt1' ... :-( Yep. > But I will have a look, if I can use numeric ids for all properties of > an edge. I would guess that unification in my Pilog solution should also > be faster, if those arguments were numeric instead of strings. :-) And > perhaps the DB behaves better then. The more I think about it, the more I believe that a plain in-memory solution is better. The I/O overhead and index tree traversals might still be heavy in comparison the the rather sequential traversal of Pilog rules in memory. If you really had millions of edges, I would consider replacing the two 'db' instances with something handmade out of an in-memory index with the 'idx' function. > That was, what made me crazy too ... I had thought, that direct access > via an index should be faster than sequential search of 360 edges again > and again and again ... Yes, but - An index is also not really a direct access, but involves searching btree nodes. In-memory, linear lookup for a few dozens or hundreds of items is usually faster. - I'm not even sure if Pilog does much searching here, of if it just iterates the lists of clauses? > If it really need millions of edges to show effect, I can safely reside > on my Pilog facts, as my tracknet will never be so large. The 360 edges > are a not so small tracknet already :-D Yeah, so let's better stay with plain Pilog. Then the "database" is just the set of rules read into memory once, and then processing can start without further I/O. A database really makes only sense when - there are more data than fit into memory - only a small subset needs to be loaded into memory for a certain operation - the data are loaded/modified/written in arbitrary ways, and need to be persistent - multiple users operate on the data and need to be synchronized Did I forget something? So a constant, small set of rules can simply be read at program start. Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Alexander Burger wrote: > Hi Cle, > > >> Nice idea! Now I reformulated my Pilog rules for usage with database >> like this: >> >>(be graph:walk (@Begin @Way) >> (@E db 'id '+DB:Edge (-> @Begin)) >> (@To get (-> @E) 'tgt) >> (graph:walk @To (@Begin) @Way) >>) >> >>(be graph:walk (@Node @Way @Way)) >> >>(be graph:walk (@FromNode @AuxWay @Way) >> (@Edges collect 'src '+DB:Edge (-> @FromNode)) >> (lst @Edge @Edges) >> (@Id get (-> @Edge) 'id) >> (@ToNode get (-> @Edge) 'tgt) >> (graph:walk @ToNode (@Id . @AuxWay) @Way) >>) >> > > Hehe, nice. Our mails crossed-over, without paying attention to a point > or signal ;-) Yeah, but also without any accident ;-))) > The solutions look pretty similar. That should be so! As they say in China: 英雄所见略同 (Heros have the same mind/opinion) ;-) > As you see, the explicit 'get' can be substituted by a more concise > 'val' predicate. To rephrase the chinese sentence once more: your solution was exactly the same as I came up with my first attempt. But due to the fact, I use '+String' for all fields, the Pilog 'db' clause did find the wrong solutions. To handle that, I did introduce a 'equal' check then. But now, if I can use '+Number' for the id's the 'equal' check could vanish, I assume. :-) But I would guess -- if I understood your and Henriks explanation correctly -- that my solution should be faster, as I use the Lisp access functions, shouldn't it? > I also notice that the '@From' in my walk/2 is not used and thus > unnecessary. So I would propose (especially if you use a numeric 'id'): > I would think, *only* if I use numeric 'id', 'src' and 'tgt'. Otherwise the 'db' clause of Pilog would find a connection 'pt11' in e.g. the 'src' field although it was requested only to search for a connection from 'src' = 'pt1' ... :-( But I will have a look, if I can use numeric ids for all properties of an edge. I would guess that unification in my Pilog solution should also be faster, if those arguments were numeric instead of strings. :-) And perhaps the DB behaves better then. But if I do this, I will need another facts, to mark, which node ('src' or 'tgt') is a start or end point of a tracknet. At the moment those are named like "End#12" or such ... (...) > I would not worry about that. It is just the other way round: If the > interpreter finds an _undefined_function_ with a colon in the name, then > it searches for a library. This doesn't inhibit us, though, to define > such functions ourselves. And here we are not dealing with functions > anyway. > A good to know! So I can go forth with that procedure :-) >> With those changes the traversion becomes a bit faster; it needed 87 >> seconds for 1000 pathes, where Pilog facts would take 3 seconds ;-) >> > > This large difference is rather strange. I would expect _some_ slowdown, > as the DB involves I/O operations while Pilog only runs in memory, but > not so much. > That was, what made me crazy too ... I had thought, that direct access via an index should be faster than sequential search of 360 edges again and again and again ... So I am really eager to understand, why this is not the case, even with my Lisp database access in Pilog? Otherwise I feel I did something wrong perhaps ... > The question is what happens if you have millions of edges. Then the > linear lookup in Pilog versus the index accesses in the DB might show > effect. > If it really need millions of edges to show effect, I can safely reside on my Pilog facts, as my tracknet will never be so large. The 360 edges are a not so small tracknet already :-D Ciao, cle. -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Hi Cle, > Nice idea! Now I reformulated my Pilog rules for usage with database > like this: > >(be graph:walk (@Begin @Way) > (@E db 'id '+DB:Edge (-> @Begin)) > (@To get (-> @E) 'tgt) > (graph:walk @To (@Begin) @Way) >) > >(be graph:walk (@Node @Way @Way)) > >(be graph:walk (@FromNode @AuxWay @Way) > (@Edges collect 'src '+DB:Edge (-> @FromNode)) > (lst @Edge @Edges) > (@Id get (-> @Edge) 'id) > (@ToNode get (-> @Edge) 'tgt) > (graph:walk @ToNode (@Id . @AuxWay) @Way) >) Hehe, nice. Our mails crossed-over, without paying attention to a point or signal ;-) The solutions look pretty similar. As you see, the explicit 'get' can be substituted by a more concise 'val' predicate. I also notice that the '@From' in my walk/2 is not used and thus unnecessary. So I would propose (especially if you use a numeric 'id'): (be walk (@Begin @Way) (db id +Test @Begin @Obj) (val @To @Obj tgt) (walk @To (@Begin) @Way) ) (be walk (@Node @Way @Way)) (be walk (@FromNode @AuxWay @Way) (db src +Test @FromNode @Obj) (val @Edge @Obj id) (val @ToNode @Obj tgt) (walk @ToNode (@Edge . @AuxWay) @Way) ) > This is an excerp from my actual code. You will find, that I use a > : prefix in my clauses to modularize the Pilog clauses and > facts. A slight misuse I know, as those prefix should be used for > function in shared libraries ... :-} I would not worry about that. It is just the other way round: If the interpreter finds an _undefined_function_ with a colon in the name, then it searches for a library. This doesn't inhibit us, though, to define such functions ourselves. And here we are not dealing with functions anyway. > With those changes the traversion becomes a bit faster; it needed 87 > seconds for 1000 pathes, where Pilog facts would take 3 seconds ;-) This large difference is rather strange. I would expect _some_ slowdown, as the DB involves I/O operations while Pilog only runs in memory, but not so much. The question is what happens if you have millions of edges. Then the linear lookup in Pilog versus the index accesses in the DB might show effect. Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Hi Cle, > Ok, here comes my story! Currently there is an application/tool written > ... ># (Edge id fromNode toNode) >(be edge ("ts1" "end#1" "points#1")) >(be edge ("ts2" "points#1" "points#2")) >(be edge ("ts3" "points#1" "points#2")) >(be edge ("ts4" "points#2" "end#2")) > ... >(be walk (@Begin @Way) > (edge @Begin @From @To) > (walk @To (@Begin) @Way) >) > >(be walk (@Node @Way @Way)) > >(be walk (@FromNode @AuxWay @Way) > (edge @Edge @FromNode @ToNode) > (walk @ToNode (@Edge . @AuxWay) @Way) >) OK, so my modification of your original DB program would be (pool "x.db") (class +Test +Entity) (rel id (+Ref +String)) (rel src (+Ref +String)) (rel tgt (+Ref +String)) (new T '(+Test) 'id "ts1" 'src "end#1" 'tgt "points#1") (new T '(+Test) 'id "ts2" 'src "points#1" 'tgt "points#2") (new T '(+Test) 'id "ts3" 'src "points#1" 'tgt "points#2") (new T '(+Test) 'id "ts4" 'src "points#2" 'tgt "end#2") (commit) (be walk (@Begin @Way) (db id +Test @Begin @Obj) (val @From @Obj src) (val @To @Obj tgt) (walk @To (@Begin) @Way) ) (be walk (@Node @Way @Way)) (be walk (@FromNode @AuxWay @Way) (db src +Test @FromNode @Obj) (val @Edge @Obj id) (val @ToNode @Obj tgt) (walk @ToNode (@Edge . @AuxWay) @Way) ) Now, I must confess that I use the 'db' Pilog predicate here, instead of the suggested Lisp 'db' function. In the latter case, the line (db id +Test @Begin @Obj) would need to be replaced with (@Lst collect 'id '+Test (-> @Begin)) (lst @Obj @Lst) and (db src +Test @FromNode @Obj) would be (@Lst collect 'src '+Test (-> @FromNode)) (lst @Obj @Lst) No big thing, but not so beautiful ;-) But I remembered a fact which would allow us to stay with the beautiful solution: The 'db' predicate matches "val%" only for string types (for some obscure practical and historical reasons). If 'id' were a numeric type, then only exact matches are returned. Would this be an option, i.e. (rel id (+Ref +Number)) or probably even (rel id (+Key +Number)) If so, then we could safely stay with the 'db' predicate. Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Alexander Burger wrote: > > Hi Cle, Hi Alex, (...) >>> If 'new' is called this way, it will not create database objects >>> but >> >> ... :-O But ... but ... but ... URGHS! :-( I had looked especially >> into the "x.db" file and found my keys "1", "11" and "111" there. >> So I thought > > Right. I never tried it this way, but the reason is that though > there were memory-only objects created, the relation-handling > side-effects of '+Ref' etc. were executed, so that the indexes were > in the file but not the objects. ahhh ... now I understand, what I've seen :-) (...) > The way I would use is the already-discussed 'collect' (...) > If you really prefer Pilog also for direct accesses, then you can > still call it the direct way: Nice idea! Now I reformulated my Pilog rules for usage with database like this: (be graph:walk (@Begin @Way) (@E db 'id '+DB:Edge (-> @Begin)) (@To get (-> @E) 'tgt) (graph:walk @To (@Begin) @Way) ) (be graph:walk (@Node @Way @Way)) (be graph:walk (@FromNode @AuxWay @Way) (@Edges collect 'src '+DB:Edge (-> @FromNode)) (lst @Edge @Edges) (@Id get (-> @Edge) 'id) (@ToNode get (-> @Edge) 'tgt) (graph:walk @ToNode (@Id . @AuxWay) @Way) ) This is an excerp from my actual code. You will find, that I use a : prefix in my clauses to modularize the Pilog clauses and facts. A slight misuse I know, as those prefix should be used for function in shared libraries ... :-} With those changes the traversion becomes a bit faster; it needed 87 seconds for 1000 pathes, where Pilog facts would take 3 seconds ;-) Any other hint? :- > I'll study your project now ... Looks interesting :-) Thank you for your interest :-) > Cheers, - Alex Ciao, cle. -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Henrik Sarvell wrote: > > Sure, Pilog works fine if you have something more complex, but you > get quite far with (db), (aux) and (collect). (...) > Then you get all objects with D as src, I think you need to come up > with something more complex in order to actually get to a reasonable > real-world Pilog example. My Pilog program will be complex. As I have written in my last mail to Alex, I will have to traverse directed graphs egde by edge and ensure rules are fulfilled. An easier rule could be formulated like this: Ensure, that for every path thru a train station beginning from every entry signal up to the exit signal, there is no path longer than 500 m without any intervening signal or auxiliary signal :-) This could be nicely solved with Pilog :-))) > My tutorial series has a few fairly simple Pilog examples here: > http://www.prodevtips.com/2008/04/28/advanced-oodb-in-pico-lisp/ I had already a fly-over over your site, it is really nice done! I will look at it again, though. But I think, I will have no problem with Pilog, but more with the database thingy :-) (...) Ciao, cle. -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Hi Cle, > Ahhh ... this was a copy/paste error from the console into the mail. My > DB file was named "test" :-) > ... > Ahhh not for me ... as I am working within another directory parallel to > picoLisp ;-) I see. This explains it ;-) > > If 'new' is called this way, it will not create database objects but >... > :-O But ... but ... but ... URGHS! :-( I had looked especially into the > "x.db" file and found my keys "1", "11" and "111" there. So I thought Right. I never tried it this way, but the reason is that though there were memory-only objects created, the relation-handling side-effects of '+Ref' etc. were executed, so that the indexes were in the file but not the objects. > ... in my real program I used 'new!' and *not* 'new'. So OK, good :-) > Ok, perhaps I could use the Lisp 'db' function when I have a direct > access :-) But anyway I will also need the Pilog 'db' clause. But I > would need it with a functionality like SQL > >select ID, SRC, TGT > from +Test >where ID = "1"' > > an *not* > >select ID, SRC, TGT > from +Test > where ID like "1%" > > is there any way to achieve my goal? The way I would use is the already-discussed 'collect' : (collect 'id '+Test "1") -> ({2}) or, more close to your 'select' : (for This (collect 'id '+Test "1") (println (: id) (: src) (: tgt)) ) "1" "A" "B" -> "B" If you really prefer Pilog also for direct accesses, then you can still call it the direct way: : (? (@Test db 'id '+Test "1")) @Test={2} or, via a variable : (? @ID "1" (@Test db 'id '+Test (-> @ID))) @ID="1" @Test={2} I'll study your project now ... Looks interesting :-) Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Hi Henrik, > Sure, Pilog works fine if you have something more complex, but you get > quite far with (db), (aux) and (collect). As far as src and tgt goes > if they are not unique you can do for instance: > > (collect 'src '+Test "D" "D") > > Then you get all objects with D as src, I think you need to come up Correct, though in this case also just (collect 'src '+Test "D") would do. 'collect' with a single key argument will return a list of all hits having that key. The list will have of course only zero or one elements if the key is unique. Otherwise (collect 'src '+Test "from" "to") will return a list of all hits between (and including) these keys. Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Alexander Burger wrote: > > Hi Cle, Hi Alex, > let me first correct some errors: > >> (pool test) > > This is a funny one. As the value of 'test' is a function > definition, you get a rather strange database file ;-) Ahhh ... this was a copy/paste error from the console into the mail. My DB file was named "test" :-) > The name "test", on the other hand, will give a conflict as there is > a directory with that name. I take "x.db" for now: Ahhh not for me ... as I am working within another directory parallel to picoLisp ;-) > (pool "x.db") Ok, I agree :-) >> (class +Test +Entity) (rel id (+Ref +String)) (rel src (+Ref >> +String)) (rel tgt (+Ref +String)) >> >> (new '(+Test) 'id "1" 'src A 'tgt B) (new '(+Test) 'id "11" 'src >> B 'tgt "C") (new '(+Test) 'id "111" 'src C 'tgt "D") > > If 'new' is called this way, it will not create database objects but > memory-resident symbols. You should call either :-O But ... but ... but ... URGHS! :-( I had looked especially into the "x.db" file and found my keys "1", "11" and "111" there. So I thought .. now I feel stupid again :-( No! I am stupid ... in my real program I used 'new!' and *not* 'new'. So in my real slower program the DB was filled but only by accident :-} (...) > This would actually return no result at all, as you pass a _number_ > '1' while the key is a _string_. I suppose the double quotes got lost > when copy/pasting the expression from the console? Yes you are right! Those missing "" were not the problem. Only the missing 'T' was a real problem in this made-up example so far! Thanks for pointing this out! (...) > Yes, the Pilog queries 'select' and its small brother 'db' are > intended to return result _sets_, useful in GUI search components. > For simple direct DB accesses Pilog is rather overkill. You typically > use the Lisp function 'db' for that: > > : (db 'id '+Test "1") -> {2} > > The Pilog 'db' is actually a misnomer. It behaves more like the Lisp > 'collect' function. Ok, perhaps I could use the Lisp 'db' function when I have a direct access :-) But anyway I will also need the Pilog 'db' clause. But I would need it with a functionality like SQL select ID, SRC, TGT from +Test where ID = "1"' an *not* select ID, SRC, TGT from +Test where ID like "1%" is there any way to achieve my goal? (...) > True. Pilog is a powerful search mechanism, but you have to > understand that for a simple indexed access the overhead of setting > up the query and communicating environments from Lisp to Pilog and > back is much larger than the actual db access. > > So it is not recommended, and was never intended, to use Pilog to > retrieve a single value from the DB. Ok! So perhaps I cannot use the DB for my purposes :-( (...) > Perhaps you should try with the corrected version (see above). Also, > when using DB _plus_ Pilog, you get the overhead from both worlds. > You could also try a DB version without Pilog. Is this possible? I do not know for sure. I originally want to use Java + Prolog for my application. Then I detected your picoLisp ... (...) > There are many ways to tune the performance. Perhaps you can > elaborate a little more on the details? Ok, here comes my story! Currently there is an application/tool written in C++, which is very hard to maintain. The task is, to read-in some XML formatted description of a rail infrastructure. To let our main application deal with the original rail infrastructure (points, tracks, signals ...) a team has to take the original planning sketches and describe them into XML. There are a lot of rules to obey to describe the infrastructure properly. To help the team with that task, a check tool was written, that read in said XML description and check them according the rules of our projection handbook/guide. My idea now is that I can write a new check tool that will have the rules formulated in Pilog (formerly Prolog). All in all I would think, that the resulting tool will be much more friendlier to maintain as the said C++ one. For that I need to formulate the edges of the tracknet somehow (for instance) as Pilog facts like this: # (Edge id fromNode toNode) (be edge ("ts1" "end#1" "points#1")) (be edge ("ts2" "points#1" "points#2")) (be edge ("ts3" "points#1" "points#2")) (be edge ("ts4" "points#2" "end#2")) Would describe a graph like this (better look with a fixed size font): ts2 ___ ts1 / \ ts4 End#1 |<---o Points#1 Points#2 o--->| End#2 \___/ ts3 Then I have a Pilog clause like this: (be walk (@Begin @Way) (edge @Begin @From @To) (walk @To (@Begin) @Way) ) (be walk (@Node @Way @Way)) (be walk (@FromNode @AuxWay @Way) (edge @Edge @FromNode @ToNode) (walk @ToNode (@Edge . @AuxWay) @Way) ) Which would allow me to iterate from a certain edge
Re: Pilog faster than DB access?
Hi Cle, > > but if I were you and needed to access by unique id I would > > first use +Key in the relations: > > > > (rel id (+Key +String)) > > Ah, that makes sense! But it will only work for my id relation ... but I > will also access them by 'src' and 'tgt'. Please note that '+Key' will not change the 'collect'-like behavior of the Pilog 'db' predicate in that regard. It only creates a unique index (while '+Ref' maintains a non-unique index). Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Sure, Pilog works fine if you have something more complex, but you get quite far with (db), (aux) and (collect). As far as src and tgt goes if they are not unique you can do for instance: (collect 'src '+Test "D" "D") Then you get all objects with D as src, I think you need to come up with something more complex in order to actually get to a reasonable real-world Pilog example. My tutorial series has a few fairly simple Pilog examples here: http://www.prodevtips.com/2008/04/28/advanced-oodb-in-pico-lisp/ /Henrik On Fri, Dec 11, 2009 at 9:54 AM, cle wrote: > > Hello Henrik, > > Henrik Sarvell wrote: >> >> I don't know exactly the circumstances of what you're trying to >> achieve, > > ok, I will try to answer this in the next mail to Alex ... > >> but if I were you and needed to access by unique id I would >> first use +Key in the relations: >> >> (rel id (+Key +String)) > > Ah, that makes sense! But it will only work for my id relation ... but I > will also access them by 'src' and 'tgt'. > >> Then I would use (db) to access (like you do but without the Pilog): >> >> (db 'id '+Test "1") > > Ok, I will try this too for sure ... thanks! :-) > >> I think that's the fastest way, see also (collect) and (aux) for >> more utility functions that will let you access the DB without using >> Pilog. > > But in the end, I *want* to use Pilog, I explain in the later mail. :-) > > (...) > > Thank you for your non-ending effort to answer my noob questions :-) > > > Ciao, > cle. > > -- > UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe > -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Hello Henrik, Henrik Sarvell wrote: > > I don't know exactly the circumstances of what you're trying to > achieve, ok, I will try to answer this in the next mail to Alex ... > but if I were you and needed to access by unique id I would > first use +Key in the relations: > > (rel id (+Key +String)) Ah, that makes sense! But it will only work for my id relation ... but I will also access them by 'src' and 'tgt'. > Then I would use (db) to access (like you do but without the Pilog): > > (db 'id '+Test "1") Ok, I will try this too for sure ... thanks! :-) > I think that's the fastest way, see also (collect) and (aux) for > more utility functions that will let you access the DB without using > Pilog. But in the end, I *want* to use Pilog, I explain in the later mail. :-) (...) Thank you for your non-ending effort to answer my noob questions :-) Ciao, cle. -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
Hi Cle, let me first correct some errors: > (pool test) This is a funny one. As the value of 'test' is a function definition, you get a rather strange database file ;-) The name "test", on the other hand, will give a conflict as there is a directory with that name. I take "x.db" for now: (pool "x.db") > (class +Test +Entity) > (rel id (+Ref +String)) > (rel src (+Ref +String)) > (rel tgt (+Ref +String)) > > (new '(+Test) 'id "1" 'src A 'tgt B) > (new '(+Test) 'id "11" 'src B 'tgt "C") > (new '(+Test) 'id "111" 'src C 'tgt "D") If 'new' is called this way, it will not create database objects but memory-resident symbols. You should call either (new T '(+Test) 'id "1" 'src A 'tgt B) if there is a single db file, or (new (db: +Test) '(+Test) 'id "1" 'src A 'tgt B) in the general case. The values passed for 'src' and 'tgt' are random, depending on the bindings of 'A' and 'B'. Probably NIL. > (commit) > > (? (db id +Test 1 @E) (show @E)) This would actually return no result at all, as you pass a _number_ '1' while the key is a _string_. I suppose the double quotes got lost when copy/pasting the expression from the console? However, you are right in your observation: > The last Pilog query will successively return three answers where I > thought to get only one. I understand, why 'db' will deliver three > results. But the question comes up, how I could formulate the query > using only exact matches! Yes, the Pilog queries 'select' and its small brother 'db' are intended to return result _sets_, useful in GUI search components. For simple direct DB accesses Pilog is rather overkill. You typically use the Lisp function 'db' for that: : (db 'id '+Test "1") -> {2} The Pilog 'db' is actually a misnomer. It behaves more like the Lisp 'collect' function. > (? (db id +Test "1" @E) (val @Id @E id) (equal @Id "1") (show @E)) > > But this would defeat the reason, why I tried to use a DB at all. I > hoped that accessing an Edge via an index would be faster than asking > Pilog to match against all its facts until it found a corresponding it > could unify against. True. Pilog is a powerful search mechanism, but you have to understand that for a simple indexed access the overhead of setting up the query and communicating environments from Lisp to Pilog and back is much larger than the actual db access. So it is not recommended, and was never intended, to use Pilog to retrieve a single value from the DB. > So could this be the reason that my DB solution of traversing all ways > thru a graph is a lot slower than using Pilog facts? Both are trying to For such a small database this may well be possible. In general, if all data fit into memory, a non-DB version might well be faster. But I wouln't expect a dramatic difference. > iterate over the same graph built by 362 edges. Whereas the Pilog facts > will allow for finding ca. 1000 ways per 1-2 seconds, the database > solution was canceled after some time, as it did not reach the first > 1000 solutions before I lost my patience ;-) Perhaps you should try with the corrected version (see above). Also, when using DB _plus_ Pilog, you get the overhead from both worlds. You could also try a DB version without Pilog. Is this possible? > Or is there any way to improve the performance of the DB? For instance > by only matching exactly in the db query for a certain edge? There are many ways to tune the performance. Perhaps you can elaborate a little more on the details? Cheers, - Alex -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Re: Pilog faster than DB access?
I don't know exactly the circumstances of what you're trying to achieve, but if I were you and needed to access by unique id I would first use +Key in the relations: (rel id (+Key +String)) Then I would use (db) to access (like you do but without the Pilog): (db 'id '+Test "1") I think that's the fastest way, see also (collect) and (aux) for more utility functions that will let you access the DB without using Pilog. /Henrik On Fri, Dec 11, 2009 at 5:25 AM, cle wrote= : > Hello, > > today I tried to verify if using a database instead of Pilog facts would > speed up my processing. While trying this, I stumbled over the > following. Suppose: > > =A0(pool test) > > =A0(class +Test +Entity) > =A0(rel id (+Ref +String)) > =A0(rel src (+Ref +String)) > =A0(rel tgt (+Ref +String)) > > =A0(new '(+Test) 'id "1" =A0'src A =A0'tgt B) > =A0(new '(+Test) 'id "11" =A0'src B =A0'tgt "C") > =A0(new '(+Test) 'id "111" =A0'src C =A0'tgt "D") > > =A0(commit) > > =A0(? (db id +Test 1 @E) (show @E)) > > The last Pilog query will successively return three answers where I > thought to get only one. I understand, why 'db' will deliver three > results. But the question comes up, how I could formulate the query > using only exact matches! > > Without this, the 'db' query would return similar matches that I have to > explicitely guarding against with e.g. 'equal' like this: > > =A0(? (db id +Test "1" @E) (val @Id @E id) (equal @Id "1") (show @E)) > > But this would defeat the reason, why I tried to use a DB at all. I > hoped that accessing an Edge via an index would be faster than asking > Pilog to match against all its facts until it found a corresponding it > could unify against. > > So could this be the reason that my DB solution of traversing all ways > thru a graph is a lot slower than using Pilog facts? Both are trying to > iterate over the same graph built by 362 edges. Whereas the Pilog facts > will allow for finding ca. 1000 ways per 1-2 seconds, the database > solution was canceled after some time, as it did not reach the first > 1000 solutions before I lost my patience ;-) > > Or is there any way to improve the performance of the DB? For instance > by only matching exactly in the db query for a certain edge? > > > Thanks and Ciao, > cle. > > -- > UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=3dunsubscribe > -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe
Pilog faster than DB access?
Hello, today I tried to verify if using a database instead of Pilog facts would speed up my processing. While trying this, I stumbled over the following. Suppose: (pool test) (class +Test +Entity) (rel id (+Ref +String)) (rel src (+Ref +String)) (rel tgt (+Ref +String)) (new '(+Test) 'id "1" 'src A 'tgt B) (new '(+Test) 'id "11" 'src B 'tgt "C") (new '(+Test) 'id "111" 'src C 'tgt "D") (commit) (? (db id +Test 1 @E) (show @E)) The last Pilog query will successively return three answers where I thought to get only one. I understand, why 'db' will deliver three results. But the question comes up, how I could formulate the query using only exact matches! Without this, the 'db' query would return similar matches that I have to explicitely guarding against with e.g. 'equal' like this: (? (db id +Test "1" @E) (val @Id @E id) (equal @Id "1") (show @E)) But this would defeat the reason, why I tried to use a DB at all. I hoped that accessing an Edge via an index would be faster than asking Pilog to match against all its facts until it found a corresponding it could unify against. So could this be the reason that my DB solution of traversing all ways thru a graph is a lot slower than using Pilog facts? Both are trying to iterate over the same graph built by 362 edges. Whereas the Pilog facts will allow for finding ca. 1000 ways per 1-2 seconds, the database solution was canceled after some time, as it did not reach the first 1000 solutions before I lost my patience ;-) Or is there any way to improve the performance of the DB? For instance by only matching exactly in the db query for a certain edge? Thanks and Ciao, cle. -- UNSUBSCRIBE: mailto:picol...@software-lab.de?subject=unsubscribe