Re: Graph database

[George-Phillip Orais]

Thank you beneroth and Joh-Tob for this impressive and insightful
explanation, very informative for me as well, thank you, I will put this on
my PicoLisp notes.

On Thu, Mar 26, 2020 at 8:29 AM Joh-Tob Schäg  wrote:

> Have you already looked at the family example?
>
> Here is my brief overview of ascending order of abstractedness:
> PicoLisp had no graph database. What is has is this:
>  - The ability to serialize/de-serialize all structures in the heap
> (Lists, numbers, functions, symbols etc)
>  - It has the ability to fetch and deserialize structures when they
> are accessed by their reference (also called external symbol) and also
> GC the loaded content when it is no longer accessable
>  - the ability to make atomic changes to such structures (by write
> locking the database).
>  - it has relations which allow the database to place pointers in one
> or multiple directions, make space for indexes etc... (I have not
> fully grasped that myself)
>  - The are indexes which automatically maintained and those are
> accessible to the database allowing you to discover objects placed in
> the database.
>  - It has an mechanism which allows you to maintain database
> consistent of different machines (replication)
>
> In PicoLisp you can solve many graph database problems with those
> structures, however PicoLisp does not care or know what a vertex is.
> It does not come support for weighted vertexes, you could build your
> own class but is usually more efficient to just implement what you
> actually need, not implement some paradigm because you heard of
> it/like it.
>
> Tailor PicoLisp to your problem, not your paradigm. This is a thing i
> had to learn over a long time.
>
> On Wed, 25 Mar 2020 at 21:46, Lawrence Bottorff  wrote:
> >
> > I'm afraid at my level of CS theory I don't really know what is meant by
> a picolisp atom being persistent, much less across distributed picolisp
> instances. Could someone give me a concrete example of what you describe
> as: "Any named bag of items automatically represents a (directed,
> undirected) graph. The name then is the node, the items in the bag then
> there represent the edges." I do understand the tree structure of a lisp
> program. But that doesn't make it a graph database. When I tried to fathom
> the Picolisp "graph database" example, I was quickly confused. The GUI
> actually added confusion, AFAIC. I'm guessing from what I could ausknobeln
> from example that the Picolisp version of a CLOS object is a vertex, and
> the inheritance of that object from other (higher, more general?) objects
> is a sort of edge. Correct me if I'm wrong. But then there was talk of
> "records." Is creating a record the same as creating an object instance --
> and this record/object is a vertex? Where, what are the edges?
> >
> > Don't get me wrong, I have long felt that Lisp -- with its parsing
> actually visible in the code you write -- is or could be very
> graphDB-friendly; however, Lisp is functional, i.e., you write functions.
> And even though they are set up as a graph-like tree in nested lists form,
> they are not in themselves data in the traditional sense, rather, code
> meant to take you from a domain/input to a range/result. This is not a
> "record" (or graph vertex?) creation/query/deletion paradigm.
> >
> > But this relates to a long-standing question I've had about software
> libraries. As it stands, they may be auto-indexed for our viewing pleasure,
> but they aren't in any real database form so that you might simply have
> your program "query-and-plug-in" a library. (Although I've heard Haskell's
> hlint almost writes your code for you!) No, you have to find the module,
> plug it in yourself. The whole "code is data", therefore, doesn't seem to
> get past the higher-order function trick of passing in a function as an
> argument. What more is there to "code is data?" In Fortran the data was in
> fact parked just below the code.
> >
> > At some point I'm just scared and rambling on
> >
> > On Wed, Mar 25, 2020 at 7:12 AM Guido Stepken 
> wrote:
> >>
> >> Lawrence, you haven't yet understood, that any Lisp, by default, is
> it's own Graph Database. Especially Picolisp, where Alex has made any
> Picolisp Atom persistent and even distributed across other Picolisp
> instances. 'Data is code, code is data'.
> >>
> >> Any named bag of items automatically represents a (directed,
> undirected) graph. The name then is the node, the items in the bag then
> there represent the edges. Even Picolisp sources you can consider a
> (directed) graph, often also called 'syntax tree'.
> >>
> >> If you like, you can put, group all "edges" with same properties into a
> new, searchable bag of edges for fast lookup. Since it's all lazy evaluated
> (even the persistent nodes), as Alex already pointed out, it's still ultra
> fast. And since in Picolisp everything can be persisted distributed,
> Picolisp automatically represents a distributed graph database (with
> sharding and 

Re: Graph database

[andreas]

Dear Lawrence

Sounds to me that your head got stuffed a bit too well with
over-complicated concepts. No offense! That is the nature of most
software education, and its even worse in the business world. And we
programmers have a high tendency to believe we are more clever when we
are working on more complex systems:"Weeks of coding can save you hours
of planning."

PicoLisp is radical in its focus on simplicity, although it is not an
easy language. Less so if one is trained by education and other
languages to think in more complicated patterns than really necessary. I
went through this struggles too, and it made me into an overall much
better software developer, also in other languages & stacks.

Some fundamentals:

  * PicoLisp has not much in common with CommonLisp, though they share
common ancestors
  * As with lisp languages in general: the source code is not a list of
instructions, but an abstract syntax tree (AST)
  o compilers (in the mainstream sense) for other programming
languages construct internally an AST from source code, and then
they optimize that AST and translate it to machine code
  o for PicoLisp, the reader (R of REPL) translates source code
(which can also be repl text input) into an AST, basically trees
of pointers
  o this AST /is/ nothing else than a data structure, the source
code (textual data) becomes a tree in RAM (lists of lists)
  o PicoLisp can modify this AST (the code data structure) during
execution - primarily by interpret it as code or data, both is
the exact same stuff in memory (more so in PicoLisp than some
other lisps)
  * PicoLisp is a multi paradigm language
  o it is not purely functional (like e.g. Haskell)
  o operations can be immutable or mutable
  * In Lisp (*l**is*t *p*rocessor) languages are based on lists
(everything with parentheses around it is a list)
  o A list is always a grouping of (possibly) multiple elements
  o Elements which are not lists themselves are called *atoms*
(single things)
  * PicoLisp has only 3 fundamental data types:
  o Number
  + signed integer of arbitrary size
  + this is an atom (atomic type)
  o Lists
  + singly linked list (so always an ordered sequence of
elements, not a unordered set)
  + may contain lists and atoms
  o Symbol
  + has a name
  + has a value
  + may have an arbitrary number of properties
  # a property consists of a name and a value
  + this is an atom (atomic type)
  o a /value/ always is of one of these 3 data types
  + other than in other languages, variables don't have a data
type (the value has a data type)
  + variables in PicoLisp are just symbols
  + the data type of an value is both static and strong (cannot
be changed)
  o all other data types (e.g. classes / objects) are based on these
3 fundamental PicoLisp types
  + non-fundamental types have practically no enforcement, no
checks - unless explicitly called
  # not entirely true, the fundamental data types have
built-in sub-variants for which certain rules apply
(e.g. transient symbols, primarily used as string type)
  +  so all non-fundamental types are *duck typed*

  + this enables easier code reuse
  o for example: the OOP system in PicoLisp is primarily based on
the symbol data type
  + classes and objects are symbols which follow certain principles
  # member variables (attributes) are stored as properties
of the symbol
  # the value of the symbol is a list containing parent
classes and methods
  * PicoLisp database mechanisms is multi paradigm
  o *on lowest level: a /key-value store/*
  + just values of the symbol data type which are persisted to disk
  + the so-called *external symbols* - a sub-variant of the
fundamental symbol data type
  + the name of such an external symbol is the logical block
address of the data within the database file(s)
  + external symbols are automatically loaded into RAM on first
access (lazy loading)
  o *external symbols combined with PicoLisp OOP system: /object
database/*
  + persistent OOP objects
  + no translation/copying between objects in RAM and database
(no ORM problem)
  + as the objects are just external symbols which follow
certain principles, they're lazy loaded when accessed
  o *relationships between database classes (entities) defined in a
schema: /graph database/*
  + not exactly following the strict academic definitions for
graph database
   

Re: Graph database

[Lawrence Bottorff]

I'm afraid at my level of CS theory I don't really know what is meant by a
picolisp atom being persistent, much less across distributed picolisp
instances. Could someone give me a concrete example of what you describe
as: "Any named bag of items automatically represents a (directed,
undirected) graph. The name then is the node, the items in the bag then
there represent the edges." I do understand the tree structure of a lisp
program. But that doesn't make it a graph database. When I tried to fathom
the Picolisp "graph database" example, I was quickly confused. The GUI
actually added confusion, AFAIC. I'm guessing from what I could
ausknobeln from example that the Picolisp version of a CLOS object is a
vertex, and the inheritance of that object from other (higher, more
general?) objects is a sort of edge. Correct me if I'm wrong. But then
there was talk of "records." Is creating a record the same as creating an
object instance -- and this record/object is a vertex? Where, what are the
edges?

Don't get me wrong, I have long felt that Lisp -- with its parsing actually
visible in the code you write -- is or could be very graphDB-friendly;
however, Lisp is functional, i.e., you write functions. And even though
they are set up as a graph-like tree in nested lists form, they are not in
themselves data in the traditional sense, rather, code meant to take you
from a domain/input to a range/result. This is not a "record" (or graph
vertex?) creation/query/deletion paradigm.

But this relates to a long-standing question I've had about software
libraries. As it stands, they may be auto-indexed for our viewing pleasure,
but they aren't in any real database form so that you might simply have
your program "query-and-plug-in" a library. (Although I've heard Haskell's
hlint almost writes your code for you!) No, you have to find the module,
plug it in yourself. The whole "code is data", therefore, doesn't seem to
get past the higher-order function trick of passing in a function as an
argument. What more is there to "code is data?" In Fortran the data was in
fact parked just below the code.

At some point I'm just scared and rambling on

On Wed, Mar 25, 2020 at 7:12 AM Guido Stepken  wrote:

> Lawrence, you haven't yet understood, that any Lisp, by default, is it's
> own Graph Database. Especially Picolisp, where Alex has made any Picolisp
> Atom persistent and even distributed across other Picolisp instances. 'Data
> is code, code is data'.
>
> Any named bag of items automatically represents a (directed, undirected)
> graph. The name then is the node, the items in the bag then there represent
> the edges. Even Picolisp sources you can consider a (directed) graph, often
> also called 'syntax tree'.
>
> If you like, you can put, group all "edges" with same properties into a
> new, searchable bag of edges for fast lookup. Since it's all lazy evaluated
> (even the persistent nodes), as Alex already pointed out, it's still ultra
> fast. And since in Picolisp everything can be persisted distributed,
> Picolisp automatically represents a distributed graph database (with
> sharding and everything) which you can build, implement on your own with
> just a few lines of code. It's a no-brainer!
>
> Picolisp is a genius strike, but most people can't see the forest for all
> the trees.
>
> Have fun!
>
> Regards, Guido Stepken
>
> P.S. Keep away from Windows and other viruses!
>
> Am Donnerstag, 12. März 2020 schrieb Lawrence Bottorff  >:
>
>> I take it the picolisp graph database follows more the Neo4j property
>> graph idea than any RDF/OWL triples, correct? That seems obvious, but I
>> thought I'd check. I haven't dived in deep, buy you seem to use Lisp
>> objects to create a vertex. But then what are the edges? Again, I'm just
>> getting started.
>>
>> LB
>> Grand Marais, MN, Oberer See
>>
>

Re: Graph database

[Guido Stepken]

Lawrence, you haven't yet understood, that any Lisp, by default, is it's
own Graph Database. Especially Picolisp, where Alex has made any Picolisp
Atom persistent and even distributed across other Picolisp instances. 'Data
is code, code is data'.

Any named bag of items automatically represents a (directed, undirected)
graph. The name then is the node, the items in the bag then there represent
the edges. Even Picolisp sources you can consider a (directed) graph, often
also called 'syntax tree'.

If you like, you can put, group all "edges" with same properties into a
new, searchable bag of edges for fast lookup. Since it's all lazy evaluated
(even the persistent nodes), as Alex already pointed out, it's still ultra
fast. And since in Picolisp everything can be persisted distributed,
Picolisp automatically represents a distributed graph database (with
sharding and everything) which you can build, implement on your own with
just a few lines of code. It's a no-brainer!

Picolisp is a genius strike, but most people can't see the forest for all
the trees.

Have fun!

Regards, Guido Stepken

P.S. Keep away from Windows and other viruses!

Am Donnerstag, 12. März 2020 schrieb Lawrence Bottorff :

> I take it the picolisp graph database follows more the Neo4j property
> graph idea than any RDF/OWL triples, correct? That seems obvious, but I
> thought I'd check. I haven't dived in deep, buy you seem to use Lisp
> objects to create a vertex. But then what are the edges? Again, I'm just
> getting started.
>
> LB
> Grand Marais, MN, Oberer See
>

Re: Graph database

[Alexander Burger]

On Fri, Mar 13, 2020 at 10:22:12PM +0100, Alexander Burger wrote:
> Perhaps this helps? https://software-lab.de/doc/tut.html#ext
> 
> It is really very simple.

Understanding PicoLisp symbols is perhaps the important point.

   https://software-lab.de/doc/ref.html#symbol

The rest is just making symbols persistent.

☺/ A!ex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Graph database

[Alexander Burger]

On Fri, Mar 13, 2020 at 04:02:44PM -0500, Lawrence Bottorff wrote:
> Could you point me to a beginner's treatment of this topic, especially an
> example of a graph database, and what exactly a picolisp pointer is?

Perhaps this helps? https://software-lab.de/doc/tut.html#ext

It is really very simple.


> I'm afraid I don't even know what you mean by a pointer in this context. I
> know from C what a pointer is, but a picolisp pointer is beyond my
> comprehension.

Exactly the same as C pointers. The 32-bit version of PicoLisp *is* even written
in C.

The external symbols *point* to each other when loaded inte memory, via
properties or arbitrary Lisp structures. And they are loaded *into* memory on
demand, implicitly when they are accessed (lazy loading).

The higher DB levels then implement entity and relation classes, B-Trees etc.,
all on top of external symbols.

☺/ A!ex

-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Re: Graph database

[Lawrence Bottorff]

Could you point me to a beginner's treatment of this topic, especially an
example of a graph database, and what exactly a picolisp pointer is?

I'm afraid I don't even know what you mean by a pointer in this context. I
know from C what a pointer is, but a picolisp pointer is beyond my
comprehension.

On Fri, Mar 13, 2020 at 3:28 PM Alexander Burger 
wrote:

> Hi Lawrence,
>
> > I take it the picolisp graph database follows more the Neo4j property
> graph
> > idea than any RDF/OWL triples, correct? That seems obvious, but I thought
> > I'd check. I haven't dived in deep, buy you seem to use Lisp objects to
> > create a vertex. But then what are the edges? Again, I'm just getting
> > started.
>
> Sorry, I know absolutely nothing about Neo4j or RDF/OWL, but the edges are
> in
> fact trivial.
>
> As everything in PicoLisp is a pointer (except short numbers), the edges
> are
> indeed just pointers to other symbols. All those symbols in a DB (called
> "external symbols") are loaded lazily on demand.
>
> ☺/ A!ex
>
>
>
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
>

Re: Graph database

[Alexander Burger]

Hi Lawrence,

> I take it the picolisp graph database follows more the Neo4j property graph
> idea than any RDF/OWL triples, correct? That seems obvious, but I thought
> I'd check. I haven't dived in deep, buy you seem to use Lisp objects to
> create a vertex. But then what are the edges? Again, I'm just getting
> started.

Sorry, I know absolutely nothing about Neo4j or RDF/OWL, but the edges are in
fact trivial.

As everything in PicoLisp is a pointer (except short numbers), the edges are
indeed just pointers to other symbols. All those symbols in a DB (called
"external symbols") are loaded lazily on demand.

☺/ A!ex




-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe