Re: 101 on simple and flexible graph query/update?
@Francois Rey : I totally forgot that Datomic could be used on regular data
structures. Thanks a lot, it might indeed be something to investigate.
@Frank Castellucci : Indeed, I guess someone working with ontologies would
have the same concerns. Thanks for all the pointers.
Le vendredi 11 juillet 2014 03:48:51 UTC+2, Frank Castellucci a écrit :
Bertrand
I've been doing a lot of Ontology (OWL) with directed graphs (with cycles)
and DAGS. One application we went with the nested map structure however now
that I've been working on a normalized BPMN/XPDL utility (
https://github.com/FrankC01/tributary
https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2FFrankC01%2Ftributarysa=Dsntz=1usg=AFQjCNGzsWatrlDLLVU4uXDM-UEbR_wDjA
) I'm becoming enamored with the in XML format produced by xml-parse and
then zipping it up. Coupled with clojures 'functional' nature, zippers
become your friend really quickly and provide a most generic way of dealing
with any graph with any number of layers.
The next step will be a normalized DSL on top of the zippers and using
things like hickory-parse/data.zip selector concepts.
For the Ontology work we use datomic as the persist mechanism.
On Thursday, July 10, 2014 4:26:33 AM UTC-4, Bertrand Dechoux wrote:
Hi,
I have various general questions about how one can perform simple and
flexible graph query/update.
If we start by a structure being nested maps then I know about
get-in/update-in/assoc-in. The common point about these functions is that
they all take a path to where the data is in the superstructure and that
this path is only a sequence of keys, which may or not be symbols.
1) What happens when one layer of this superstructure is not a map but a
sequence? For the sake of the argument, we can keep the discussion simple
by only arguing about the vector case.
Typically instead of
{ :key1 { :key2 val }}
we now have
{ :key1 [{ :key2 val }]}
Of course, one can perform get/update/assoc using map/reduce. However, it
means that the code using this superstructure is taking a hit on
complexity/maintenance each time a non-map layer must be crossed. Of
course, the simplicity/beauty of the current implementation is that a path
can point to at most one target substructure/value. Is there a way to do
something similar with a more general definition of a path? One would
assume that it is something that might be handled by xml/html modification
libraries (think xpath or css selector) but this problem is more general
and not domain specific. Is there a clean library that handle that case
without being tied to html/xml?
2) How to deal with more general superstructure like a graph?
One could build an immutable graph but I don't believe it is something
that can be done by nesting immutable maps and vectors. One solution to
deal with 'entity'--'entity' relationships is for one of the member to own
only an identifier to get the other member from another reference structure.
(from the basic OM tutorial:)
{:people
[{:type :student :first Ben :last Bitdiddle}
{:type :professor :first Gerald :last Sussman :classes [:6001
:6946]}]
:classes
{:6001 The Structure and Interpretation of Computer Programs
:6946 The Structure and Interpretation of Classical Mechanics
:1806 Linear Algebra}}
It is, of course, again, manually possible to get/update/assoc using
map/reduce. But then, again, this is the same problematic as with the
first question : each time a relation needs to be crossed, the code using
this superstructure is taking a hit on complexity/maintenance. How do you
usually deal with this problematic?
For both question 1) and 2), a more appropriate data structure might be
the answer like a graph/semantic-like store (Datomic or something else).
The questions are about intermediary solutions which would be less heavier.
3) How does Demeter lives with graph traversal?
This law is often heard in the OOP world but it is a bit more general
than that. When a long path on a superstructure is specified then if one
intermediary layer is introduced later, all hardcoded paths will be broken
ie in multiple locations in the code base. One would like to store local
structure knowledge in a single place. How do you usually deal with this
problematic?
I have yet to take a serious look at lenses and their composition, they
are probably an element of answer, but they are more often seen in more
statically typed langage.
Thanks for any feedback.
Bertrand
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
101 on simple and flexible graph query/update?
Hi,
I have various general questions about how one can perform simple and
flexible graph query/update.
If we start by a structure being nested maps then I know about
get-in/update-in/assoc-in. The common point about these functions is that
they all take a path to where the data is in the superstructure and that
this path is only a sequence of keys, which may or not be symbols.
1) What happens when one layer of this superstructure is not a map but a
sequence? For the sake of the argument, we can keep the discussion simple
by only arguing about the vector case.
Typically instead of
{ :key1 { :key2 val }}
we now have
{ :key1 [{ :key2 val }]}
Of course, one can perform get/update/assoc using map/reduce. However, it
means that the code using this superstructure is taking a hit on
complexity/maintenance each time a non-map layer must be crossed. Of
course, the simplicity/beauty of the current implementation is that a path
can point to at most one target substructure/value. Is there a way to do
something similar with a more general definition of a path? One would
assume that it is something that might be handled by xml/html modification
libraries (think xpath or css selector) but this problem is more general
and not domain specific. Is there a clean library that handle that case
without being tied to html/xml?
2) How to deal with more general superstructure like a graph?
One could build an immutable graph but I don't believe it is something that
can be done by nesting immutable maps and vectors. One solution to deal
with 'entity'--'entity' relationships is for one of the member to own only
an identifier to get the other member from another reference structure.
(from the basic OM tutorial:)
{:people
[{:type :student :first Ben :last Bitdiddle}
{:type :professor :first Gerald :last Sussman :classes [:6001 :6946]}]
:classes
{:6001 The Structure and Interpretation of Computer Programs
:6946 The Structure and Interpretation of Classical Mechanics
:1806 Linear Algebra}}
It is, of course, again, manually possible to get/update/assoc using
map/reduce. But then, again, this is the same problematic as with the
first question : each time a relation needs to be crossed, the code using
this superstructure is taking a hit on complexity/maintenance. How do you
usually deal with this problematic?
For both question 1) and 2), a more appropriate data structure might be the
answer like a graph/semantic-like store (Datomic or something else). The
questions are about intermediary solutions which would be less heavier.
3) How does Demeter lives with graph traversal?
This law is often heard in the OOP world but it is a bit more general than
that. When a long path on a superstructure is specified then if one
intermediary layer is introduced later, all hardcoded paths will be broken
ie in multiple locations in the code base. One would like to store local
structure knowledge in a single place. How do you usually deal with this
problematic?
I have yet to take a serious look at lenses and their composition, they are
probably an element of answer, but they are more often seen in more
statically typed langage.
Thanks for any feedback.
Bertrand
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: 101 on simple and flexible graph query/update?
I usually just give every ENTITY in my system a primary id and structure
everything in a nested map.
{:people {1 {:id 1 :type :student :name Ben :taking-classes #{1 2}}
2 ...}
:classes {1 {:id 1 :name Clojure 101}
2 ...}}
This basically looks like a database table. You can also go with something
a little more flat.
{1 {:type :student, :name Ben :taking-classes #{3}}
3 {:type :class, :name Clojure 101}}
Usually needs some form of indexing otherwise you are taking some
performance hits if you have alot of entities in your system. You dont want
to (- (vals @data) (filter #(= :class (:type %))) every time you want to
list all :classes.
Nothing fancy but it is simple.
On Thursday, July 10, 2014 10:26:33 AM UTC+2, Bertrand Dechoux wrote:
Hi,
I have various general questions about how one can perform simple and
flexible graph query/update.
If we start by a structure being nested maps then I know about
get-in/update-in/assoc-in. The common point about these functions is that
they all take a path to where the data is in the superstructure and that
this path is only a sequence of keys, which may or not be symbols.
1) What happens when one layer of this superstructure is not a map but a
sequence? For the sake of the argument, we can keep the discussion simple
by only arguing about the vector case.
Typically instead of
{ :key1 { :key2 val }}
we now have
{ :key1 [{ :key2 val }]}
Of course, one can perform get/update/assoc using map/reduce. However, it
means that the code using this superstructure is taking a hit on
complexity/maintenance each time a non-map layer must be crossed. Of
course, the simplicity/beauty of the current implementation is that a path
can point to at most one target substructure/value. Is there a way to do
something similar with a more general definition of a path? One would
assume that it is something that might be handled by xml/html modification
libraries (think xpath or css selector) but this problem is more general
and not domain specific. Is there a clean library that handle that case
without being tied to html/xml?
2) How to deal with more general superstructure like a graph?
One could build an immutable graph but I don't believe it is something
that can be done by nesting immutable maps and vectors. One solution to
deal with 'entity'--'entity' relationships is for one of the member to own
only an identifier to get the other member from another reference structure.
(from the basic OM tutorial:)
{:people
[{:type :student :first Ben :last Bitdiddle}
{:type :professor :first Gerald :last Sussman :classes [:6001
:6946]}]
:classes
{:6001 The Structure and Interpretation of Computer Programs
:6946 The Structure and Interpretation of Classical Mechanics
:1806 Linear Algebra}}
It is, of course, again, manually possible to get/update/assoc using
map/reduce. But then, again, this is the same problematic as with the
first question : each time a relation needs to be crossed, the code using
this superstructure is taking a hit on complexity/maintenance. How do you
usually deal with this problematic?
For both question 1) and 2), a more appropriate data structure might be
the answer like a graph/semantic-like store (Datomic or something else).
The questions are about intermediary solutions which would be less heavier.
3) How does Demeter lives with graph traversal?
This law is often heard in the OOP world but it is a bit more general than
that. When a long path on a superstructure is specified then if one
intermediary layer is introduced later, all hardcoded paths will be broken
ie in multiple locations in the code base. One would like to store local
structure knowledge in a single place. How do you usually deal with this
problematic?
I have yet to take a serious look at lenses and their composition, they
are probably an element of answer, but they are more often seen in more
statically typed langage.
Thanks for any feedback.
Bertrand
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: 101 on simple and flexible graph query/update?
On 10/07/14 10:26, Bertrand Dechoux wrote: For both question 1) and 2), a more appropriate data structure might be the answer like a graph/semantic-like store (Datomic or something else). The questions are about intermediary solutions which would be less heavier. I would suggest you look into the walk http://clojuredocs.org/clojure_core/clojure.walk/walk/prewalk http://clojuredocs.org/clojure_core/clojure.walk/prewalk/postwalk http://clojuredocs.org/clojure_core/clojure.walk/postwalk functions. These are able to traverse arbitrary nested data structures while building a different or modified data structure. 3) How does Demeter lives with graph traversal? This law is often heard in the OOP world but it is a bit more general than that. When a long path on a superstructure is specified then if one intermediary layer is introduced later, all hardcoded paths will be broken ie in multiple locations in the code base. One would like to store local structure knowledge in a single place. How do you usually deal with this problematic? I have yet to take a serious look at lenses and their composition, they are probably an element of answer, but they are more often seen in more statically typed langage. The walk http://clojuredocs.org/clojure_core/clojure.walk/walk/prewalk http://clojuredocs.org/clojure_core/clojure.walk/prewalk/postwalk http://clojuredocs.org/clojure_core/clojure.walk/postwalk can support some resilience with respects to structural changes if one can recognize what any data structure represents at any level based on its content (e.g. a :type key). -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: 101 on simple and flexible graph query/update?
Is there a way to do something similar with a more general definition of a path? The lens library Fresnel (https://github.com/ckirkendall/fresnel) might be worth a look - it abstracts get-in/assoc-in into lenses which can store/retrieve state from complex structures (and can be composed). -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: 101 on simple and flexible graph query/update?
*@*Thomas Heller: It's a bit cleaner indeed. That way, a 'join-like'
operation does not need to know about the structure. But the three
problematics from my post are still unsolved.
*@Francois Rey:* I believe walk/prewalk/postwalk (and zipper?) might be the
building blocks for implementing query/update based on a more flexible path
definition. But by themselves, they are a bit low level. They indeed are
beautiful and useful abstractions but I think another abstraction, even
higher, could be built.
*@Daniel Neal:* I will look into it. I am new to lenses as I said. I don't
know if a lens can target multiple structures at once (first question).
(def s { :key1 [{ :key2 { :key3 val } }]})
(update-in s [:key1 :all :key2 :key3] inc)
The principle might work with a custom lens but I guess it is likely the
implementation for the traversal must be changed in order to handle a
sequence at each step. But at least I have a place where I can start from.
Bertrand
Le jeudi 10 juillet 2014 13:26:57 UTC+2, Daniel Neal a écrit :
Is there a way to do something similar with a more general definition of
a path?
The lens library Fresnel (https://github.com/ckirkendall/fresnel) might
be worth a look - it abstracts
get-in/assoc-in into lenses which can store/retrieve state from complex
structures (and can be composed).
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: 101 on simple and flexible graph query/update?
Also of interest is datomic datalog on regular clojure data structures and records: https://gist.github.com/stuarthalloway/2645453 https://gist.github.com/stuarthalloway/3068749 -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: 101 on simple and flexible graph query/update?
Bertrand
I've been doing a lot of Ontology (OWL) with directed graphs (with cycles)
and DAGS. One application we went with the nested map structure however now
that I've been working on a normalized BPMN/XPDL utility
( https://github.com/FrankC01/tributary ) I'm becoming enamored with the in
XML format produced by xml-parse and then zipping it up. Coupled with
clojures 'functional' nature, zippers become your friend really quickly and
provide a most generic way of dealing with any graph with any number of
layers.
The next step will be a normalized DSL on top of the zippers and using
things like hickory-parse/data.zip selector concepts.
For the Ontology work we use datomic as the persist mechanism.
On Thursday, July 10, 2014 4:26:33 AM UTC-4, Bertrand Dechoux wrote:
Hi,
I have various general questions about how one can perform simple and
flexible graph query/update.
If we start by a structure being nested maps then I know about
get-in/update-in/assoc-in. The common point about these functions is that
they all take a path to where the data is in the superstructure and that
this path is only a sequence of keys, which may or not be symbols.
1) What happens when one layer of this superstructure is not a map but a
sequence? For the sake of the argument, we can keep the discussion simple
by only arguing about the vector case.
Typically instead of
{ :key1 { :key2 val }}
we now have
{ :key1 [{ :key2 val }]}
Of course, one can perform get/update/assoc using map/reduce. However, it
means that the code using this superstructure is taking a hit on
complexity/maintenance each time a non-map layer must be crossed. Of
course, the simplicity/beauty of the current implementation is that a path
can point to at most one target substructure/value. Is there a way to do
something similar with a more general definition of a path? One would
assume that it is something that might be handled by xml/html modification
libraries (think xpath or css selector) but this problem is more general
and not domain specific. Is there a clean library that handle that case
without being tied to html/xml?
2) How to deal with more general superstructure like a graph?
One could build an immutable graph but I don't believe it is something
that can be done by nesting immutable maps and vectors. One solution to
deal with 'entity'--'entity' relationships is for one of the member to own
only an identifier to get the other member from another reference structure.
(from the basic OM tutorial:)
{:people
[{:type :student :first Ben :last Bitdiddle}
{:type :professor :first Gerald :last Sussman :classes [:6001
:6946]}]
:classes
{:6001 The Structure and Interpretation of Computer Programs
:6946 The Structure and Interpretation of Classical Mechanics
:1806 Linear Algebra}}
It is, of course, again, manually possible to get/update/assoc using
map/reduce. But then, again, this is the same problematic as with the
first question : each time a relation needs to be crossed, the code using
this superstructure is taking a hit on complexity/maintenance. How do you
usually deal with this problematic?
For both question 1) and 2), a more appropriate data structure might be
the answer like a graph/semantic-like store (Datomic or something else).
The questions are about intermediary solutions which would be less heavier.
3) How does Demeter lives with graph traversal?
This law is often heard in the OOP world but it is a bit more general than
that. When a long path on a superstructure is specified then if one
intermediary layer is introduced later, all hardcoded paths will be broken
ie in multiple locations in the code base. One would like to store local
structure knowledge in a single place. How do you usually deal with this
problematic?
I have yet to take a serious look at lenses and their composition, they
are probably an element of answer, but they are more often seen in more
statically typed langage.
Thanks for any feedback.
Bertrand
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
