Re: [Neo4j] Cats and Dogs, living together

[Javier de la Rosa]

On Tue, Dec 7, 2010 at 10:39, Peter Neubauer
 wrote:
> There is a strong desire for having some graph visualization in the
> Neo4j Admin console, so - if you think it is interesting, I think
> there might be a strong case for the projects working together on the
> visualization component. I don't have the timeframe laid out yet but
> Neo Technology can dedicate resources to it early next year.
>
> Would that make sense to you?

Of course yes. We're not very familiar using Java, but I think it's
not needed, because the most of the code in the web admin console
would be Javascript. The only thing to decide is the format of data to
interchange between the server and the browser to paint the graph and
the properties.

Anyways, we will keep working on Sylva and its visual component,
although in a slower way than we would like.
For whatever, I'm always available at my e-mail.

>
> Cheers,
>
> /peter neubauer
>
> GTalk:      neubauer.peter
> Skype       peter.neubauer
> Phone       +46 704 106975
> LinkedIn   http://www.linkedin.com/in/neubauer
> Twitter      http://twitter.com/peterneubauer
>
> http://www.neo4j.org               - Your high performance graph database.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>
>
>
> On Tue, Dec 7, 2010 at 3:44 PM, Javier de la Rosa  wrote:
>> On Mon, Dec 6, 2010 at 16:57, Peter Neubauer
>>  wrote:
>>> Very very cool Javier!
>>
>> Thank you :)
>>
>>> Is this built using the Neo4j Python bindings or pure REST? Also, is
>>> there a public website available to refer to?
>>
>> By now we are using only REST, but the performance is not what we
>> expected when we process large nodes returned by a traversal (so much
>> HTTP requests). So we are now evaluating whether using de Python
>> binding or building a Java socket server and a Python socket client
>> could be better. When you make a traversal or use the indices, the
>> REST API returns the URLs of each of the nodes returned, so we need
>> make one HTTP request more per node. It would be great if we could
>> send an optional param to make the server returned all properties.
>>
>>> Another question - regarding visualisation, what was your experience
>>> of the best performing lib for JavaScript out there regarding large
>>> amount of nodes and relationships to render, and adaptability for UI?
>>> Currently, it seems there is
>>>
>>> - TheJIT
>>> - Processing.js
>>> - Graphdracula
>>
>> TheJIT was our first approach, but with large datasets the behaviour
>> is not very fast. Besides, the interaction ways are a bit limited and
>> hard to expand.
>> Processing.js is, with no doubt, the most promising solution. We were
>> happy using Porcessing.js, but you need build all you need to
>> represent graphs, nodes and edges. It's very low level programation
>> and by now the browsers can't with it, therefore we used a mixed
>> version between Processing.js in the browser and NetworkX in the
>> server side for some calculations of layout and etc.
>> Graphdracula was an inspiration for us. It's very beautiful, but it's
>> also very incomplete yet. I guess in the near future will be very
>> useful, but we need total control of all happens in the UI. So, now we
>> are using Räphael, the core library of Graphdracula, and we are
>> implementing several layout algorithms in Javascript and some ways to
>> interact with nodes to expand the graph by browsing.
>>
>> I hope to setup a Sylva test site soon, then I will e-mail to this
>> list, if that's alright with you.
>>
>> Best regards.
>>
>>>
>>>
>>> Cheers,
>>>
>>> /peter neubauer
>>>
>>> GTalk:      neubauer.peter
>>> Skype       peter.neubauer
>>> Phone       +46 704 106975
>>> LinkedIn   http://www.linkedin.com/in/neubauer
>>> Twitter      http://twitter.com/peterneubauer
>>>
>>> http://www.neo4j.org               - Your high performance graph database.
>>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>>>
>>>
>>>
>>> On Thu, Dec 2, 2010 at 6:03 PM, Javier de la Rosa  wrote:
 Hi, everybody,

 I work in a lab at University of Western Ontario, with humanists
 people. The needs of the humanist research make useless the current
 SQL databases because it's hard to change the schemas or build queries
 with several JOINs. So, we are developing a system which mixes a
 relational database and Neo4j.

 Sylva [1], as it's called (previously Graphgamel), stores all data in
 the Neo4j database as a graph. On the other hand, de multimedia files
 (image, video and audio files) are stores using the relational one
 (over Django). Besides, the relational part allows the definition of
 lazy and dynamic schemas, very usefull to model the world from the
 humanistic point of view. The users can create nodes and relationships
 but according to a certain kind of integrity defined in the schema.

 Sylva also has a very early version of visualization through Raphäel
 and Processing.js.
 Her

Re: [Neo4j] Cats and Dogs, living together

[Peter Neubauer]

Javier,
thanks for the feedback!

There is a strong desire for having some graph visualization in the
Neo4j Admin console, so - if you think it is interesting, I think
there might be a strong case for the projects working together on the
visualization component. I don't have the timeframe laid out yet but
Neo Technology can dedicate resources to it early next year.

Would that make sense to you?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Tue, Dec 7, 2010 at 3:44 PM, Javier de la Rosa  wrote:
> On Mon, Dec 6, 2010 at 16:57, Peter Neubauer
>  wrote:
>> Very very cool Javier!
>
> Thank you :)
>
>> Is this built using the Neo4j Python bindings or pure REST? Also, is
>> there a public website available to refer to?
>
> By now we are using only REST, but the performance is not what we
> expected when we process large nodes returned by a traversal (so much
> HTTP requests). So we are now evaluating whether using de Python
> binding or building a Java socket server and a Python socket client
> could be better. When you make a traversal or use the indices, the
> REST API returns the URLs of each of the nodes returned, so we need
> make one HTTP request more per node. It would be great if we could
> send an optional param to make the server returned all properties.
>
>> Another question - regarding visualisation, what was your experience
>> of the best performing lib for JavaScript out there regarding large
>> amount of nodes and relationships to render, and adaptability for UI?
>> Currently, it seems there is
>>
>> - TheJIT
>> - Processing.js
>> - Graphdracula
>
> TheJIT was our first approach, but with large datasets the behaviour
> is not very fast. Besides, the interaction ways are a bit limited and
> hard to expand.
> Processing.js is, with no doubt, the most promising solution. We were
> happy using Porcessing.js, but you need build all you need to
> represent graphs, nodes and edges. It's very low level programation
> and by now the browsers can't with it, therefore we used a mixed
> version between Processing.js in the browser and NetworkX in the
> server side for some calculations of layout and etc.
> Graphdracula was an inspiration for us. It's very beautiful, but it's
> also very incomplete yet. I guess in the near future will be very
> useful, but we need total control of all happens in the UI. So, now we
> are using Räphael, the core library of Graphdracula, and we are
> implementing several layout algorithms in Javascript and some ways to
> interact with nodes to expand the graph by browsing.
>
> I hope to setup a Sylva test site soon, then I will e-mail to this
> list, if that's alright with you.
>
> Best regards.
>
>>
>>
>> Cheers,
>>
>> /peter neubauer
>>
>> GTalk:      neubauer.peter
>> Skype       peter.neubauer
>> Phone       +46 704 106975
>> LinkedIn   http://www.linkedin.com/in/neubauer
>> Twitter      http://twitter.com/peterneubauer
>>
>> http://www.neo4j.org               - Your high performance graph database.
>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>>
>>
>>
>> On Thu, Dec 2, 2010 at 6:03 PM, Javier de la Rosa  wrote:
>>> Hi, everybody,
>>>
>>> I work in a lab at University of Western Ontario, with humanists
>>> people. The needs of the humanist research make useless the current
>>> SQL databases because it's hard to change the schemas or build queries
>>> with several JOINs. So, we are developing a system which mixes a
>>> relational database and Neo4j.
>>>
>>> Sylva [1], as it's called (previously Graphgamel), stores all data in
>>> the Neo4j database as a graph. On the other hand, de multimedia files
>>> (image, video and audio files) are stores using the relational one
>>> (over Django). Besides, the relational part allows the definition of
>>> lazy and dynamic schemas, very usefull to model the world from the
>>> humanistic point of view. The users can create nodes and relationships
>>> but according to a certain kind of integrity defined in the schema.
>>>
>>> Sylva also has a very early version of visualization through Raphäel
>>> and Processing.js.
>>> Here you can see some screenshots [2, 3, 4] and a video demo [5]. Our
>>> goal is to adapt the django-qbe project [6] to our schema tool in
>>> order to produce Gremlin queries in a visual way.
>>>
>>> But by now we are using the Neo4 REST component but it's not very fast
>>> and it has some limitations.
>>>
>>> It's an alpha version, but it goes without saying :-)
>>>
>>> [1] https://github.com/escalant3/graphgamel
>>> [2] http://dl.dropbox.com/u/2630535/sylva.png
>>> [3] http://dl.dropbox.com/u/2630535/plexigraph.png
>>> [4] http://dl.dropbox.com/u/2630535/grafo.png
>>> [5] http://www.youtub

Re: [Neo4j] Cats and Dogs, living together

[Javier de la Rosa]

On Mon, Dec 6, 2010 at 16:57, Peter Neubauer
 wrote:
> Very very cool Javier!

Thank you :)

> Is this built using the Neo4j Python bindings or pure REST? Also, is
> there a public website available to refer to?

By now we are using only REST, but the performance is not what we
expected when we process large nodes returned by a traversal (so much
HTTP requests). So we are now evaluating whether using de Python
binding or building a Java socket server and a Python socket client
could be better. When you make a traversal or use the indices, the
REST API returns the URLs of each of the nodes returned, so we need
make one HTTP request more per node. It would be great if we could
send an optional param to make the server returned all properties.

> Another question - regarding visualisation, what was your experience
> of the best performing lib for JavaScript out there regarding large
> amount of nodes and relationships to render, and adaptability for UI?
> Currently, it seems there is
>
> - TheJIT
> - Processing.js
> - Graphdracula

TheJIT was our first approach, but with large datasets the behaviour
is not very fast. Besides, the interaction ways are a bit limited and
hard to expand.
Processing.js is, with no doubt, the most promising solution. We were
happy using Porcessing.js, but you need build all you need to
represent graphs, nodes and edges. It's very low level programation
and by now the browsers can't with it, therefore we used a mixed
version between Processing.js in the browser and NetworkX in the
server side for some calculations of layout and etc.
Graphdracula was an inspiration for us. It's very beautiful, but it's
also very incomplete yet. I guess in the near future will be very
useful, but we need total control of all happens in the UI. So, now we
are using Räphael, the core library of Graphdracula, and we are
implementing several layout algorithms in Javascript and some ways to
interact with nodes to expand the graph by browsing.

I hope to setup a Sylva test site soon, then I will e-mail to this
list, if that's alright with you.

Best regards.

>
>
> Cheers,
>
> /peter neubauer
>
> GTalk:      neubauer.peter
> Skype       peter.neubauer
> Phone       +46 704 106975
> LinkedIn   http://www.linkedin.com/in/neubauer
> Twitter      http://twitter.com/peterneubauer
>
> http://www.neo4j.org               - Your high performance graph database.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>
>
>
> On Thu, Dec 2, 2010 at 6:03 PM, Javier de la Rosa  wrote:
>> Hi, everybody,
>>
>> I work in a lab at University of Western Ontario, with humanists
>> people. The needs of the humanist research make useless the current
>> SQL databases because it's hard to change the schemas or build queries
>> with several JOINs. So, we are developing a system which mixes a
>> relational database and Neo4j.
>>
>> Sylva [1], as it's called (previously Graphgamel), stores all data in
>> the Neo4j database as a graph. On the other hand, de multimedia files
>> (image, video and audio files) are stores using the relational one
>> (over Django). Besides, the relational part allows the definition of
>> lazy and dynamic schemas, very usefull to model the world from the
>> humanistic point of view. The users can create nodes and relationships
>> but according to a certain kind of integrity defined in the schema.
>>
>> Sylva also has a very early version of visualization through Raphäel
>> and Processing.js.
>> Here you can see some screenshots [2, 3, 4] and a video demo [5]. Our
>> goal is to adapt the django-qbe project [6] to our schema tool in
>> order to produce Gremlin queries in a visual way.
>>
>> But by now we are using the Neo4 REST component but it's not very fast
>> and it has some limitations.
>>
>> It's an alpha version, but it goes without saying :-)
>>
>> [1] https://github.com/escalant3/graphgamel
>> [2] http://dl.dropbox.com/u/2630535/sylva.png
>> [3] http://dl.dropbox.com/u/2630535/plexigraph.png
>> [4] http://dl.dropbox.com/u/2630535/grafo.png
>> [5] http://www.youtube.com/watch?v=r04eV7vghfs (sorry, not subtitles
>> or audio yet)
>> [6] http://versae.github.com/qbe/
>>
>> On Wed, Dec 1, 2010 at 12:52, Andreas Kollegger
>>  wrote:
>>> Would anybody be willing to share experiences with trying to introduce 
>>> Neo4j into a system with another relational (or other NoSQL) database?
>>>
>>> We're starting to think about best practices for integration:
>>> * Hybrid data-modeling: what goes where?
>>> * XA transactions
>>> * message queues for data distribution
>>> * data migration strategies
>>>
>>> Any problems or feature-requests related to living in a 
>>> multi-storage-platform world are welcome.
>>>
>>> Cheers,
>>> Andreas
>>>
>>>
>>> ___
>>> Neo4j mailing list
>>> User@lists.neo4j.org
>>> https://lists.neo4j.org/mailman/listinfo/user
>>>
>>
>>
>>
>> --
>> Javier de la Rosa
>> http://versae.es
>> __

Re: [Neo4j] Cats and Dogs, living together

[Peter Neubauer]

Very very cool Javier!

Is this built using the Neo4j Python bindings or pure REST? Also, is
there a public website available to refer to?

Another question - regarding visualisation, what was your experience
of the best performing lib for JavaScript out there regarding large
amount of nodes and relationships to render, and adaptability for UI?
Currently, it seems there is

- TheJIT
- Processing.js
- Graphdracula


Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Thu, Dec 2, 2010 at 6:03 PM, Javier de la Rosa  wrote:
> Hi, everybody,
>
> I work in a lab at University of Western Ontario, with humanists
> people. The needs of the humanist research make useless the current
> SQL databases because it's hard to change the schemas or build queries
> with several JOINs. So, we are developing a system which mixes a
> relational database and Neo4j.
>
> Sylva [1], as it's called (previously Graphgamel), stores all data in
> the Neo4j database as a graph. On the other hand, de multimedia files
> (image, video and audio files) are stores using the relational one
> (over Django). Besides, the relational part allows the definition of
> lazy and dynamic schemas, very usefull to model the world from the
> humanistic point of view. The users can create nodes and relationships
> but according to a certain kind of integrity defined in the schema.
>
> Sylva also has a very early version of visualization through Raphäel
> and Processing.js.
> Here you can see some screenshots [2, 3, 4] and a video demo [5]. Our
> goal is to adapt the django-qbe project [6] to our schema tool in
> order to produce Gremlin queries in a visual way.
>
> But by now we are using the Neo4 REST component but it's not very fast
> and it has some limitations.
>
> It's an alpha version, but it goes without saying :-)
>
> [1] https://github.com/escalant3/graphgamel
> [2] http://dl.dropbox.com/u/2630535/sylva.png
> [3] http://dl.dropbox.com/u/2630535/plexigraph.png
> [4] http://dl.dropbox.com/u/2630535/grafo.png
> [5] http://www.youtube.com/watch?v=r04eV7vghfs (sorry, not subtitles
> or audio yet)
> [6] http://versae.github.com/qbe/
>
> On Wed, Dec 1, 2010 at 12:52, Andreas Kollegger
>  wrote:
>> Would anybody be willing to share experiences with trying to introduce Neo4j 
>> into a system with another relational (or other NoSQL) database?
>>
>> We're starting to think about best practices for integration:
>> * Hybrid data-modeling: what goes where?
>> * XA transactions
>> * message queues for data distribution
>> * data migration strategies
>>
>> Any problems or feature-requests related to living in a 
>> multi-storage-platform world are welcome.
>>
>> Cheers,
>> Andreas
>>
>>
>> ___
>> Neo4j mailing list
>> User@lists.neo4j.org
>> https://lists.neo4j.org/mailman/listinfo/user
>>
>
>
>
> --
> Javier de la Rosa
> http://versae.es
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Cats and Dogs, living together

[Javier de la Rosa]

Hi, everybody,

I work in a lab at University of Western Ontario, with humanists
people. The needs of the humanist research make useless the current
SQL databases because it's hard to change the schemas or build queries
with several JOINs. So, we are developing a system which mixes a
relational database and Neo4j.

Sylva [1], as it's called (previously Graphgamel), stores all data in
the Neo4j database as a graph. On the other hand, de multimedia files
(image, video and audio files) are stores using the relational one
(over Django). Besides, the relational part allows the definition of
lazy and dynamic schemas, very usefull to model the world from the
humanistic point of view. The users can create nodes and relationships
but according to a certain kind of integrity defined in the schema.

Sylva also has a very early version of visualization through Raphäel
and Processing.js.
Here you can see some screenshots [2, 3, 4] and a video demo [5]. Our
goal is to adapt the django-qbe project [6] to our schema tool in
order to produce Gremlin queries in a visual way.

But by now we are using the Neo4 REST component but it's not very fast
and it has some limitations.

It's an alpha version, but it goes without saying :-)

[1] https://github.com/escalant3/graphgamel
[2] http://dl.dropbox.com/u/2630535/sylva.png
[3] http://dl.dropbox.com/u/2630535/plexigraph.png
[4] http://dl.dropbox.com/u/2630535/grafo.png
[5] http://www.youtube.com/watch?v=r04eV7vghfs (sorry, not subtitles
or audio yet)
[6] http://versae.github.com/qbe/

On Wed, Dec 1, 2010 at 12:52, Andreas Kollegger
 wrote:
> Would anybody be willing to share experiences with trying to introduce Neo4j 
> into a system with another relational (or other NoSQL) database?
>
> We're starting to think about best practices for integration:
> * Hybrid data-modeling: what goes where?
> * XA transactions
> * message queues for data distribution
> * data migration strategies
>
> Any problems or feature-requests related to living in a 
> multi-storage-platform world are welcome.
>
> Cheers,
> Andreas
>
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
Javier de la Rosa
http://versae.es
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Cats and Dogs, living together

[Chris Gioran]

> I will now go and write a post explaining in more detail how to build,
> install and use the above.

And it is available at

http://digitalstain.blogspot.com/2010/12/towards-neo4j-connector-implementation.html

be gentle.

thank you for your time,
CG
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Cats and Dogs, living together

[Andres Taylor]

Kalin (and list),

My lab project for a few weeks now has been an alternative shell to Neo4j.

Coming from a DBA background, I felt that we are lacking tools to do
DBA-work - import/export data, tune indexes, browse the data in an easy way,
and make backups. So that's been the focus for my shell.

It hosts Rhino, so you write scripts in Javascript. This gives you a Turing
complete scripting environment, which I missed.

The API looks like the Java API, but has been simplified, to make REPL more
enjoyable (stuff like auto-commits). I've also built in some support to make
it easy to import CSV-files.

It's available 
hereif
anyone want to take a look. It's still just a lab project, so be
cautious
if you test drive it.

I've been doing this pretty much on my own, so I'd love feedback if there's
any to be had.

Cheers,

Andrés


On Thu, Dec 2, 2010 at 3:02 PM, Kalin Wilson Development <
d...@kalinwilson.com> wrote:

> Peter,
> Those are certainly viable alternatives (need to investigate further), but
> most involve learning another language/interface or working at the command
> line or within a script.
> The ubiquity of SQL access to a database is a big factor when considering
> migration. One of my teammates is much better with SQL than I am and his
> preferred language is Visual Basic. His role in supporting a database driven
> app has fallen to mostly data maintenance which he accomplishes either
> through the Mysql workbench using it's browsing capabilities and direct SQL,
> or quick and dirty utilities he can write in VB to manipulate the data.
> So as I consider implementing part of the system with Neo4j, I have to
> consider that I'll be removing access to and manipulation of that data from
> other teammates who prefer or have expertise with SQL and require that
> access to the Neo4j data be programmatic or via shell/command line which can
> be limiting in the view you have.
>
> The flexible, schema-less nature of Neo4j is a great feature. But at some
> point in order to be able to traverse and understand a graph, a user needs
> to understand how the graph has been built and it's semantics. With tools
> like Mysql workbench you can browse the DB metadata to discover or remember
> table/field names and types, relationships, etc. Not having such a browser
> to discover and visualize a Neo4j graph requires more rigor among a team to
> document and adhere to the network model. Rigor is good but not always
> followed up on. I guess you could use the shell to traverse a graph and dump
> all of the relationship types, etc, but could you dump all of the properties
> on nodes and relationships to get a view of the model? Even with that would
> you have enough information to begin writing traversals to extract or
> manipulate data? Would you understand the semantics? This sort of tool
> support and general access capability will be very useful to Neo4j when its
> available, IMO.
>
> Thanks for the dialog!
>
> Kalin
>
> On Dec 2, 2010, at 5:48 AM, Peter Neubauer wrote:
>
> > Kalin,
> > I think the different shell access possibilities are good candidates
> > for fast integration. You have at least
> >
> > Neo4j Shell, distributed with Neo4j downloads, see
> >
> http://wiki.neo4j.org/content/Getting_Started_with_Neo4j_Server#Enter_the_Shell
> >
> > JRuby scripts, see http://neo4j.rubyforge.org/ and
> > https://github.com/andreasronge/neo4j
> >
> > Gremlin, https://github.com/tinkerpop/gremlin/wiki/Getting-Started
> >
> > Python and others,
> >
> > many of these can be used from the command line to read and pipe data
> > into Neo4j. Would one of these be an option? Especially the Neo4j
> > Shell connects remotely (the others will when the RemoteGraphDB
> > component is fixed up) and should be able to do most of the usual
> > tasks...
> >
> > WDYT?
> >
> > Cheers,
> >
> > /peter neubauer
> >
> > GTalk:  neubauer.peter
> > Skype   peter.neubauer
> > Phone   +46 704 106975
> > LinkedIn   http://www.linkedin.com/in/neubauer
> > Twitter  http://twitter.com/peterneubauer
> >
> > http://www.neo4j.org   - Your high performance graph
> database.
> > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
> >
> >
> >
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Cats and Dogs, living together

[Kalin Wilson Development]

Peter,
Those are certainly viable alternatives (need to investigate further), but most 
involve learning another language/interface or working at the command line or 
within a script.
The ubiquity of SQL access to a database is a big factor when considering 
migration. One of my teammates is much better with SQL than I am and his 
preferred language is Visual Basic. His role in supporting a database driven 
app has fallen to mostly data maintenance which he accomplishes either through 
the Mysql workbench using it's browsing capabilities and direct SQL, or quick 
and dirty utilities he can write in VB to manipulate the data.
So as I consider implementing part of the system with Neo4j, I have to consider 
that I'll be removing access to and manipulation of that data from other 
teammates who prefer or have expertise with SQL and require that access to the 
Neo4j data be programmatic or via shell/command line which can be limiting in 
the view you have.

The flexible, schema-less nature of Neo4j is a great feature. But at some point 
in order to be able to traverse and understand a graph, a user needs to 
understand how the graph has been built and it's semantics. With tools like 
Mysql workbench you can browse the DB metadata to discover or remember 
table/field names and types, relationships, etc. Not having such a browser to 
discover and visualize a Neo4j graph requires more rigor among a team to 
document and adhere to the network model. Rigor is good but not always followed 
up on. I guess you could use the shell to traverse a graph and dump all of the 
relationship types, etc, but could you dump all of the properties on nodes and 
relationships to get a view of the model? Even with that would you have enough 
information to begin writing traversals to extract or manipulate data? Would 
you understand the semantics? This sort of tool support and general access 
capability will be very useful to Neo4j when its available, IMO.

Thanks for the dialog!

Kalin

On Dec 2, 2010, at 5:48 AM, Peter Neubauer wrote:

> Kalin,
> I think the different shell access possibilities are good candidates
> for fast integration. You have at least
> 
> Neo4j Shell, distributed with Neo4j downloads, see
> http://wiki.neo4j.org/content/Getting_Started_with_Neo4j_Server#Enter_the_Shell
> 
> JRuby scripts, see http://neo4j.rubyforge.org/ and
> https://github.com/andreasronge/neo4j
> 
> Gremlin, https://github.com/tinkerpop/gremlin/wiki/Getting-Started
> 
> Python and others,
> 
> many of these can be used from the command line to read and pipe data
> into Neo4j. Would one of these be an option? Especially the Neo4j
> Shell connects remotely (the others will when the RemoteGraphDB
> component is fixed up) and should be able to do most of the usual
> tasks...
> 
> WDYT?
> 
> Cheers,
> 
> /peter neubauer
> 
> GTalk:  neubauer.peter
> Skype   peter.neubauer
> Phone   +46 704 106975
> LinkedIn   http://www.linkedin.com/in/neubauer
> Twitter  http://twitter.com/peterneubauer
> 
> http://www.neo4j.org   - Your high performance graph database.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
> 
> 
> 

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Cats and Dogs, living together

[Chris Gioran]

> Would anybody be willing to share experiences with trying to introduce Neo4j 
> into a system with another relational (or other NoSQL) database?

Let me contribute my 2 cents here.

I have developed a rough demo that shows what my thoughts are for
integration of Neo4j instances in a managed environment. Using this
JCA 1.6 compliant connector module I have successfully

- Injected a ConnectionFactory (akin to a DataSource from JDBC) that
returns NeoConnections (similar to javax.sql.Connection).
- Performed operations on it (node creations/property settings) in
tandem with operations on a mySQL instance. That is, in the same EJB
method, with a container managed transaction. Both resources were
enlisted in the top level tx and were successfully committed or
rolledback (when I violated constraints in either db, for instance),
in proper 2PC manner.

So far, based on my experience working with JDBC, the feeling is the
same, as far as usability goes. The GraphDatabaseService interface is
not yet completely available.

The code is available at the repository:

The transaction manager to use (based on previous work):
https://svn.neo4j.org/laboratory/users/cgioran/ContainerProvidedTxManager/

The connector module:
https://svn.neo4j.org/laboratory/users/cgioran/NeoJCA/

The modified kernel (nothing major, just exposed retrieval of
XAResources and enforced the new txm)
https://svn.neo4j.org/laboratory/users/cgioran/neo4j-kernel-jta/

The idea is that you build/install all three above, add the jta kernel
and the txm service in your ear's libraries and the connector module
as an ear connector module (in application.xml). Create a pool and
bind it to the JNDI and retrieve it from there in your EJB/Servlet.

DISCLAIMER: The above code is a toy and completely temporary. I have
not verified its operation in any significant way, I do not even know
if (in fact I doubt that) the pooling mechanism works properly. Do not
use it in any way that might be remotely important.

The connector will work only in glassfish. Other app servers (JonAS
and Geronimo for certain) require additional descriptor files apart
from ra.xml If you want to deploy in another container, please consult
its manual on the proper procedure.

I will now go and write a post explaining in more detail how to build,
install and use the above. If, in the meantime, someone feels
adventurous and decides to look at the code, feedback is always
welcome. The important thing for me is not the correctness of the code
- there is not much there anyway. What I am interested in is its
dynamics and the possible interactions in an app server.

cheers,
CG
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Cats and Dogs, living together

[Peter Neubauer]

Kalin,
I think the different shell access possibilities are good candidates
for fast integration. You have at least

Neo4j Shell, distributed with Neo4j downloads, see
http://wiki.neo4j.org/content/Getting_Started_with_Neo4j_Server#Enter_the_Shell

JRuby scripts, see http://neo4j.rubyforge.org/ and
https://github.com/andreasronge/neo4j

Gremlin, https://github.com/tinkerpop/gremlin/wiki/Getting-Started

Python and others,

many of these can be used from the command line to read and pipe data
into Neo4j. Would one of these be an option? Especially the Neo4j
Shell connects remotely (the others will when the RemoteGraphDB
component is fixed up) and should be able to do most of the usual
tasks...

WDYT?

Cheers,

/peter neubauer

GTalk:      neubauer.peter
Skype       peter.neubauer
Phone       +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter      http://twitter.com/peterneubauer

http://www.neo4j.org               - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Thu, Dec 2, 2010 at 8:12 AM, Kalin Wilson Development
 wrote:
> I'm currently prototyping an application that will use both a RDBMS and Neo4j 
> in parallel. The RDBMS currently exists as the backing store for a web 
> application. While it's feasible to move all of the data into Neo4j, that's 
> not politically palatable right now. So I plan to use Neo4j to store 
> dependency relationships and associated data with references back to the 
> RDBMS. To minimize synchronization issues, I plan to only store entity type 
> and id information in nodes unless specific information is needed to control 
> traversals.
> Right now most of the data access is via JPA using hibernate through a pretty 
> good DAO-domain abstraction. I plan to extend the domain model to include 
> neo4j nodes and relationships along the lines of your examples and extend the 
> DAOs to include management of the embedded Neo4j instance.
> My prototype is a Spring app and currently uses aspect oriented transaction 
> management, but in order to manage the transactions on the two databases, 
> I'll probably have to handle transactions programmatically, probably wrapping 
> one in the other. It would be nice to combine the transaction management 
> somehow.
>
> In terms of data migration, although I'm trying to minimize migration right 
> now, I have thought about how I might migrate if I were to use Neo4j 
> exclusively. I'm still not sure what the best approaches are to properties vs 
> nodes for entity attributes, indexes (lucene) vs type subnodes, etc.
>
> One of the hurdles to migrating to Neo4j is the lack of tool support, 
> particularly for general access. This has been acknowledged by Emil and 
> others, so I'm not complaining. But with our current app, if we need to 
> import data or fix general data problems, we can use a SQL workbench to 
> directly access the database rather than write a special capability in the 
> app. That's not always the best approach but it does allow other developers 
> more familiar with other technologies the ability to access the DB with their 
> tools/language due to the prevalence of SQL. With Neo4j it seems that 
> programmatic access will be required. Granted, I haven't explored the REST 
> server, SPARQL support, or the shell very much. They may offer more 
> generalized access.
>
> I'm interested to hear how others are approaching the polyglot-persistence 
> task. As I move forward, I'll share what I learn or have problems with.
>
> Thanks,
>  Kalin
>
> On Dec 1, 2010, at 10:52 AM, Andreas Kollegger wrote:
>
>> Would anybody be willing to share experiences with trying to introduce Neo4j 
>> into a system with another relational (or other NoSQL) database?
>>
>> We're starting to think about best practices for integration:
>> * Hybrid data-modeling: what goes where?
>> * XA transactions
>> * message queues for data distribution
>> * data migration strategies
>>
>> Any problems or feature-requests related to living in a 
>> multi-storage-platform world are welcome.
>>
>> Cheers,
>> Andreas
>>
>>
>> ___
>> Neo4j mailing list
>> User@lists.neo4j.org
>> https://lists.neo4j.org/mailman/listinfo/user
>>
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Cats and Dogs, living together

[Kalin Wilson Development]

I'm currently prototyping an application that will use both a RDBMS and Neo4j 
in parallel. The RDBMS currently exists as the backing store for a web 
application. While it's feasible to move all of the data into Neo4j, that's not 
politically palatable right now. So I plan to use Neo4j to store dependency 
relationships and associated data with references back to the RDBMS. To 
minimize synchronization issues, I plan to only store entity type and id 
information in nodes unless specific information is needed to control 
traversals.
Right now most of the data access is via JPA using hibernate through a pretty 
good DAO-domain abstraction. I plan to extend the domain model to include neo4j 
nodes and relationships along the lines of your examples and extend the DAOs to 
include management of the embedded Neo4j instance. 
My prototype is a Spring app and currently uses aspect oriented transaction 
management, but in order to manage the transactions on the two databases, I'll 
probably have to handle transactions programmatically, probably wrapping one in 
the other. It would be nice to combine the transaction management somehow.

In terms of data migration, although I'm trying to minimize migration right 
now, I have thought about how I might migrate if I were to use Neo4j 
exclusively. I'm still not sure what the best approaches are to properties vs 
nodes for entity attributes, indexes (lucene) vs type subnodes, etc. 

One of the hurdles to migrating to Neo4j is the lack of tool support, 
particularly for general access. This has been acknowledged by Emil and others, 
so I'm not complaining. But with our current app, if we need to import data or 
fix general data problems, we can use a SQL workbench to directly access the 
database rather than write a special capability in the app. That's not always 
the best approach but it does allow other developers more familiar with other 
technologies the ability to access the DB with their tools/language due to the 
prevalence of SQL. With Neo4j it seems that programmatic access will be 
required. Granted, I haven't explored the REST server, SPARQL support, or the 
shell very much. They may offer more generalized access.

I'm interested to hear how others are approaching the polyglot-persistence 
task. As I move forward, I'll share what I learn or have problems with.

Thanks,
 Kalin

On Dec 1, 2010, at 10:52 AM, Andreas Kollegger wrote:

> Would anybody be willing to share experiences with trying to introduce Neo4j 
> into a system with another relational (or other NoSQL) database? 
> 
> We're starting to think about best practices for integration:
> * Hybrid data-modeling: what goes where?
> * XA transactions
> * message queues for data distribution
> * data migration strategies
> 
> Any problems or feature-requests related to living in a 
> multi-storage-platform world are welcome.
> 
> Cheers,
> Andreas
> 
> 
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
> 

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

[Neo4j] Cats and Dogs, living together

[Andreas Kollegger]

Would anybody be willing to share experiences with trying to introduce Neo4j 
into a system with another relational (or other NoSQL) database? 

We're starting to think about best practices for integration:
* Hybrid data-modeling: what goes where?
* XA transactions
* message queues for data distribution
* data migration strategies

Any problems or feature-requests related to living in a multi-storage-platform 
world are welcome.

Cheers,
Andreas


___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user