[Neo4j] Querying Neo4j

[Boban Erakovic]

Hi,
the reason of using neo4j is the main problem of my question. Just as Toni
Menzel said "Directly falling back to the index service does not really
embrace why you are a graph db at all." The question is, because my employee
want to perform some project, within what he (me) will demonstrate the use
of graph db. That is the problem, because there is no any particular
problem, I am aiming to resolve, and for that and that I need graph db.
However, one thing which bothering me is : Can I perform selection like in
relational database (so I have saved some data some time ago, and I want to
retrieve it again)?
In all neo4j examples, data is retrieved from the same part of the code
which perform saving earlier.
Is it at all possible to select data, like 10 days after saving?

Every example is like this :

GraphDatabaseService graphDb = new
EmbeddedGraphDatabase("poc_test");

Transaction tx = graphDb.beginTx();
try {
for(CSVObject object : csv){
Node trigramme = graphDb.createNode();
Node cp = graphDb.createNode();
Node description = graphDb.createNode();
Node category = graphDb.createNode();

Relationship ispartOf = trigramme.createRelationshipTo(cp,
RelTypes.IS_PART_OF);
Relationship isDescribedBy =
trigramme.createRelationshipTo(description, RelTypes.IS_DESCRIBED_BY);
Relationship isCategorizedIn =
description.createRelationshipTo(category, RelTypes.IS_CATEGORIZED_IN);

trigramme.setProperty("value", object.getTrigram());
cp.setProperty("value", object.getCp());
description.setProperty("value", object.getDescription());
category.setProperty("value", object.getCategory());

ispartOf.setProperty("relationship", "ispartOf ");
isDescribedBy.setProperty("relationship", "ispartOf");
isCategorizedIn.setProperty("relationship", "ispartOf");

Traverser friendsTraverser = description.traverse(
Traverser.Order.BREADTH_FIRST,
StopEvaluator.DEPTH_ONE,
ReturnableEvaluator.ALL_BUT_START_NODE,
RelTypes.IS_DESCRIBED_BY,
Direction.INCOMING );
// Traverse the node space and print out the result
//System.out.println( "Testing values : " );
for ( Node friend : friendsTraverser )
{

System.out.println( "At depth : "+
friendsTraverser.currentPosition().depth()+" with
value : "+
friend.getProperty( "value" ) );
}
}

tx.success();
}catch (Exception e){
e.printStackTrace();
}
finally {
tx.finish();
graphDb.shutdown();
}
  }

So Traverser is using the node previously created, in order to iterate over
database.
But, how can I perform (simple example), to save user's data when he
register, and after 10 days when he log in again, to retrieve that data? And
is this kind of concept, which I use in relational db, possible to apply to
graph database? I mean, if I cannot get saved data after some time, what is
the purpose of the graph db?
Hope you understood my problem and question, and thanks in advance to your
answers !
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Meta-model component: problem with renaming classes

[Niels Hoogeveen]

The modifications have been applied to trunk.
Niels

> Date: Mon, 5 Jul 2010 20:27:49 +0200
> From: matt...@neotechnology.com
> To: user@lists.neo4j.org
> Subject: Re: [Neo4j] Meta-model component: problem with renaming classes
> 
> 2010/7/5 Niels Hoogeveen 
> 
> >
> > If Mattias agrees, I suggest adding the methods:
> > MetaModelNamespace#rename(String oldName, String newName)
> > MetaModelNamespace#remove(String name, Boolean forced)
> > If forced is false, throw exception when relations exist to other
> > MetaModelObjects or in case of a MetaModelClass, has associated instances.
> > If forced is true, all relationships are removed together with the
> > MetaModelObject.
> > I am willing to make the changes, but want approval from the Neo4J team
> > before making these adjustments.
> > Niels
> >
> 
> Sure, I don't see why not. But shouldn't those be on MetaModelObject or some
> other class (not MetaModelNamespace)?
> 
> 
> > > From: j...@deepamehta.de
> > > Date: Mon, 5 Jul 2010 13:54:10 +0200
> > > To: user@lists.neo4j.org
> > > Subject: Re: [Neo4j] Meta-model component: problem with renaming classes
> > >
> > >
> > > On Jul 5, 2010, at 9:13, Niels Hoogeveen wrote:
> > >
> > > > MetaModelClasses are cached in a map, so while the property may be
> > renamed and reindexed, the map still holds the original name value. After a
> > restart of the application the new value should be applied. To make renaming
> > possible a method needs to be added to MetaModelNamespace.
> > > > Niels
> > >
> > > Thank you for the info!
> > > Restarting the application is not an option in dynamic environments,
> > where classes come and go (and change their names) at runtime.
> > > BTW: deleting a class is also an issue with the meta-model component.
> > >
> > > Is it possible for you to add a setName() method to MetaModelObject (or
> > elsewhere)?
> > >
> > > This would allow developers to use the meta-model component in dynamic
> > applications.
> > >
> > > Cheers
> > > Jörg
> > >
> > > ___
> > > Neo4j mailing list
> > > User@lists.neo4j.org
> > > https://lists.neo4j.org/mailman/listinfo/user
> >
> > _
> > Express yourself instantly with MSN Messenger! Download today it's FREE!
> > http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
> > ___
> > Neo4j mailing list
> > User@lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
> >
> 
> 
> 
> -- 
> Mattias Persson, [matt...@neotechnology.com]
> Hacker, Neo Technology
> www.neotechnology.com
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
  
_
New Windows 7: Simplify what you do everyday. Find the right PC for you.
http://windows.microsoft.com/shop
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Importing data

[Craig Taverner]

Since your data is sequential, it seems to me that the search for nodes with
the same visit-id or same visitor-id is limited to only recent entries from
the input log. For example, you say that the visit-id changes after
20minutes, so you only need to search back the last 20 minutes.

For extremely low memory usage, keep the visit nodes in a chain of NEXT
relationships, and you can search backwards when needed. Of course that is
not the best solution. I think for your case, just keep an in-memory cash of
recent visit-ids. If you only need to store a 20-minute window, and only of
the visit and visitor ids, that is a small memory cache compared to the
total 100k page views you are loading.

And the final option, use a lucene index on the visit and visitor nodes to
find them when you need to. This is best if you cannot rely on the
time-window in-memory cache idea.

On Mon, Jul 5, 2010 at 4:57 PM, Logo Bogo  wrote:

> Hi,
>
> I want to use neo4j to analyse apache logs. Each visitor is identified by a
> session ID, and each visit has an ID too - a new ID is assigned after 20
> minutes between page views. The graph consists of Visit, Visitor and Page
> nodes where a Visitor -> multiple Visit nodes, and a Visit -> multiple Page
> nodes. The Visit -> Page relationship has a property to indicate when in the
> visit the page was visited (i.e. 1 = first page visited, 2 = second page in
> the visit, etc).
>
> How would I best go about importing data into this graph? I'd use the batch
> inserter, but before I create a new Visitor or Visit I need to check whether
> a node exists already with the same ID. I've read that it's better to use
> the EmbeddedGraphDatabase, but I'm going to be inserting ~ 100K nodes 3
> times per day. In the past using MySQL in a similar way performance was
> abysmal, so I couldn't do take this approach.
>
> Will I be able to just use an EmbeddedGraphDatabase, or should I have a
> rethink? How is performance likely to be for these inserts?
>
> Thanks
> Tim
>
>
>
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] memory consumption

[Craig Taverner]

I believe the standard traverser keeps a cache of visited nodes (ids) to
prevent covering the same ground again, and that will grow the memory on
very deep traversals, but I would be surprised if it grew it as much as you
see. I think the neo4j developers will know better.

I would probably still go ahead and investigate the queries, and if they are
getting extremely deep, consider structuring the graph differently to ensure
shorter traversals.

On Mon, Jul 5, 2010 at 10:42 AM, Qiuyan Xu
wrote:

> Maybe it's the reason. Does query consume a lot of memory when the
> graph is large? When the folder of the graph is about 300M, 1G memory
> seems to be not enough for the program.
>
> Cheers,
> Qiuyan
>
> Quoting Craig Taverner :
>
> > You mentioned that your also do queries or searches as part of the
> process.
> > If the graph is growing in complexity, perhaps the queries are getting
> > slower?
> >
> > On Mon, Jul 5, 2010 at 10:16 AM, Qiuyan Xu
> > wrote:
> >
> >> I've just checked my code. It turns out that tx.finish() is already
> >> there and I use a normal inserter.
> >>
> >> In each program I try to insert 10 data and I restart the
> >> transaction every 500 insertions. The problem is, when I execute the
> >> program for about the 15th time, it runs slowly shortly after the
> >> program is started.
> >>
> >> Cheers,
> >> Qiuyan
> >>
> >>
> >> Quoting Tobias Ivarsson :
> >>
> >> > Just throwing this out there (since people have made mistakes with
> this
> >> > before):
> >> >
> >> > Remember that commit is spelled:
> >> > tx.success(); tx.finish();
> >> >
> >> > If you forget tx.finish(); the transaction will still be alive, and
> >> continue
> >> > to grow and eat more memory.
> >> >
> >> > Cheers,
> >> > Tobias
> >> >
> >> > On Sun, Jul 4, 2010 at 4:33 PM, Qiuyan Xu
> >> > wrote:
> >> >
> >> >> Thanks for the answer. I can't remember exactly which inserter I use
> >> >> currently. I will check it tomorrow. But I do commit and restart a
> new
> >> >> transaction after a fixed number of insertions.
> >> >>
> >> >> At the same time, I execute some queries on the database while I
> >> >> insert the data, since such insertions are conditional and always
> >> >> require some checks before the insertions are executed.
> >> >>
> >> >> cheers,
> >> >> Qiuyan
> >> >>
> >> >> Quoting Martin Neumann :
> >> >>
> >> >> > Do you use the Batchinserter or a normal Transaction?
> >> >> > When using a normal Transaction to insert huge amounts of data I
> >> always
> >> >> > submit and create a new transaction every X Items. This keeps the
> >> >> > transaction small and reduces the memory used.
> >> >> >
> >> >> > cheers Martin
> >> >> >
> >> >> > On Sun, Jul 4, 2010 at 4:13 PM, 
> >> wrote:
> >> >> >
> >> >> >> Hallo,
> >> >> >>
> >> >> >> I'm currently working with neo4j database and want to insert a
> bunch
> >> >> >> of data into it.
> >> >> >>
> >> >> >> At the very beginning the program works quite well. But as more
> data
> >> >> >> has been inserted into the database, the insertion runs more and
> more
> >> >> >> slowly and I noticed that the program consumes really a lot of
> >> memory.
> >> >> >> Even though I splitted the input file into small pieces so that
> each
> >> >> >> time the program tries only to insert a small part of data, the
> >> >> >> problem occurs. That means, as there exists already much data in
> the
> >> >> >> database, the program consumes a lot of memory as soon as it
> begins
> >> so
> >> >> >> that the insertion is so slow that it seems that it won't be able
> to
> >> >> >> finish.
> >> >> >>
> >> >> >> I wonder if there's some solutions to save the memory. Thanks in
> >> >> advance.
> >> >> >>
> >> >> >> Cheers,
> >> >> >> Qiuyan
> >> >> >>
> >> >> >> ___
> >> >> >> Neo4j mailing list
> >> >> >> User@lists.neo4j.org
> >> >> >> https://lists.neo4j.org/mailman/listinfo/user
> >> >> >>
> >> >> > ___
> >> >> > Neo4j mailing list
> >> >> > User@lists.neo4j.org
> >> >> > https://lists.neo4j.org/mailman/listinfo/user
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >> ___
> >> >> Neo4j mailing list
> >> >> User@lists.neo4j.org
> >> >> https://lists.neo4j.org/mailman/listinfo/user
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Tobias Ivarsson 
> >> > Hacker, Neo Technology
> >> > www.neotechnology.com
> >> > Cellphone: +46 706 534857
> >> > ___
> >> > Neo4j mailing list
> >> > User@lists.neo4j.org
> >> > https://lists.neo4j.org/mailman/listinfo/user
> >> >
> >> >
> >>
> >>
> >> ___
> >> Neo4j mailing list
> >> User@lists.neo4j.org
> >> https://lists.neo4j.org/mailman/listinfo/user
> >>
> > ___
> > Neo4j mailing list
> > User@lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
> >
> >
>
>
> _

Re: [Neo4j] Querying Neo4j

[Craig Taverner]

The way I view this point is that you query what you create. In other words,
you should create a graph structure to suite the kinds of queries you plan
to make. So if you are going to come back to the graph later on and have no
idea how to find a starting point for the traversal, then be sure to have
relationships from the reference node into some key point on your graph, so
you can traverse from the reference node.

In my opinion traversing from the reference node is a very good model, and
therefore building a graph linked to the reference node is also a good
model. In my work, I typically have projects composed of datasets composed
of complex graphs of data. So I generally have a tree structure rooted at
the reference node: ref->project(s)->dataset(s)->data-graph, etc. Then I can
always find what I want later on with a simple traversal, without needing
lucene and without having to know any node-id in advance.

Lucene is good for very flat structures, where you have a lot of similar
data and want to later on find parts of that data based on a key or
property. You can build an index tree yourself, of course, but few both too.

On Mon, Jul 5, 2010 at 8:42 PM, Toni Menzel  wrote:

> Hi Boban,
>
> aside from using an indexing service to grab certain information from
> neo4j (as Paddy mentioned correctly) you certainly should get yourself
> learning about traversal concepts.Traversing does not mean just
> iterating over "elements over a single node" but traversing the whole
> graph (if you want to) : hence the naming "traverse".
> So, from the way you ask about stuff in you mail, i think its worth
> understand that unique concept of graph database. Directly falling
> back to the index service does not really embrace why you are a graph
> db at all.
> As a starter, looking at the javadoc of Node.traverse or its neo4j
> 1.1.0-SNAPSHOT successor TraversalDescription and friends helps a lot
> already.
>
> If you cannot succeed that way, read for common misconceptions like that
> one [1]
> Also try to adopt some design best practices when creating the graph [2]
>
> cheers,
> Toni
> [1] http://wiki.neo4j.org/content/Neo_Mistakes#Unbalanced_graph
> [2] http://wiki.neo4j.org/content/Design_Guide
>
>
> On Mon, Jul 5, 2010 at 4:19 PM, Boban Erakovic 
> wrote:
> > Greetings, I have some misunderstanding about neo4j.
> > Once, when I store some nodes and relations into database, how can I
> query
> > it?
> > So I store information 5 days ago, and now I want to query? As much as I
> > saw, I am using :
> >
> > Traverser traverser = node.traverse();
> >
> > So I need a node to in order to iterate over database.
> > In every example, iteration over data is perform in the same code within
> > where we had create already some nodes and relation, and then just reuse
> > created node to communicate to database.
> > But how to perform querying if have NO any node. I just want to get some
> > data from database???
> > I tried getReferenceNode() method, of EmbeddedGraphDatabase class, but it
> is
> > not correct.
> > So, how can I just get data from database?
> > Thanks in advance!
> > ___
> > Neo4j mailing list
> > User@lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
> >
>
>
>
> --
> Toni Menzel
> Independent Software Developer
> Professional Profile: http://okidokiteam.com
> t...@okidokiteam.com
> http://www.ops4j.org - New Energy for OSS Communities - Open
> Participation Software.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Meta-model component: problem with renaming classes

[Niels Hoogeveen]

The caching for MetaModelObjects takes place in MetaModelNamespace, so renaming 
and removal should take place there.

> Date: Mon, 5 Jul 2010 20:27:49 +0200
> From: matt...@neotechnology.com
> To: user@lists.neo4j.org
> Subject: Re: [Neo4j] Meta-model component: problem with renaming classes
> 
> 2010/7/5 Niels Hoogeveen 
> 
> >
> > If Mattias agrees, I suggest adding the methods:
> > MetaModelNamespace#rename(String oldName, String newName)
> > MetaModelNamespace#remove(String name, Boolean forced)
> > If forced is false, throw exception when relations exist to other
> > MetaModelObjects or in case of a MetaModelClass, has associated instances.
> > If forced is true, all relationships are removed together with the
> > MetaModelObject.
> > I am willing to make the changes, but want approval from the Neo4J team
> > before making these adjustments.
> > Niels
> >
> 
> Sure, I don't see why not. But shouldn't those be on MetaModelObject or some
> other class (not MetaModelNamespace)?
> 
> 
> > > From: j...@deepamehta.de
> > > Date: Mon, 5 Jul 2010 13:54:10 +0200
> > > To: user@lists.neo4j.org
> > > Subject: Re: [Neo4j] Meta-model component: problem with renaming classes
> > >
> > >
> > > On Jul 5, 2010, at 9:13, Niels Hoogeveen wrote:
> > >
> > > > MetaModelClasses are cached in a map, so while the property may be
> > renamed and reindexed, the map still holds the original name value. After a
> > restart of the application the new value should be applied. To make renaming
> > possible a method needs to be added to MetaModelNamespace.
> > > > Niels
> > >
> > > Thank you for the info!
> > > Restarting the application is not an option in dynamic environments,
> > where classes come and go (and change their names) at runtime.
> > > BTW: deleting a class is also an issue with the meta-model component.
> > >
> > > Is it possible for you to add a setName() method to MetaModelObject (or
> > elsewhere)?
> > >
> > > This would allow developers to use the meta-model component in dynamic
> > applications.
> > >
> > > Cheers
> > > Jörg
> > >
> > > ___
> > > Neo4j mailing list
> > > User@lists.neo4j.org
> > > https://lists.neo4j.org/mailman/listinfo/user
> >
> > _
> > Express yourself instantly with MSN Messenger! Download today it's FREE!
> > http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
> > ___
> > Neo4j mailing list
> > User@lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
> >
> 
> 
> 
> -- 
> Mattias Persson, [matt...@neotechnology.com]
> Hacker, Neo Technology
> www.neotechnology.com
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
  
_
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Querying Neo4j

[Toni Menzel]

Hi Boban,

aside from using an indexing service to grab certain information from
neo4j (as Paddy mentioned correctly) you certainly should get yourself
learning about traversal concepts.Traversing does not mean just
iterating over "elements over a single node" but traversing the whole
graph (if you want to) : hence the naming "traverse".
So, from the way you ask about stuff in you mail, i think its worth
understand that unique concept of graph database. Directly falling
back to the index service does not really embrace why you are a graph
db at all.
As a starter, looking at the javadoc of Node.traverse or its neo4j
1.1.0-SNAPSHOT successor TraversalDescription and friends helps a lot
already.

If you cannot succeed that way, read for common misconceptions like that one [1]
Also try to adopt some design best practices when creating the graph [2]

cheers,
Toni
[1] http://wiki.neo4j.org/content/Neo_Mistakes#Unbalanced_graph
[2] http://wiki.neo4j.org/content/Design_Guide


On Mon, Jul 5, 2010 at 4:19 PM, Boban Erakovic  wrote:
> Greetings, I have some misunderstanding about neo4j.
> Once, when I store some nodes and relations into database, how can I query
> it?
> So I store information 5 days ago, and now I want to query? As much as I
> saw, I am using :
>
> Traverser traverser = node.traverse();
>
> So I need a node to in order to iterate over database.
> In every example, iteration over data is perform in the same code within
> where we had create already some nodes and relation, and then just reuse
> created node to communicate to database.
> But how to perform querying if have NO any node. I just want to get some
> data from database???
> I tried getReferenceNode() method, of EmbeddedGraphDatabase class, but it is
> not correct.
> So, how can I just get data from database?
> Thanks in advance!
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
Toni Menzel
Independent Software Developer
Professional Profile: http://okidokiteam.com
t...@okidokiteam.com
http://www.ops4j.org - New Energy for OSS Communities - Open
Participation Software.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Meta-model component: problem with renaming classes

[Mattias Persson]

2010/7/5 Niels Hoogeveen 

>
> If Mattias agrees, I suggest adding the methods:
> MetaModelNamespace#rename(String oldName, String newName)
> MetaModelNamespace#remove(String name, Boolean forced)
> If forced is false, throw exception when relations exist to other
> MetaModelObjects or in case of a MetaModelClass, has associated instances.
> If forced is true, all relationships are removed together with the
> MetaModelObject.
> I am willing to make the changes, but want approval from the Neo4J team
> before making these adjustments.
> Niels
>

Sure, I don't see why not. But shouldn't those be on MetaModelObject or some
other class (not MetaModelNamespace)?


> > From: j...@deepamehta.de
> > Date: Mon, 5 Jul 2010 13:54:10 +0200
> > To: user@lists.neo4j.org
> > Subject: Re: [Neo4j] Meta-model component: problem with renaming classes
> >
> >
> > On Jul 5, 2010, at 9:13, Niels Hoogeveen wrote:
> >
> > > MetaModelClasses are cached in a map, so while the property may be
> renamed and reindexed, the map still holds the original name value. After a
> restart of the application the new value should be applied. To make renaming
> possible a method needs to be added to MetaModelNamespace.
> > > Niels
> >
> > Thank you for the info!
> > Restarting the application is not an option in dynamic environments,
> where classes come and go (and change their names) at runtime.
> > BTW: deleting a class is also an issue with the meta-model component.
> >
> > Is it possible for you to add a setName() method to MetaModelObject (or
> elsewhere)?
> >
> > This would allow developers to use the meta-model component in dynamic
> applications.
> >
> > Cheers
> > Jörg
> >
> > ___
> > Neo4j mailing list
> > User@lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
>
> _
> Express yourself instantly with MSN Messenger! Download today it's FREE!
> http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Meta-model component: problem with renaming classes

[Jörg Richter]

Thank you!

Highly appreciating that :-)
I'm looking forward ...

Cheers
Jörg


On Jul 5, 2010, at 17:54, Niels Hoogeveen wrote:

> If Mattias agrees, I suggest adding the methods:
> MetaModelNamespace#rename(String oldName, String newName) 
> MetaModelNamespace#remove(String name, Boolean forced)
> If forced is false, throw exception when relations exist to other 
> MetaModelObjects or in case of a MetaModelClass, has associated instances. If 
> forced is true, all relationships are removed together with the 
> MetaModelObject.
> I am willing to make the changes, but want approval from the Neo4J team 
> before making these adjustments.
> Niels
>> From: j...@deepamehta.de
>> Date: Mon, 5 Jul 2010 13:54:10 +0200
>> To: user@lists.neo4j.org
>> Subject: Re: [Neo4j] Meta-model component: problem with renaming classes
>> 
>> 
>> On Jul 5, 2010, at 9:13, Niels Hoogeveen wrote:
>> 
>>> MetaModelClasses are cached in a map, so while the property may be renamed 
>>> and reindexed, the map still holds the original name value. After a restart 
>>> of the application the new value should be applied. To make renaming 
>>> possible a method needs to be added to MetaModelNamespace.
>>> Niels
>> 
>> Thank you for the info!
>> Restarting the application is not an option in dynamic environments, where 
>> classes come and go (and change their names) at runtime.
>> BTW: deleting a class is also an issue with the meta-model component.
>> 
>> Is it possible for you to add a setName() method to MetaModelObject (or 
>> elsewhere)?
>> 
>> This would allow developers to use the meta-model component in dynamic 
>> applications.
>> 
>> Cheers
>> Jörg

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Meta-model component: problem with renaming classes

[Niels Hoogeveen]

If Mattias agrees, I suggest adding the methods:
MetaModelNamespace#rename(String oldName, String newName) 
MetaModelNamespace#remove(String name, Boolean forced)
If forced is false, throw exception when relations exist to other 
MetaModelObjects or in case of a MetaModelClass, has associated instances. If 
forced is true, all relationships are removed together with the MetaModelObject.
I am willing to make the changes, but want approval from the Neo4J team before 
making these adjustments.
Niels
> From: j...@deepamehta.de
> Date: Mon, 5 Jul 2010 13:54:10 +0200
> To: user@lists.neo4j.org
> Subject: Re: [Neo4j] Meta-model component: problem with renaming classes
> 
> 
> On Jul 5, 2010, at 9:13, Niels Hoogeveen wrote:
> 
> > MetaModelClasses are cached in a map, so while the property may be renamed 
> > and reindexed, the map still holds the original name value. After a restart 
> > of the application the new value should be applied. To make renaming 
> > possible a method needs to be added to MetaModelNamespace.
> > Niels
> 
> Thank you for the info!
> Restarting the application is not an option in dynamic environments, where 
> classes come and go (and change their names) at runtime.
> BTW: deleting a class is also an issue with the meta-model component.
> 
> Is it possible for you to add a setName() method to MetaModelObject (or 
> elsewhere)?
> 
> This would allow developers to use the meta-model component in dynamic 
> applications.
> 
> Cheers
> Jörg
> 
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
  
_
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] neo4j question

[Alex Averbuch]

Hi Paddy,
Just for your information, I've been running a lot of shortest path
operations on a graph of similar size to yours (approx 1 million Nodes & 5
Million Relationships).
I'm using an Amazon instance with 32GB of RAM, but am only using a heap size
of 8GB (as other people are sharing the same instance).
I choose the start and end Nodes randomly, so occasionally the AStar
algorithm actually exhausts the entire graph during a search.

What I've found is that the first 1-3 search operations may take up to a
minute in the worse case (when entire graph is exhausted) but after this I
never see run times of over 10 seconds (entire graph exhausted), and in most
cases it completes in between 0-4 seconds

Cheers,
Alex

On Mon, Jul 5, 2010 at 5:05 PM, Peter Neubauer <
peter.neuba...@neotechnology.com> wrote:

> Paddy,
> posting on the user list - yes, I think you should add RAM to hold
> your graph in memory, and then maybe write some routing to warm up the
> caches. That is, read all nodes up into RAM before you do the first
> search, or e.g. perform some interesting searches to load all
> interesting data.
>
> Would that be possible? What hardware config are you running on right now?
>
> Cheers,
>
> /peter neubauer
>
> COO and Sales, Neo Technology
>
> GTalk:  neubauer.peter
> Skype   peter.neubauer
> Phone   +46 704 106975
> LinkedIn   http://www.linkedin.com/in/neubauer
> Twitter  http://twitter.com/peterneubauer
>
> http://www.neo4j.org   - Your high performance graph database.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>
>
>
> On Sun, Jul 4, 2010 at 4:34 AM, Paddy  wrote:
> > hi peter,
> > Hope all is well, I have modeled the bus network with timetables and
> stops
> > connecting by walking distance.
> > There are approx 10 million and 1 million nodes. the graph is approx 1gb
> > Running on my local machine, to find the best route , it takes 5 seconds
> to
> > do approx 100 lucene query searches and add approx 100 relationships to
> the
> > graph
> > it can take from 3 - 10 seconds to find a shortest path , with the a star
> > algorithm.
> >
> > what steps could i take to speed up the app?  More RAM? Run in the cloud?
> > Thanks
> > Paddy
> >
> >
> >
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Querying Neo4j

[Paddy]

here is an example i used too test the LuceneFulltextQueryIndexService
search



package test;

import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.Transaction;
import org.neo4j.index.IndexHits;
import org.neo4j.index.IndexService;
import org.neo4j.index.lucene.LuceneFulltextIndexService;
import org.neo4j.index.lucene.LuceneFulltextQueryIndexService;
import org.neo4j.index.lucene.LuceneIndexService;

import org.neo4j.kernel.EmbeddedGraphDatabase;

import datastructure.neo4j.StopTime;

public class LuceneTest {




public static void main( final String[] args ){
GraphDatabaseService graphDb= new
EmbeddedGraphDatabase("target/neo4j-db");

Transaction tx = graphDb.beginTx();
try
{

IndexService index= new
LuceneFulltextQueryIndexService(graphDb);
Node andy = graphDb.createNode();
Node larry = graphDb.createNode();

andy.setProperty( "name", "Andy Wachowski" );
andy.setProperty( "title", "Director" );
larry.setProperty( "name", "Larry Wachowski" );
larry.setProperty( "title", "Director" );
index.index( andy, "name", andy.getProperty( "name" ) );
index.index( andy, "title", andy.getProperty( "title" ) );
index.index( larry, "name", larry.getProperty( "name" ) );
index.index( larry, "title", larry.getProperty( "title" ) );


IndexHits nodes =index.getNodes( "name", "wachow* andy"
); // --> andy and larry
System.out.println("1: nodes size " + nodes.size());
nodes = index.getNodes( "name", "Andy" ); // --> andy
System.out.println("2: nodes size " + nodes.size());
nodes = index.getNodes( "name", "andy" ); // --> andy
System.out.println("3: nodes size " + nodes.size());
nodes = index.getNodes( "name", "wachowski" ); // --> andy and
larry
System.out.println("4: nodes size " + nodes.size());
nodes = index.getNodes( "name", "+wachow* +larry" ); // -->
larry
System.out.println("5: nodes size " + nodes.size());
nodes = index.getNodes( "name", "andy AND larry" ); // -->
System.out.println("6: nodes size " + nodes.size());
nodes = index.getNodes( "name", "andy OR larry" ); // --> andy
and larry
System.out.println("7: nodes size " + nodes.size());
nodes = index.getNodes( "name", "Wachowski AND larry" ); // -->
larry
System.out.println("8: nodes size " + nodes.size());

tx.success();

}
finally
{
tx.finish();
graphDb.shutdown();

}
}

}

output:
1: nodes size 2
2: nodes size 1
3: nodes size 1
4: nodes size 2
5: nodes size 1
6: nodes size 0
7: nodes size 2
8: nodes size 1


On Mon, Jul 5, 2010 at 7:19 AM, Boban Erakovic wrote:

> Greetings, I have some misunderstanding about neo4j.
> Once, when I store some nodes and relations into database, how can I query
> it?
> So I store information 5 days ago, and now I want to query? As much as I
> saw, I am using :
>
> Traverser traverser = node.traverse();
>
> So I need a node to in order to iterate over database.
> In every example, iteration over data is perform in the same code within
> where we had create already some nodes and relation, and then just reuse
> created node to communicate to database.
> But how to perform querying if have NO any node. I just want to get some
> data from database???
> I tried getReferenceNode() method, of EmbeddedGraphDatabase class, but it
> is
> not correct.
> So, how can I just get data from database?
> Thanks in advance!
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] neo4j question

[Peter Neubauer]

Paddy,
posting on the user list - yes, I think you should add RAM to hold
your graph in memory, and then maybe write some routing to warm up the
caches. That is, read all nodes up into RAM before you do the first
search, or e.g. perform some interesting searches to load all
interesting data.

Would that be possible? What hardware config are you running on right now?

Cheers,

/peter neubauer

COO and Sales, Neo Technology

GTalk:  neubauer.peter
Skype   peter.neubauer
Phone   +46 704 106975
LinkedIn   http://www.linkedin.com/in/neubauer
Twitter  http://twitter.com/peterneubauer

http://www.neo4j.org   - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.



On Sun, Jul 4, 2010 at 4:34 AM, Paddy  wrote:
> hi peter,
> Hope all is well, I have modeled the bus network with timetables and stops
> connecting by walking distance.
> There are approx 10 million and 1 million nodes. the graph is approx 1gb
> Running on my local machine, to find the best route , it takes 5 seconds to
> do approx 100 lucene query searches and add approx 100 relationships to the
> graph
> it can take from 3 - 10 seconds to find a shortest path , with the a star
> algorithm.
>
> what steps could i take to speed up the app?  More RAM? Run in the cloud?
> Thanks
> Paddy
>
>
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Querying Neo4j

[Paddy]

you can query neo4j  using the
LuceneFulltextQueryIndexService

here is a good example:
http://wiki.neo4j.org/content/Indexing_with_IndexService#Fulltext_indexing_with_lucene_query_syntax

Paddy

On Mon, Jul 5, 2010 at 7:19 AM, Boban Erakovic wrote:

> Greetings, I have some misunderstanding about neo4j.
> Once, when I store some nodes and relations into database, how can I query
> it?
> So I store information 5 days ago, and now I want to query? As much as I
> saw, I am using :
>
> Traverser traverser = node.traverse();
>
> So I need a node to in order to iterate over database.
> In every example, iteration over data is perform in the same code within
> where we had create already some nodes and relation, and then just reuse
> created node to communicate to database.
> But how to perform querying if have NO any node. I just want to get some
> data from database???
> I tried getReferenceNode() method, of EmbeddedGraphDatabase class, but it
> is
> not correct.
> So, how can I just get data from database?
> Thanks in advance!
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

[Neo4j] Importing data

[Logo Bogo]

Hi,

I want to use neo4j to analyse apache logs. Each visitor is identified by a 
session ID, and each visit has an ID too - a new ID is assigned after 20 
minutes between page views. The graph consists of Visit, Visitor and Page nodes 
where a Visitor -> multiple Visit nodes, and a Visit -> multiple Page nodes. 
The Visit -> Page relationship has a property to indicate when in the visit the 
page was visited (i.e. 1 = first page visited, 2 = second page in the visit, 
etc).

How would I best go about importing data into this graph? I'd use the batch 
inserter, but before I create a new Visitor or Visit I need to check whether a 
node exists already with the same ID. I've read that it's better to use the 
EmbeddedGraphDatabase, but I'm going to be inserting ~ 100K nodes 3 times per 
day. In the past using MySQL in a similar way performance was abysmal, so I 
couldn't do take this approach.

Will I be able to just use an EmbeddedGraphDatabase, or should I have a 
rethink? How is performance likely to be for these inserts?

Thanks
Tim



  
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

[Neo4j] Querying Neo4j

[Boban Erakovic]

Greetings, I have some misunderstanding about neo4j.
Once, when I store some nodes and relations into database, how can I query
it?
So I store information 5 days ago, and now I want to query? As much as I
saw, I am using :

Traverser traverser = node.traverse();

So I need a node to in order to iterate over database.
In every example, iteration over data is perform in the same code within
where we had create already some nodes and relation, and then just reuse
created node to communicate to database.
But how to perform querying if have NO any node. I just want to get some
data from database???
I tried getReferenceNode() method, of EmbeddedGraphDatabase class, but it is
not correct.
So, how can I just get data from database?
Thanks in advance!
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Meta-model component: problem with renaming classes

[Jörg Richter]

On Jul 5, 2010, at 9:13, Niels Hoogeveen wrote:

> MetaModelClasses are cached in a map, so while the property may be renamed 
> and reindexed, the map still holds the original name value. After a restart 
> of the application the new value should be applied. To make renaming possible 
> a method needs to be added to MetaModelNamespace.
> Niels

Thank you for the info!
Restarting the application is not an option in dynamic environments, where 
classes come and go (and change their names) at runtime.
BTW: deleting a class is also an issue with the meta-model component.

Is it possible for you to add a setName() method to MetaModelObject (or 
elsewhere)?

This would allow developers to use the meta-model component in dynamic 
applications.

Cheers
Jörg

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Meta-model component: problem with renaming classes

[Jörg Richter]

On Jul 5, 2010, at 8:57, Mattias Persson wrote:

> Maybe there's some bug in the removeIndex(Node,String) method. Instead
> try to do a:
>   
> index.removeIndex(classNode1,MetaModelProperty.KEY_NAME,classNode1.getName());
> 
> before you set the new name

I've tested this already. Indexing works OK. The problem is in 
namespace.getMetaClass()

> What is your use case for renaming a meta model class? Is that really
> functionality that should be used at runtime? My gut feeling says that
> this is a develpment time thing and once a class is created with a
> name it sticks with that name. Or are you doing some kind of migration
> of your data?

Renaming a class is required when building a graphical modelling tool. That is, 
the user creates classes and properties and he want to rename them. This 
happens at runtime.

I'm currently working on a rewrite of the DeepaMehta open source project. 
DeepaMehta is a platform for collaboration and knowledge management. In 
existence since 10 years, 6 years in production. Part of DeepaMehta is a 
modelling feature. For the DeepaMehta rewrite I switched from MySQL to CouchDB, 
and now to Neo4j. Neo4j fits the DeepaMehta model very well and Neo4j storage 
runs smooth since 2 months. Now, while working on the modelling feature I hit 
the wall at meta-model component. I would love to release DeepaMehta 3 v0.4 
(the first version based on Neo4j) soon.

Is there a reason to not have a setName() method on MetaModelObject?

Cheers
Jörg

___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] memory consumption

[Qiuyan Xu]

Maybe it's the reason. Does query consume a lot of memory when the  
graph is large? When the folder of the graph is about 300M, 1G memory  
seems to be not enough for the program.

Cheers,
Qiuyan

Quoting Craig Taverner :

> You mentioned that your also do queries or searches as part of the process.
> If the graph is growing in complexity, perhaps the queries are getting
> slower?
>
> On Mon, Jul 5, 2010 at 10:16 AM, Qiuyan Xu
> wrote:
>
>> I've just checked my code. It turns out that tx.finish() is already
>> there and I use a normal inserter.
>>
>> In each program I try to insert 10 data and I restart the
>> transaction every 500 insertions. The problem is, when I execute the
>> program for about the 15th time, it runs slowly shortly after the
>> program is started.
>>
>> Cheers,
>> Qiuyan
>>
>>
>> Quoting Tobias Ivarsson :
>>
>> > Just throwing this out there (since people have made mistakes with this
>> > before):
>> >
>> > Remember that commit is spelled:
>> > tx.success(); tx.finish();
>> >
>> > If you forget tx.finish(); the transaction will still be alive, and
>> continue
>> > to grow and eat more memory.
>> >
>> > Cheers,
>> > Tobias
>> >
>> > On Sun, Jul 4, 2010 at 4:33 PM, Qiuyan Xu
>> > wrote:
>> >
>> >> Thanks for the answer. I can't remember exactly which inserter I use
>> >> currently. I will check it tomorrow. But I do commit and restart a new
>> >> transaction after a fixed number of insertions.
>> >>
>> >> At the same time, I execute some queries on the database while I
>> >> insert the data, since such insertions are conditional and always
>> >> require some checks before the insertions are executed.
>> >>
>> >> cheers,
>> >> Qiuyan
>> >>
>> >> Quoting Martin Neumann :
>> >>
>> >> > Do you use the Batchinserter or a normal Transaction?
>> >> > When using a normal Transaction to insert huge amounts of data I
>> always
>> >> > submit and create a new transaction every X Items. This keeps the
>> >> > transaction small and reduces the memory used.
>> >> >
>> >> > cheers Martin
>> >> >
>> >> > On Sun, Jul 4, 2010 at 4:13 PM, 
>> wrote:
>> >> >
>> >> >> Hallo,
>> >> >>
>> >> >> I'm currently working with neo4j database and want to insert a bunch
>> >> >> of data into it.
>> >> >>
>> >> >> At the very beginning the program works quite well. But as more data
>> >> >> has been inserted into the database, the insertion runs more and more
>> >> >> slowly and I noticed that the program consumes really a lot of
>> memory.
>> >> >> Even though I splitted the input file into small pieces so that each
>> >> >> time the program tries only to insert a small part of data, the
>> >> >> problem occurs. That means, as there exists already much data in the
>> >> >> database, the program consumes a lot of memory as soon as it begins
>> so
>> >> >> that the insertion is so slow that it seems that it won't be able to
>> >> >> finish.
>> >> >>
>> >> >> I wonder if there's some solutions to save the memory. Thanks in
>> >> advance.
>> >> >>
>> >> >> Cheers,
>> >> >> Qiuyan
>> >> >>
>> >> >> ___
>> >> >> Neo4j mailing list
>> >> >> User@lists.neo4j.org
>> >> >> https://lists.neo4j.org/mailman/listinfo/user
>> >> >>
>> >> > ___
>> >> > Neo4j mailing list
>> >> > User@lists.neo4j.org
>> >> > https://lists.neo4j.org/mailman/listinfo/user
>> >> >
>> >> >
>> >>
>> >>
>> >> ___
>> >> Neo4j mailing list
>> >> User@lists.neo4j.org
>> >> https://lists.neo4j.org/mailman/listinfo/user
>> >>
>> >
>> >
>> >
>> > --
>> > Tobias Ivarsson 
>> > Hacker, Neo Technology
>> > www.neotechnology.com
>> > Cellphone: +46 706 534857
>> > ___
>> > Neo4j mailing list
>> > User@lists.neo4j.org
>> > https://lists.neo4j.org/mailman/listinfo/user
>> >
>> >
>>
>>
>> ___
>> Neo4j mailing list
>> User@lists.neo4j.org
>> https://lists.neo4j.org/mailman/listinfo/user
>>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
>


___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] memory consumption

[Craig Taverner]

You mentioned that your also do queries or searches as part of the process.
If the graph is growing in complexity, perhaps the queries are getting
slower?

On Mon, Jul 5, 2010 at 10:16 AM, Qiuyan Xu
wrote:

> I've just checked my code. It turns out that tx.finish() is already
> there and I use a normal inserter.
>
> In each program I try to insert 10 data and I restart the
> transaction every 500 insertions. The problem is, when I execute the
> program for about the 15th time, it runs slowly shortly after the
> program is started.
>
> Cheers,
> Qiuyan
>
>
> Quoting Tobias Ivarsson :
>
> > Just throwing this out there (since people have made mistakes with this
> > before):
> >
> > Remember that commit is spelled:
> > tx.success(); tx.finish();
> >
> > If you forget tx.finish(); the transaction will still be alive, and
> continue
> > to grow and eat more memory.
> >
> > Cheers,
> > Tobias
> >
> > On Sun, Jul 4, 2010 at 4:33 PM, Qiuyan Xu
> > wrote:
> >
> >> Thanks for the answer. I can't remember exactly which inserter I use
> >> currently. I will check it tomorrow. But I do commit and restart a new
> >> transaction after a fixed number of insertions.
> >>
> >> At the same time, I execute some queries on the database while I
> >> insert the data, since such insertions are conditional and always
> >> require some checks before the insertions are executed.
> >>
> >> cheers,
> >> Qiuyan
> >>
> >> Quoting Martin Neumann :
> >>
> >> > Do you use the Batchinserter or a normal Transaction?
> >> > When using a normal Transaction to insert huge amounts of data I
> always
> >> > submit and create a new transaction every X Items. This keeps the
> >> > transaction small and reduces the memory used.
> >> >
> >> > cheers Martin
> >> >
> >> > On Sun, Jul 4, 2010 at 4:13 PM, 
> wrote:
> >> >
> >> >> Hallo,
> >> >>
> >> >> I'm currently working with neo4j database and want to insert a bunch
> >> >> of data into it.
> >> >>
> >> >> At the very beginning the program works quite well. But as more data
> >> >> has been inserted into the database, the insertion runs more and more
> >> >> slowly and I noticed that the program consumes really a lot of
> memory.
> >> >> Even though I splitted the input file into small pieces so that each
> >> >> time the program tries only to insert a small part of data, the
> >> >> problem occurs. That means, as there exists already much data in the
> >> >> database, the program consumes a lot of memory as soon as it begins
> so
> >> >> that the insertion is so slow that it seems that it won't be able to
> >> >> finish.
> >> >>
> >> >> I wonder if there's some solutions to save the memory. Thanks in
> >> advance.
> >> >>
> >> >> Cheers,
> >> >> Qiuyan
> >> >>
> >> >> ___
> >> >> Neo4j mailing list
> >> >> User@lists.neo4j.org
> >> >> https://lists.neo4j.org/mailman/listinfo/user
> >> >>
> >> > ___
> >> > Neo4j mailing list
> >> > User@lists.neo4j.org
> >> > https://lists.neo4j.org/mailman/listinfo/user
> >> >
> >> >
> >>
> >>
> >> ___
> >> Neo4j mailing list
> >> User@lists.neo4j.org
> >> https://lists.neo4j.org/mailman/listinfo/user
> >>
> >
> >
> >
> > --
> > Tobias Ivarsson 
> > Hacker, Neo Technology
> > www.neotechnology.com
> > Cellphone: +46 706 534857
> > ___
> > Neo4j mailing list
> > User@lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
> >
> >
>
>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] memory consumption

[Qiuyan Xu]

I've just checked my code. It turns out that tx.finish() is already  
there and I use a normal inserter.

In each program I try to insert 10 data and I restart the  
transaction every 500 insertions. The problem is, when I execute the  
program for about the 15th time, it runs slowly shortly after the  
program is started.

Cheers,
Qiuyan


Quoting Tobias Ivarsson :

> Just throwing this out there (since people have made mistakes with this
> before):
>
> Remember that commit is spelled:
> tx.success(); tx.finish();
>
> If you forget tx.finish(); the transaction will still be alive, and continue
> to grow and eat more memory.
>
> Cheers,
> Tobias
>
> On Sun, Jul 4, 2010 at 4:33 PM, Qiuyan Xu  
> wrote:
>
>> Thanks for the answer. I can't remember exactly which inserter I use
>> currently. I will check it tomorrow. But I do commit and restart a new
>> transaction after a fixed number of insertions.
>>
>> At the same time, I execute some queries on the database while I
>> insert the data, since such insertions are conditional and always
>> require some checks before the insertions are executed.
>>
>> cheers,
>> Qiuyan
>>
>> Quoting Martin Neumann :
>>
>> > Do you use the Batchinserter or a normal Transaction?
>> > When using a normal Transaction to insert huge amounts of data I always
>> > submit and create a new transaction every X Items. This keeps the
>> > transaction small and reduces the memory used.
>> >
>> > cheers Martin
>> >
>> > On Sun, Jul 4, 2010 at 4:13 PM,  wrote:
>> >
>> >> Hallo,
>> >>
>> >> I'm currently working with neo4j database and want to insert a bunch
>> >> of data into it.
>> >>
>> >> At the very beginning the program works quite well. But as more data
>> >> has been inserted into the database, the insertion runs more and more
>> >> slowly and I noticed that the program consumes really a lot of memory.
>> >> Even though I splitted the input file into small pieces so that each
>> >> time the program tries only to insert a small part of data, the
>> >> problem occurs. That means, as there exists already much data in the
>> >> database, the program consumes a lot of memory as soon as it begins so
>> >> that the insertion is so slow that it seems that it won't be able to
>> >> finish.
>> >>
>> >> I wonder if there's some solutions to save the memory. Thanks in
>> advance.
>> >>
>> >> Cheers,
>> >> Qiuyan
>> >>
>> >> ___
>> >> Neo4j mailing list
>> >> User@lists.neo4j.org
>> >> https://lists.neo4j.org/mailman/listinfo/user
>> >>
>> > ___
>> > Neo4j mailing list
>> > User@lists.neo4j.org
>> > https://lists.neo4j.org/mailman/listinfo/user
>> >
>> >
>>
>>
>> ___
>> Neo4j mailing list
>> User@lists.neo4j.org
>> https://lists.neo4j.org/mailman/listinfo/user
>>
>
>
>
> --
> Tobias Ivarsson 
> Hacker, Neo Technology
> www.neotechnology.com
> Cellphone: +46 706 534857
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
>


___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Neo4j REST, python, prune evaluator trick/hack, and some more Q.

[Mattias Persson]

2010/7/5 Sergey Nikitin 

> Hi! I'v found that it is possible to alter database data inside
> traversal query like this:
>
> *PYTHON DICT*
> ...
>"prune evaluator": {
>  "language": "javascript",
>  "body": """
>  if(position.node().getId().equals(12)){
>position.node().setProperty('some_property','some_value');
>false;
>  }else{
>true;
>  }
>},
>
> As I understand I can do the same with relationships but I didn't find
> how to get current Database Service instance (graphdb), can anybody
> tell me is it possible and has this trick right to life ?
>

Yep that's right, you can do modifying operations within a traversal in
neo4j. You can get the graph database instance from any node or relationship
via the method getGraphDatabase() (which sits on the PropertyContainer
interface).

>
>
> Also: if graphdb can bee accessed via javasctipt, is it means that I
> can do transactional operations via traversal query ?
>

on the REST server each request executes in its own transaction so that
managing transactions within a traversal in the REST request won't actually
start new transactions (since neo4j won't treats such transactions as true
nested transaction, see
http://wiki.neo4j.org/content/Transactions#Nested_transactions for more
information).

>
> Also too: It would bу nice to pass a custom JAVA code to REST with a
> custom output.
> for example: prepare some output on serverside with one query instead
> of quering dozens when I need to get relationships and some properties
> of a Nodes.
>

Yes, it's a good idea to expose your own a domain specific layer around the
REST API just like you said. There aren't any good documentation about how
to do that yet (that I'm aware of), but it will come I'm sure.


>
> --
> Best Regards, Nikitin Sergey
>
> ICQ 343833010
> phone:
> 8(926)013-55-66
>
> site: http://nikitinsm.ru
> email: nikiti...@gmail.com
> skype: nikitinsm
> jabber: nikiti...@gmail.com
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>



-- 
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Re: [Neo4j] Meta-model component: problem with renaming classes

[Niels Hoogeveen]

MetaModelClasses are cached in a map, so while the property may be renamed and 
reindexed, the map still holds the original name value. After a restart of the 
application the new value should be applied. To make renaming possible a method 
needs to be added to MetaModelNamespace.
Niels

> Date: Mon, 5 Jul 2010 08:57:09 +0200
> From: matt...@neotechnology.com
> To: user@lists.neo4j.org
> Subject: Re: [Neo4j] Meta-model component: problem with renaming classes
> 
> 2010/7/5, Jörg Richter :
> >
> > How can a MetaModelClass be renamed?
> > There seems to be no obvious way.
> >
> > I tried to change the "meta_model_name" property of the wrapped node and
> > re-index the node. This basically works, but namespace.getMetaClass() still
> > returns a MetaModelClass object when called with the old name. IMO it would
> > be correct if null is returned in this case, like for any unknown class.
> > Otherwise the application could get confused.
> >
> > Example code:
> >
> > // creating a class and getting the wrapped node
> > MetaModelClass class1 = namespace.getMetaClass("My Class", true);
> > Node classNode1 = class1.node();
> >
> > // change the node's name property and re-index
> > classNode1.setProperty(MetaModelProperty.KEY_NAME, "New Class Name");
> > index.removeIndex(classNode1, MetaModelProperty.KEY_NAME);
> > index.index(classNode1, MetaModelProperty.KEY_NAME, "New Class Name");
> >
> 
> Maybe there's some bug in the removeIndex(Node,String) method. Instead
> try to do a:
>
> index.removeIndex(classNode1,MetaModelProperty.KEY_NAME,classNode1.getName());
> 
> before you set the new name
> 
> > // both calls return a class, the second one shouldn't
> > namespace.getMetaClass("New Class Name", false);
> > namespace.getMetaClass("My Class", false);
> >
> > Tested with latest snapshots of neo4j kernel, index, and meta-model
> > components and Lucene 2.9.2
> >
> >
> > API proposal:
> >
> > Class MetaModelObject should have a setName(String) method for easy and
> > consistent renaming of classes and properties.
> > Furthermore, namespace.getMetaClass() should return null when called with a
> > stale name.
> >
> 
> What is your use case for renaming a meta model class? Is that really
> functionality that should be used at runtime? My gut feeling says that
> this is a develpment time thing and once a class is created with a
> name it sticks with that name. Or are you doing some kind of migration
> of your data?
> 
> > BTW: one big up to the great Neo4j team and community!
> 
> Cheers!
> 
> > Cheers,
> > Jörg
> >
> >
> > ___
> > Neo4j mailing list
> > User@lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
> >
> 
> 
> -- 
> Mattias Persson, [matt...@neotechnology.com]
> Hacker, Neo Technology
> www.neotechnology.com
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
  
_
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

[Neo4j] Neo4j REST, python, prune evaluator trick/hack, and some more Q.

[Sergey Nikitin]

Hi! I'v found that it is possible to alter database data inside
traversal query like this:

*PYTHON DICT*
...
"prune evaluator": {
  "language": "javascript",
  "body": """
  if(position.node().getId().equals(12)){
position.node().setProperty('some_property','some_value');
false;
  }else{
true;
  }
},

As I understand I can do the same with relationships but I didn't find
how to get current Database Service instance (graphdb), can anybody
tell me is it possible and has this trick right to life ?


Also: if graphdb can bee accessed via javasctipt, is it means that I
can do transactional operations via traversal query ?

Also too: It would bу nice to pass a custom JAVA code to REST with a
custom output.
for example: prepare some output on serverside with one query instead
of quering dozens when I need to get relationships and some properties
of a Nodes.

-- 
Best Regards, Nikitin Sergey

ICQ 343833010
phone:
8(926)013-55-66

site: http://nikitinsm.ru
email: nikiti...@gmail.com
skype: nikitinsm
jabber: nikiti...@gmail.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user