Jake,
Thanks for looking into this. I'm going to try a few extra things
including going to 2.0.1. To answer your immediate questions, the problem
happens consistently when I run particular cypher queries - the one I sent
is one such query. It's consistent in that whenever I run certain methods
that contain those queries, this results. It's not consistent in that I
have many other read-only cypher queries that don't display this same
behavior.
I doubt I'm going to be able to remove code until I can give you something
functional; our test database is populated with stuff generated by other
classes. Based on how the code is written I fear I'd have to send you at
least a dozen or more classes, it would take substantial work to trim
things out, and we haven't yet obtained permission to release this (working
on eventual open source release of our software, as it happens).
Can you give me any tips on how I might diagnose/debug myself? I.e. a
different way of logging what neo is doing to look for the problem as an
alternative if I can't ship you a working code base? Also any tips you
might suggest about where to look. What are the possible causes of failure
here? Why is neo even attempting to "commit" a transaction? (What's to
commit when you're only fetching data?)
I'm not manually using locks; we do use constraints. All of our
constraints look like this:
CREATE CONSTRAINT ON (node:MyLabel) ASSERT node.myID IS UNIQUE;
FWIW, before we moved from 1.9.3 -> 2.0.0, we didn't have this problem;
this is recent after I started wrapping cypher queries in transactions. In
1.9.3, I would just run the query and all was fine; I started getting
exceptions after moving to 2.0.0, and advice on this list said that
wrapping those read-only cypher queries in transactions was necessary, so I
started doing it in the java 7 style: try (Transaction tx =
db.beginTx()) { blah(); tx.success(); }
Presently my workaround is to catch and ignore these transactions. The
data coming back from the read-only query looks correct, and so these
exceptions are an annoyance more than a blocker, but I'm assuming they're
symptomatic of either a bug or something I'm doing wrong that will bite me
later if I ignore it.
Thanks
On Mon, Feb 10, 2014 at 4:50 AM, Jacob Hansson <[email protected]>wrote:
> Hey,
>
> I'm sorry to say I've gone through the messages.log, and it contains from
> what I can see no hints to your problem. The upside of that is that it's
> not a "global" problem, eg. the database itself is not concerned about
> these exceptions (and thus does not log them), it only affects these
> specific transactions. That could mean that what you are seeing is
> something like a constraint violation or a detected deadlock.
>
> Does this exception happen every time you run the code, or intermittently?
> Are you using constraints, or are you manually taking locks with
> tx.acquireXXLock()?
>
> I'll need you to help me out to get further. Would you be able to take the
> code you have, and slowly remove parts until you get it down to a
> stand-alone java class with a main method that replicates the problem?
> Without running code that replicates the problem, or the cause of the
> exceptions you are seeing, it's nearly impossible to tell what is wrong.
>
> It could be worth trying to upgrade to 2.0.1, which contains a lot of bug
> fixes - but fair warning, I haven't seen any issues like this that was a
> "real" bug, so it is not at all certain that upgrading would resolve your
> issue.
>
>
> Sorry this is a bit tricky to debug, hang in there and I'm sure we'll get
> it sorted.
> jake
>
>
>
> On Fri, Feb 7, 2014 at 8:15 PM, Jacob Hansson <[email protected]>wrote:
>
>> Sweet, thanks man, sorry for missing it, was on my phone earlier today.
>> Ill have a look at it on monday.
>>
>> jake
>>
>> Sent from my phone, please excuse typos and brevity.
>> On Feb 7, 2014 3:15 PM, "M. David Allen" <[email protected]> wrote:
>>
>>> Attached is the messages.log - it was also attached to the original
>>> message in this thread.
>>>
>>> In my original message, I didn't include all the details of neo version,
>>> JVM, OS, etc because that all appears to be in this log anyway.
>>>
>>> Thanks!
>>>
>>>
>>>
>>>
>>> On Fri, Feb 7, 2014 at 6:25 AM, Jacob Hansson <[email protected]>wrote:
>>>
>>>> Thanks for bumping this David, it fell off my radar, sorry.
>>>>
>>>> Im gonna have a look at your original sample code, and Ill get back to
>>>> you.
>>>>
>>>> in the mean time, would you be able to send me the messages.log file
>>>> that neo puts in its database folder? If you are running the server, thats
>>>> in data/graph.db/, otherwise its whatever folder you tell the embedded
>>>> database to use.
>>>>
>>>> jake
>>>>
>>>> Sent from my phone, please excuse typos and brevity.
>>>> On Feb 6, 2014 2:19 PM, "M. David Allen" <[email protected]>
>>>> wrote:
>>>>
>>>>> I don't mean to be a pest, but I would really appreciate some feedback
>>>>> on this from the devs. It's something I've been wrestling with since I
>>>>> made the move to 2.0.0, and I'd really like to figure it out.
>>>>>
>>>>> I am more than willing to run any sample code or test cases that will
>>>>> help diagnose this, or provide any additional information that would be
>>>>> useful to get to the bottom of this.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Wednesday, February 5, 2014 10:24:09 AM UTC-5, M. David Allen wrote:
>>>>>>
>>>>>> No, not that I can see.
>>>>>>
>>>>>> I removed the catch code and re-ran to see what else it might come up
>>>>>> with, and below is what I got:
>>>>>>
>>>>>> Exception in thread "main" org.neo4j.graphdb.TransactionFailureException:
>>>>>> Unable to commit transaction
>>>>>> at org.neo4j.kernel.TopLevelTransaction.close(
>>>>>> TopLevelTransaction.java:134)
>>>>>> at org.mitre.provenance.db.neo4j.Neo4JStorage.getMembers(
>>>>>> Neo4JStorage.java:545)
>>>>>> at org.mitre.provenance.test.Stub.main(Stub.java:17)
>>>>>> Caused by: javax.transaction.RollbackException: Failed to commit,
>>>>>> transaction rolled back
>>>>>> at org.neo4j.kernel.impl.transaction.TxManager.
>>>>>> rollbackCommit(TxManager.java:623)
>>>>>> at org.neo4j.kernel.impl.transaction.TxManager.commit(
>>>>>> TxManager.java:402)
>>>>>> at org.neo4j.kernel.impl.transaction.TransactionImpl.
>>>>>> commit(TransactionImpl.java:122)
>>>>>> at org.neo4j.kernel.TopLevelTransaction.close(
>>>>>> TopLevelTransaction.java:124)
>>>>>> ... 2 more
>>>>>>
>>>>>>
>>>>>> On Wednesday, February 5, 2014 10:13:55 AM UTC-5, Jacob Hansson wrote:
>>>>>>>
>>>>>>> David, are there any additional cause exceptions available for that
>>>>>>> stack trace?
>>>>>>>
>>>>>>> Sent from my phone, please excuse typos and brevity.
>>>>>>> On Feb 5, 2014 3:56 PM, "M. David Allen" <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I'm running into exceptions where my transactions fail to commit,
>>>>>>>> even though they're read-only cypher queries. I created a small test
>>>>>>>> database that's attached to some software I've written (running
>>>>>>>> embedded
>>>>>>>> DB). I'm attaching messages.log for a lot of diagnostics. I ran some
>>>>>>>> simple stub code to populate the test database, then do some querying
>>>>>>>> on it
>>>>>>>> using methods I wrote.
>>>>>>>>
>>>>>>>> I'm hoping to find some help with why this is happening. I can
>>>>>>>> actually safely ignore these transaction exceptions, and nothing seems
>>>>>>>> to
>>>>>>>> go horribly wrong, but they just shouldn't be there and I don't want to
>>>>>>>> ignore them as my strategy. Below I'm providing what the exceptions
>>>>>>>> look
>>>>>>>> like, and the code of the method that's creating these exceptions. In
>>>>>>>> other parts of my code base I have other similar queries (always
>>>>>>>> read-only
>>>>>>>> cypher queries wrapped in transactions) that have the same problems.
>>>>>>>> I
>>>>>>>> ran this stub to try to illustrate the problem in one specific compact
>>>>>>>> way.
>>>>>>>>
>>>>>>>> The exception dumps --
>>>>>>>>
>>>>>>>> SEVERE: Failed transaction: Unable to commit transaction
>>>>>>>> org.neo4j.graphdb.TransactionFailureException: Unable to commit
>>>>>>>> transaction
>>>>>>>> at org.neo4j.kernel.TopLevelTransaction.close(
>>>>>>>> TopLevelTransaction.java:134)
>>>>>>>> at org.mitre.provenance.db.neo4j.Neo4JStorage.getMembers(
>>>>>>>> Neo4JStorage.java:545)
>>>>>>>> at org.mitre.provenance.test.Stub.main(Stub.java:17)
>>>>>>>> Caused by: javax.transaction.RollbackException: Failed to commit,
>>>>>>>> transaction rolled back
>>>>>>>> at org.neo4j.kernel.impl.transaction.TxManager.
>>>>>>>> rollbackCommit(TxManager.java:623)
>>>>>>>> at org.neo4j.kernel.impl.transaction.TxManager.commit(
>>>>>>>> TxManager.java:402)
>>>>>>>> at org.neo4j.kernel.impl.transaction.TransactionImpl.
>>>>>>>> commit(TransactionImpl.java:122)
>>>>>>>> at org.neo4j.kernel.TopLevelTransaction.close(
>>>>>>>> TopLevelTransaction.java:124)
>>>>>>>>
>>>>>>>> The code that's throwing the exception:
>>>>>>>>
>>>>>>>> public static ProvenanceCollection getMembers(PLUSWorkflow wf,
>>>>>>>> User user, int maximum) {
>>>>>>>> ViewedCollection d = new ViewedCollection(user);
>>>>>>>> if(db == null) initialize();
>>>>>>>>
>>>>>>>> if(maximum <= 0 || maximum > Neo4JPLUSObjectFactory.MAX_
>>>>>>>> OBJECTS)
>>>>>>>> maximum = 100;
>>>>>>>>
>>>>>>>> Map<String,Object>params = new HashMap<String,Object>();
>>>>>>>> params.put("wf", wf.getId());
>>>>>>>> String query = "start
>>>>>>>> r=relationship:relationship_auto_index(workflow={wf})
>>>>>>>> " +
>>>>>>>> "return r " +
>>>>>>>> "limit " + maximum;
>>>>>>>>
>>>>>>>> try (Transaction tx = db.beginTx()) {
>>>>>>>> ResourceIterator<Relationship> rs =
>>>>>>>> Neo4JStorage.execute(query, params).columnAs("r");
>>>>>>>>
>>>>>>>> try {
>>>>>>>> while(rs.hasNext()) {
>>>>>>>> Relationship r = rs.next();
>>>>>>>>
>>>>>>>> d.addNode(Neo4JPLUSObjectFactory.
>>>>>>>> newObject(r.getStartNode()));
>>>>>>>> d.addNode(Neo4JPLUSObjectFactory.
>>>>>>>> newObject(r.getEndNode()));
>>>>>>>> d.addEdge(Neo4JPLUSObjectFactory.newEdge(r));
>>>>>>>>
>>>>>>>> }
>>>>>>>> } catch(PLUSException exc) {
>>>>>>>> exc.printStackTrace();
>>>>>>>> }
>>>>>>>>
>>>>>>>> rs.close();
>>>>>>>> tx.success();
>>>>>>>> } catch(TransactionFailureException exc) {
>>>>>>>> log.severe("Failed transaction: " + exc.getMessage());
>>>>>>>> exc.printStackTrace();
>>>>>>>> }
>>>>>>>>
>>>>>>>> return d;
>>>>>>>> } // End getMembers
>>>>>>>>
>>>>>>>> Another relevant method (Neo4JStorage.execute): (db is a
>>>>>>>> GraphDatabaseService)
>>>>>>>>
>>>>>>>> public static ExecutionResult execute(String cypherQuery,
>>>>>>>> Map<String,Object>params) {
>>>>>>>> if(db == null) initialize();
>>>>>>>>
>>>>>>>> ExecutionEngine engine = new ExecutionEngine(db);
>>>>>>>>
>>>>>>>> assert(db.index().getNodeAutoIndexer().isEnabled());
>>>>>>>>
>>>>>>>> StringBuffer sb = new StringBuffer("");
>>>>>>>> for(String k : params.keySet()) sb.append(" " + k + "=" +
>>>>>>>> params.get(k));
>>>>>>>>
>>>>>>>> //log.info("EXECUTING: " + cypherQuery + " /" +sb);
>>>>>>>> return engine.execute(cypherQuery + " ", params);
>>>>>>>> }
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "Neo4j" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to [email protected].
>>>>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>>>>
>>>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Neo4j" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>
>>>> --
>>>> You received this message because you are subscribed to a topic in the
>>>> Google Groups "Neo4j" group.
>>>> To unsubscribe from this topic, visit
>>>> https://groups.google.com/d/topic/neo4j/w1L_21z0z04/unsubscribe.
>>>> To unsubscribe from this group and all its topics, send an email to
>>>> [email protected].
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>
>>>
>>>
>>> --
>>> M. David Allen
>>> Mobile: (804) 787-0289
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Neo4j" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Neo4j" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/neo4j/w1L_21z0z04/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
M. David Allen
Mobile: (804) 787-0289
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
