Re: [Neo4j] Neo4j, third party JTA Transaction M anager integration and potential bug.
Francesco, That's fantastic to hear. If you don't mind sharing the patch and test cases I'd like to re-consider the way we currently integrate giving it another try with real JTA/XA. -Andreas On 8/14/2010 10:14 AM, Degrassi Francesco wrote: Hi Andreas, thanks for your hint, i'll look into the chained-transaction strategy as a fallback option. I am very curious how you will end up using Neo4j with other transaction managers. So far, I haven't seen any concrete example of Database-Neo4j JTA/XA transactions in practice other than many claiming it should work ;) Actually right now, after our hacking, our setup _seems_ to work correctly and fully, we're writing a couple of integration tests to verify the behaviour in some corner cases; also I am not sure if there are other aspects I've not considered. Regards, Francesco ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Announcing Neo4j 1.0
Congratulations guys! I look forward to the upgrade. Keep up the fantastic work on a truly amazing product. -Andreas Johan Svensson wrote: Friends, After ten years of development, we are happy to finally announce the release of Neo4j 1.0. It's available here: http://neo4j.org/download http://dist.neo4j.org/neo4j-kernel-1.0-binary.zip http://dist.neo4j.org/neo4j-kernel-1.0-binary.tar.gz For the kernel component this release includes documentation updates together with bug fixes for all known bugs. For more information see: http://dist.neo4j.org/CHANGES.txt Also included in this release is the Neo4j index component: http://components.neo4j.org/neo4j-index/ (version 1.0) You can download the kernel and index (together with some other useful components) bundled together in the apoc package: http://dist.neo4j.org/neo4j-apoc-1.0.zip http://dist.neo4j.org/neo4j-apoc-1.0.tar.gz If you are using maven you can depend on the following (group id=org.neo4j): neo4j-apoc 1.0 or individual components: neo4j-kernel 1.0 neo4j-index 1.0 Finally, let us just offer a huge thanks to everyone on this list, on twitter and in the broader community. Without the feedback and energy and passion and interest from all of you guys, all the endless nights of staring through java.nio stacktraces would never be worth it. We truly feel that 2010 is the year of the graph. Let's change the world. -- Regards, The Neo4j team ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] NPE during XA transaction
Well, as it turns out all my fault as another backup process I have kicked in and interfered with my reprocessing. After I synchronized these two competing resources everything runs as smooth as usual again. Sorry for bothering but we are close to end of our release cycle ;) -Andreas P.S. Neo rocks! -Ursprüngliche Nachricht- Von: Andreas Guenther andreas.guent...@web.de Gesendet: 11.11.09 20:57:51 An: user@lists.neo4j.org Betreff: [Neo] NPE during XA transaction Hi, I am trying to reprocess requests available from a queue into my neo store. I intentionally avoid using the bulk load feature. The idea behind the feature is to have the ability synchronizing neo with the database in case neo missed transactions and also to be able to reconstruct the store from scratch if needed. Everything works just fine when processing the queue in production, that is, where requests come in infrequently, all distributed during the day. It unfortunately fails when trying to reprocess them in fast order. It consistently happens after approx 2000 processing requests. Any idea what could have caused the NPE as pasted below? Looks like the 'LogBuffer writeBuffer' got assigned to NULL at one point, probably during either releaseCurrentLogFile() or close(). Anything related to log rotation, maybe? This happens on Linux with Neo4j 1.0-b10. Any workarounds or explanations appreciated, -Andreas org.neo4j.impl.transaction.TransactionFailureException: Unable to commit transaction ... Caused by: java.lang.NullPointerException at org.neo4j.impl.transaction.xaframework.XaLogicalLog.done(XaLogicalLog.java:462) at org.neo4j.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:400) at org.neo4j.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) at org.neo4j.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:514) at org.neo4j.impl.transaction.TxManager.commit(TxManager.java:573) at org.neo4j.impl.transaction.TxManager.commit(TxManager.java:543) at org.neo4j.impl.transaction.TransactionImpl.commit(TransactionImpl.java:102) at org.neo4j.api.core.EmbeddedNeoImpl$TransactionImpl.finish(EmbeddedNeoImpl.java:395) __ GRATIS für alle WEB.DE-Nutzer: Die maxdome Movie-FLAT! Jetzt freischalten unter http://movieflat.web.de ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user __ GRATIS für alle WEB.DE-Nutzer: Die maxdome Movie-FLAT! Jetzt freischalten unter http://movieflat.web.de ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Infrequent DeadlockDetectedException
Johan,
I tried your second suggestion but unfortunately, I am running into the same
dead-lock, only this time while calling LockManager.getWriteLock(). I pasted
how I am using this feature and below the new exception output. Is the usage
maybe incorrect? Note, that I rely on Spring transactions, which appears to
work just fine regarding the release.
public void aquireTransactionalWriteLock(Node node) {
LockReleaser lockReleaser =
embeddedNeoService.getConfig().getLockReleaser();
lockReleaser.addLockToTransaction(node, LockType.WRITE);
LockManager lockManager =
embeddedNeoService.getConfig().getLockManager();
lockManager.getWriteLock(node);
}
Thanks,
-Andreas
vv
org.neo4j.impl.transaction.DeadlockDetectedException:
Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|1257353507304|1887584],
BranchId
[ 49 54 50 51 55 51 ]]
xaresource[org.neo4j.util.index.lucenexaconnection$lucenexaresou...@1b65dd9]
Status[ENLISTED],Xid[GlobalId[NEOKERNL|1257353507304|1
887584], BranchId[ 52 49 52 49 52 49 ]]
xaresource[org.neo4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@1cb83ad]
Status[ENLISTED] can't wait o
n resource RWLock[Node[310788]] since =
Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|1257353507304|1887584],
BranchId[ 49 54 50 51
55 51 ]]
xaresource[org.neo4j.util.index.lucenexaconnection$lucenexaresou...@1b65dd9]
Status[ENLISTED],Xid[GlobalId[NEOKERNL|1257353507304|1887584], Bran
chId[ 52 49 52 49 52 49 ]]
xaresource[org.neo4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@1cb83ad]
Status[ENLISTED] - RWLock[Node[302033]]
-
Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|1257353507304|1887584],
BranchId[ 49 54 50 51 55 51 ]] XAResource[org.neo4j.util.ind
ex.lucenexaconnection$lucenexaresou...@1b65dd9]
Status[ENLISTED],Xid[GlobalId[NEOKERNL|1257353507304|1887584], BranchId[ 52 49
52 49 52 49 ]] XAResource[o
rg.neo4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@1cb83ad]
Status[ENLISTED] - RWLock[Node[310788]]
at
org.neo4j.impl.transaction.RagManager.checkWaitOnRecursive(RagManager.java:208)
at
org.neo4j.impl.transaction.RagManager.checkWaitOnRecursive(RagManager.java:236)
at
org.neo4j.impl.transaction.RagManager.checkWaitOn(RagManager.java:176)
at org.neo4j.impl.transaction.RWLock.acquireWriteLock(RWLock.java:298)
at
org.neo4j.impl.transaction.LockManager.getWriteLock(LockManager.java:122)
-Ursprüngliche Nachricht-
Von: Johan Svensson jo...@neotechnology.com
Gesendet: 03.11.09 15:16:18
An: Neo user discussions user@lists.neo4j.org
Betreff: Re: [Neo] Infrequent DeadlockDetectedException
Lock handling has not been exposed in the real API (yet) but you can
still access the internal lock manager. Here is an example on how to
do it:
// get lock manager
LockManager lockManager = ((EmbeddedNeo)
neo).getConfig().getLockManager();
// start transaction that sometimes deadlocks with other transactions
Transaction tx = neo.beginTx();
// first thing we do is grab a write lock on something,
// for example a node (the other transactions then have to
lock on same node)
lockManager.getWriteLock( node );
// perform work that previously deadlocked
// release write lock
lockManager.releaseWriteLock( node );
// commit transaction
tx.success();
tx.finish();
There exist a lock releaser adding the lock to the running
transaction. The lock will then be released upon commit or rollback of
the transaction:
Transaction tx = neo.beginTx();
lockManager.getWriteLock( node );
// add write lock on node to transaction
LockReleaser lockReleaser = ((EmbeddedNeo)
neo).getConfig().getLockReleaser();
lockManager.addLockToTransaction( node, LockType.WRITE );
// perform work, lock will be released on commit/rollback
Hope this helps.
Regards,
-Johan
On Tue, Nov 3, 2009 at 12:49 AM, Andreas Guenther
andreas.guent...@web.de wrote:
Johan,
I like to consider the node lock idea. Can you provide a quick code
snippet on the recommended approach for this?
Thank you in advance,
-Andreas
Johan Svensson wrote:
On Mon, Nov 2, 2009 at 9:06 PM, Andreas Guenther
andreas.guent...@web.de wrote:
Hi,
I am getting the below deadlock exception occasionally and so far only
on relationship operations. The code base is trunk. The system this is
running on is Linux and the Java app integrates Neo with Spring
transaction manager. Multiple threads are actively updating the Neo
storage in different ways. The only way for us to currently mitigate
this problem is to retry the entire transaction call around this, but
this only succeeds to 99%. I am using the LuceneIndexer to store a
simple Long ID value representing
[Neo] Infrequent DeadlockDetectedException
Hi, I am getting the below deadlock exception occasionally and so far only on relationship operations. The code base is trunk. The system this is running on is Linux and the Java app integrates Neo with Spring transaction manager. Multiple threads are actively updating the Neo storage in different ways. The only way for us to currently mitigate this problem is to retry the entire transaction call around this, but this only succeeds to 99%. I am using the LuceneIndexer to store a simple Long ID value representing a node in case that helps based on the exception message. I am currently thinking how to best extract this problem into a little unit test but based on how often this happens, I haven't been able to reproduce it in a self-contained test suite. Any suggestions on how to get this under control? Thanks for any help, -Andreas org.neo4j.impl.transaction.DeadlockDetectedException: Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|1257104121765|6930039], BranchId[ 49 54 50 51 55 51 ]] xaresource[org.neo4j.util.index.lucenexaconnection$lucenexaresou...@cd66f1] Status[ENLISTED],Xid[GlobalId[NEOKERNL|1257104121765|6930 039], BranchId[ 52 49 52 49 52 49 ]] xaresource[org.neo4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@12161db] Status[ENLISTED] can't wait on re source RWLock[NodeImpl#429819] since = Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|1257104121765|6930039], BranchId[ 49 54 50 51 5 5 51 ]] xaresource[org.neo4j.util.index.lucenexaconnection$lucenexaresou...@cd66f1] Status[ENLISTED],Xid[GlobalId[NEOKERNL|1257104121765|6930039], BranchId [ 52 49 52 49 52 49 ]] xaresource[org.neo4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@12161db] Status[ENLISTED] - RWLock[NodeImpl#269645] - Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|1257104121765|6930039], BranchId[ 49 54 50 51 55 51 ]] XAResource[org.neo4j.util.index. lucenexaconnection$lucenexaresou...@cd66f1] Status[ENLISTED],Xid[GlobalId[NEOKERNL|1257104121765|6930039], BranchId[ 52 49 52 49 52 49 ]] XAResource[org.ne o4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@12161db] Status[ENLISTED] - RWLock[NodeImpl#429819] at org.neo4j.impl.transaction.RagManager.checkWaitOnRecursive(RagManager.java:208) at org.neo4j.impl.transaction.RagManager.checkWaitOnRecursive(RagManager.java:236) at org.neo4j.impl.transaction.RagManager.checkWaitOn(RagManager.java:176) at org.neo4j.impl.transaction.RWLock.acquireWriteLock(RWLock.java:298) at org.neo4j.impl.transaction.LockManager.getWriteLock(LockManager.java:122) at org.neo4j.impl.core.NodeManager.acquireLock(NodeManager.java:682) at org.neo4j.impl.core.NodeManager.createRelationship(NodeManager.java:303) at org.neo4j.impl.core.NodeImpl.createRelationshipTo(NodeImpl.java:357) at org.neo4j.impl.core.NodeProxy.createRelationshipTo(NodeProxy.java:177) org.neo4j.impl.transaction.DeadlockDetectedException: Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|1256778708568|13441], BranchId[ 49 54 50 51 55 51 ]] xaresource[org.neo4j.util.index.lucenexaconnection$lucenexaresou...@467cb704] Status[ENLISTED],Xid[GlobalId[NEOKERNL|1256778708568|13441], BranchId[ 52 49 52 49 52 49 ]] xaresource[org.neo4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@25d38bc3] Status[ENLISTED] can't wait on resource RWLock[RelationshipImpl #22 of type CUSTOM_TYPE between Node[1] and Node[14]] since = Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|125 6778708568|13441], BranchId[ 49 54 50 51 55 51 ]] xaresource[org.neo4j.util.index.lucenexaconnection$lucenexaresou...@467cb704] Status[ENLISTED],Xid[GlobalId [NEOKERNL|1256778708568|13441], BranchId[ 52 49 52 49 52 49 ]] xaresource[org.neo4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@25d38bc3] Status[E NLISTED] - RWLock[NodeImpl#1] - Transaction[Status=STATUS_ACTIVE,ResourceList=Xid[GlobalId[NEOKERNL|1256778708568|13441], BranchId[ 49 54 50 51 55 51 ]] XA resource[org.neo4j.util.index.lucenexaconnection$lucenexaresou...@467cb704] Status[ENLISTED],Xid[GlobalId[NEOKERNL|1256778708568|13441], BranchId[ 52 49 52 4 9 52 49 ]] xaresource[org.neo4j.impl.nioneo.xa.neostorexaconnection$neostorexaresou...@25d38bc3] Status[ENLISTED] - RWLock[RelationshipImpl #22 of type CUSTOM_TYPE between Node[1] and Node[14]] at org.neo4j.impl.transaction.RagManager.checkWaitOnRecursive(RagManager.java:208) at org.neo4j.impl.transaction.RagManager.checkWaitOnRecursive(RagManager.java:236) at org.neo4j.impl.transaction.RagManager.checkWaitOn(RagManager.java:176) at org.neo4j.impl.transaction.RWLock.acquireWriteLock(RWLock.java:298) at org.neo4j.impl.transaction.LockManager.getWriteLock(LockManager.java:122) at org.neo4j.impl.nioneo.xa.NeoTransaction.getWriteLock(NeoTransaction.java:664)
Re: [Neo] Infrequent DeadlockDetectedException
Johan, I like to consider the node lock idea. Can you provide a quick code snippet on the recommended approach for this? Thank you in advance, -Andreas Johan Svensson wrote: On Mon, Nov 2, 2009 at 9:06 PM, Andreas Guenther andreas.guent...@web.de wrote: Hi, I am getting the below deadlock exception occasionally and so far only on relationship operations. The code base is trunk. The system this is running on is Linux and the Java app integrates Neo with Spring transaction manager. Multiple threads are actively updating the Neo storage in different ways. The only way for us to currently mitigate this problem is to retry the entire transaction call around this, but this only succeeds to 99%. I am using the LuceneIndexer to store a simple Long ID value representing a node in case that helps based on the exception message. I am currently thinking how to best extract this problem into a little unit test but based on how often this happens, I haven't been able to reproduce it in a self-contained test suite. Any suggestions on how to get this under control? Hi, Looks like a standard deadlock, two or more transactions try to modify the same data and end up in a deadlock. There are several ways to get around this. I would suggest either do some external synchronization at the beginning of the transaction (for example get a write lock on a node so transactions will execute after each other instead of race towards a potential deadlock) or modify the node space layout so there is a lower chance of transactions waiting on each other (similar to lock striping but you do it by having a different layout of the graph). Regards, -Johan ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Counting nodes
The service API has a getAllNodes() call. Iterate and count through it. -AndreasAndreas Kollegger wrote: Hi all, For nodes that are kept in the indexed service, is there a more clever (and hopefully efficient) way to get the node count than manually iterating over all the nodes in the index and actually counting. I don't see anything obvious in the IndexService() interface, and am not sure if I've overlooked something useful somewhere else. I suppose I could always keep a running tally somewhere, but then I'd have to be careful to keep it in sync with reality. Ideally the count would be closer to the metal. Thanks, Andreas ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Counting nodes
actually, disregard my suggestion as I didn't read the word index in your question before. -AndreasAndreas Guenther wrote: The service API has a getAllNodes() call. Iterate and count through it. -AndreasAndreas Kollegger wrote: Hi all, For nodes that are kept in the indexed service, is there a more clever (and hopefully efficient) way to get the node count than manually iterating over all the nodes in the index and actually counting. I don't see anything obvious in the IndexService() interface, and am not sure if I've overlooked something useful somewhere else. I suppose I could always keep a running tally somewhere, but then I'd have to be careful to keep it in sync with reality. Ideally the count would be closer to the metal. Thanks, Andreas ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Next release?
Hello, Any update on the b9 release? I am really looking forward to all the recent and great changes to be available as a release. Thanks, -Andreas Emil Eifrem wrote: On Fri, May 29, 2009 at 2:29 AM, Andreas Guenther aguent...@flying-orange.com wrote: Hi, Based on recent changes and using trunk Neo together with various other Neo components I am somewhat starting to lose track of all these custom trunk dependencies. If the Neo team could pull of a new release, meaning to have tags, downloads, and maven repo entries for Neo and all its related components, that'll be just grand! Especially Neoclipse ;) Hi Andreas, So the plan is actually to release b9 in about 4 weeks time (end of June). The unsigned branch is some fairly substantial work which requires a lot of testing and it'll take some time to whip it into shape. We also want to get some more API goodness in there. Otherwise, things are progressing very well using Neo so far, only manually juggling the version dependencies in our local repo ;( Great to hear! We really appreciate your feedback and contributions. Is there anything else we can do before b9 that would help you out? For example, pay particular attention to having some components always integrated on trunk level. We really need to get that CI server up and running soon... Cheers, ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Online backup fails with suggested Spring transaction integration
Johan, Understood. In that case I suggest a simple Configuration interface with getConfig(), implemented by EmbeddedNeoService. This would suffice as a more appropriate solution to the configuration workaround IMO because then you only have to pass that interface to the transaction manager. Certainly, you had to extract the inner class Configuration then, too ;) Regardless, I am happy to wait until the next release for a proper solution and in the meantime can live with my own hacked version. Thanks for clarifying. -Andreas Johan Svensson wrote: Andreas, I applied most of it, both SpringTransactionManager and UserTransactionImpl now have the get/set method for transaction manager and the default constructor. I did not add the new constructor in EmbeddedNeo because it didn't feel right to have a constructor taking SpringTransactionManager there (NeoService knows nothing about Spring). We will add an EmbeddedNeo constructor that can take a javax.transaction.TransactionManager instead (probably through configuration map). -Johan On Thu, May 28, 2009 at 10:07 PM, Andreas Guenther aguent...@flying-orange.com wrote: Johan, What is the reason that you didn't apply the SpringTransactionManager injection changes to SpringTransactionManager, UserTransactionImpl, EmbeddedNeo, and NeoJvmInstance? -Andreas ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Next release?
Emil Eifrem wrote: On Fri, May 29, 2009 at 2:29 AM, Andreas Guenther aguent...@flying-orange.com wrote: Hi, Based on recent changes and using trunk Neo together with various other Neo components I am somewhat starting to lose track of all these custom trunk dependencies. If the Neo team could pull of a new release, meaning to have tags, downloads, and maven repo entries for Neo and all its related components, that'll be just grand! Especially Neoclipse ;) Hi Andreas, So the plan is actually to release b9 in about 4 weeks time (end of June). The unsigned branch is some fairly substantial work which requires a lot of testing and it'll take some time to whip it into shape. We also want to get some more API goodness in there. That's fantastic to hear, especially around EmbeddedNeoService would be appreciated. Otherwise, things are progressing very well using Neo so far, only manually juggling the version dependencies in our local repo ;( Great to hear! We really appreciate your feedback and contributions. Is there anything else we can do before b9 that would help you out? For example, pay particular attention to having some components always integrated on trunk level. We really need to get that CI server up and running soon... That'll be the answer to pretty much my problems because then I can depend on at least regularly tagged snapshots. I really dislike to depend on trunk and that's why I created internal tags. I simply wait until the release for now. -Andreas Cheers, ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] How to build Neoclipse?
Anders, Thanks for the pointers. That's actually awesome to be able to build the executable ;) Shame it doesn't build out-of-the-box with Maven. -Andreas Anders Nawroth wrote: Andreas Guenther skrev: I am running into the below error. Am I missing anything? Nope, this is a known problem with Maven. Your choices are: 1. Run it from source from Eclipse: http://wiki.neo4j.org/content/Neoclipse#Running_from_source_code 2. Build the plugin using Tycho: http://www.sonatype.com/people/2008/11/building-eclipse-plugins-with-maven-tycho/ (the article actually uses Neoclipse as an example) 3. If you want the RCP version, export it from the Product Configuration Editor in Eclipse (opening the neoclipse.product file). My main reason for building Neoclipse is that none of the downloadable versions works with trunk. Neoclipse simply crashes when trying to open against non trunk stores. Thanks for point this out! /anders ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] How to build Neoclipse?
I am running into the below error. Am I missing anything? I installed Eclipse 3.4.2 and mavenized the plugins as outlined in the readme. Ideally, I am able to build the executable. My main reason for building Neoclipse is that none of the downloadable versions works with trunk. Neoclipse simply crashes when trying to open against non trunk stores. -Andreas - C:\svn\neo\components\neoclipse\trunk\org.neo4j.neoclipsemvn clean install -DskipTests [DEBUG] -- end configuration -- [INFO] [clean:clean] [INFO] [ERROR] BUILD ERROR [INFO] [INFO] Cannot find lifecycle mapping for packaging: 'eclipse-plugin'. Component descriptor cannot be found in the component repository: org.apache.maven.lifecycle.mapping.LifecycleMappingeclipse-plugin. [INFO] [DEBUG] Trace org.apache.maven.lifecycle.LifecycleExecutionException: Cannot find lifecycle mapping for packaging: 'eclipse-plugin'. at org.apache.maven.lifecycle.DefaultLifecycleExecutor.findMappingsForLifecycle(DefaultLifecycleExecutor.java:1088) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.bindLifecycleForPackaging(DefaultLifecycleExecutor.java:1014) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.constructLifecycleMappings(DefaultLifecycleExecutor.java:997) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:477) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:330) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:291) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:142) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:336) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:129) at org.apache.maven.cli.MavenCli.main(MavenCli.java:287) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315) at org.codehaus.classworlds.Launcher.launch(Launcher.java:255) at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430) at org.codehaus.classworlds.Launcher.main(Launcher.java:375) Caused by: org.codehaus.plexus.component.repository.exception.ComponentLookupException: Component descriptor cannot be found in the component repository: org.apache.maven.lifecycle .mapping.LifecycleMappingeclipse-plugin. at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:323) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:440) at org.apache.maven.execution.MavenSession.lookup(MavenSession.java:125) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.findMappingsForLifecycle(DefaultLifecycleExecutor.java:1081) ... 17 more [INFO] [INFO] Total time: 1 second [INFO] Finished at: Sun May 24 16:37:09 PDT 2009 [INFO] Final Memory: 7M/12M [INFO] ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Online backup fails with suggested Spring transaction integration
Johan,
First of all thank you for quickly addressing my issue. I tested it and things
work just fine.
Still, I took the liberty to actually refactor your solution slightly different
hoping you could review the change and commit it to trunk. My code change
mainly frees the transaction manager from knowing anything about NeoService and
instead have the NeoService implementation (EmbeddedNeo) manage the
configuration. This also avoids casting NeoService down to EmbeddedNeo, which I
frankly have seen too many times in the code base, and really suffered under
with my own NeoService wrapper implementation ;) Anyhow, I understand that your
changes are workarounds to get things going until someone has time to properly
address the underlying transaction manager integration issue.
I hope my attached patch goes through ;)
Cheers,
-Andreas
On Wed, May 20, 2009 at 04:31:09PM +0200, Johan Svensson wrote:
Andreas,
I had a look at this and realized problem is caused by the static
nature of SpringTransactionManager and UserTransactionImpl. A fix for
this has been on my todo for some time but to make it right we need to
expose some new stuff in NeoService (mainly tied to configuration API
that has yet to be specified). For now I did a quick fix in
b9-SNAPSHOT changing the SpringTransactionManager
UserTransactionImpl classes to take NeoService in constructor.
Previously those two classes were singletons and the startup of a new
NeoService did just set a static field in them (not right). This
caused the backup slave's TM to be configured as the Spring TM and
that would in turn cause declarative transactions to fail.
I had to change your NeoServiceWrapper to take NeoService instead of a
string path to the neo db. This is because the NeoService you are
running with must be the same NeoService that was used to configure
and setup the Spring transaction manager.
I committed those changes to the test transaction project in your laboratory.
NOTE: The old suggested spring configuration (found on the wiki, use
it with neo-b8) will not work anymore when running neo-b9-SNAPSHOT.
Instead one has to do something like:
bean id=neo class=org.neo4j.api.core.EmbeddedNeo
destroy-method=shutdown lazy-init=false
constructor-arg
value${neo.store}/value
/constructor-arg
/bean
tx:annotation-driven transaction-manager=transactionManager/
alias name=neoTransactionManager alias=transactionManager/
bean id=neoTransactionManagerService
class=org.neo4j.impl.transaction.SpringTransactionManager
constructor-arg ref=neo/
/bean
bean id=neoUserTransactionService
class=org.neo4j.impl.transaction.UserTransactionImpl
constructor-arg ref=neo/
/bean
bean id=neoTransactionManager
class=org.springframework.transaction.jta.JtaTransactionManager
property name=transactionManager ref=neoTransactionManagerService/
property name=userTransaction ref=neoUserTransactionService/
/bean
Regards,
Johan
On Wed, May 20, 2009 at 2:00 AM, Andreas Guenther
aguent...@flying-orange.com wrote:
Johan, Emil,
Oki, I committed a test project resembling a simplification of what I am
using in our application.
https://svn.neo4j.org/laboratory/users/aguenther/testing/transaction
The two relevant test cases I am referring to are:
#1 ExplicitTransactionBackupTest (succeeds)
#2 SpringBasedTransactionBackupTest (fails)
The second test case relies on Spring's declarative transactions which
successfully work otherwise in our application.
I hope the code makes enough sense to spot the underlying problem. Ideally
it's just me not properly using Neo ;)
Thanks in advance,
-Andreas
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Index: src/test/org/neo4j/impl/nioneo/store/TestXa.java
===
--- src/test/org/neo4j/impl/nioneo/store/TestXa.java(revision 2869)
+++ src/test/org/neo4j/impl/nioneo/store/TestXa.java(working copy)
@@ -672,6 +672,7 @@
ds.rotateLogicalLog();
assertTrue( ds.getLogicalLog( currentVersion ) != null );
assertTrue( ds.getLogicalLog( currentVersion + 1 ) != null );
+ds.keepLogicalLogs( false );
}
catch ( Exception e )
{
Index: src/java/org/neo4j/impl/transaction/SpringTransactionManager.java
===
--- src/java/org/neo4j/impl/transaction/SpringTransactionManager.java
(revision 2869)
+++ src/java/org/neo4j/impl/transaction/SpringTransactionManager.java
(working copy)
@@ -33,12 +33,7 @@
public class SpringTransactionManager implements TransactionManager
{
-private final TransactionManager tm;
-
-public SpringTransactionManager( NeoService neo )
-{
-this.tm = ((EmbeddedNeo) neo
Re: [Neo] Online backup fails with suggested Spring transaction integration
I should probably outline how spring configuration looks like with
my change. It's pretty similar to before with two constructor changes:
bean id=neoTransactionManagerService
class=org.neo4j.impl.transaction.SpringTransactionManager lazy-init=false/
bean id=neoUserTransactionService
class=org.neo4j.impl.transaction.UserTransactionImpl
constructor-arg ref=neoTransactionManagerService/
/bean
bean id=neoTransactionManager
class=org.springframework.transaction.jta.JtaTransactionManager
property name=transactionManager ref=neoTransactionManagerService/
property name=userTransaction ref=neoUserTransactionService/
/bean
bean id=neo class=org.neo4j.api.core.EmbeddedNeo
destroy-method=shutdown lazy-init=false
constructor-arg index=0 value=${neo.store}/
constructor-arg index=1 value=neoTransactionManagerService/
/bean
-Andreas
On Wed, May 20, 2009 at 05:06:56PM -0700, Andreas Guenther wrote:
Johan,
First of all thank you for quickly addressing my issue. I tested it and
things work just fine.
Still, I took the liberty to actually refactor your solution slightly
different hoping you could review the change and commit it to trunk. My code
change mainly frees the transaction manager from knowing anything about
NeoService and instead have the NeoService implementation (EmbeddedNeo)
manage the configuration. This also avoids casting NeoService down to
EmbeddedNeo, which I frankly have seen too many times in the code base, and
really suffered under with my own NeoService wrapper implementation ;)
Anyhow, I understand that your changes are workarounds to get things going
until someone has time to properly address the underlying transaction manager
integration issue.
I hope my attached patch goes through ;)
Cheers,
-Andreas
On Wed, May 20, 2009 at 04:31:09PM +0200, Johan Svensson wrote:
Andreas,
I had a look at this and realized problem is caused by the static
nature of SpringTransactionManager and UserTransactionImpl. A fix for
this has been on my todo for some time but to make it right we need to
expose some new stuff in NeoService (mainly tied to configuration API
that has yet to be specified). For now I did a quick fix in
b9-SNAPSHOT changing the SpringTransactionManager
UserTransactionImpl classes to take NeoService in constructor.
Previously those two classes were singletons and the startup of a new
NeoService did just set a static field in them (not right). This
caused the backup slave's TM to be configured as the Spring TM and
that would in turn cause declarative transactions to fail.
I had to change your NeoServiceWrapper to take NeoService instead of a
string path to the neo db. This is because the NeoService you are
running with must be the same NeoService that was used to configure
and setup the Spring transaction manager.
I committed those changes to the test transaction project in your
laboratory.
NOTE: The old suggested spring configuration (found on the wiki, use
it with neo-b8) will not work anymore when running neo-b9-SNAPSHOT.
Instead one has to do something like:
bean id=neo class=org.neo4j.api.core.EmbeddedNeo
destroy-method=shutdown lazy-init=false
constructor-arg
value${neo.store}/value
/constructor-arg
/bean
tx:annotation-driven transaction-manager=transactionManager/
alias name=neoTransactionManager alias=transactionManager/
bean id=neoTransactionManagerService
class=org.neo4j.impl.transaction.SpringTransactionManager
constructor-arg ref=neo/
/bean
bean id=neoUserTransactionService
class=org.neo4j.impl.transaction.UserTransactionImpl
constructor-arg ref=neo/
/bean
bean id=neoTransactionManager
class=org.springframework.transaction.jta.JtaTransactionManager
property name=transactionManager ref=neoTransactionManagerService/
property name=userTransaction ref=neoUserTransactionService/
/bean
Regards,
Johan
On Wed, May 20, 2009 at 2:00 AM, Andreas Guenther
aguent...@flying-orange.com wrote:
Johan, Emil,
Oki, I committed a test project resembling a simplification of what I am
using in our application.
https://svn.neo4j.org/laboratory/users/aguenther/testing/transaction
The two relevant test cases I am referring to are:
#1 ExplicitTransactionBackupTest (succeeds)
#2 SpringBasedTransactionBackupTest (fails)
The second test case relies on Spring's declarative transactions which
successfully work otherwise in our application.
I hope the code makes enough sense to spot the underlying problem.
Ideally it's just me not properly using Neo ;)
Thanks in advance,
-Andreas
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Index: src/test/org/neo4j/impl
Re: [Neo] Online backup release tag?
Anders, Awesome! Thank you. -Andreas Anders Nawroth wrote: Hi Andreas! Now there is a 0.1 release of online-backup. There will soon be a 0.2-snapshot too, but the only difference is some project cleanup, so no need to care about it at the moment. /anders Andreas Guenther skrev: Are there any objections in creating a first release for online-backup component? Any candidate, alpha or beta, will do, really something that is not a moving target and I can create a safe Maven dependency against ;) Thanks, -Andreas ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Online backup fails with suggested Spring transaction integration
Emil, I'd prefer to upload what I have to the laboratory once my CLA is accepted and a sub folder is available. This will make it easier to keep all my posted issues and testing code in the Neo space for future reference. -Andreas Emil Eifrem wrote: On Sun, May 17, 2009 at 23:45, Andreas Guenther aguent...@flying-orange.com wrote: Johan, Looks like there's no unexpected shutdown going on according to my debugging. I do have a full test project available for you but it's too big for pasting it here. Any suggestion on where I can place you a Maven test project tarball? Hi Andreas, Well, I'd suggest using something like dropbox [1] or yousendit [2]. Or if you do want to submit patches as discussed previously and agree with our contributor policy [3], then you can just sign the CLA [4]. After that, I'd be glad to give you write access to the svn repo and you can add it to your laboratory. 1] http://getdropbox.com 2] http://yousendit.com/ 3] http://wiki.neo4j.org/content/Code_Contributor%27s_Guide 4] http://wiki.neo4j.org/content/About_Contributor_License_Agreement Cheers, ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Online backup fails with suggested Spring transaction integration
Johan, Emil, Oki, I committed a test project resembling a simplification of what I am using in our application. https://svn.neo4j.org/laboratory/users/aguenther/testing/transaction The two relevant test cases I am referring to are: #1 ExplicitTransactionBackupTest (succeeds) #2 SpringBasedTransactionBackupTest (fails) The second test case relies on Spring's declarative transactions which successfully work otherwise in our application. I hope the code makes enough sense to spot the underlying problem. Ideally it's just me not properly using Neo ;) Thanks in advance, -Andreas On Tue, May 19, 2009 at 11:34:21PM +0200, Emil Eifrem wrote: On Tue, May 19, 2009 at 18:17, Andreas Guenther aguent...@flying-orange.com wrote: Emil, I'd prefer to upload what I have to the laboratory once my CLA is accepted and a sub folder is available. This will make it easier to keep all my posted issues and testing code in the Neo space for future reference. Hi Andreas, Good that you brought this up, cause the CLA you sent was actually caught by Google's spam filters! :( I'll set it up for you immediately. Cheers, -- Emil Eifrém, CEO [e...@neotechnology.com] Neo Technology, www.neotechnology.com Cell: +46 733 462 271 | US: 206 403 8808 http://twitter.com/emileifrem ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Online backup release tag?
Are there any objections in creating a first release for online-backup component? Any candidate, alpha or beta, will do, really something that is not a moving target and I can create a safe Maven dependency against ;) Thanks, -Andreas ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Online backup fails with suggested Spring transaction integration
Johan, Looks like there's no unexpected shutdown going on according to my debugging. I do have a full test project available for you but it's too big for pasting it here. Any suggestion on where I can place you a Maven test project tarball? -Andreas Johan Svensson wrote: On Sat, May 16, 2009 at 10:25 PM, Andreas Guenther aguent...@flying-orange.com wrote: I have been poking around with online backup for a couple of days now and are happy with the results in general, especially the runtime backup capability. Most of the issues I came across I was able to work around but there is one obstacles left that I need some help on. As long as I use Neo's transactions programmatically taking backups work just fine at any given point in time during regular Neo usage. But once I switch to declarative transactions through the suggested Spring configuration taken from the IMdb example I am running into all sorts of exceptions after the first backup is taken. Below is the Spring configuration and some exceptions I got. If helpful I could extend the online-backup test suite with declarative transactions. Could there be some log file access conflicts going on? It's probably worth mentioning that I am using declarative transactions throughout the rest of our app and it's been working just great after many stress tests and without using online backup. Any thoughts on this one? -Andreas Looks like the global transaction log has been closed. Can you verify that NeoService.shutdown() hasn't been invoked? Another possible cause could be wrong use of Thread.interrupt() somewhere. Would be great if you could produce a test case for this problem so I can have a look at it. Regards, Johan ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Maven source plugin across all Neo components
Can someone please add the maven source plugin to all Neo projects. This will make debugging and understanding the code base much easier for me while integrating into my own apps. Thank you, -Andreas plugin artifactIdmaven-source-plugin/artifactId executions execution goals goaljar/goal /goals /execution /executions /plugin ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Online backup fails with suggested Spring transaction integration
I have been poking around with online backup for a couple of days now and are happy with the results in general, especially the runtime backup capability. Most of the issues I came across I was able to work around but there is one obstacles left that I need some help on. As long as I use Neo's transactions programmatically taking backups work just fine at any given point in time during regular Neo usage. But once I switch to declarative transactions through the suggested Spring configuration taken from the IMdb example I am running into all sorts of exceptions after the first backup is taken. Below is the Spring configuration and some exceptions I got. If helpful I could extend the online-backup test suite with declarative transactions. Could there be some log file access conflicts going on? It's probably worth mentioning that I am using declarative transactions throughout the rest of our app and it's been working just great after many stress tests and without using online backup. Any thoughts on this one? -Andreas --- Spring configuration --- bean id=neoTransactionManagerService class=org.neo4j.impl.transaction.SpringTransactionManager/ bean id=neoUserTransactionService class=org.neo4j.impl.transaction.UserTransactionImpl/ bean id=neoTransactionManager class=org.springframework.transaction.jta.JtaTransactionManager property name=transactionManager ref=neoTransactionManagerService/ property name=userTransaction ref=neoUserTransactionService/ /bean --- --- Exceptions --- java.nio.channels.ClosedChannelException at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:91) at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:192) at org.neo4j.impl.transaction.TxLog.txStart(TxLog.java:143) at org.neo4j.impl.transaction.TxManager.begin(TxManager.java:502) at org.neo4j.impl.transaction.UserTransactionImpl.begin(UserTransactionImpl.java:47) at org.springframework.transaction.jta.JtaTransactionManager.doJtaBegin(JtaTransactionManager.java:886) at org.springframework.transaction.jta.JtaTransactionManager.doBegin(JtaTransactionManager.java:843) at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:374) ... org.springframework.transaction.CannotCreateTransactionException: JTA failure on begin; nested exception is javax.transaction.SystemException: TM encountered a problem, error writing transaction log,java.nio.channels.ClosedChannelException at org.springframework.transaction.jta.JtaTransactionManager.doBegin(JtaTransactionManager.java:856) at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:374) ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Online backup tests failing
I don't understand the internals well enough but to get online-backup going I removed the log truncate operation from neo core in XaLogicalLog class which seems to conflict with the log still in use. I furthermore removed the timestamp checks in NeoBackup.prepare() because whenever I e.g. use Neoclipse to access either storage file timestamps are changed, leading to failing backups afterwards. I suspect the version should be fine for now. An overall checksum in the Neo store is something to consider in the long run I guess. Can someone please confirm that my above changes are safe? I am happy to provide the patch for these then. It also would be helpful to get some design understanding around online-backups. Otherwise it's all trial and error ;( -Andreas On Thu, May 14, 2009 at 06:35:28PM -0700, Andreas Guenther wrote: Hi, I tried running the online-backup component but most if not all all test cases are failing with the same error pasted below. I hope it's something trivial as I really would like to get the backup feature going. Any ideas? -Andreas at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:997) May 14, 2009 6:19:27 PM org.neo4j.impl.transaction.xaframework.XaLogicalLog renameCurrentLogFileAndIncrementVersion WARNING: Failed to truncate log at correct size java.io.IOException: The requested operation cannot be performed on a file with a user-mapped section open at sun.nio.ch.FileChannelImpl.truncate0(Native Method) at sun.nio.ch.FileChannelImpl.truncate(FileChannelImpl.java:340) at org.neo4j.impl.transaction.xaframework.XaLogicalLog.renameCurrentLogFileAndIncrementVersion(XaLogicalLog.java:753) at org.neo4j.impl.transaction.xaframework.XaLogicalLog.close(XaLogicalLog.java:929) at org.neo4j.impl.transaction.xaframework.XaContainer.close(XaContainer.java:100) at org.neo4j.impl.nioneo.xa.NeoStoreXaDataSource.close(NeoStoreXaDataSource.java:232) at org.neo4j.impl.nioneo.xa.NioNeoDbPersistenceSource.stop(NioNeoDbPersistenceSource.java:71) at org.neo4j.api.core.NeoJvmInstance.shutdown(NeoJvmInstance.java:228) at org.neo4j.api.core.EmbeddedNeo.shutdown(EmbeddedNeo.java:195) at org.neo4j.onlinebackup.Util.stopNeo(Util.java:88) at org.neo4j.onlinebackup.Util.stopNeo(Util.java:94) at org.neo4j.onlinebackup.MultiRunningTest.backup(MultiRunningTest.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.internal.runners.TestMethodRunner.executeMethodBody(TestMethodRunner.java:99) at org.junit.internal.runners.TestMethodRunner.runUnprotected(TestMethodRunner.java:81) at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34) at org.junit.internal.runners.TestMethodRunner.runMethod(TestMethodRunner.java:75) at org.junit.internal.runners.TestMethodRunner.run(TestMethodRunner.java:45) at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod(TestClassMethodsRunner.java:71) at org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:35) at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42) at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34) at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52) at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62) at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140) at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127) at org.apache.maven.surefire.Surefire.run(Surefire.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:338) ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Online backup tests failing
I in fact developed this on Windows but it's supposed to run on Linux eventually. As for the version and Neoclipse changing it, I observed simply starting and stopping it against the repo results in store changes. Hence I have to live with yet another copy of the store. The usage of a backup in my case will be two-folded. One scenario is to have a backup and the other is to route analysis and reporting against the copy. How would I model this scenario because I unfortunately can't have another application point against the copy while taking the backup. Taking yet another copy won't really work for me in the long run solution. I haven't looked at remote neo yet but hope this to be the answer. -Andreas On Fri, May 15, 2009 at 09:51:25PM +0200, Johan Svensson wrote: Hi Andreas, Thanks for reporting this problem. I take it you have been testing this on a Windows machine? I created ticked for the problem with log rotation configured to save logical logs. See https://trac.neo4j.org/ticket/169 Regarding the timestamps, if you access the backup with Neoclipse you could potentially modify the backup so it would not be a snapshot of the original source. Every non backup slave startup of a Neo store will increment the version number of that store so this is expected behavior. If you need to validate the backup and that everything is working correct I would suggest you make a full copy of the backup store to somewhere else then connect with Neoclipse to it. Regards, Johan On Fri, May 15, 2009 at 9:29 PM, Andreas Guenther aguent...@flying-orange.com wrote: I don't understand the internals well enough but to get online-backup going I removed the log truncate operation from neo core in XaLogicalLog class which seems to conflict with the log still in use. I furthermore removed the timestamp checks in NeoBackup.prepare() because whenever I e.g. use Neoclipse to access either storage file timestamps are changed, leading to failing backups afterwards. I suspect the version should be fine for now. An overall checksum in the Neo store is something to consider in the long run I guess. Can someone please confirm that my above changes are safe? I am happy to provide the patch for these then. It also would be helpful to get some design understanding around online-backups. Otherwise it's all trial and error ;( -Andreas On Thu, May 14, 2009 at 06:35:28PM -0700, Andreas Guenther wrote: Hi, I tried running the online-backup component but most if not all all test cases are failing with the same error pasted below. I hope it's something trivial as I really would like to get the backup feature going. Any ideas? -Andreas at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:997) May 14, 2009 6:19:27 PM org.neo4j.impl.transaction.xaframework.XaLogicalLog renameCurrentLogFileAndIncrementVersion WARNING: Failed to truncate log at correct size java.io.IOException: The requested operation cannot be performed on a file with a user-mapped section open at sun.nio.ch.FileChannelImpl.truncate0(Native Method) at sun.nio.ch.FileChannelImpl.truncate(FileChannelImpl.java:340) at org.neo4j.impl.transaction.xaframework.XaLogicalLog.renameCurrentLogFileAndIncrementVersion(XaLogicalLog.java:753) at org.neo4j.impl.transaction.xaframework.XaLogicalLog.close(XaLogicalLog.java:929) at org.neo4j.impl.transaction.xaframework.XaContainer.close(XaContainer.java:100) at org.neo4j.impl.nioneo.xa.NeoStoreXaDataSource.close(NeoStoreXaDataSource.java:232) at org.neo4j.impl.nioneo.xa.NioNeoDbPersistenceSource.stop(NioNeoDbPersistenceSource.java:71) at org.neo4j.api.core.NeoJvmInstance.shutdown(NeoJvmInstance.java:228) at org.neo4j.api.core.EmbeddedNeo.shutdown(EmbeddedNeo.java:195) at org.neo4j.onlinebackup.Util.stopNeo(Util.java:88) at org.neo4j.onlinebackup.Util.stopNeo(Util.java:94) at org.neo4j.onlinebackup.MultiRunningTest.backup(MultiRunningTest.java:132) ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] Which source matches release 1.0-b8?
Hi, I need to hack Neo for some transactional enhancements regarding external resources. At first I intended to use the 1.0-b8 branch but using the tagged version 1.0-b8-2009-01-05 under https://svn.neo4j.org/components/neo it does not seem to match the latest 1.0-b8 release. I tried trunk with the same error and then fell back to 1.0-b7 which in fact works. Can someone please double-check the code base against the latest released version and point me to it? Here are the ones I saw in svn: 1.0-b5/ 1.0-b6/ 1.0-b6-before-opti/ first-import/ neo-1.0-b7/ neo-1.0-b7-pre1/ neo-1.0-b8-2009-01-05/ neo-1.0-rc1-080612/ neo-tx-fix-1.0-rc1-080528/ Furthermore, when using a trunk build, my Neoclipse crashed right away when pointing it to the neo database files. I haven't tried downloading version 4 yet but are wondering what the cause is. Did the underlying storage change? Unless trunk represents 1.0 I generally suggest to keep a version branch around so people don't have to hack the tag, e.g. branches/neo-1.0. Thanks, -Andreas ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] org.neo4j.api.core.NotFoundException: Node[7] not found.
Hi,
I am trying to verify simple transactional behavior of EmbeddedNeo
together with NeoIndexService and run into a strange and rather
inconsistent errors. Attached are the relevant parts from a simple Maven
project I used for testing. Simply issue 'mvn archetype:create -e
-DgroupId=com.neo4j.test -DartifactId=transaction' and copy the files
into the respective folders. Note, that the order in how I execute tests
are relevant. Furthermore, I tried this with and without Spring
integration and Neo 1.0-b7, 1.0-b8, and trunk. So, here goes:
[mvn clean install]
This will lead to both test classes failing due to
'org.neo4j.api.core.NotFoundException: Node[7] not found' exception.
[mvn test -Dtest=org.neo4j.test.TransactionRollbackTest]
Same error as above for this test.
[mvn test -Dtest=org.neo4j.test.TransactionCommitTest]
Surprisingly succeeds.
[mvn test -Dtest=org.neo4j.test.TransactionRollbackTest]
Surprisingly succeeds this time.
[mvn clean install]
Surprisingly succeeds this time for all tests.
Here's also a more detailed example of the exception:
org.neo4j.api.core.NotFoundException: Node[7] not found.
at org.neo4j.impl.core.NodeManager.getNodeForProxy(NodeManager.java:431)
at
org.neo4j.impl.core.NodeProxy.getSingleRelationship(NodeProxy.java:98)
at org.neo4j.util.btree.TreeNode.getFirstEntry(TreeNode.java:181)
at org.neo4j.util.btree.TreeNode.getEntry(TreeNode.java:398)
at org.neo4j.util.btree.BTree.getAsKeyEntry(BTree.java:341)
at
org.neo4j.util.index.AbstractIndex.getSingleNodeFor(AbstractIndex.java:393)
at
org.neo4j.util.index.MultiValueIndex.getSingleNodeFor(MultiValueIndex.java:34)
at
org.neo4j.util.index.NeoIndexService.getSingleNode(NeoIndexService.java:140)
at
org.neo4j.test.TransactionRollbackTest.hasNewNode(TransactionRollbackTest.java:66)
at
org.neo4j.test.TransactionRollbackTest.testProgramaticNeoMultiRollback(TransactionRollbackTest.java:32)
As mentioned above already I tried this test scenario with and without
the Spring test integration and arrived at the same results. Can anyone
explain the reason for these inconsistencies or what I am doing wrong?
Help is very much appreciated!
Thank you,
-Andreas
---
package org.neo4j.test;
import org.junit.runner.RunWith;
import org.junit.Test;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.beans.factory.annotation.Autowired;
import org.neo4j.api.core.NeoService;
import org.neo4j.api.core.Node;
import org.neo4j.api.core.Transaction;
import org.neo4j.api.core.RelationshipType;
import org.neo4j.util.index.IndexService;
import static junit.framework.Assert.assertTrue;
import static junit.framework.Assert.fail;
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = {/applicationContext-test.xml})
public class TransactionCommitTest {
@Autowired
protected NeoService neoService = null;
@Autowired
protected IndexService neoIndexService = null;
@Test
public void testProgramaticNeoMultiCommit() {
String testValue = String.valueOf(System.nanoTime());
assertTrue(No node expected, !hasNewNode(testValue));
insertNewNodeAndCommit(testValue);
assertTrue(Node expected, hasNewNode(testValue));
testValue = String.valueOf(System.nanoTime());
assertTrue(No node expected, !hasNewNode(testValue));
insertNewNodeAndCommit(testValue);
assertTrue(Node expected, hasNewNode(testValue));
}
private void insertNewNodeAndCommit(String testValue) {
Transaction tx = null;
try {
tx = neoService.beginTx();
Node newNode = neoService.createNode();
newNode.setProperty(id, testValue);
neoService.getReferenceNode().createRelationshipTo(newNode,
NeoRelationshipTypes.LINKS_TO);
neoIndexService.index(newNode, id, testValue);
tx.success();
} catch (Exception e) {
e.printStackTrace();
fail();
} finally {
tx.finish();
}
}
private boolean hasNewNode(String testValue) {
Transaction tx = null;
Node node = null;
try {
tx = neoService.beginTx();
node = neoIndexService.getSingleNode(id, testValue);
tx.success();
} finally {
tx.finish();
}
return node != null;
}
public static enum NeoRelationshipTypes implements RelationshipType {
LINKS_TO
}
}
---
package org.neo4j.test;
import org.junit.runner.RunWith;
import org.junit.Test;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.beans.factory.annotation.Autowired;
import org.neo4j.api.core.NeoService;
import
Re: [Neo] org.neo4j.api.core.NotFoundException: Node[7] not found.
Johan, Thanks for opening a ticket. What really is the difference between all of the different index implementations and what is the recommendation to use either one? I even found a LuceneFulltextIndexService. We are currently considering the alternative to use Compass for node id indexing. Besides transactional and maybe future Neoclipse search features, is there any reason to not consider Neo external search functionality? I assume all Neo IndexService implementations are thread-safe ;) -Andreas Johan Svensson wrote: Hi Andreas, I was able to reproduce this. If I start from a clean db and only execute those 2 tests over and over again the 4th run will always fail on the multi rollback test. I created a ticket for it https://trac.neo4j.org/ticket/168 and will look into it as soon as possible. Regards, -Johan On Wed, May 13, 2009 at 8:40 PM, Andreas Guenther aguent...@flying-orange.com wrote: Hi, I am trying to verify simple transactional behavior of EmbeddedNeo together with NeoIndexService and run into a strange and rather inconsistent errors. Attached are the relevant parts from a simple Maven project I used for testing. Simply issue 'mvn archetype:create -e -DgroupId=com.neo4j.test -DartifactId=transaction' and copy the files into the respective folders. Note, that the order in how I execute tests are relevant. Furthermore, I tried this with and without Spring integration and Neo 1.0-b7, 1.0-b8, and trunk. So, here goes: ... ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] MySQL and Neo within same transaction configured in Spring
Johan, Thank you for the pointers. I will give it a shot based on the below comments but look forward to a more proper solution in the upcoming 1.0-b9 branch. In case I am able to come up with anything useful I will keep you posted. -Andreas On Thu, Apr 30, 2009 at 05:12:47PM +0200, Johan Svensson wrote: Hi Andreas, On Thu, Apr 30, 2009 at 7:22 AM, Andreas Guenther aguent...@flying-orange.com wrote: Hi, I am in the process of considering to use Neo for my company. So far I am impressed with the results and capabilities. That said, after a week Good to hear! of getting to know Neo on the feature side I came across an integration obstacle regarding transactions. Basically, I have been trying to use MySQL and Neo together in the same transaction context using JTA/XA and Spring. #1 Is it at all possible to use a single transaction manager for MySQL and Neo, ideally out of the box? It would require some hacking wrapping the MySQL datasource in a org.neo4j.impl.transaction.xaframework.XaDataSource and using Neo's TM (so not really out of the box...). #2 Does anyone have an example on how this could be configured in Spring? I am no good at Spring configuration but I am sure it can be done. What needs to be accomplished is to register the MySQL data source so Neo TM knows about it. Basically that means assigning a name and branch id for it. In code one could do something like: ((EmbeddedNeo) neo).getConfig().getTxModule().getXaDataSourceManager().registerDataSource( name, mySqlXaDsWrapper, branchId ); Not pretty and this is not via the standard Neo API (we have yet to specify the API for configuration like this, it is hard to get it right for both a simple embedded environment and a container). Ideal and what we are aiming for is to be able to run Neo together with other XA resources in Spring using Neo's TM or other. Next week we will be opening the Neo-1.0-b9 branch. We could then try get some code in to simplify your use case. Would that work for you? Regards, -Johan ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] MySQL and Neo within same transaction configured in Spring
Hi,
I am in the process of considering to use Neo for my company. So far I
am impressed with the results and capabilities. That said, after a week
of getting to know Neo on the feature side I came across an integration
obstacle regarding transactions. Basically, I have been trying to use
MySQL and Neo together in the same transaction context using JTA/XA and
Spring.
#1 Is it at all possible to use a single transaction manager for MySQL
and Neo, ideally out of the box?
#2 Does anyone have an example on how this could be configured in Spring?
Below is a snippet I somewhat started but got stuck on:
bean id=jotm
class=org.springframework.transaction.jta.JotmFactoryBean/
bean id=transactionManager
class=org.springframework.transaction.jta.JtaTransactionManager
property name=userTransaction value=jotm/
/bean
bean id=mysqlXADataSource
class=org.enhydra.jdbc.pool.StandardXAPoolDataSource
destroy-method=shutdown
property name=dataSource
bean class=org.enhydra.jdbc.standard.StandardXADataSource
destroy-method=shutdown
property name=transactionManager ref=jotm /
property name=driverName value=${jdbc.driver} /
property name=url
value=${jdbc.url}/test?zeroDateTimeBehavior=convertToNull /
/bean
/property
property name=user value=${jdbc.username}/
property name=password value=${jdbc.password}/
/bean
!-- This doesn't seem right to me, especially since I can't find a
place to tell Neo about jotm --
bean id=neoXADataSource
class=org.neo4j.impl.nioneo.xa.NeoStoreXaDataSource
constructor-arg
map
entry key=store_dir value=target/neo-db/
/map
/constructor-arg
/bean
!-- The following is what I used to start out with based on IMDb
example
bean id=neoTransactionManagerService
class=org.neo4j.impl.transaction.SpringTransactionManager/
bean id=neoUserTransactionService
class=org.neo4j.impl.transaction.UserTransactionImpl/
bean id=transactionManager
class=org.springframework.transaction.jta.JtaTransactionManager
property name=transactionManager
ref=neoTransactionManagerService/
property name=userTransaction ref=neoUserTransactionService/
/bean
--
Thanks in advance,
-Andreas
___
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
