Next release of Neo4j (b9) will have an option not to use memory mapped buffers but instead either use direct buffers or normal Java buffers. A .NET port could replace the org.neo4j.impl.nioneo.store.MappedPersistenceWindow with a different implementation that does not use the MappedByteBuffer (b9 will soon have such an implementation).
In the transaction package any usage of MappedByteBuffer can be replaced with a direct or normal buffer also (they all just append data to some file). Hopefully I will get my test code of the new persistence windows into trunk soon. Once that is done a .NET port could make use of that and replace all other usages of memory mapped I/O with normal buffers. Regards, Johan On Sun, May 3, 2009 at 10:03 PM, Antonello Provenzano <antone...@deveel.com> wrote: > I dug a bit the NIO package and the use done in neo4j: unfortunately > there's not counter part for it on .NET nor the possibility to > recreate [easily] classes to support it. In fact, you make a wide and > important use of MappedByteBuffer on files: although implement the > logics of a ByteBuffer class is not difficult (in .NET schema is the > association of two classes: MemoryStream and > BinaryWriter/BinaryReader), the behavior of MappedByteBuffer appears > to be more complex, since access to portion of data in file which > access must be coordinated with the memory, and this is not a simple > issue. > > As of Java implementation, this is done natively and it's a complex > operation to replicate the same functionality. > > Once this is solved, I see no further [major] obstacles to the port: > avoiding the integration with the System.Transactions mechanism, the > base of neo4j port would work in its core functionalities. > > Cheers. > Antonello > > > On Sun, May 3, 2009 at 4:51 PM, Johan Svensson <jo...@neotechnology.com> > wrote: >> See my comments inline: >> >> On Sun, May 3, 2009 at 4:05 PM, Antonello Provenzano >> <antone...@deveel.com> wrote: >>> Johan, >>> >>> I see. My knowledge of NIO is pretty limited indeed: since you're >>> telling me it's re-designable with traditional java.io >>> implementations, I see no particular issues on that side (also >>> because, as you described some of the core functionalities, many >>> features are already present in System.IO). In fact, what i did know >>> was that NIO was used to stream remotely, opening communication >>> channels between the local JVM and the remote endpoint (which could >>> have been another running application in the same JVM). >>> >> >> Yes, but we only use the native I/O part for files. Simply put, we >> want to avoid the extra copy of buffers that takes place from outside >> jvm memory to jvm buffers. NIO allows us to allocation direct buffers >> (that are allocated and managed outside the normal java heap). I also >> like the design of the nio package with channels and it is good that >> you can use the same interfaces to shuffle data over the network but >> Neo does not use those features (yet). >> >>> Coming back to Transactions: it's true that JTA is just a set of >>> specifications abstracted by the interfaces provided (which are easily >>> convertible into .NET with a custom JTA packaging), but it's also true >>> that .NET provides a "framework" for handling application transactions >>> in the same fashion as JTA. >>> In fact, when a developer instantiate a TransactionScope class, opens >>> a transactional context for th execution of the calls withing that >>> instance: to support this the JTA support present in neo4j should be >>> adapted to the specifications provided by System.Transactions, which >>> don't support XA nor 2PC (heavily used by neo4j). >>> >> >> It should be possible to port Neo4j's transaction handling to support >> .NET's way of handling transactions. Neo will only make use of 2PC if >> there are multiple resources participating in the transaction. If you >> only use Neo and nothing else all commits will be optimized one phase >> commits. >> >> How would .NET handle a transaction that would like to modify >> something in two different databases in the same transactional >> context? That is what we use the two phase commit protocol for but if >> you only use Neo4j there is only one resources enlisted in the >> transaction so then it is not needed. >> >>> It's true that it would be possible to create a custom transaction >>> namespace within the ported neo4j, but this would change the behavior >>> of the application: I don't know for sure, but JTA interacts with >>> underlying systems for the management of transactions, as >>> System.Transaction does; having a custom transaction mechanism would >>> bypass all this. >>> >> >> Yes, also it might be possible to just write some glue code between >> the .NET TransactionScope and the NeoService.beginTx() & >> org.neo4j.api.core.Transaction classes. If all you need is single >> resource, begin,commit and rollback it would be much easier to >> integrate there and just do a straightforward port of the transaction >> package. >> >> Regards, >> Johan >> >>> Cheers. >>> Antonello _______________________________________________ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
