Andy, this is really great to hear about, congratulations! A great beginning to the new year. A few general questions and then a few questions in-line.
* Is the source in the TDB2 sections of https://github.com/afs/mantis ? * What kind of trajectory do you expect this project to take this year (e.g. towards integration as part of the Jena release)? * What kind of input are you hoping for immediately? I.e. are you looking for active development contributors, sites willing to do testing at scale, or just basic feedback on the code itself and small-scale testing? * Do you have a sense of how far away from production-readiness this code is? Is there anything missing that could be supplied to change that timeline? * Do you expect the ideas about clustering and distribution with which you've been working to come back into TDB2? --- A. Soroka The University of Virginia Library > On Jan 1, 2017, at 12:17 PM, Andy Seaborne <[email protected]> wrote: > > TDB2 is an upgrade of TDB. It provides fully scalable transactions,e.g. > loading 100's of million triples into a live database. <snipped> > So what are the differences between TDB1 and TDB2? > > = Indexes <details snipped> Given the common use of persistent/functional structures between TIM and TDB2, do you expect us eventually to be able to factor out common behavior? Or is the difference between in-memory and on-disk budgeting too great, or is it just too soon to say? > = Nodes > > The node data is now held in a binary form (using RDF/Thrift). Does that mean just the same as: https://jena.apache.org/documentation/io/rdf-binary.html ? <snipped> > = Transactions > > There is a completely new transaction mechanism. It is now a general > framework that can work with multiple components. A TDB2 database is a > number of such components - one per index, and also the node table. It could > be enhanced to provide multiple dataset transactions and work with external > indexes. The API on datasets is unchanged. Along the lines of my ask above about persistent structures and TIM and TDB2, do you expect us eventually to be able to migrate Jena itself to use this new more-flexible approach? Does the new approach finally separate threads and transactions? <snipped> > == Possibilities > > Given this design, some features are possible, i.e. could be done but aren't. > > "See into the past" - a read-transaction can be started that sees some > specific committed state from the past, not the latest commit. The database > does not forget any committed changes unless storage is reclaimed. TDB2 is still MRSW, not MR+SW?
