On Thu, Mar 1, 2012 at 11:49 AM, Andy Seaborne <[email protected]> wrote: > > The TDB bulk loaders all work on the assumption they are building a new > database from empty. You give all the files at once to the bulkloader.
OK that's good to know; I take the point that the incremental loading is meant for occasional small updates rather than staged bulk loads. > The other point that is worth mentioned when working with a lot of data is > to make sure that the data is all valid - if some RDF is broken but only > encountered late in the load,then a lot of time is used only to be thrown > away. Thanks - the RDF files in this particular case are OK but I appreciate the advice about checking first. >> Is TDB even appropriate for this? Would (say) a MySQL-backed SDB >> instance be better? Or three separate TDB instances? Obviously the >> later would require some sort of query federation layer. > > SDB does not scale as well. OK, that's good to know. > If there is a natural split to 3 (say) servers, then that might work but > without a clever query federation layer the details get exposed to the > application. In this case I'd prefer to keep everything on one server; I'll let you know how I get on. Regards Glenn.
