Andy, others,
we do not use SDB because it is way too slow for us. Although I'm sure it
can be improved as you suggested below, we do not believe it will ever
come close to TDBs performance because of how SDB is designed. The
fact it
is used at all keeps surprising me, but it probably doesn't matter for
simple use cases especially if the dataset remains small. Btw, a little
while back we were reconsidering it because SDB supports multiple vendors
and our use-case was not very performance sensitive, but it turned out
that it was still too slow for our needs
As for motivations why some people may prefer SDB over TDB, I don't think
it is just "SQL" and corporate acceptance. There are some very good
reasons why file-based systems like TDB are difficult to use in
commercial
deployments. Corporate Java-based server deployments are almost always
based on one or more app servers (JEE, but sometimes just a regular web
server) where *all* persisted state goes into a relational database.
Storing state on the file system is generally taboo for many reasons
including: The inability to cluster the app server - a critical step to
scale up beyond a few hundred users; the fact that organizations
generally
have a hard time understanding and managing file system state as opposed
to standard relational database management. For instance, how do you
perform online backups and when and how does the state corrupt. For
instance, on a DB server, admins are watchful for running out of disk
space. This sort of monitoring is usually less critical on the app
server,
but now, suddenly, there is this growing "thing" they have to backup
because it will corrupt if you run out of disk space. On top of that, DB
servers usually use very fast and expensive disk systems (RAIDs, SSDs,
etc.) This is usually not the case for the app server. On top of that,
when customers realize there is this large set of data on the file
system,
they have a tendency to put it on larger disks connected via NFS,
which is
unfortunately very dangerous because even short network glitches can
corrupt TDB. All of this is manageable if you carefully encapsulate TDB
and provide good administration tools on top of your system, but it is
not
trivial, and it doesn't come out of the box. Especially cluster
management
is quite tricky as you can imagine.
Just wanted to set that out there as to why SQL-based systems remain
attractive. I would advise though to more clearly state on the SDB
download page that SDB is deprecated and no longer actively supported
(unless that changes of course)
Simon
From:
Andy Seaborne <[email protected]>
To:
[email protected],
Date:
05/25/2013 06:19 AM
Subject:
[DISCUSS] SDB future
Yes - I'm conflicted as well, flip-flopping between opt1 and opt3.
There is enough user@ traffic to suggest it's used. I'm guessing it's
the "SQL" part makes it easier in corp IT. TDB is faster, scales better
and is better supported (and I'm not corp IT bound).
There are ways to improve it's performance - pushing some filters into
SQL for example - so theres scoep for development.
Option 1 - add to the main distribution, remove if it becomes a block -
means that there is no additional work on a release vote.
Testing SDB using Derby only (that's what the junit does by default) is
easy to setup because it's pulled in by maven. It only runs embedded,
not as a server but it does check the code generation. Unliek other
jjava SQL DBs, Derby implements tree join plans (the other onyl do
linear join plans which makes some optional cases impossible - the code
fall back to brute-force-and-ignorance in these cases). Derby is quite
picky about it's SQL 92. SDB tests without additional setup.
We state on users@ this is the position as a indication that we stil
have option 3 (retirement) available. Unless we shake the tree a bit,
Proposal: (option 1)
add to apache-jena, remove at the first sign of trouble.
make a clear statement of situation on users@ including
encouraging people to come forward
option 3 still on the cards.
I've added DISCUSS to the subject line for now to leave open the
possibility of a vote because it affects the whole project.
But all PMC chipping in here is enough.
I don't see a rush to make a choice just yet.
Andy
On 22/05/13 11:47, Claude Warren wrote:
I am conflicted about this one. I think we need SDB (or something
similar)
that will allow users to use standard infrastructure (shared/pooled DB
is
fairly common). But I don't have the bandwidth to support it.
Is there a status where in we release SDB package with each release of
Jena
but only ensure that the current test cases work -- that is the latest
release doesn't break something? Perhaps with a reduced set of
supported
DBs (perhaps Derby & MySQL)
If not then I think we take Andy's approach and release one more before
putting it on the shelf.
On Wed, May 22, 2013 at 10:35 AM, Charles Li
<[email protected]>wrote:
What are alternatives to SDB? I have a 4GB RDF/XML to load for later
queries
Thanks!
- Charles
On May 21, 2013, at 12:00 PM, Stephen Owens
<[email protected]>
wrote:
+1 for option 3 if no one currently is taking ownership of that
project.
I think it's a useful signal to potential adopters about what they
should
expect.
On 2013-05-21, at 12:47 PM, Andy Seaborne <[email protected]> wrote:
SDB is getting some user attention but not much developer attention.
I
was hoping that there would be a contribution to go with JENA-447 but
nothing has come in. I don't have the bandwidth to even answer
questions
about it properly, partly because I don't use it. I guess others are
in a
similar position.
I do think we should be clear as to it's status.
In the future, I see these options:
1/ Add jena-sdb to the main distribution.
(If it becomes a block on a release, remove it.)
2/ As is - release "sometimes".
3/ Dormant SDB.
This is the last release unless some activity arises to maintain
it.
Keep the source around but move out of trunk.
Can be built from source.
4/ Legacy SDB.
More definite statement than (3) that it is dropped.
Keep the source around.
For 3 and 4, where there are no plans to release again if nothing
changes, the snapshot builds should be stopped. Users can build from
source if they want to but the current snapshot should not become a
distribution-under-the-radar which I feel it becomes if there are no
plans
to make it a formal release.
Thoughts?
I'm tending towards doing this one last release then (3).
Andy
