Re: [sqlite] Newbie coherency questions ... poor man's replication? Proper replication approaches?

2005-10-04 Thread Bryan J. Smith
Jay Sprenkle <[EMAIL PROTECTED]> wrote:
> You're still going to have to understand locking semaphores
> and threads.

Yeah, I was thinking about that.
There are still structures in memory and other access issues.

> All sqlite does for you is give you a good interface
> to access the data, transactions, and hardware fault
> tolerance.

Yep.  I forgot some of my fundamentals there for a second.


-- 
Bryan J. Smith| Sent from Yahoo Mail
mailto:[EMAIL PROTECTED] |  (please excuse any
http://thebs413.blogspot.com/ |   missing headers)


[sqlite] Newbie coherency questions ... poor man's replication? Proper replication approaches?

2005-10-03 Thread Bryan J. Smith
I've heard about this project for so many years, but I don't
know why I ignored it until now.  I was just reading the FAQ
on coherency and threading:  
  http://www.sqlite.org/faq.html#q7  

I believe this is outstanding for an embedded DB engine,
something I've never seen in anything else.  If I am assuming
correctly, while SQLite may not be a "true" multi-write DB in
the eyes of some -- I assume by having difference processes
locking control access to the DB, you can effectively have
multi-write (just one at a time), correct?

I assume that only works between programs using SQLite
methods in their code?  Or does it mean I could possibly test
for control, lock the DB and even rsync the DB file remotely?
 I assume not, but I have to ask if this "poor man's
replication" approach could work.  Assuming that would not,
so read on (and please comment on) what I believe might be
more proper design.

We're building a multi-master, peer-to-peer network service,
so we need a way to replicate changes between systems. 
Before finding SQLite, I figured I'd have to write an
"all-in-one daemon."  E.g., an application that handles the
app's data itself with client service access plus peer
replication service access to the same DB -- probably with
threads, semaphores and other internal locking issues.

But now I think SQLite solves the problem nicely, if I'm
reading the FAQ correctly.  What I'm considering is a 2
daemon approach, one for the app/client services and another
for the replication services.  I would just thread as many
daemons as configured for each (creating a new SQLite DB
connection instance to the file for each thread, per FAQ #8),
and handle client service changes and replication service
changes appropriately with a write lock.

If that is not feasible/recommended, alternatively, I was
thinking of keeping separate databases for the peer
replication, whereby the app/client services daemon and
replication services daemon never have write access to each
other's databases.  There would be incoming databases of
changes from each peer, and a single outgoing database for
all other peers (of local changes)

Or should I even be looking at maintaining local copies of
remote databases, possibly using an rsync at startup, and
then doing local resolution?  I guess there's a lot of
options/issues here, and I'm looking for suggestions/best
practices.

I know this is a broad set of questions/commentary, but any
insight into what SQLite can and can't do, and what practices
are best for setting up a multi-master, peer-replicating
service (of which, each "master" is also servicing clients,
which are where the changes initially come from) would be
ideal.

Thanx in advance.


-- 
Bryan J. Smith| Sent from Yahoo Mail
mailto:[EMAIL PROTECTED] |  (please excuse any
http://thebs413.blogspot.com/ |   missing headers)