Re: [fossil-users] fossil-scm as an SQLite db with schema/data revision control

[Adam Jensen]

On 07/23/2016 07:05 PM, Ron W wrote:
> Hello,
> 
> What you are thinking is not impossible.
> 

Thanks for the confirmation, I've been speculating about novel system
architectures and unusual use-cases without sufficient fundamental
knowledge and experience - that makes for dangerous propositions and a
very delicate position. Thanks again, for the sanity check.

> You do need to be aware that Fossil only uses SQLite as an index engine.
> Fossil does not synchronize the actual databases, only the "artifacts"
> (stored as "blobs" in the database). Aside from the blob table, the
> tables in the database are derived from the information in the artifacts.
> 
> When information in a Fossil repository is added or updated, Fossil
> creates new artifacts to contain the new or update information, stores
> those in the blob table, then updates the various other tables.
> 

We're probably on the same page here. I've recently opened a fossil
repository file with the sqlite3 application and explored the schema
(superficially).

> When Fossil pushes to or pulls from another repository, only blobs are
> transferred. The receiving Fossil adds those to it's blob table, then
> updates the other tables as needed.
> 

I think I get the gist of what you are saying. At some point it would
probably be worthwhile [for me] to explore the communication protocol.

> Fossil could be used to synchronize 2 or more SQLite databases. Schema
> updates could be handled as tickets, the body of each ticket containing
> the SQL commands required to make the schema changes. For new clones of
> the database, a serialized version of the schema could be used, rather
> than "replaying" the schema tickets. Synchronizing tables would require
> serializing their content into artifacts, which Could then be
> pushed/pulled normally by Fossil.
> 
> The serialization of the schema and the table contents would have to be
> done by an external application. Maybe libfossil could help with that.
> Otherwise, each new artifact could be created in to a file, then the
> files committed into Fossil using system() calls to invoke Fossil's
> command interface. Alternately, the artifacts could be added as wiki
> pages via Fossil's HTTP interface (which would avoid launching the
> Fossil application multiple times, though the push/pull would still have
> to be done via the command interface).
> 

That might be an interesting approach to putting together a prototype
demonstration-of-concept system, just using standard Fossil and SQLite
and some scripting.

[The Session Extension](http://www.sqlite.org/sessionintro.html) pointed
out by Eduardo seems to have a lot of potential.

"The session extension provides a mechanism for recording changes to
some or all of the rowid tables in an SQLite database, and packaging
those changes into a "changeset" or "patchset" file that can later be
used to apply the same set of changes to another database with the same
schema and compatible starting data. A "changeset" may also be inverted
and used to "undo" a session."

Wow, whoever designed that extension had Vision.

Given this capability, would you still use Fossil tickets to record,
manage, and communicate the changesets or do you suppose the changesets
(or some variation) might somehow be stored in the blob table?

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil-scm as an SQLite db with schema/data revision control

[Eduardo Morras]

On Thu, 14 Jul 2016 14:31:46 -0400
Adam Jensen  wrote:

> I sent a message last night before joining the mailing list (today)
> and I'm not certain if it (last night's post) was distributed to the
> list or not. It's not displayed in the list archive yet. Just in
> case, here it is:
> 
> On 07/13/2016 06:18 PM, Adam Jensen wrote:
> > This might be a bit convoluted but is it possible to use fossil-scm
> > as an SQLite db with the schema/data under revision control? If this
> > functionality doesn't already exist in fossil, would it be
> > difficult, awkward, or crazy to try to implement it?
> > 
> > It seems like such an obvious thing to do that I guess it either
> > already exists or it's so technically twisted that it's virtually
> > impossible.
> > 
> > If this functionality doesn't exist, how would you do it?
> 
> After thinking a little more about this, it seems like maybe this
> isn't something that should be incorporated into Fossil [or SQLite]
> but rather this could be a third system that is based on the other
> two.
> 
> To add a little more verbiage and description to the basic idea, what
> I am imagining is (I'm going to assume that in this new system a
> single db file can support multiple databases) an SQLite database
> with version control on the schema and data. This might be
> accomplished in a fashion similar to the original SCCS - every SQL
> command that causes changes to the [working copy] database is saved
> in the version history (a fossil-like database in this common db
> file). With this, any version of the database could be recreated by
> replaying the history. This process could be sped up by saving or
> creating snapshots of the database at various moments in its history
> (all in the common db file (and, perhaps, read-only)) and then a
> specific version can be created by replaying the relevant set of SQL
> changes made after the snapshot. (This description is just to get the
> idea across; there are probably more clever ways to implement it).

You can use the Sqlite session extension to produce patch files between
db file versions. It has some restrictions on schema and documentation
is scarce, but if I had the same goal as you, I'll try that way.

To use it you need Sqlite version >= 3.13.0

www.sqlite.org/sessionintro.html
 
> Another aspect of this third system is the typical fossil-like data
> sharing. So basically, this could be a distributed, multi-user
> database. Each user would have a local copy (fast access) with their
> own branches (write control) but there could be data sharing through
> merges. (This probably sounds obvious to fossil users but think about
> it from the perspective of sharing database content rather than
> sharing a pile of files (source code).
> 
> For fun, and somewhat whimsically, a name for this Fossil/SQLite
> hybrid might be DVCSQLite .


---   ---
Eduardo Morras 
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users