Re: [docbook-apps] Re: XML databases

[Dave Pawson]

Declaration: I'm not a db user (never found a real use for them).
My docbook (primary) use is the accretion of files over time.
Mostly less than a few hundred.
Build tool is plain text (ant / xslt).

My logic (not wishing to call Norm a liar) is that a file system can fail
(I do take care).
Adding another point of failure (the db logic...) makes a db 'worse' i.e.
more likely to fail.

I get indexing via XSLT (thanks docbook team) where I need it.
I believe Norm when he says that a db indexes 'better' (for some
definition of better - what of secondary terms / index term != content?)

So for my 'small' (relative) number of files, coupled with my db ignorance
(I did spend time once playing with one of the XML db products, v.easy
to import)
I'm biassed towards simplicity... One less point of failure.

I'm sure there is a gray area in between high file count / low complexity etc
where both are 'equal'. Forward planning perhaps?

Not wishing to tempt fate, I won't say "No failures in n years!"  :-)

Dave




On 7 April 2018 at 09:06, Peter Fleck  wrote:
> This is a useful discussion.
>
> We make extensive use of git, especially at the beginning of a project.
>
> Do you have any tips for integrating the database approach with a vcs like
> git?
>
> Peter
>
>
> On 06/04/2018 12:35, Norman Walsh wrote:
>>
>> Camille Bégnis  writes:
>>>
>>> thanks for this interesting discussion, what DB would you use or suggest
>>> for XML?
>>
>> I’m strongly biased to suggest a particular commercial database, one
>> that you can download and use for free from developer.marklogic.com.
>>
>> But I hear good things about BaseX as well and eXist has been around
>> for ages.
>>
>>  Be seeing you,
>>norm
>>
>
>
> -
> To unsubscribe, e-mail: docbook-apps-unsubscr...@lists.oasis-open.org
> For additional commands, e-mail: docbook-apps-h...@lists.oasis-open.org
>



-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.

-
To unsubscribe, e-mail: docbook-apps-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: docbook-apps-h...@lists.oasis-open.org

Re: [docbook-apps] Re: XML databases

[Peter Fleck]

This is a useful discussion.

We make extensive use of git, especially at the beginning of a project.

Do you have any tips for integrating the database approach with a vcs 
like git?


Peter


On 06/04/2018 12:35, Norman Walsh wrote:

Camille Bégnis  writes:

thanks for this interesting discussion, what DB would you use or suggest
for XML?

I’m strongly biased to suggest a particular commercial database, one
that you can download and use for free from developer.marklogic.com.

But I hear good things about BaseX as well and eXist has been around
for ages.

 Be seeing you,
   norm




-
To unsubscribe, e-mail: docbook-apps-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: docbook-apps-h...@lists.oasis-open.org

[docbook-apps] Re: XML databases

[Norman Walsh]

Camille Bégnis  writes:
> thanks for this interesting discussion, what DB would you use or suggest
> for XML?

I’m strongly biased to suggest a particular commercial database, one
that you can download and use for free from developer.marklogic.com.

But I hear good things about BaseX as well and eXist has been around
for ages.

Be seeing you,
  norm

-- 
Norman Walsh  | There has never been a perfect
http://nwalsh.com/| government, because men have passions;
  | and if they did not have passions,
  | there would be no need for
  | government.--Voltaire


signature.asc
Description: PGP signature

Re: [docbook-apps] Re: XML databases

[Camille Bégnis]

Hi all,

thanks for this interesting discussion, what DB would you use or suggest
for XML?

Camille.


Le 06/04/2018 à 11:43, Norman Walsh a écrit :
> Dave Pawson  writes:
>> Agree with your logic. Good for thousands (hard to index)
>> Less so for hundreds (I use db indexing)
> Yes, but as I said, it depends on the app you’re trying to build.
>
> drinks.nwalsh.com: ~200 small documents, easy to build in a DB, harder 
> outside.
> so.nwalsh.com: ~600 documents and growing, easy to build in a DB, *much* 
> harder outside
> photos.nwalsh.com: ~14,000 documents, same story
> tzinfo.nwalsh.com: ~225,000 documents, probably not practical any other way
>
> (Drinks.nwalsh.com is a simple app, you could do that off the
> filesystem with a little bit of Python and some cleverness.
> So.nwalsh.com would be much harder because it’s using full-text,
> semantic, and geospatial indexes and runs queries in real time that
> rely on those indexes to perform well.)
>
>> I'd hate to have the data in a corrupt database
> Or a corrupt filesystem. I don’t think databases are inherently a
> riskier place to put your data. And if having them in a database
> encourages you to have a more reliably backup strategy, they’re
> arguably less risky.
>
> Backup early. Backup often. And remember: if you copy data to a backup
> drive, then remove the data from your computer, you don’t have a
> backup, you have a vulnerable data set on a single external drive.
>
> Be seeing you,
>   norm
>


-
To unsubscribe, e-mail: docbook-apps-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: docbook-apps-h...@lists.oasis-open.org

[docbook-apps] Re: XML databases

[Norman Walsh]

Dave Pawson  writes:
> Agree with your logic. Good for thousands (hard to index)
> Less so for hundreds (I use db indexing)

Yes, but as I said, it depends on the app you’re trying to build.

drinks.nwalsh.com: ~200 small documents, easy to build in a DB, harder outside.
so.nwalsh.com: ~600 documents and growing, easy to build in a DB, *much* harder 
outside
photos.nwalsh.com: ~14,000 documents, same story
tzinfo.nwalsh.com: ~225,000 documents, probably not practical any other way

(Drinks.nwalsh.com is a simple app, you could do that off the
filesystem with a little bit of Python and some cleverness.
So.nwalsh.com would be much harder because it’s using full-text,
semantic, and geospatial indexes and runs queries in real time that
rely on those indexes to perform well.)

> I'd hate to have the data in a corrupt database

Or a corrupt filesystem. I don’t think databases are inherently a
riskier place to put your data. And if having them in a database
encourages you to have a more reliably backup strategy, they’re
arguably less risky.

Backup early. Backup often. And remember: if you copy data to a backup
drive, then remove the data from your computer, you don’t have a
backup, you have a vulnerable data set on a single external drive.

Be seeing you,
  norm

-- 
Norman Walsh  | Limited in his nature, infinite in his
http://nwalsh.com/| desires, man is a fallen god who
  | remembers heaven.--Lamartine


signature.asc
Description: PGP signature

Re: [docbook-apps] Re: XML databases

[Dave Pawson]

Thanks Norm.
   It's the fear of losing it all that holds me back.

Agree with your logic. Good for thousands (hard to index)
Less so for hundreds (I use db indexing)

Also agree with the backup (in either case).

I'd hate to have the data in a corrupt database

regards

On 5 April 2018 at 16:59, Norman Walsh  wrote:
> Dave Pawson  writes:
>> What use cases are there for dropping a few hundred
>> XML files into a (purpose built for XML) database?
>
> I put XML in a database for the ability to index and search it,
> primarily. Here’s a screenshot of my personal “evernote clone” that
> stores a combination of XML and other formats.
>
>
>
> The documents that contained the word DocBook (stemmed appropriately,
> so DocBooking and DocBooked, if they were words, would also have
> matched) are found quickly. The facets are constructed from other
> fields in the those documents.
>
> The ability to quickly search and use indexes to build facets allows
> me to make an application that would be more difficult without a
> database.
>
>>  I can see a risk (db failure) above the file system
>> failure risks.
>
> Backups. You want to have backups!
>
>> Has anyone done that assessment and decided in
>> favour of a database over the file system?
>
> For a few hundred documents, it’s probably hard to make a compelling
> argument for a database unless you want to build applications like the
> one I described above.
>
> For a few hundred thousand documents, ti’s probably hard to make a
> compelling case for the filesystem.
>
> Be seeing you,
>   norm
>
> --
> Norman Walsh  | The finest amusements are the most
> http://nwalsh.com/| pointless ones.--Jacques Chardonne
>



-- 
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.

-
To unsubscribe, e-mail: docbook-apps-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: docbook-apps-h...@lists.oasis-open.org

[docbook-apps] Re: XML databases

[Norman Walsh]

Dave Pawson  writes:
> What use cases are there for dropping a few hundred
> XML files into a (purpose built for XML) database?

I put XML in a database for the ability to index and search it,
primarily. Here’s a screenshot of my personal “evernote clone” that
stores a combination of XML and other formats.


The documents that contained the word DocBook (stemmed appropriately,
so DocBooking and DocBooked, if they were words, would also have
matched) are found quickly. The facets are constructed from other
fields in the those documents.

The ability to quickly search and use indexes to build facets allows
me to make an application that would be more difficult without a
database.

>  I can see a risk (db failure) above the file system
> failure risks.

Backups. You want to have backups!

> Has anyone done that assessment and decided in
> favour of a database over the file system?

For a few hundred documents, it’s probably hard to make a compelling
argument for a database unless you want to build applications like the
one I described above.

For a few hundred thousand documents, ti’s probably hard to make a
compelling case for the filesystem.

Be seeing you,
  norm

-- 
Norman Walsh  | The finest amusements are the most
http://nwalsh.com/| pointless ones.--Jacques Chardonne


signature.asc
Description: PGP signature