Another issue to consider when evaluating file system versus database is
synchronizing your database backup with the file system backup if you are
keeping file state information in the database. I've managed a couple of
document systems where the documents are in a file system and the state of
the doc is in the database. It was difficult to do any type of incremental
backup that kept the database and the documents in synch. We ended up do
full backups everytime just to be safe.
Jim Berg
Sr. Software Specialist
PSCI
[EMAIL PROTECTED]
"Gavin Terrill"
<[EMAIL PROTECTED]>
To: "Struts Users Mailing List"
03-Dec-2001 11:03
Please respond to cc:
"Struts Users Mailing Subject: RE: When to store files
in DB vs. File system
List"
<[EMAIL PROTECTED]
pache.org>
I would recommend using the file system, and storing the location in the
db:
- the file system is faster
- storing files in the DB tends to generate enormous database physical
files, so back ups become an issue
- integrating with a freetext search engine becomes harder (you may need
this down the track)
We have not seen issues with the number of files in a directory,
although we also partition files across directories to minimize the
number of files and avoid OS limitations (a simple way to do this is to
use YYYY\MM\DD as the directory structure).
<vendor>
You may also want to check out: http://www.karora.com/DocConn.htm
</vendor>
Gavin Terrill
CTO, Karora Technologies Inc.
http://www.karora.com
-----Original Message-----
From: Ted Husted [mailto:[EMAIL PROTECTED]]
Sent: Monday, December 03, 2001 6:48 AM
To: Struts Users Mailing List
Subject: Re: When to store files in DB vs. File system
I've seen a lot of implementations do this sort of thing, especially
when archive emails and the like.
I've also heard that people find many OS's start to slow down with even
1000 files in a directory, so I believe many such systems use that as an
increment /1/000 - /1/999.
A general piece of good advice is that databases excel at storing
itty-bitty pieces of information, and files systems are good for storing
a larger blocks. If you decouple the actual files from the rest of the
database, you also have the opportunity to centralize that stored on a
machine optimized for that purpose.
If you need something to index those files, I'd have a look at Lucene
;-)
http://jakarta.apache.org/lucene
-- Ted Husted, Husted dot Com, Fairport NY USA.
-- Custom Software ~ Technical Services.
-- Tel +1 716 737-3463
-- http://www.husted.com/struts/
Darryl Nortje wrote:
>
> Hi.
>
> Everyone is going to tell you that you must store it in the database.
> The reason are that in different OS's, there is a limit to how many
> files you can store in each directory. (windows i think is 10000). I
> was faced with this same problem under a time constraint, so never
> took the time to figure out how to store the file directly into the
> database. So i went the a directory storing route. What I did was
> Create a directory root\1\1a\1aa\file.extension like this. Set a limit
> to the amount of files in this directory and when that limit was
> reached incremented the directory name to root\1\1a\1ab etc etc. And
> then just stored the directory and filename in the database and
> retrieved it like that. I'm pretty sure that this is not the correct
> way to do this sort of thing, but it does work. I will be watching
> this mail to find out how to do it properly.
>
> -----Original Message-----
> From: Andy Timm [mailto:[EMAIL PROTECTED]]
> Sent: Monday, December 03, 2001 9:53 AM
> To: [EMAIL PROTECTED]
> Subject: When to store files in DB vs. File system
>
> Hello, I am developing a webapp where users need to
> store files (<200k). I am wondering if these should
> be stored in the (mysql) database, or in the file
> system. I am concerned about the number of files I
> could get in the database (possibility for 20,000+
> users, so 100,000 files). On the other hand, I plan
> on replicating my database across multiple servers.
> Do I then also replicate the download of each file, or
> run a batch that copies them periodically? I would
> greatly appreciate advice from anyone who has dealt
> with these design issues. Thank you, Andrew Timm
>
> __________________________________________________
> Do You Yahoo!?
> Buy the perfect holiday gifts at Yahoo! Shopping.
> http://shopping.yahoo.com
>
> --
> To unsubscribe, e-mail:
> <mailto:[EMAIL PROTECTED]>
> For additional commands, e-mail:
> <mailto:[EMAIL PROTECTED]>
>
> --
> To unsubscribe, e-mail:
<mailto:[EMAIL PROTECTED]>
> For additional commands, e-mail:
> <mailto:[EMAIL PROTECTED]>
--
To unsubscribe, e-mail:
<mailto:[EMAIL PROTECTED]>
For additional commands, e-mail:
<mailto:[EMAIL PROTECTED]>
--
To unsubscribe, e-mail: <
mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <
mailto:[EMAIL PROTECTED]>
--
To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
