2007/1/6, Maurice Aubrey [EMAIL PROTECTED]:
Clodoaldo wrote:
But the main factor to push me in the file system direction is the
HTTP cache management. I want the internet web clients and proxies to
cache the images. The Apache web server has it ready and easy. If
the images where to be
apache has very good page and image caching. You could take advantage
of that using this technique.
I wonder why this HTTP cache headers argument didn't surface in this
heated debate.
I did other up this argument by the way.
Andrew
Clodoaldo wrote:
2007/1/5, Jorge Godoy [EMAIL
Clodoaldo wrote:
But the main factor to push me in the file system direction is the
HTTP cache management. I want the internet web clients and proxies to
cache the images. The Apache web server has it ready and easy. If
the images where to be stored in the DB I would have to handle the
HTTP
Clodoaldo wrote:
But the main factor to push me in the file system direction is the
HTTP cache management. I want the internet web clients and proxies to
cache the images. The Apache web server has it ready and easy. If
the images where to be stored in the DB I would have to handle the
HTTP
My point is: if I need to be 100% sure that what is referenced on the database
is accessible all the time when the reference is, then I need to have this on
the database
Not necessarily. It does take carefully controlling access, with a good deal
of thought and error-checking on the part of
Clodoaldo wrote:
5 Jan 2007 06:59:18 -0800, imageguy [EMAIL PROTECTED]:
I think I know the answer,
If you know the answer please tell it as I have read some discussions
on the web and although I have decided on a solution I'm still not
sure about the best answer, if there is a best
2007/1/5, Jorge Godoy [EMAIL PROTECTED]:
Andrew Chernow [EMAIL PROTECTED] writes:
meet those requirements. It is far more effecient to have apache access
them
Where weren't we meeting his/her requirements? All the discussion is around
available means to do that. One option is having the
2007/1/6, Andrew Chernow [EMAIL PROTECTED]:
apache has very good page and image caching. You could take advantage
of that using this technique.
I wonder why this HTTP cache headers argument didn't surface in this
heated debate.
I did other up this argument by the way.
Sorry, I
On 1/6/07, Jorge Godoy [EMAIL PROTECTED] wrote:
Andrew Chernow [EMAIL PROTECTED] writes:
I mean, how do you handle integrity with data
outside the database?
You don't, the file system handles integrity of the stored data. Although,
one must careful to avoid db and fs orphans. Meaning, a
I wonder why this HTTP cache headers argument didn't surface in this
heated debate.
I mentioned this earlier as well. Although you could do it in the app
layer - it would be easier to just let the web server handle it.
---(end of broadcast)---
Is there any overwhelming reason you can't just stick an apache server
on your DB server? Unless you expect this thing to get hit *hard*, the
performance of having them both on the same machine is pretty acceptable
(I know, everyone's opinion about what constitutes heavy usage
differs). If
On 1/5/07, Jorge Godoy [EMAIL PROTECTED] wrote:
Andrew Chernow [EMAIL PROTECTED] writes:
And how do you guarantee that after a failure? You're restoring two
different sets of data here:
How do you link them together on that specific operation? Or even on a daily
basis, if you get
Scott Ribe wrote:
Personally, I'd put them on the file system, because then backup software
can perform incremental backups. In the database, that becomes more of a
difficulty. One suggestion, don't use a file name from a hash to store the
image, just use the serial id, and break them up by
5 Jan 2007 06:59:18 -0800, imageguy [EMAIL PROTECTED]:
I think I know the answer,
If you know the answer please tell it as I have read some discussions
on the web and although I have decided on a solution I'm still not
sure about the best answer, if there is a best answer after all.
but if
It's almost always better to store the images on the file system and
just store the filename or relative path in the database.
This is more efficient, doesn't bloat the database by storing files in
it, and is easier to get proper browser caching behavior (depending on
how your app is setup).
Don't store your images in the database. Store them on the filesystem
and store their path in the database. Anyone that tells you otherwise
is a stark raving madman :)
My system is very heavily used, and our pg_dump is only a few gigs.
Meanwhile our images/documents storage is well over a
John McCawley [EMAIL PROTECTED] writes:
Don't store your images in the database. Store them on the filesystem and
store their path in the database. Anyone that tells you otherwise is a stark
raving madman :)
My system is very heavily used, and our pg_dump is only a few gigs. Meanwhile
... and Moses said unto them, 'The eleventh commandment : thou shalt
store images in a database!'...
What if you had another database where you stored just the images and
not back it up if you don't want to?
As an application developer, I like the idea of storing files and images
in the
Don't store your images in the database. Store them on the filesystem and
store their path in the database
I 100% agree. Use the database as a lookup into the filesystem. Don't load the
database up with terabytes of non-searchable binary data? not sure how that
would help you?
Here is
This is a web app, so in my example all of the images live on a web
server, and our data lives on a separate database server. We have a
completely duplicated setup offsite, and mirror images of every server
at the backup site. Every night we use rsync to duplicate everything
offsite. Also,
,
-Jeanna
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of James Neff
Sent: Friday, January 05, 2007 2:27 PM
To: John McCawley
Cc: Clodoaldo; imageguy; pgsql-general@postgresql.org
Subject: Re: [GENERAL] Database versus filesystem for storing images
... and Moses
My comment about stark raving madmen was somewhat tongue-in-cheek.
There is certainly a case to be made for images in a database under some
circumstances. However, for the average Joe web developer, and
certainly someone new to the whole problem, I think storing them on the
filesystem is
How does it make it easier to control access and security? If your web
app makes a decision about allowing access to the database, it can just
as easily make a decision about allowing access to the filesystem.
Storing the images on the file system doesn't mean that there isn't a
piece of code
John McCawley [EMAIL PROTECTED] writes:
This is a web app, so in my example all of the images live on a web server,
and our data lives on a separate database server. We have a completely
duplicated setup offsite, and mirror images of every server at the backup
site. Every night we use rsync
Andrew Chernow [EMAIL PROTECTED] writes:
I mean, how do you handle integrity with data
outside the database?
You don't, the file system handles integrity of the stored data. Although,
one must careful to avoid db and fs orphans. Meaning, a record with no
corresponding file or a file with no
And how do you guarantee that after a failure? You're restoring two
different sets of data here:
How do you link them together on that specific operation? Or even on a daily
basis, if you get corrupted data...
I answered that already.
Not counting that depending on your choice of
On Fri, Jan 05, 2007 at 15:26:45 -0500,
James Neff [EMAIL PROTECTED] wrote:
... and Moses said unto them, 'The eleventh commandment : thou shalt
store images in a database!'...
What if you had another database where you stored just the images and
not back it up if you don't want to?
I
Yeah - it can make it easier to implement transactional semantics by
storing them in the database, although for simple operations it wouldn't
be hard to replicate this manually. And you are going to incur a
performance penalty by storing them in the database.
Another thing to consider is that
On Fri, 2007-01-05 at 15:54, Jeremy Haile wrote:
Yeah - it can make it easier to implement transactional semantics by
storing them in the database, although for simple operations it wouldn't
be hard to replicate this manually. And you are going to incur a
performance penalty by storing them
Jeremy Haile [EMAIL PROTECTED] writes:
Another thing to consider is that storing them in the file system makes
it much easier to browse the images using third-party tools, update
them, archive them (by gzipping or whatever). This is much more
difficult if they are stored in the database.
If
Or am
I the only one that is thinking about referential integrity with those files?
Not at all. I'm not sure how 3rd party tools like apache, `ls`, `gzip`, `find`,
nfs, etc... are breaking integrity. Any php, jsp, C or shell script you write
would be doing the same thing, accessing the data.
Andrew Chernow [EMAIL PROTECTED] writes:
And how do you guarantee that after a failure? You're restoring two
different sets of data here:
How do you link them together on that specific operation? Or even on a daily
basis, if you get corrupted data...
I answered that already.
I'm sorry.
Referential integrity would be one positive for storing the files in the
database. I wasn't responding to that. I'm simply saying that browsing
them with third-party tools, updating, compressing/archiving, etc. is
easier if they are not in the database. Those are all actions that I've
found
On fös, 2007-01-05 at 15:49 -0500, Andrew Chernow wrote:
I 100% agree. Use the database as a lookup into the filesystem. Don't load
the
database up with terabytes of non-searchable binary data? not sure how that
would help you?
I mean, how do you handle integrity with data
what happens if you rollback a transaction that just updated
an image file?
for that matter, what happens if one transaction is using or
even reading an image while another is updating it?
One thing I mentioned was about a point in time backup, not updating the image.
This would rollback
copying 3 billion files and a few hundred terabytes while still maintaining an
adequate service rate with part of its infra-structure down, just to use your
I wasn't saying to do this each time you run a backup, geez that would be
horrible. Pickup from where you left off the last time you
Ragnar [EMAIL PROTECTED] writes:
for that matter, what happens if one transaction is using or
even reading an image while another is updating it?
I believe that this also depends on how the file is updated. Some
applications create a temporary file with the new (or changed) content and
then
Andrew Chernow [EMAIL PROTECTED] writes:
Or am
I the only one that is thinking about referential integrity with those files?
Not at all. I'm not sure how 3rd party tools like apache, `ls`, `gzip`,
`find`, nfs, etc... are breaking integrity. Any php, jsp, C or shell script
For gzip, for
And introducint more failing points.
depends on how you do it. not everything has to go in a database to be
reliable. Part of good engineering is realizing when to use a tool and when not
to.
I think a 10K view of the issue is in order. The bigger picture is being
missed, or I am not
Andrew Chernow [EMAIL PROTECTED] writes:
I wasn't saying to do this each time you run a backup, geez that would be
horrible. Pickup from where you left off the last time you backed up
data/records. How many images and how much data is being generated in a 60
second period? I dought 3
Andrew Chernow [EMAIL PROTECTED] writes:
depends on how you do it. not everything has to go in a database to be
reliable. Part of good engineering is realizing when to use a tool and when
not to.
I think a 10K view of the issue is in order. The bigger picture is being
missed, or I am not
Personally, I'd put them on the file system, because then backup software
can perform incremental backups. In the database, that becomes more of a
difficulty. One suggestion, don't use a file name from a hash to store the
image, just use the serial id, and break them up by hundreds or thousands,
A web application will store hundreds of thousands images.
The images meta data will be stored in postgresql but I'm in doubt
about where to store the images themselves, if in the database as byte
strings or in the file system.
Would one of them have a faster image search than the other?
In
Clodoaldo wrote:
A web application will store hundreds of thousands images.
snip
What is the best practice in this situation? I mean not only the two
options as above but any.
This discussion comes up regularly. See the archives for a thread
titled Storing images in PostgreSQL databases
44 matches
Mail list logo
