Re: RE: image storage confusion ?? -- UUDECODE
I think that if you use Bfile oracle will not copy them to the standby database. It will only copy the pointers in the database. You will have to copy the files yourself. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 10, 2002 3:23 AM Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from
Re: Re: Re: RE: image storage confusion ?? -- UUDECODE
On NTFS there is an allocation unit and the Os gives space in multiple of allocations units which is 4k or 8k. So if you use bfile you will waste 3k for each file, increasing your disk space by 60%. I am not sure about the size of the allocation unit. Please verify this with your sysadmin. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 10, 2002 8:03 AM Thanx Chao. Still under confusion , whether to store 20,000,000 images of 5k each either in -oracle 8.1.6 on win2k ( BLOB or BFILE ) OR - just on plain linux file system - OR - any file server like NetApp on linux --- which will be better ? how do people usually handle data of such volume ? plz let me know how it is done normally. TIA. Jp. On Tue, 10 Dec 2002 chao_ping wrote : oraora oraora, For blob, it is ok, since it is all in the database, while for bfile, you actually store them in filesystem, so you have to backup those files indivudually. Regards zhu chao [EMAIL PROTECTED] www.happyit.net www.cnoug.org(Chinese Oracle User Group) === 2002-12-09 17:23:00 ,you wrote£º=== Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files
Re: RE: image storage confusion ?? -- UUDECODE
Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED
Re: Re: RE: image storage confusion ?? -- UUDECODE
oraora oraora, For blob, it is ok, since it is all in the database, while for bfile, you actually store them in filesystem, so you have to backup those files indivudually. Regards zhu chao [EMAIL PROTECTED] www.happyit.net www.cnoug.org(Chinese Oracle User Group) === 2002-12-09 17:23:00 ,you wrote£º=== Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above
Re: Re: Re: RE: image storage confusion ?? -- UUDECODE
Thanx Chao. Still under confusion , whether to store 20,000,000 images of 5k each either in -oracle 8.1.6 on win2k ( BLOB or BFILE ) OR - just on plain linux file system - OR - any file server like NetApp on linux --- which will be better ? how do people usually handle data of such volume ? plz let me know how it is done normally. TIA. Jp. On Tue, 10 Dec 2002 chao_ping wrote : oraora oraora, For blob, it is ok, since it is all in the database, while for bfile, you actually store them in filesystem, so you have to backup those files indivudually. Regards zhu chao [EMAIL PROTECTED] www.happyit.net www.cnoug.org(Chinese Oracle User Group) === 2002-12-09 17:23:00 ,you wrote£º=== Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already
Re: image storage confusion ??
Not to mention, backups can take a REALLY long time, unless you backup the raw volume image, in which case you can only restore at the filesystem level. On Tue, 2002-12-03 at 14:44, Connor McDonald wrote: Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Connor McDonald http://www.oracledba.co.uk http://www.oaktable.net GIVE a man a fish and he will eat for a day. But TEACH him how to fish, and...he will sit in a boat and drink beer all day __ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: =?iso-8859-1?q?Connor=20McDonald?= INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego
RE: image storage confusion ?? -- UUDECODE
As Cary as mentioned, there's some new goodies in v9. In our case, the controlling of attachments etc is done before processing into the db. I can't remember the specifics (read: I'm no longer at that site) but we found some shareware (mimencode? + a few other little things) we separated encoded emails into separate files which then got processed into separate lobs. Cheers Connor --- MacGregor, Ian A. [EMAIL PROTECTED] wrote: Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing
image storage confusion ??
Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
Re: image storage confusion ??
Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Connor McDonald http://www.oracledba.co.uk http://www.oaktable.net GIVE a man a fish and he will eat for a day. But TEACH him how to fish, and...he will sit in a boat and drink beer all day __ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: =?iso-8859-1?q?Connor=20McDonald?= INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
Re: image storage confusion ??
Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Connor McDonald http://www.oracledba.co.uk http://www.oaktable.net GIVE a man a fish and he will eat for a day. But TEACH him how to fish, and...he will sit in a boat and drink beer all day __ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: =?iso-8859-1?q?Connor=20McDonald?= INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Arup Nanda INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: image storage confusion ??
Excellent points all around. I am dealing with these issues currently. The original design was to use BFILE and external files. Now, they are leaning towards BLOBs. Here are some points to consider. If the system design calls for a standby or other backup site, you have to remember to replicate database and external changes. Recovery of the database may require restoration of external files. If you use BFILEs, are they on a dedicated drive or sharing space with other data/files? If you use BLOBs and they are not updated, set them in separate tablespaces and make the ts read only. Backup once and then take it out of the backup cycle, but make sure you save the backup for a good, long time! Dan Fink -Original Message- Sent: Tuesday, December 03, 2002 8:14 AM To: Multiple recipients of list ORACLE-L Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Connor McDonald http://www.oracledba.co.uk http://www.oaktable.net GIVE a man a fish and he will eat for a day. But TEACH him how to fish, and...he will sit in a boat and drink beer all day __ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: =?iso-8859-1?q?Connor=20McDonald?= INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Arup Nanda INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to:
Re: image storage confusion ??
You should/must do a benchmark. If not, how can you justify your decision ? If your management do not ask for a benchmack then you have bad management (and that's no good for you either...) Anyway, in a previous life, we did a benchmarck with files of similar size and it was faster on the OS. The environment was Oracle 816/Sun with Vignette Story Server. HTH --- oraora oraora [EMAIL PROTECTED] a écrit : Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Stéphane Paquette DBA Oracle et DB2, consultant entrepôt de données Oracle and DB2 DBA, datawarehouse consultant [EMAIL PROTECTED] __ Lèche-vitrine ou lèche-écran ? magasinage.yahoo.ca -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: =?iso-8859-1?q?Stephane=20Paquette?= INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: image storage confusion ??
oraora I've provided a couple of thoughts before in response to this issue, but I think that the only way you will get a satisfactory answer is to code a simple test. I hope someone else has extensive experience with 5k images and will reply to you. However, I think the answer to your question will vary quite a bit, depending for example on image size. The experience of someone based on 50k images, for example might not give you a good result. The answer may also depend on the operating system itself. The fact that you are using Java for access may also make a difference compared other languages. I think that you are going to have to code it both ways and run a few tests. If performance is really, really important to your company, you are probably going to be testing quite a few programming alternatives. Dennis Williams DBA, 40%OCP Lifetouch, Inc. [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 4:49 AM To: Multiple recipients of list ORACLE-L Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: DENNIS WILLIAMS INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
Re: image storage confusion ??
Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Connor McDonald http://www.oracledba.co.uk http://www.oaktable.net GIVE a man a fish and he will eat for a day. But TEACH him how to fish, and...he will sit in a boat and drink beer all day __ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: =?iso-8859-1?q?Connor=20McDonald?= INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Arup Nanda INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL
Re: image storage confusion ??
Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Connor McDonald http://www.oracledba.co.uk http://www.oaktable.net GIVE a man a fish and he will eat for a day. But TEACH him how to fish, and...he will sit in a boat and drink beer all day __ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: =?iso-8859-1?q?Connor=20McDonald?= INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You
Re: image storage confusion ??
FWIW ... in my last job we stored 13 million CLOB documents (max size of 5KB, stored out-of-line in their own CLOB segments) in an 8.1.7.3 DB under Win2k (2 CPU HP NetServer with 4GB RAM and twenty-three 36GB drives; JBOD). The CLOB documents were all interMedia Text indexed. Contains queries returned results in a second or less, unless the query was not very specific and hit thousands of documents. Java processes loaded up to 250,000 new documents every night and the interMedia index was refreshed shortly thereafter - took up to a couple of hours. The front-end was Cold Fusion, so that's not applicable to your situation. This ~200GB database had a Standby DB on a sister machine. Everything worked very well. RMan level 0 backups took a couple of hours. I vote for out-of-line BLOBs in their own tablespace(s) on as many spindles as you can afford. Jack C. Applewhite Database Administrator Austin Independent School District Austin, Texas 512.414.9715 (wk) 512.935.5929 (pager) [EMAIL PROTECTED] Stephane Paquette stephane_paquette@ To: Multiple recipients of list ORACLE-L YAHOO.COM [EMAIL PROTECTED] Sent by: cc: [EMAIL PROTECTED] Subject: Re: image storage confusion ?? 12/03/2002 08:32 AM Please respond to ORACLE-L You should/must do a benchmark. If not, how can you justify your decision ? If your management do not ask for a benchmack then you have bad management (and that's no good for you either...) Anyway, in a previous life, we did a benchmarck with files of similar size and it was faster on the OS. The environment was Oracle 816/Sun with Vignette Story Server. HTH --- oraora oraora [EMAIL PROTECTED] a écrit : Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Author: oraora oraora INET: [EMAIL PROTECTED] = Stéphane Paquette DBA Oracle et DB2, consultant entrepôt de données Oracle and DB2 DBA, datawarehouse consultant [EMAIL PROTECTED] -- Author: =?iso-8859-1?q?Stephane=20Paquette?= INET: [EMAIL PROTECTED] -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: image storage confusion ??
I would test it on raw disk device because it avoids double buffering. -Original Message- Sent: Tuesday, December 03, 2002 2:49 AM To: Multiple recipients of list ORACLE-L Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Wong, Bing INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: image storage confusion ?? -- UUDECODE
Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = Connor McDonald http://www.oracledba.co.uk http://www.oaktable.net GIVE a man a fish and he will eat for a day. But TEACH him how to fish, and...he will sit in a boat and drink beer all day __ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com -- Please see the official ORACLE-L FAQ: http
RE: image storage confusion ?? -- UUDECODE
Ian, There are UUENCODE and UUDECODE functions in 9.0.1. http://download-west.oracle.com/docs/cd/A91202_01/901_doc/appdev.901/a89 852/u_encod5.htm#1004065 It's also a one-liner in Perl, which would presumably be a lot faster than doing it in PL/SQL if you're not constrained to doing the operation inside the database. http://search.cpan.org/author/ANDK/Convert-UU-0.40/lib/Convert/UU.pm Cary Millsap Hotsos Enterprises, Ltd. http://www.hotsos.com Upcoming events: - Hotsos Clinic, Dec 9-11 Honolulu - Hotsos Clinic 101, Jan 7-9 Knoxville - Steve Adams's Miracle Master Class, Jan 13-15 Copenhagen - 2003 Hotsos Symposium, Feb 9-12 Dallas -Original Message- Ian A. Sent: Tuesday, December 03, 2002 5:02 PM To: Multiple recipients of list ORACLE-L Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list and web hosting services - To REMOVE yourself from this mailing list, send an E
RE: image storage confusion ?? -- UUDECODE
Thank you very much for the pointers. Ian -Original Message- Sent: Tuesday, December 03, 2002 7:39 PM To: Multiple recipients of list ORACLE-L Ian, There are UUENCODE and UUDECODE functions in 9.0.1. http://download-west.oracle.com/docs/cd/A91202_01/901_doc/appdev.901/a89 852/u_encod5.htm#1004065 It's also a one-liner in Perl, which would presumably be a lot faster than doing it in PL/SQL if you're not constrained to doing the operation inside the database. http://search.cpan.org/author/ANDK/Convert-UU-0.40/lib/Convert/UU.pm Cary Millsap Hotsos Enterprises, Ltd. http://www.hotsos.com Upcoming events: - Hotsos Clinic, Dec 9-11 Honolulu - Hotsos Clinic 101, Jan 7-9 Knoxville - Steve Adams's Miracle Master Class, Jan 13-15 Copenhagen - 2003 Hotsos Symposium, Feb 9-12 Dallas -Original Message- Ian A. Sent: Tuesday, December 03, 2002 5:02 PM To: Multiple recipients of list ORACLE-L Were doing the same thing with mail between collaborators on one of our physics projects. But the volume is small, just unDer 250,000 so far. Did the mail ou were saving contain attachments, and if so did you write any code to break off the attachment uudecode it and place it in a blob? If so, I am keenly interested in that code. Is there a publicly available package to do uuencode and uudecode? Ian MacGregor Stanford Linear Accelerator Center [EMAIL PROTECTED] -Original Message- Sent: Tuesday, December 03, 2002 11:44 AM To: Multiple recipients of list ORACLE-L Exactly. We had a system that used to drag emails (from qmail so each mail was a file) into clobs in the database. After a year or so we had about 15 million emails in the database - no problems at all. Then one day the some idiot (aka me) put a new version of the program in which successfully loaded the clob but (to cut a long story short) started replicating the email files left, right and centre...It took literally days to clean up millions of (zero byte size) files...after which point that file system needed to be rebuilt anyway, the directory structure was in such a mess Cheers Connor --- [EMAIL PROTECTED] wrote: Arup, What Connor may have been referring to is the inefficiency of managing 20 million files in a filesystem. That's a lot of inodes ( assuming unix ). It's a bit much for a filesystem to deal with. Jared Arup Nanda [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2002 07:14 AM Please respond to ORACLE-L To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc: Subject:Re: image storage confusion ?? Connor, I seem to think otherwise. Storing 100 GB of image is not necessarily a pretty proposition either when you consider hot backups and archived log generation, etc. I presume you are concerned about the management of the image files considering the sheer volume of it. But that's precisely what BFILE is expected to help with. The images are in a file ssytem and the pointers are in the database and that's managed pretty well. However I do concede tht this might pose a problem on two fronts - (1) Security - beign on filesystem anyone can potentially see these. However this is not necesarily a concern at all sites. Good OS security can prevent this. (2) Backup - the ssy admin has to explicitly backup all these files. This, again, may not be that bad when you store your files on a single filesystem and a backup software can be easily programmed to check only the changed files, based on timestamp. Just my two cents. Arup Nanda - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Tuesday, December 03, 2002 7:39 AM Managing 20mil of anything (images/text/etc) in a file system isn't a nice proposition. Go with the database hth connor --- oraora oraora [EMAIL PROTECTED] wrote: Guys , i posted this already and this time my question is a bit different . I have to store 20,000,000 images of 5k each either in DB ( on win2k) or linux o/s file system. the images are to be displayed over mobile phones.so the time to retrieve the images should be minimum. for this to be achieved , i am confused , whether the images should be stored in o/s file system or in the DB ? --selecting a file from linux o/s file system --- or --querying it from oracle DB ( on win2k) --- which of the above will be faster ? all these will be done with java . this being condition , i would like to know ur suggestion guys. my DB is oracle 8.1.6 on Win2k. TIA -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: oraora oraora INET: [EMAIL PROTECTED] Fat City Network Services-- 858-538-5051 http://www.fatcity.com San Diego, California-- Mailing list
