Re: Re: Re: RE: image storage confusion ?? --> UUDECODE
On NTFS there is an allocation unit and the Os gives space in multiple of allocations units which is 4k or 8k. So if you use bfile you will waste 3k for each file, increasing your disk space by 60%. I am not sure about the size of the allocation unit. Please verify this with your sysadmin. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L <[EMAIL PROTECTED]> Sent: Tuesday, December 10, 2002 8:03 AM > Thanx Chao. > Still under confusion , whether to store 20,000,000 images of 5k > each either in > -oracle 8.1.6 on win2k ( BLOB or BFILE ) > OR > - just on plain linux file system - > OR > - any file server like NetApp on linux --- > > which will be better ? > how do people usually handle data of such volume ? > plz let me know how it is done normally. > > TIA. > Jp. > > > On Tue, 10 Dec 2002 chao_ping wrote : > >oraora oraora, > > For blob, it is ok, since it is all in the database, while for > >bfile, you actually store them in filesystem, so you have to > >backup those files indivudually. > > > > > > > > > > > >Regards > >zhu chao > >[EMAIL PROTECTED] > >www.happyit.net > >www.cnoug.org(Chinese Oracle User Group) > > > >=== 2002-12-09 17:23:00 ,you wrote£º=== > > > > >Guys, > > > > > >Suppose I store the images (~100G)in DB ( either as BLOB or > >BFILE > > >). > > >I want to have a standby DB for this Prod. DB. > > >Will there be any problem ? any known issues ? > > >what are the things to be taken care of ??? > > > > > >Kindly let me know Guys. > > > > > >TIA. > > >Jp. > > > > > > > > >On Wed, 04 Dec 2002 Connor McDonald wrote : > > >>As Cary as mentioned, there's some new goodies in v9. > > >> > > >>In our case, the controlling of attachments etc is > > >>done before processing into the db. I can't remember > > >>the specifics (read: I'm no longer at that site) but > > >>we found some shareware (mimencode? + a few other > > >>little things) we separated encoded emails into > > >>separate files which then got processed into separate > > >>lobs. > > >> > > >>Cheers > > >>Connor > > >> > > >> --- "MacGregor, Ian A." <[EMAIL PROTECTED]> > > >>wrote: > Were doing the same thing with mail between > > >> > collaborators on one of our physics projects. But > > >> > the volume is small, just unDer 250,000 so far. Did > > >> > the mail ou were saving contain attachments, and if > > >> > so did you write any code to break off the > > >> > attachment uudecode it and place it in a blob? If > > >> > so, I am keenly interested in that code. > > >> > > > >> > Is there a publicly available package to do uuencode > > >> > and uudecode? > > >> > > > >> > Ian MacGregor > > >> > Stanford Linear Accelerator Center > > >> > [EMAIL PROTECTED] > > >> > > > >> > -Original Message- > > >> > Sent: Tuesday, December 03, 2002 11:44 AM > > >> > To: Multiple recipients of list ORACLE-L > > >> > > > >> > > > >> > Exactly. > > >> > > > >> > We had a system that used to drag emails (from qmail > > >> > so each mail was a file) into clobs in the database. > > >> > > > >> > After a year or so we had about 15 million emails in > > >> > the database - no problems at all. > > >> > > > >> > Then one day the some idiot (aka me) put a new > > >> > version > > >> > of the program in which successfully loaded the clob > > >> > but (to cut a long story short) started replicating > > >> > the email files left, right and centre...It took > > >> > literally days to clean up millions of (zero byte > > >> > size) files...after which point that file system > > >> > needed to be rebuilt anyway, the directory structure > > >> > was in such a mess > > >> > > > >> > Cheers > > >> > Connor > > >> > > > >> > --- [EMAIL PROTECTED] wrote: > Arup, > > >> > > > > >> > > What Connor may have been referring to is the > > >> > > inefficiency > > >> > > of managing 20 million files in a filesystem. > > >> > > > > >> > > That's a lot of inodes ( assuming unix ). It's a > > >> > > bit much > > >> > > for a filesystem to deal with. > > >> > > > > >> > > Jared > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > "Arup Nanda" <[EMAIL PROTECTED]> > > >> > > Sent by: [EMAIL PROTECTED] > > >> > > 12/03/2002 07:14 AM > > >> > > Please respond to ORACLE-L > > >> > > > > >> > > > > >> > > To: Multiple recipients of list > > >> > ORACLE-L > > >> > > <[EMAIL PROTECTED]> > > >> > > cc: > > >> > > Subject:Re: image storage > > >> > confusion > > >> > > ?? > > >> > > > > >> > > > > >> > > Connor, > > >> > > > > >> > > I seem to think otherwise. Storing 100 GB of image > > >> > > is not necessarily a > > >> > > pretty proposition either when you consider hot > > >> > > backups and archived log > > >> > > generation, etc. I presume you are concerned about > > >> > > the management of the > > >> > > image files considering the sheer volume of it. > > >> > But
Re: RE: image storage confusion ?? --> UUDECODE
I think that if you use Bfile oracle will not copy them to the standby database. It will only copy the pointers in the database. You will have to copy the files yourself. Yechiel Adar Mehish - Original Message - To: Multiple recipients of list ORACLE-L <[EMAIL PROTECTED]> Sent: Tuesday, December 10, 2002 3:23 AM > Guys, > > Suppose I store the images (~100G)in DB ( either as BLOB or BFILE > ). > I want to have a standby DB for this Prod. DB. > Will there be any problem ? any known issues ? > what are the things to be taken care of ??? > > Kindly let me know Guys. > > TIA. > Jp. > > > On Wed, 04 Dec 2002 Connor McDonald wrote : > >As Cary as mentioned, there's some new goodies in v9. > > > >In our case, the controlling of attachments etc is > >done before processing into the db. I can't remember > >the specifics (read: I'm no longer at that site) but > >we found some shareware (mimencode? + a few other > >little things) we separated encoded emails into > >separate files which then got processed into separate > >lobs. > > > >Cheers > >Connor > > > > --- "MacGregor, Ian A." <[EMAIL PROTECTED]> > >wrote: > Were doing the same thing with mail between > > > collaborators on one of our physics projects. But > > > the volume is small, just unDer 250,000 so far. Did > > > the mail ou were saving contain attachments, and if > > > so did you write any code to break off the > > > attachment uudecode it and place it in a blob? If > > > so, I am keenly interested in that code. > > > > > > Is there a publicly available package to do uuencode > > > and uudecode? > > > > > > Ian MacGregor > > > Stanford Linear Accelerator Center > > > [EMAIL PROTECTED] > > > > > > -Original Message- > > > Sent: Tuesday, December 03, 2002 11:44 AM > > > To: Multiple recipients of list ORACLE-L > > > > > > > > > Exactly. > > > > > > We had a system that used to drag emails (from qmail > > > so each mail was a file) into clobs in the database. > > > > > > After a year or so we had about 15 million emails in > > > the database - no problems at all. > > > > > > Then one day the some idiot (aka me) put a new > > > version > > > of the program in which successfully loaded the clob > > > but (to cut a long story short) started replicating > > > the email files left, right and centre...It took > > > literally days to clean up millions of (zero byte > > > size) files...after which point that file system > > > needed to be rebuilt anyway, the directory structure > > > was in such a mess > > > > > > Cheers > > > Connor > > > > > > --- [EMAIL PROTECTED] wrote: > Arup, > > > > > > > > What Connor may have been referring to is the > > > > inefficiency > > > > of managing 20 million files in a filesystem. > > > > > > > > That's a lot of inodes ( assuming unix ). It's a > > > > bit much > > > > for a filesystem to deal with. > > > > > > > > Jared > > > > > > > > > > > > > > > > > > > > > > > > > > > > "Arup Nanda" <[EMAIL PROTECTED]> > > > > Sent by: [EMAIL PROTECTED] > > > > 12/03/2002 07:14 AM > > > > Please respond to ORACLE-L > > > > > > > > > > > > To: Multiple recipients of list > > > ORACLE-L > > > > <[EMAIL PROTECTED]> > > > > cc: > > > > Subject:Re: image storage > > > confusion > > > > ?? > > > > > > > > > > > > Connor, > > > > > > > > I seem to think otherwise. Storing 100 GB of image > > > > is not necessarily a > > > > pretty proposition either when you consider hot > > > > backups and archived log > > > > generation, etc. I presume you are concerned about > > > > the management of the > > > > image files considering the sheer volume of it. > > > But > > > > that's precisely what > > > > BFILE is expected to help with. The images are in > > > a > > > > file ssytem and the > > > > pointers are in the database and that's managed > > > > pretty well. > > > > > > > > However I do concede tht this might pose a problem > > > > on two fronts - > > > > (1) Security - beign on filesystem anyone can > > > > potentially see these. > > > > However > > > > this is not necesarily a concern at all sites. > > > Good > > > > OS security can > > > > prevent > > > > this. > > > > (2) Backup - the ssy admin has to explicitly > > > backup > > > > all these files. This, > > > > again, may not be that bad when you store your > > > files > > > > on a single > > > > filesystem > > > > and a backup software can be easily programmed to > > > > check only the changed > > > > files, based on timestamp. > > > > > > > > Just my two cents. > > > > > > > > Arup Nanda > > > > > > > > - Original Message - > > > > To: "Multiple recipients of list ORACLE-L" > > > <[EMAIL PROTECTED]> > > > > Sent: Tuesday, December 03, 2002 7:39 AM > > > > > > > > > > > > > Managing 20mil of anything (images/text/etc) in > > > a > > > > file > > > > > system isn't a nice proposition. Go with the > > > > database > > > > > > > > > > hth > > > > > connor > > > > > > > > > > --- oraora oraora <[EMAIL PROTECTED]> > > > > wrote
Re: Re: Re: RE: image storage confusion ?? --> UUDECODE
Thanx Chao. Still under confusion , whether to store 20,000,000 images of 5k each either in -oracle 8.1.6 on win2k ( BLOB or BFILE ) OR - just on plain linux file system - OR - any file server like NetApp on linux --- which will be better ? how do people usually handle data of such volume ? plz let me know how it is done normally. TIA. Jp. On Tue, 10 Dec 2002 chao_ping wrote : >oraora oraora, > For blob, it is ok, since it is all in the database, while for >bfile, you actually store them in filesystem, so you have to >backup those files indivudually. > > > > > >Regards >zhu chao >[EMAIL PROTECTED] >www.happyit.net >www.cnoug.org(Chinese Oracle User Group) > >=== 2002-12-09 17:23:00 ,you wrote£º=== > > >Guys, > > > >Suppose I store the images (~100G)in DB ( either as BLOB or >BFILE > >). > >I want to have a standby DB for this Prod. DB. > >Will there be any problem ? any known issues ? > >what are the things to be taken care of ??? > > > >Kindly let me know Guys. > > > >TIA. > >Jp. > > > > > >On Wed, 04 Dec 2002 Connor McDonald wrote : > >>As Cary as mentioned, there's some new goodies in v9. > >> > >>In our case, the controlling of attachments etc is > >>done before processing into the db. I can't remember > >>the specifics (read: I'm no longer at that site) but > >>we found some shareware (mimencode? + a few other > >>little things) we separated encoded emails into > >>separate files which then got processed into separate > >>lobs. > >> > >>Cheers > >>Connor > >> > >> --- "MacGregor, Ian A." <[EMAIL PROTECTED]> > >>wrote: > Were doing the same thing with mail between > >> > collaborators on one of our physics projects. But > >> > the volume is small, just unDer 250,000 so far. Did > >> > the mail ou were saving contain attachments, and if > >> > so did you write any code to break off the > >> > attachment uudecode it and place it in a blob? If > >> > so, I am keenly interested in that code. > >> > > >> > Is there a publicly available package to do uuencode > >> > and uudecode? > >> > > >> > Ian MacGregor > >> > Stanford Linear Accelerator Center > >> > [EMAIL PROTECTED] > >> > > >> > -Original Message- > >> > Sent: Tuesday, December 03, 2002 11:44 AM > >> > To: Multiple recipients of list ORACLE-L > >> > > >> > > >> > Exactly. > >> > > >> > We had a system that used to drag emails (from qmail > >> > so each mail was a file) into clobs in the database. > >> > > >> > After a year or so we had about 15 million emails in > >> > the database - no problems at all. > >> > > >> > Then one day the some idiot (aka me) put a new > >> > version > >> > of the program in which successfully loaded the clob > >> > but (to cut a long story short) started replicating > >> > the email files left, right and centre...It took > >> > literally days to clean up millions of (zero byte > >> > size) files...after which point that file system > >> > needed to be rebuilt anyway, the directory structure > >> > was in such a mess > >> > > >> > Cheers > >> > Connor > >> > > >> > --- [EMAIL PROTECTED] wrote: > Arup, > >> > > > >> > > What Connor may have been referring to is the > >> > > inefficiency > >> > > of managing 20 million files in a filesystem. > >> > > > >> > > That's a lot of inodes ( assuming unix ). It's a > >> > > bit much > >> > > for a filesystem to deal with. > >> > > > >> > > Jared > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > "Arup Nanda" <[EMAIL PROTECTED]> > >> > > Sent by: [EMAIL PROTECTED] > >> > > 12/03/2002 07:14 AM > >> > > Please respond to ORACLE-L > >> > > > >> > > > >> > > To: Multiple recipients of list > >> > ORACLE-L > >> > > <[EMAIL PROTECTED]> > >> > > cc: > >> > > Subject:Re: image storage > >> > confusion > >> > > ?? > >> > > > >> > > > >> > > Connor, > >> > > > >> > > I seem to think otherwise. Storing 100 GB of image > >> > > is not necessarily a > >> > > pretty proposition either when you consider hot > >> > > backups and archived log > >> > > generation, etc. I presume you are concerned about > >> > > the management of the > >> > > image files considering the sheer volume of it. > >> > But > >> > > that's precisely what > >> > > BFILE is expected to help with. The images are in > >> > a > >> > > file ssytem and the > >> > > pointers are in the database and that's managed > >> > > pretty well. > >> > > > >> > > However I do concede tht this might pose a problem > >> > > on two fronts - > >> > > (1) Security - beign on filesystem anyone can > >> > > potentially see these. > >> > > However > >> > > this is not necesarily a concern at all sites. > >> > Good > >> > > OS security can > >> > > prevent > >> > > this. > >> > > (2) Backup - the ssy admin has to explicitly > >> > backup > >> > > all these files. This, > >> > > again, may not be that bad when you store your > >> > files > >> > > on a single
Re: Re: RE: image storage confusion ?? --> UUDECODE
oraora oraora, For blob, it is ok, since it is all in the database, while for bfile, you actually store them in filesystem, so you have to backup those files indivudually. Regards zhu chao [EMAIL PROTECTED] www.happyit.net www.cnoug.org(Chinese Oracle User Group) === 2002-12-09 17:23:00 ,you wrote£º=== >Guys, > >Suppose I store the images (~100G)in DB ( either as BLOB or BFILE >). >I want to have a standby DB for this Prod. DB. >Will there be any problem ? any known issues ? >what are the things to be taken care of ??? > >Kindly let me know Guys. > >TIA. >Jp. > > >On Wed, 04 Dec 2002 Connor McDonald wrote : >>As Cary as mentioned, there's some new goodies in v9. >> >>In our case, the controlling of attachments etc is >>done before processing into the db. I can't remember >>the specifics (read: I'm no longer at that site) but >>we found some shareware (mimencode? + a few other >>little things) we separated encoded emails into >>separate files which then got processed into separate >>lobs. >> >>Cheers >>Connor >> >> --- "MacGregor, Ian A." <[EMAIL PROTECTED]> >>wrote: > Were doing the same thing with mail between >> > collaborators on one of our physics projects. But >> > the volume is small, just unDer 250,000 so far. Did >> > the mail ou were saving contain attachments, and if >> > so did you write any code to break off the >> > attachment uudecode it and place it in a blob? If >> > so, I am keenly interested in that code. >> > >> > Is there a publicly available package to do uuencode >> > and uudecode? >> > >> > Ian MacGregor >> > Stanford Linear Accelerator Center >> > [EMAIL PROTECTED] >> > >> > -Original Message- >> > Sent: Tuesday, December 03, 2002 11:44 AM >> > To: Multiple recipients of list ORACLE-L >> > >> > >> > Exactly. >> > >> > We had a system that used to drag emails (from qmail >> > so each mail was a file) into clobs in the database. >> > >> > After a year or so we had about 15 million emails in >> > the database - no problems at all. >> > >> > Then one day the some idiot (aka me) put a new >> > version >> > of the program in which successfully loaded the clob >> > but (to cut a long story short) started replicating >> > the email files left, right and centre...It took >> > literally days to clean up millions of (zero byte >> > size) files...after which point that file system >> > needed to be rebuilt anyway, the directory structure >> > was in such a mess >> > >> > Cheers >> > Connor >> > >> > --- [EMAIL PROTECTED] wrote: > Arup, >> > > >> > > What Connor may have been referring to is the >> > > inefficiency >> > > of managing 20 million files in a filesystem. >> > > >> > > That's a lot of inodes ( assuming unix ). It's a >> > > bit much >> > > for a filesystem to deal with. >> > > >> > > Jared >> > > >> > > >> > > >> > > >> > > >> > > >> > > "Arup Nanda" <[EMAIL PROTECTED]> >> > > Sent by: [EMAIL PROTECTED] >> > > 12/03/2002 07:14 AM >> > > Please respond to ORACLE-L >> > > >> > > >> > > To: Multiple recipients of list >> > ORACLE-L >> > > <[EMAIL PROTECTED]> >> > > cc: >> > > Subject:Re: image storage >> > confusion >> > > ?? >> > > >> > > >> > > Connor, >> > > >> > > I seem to think otherwise. Storing 100 GB of image >> > > is not necessarily a >> > > pretty proposition either when you consider hot >> > > backups and archived log >> > > generation, etc. I presume you are concerned about >> > > the management of the >> > > image files considering the sheer volume of it. >> > But >> > > that's precisely what >> > > BFILE is expected to help with. The images are in >> > a >> > > file ssytem and the >> > > pointers are in the database and that's managed >> > > pretty well. >> > > >> > > However I do concede tht this might pose a problem >> > > on two fronts - >> > > (1) Security - beign on filesystem anyone can >> > > potentially see these. >> > > However >> > > this is not necesarily a concern at all sites. >> > Good >> > > OS security can >> > > prevent >> > > this. >> > > (2) Backup - the ssy admin has to explicitly >> > backup >> > > all these files. This, >> > > again, may not be that bad when you store your >> > files >> > > on a single >> > > filesystem >> > > and a backup software can be easily programmed to >> > > check only the changed >> > > files, based on timestamp. >> > > >> > > Just my two cents. >> > > >> > > Arup Nanda >> > > >> > > - Original Message - >> > > To: "Multiple recipients of list ORACLE-L" >> > <[EMAIL PROTECTED]> >> > > Sent: Tuesday, December 03, 2002 7:39 AM >> > > >> > > >> > > > Managing 20mil of anything (images/text/etc) in >> > a >> > > file >> > > > system isn't a nice proposition. Go with the >> > > database >> > > > >> > > > hth >> > > > connor >> > > > >> > > > --- oraora oraora <[EMAIL PROTECTED]> >> > > wrote: >> > > > > Guys , i posted this already and this time my >> > > > > question is a bit >> > > > > different . >> > > > > >> > > > > I have to
Re: RE: image storage confusion ?? --> UUDECODE
Guys, Suppose I store the images (~100G)in DB ( either as BLOB or BFILE ). I want to have a standby DB for this Prod. DB. Will there be any problem ? any known issues ? what are the things to be taken care of ??? Kindly let me know Guys. TIA. Jp. On Wed, 04 Dec 2002 Connor McDonald wrote : >As Cary as mentioned, there's some new goodies in v9. > >In our case, the controlling of attachments etc is >done before processing into the db. I can't remember >the specifics (read: I'm no longer at that site) but >we found some shareware (mimencode? + a few other >little things) we separated encoded emails into >separate files which then got processed into separate >lobs. > >Cheers >Connor > > --- "MacGregor, Ian A." <[EMAIL PROTECTED]> >wrote: > Were doing the same thing with mail between > > collaborators on one of our physics projects. But > > the volume is small, just unDer 250,000 so far. Did > > the mail ou were saving contain attachments, and if > > so did you write any code to break off the > > attachment uudecode it and place it in a blob? If > > so, I am keenly interested in that code. > > > > Is there a publicly available package to do uuencode > > and uudecode? > > > > Ian MacGregor > > Stanford Linear Accelerator Center > > [EMAIL PROTECTED] > > > > -Original Message- > > Sent: Tuesday, December 03, 2002 11:44 AM > > To: Multiple recipients of list ORACLE-L > > > > > > Exactly. > > > > We had a system that used to drag emails (from qmail > > so each mail was a file) into clobs in the database. > > > > After a year or so we had about 15 million emails in > > the database - no problems at all. > > > > Then one day the some idiot (aka me) put a new > > version > > of the program in which successfully loaded the clob > > but (to cut a long story short) started replicating > > the email files left, right and centre...It took > > literally days to clean up millions of (zero byte > > size) files...after which point that file system > > needed to be rebuilt anyway, the directory structure > > was in such a mess > > > > Cheers > > Connor > > > > --- [EMAIL PROTECTED] wrote: > Arup, > > > > > > What Connor may have been referring to is the > > > inefficiency > > > of managing 20 million files in a filesystem. > > > > > > That's a lot of inodes ( assuming unix ). It's a > > > bit much > > > for a filesystem to deal with. > > > > > > Jared > > > > > > > > > > > > > > > > > > > > > "Arup Nanda" <[EMAIL PROTECTED]> > > > Sent by: [EMAIL PROTECTED] > > > 12/03/2002 07:14 AM > > > Please respond to ORACLE-L > > > > > > > > > To: Multiple recipients of list > > ORACLE-L > > > <[EMAIL PROTECTED]> > > > cc: > > > Subject:Re: image storage > > confusion > > > ?? > > > > > > > > > Connor, > > > > > > I seem to think otherwise. Storing 100 GB of image > > > is not necessarily a > > > pretty proposition either when you consider hot > > > backups and archived log > > > generation, etc. I presume you are concerned about > > > the management of the > > > image files considering the sheer volume of it. > > But > > > that's precisely what > > > BFILE is expected to help with. The images are in > > a > > > file ssytem and the > > > pointers are in the database and that's managed > > > pretty well. > > > > > > However I do concede tht this might pose a problem > > > on two fronts - > > > (1) Security - beign on filesystem anyone can > > > potentially see these. > > > However > > > this is not necesarily a concern at all sites. > > Good > > > OS security can > > > prevent > > > this. > > > (2) Backup - the ssy admin has to explicitly > > backup > > > all these files. This, > > > again, may not be that bad when you store your > > files > > > on a single > > > filesystem > > > and a backup software can be easily programmed to > > > check only the changed > > > files, based on timestamp. > > > > > > Just my two cents. > > > > > > Arup Nanda > > > > > > - Original Message - > > > To: "Multiple recipients of list ORACLE-L" > > <[EMAIL PROTECTED]> > > > Sent: Tuesday, December 03, 2002 7:39 AM > > > > > > > > > > Managing 20mil of anything (images/text/etc) in > > a > > > file > > > > system isn't a nice proposition. Go with the > > > database > > > > > > > > hth > > > > connor > > > > > > > > --- oraora oraora <[EMAIL PROTECTED]> > > > wrote: > > > > > Guys , i posted this already and this time my > > > > > question is a bit > > > > > different . > > > > > > > > > > I have to store 20,000,000 images of 5k each > > > either > > > > > in DB ( on > > > > > win2k) or linux o/s file system. > > > > > > > > > > the images are to be displayed over mobile > > > phones.so > > > > > the time to > > > > > retrieve the images should be minimum. > > > > > > > > > > for this to be achieved , i am confused , > > > whether > > > > > the images > > > > > should be stored in o/s file system or in > > the > > > DB ? > > > > > > > > > > --selecting a file from linux o/s file > > sys
