Re[2]: [rt-users] Attachments table of RT's Mysql database
Wiki would be enough, but there is a problem in current patch state: it uses special constant string inplace of real content to indicate that file is located out of DB. Right way is to use flag in table. I do not think that changing scheme witch patching is a good idea, that is why this patch must be merged into RT. Now about different strokes for different folks. This way of storing attachments is reasonable when you get 50Mb+ of mail each week (scans, faxes, documents, etc). -- Boris Lytochkin, JSC e-port, Moscow web: www.e-port.ru, wap: wap.e-port.ru tel: +7 (495) 777 1872, ext. 251 __ From: Drew Barnes Sent: 7 августа 2007 г., 21:08 To: Boris Lytochkin Subject: [rt-users] Attachments table of RT's Mysql database I guess it all comes down to different strokes for different folks. Perhaps this could be put up on the wiki and then if someone needs this capability they can easily get it? DB Boris Lytochkin wrote: Ken, First, if everything is inside a database, then a simple backup of the database will get everything related to a particular RT instance. Wrong. We _stopped_ backup process of RT database due to LARGE amount data every day. We have no such amount of tape to store DB's everyday backups. Now, DB backup is done every day and attachment backup is done separately. As a result we have everyday SQL-backup of DB and incremental backup for attachments. It uses much less space. Second, in many cases you would like to isolate the front-end from the back-end information store. Once you need access to the filesystem, everything becomes much more involved. I understand that storing attachments out of RT involves much more than DB-only solution, BUT 10 Gb DB with 9.5 Gb of images involves much more. Anyway, it is up to admin to decide whether to store attachments separate or not. Tuesday, August 7, 2007, 4:22:49 PM, you wrote: Dear Mr. Lytochkin, There are two very good reasons to not store attachments outside of the database. First, if everything is inside a database, then a simple backup of the database will get everything related to a particular RT instance. Second, in many cases you would like to isolate the front-end from the back-end information store. Once you need access to the filesystem, everything becomes much more involved. I am certain that there are other reasons, but those two are certainly enough for me. I have appreciated the ease of generating a consistent backup of my RT information store. Ken On Tue, Aug 07, 2007 at 04:15:02PM +0400, Boris Lytochkin wrote: I wrote a patch that allows to store non-text attachments to be stored out of DB - in my case it greatly reduced DB swelling. Just for now it uses constant string in Attachments-Content to indicate that file is written to FS. You will need to specify some variables in RT_Siteconfig.pm: Set($AttachmentsDirectory, '/var/RT/attachments'); Set($LogAttachmentsLoading, 1); Set($LogAttachmentsSaving, 1); Set($StoreNonTextAttachmensInDB, undef); #Set($StoreNonTextAttachmensInDB, 1); A new share/html/Ticket/Attachment/dhandler and attach.patch for rest of RT distribution is in attachment. Gregory Harper, you can find more complex set of patches allowing to produce show image thumbs automaticly in attachment too. Some more variables must be specified in RT_Siteconfig.pm Set($ShowTransactionImages, 1); Set($ProduceImageThumbs, 1); Set($ImageThumbsDirectory, '/var/RT/thumbs'); I wonder why bestprcactical is not interested in intergating these patches into RT: From: Jesse Vincent Sent: 21 march 2007 ?., 23:53 To: lytochkin Subject: [RT 3.6] Storing attachments away from DB Hi Boris, Thanks very much for the mail, but I think we're not really interested in offering this feature within RT. Best, Jesse -- Boris Lytochkin, JSC e-port, Moscow web: www.e-port.ru, wap: wap.e-port.ru tel: +7 (495) 777 1872, ext. 251 Date: Mon, 06 Aug 2007 12:31:31 -0500 From: Gregory Harper [EMAIL PROTECTED] Subject: Re: [rt-users] Attachments table of RT's Mysql database To: Justin Brodley [EMAIL PROTECTED] Cc: rt-users@lists.bestpractical.com Message-ID: [EMAIL PROTECTED] Content-Type: text/plain; charset=ISO-8859-1; format=flowed Justin Brodley wrote: We actually had to disable the attachment feature as we were having our customers attach enormous files and killed our DB processing. Ultimately we are looking into rewriting the attachment feature to store the attachments on the web server to alleviate this overhead from the DB. I understand that the attachment table also stores all updates to a ticket, not just the attachments. Justin Brodley -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gregory Harper Sent: Wednesday, August 01, 2007
Re: Re[2]: [rt-users] Attachments table of RT's Mysql database
On Aug 8, 2007, at 5:17 AM, Boris Lytochkin wrote: Wiki would be enough, but there is a problem in current patch state: it uses special constant string inplace of real content to indicate that file is located out of DB. Right way is to use flag in table. I do not think that changing scheme witch patching is a good idea, that is why this patch must be merged into RT. This might be a good place to use the ContentEncoding field to describe an 'ondisk' encoding. -jesse PGP.sig Description: This is a digitally signed message part ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: [EMAIL PROTECTED] Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re[2]: [rt-users] Attachments table of RT's Mysql database
I wrote a patch that allows to store non-text attachments to be stored out of DB - in my case it greatly reduced DB swelling. Just for now it uses constant string in Attachments-Content to indicate that file is written to FS. You will need to specify some variables in RT_Siteconfig.pm: Set($AttachmentsDirectory, '/var/RT/attachments'); Set($LogAttachmentsLoading, 1); Set($LogAttachmentsSaving, 1); Set($StoreNonTextAttachmensInDB, undef); #Set($StoreNonTextAttachmensInDB, 1); A new share/html/Ticket/Attachment/dhandler and attach.patch for rest of RT distribution is in attachment. Gregory Harper, you can find more complex set of patches allowing to produce show image thumbs automaticly in attachment too. Some more variables must be specified in RT_Siteconfig.pm Set($ShowTransactionImages, 1); Set($ProduceImageThumbs, 1); Set($ImageThumbsDirectory, '/var/RT/thumbs'); I wonder why bestprcactical is not interested in intergating these patches into RT: From: Jesse Vincent Sent: 21 march 2007 г., 23:53 To: lytochkin Subject: [RT 3.6] Storing attachments away from DB Hi Boris, Thanks very much for the mail, but I think we're not really interested in offering this feature within RT. Best, Jesse -- Boris Lytochkin, JSC e-port, Moscow web: www.e-port.ru, wap: wap.e-port.ru tel: +7 (495) 777 1872, ext. 251 Date: Mon, 06 Aug 2007 12:31:31 -0500 From: Gregory Harper [EMAIL PROTECTED] Subject: Re: [rt-users] Attachments table of RT's Mysql database To: Justin Brodley [EMAIL PROTECTED] Cc: rt-users@lists.bestpractical.com Message-ID: [EMAIL PROTECTED] Content-Type: text/plain; charset=ISO-8859-1; format=flowed Justin Brodley wrote: We actually had to disable the attachment feature as we were having our customers attach enormous files and killed our DB processing. Ultimately we are looking into rewriting the attachment feature to store the attachments on the web server to alleviate this overhead from the DB. I understand that the attachment table also stores all updates to a ticket, not just the attachments. Justin Brodley -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gregory Harper Sent: Wednesday, August 01, 2007 12:41 PM To: rt-users@lists.bestpractical.com Subject: [rt-users] Attachments table of RT's Mysql database Hello everybody. We've been using RT for more than three months as part of our customer concern processes. Overall, things have been going well. The configuration includes Mysql, Apache2 and Postfix running on Ubuntu 6.06. I've made no modifications to the databases. The primary concern at this point is that the Attachments table of the Mysql database is growing significantly. Our CSR's want to attach PDFs, jpegs, etc. to the tickets with the jpegs usually created by our customers. The digital photos are the main culprit. I've read about scaling back the photos, creating thumbnails, etc. and we need to find a way to limit the attachment size prior to attachment. Has anyone else using RT had this type of problem? What are the best approaches for minimizing and controlling the size of the Attachments table? Any information, feedback and guidance are appreciated. thanks - Gregory Harper , Stevens Industries ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: [EMAIL PROTECTED] Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Thanks Justin for the feedback. Anyone else have input regarding their experiences with Attachments and RT? thanks - Greg -- Message: 3 Date: Mon, 6 Aug 2007 11:53:13 -0700 (PDT) From: Ed Matthews [EMAIL PROTECTED] Subject: [rt-users] Kwiki Table Rendering? To: rt-users@lists.bestpractical.com Message-ID: [EMAIL PROTECTED] Content-Type: text/plain; charset=us-ascii Happy Monday. Does anyone know why Kwiki renders tables on this page, http://wiki.bestpractical.com/view/ManualScrips but not on this page http://wiki.bestpractical.com/view/ManualRights nor in my personal page . I tried copy pasting the first ManualScrips table wiki markup into another edit page and saving, and it wouldn't render there either after save. Ed Matthews [EMAIL PROTECTED] Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos more. http://mobile.yahoo.com/go?refer=1GNXIC -- Message: 4 Date: Tue, 7 Aug 2007 10:20:33 +0100 From: Luke E Morgan [EMAIL PROTECTED] Subject: [rt-users] Trying to change the logo. Fedora Core 7, RT3.6.3 yum install To: rt-users@lists.bestpractical.com Message-ID: [EMAIL PROTECTED] Content-Type: text/plain;
Re[2]: [rt-users] Attachments table of RT's Mysql database
Ken, First, if everything is inside a database, then a simple backup of the database will get everything related to a particular RT instance. Wrong. We _stopped_ backup process of RT database due to LARGE amount data every day. We have no such amount of tape to store DB's everyday backups. Now, DB backup is done every day and attachment backup is done separately. As a result we have everyday SQL-backup of DB and incremental backup for attachments. It uses much less space. Second, in many cases you would like to isolate the front-end from the back-end information store. Once you need access to the filesystem, everything becomes much more involved. I understand that storing attachments out of RT involves much more than DB-only solution, BUT 10 Gb DB with 9.5 Gb of images involves much more. Anyway, it is up to admin to decide whether to store attachments separate or not. Tuesday, August 7, 2007, 4:22:49 PM, you wrote: Dear Mr. Lytochkin, There are two very good reasons to not store attachments outside of the database. First, if everything is inside a database, then a simple backup of the database will get everything related to a particular RT instance. Second, in many cases you would like to isolate the front-end from the back-end information store. Once you need access to the filesystem, everything becomes much more involved. I am certain that there are other reasons, but those two are certainly enough for me. I have appreciated the ease of generating a consistent backup of my RT information store. Ken On Tue, Aug 07, 2007 at 04:15:02PM +0400, Boris Lytochkin wrote: I wrote a patch that allows to store non-text attachments to be stored out of DB - in my case it greatly reduced DB swelling. Just for now it uses constant string in Attachments-Content to indicate that file is written to FS. You will need to specify some variables in RT_Siteconfig.pm: Set($AttachmentsDirectory, '/var/RT/attachments'); Set($LogAttachmentsLoading, 1); Set($LogAttachmentsSaving, 1); Set($StoreNonTextAttachmensInDB, undef); #Set($StoreNonTextAttachmensInDB, 1); A new share/html/Ticket/Attachment/dhandler and attach.patch for rest of RT distribution is in attachment. Gregory Harper, you can find more complex set of patches allowing to produce show image thumbs automaticly in attachment too. Some more variables must be specified in RT_Siteconfig.pm Set($ShowTransactionImages, 1); Set($ProduceImageThumbs, 1); Set($ImageThumbsDirectory, '/var/RT/thumbs'); I wonder why bestprcactical is not interested in intergating these patches into RT: From: Jesse Vincent Sent: 21 march 2007 ?., 23:53 To: lytochkin Subject: [RT 3.6] Storing attachments away from DB Hi Boris, Thanks very much for the mail, but I think we're not really interested in offering this feature within RT. Best, Jesse -- Boris Lytochkin, JSC e-port, Moscow web: www.e-port.ru, wap: wap.e-port.ru tel: +7 (495) 777 1872, ext. 251 Date: Mon, 06 Aug 2007 12:31:31 -0500 From: Gregory Harper [EMAIL PROTECTED] Subject: Re: [rt-users] Attachments table of RT's Mysql database To: Justin Brodley [EMAIL PROTECTED] Cc: rt-users@lists.bestpractical.com Message-ID: [EMAIL PROTECTED] Content-Type: text/plain; charset=ISO-8859-1; format=flowed Justin Brodley wrote: We actually had to disable the attachment feature as we were having our customers attach enormous files and killed our DB processing. Ultimately we are looking into rewriting the attachment feature to store the attachments on the web server to alleviate this overhead from the DB. I understand that the attachment table also stores all updates to a ticket, not just the attachments. Justin Brodley -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gregory Harper Sent: Wednesday, August 01, 2007 12:41 PM To: rt-users@lists.bestpractical.com Subject: [rt-users] Attachments table of RT's Mysql database Hello everybody. We've been using RT for more than three months as part of our customer concern processes. Overall, things have been going well. The configuration includes Mysql, Apache2 and Postfix running on Ubuntu 6.06. I've made no modifications to the databases. The primary concern at this point is that the Attachments table of the Mysql database is growing significantly. Our CSR's want to attach PDFs, jpegs, etc. to the tickets with the jpegs usually created by our customers. The digital photos are the main culprit. I've read about scaling back the photos, creating thumbnails, etc. and we need to find a way to limit the attachment size prior to attachment. Has anyone else using RT had this type of problem? What are the best approaches for minimizing and controlling the size of the Attachments table? Any information, feedback and guidance are appreciated. thanks - Gregory
RE: Re[2]: [rt-users] Attachments table of RT's Mysql database
I personally agree with Boris on this. This should be a configuration option to either store attachments in the database or on the web front end. In most enterprise applications you don't store files or session state in the database unless you have a really good reason too. The overhead added by sessions and files results in increased reads from the web server to verify session or large database sizes to store the files in blobs. Justin Brodley -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Boris Lytochkin Sent: Tuesday, August 07, 2007 6:31 AM To: rt-users@lists.bestpractical.com Subject: Re[2]: [rt-users] Attachments table of RT's Mysql database Ken, First, if everything is inside a database, then a simple backup of the database will get everything related to a particular RT instance. Wrong. We _stopped_ backup process of RT database due to LARGE amount data every day. We have no such amount of tape to store DB's everyday backups. Now, DB backup is done every day and attachment backup is done separately. As a result we have everyday SQL-backup of DB and incremental backup for attachments. It uses much less space. Second, in many cases you would like to isolate the front-end from the back-end information store. Once you need access to the filesystem, everything becomes much more involved. I understand that storing attachments out of RT involves much more than DB-only solution, BUT 10 Gb DB with 9.5 Gb of images involves much more. Anyway, it is up to admin to decide whether to store attachments separate or not. Tuesday, August 7, 2007, 4:22:49 PM, you wrote: Dear Mr. Lytochkin, There are two very good reasons to not store attachments outside of the database. First, if everything is inside a database, then a simple backup of the database will get everything related to a particular RT instance. Second, in many cases you would like to isolate the front-end from the back-end information store. Once you need access to the filesystem, everything becomes much more involved. I am certain that there are other reasons, but those two are certainly enough for me. I have appreciated the ease of generating a consistent backup of my RT information store. Ken On Tue, Aug 07, 2007 at 04:15:02PM +0400, Boris Lytochkin wrote: I wrote a patch that allows to store non-text attachments to be stored out of DB - in my case it greatly reduced DB swelling. Just for now it uses constant string in Attachments-Content to indicate that file is written to FS. You will need to specify some variables in RT_Siteconfig.pm: Set($AttachmentsDirectory, '/var/RT/attachments'); Set($LogAttachmentsLoading, 1); Set($LogAttachmentsSaving, 1); Set($StoreNonTextAttachmensInDB, undef); #Set($StoreNonTextAttachmensInDB, 1); A new share/html/Ticket/Attachment/dhandler and attach.patch for rest of RT distribution is in attachment. Gregory Harper, you can find more complex set of patches allowing to produce show image thumbs automaticly in attachment too. Some more variables must be specified in RT_Siteconfig.pm Set($ShowTransactionImages, 1); Set($ProduceImageThumbs, 1); Set($ImageThumbsDirectory, '/var/RT/thumbs'); I wonder why bestprcactical is not interested in intergating these patches into RT: From: Jesse Vincent Sent: 21 march 2007 ?., 23:53 To: lytochkin Subject: [RT 3.6] Storing attachments away from DB Hi Boris, Thanks very much for the mail, but I think we're not really interested in offering this feature within RT. Best, Jesse -- Boris Lytochkin, JSC e-port, Moscow web: www.e-port.ru, wap: wap.e-port.ru tel: +7 (495) 777 1872, ext. 251 Date: Mon, 06 Aug 2007 12:31:31 -0500 From: Gregory Harper [EMAIL PROTECTED] Subject: Re: [rt-users] Attachments table of RT's Mysql database To: Justin Brodley [EMAIL PROTECTED] Cc: rt-users@lists.bestpractical.com Message-ID: [EMAIL PROTECTED] Content-Type: text/plain; charset=ISO-8859-1; format=flowed Justin Brodley wrote: We actually had to disable the attachment feature as we were having our customers attach enormous files and killed our DB processing. Ultimately we are looking into rewriting the attachment feature to store the attachments on the web server to alleviate this overhead from the DB. I understand that the attachment table also stores all updates to a ticket, not just the attachments. Justin Brodley -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gregory Harper Sent: Wednesday, August 01, 2007 12:41 PM To: rt-users@lists.bestpractical.com Subject: [rt-users] Attachments table of RT's Mysql database Hello everybody. We've been using RT for more than three months as part of our customer concern processes. Overall, things have been going well. The configuration includes Mysql, Apache2 and Postfix running on Ubuntu
