Re[4]: [rt-users] Attachments table of RT's Mysql database

2007-08-08 Thread Boris Lytochkin
Wednesday, August 8, 2007, 6:22:31 PM, you wrote:

 This might be a good place to use the ContentEncoding field to
 describe an 'ondisk' encoding.

How do you expect to use this filed?
Something like '(ondisk|inDB),(base64|quoted-printable|...)'?


 On Aug 8, 2007, at 5:17 AM, Boris Lytochkin wrote:

 Wiki would be enough, but there is a problem in current patch state:
 it uses special constant string inplace of real content to indicate
 that file is located out of DB.
 Right way is to use flag in table. I do not think that changing scheme
  witch patching is a good idea, that is why this patch must be  
 merged into RT.

 This might be a good place to use the ContentEncoding field to  
 describe an 'ondisk' encoding.

 -jesse



-- 
Best regards,
 Boris Lytochkinmailto:[EMAIL PROTECTED]

___
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com
Commercial support: [EMAIL PROTECTED]


Discover RT's hidden secrets with RT Essentials from O'Reilly Media. 
Buy a copy at http://rtbook.bestpractical.com


RE: Re[4]: [rt-users] Attachments table of RT's Mysql database

2007-08-08 Thread Schincke, Keith D. (JSC-IT)[MEI]
You could do that or just have 'ondisk' be a new encoding type added at
the end of the list. 
The Content field of the data base could store the path to the file
relative to a config option.

I would suggest a file size threshold option if RT starts using the file
system to store attachements. 

The folowing config options could be added:
Set( $AttachmentLocation, 'indb'|'ondisk'|'size' ) ;
Set( $AttachmentDiskStorageThreshhold, 4096 ) ;
Set( $AttachmentDiskPath, '/some/location' ) ;

Keith

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Boris
Lytochkin
Sent: Wednesday, August 08, 2007 9:48 AM
To: Jesse Vincent
Cc: rt-users@lists.bestpractical.com
Subject: Re[4]: [rt-users] Attachments table of RT's Mysql database

Wednesday, August 8, 2007, 6:22:31 PM, you wrote:

 This might be a good place to use the ContentEncoding field to 
 describe an 'ondisk' encoding.

How do you expect to use this filed?
Something like '(ondisk|inDB),(base64|quoted-printable|...)'?


 On Aug 8, 2007, at 5:17 AM, Boris Lytochkin wrote:

 Wiki would be enough, but there is a problem in current patch state:
 it uses special constant string inplace of real content to indicate 
 that file is located out of DB.
 Right way is to use flag in table. I do not think that changing 
 scheme  witch patching is a good idea, that is why this patch must be

 merged into RT.

 This might be a good place to use the ContentEncoding field to 
 describe an 'ondisk' encoding.

 -jesse



--
Best regards,
 Boris Lytochkin
mailto:[EMAIL PROTECTED]

___
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com Commercial support:
[EMAIL PROTECTED]


Discover RT's hidden secrets with RT Essentials from O'Reilly Media. 
Buy a copy at http://rtbook.bestpractical.com
___
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com
Commercial support: [EMAIL PROTECTED]


Discover RT's hidden secrets with RT Essentials from O'Reilly Media.
Buy a copy at http://rtbook.bestpractical.com


Re: Re[4]: [rt-users] Attachments table of RT's Mysql database

2007-08-08 Thread Jesse Vincent


On Aug 8, 2007, at 10:47 AM, Boris Lytochkin wrote:


Wednesday, August 8, 2007, 6:22:31 PM, you wrote:


This might be a good place to use the ContentEncoding field to
describe an 'ondisk' encoding.


How do you expect to use this filed?
Something like '(ondisk|inDB),(base64|quoted-printable|...)'?


I think I was envisioning ondisk as a new encoding meaning the  
content is the path to the attachment's content on disk (presumably  
named as a sha1 sum of its content) and not a prefox to the existing  
encodings. There's not much sense in storing stuff on disk in  
something other than raw binary format.



Best,
Jesse


PGP.sig
Description: This is a digitally signed message part
___
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com
Commercial support: [EMAIL PROTECTED]


Discover RT's hidden secrets with RT Essentials from O'Reilly Media. 
Buy a copy at http://rtbook.bestpractical.com

Re[4]: [rt-users] Attachments table of RT's Mysql database

2007-08-07 Thread Viktor
 I personally agree with Boris on this. This should be a configuration option 
 to either store attachments in the database or on the web front end.
So do I.
Moreover, it might be very convenient to store only _unique_ attachments and to 
have a counter of 'uniqueness' for each one. After all, RT is a tracker. Some 
users may tend to send the same files repeatedly (for instance, request 
handlers might send the same patch to a limited number of RT users).
A configuration option to store attachments in server's local FS instead of any 
DBMS would be nice for some configurations.

We currently have RT configuration with ~50Gb pgsql database!
 
 In most enterprise applications you don't store files or session state in the 
 database unless you have a really good reason too. The overhead added by 
 sessions and files results in increased reads from the web server to verify 
 session or large database sizes to store the files in blobs.  

Good point, but for DB transactions facility...

 
 
 
 
 Justin Brodley 
 
 
 -Original Message-
 From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Boris Lytochkin
 Sent: Tuesday, August 07, 2007 6:31 AM
 To: rt-users@lists.bestpractical.com
 Subject: Re[2]: [rt-users] Attachments table of RT's Mysql database
 
 Ken,
 
  First, if everything is inside a database, then a
  simple backup of the database will get everything related to a
  particular RT instance.
 Wrong. We _stopped_ backup process of RT database due to LARGE amount
 data every day. We have no such amount of tape to store DB's everyday
 backups.
 Now, DB backup is done every day and attachment backup is done
 separately. As a result we have everyday SQL-backup of DB and
 incremental backup for attachments. It uses much less space.
 
  Second, in many cases you would like to
  isolate the front-end from the back-end information store. Once
  you need access to the filesystem, everything becomes much more
  involved.
 
 I understand that storing attachments out of RT involves much more
 than DB-only solution, BUT 10 Gb DB with 9.5 Gb of images involves much more.
 
 Anyway, it is up to admin to decide whether to store attachments
 separate or not.
 
 
 Tuesday, August 7, 2007, 4:22:49 PM, you wrote:
 
  Dear Mr. Lytochkin,
 
  There are two very good reasons to not store attachments outside
  of the database. First, if everything is inside a database, then a
  simple backup of the database will get everything related to a
  particular RT instance. Second, in many cases you would like to
  isolate the front-end from the back-end information store. Once
  you need access to the filesystem, everything becomes much more
  involved. I am certain that there are other reasons, but those
  two are certainly enough for me. I have appreciated the ease of
  generating a consistent backup of my RT information store.
 
  Ken
 
  On Tue, Aug 07, 2007 at 04:15:02PM +0400, Boris Lytochkin wrote:
  I wrote a patch that allows to store non-text attachments to be
  stored out of DB - in my case it greatly reduced DB swelling.
  Just for now it uses constant string in Attachments-Content to
  indicate that file is written to FS.
  
  You will need to specify some variables in RT_Siteconfig.pm:
  Set($AttachmentsDirectory, '/var/RT/attachments');
  Set($LogAttachmentsLoading, 1);
  Set($LogAttachmentsSaving, 1);
  Set($StoreNonTextAttachmensInDB, undef);
  #Set($StoreNonTextAttachmensInDB, 1);
  
  A new share/html/Ticket/Attachment/dhandler and attach.patch for
  rest of RT distribution is in attachment.
  
  
  Gregory Harper, you can find more complex set of patches allowing to
  produce  show image thumbs automaticly in attachment too.
  Some more variables must be specified in RT_Siteconfig.pm
  
  Set($ShowTransactionImages, 1);
  Set($ProduceImageThumbs, 1);
  Set($ImageThumbsDirectory, '/var/RT/thumbs');
  
  
  I wonder why bestprcactical is not interested in intergating these
  patches into RT:
  From: Jesse Vincent
  Sent: 21 march 2007 ?., 23:53
  To:   lytochkin
  Subject: [RT 3.6] Storing attachments away from DB
  Hi Boris,
  
  Thanks very much for the mail, but I think we're not really  
  interested in offering this feature within RT.
  Best,
  Jesse
  
  
  
  
  -- 
  Boris Lytochkin,
  JSC e-port, Moscow
  web: www.e-port.ru, wap: wap.e-port.ru
  tel: +7 (495) 777 1872, ext. 251
  
  
  
  
  Date: Mon, 06 Aug 2007 12:31:31 -0500
  From: Gregory Harper [EMAIL PROTECTED]
  Subject: Re: [rt-users] Attachments table of RT's Mysql database
  To: Justin Brodley [EMAIL PROTECTED]
  Cc: rt-users@lists.bestpractical.com
  Message-ID: [EMAIL PROTECTED]
  Content-Type: text/plain; charset=ISO-8859-1; format=flowed
  
  Justin Brodley wrote:
   We actually had to disable the attachment feature as we were having our 
   customers attach enormous files and killed our DB processing. Ultimately 
   we
  are looking
   into rewriting the attachment feature to store the attachments on the 
   web