Re: [PHP-DEV] [PATCH] PHP_IMAP New Function imap_partialbody to fetch chunks of a MIME attachment
How about the idea to make a function like imap_getbodystream(...) that
returns a PHP stream that can be read with fread() etc. and then closed by
fclose()?. This could be done by using Wez's PHP stream wrappers in PHP5
(even in PHP4.3).
Uwe
At 00:47 15.08.2004, you wrote:
This new function is intended to add chunking capability to fetching
attachments from an email using the IMAP library. Currently the attachment
fetch is done with imap_fetchbody , and is placed entirely in memory, ergo
- if the attachment is larger than the available memory it will fail.
This function has the following prototype - string
imap_partialbody(resource stream_id, int msg_no, int section, int start,
int len [, int options])
It's exactly like imap_fetchbody, except it adds the parameters start
(for the start position offset) and len (for the number of bytes to
retrieve). I've also made sure that the correct checks for message number
are in the code (as per the recent change to imap_fetchbody).
It uses the function mail_partial_body in the C Client, which doesn't
appear in any of the very dated documentation, but I found a few postings
by Mark Crispin explaining how to use it, and it's also been used in pine.
One complication is that it uses a callback to populate the buffer, but I
found some hints on how to use a spare pointer element in the IMAP
stream structure from a post by him in one of the online IMAP forums here
- http://www.webservertalk.com/message299504.html
Obviously the benefit is that a new attachment retrieval could be
implemented in webmail such as IMP that doesn't require the memlimit on
the server to be set to some absurd value, and would be less likely bring
the server to its knees when some idiot mails a 50M video to a group of
his friends. Note however c-client is still greedy about memory for its
cache (theres nothing that can be done about that) - but this doesn't
figure in the PHP memlimit.
A snippet of some resulting PHP using the function (struct is the return
from imap_fetchstructure - note I haven't put anything in to decode the
structure or errorcheck the return for brevity). CHUNKSIZE is how much to
fetch at a time (100K works quite well)
$size=$struct-parts[$partindex]-bytes;
$fp=fopen($attachfile,w);
for ($i=0;$i$size;$i+=CHUNKSIZE)
{
$len = (($size-$i)CHUNKSIZE) ? $size-$i:CHUNKSIZE;
$pbody = imap_partialbody($imap,$msgno,$partno,$i,$len);
fwrite($fp,$pbody);
}
You will find this code works fine on a large attachment, and doesn't blow
the memlimit when its set to the default of 8M.
The next problem of course is that php/php_imap doesn't have a streamed
Base64 decoder - though I've used a simple command line one quite effectively.
I'm guessing if this is of interest, it will be of most interest to the
HORDE/IMP team?
Crispin Olson
-
Uwe Schindler
[EMAIL PROTECTED] - http://www.php.net
NSAPI SAPI developer
Erlangen, Germany
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [PATCH] PHP_IMAP New Function imap_partialbody to fetch chunks of a MIME attachment
Uwe Schindler wrote: How about the idea to make a function like imap_getbodystream(...) that returns a PHP stream that can be read with fread() etc. and then closed by fclose()?. This could be done by using Wez's PHP stream wrappers in PHP5 (even in PHP4.3). Uwe Uwe, Not a bad idea, and I had thought of it too. Surely I need to be looking at the Streams API for extension authors though, rather than the wrapper functions if I want to create and maintain my own stream? At this point I've kept it simple, because for the most part, php_imap is a simple wrapper layer onto the UW c-client, and that is also true for this function. Crispin -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [PATCH] PHP_IMAP New Function imap_partialbody to fetch chunks of a MIME attachment
This would be also interesting: For decoding base64 binarys you could use the string.base64 stream filter then (PHP5)! :) Uwe At 18:50 15.08.2004, Crispin Olson wrote: Uwe Schindler wrote: How about the idea to make a function like imap_getbodystream(...) that returns a PHP stream that can be read with fread() etc. and then closed by fclose()?. This could be done by using Wez's PHP stream wrappers in PHP5 (even in PHP4.3). Uwe Uwe, Not a bad idea, and I had thought of it too. Surely I need to be looking at the Streams API for extension authors though, rather than the wrapper functions if I want to create and maintain my own stream? At this point I've kept it simple, because for the most part, php_imap is a simple wrapper layer onto the UW c-client, and that is also true for this function. Crispin -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php - Uwe Schindler [EMAIL PROTECTED] - http://www.php.net NSAPI SAPI developer Erlangen, Germany -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] [PATCH] PHP_IMAP New Function imap_partialbody to fetch chunks of a MIME attachment
This new function is intended to add chunking capability to fetching
attachments from an email using the IMAP library. Currently the
attachment fetch is done with imap_fetchbody , and is placed entirely in
memory, ergo - if the attachment is larger than the available memory it
will fail.
This function has the following prototype - string
imap_partialbody(resource stream_id, int msg_no, int section, int start,
int len [, int options])
It's exactly like imap_fetchbody, except it adds the parameters start
(for the start position offset) and len (for the number of bytes to
retrieve). I've also made sure that the correct checks for message
number are in the code (as per the recent change to imap_fetchbody).
It uses the function mail_partial_body in the C Client, which doesn't
appear in any of the very dated documentation, but I found a few
postings by Mark Crispin explaining how to use it, and it's also been
used in pine. One complication is that it uses a callback to populate
the buffer, but I found some hints on how to use a spare pointer
element in the IMAP stream structure from a post by him in one of the
online IMAP forums here - http://www.webservertalk.com/message299504.html
Obviously the benefit is that a new attachment retrieval could be
implemented in webmail such as IMP that doesn't require the memlimit on
the server to be set to some absurd value, and would be less likely
bring the server to its knees when some idiot mails a 50M video to a
group of his friends. Note however c-client is still greedy about memory
for its cache (theres nothing that can be done about that) - but this
doesn't figure in the PHP memlimit.
A snippet of some resulting PHP using the function (struct is the return
from imap_fetchstructure - note I haven't put anything in to decode the
structure or errorcheck the return for brevity). CHUNKSIZE is how much
to fetch at a time (100K works quite well)
$size=$struct-parts[$partindex]-bytes;
$fp=fopen($attachfile,w);
for ($i=0;$i$size;$i+=CHUNKSIZE)
{
$len = (($size-$i)CHUNKSIZE) ? $size-$i:CHUNKSIZE;
$pbody = imap_partialbody($imap,$msgno,$partno,$i,$len);
fwrite($fp,$pbody);
}
You will find this code works fine on a large attachment, and doesn't
blow the memlimit when its set to the default of 8M.
The next problem of course is that php/php_imap doesn't have a streamed
Base64 decoder - though I've used a simple command line one quite
effectively.
I'm guessing if this is of interest, it will be of most interest to the
HORDE/IMP team?
Crispin Olson
Index: php_imap.c
===
RCS file: /repository/php-src/ext/imap/php_imap.c,v
retrieving revision 1.142.2.28
diff -u -r1.142.2.28 php_imap.c
--- php_imap.c 12 Aug 2004 19:32:59 - 1.142.2.28
+++ php_imap.c 14 Aug 2004 22:44:38 -
@@ -92,6 +92,7 @@
PHP_FE(imap_body,
NULL)
PHP_FE(imap_bodystruct, NULL)
PHP_FE(imap_fetchbody, NULL)
+ PHP_FE(imap_partialbody,
NULL)
PHP_FE(imap_fetchheader,NULL)
PHP_FE(imap_fetchstructure, NULL)
PHP_FE(imap_expunge,NULL)
@@ -1792,6 +1793,65 @@
/* }}} */
+/* Author: CNO - callback for mail_partial_body */
+char* mm_gets(readfn_t f,void* stream,unsigned long size,GETS_DATA* md)
+{
+ SIZEDTEXT *buffer;
+ buffer = emalloc(sizeof(SIZEDTEXT));
+ buffer-data = emalloc(size+1);
+ f(stream,size,buffer-data);
+ buffer-size = size;
+ md-stream-sparep=buffer;
+ return buffer-data;
+}
+/* {{{ proto string imap_partialbody(resource stream_id, int msg_no, int section, int
start, int len [, int options])
+ Get a specific body section chunk */
+PHP_FUNCTION(imap_partialbody)
+{
+ zval **streamind, **msgno, **sec, **start, **len, **flags;
+ pils *imap_le_struct;
+ long status;
+ char *body;
+ SIZEDTEXT *retbuff;
+ int myargc=ZEND_NUM_ARGS();
+
+ if (myargc 5 || myargc 6 || zend_get_parameters_ex(myargc, streamind,
msgno, sec,
+ start, len, flags) == FAILURE) {
+ ZEND_WRONG_PARAM_COUNT();
+ }
+
+ ZEND_FETCH_RESOURCE(imap_le_struct, pils *, streamind, -1, imap, le_imap);
+
+ convert_to_long_ex(msgno);
+ convert_to_string_ex(sec);
+ convert_to_long_ex(start);
+ convert_to_long_ex(len);
+ if (myargc == 6) {
+ convert_to_long_ex(flags);
+ }
+
+ if (myargc 6 || !(Z_LVAL_PP(flags) FT_UID)) {
+ /* only perform the check if the msgno is a message number and not a
UID */
+
