[PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Hi all, I'm making an effort to clean up the streams code ready for release. I've already commited some docs to the phpdoc repository that will aid extension developers use streams. However, there is more to streams than simple file IO; the new wrapper system provides a much richer set of operations. For example, stat and opendir are routed through the wrapper subsystem, which means that it is possible for a lot of filesystem functions to obtain information about remote files. I've put the infrastructure in place, but now I need your help: I need people that know the protocols to implement the stat and readdir functionality for FTP and HTTP. I'd like to see the remote file stat capability stuff in place for the 4.3 release: readdir is slightly more involved, so I can wait for that. FTP: From the code already written for the ftp wrapper, I can see how to determine the file size, but I don't know how to determine things like ctime and file type. For opendir/readdir, we need to enumerate/list files and folders in a dir. HTTP: = Again, we need to determine file size and change time. This seems quite easy - just do a HEAD request and parse the relevant headers. For readdir, I think we might need to implement a very simple DAV client? Opinions please! I'm looking for 2 volunteers, one for FTP and one for HTTP, to actually write the code - it's not much code to write, you just need to have a clue about the protocols ! :-) --Wez. -- Wez Furlong The Brain Room Ltd. -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Again, we need to determine file size and change time. This seems quite easy - just do a HEAD request and parse the relevant headers. For readdir, I think we might need to implement a very simple DAV client? Opinions please! Yeah, you simply send a PROPFIND request and parse the resulting XML reply. But I am not convinced this is all that useful. DAV really hasn't made too many inroads, so for most users this won't work. -Rasmus -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Hi all, I'm making an effort to clean up the streams code ready for release. I've already commited some docs to the phpdoc repository that will aid extension developers use streams. However, there is more to streams than simple file IO; the new wrapper system provides a much richer set of operations. For example, stat and opendir are routed through the wrapper subsystem, which means that it is possible for a lot of filesystem functions to obtain information about remote files. I've put the infrastructure in place, but now I need your help: I need people that know the protocols to implement the stat and readdir functionality for FTP and HTTP. I'd like to see the remote file stat capability stuff in place for the 4.3 release: readdir is slightly more involved, so I can wait for that. FTP: From the code already written for the ftp wrapper, I can see how to determine the file size, but I don't know how to determine things like ctime and file type. For opendir/readdir, we need to enumerate/list files and folders in a dir. HTTP: = Again, we need to determine file size and change time. This seems quite easy - just do a HEAD request and parse the relevant headers. For readdir, I think we might need to implement a very simple DAV client? Opinions please! I'm looking for 2 volunteers, one for FTP and one for HTTP, to actually write the code - it's not much code to write, you just need to have a clue about the protocols ! :-) Wez, This seems like an ideal time to me to start working on the bundling of cURL into PHP, no? -Sterling --Wez. -- Wez Furlong The Brain Room Ltd. -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
On 08/11/02, Rasmus Lerdorf [EMAIL PROTECTED] wrote: For readdir, I think we might need to implement a very simple DAV client? Opinions please! Yeah, you simply send a PROPFIND request and parse the resulting XML reply. But I am not convinced this is all that useful. DAV really hasn't made too many inroads, so for most users this won't work. I thought that Apache 2 would result in a more widespread usage of DAV (I'm not sure why I think that - something to do with subversion, IIRC)? I suppose we can leave out DAV from the HTTP wrapper; frameworks like PEAR can always implement a DAV protocol wrapper using script anyway, and it'll probably easier to implement an XML parser in script too. --Wez. -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Hi Sterling, I'm not sure :-) I'm 50/50 on integrating curl, because I know that it supports more features than the current wrappers, but I'm not sure if it supports a rich enough API to do the things that the streams API supports. So, I need your opinion: Does libcurl allow you to have progress notification callbacks, so that scripts/extensions can act on notifications such as mime type, file size, and overall progress? (I know it can print out a progress bar, but we need something a little bit more useful than that :-) Can we wrap a curl handle up into a stream without too much trouble? Is libcurl thread-safe? The other stuff should be relatively easy (like HEAD requests and so on), but if those two things look too hard, I'd be tempted to spend the effort on improving the streams code instead of trying to force libcurl to fit into streams. Last time I looked, I didn't like the look of libcurl, but it's been a while, so things might be better now. Opinions please! :-) --Wez. On 08/11/02, Sterling Hughes [EMAIL PROTECTED] wrote: Wez, This seems like an ideal time to me to start working on the bundling of cURL into PHP, no? -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
On Sun, Aug 11, 2002 at 03:49:45PM +0100, Wez Furlong wrote: I thought that Apache 2 would result in a more widespread usage of DAV (I'm not sure why I think that - something to do with subversion, IIRC)? httpd-2.0 includes mod_dav in the stock distribution. -- justin -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Wez, I haven't looked into the streams stuff much, but have had it in the back of my head a lot. Does the streams implementation support 'filters' on streams? With this, whatever gets written to a stream would go through a filter prior to going across the wire. Reason I ask is that this would be a perfect fit for protocols like DIME. Shane Wez Furlong wrote: Hi Sterling, I'm not sure :-) I'm 50/50 on integrating curl, because I know that it supports more features than the current wrappers, but I'm not sure if it supports a rich enough API to do the things that the streams API supports. So, I need your opinion: Does libcurl allow you to have progress notification callbacks, so that scripts/extensions can act on notifications such as mime type, file size, and overall progress? (I know it can print out a progress bar, but we need something a little bit more useful than that :-) Can we wrap a curl handle up into a stream without too much trouble? Is libcurl thread-safe? The other stuff should be relatively easy (like HEAD requests and so on), but if those two things look too hard, I'd be tempted to spend the effort on improving the streams code instead of trying to force libcurl to fit into streams. Last time I looked, I didn't like the look of libcurl, but it's been a while, so things might be better now. Opinions please! :-) --Wez. On 08/11/02, Sterling Hughes [EMAIL PROTECTED] wrote: Wez, This seems like an ideal time to me to start working on the bundling of cURL into PHP, no? -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
On 08/11/02, Rasmus Lerdorf [EMAIL PROTECTED] wrote: For readdir, I think we might need to implement a very simple DAV client? Opinions please! Yeah, you simply send a PROPFIND request and parse the resulting XML reply. But I am not convinced this is all that useful. DAV really hasn't made too many inroads, so for most users this won't work. I thought that Apache 2 would result in a more widespread usage of DAV (I'm not sure why I think that - something to do with subversion, IIRC)? Yes, DAV is bundled with Apache2, but Apache2 is deployed on even less sites than mod_dav. When, if ever, someone fixes all the 3rd party libs on UNIX to be threadsafe and someone adds something to Apache2 that is actually a compelling reason to migrate, then we might start seeing more widespread deployment. I guess my point is that there is no rush on this one. Skip it for 4.3. -Rasmus -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Hi Sterling, I'm not sure :-) I'm 50/50 on integrating curl, because I know that it supports more features than the current wrappers, but I'm not sure if it supports a rich enough API to do the things that the streams API supports. So, I need your opinion: Does libcurl allow you to have progress notification callbacks, so that scripts/extensions can act on notifications such as mime type, file size, and overall progress? (I know it can print out a progress bar, but we need something a little bit more useful than that :-) cURL can handle such things (progress notifications can go to a function). I'm not sure why you would want MIME type notification, but we can add that no problem, and maybe even cURL can be modified to make that easier... cURL does support posting customized headers, btw, if that's where you were going. Can we wrap a curl handle up into a stream without too much trouble? Depends on how you define too-much-trouble :P Using smart_str() and cURL's new multi interface. there are some things that would be different, i would say optimized, mainly that read () requests would be optimal case, not specified case, ie, if the user specifies 1024 bytes to read, and 4096 are available without blocking, then 4096 will be read (it goes without saying that we can buffer this and return 1024). An option could theortically be added to handle specified case instead of optimized case. One of the great things about cURL is that the author is very responsive I've forwarded him this message so that (i forget if currently he's on vacation or not, which is the only time I've ever head a reply lag) he can answer any questions that arise as well... Is libcurl thread-safe? yes, as long as you don't make it un-threadsafe :) When using global dns caching, then it becomes un-ts, but if you avoid this, than you should be fine. Daniel wrote up an API for mutexing globals, and I've got this almost working (a bit more testing before I post it), so this will also be an option (global DNS caching should *greatly* improve speed for php apps, it is already somewhat done i believe, but it can be handled better, and cURL does so). You don't need to use the global dns cache, of course, and if you don't everything else is threadsafe. cURL is heavily used in many multi-thread applications (more so than php, i would gander). The other stuff should be relatively easy (like HEAD requests and so on), but if those two things look too hard, I'd be tempted to spend the effort on improving the streams code instead of trying to force libcurl to fit into streams. Last time I looked, I didn't like the look of libcurl, but it's been a while, so things might be better now. Well, if we are talking about a permanent solution, I think cURL is a much better choice, simply because: 1) its about 3 times as fast as the code in there now 2) its already gone to the trouble of messing with fucked up http, and has plenty of debugging going into it (its becoming the replacement for libwww btw). 3) it offers much richer support for http and ftp, as well as a miriad of other protocols, I want my LDAP streams and I want them now! 4) Reuse, http and friends are constantly changing, we don't have the resources properly, the cURL folks do. I would rather see one php developer improving cURL to meet php's needs, than one php developer maintaining such code in php (if that was even achievable). -Sterling -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Hi Shane, It's planned; depending on how productive I can be with my PHP time, it might make it into PHP 4.3. --Wez. On 08/11/02, Shane Caraveo [EMAIL PROTECTED] wrote: Wez, I haven't looked into the streams stuff much, but have had it in the back of my head a lot. Does the streams implementation support 'filters' on streams? With this, whatever gets written to a stream would go through a filter prior to going across the wire. Reason I ask is that this would be a perfect fit for protocols like DIME. Shane -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Sterling, Thats all I needed to know; I'll take a closer look at the library and see if I can cook up a curl based stream/wrapper. Is there a minimum version of the library (release preferred, of course) that has this new multi-interface, or should I just use the CVS version for now? --Wez. On 08/11/02, Sterling Hughes [EMAIL PROTECTED] wrote: [ Stuff in favour of libcurl ] -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
Sterling, Thats all I needed to know; I'll take a closer look at the library and see if I can cook up a curl based stream/wrapper. Is there a minimum version of the library (release preferred, of course) that has this new multi-interface, or should I just use the CVS version for now? Go with the CVS version, its been in there for a while, but I think just CVS contains the multi version in all its glory (ie, with exported header files). -Sterling -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Streams: FTP/HTTP wrapper gurus wanted
I'm checking it out now: I had 7.9.8 already downloaded, but I think I ran into a bug with the multi interface (it doesn't seem to actually follow HTTP redirects?! - and FOLLOWLOCATION is enabled). The good news is that I have an alpha curl stream/wrapper already working (except for not following redirects!). --Wez. On 08/11/02, Sterling Hughes [EMAIL PROTECTED] wrote: Go with the CVS version, its been in there for a while, but I think just CVS contains the multi version in all its glory (ie, with exported header files). -- PHP Development Mailing List http://www.php.net/ To unsubscribe, visit: http://www.php.net/unsub.php
