Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
I will again suggest http://radinks.com/upload/ . It does not use POST to transfer huge files. Thanks everyone for replying. I had a look at that. My problem is that along with uploading file I also have to post some other information as well. File names are randomly generated. Jupload seems to do the work other than that POST problem. Client is asking for web based solution. I cannot ask him to do ftp. PHP's handling of uploaded files is confusing. I have added logging to the script to which jupload posts. Logging suggests that php file is only accessed after file is completely uploaded by jupload. I also checked apache access logs. Entry in apache log appears only after file is completely uploaded. Another thing that came up with this jupload is that as soon as I hit upload button, a file named phpxx.tmp (xx is any number) is created in temporary directory and its size starts growing as upload happens. If applet doesn't post to php even at start then how come the temporary file is named phpxx.tmp. I tried to upload file on internet using jupload and it seems to me script is timing out even when I have specified 1 hour as timeout. set_time_limit(216000); 137 MB upload seemed to time out. and the set_time_limit doesn't seem to have any affect. Now the even if I accept php's 1.99 GB limit. What to do about this timeout? Do I have to set it in php.ini? Why woudn't it work through the above function? Stut is suggesting it has to be custom made as this kind of solution doesn't seem to already exit. A solution where some applet etc. only passes that information to php which is needed to move the file and update the database. Thanks, Sukhwinder Singh - Original Message - From: Jim Moseby [EMAIL PROTECTED] To: 'Sukhwinder Singh' [EMAIL PROTECTED]; Abdullah Ramazanoglu [EMAIL PROTECTED] Cc: php-general@lists.php.net Sent: Friday, June 08, 2007 5:40 PM Subject: RE: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs Any suggestion to use some other applet. Any freeware etc. Or one not so expensive and which overcomes this php's 1.99 gb limit. I will again suggest http://radinks.com/upload/ . It does not use POST to transfer huge files. JM -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Sukhwinder Singh wrote: PHP's handling of uploaded files is confusing. I have added logging to the script to which jupload posts. Logging suggests that php file is only accessed after file is completely uploaded by jupload. I also checked apache access logs. Entry in apache log appears only after file is completely uploaded. Absolutely correct - the data sent with an HTTP POST will be completely uploaded before the PHP script is started. Likewise, the entry in the apache log will only be done once the request is completed otherwise it doesn't know all the information it needs. Another thing that came up with this jupload is that as soon as I hit upload button, a file named phpxx.tmp (xx is any number) is created in temporary directory and its size starts growing as upload happens. If applet doesn't post to php even at start then how come the temporary file is named phpxx.tmp. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). I tried to upload file on internet using jupload and it seems to me script is timing out even when I have specified 1 hour as timeout. set_time_limit(216000); 137 MB upload seemed to time out. and the set_time_limit doesn't seem to have any affect. Now the even if I accept php's 1.99 GB limit. What to do about this timeout? Do I have to set it in php.ini? Why woudn't it work through the above function? The timeout is likely coming from Apache. It has a limits similar to PHP. Stut is suggesting it has to be custom made as this kind of solution doesn't seem to already exit. A solution where some applet etc. only passes that information to php which is needed to move the file and update the database. Indeed, that is what I'm suggesting. However, I *do* think the solution exists, but I've only ever seen it on specific websites. I've never seen a product that can do this sort of thing. If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big one? -Stut -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Stut, thank you very much for explaining things and trying to help. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). So it seems applet posts complete data to that php file handling script (rather php engine),*as soon as upload button is clicked*, rather than uploading first and then posting to php. That is the reason for name phpxx.tmp of temporary file. As that seems to be the case; there is no real benefit in using applet (in case of http) other than that progress bar which is already supported by php 5.2 It also seems that php engine only hands over control to php file after the file is completely uploaded that is why my custom logging doesn't start as soon as upload starts. It only starts when file is already uploaded. The timeout is likely coming from Apache. It has a limits similar to PHP. Tested on two servers. Both cases it seems to fail if it takes more than 30 seconds. I get an error: [08-Jun-2007 20:15:06] PHP Fatal error: Maximum execution time of 30 seconds exceeded in \handle_upload.php on line 2 If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big one? It is not people that are uploading things. It is admin, and admin is local to server. I won't want to try uploading 4 GB over internet. And yes these are dvd files. They are going to be uploaded for streaming. But along with uploading of files som other fields also have to be uploaded. Client is showing examples like http://www.attachmore.com/Whysubscribe.aspx which claim unlimited upload size. I assume they are using custom activex controls. Looks like I have to post that php's upload limit of 1.99 gb to php internal mailing list. Thanks again, Sukhwinder Singh . - Original Message - From: Stut [EMAIL PROTECTED] To: Sukhwinder Singh [EMAIL PROTECTED] Cc: Jim Moseby [EMAIL PROTECTED]; php-general@lists.php.net Sent: Friday, June 08, 2007 6:42 PM Subject: Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs Sukhwinder Singh wrote: PHP's handling of uploaded files is confusing. I have added logging to the script to which jupload posts. Logging suggests that php file is only accessed after file is completely uploaded by jupload. I also checked apache access logs. Entry in apache log appears only after file is completely uploaded. Absolutely correct - the data sent with an HTTP POST will be completely uploaded before the PHP script is started. Likewise, the entry in the apache log will only be done once the request is completed otherwise it doesn't know all the information it needs. Another thing that came up with this jupload is that as soon as I hit upload button, a file named phpxx.tmp (xx is any number) is created in temporary directory and its size starts growing as upload happens. If applet doesn't post to php even at start then how come the temporary file is named phpxx.tmp. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). I tried to upload file on internet using jupload and it seems to me script is timing out even when I have specified 1 hour as timeout. set_time_limit(216000); 137 MB upload seemed to time out. and the set_time_limit doesn't seem to have any affect. Now the even if I accept php's 1.99 GB limit. What to do about this timeout? Do I have to set it in php.ini? Why woudn't it work through the above function? The timeout is likely coming from Apache. It has a limits similar to PHP. Stut is suggesting it has to be custom made as this kind of solution doesn't seem to already exit. A solution where some applet etc. only passes that information to php which is needed to move the file and update the database. Indeed, that is what I'm suggesting. However, I *do* think the solution exists, but I've only ever seen it on specific websites. I've never seen a product that can do this sort of thing. If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big one? -Stut -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Sukhwinder Singh wrote: Stut, thank you very much for explaining things and trying to help. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). So it seems applet posts complete data to that php file handling script (rather php engine),*as soon as upload button is clicked*, rather than uploading first and then posting to php. That is the reason for name phpxx.tmp of temporary file. As that seems to be the case; there is no real benefit in using applet (in case of http) other than that progress bar which is already supported by php 5.2 It also seems that php engine only hands over control to php file after the file is completely uploaded that is why my custom logging doesn't start as soon as upload starts. It only starts when file is already uploaded. The timeout is likely coming from Apache. It has a limits similar to PHP. Tested on two servers. Both cases it seems to fail if it takes more than 30 seconds. I get an error: [08-Jun-2007 20:15:06] PHP Fatal error: Maximum execution time of 30 seconds exceeded in \handle_upload.php on line 2 If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big one? It is not people that are uploading things. It is admin, and admin is local to server. I won't want to try uploading 4 GB over internet. And yes these are dvd files. They are going to be uploaded for streaming. But along with uploading of files som other fields also have to be uploaded. Client is showing examples like http://www.attachmore.com/Whysubscribe.aspx which claim unlimited upload size. I assume they are using custom activex controls. Looks like I have to post that php's upload limit of 1.99 gb to php internal mailing list. Thanks again, Sukhwinder Singh . - Original Message - From: Stut [EMAIL PROTECTED] To: Sukhwinder Singh [EMAIL PROTECTED] Cc: Jim Moseby [EMAIL PROTECTED]; php-general@lists.php.net Sent: Friday, June 08, 2007 6:42 PM Subject: Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs Sukhwinder Singh wrote: PHP's handling of uploaded files is confusing. I have added logging to the script to which jupload posts. Logging suggests that php file is only accessed after file is completely uploaded by jupload. I also checked apache access logs. Entry in apache log appears only after file is completely uploaded. Absolutely correct - the data sent with an HTTP POST will be completely uploaded before the PHP script is started. Likewise, the entry in the apache log will only be done once the request is completed otherwise it doesn't know all the information it needs. Another thing that came up with this jupload is that as soon as I hit upload button, a file named phpxx.tmp (xx is any number) is created in temporary directory and its size starts growing as upload happens. If applet doesn't post to php even at start then how come the temporary file is named phpxx.tmp. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). I tried to upload file on internet using jupload and it seems to me script is timing out even when I have specified 1 hour as timeout. set_time_limit(216000); 137 MB upload seemed to time out. and the set_time_limit doesn't seem to have any affect. Now the even if I accept php's 1.99 GB limit. What to do about this timeout? Do I have to set it in php.ini? Why woudn't it work through the above function? The timeout is likely coming from Apache. It has a limits similar to PHP. Stut is suggesting it has to be custom made as this kind of solution doesn't seem to already exit. A solution where some applet etc. only passes that information to php which is needed to move the file and update the database. Indeed, that is what I'm suggesting. However, I *do* think the solution exists, but I've only ever seen it on specific websites. I've never seen a product that can do this sort of thing. If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big one? -Stut Have a look at Filechucker http://encodable.com/filechucker/ My company purchased this program for a small project; It's quite inexpensive ($39 USD) and works well. It's written in perl and does not post to PHP, and we've used it to upload large files and it seems to handle them just fine. HTH, Brad -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Brad Fuller wrote: Have a look at Filechucker http://encodable.com/filechucker/ My company purchased this program for a small project; It's quite inexpensive ($39 USD) and works well. It's written in perl and does not post to PHP, and we've used it to upload large files and it seems to handle them just fine. The OP might want to take note of this comment on that page... If you want to upload files larger than 2 gigabytes, then you can't use Apache 1.3.x or 2.0.x. You need to use Apache 2.2.x, as explained on the New Features in Apache 2.2 page: Large File Support httpd is now built with support for files larger than 2GB on modern 32-bit Unix systems. Support for handling 2GB request bodies has also been added. -Stut -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Sukhwinder Singh wrote: Stut, thank you very much for explaining things and trying to help. That's no problem, it's what I'm here for (I knew there was a reason). The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). So it seems applet posts complete data to that php file handling script (rather php engine),*as soon as upload button is clicked*, rather than uploading first and then posting to php. That is the reason for name phpxx.tmp of temporary file. As that seems to be the case; there is no real benefit in using applet (in case of http) other than that progress bar which is already supported by php 5.2 It also seems that php engine only hands over control to php file after the file is completely uploaded that is why my custom logging doesn't start as soon as upload starts. It only starts when file is already uploaded. I think you need to read up on how an HTTP POST request works when it's handled by PHP because some of what you've written there indicates that you don't quite have it. When you hit the upload button the browser makes a connection to the server and the PHP engine gets invoked to handle the request. The engine then reads the entire HTTP request, including the file data that is being uploaded. That temporary file is created by the PHP engine to temporarily store the incoming data. Only when the entire request has been read does the PHP engine start running your PHP script. The timeout is likely coming from Apache. It has a limits similar to PHP. Tested on two servers. Both cases it seems to fail if it takes more than 30 seconds. I get an error: [08-Jun-2007 20:15:06] PHP Fatal error: Maximum execution time of 30 seconds exceeded in \handle_upload.php on line 2 That's the PHP timeout kicking in. There is another setting in php.ini (I forget what it's called) that tells it how long it is allowed to wait for the request to be read, but looking at that error I'm not convinced that's what's causing that timeout. It's very difficult to tell without seeing your script. If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big one? It is not people that are uploading things. It is admin, and admin is local to server. I won't want to try uploading 4 GB over internet. And yes these are dvd files. They are going to be uploaded for streaming. But along with uploading of files som other fields also have to be uploaded. Client is showing examples like http://www.attachmore.com/Whysubscribe.aspx which claim unlimited upload size. I assume they are using custom activex controls. If you look at the screenshots on that site you will see that they install something on your machine that provides the upload capability. Look specifically at the screenshot that shows the user right-clicking on a file in Windows and selecting to upload it. That's not a web-based solution, and if your client is pointing to it as such then they don't really know what they've talking about (like most clients in my experience). Looks like I have to post that php's upload limit of 1.99 gb to php internal mailing list. Definitely worth doing, but don't be surprised if you get the same reaction. -Stut -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
2007. 06. 8, péntek keltezéssel 20.17-kor Sukhwinder Singh ezt írta: Stut, thank you very much for explaining things and trying to help. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). So it seems applet posts complete data to that php file handling script (rather php engine),*as soon as upload button is clicked*, rather than uploading first and then posting to php. That is the reason for name phpxx.tmp of temporary file. The applet you use currently might be doing that, yes. So you should write another applet, which transfers the file by ftp and posts other data and file location to php after the file transfer is complete. I think that's what Stut and others are telling you for a while now... greets Zoltán Németh As that seems to be the case; there is no real benefit in using applet (in case of http) other than that progress bar which is already supported by php 5.2 It also seems that php engine only hands over control to php file after the file is completely uploaded that is why my custom logging doesn't start as soon as upload starts. It only starts when file is already uploaded. The timeout is likely coming from Apache. It has a limits similar to PHP. Tested on two servers. Both cases it seems to fail if it takes more than 30 seconds. I get an error: [08-Jun-2007 20:15:06] PHP Fatal error: Maximum execution time of 30 seconds exceeded in \handle_upload.php on line 2 If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big one? It is not people that are uploading things. It is admin, and admin is local to server. I won't want to try uploading 4 GB over internet. And yes these are dvd files. They are going to be uploaded for streaming. But along with uploading of files som other fields also have to be uploaded. Client is showing examples like http://www.attachmore.com/Whysubscribe.aspx which claim unlimited upload size. I assume they are using custom activex controls. Looks like I have to post that php's upload limit of 1.99 gb to php internal mailing list. Thanks again, Sukhwinder Singh . - Original Message - From: Stut [EMAIL PROTECTED] To: Sukhwinder Singh [EMAIL PROTECTED] Cc: Jim Moseby [EMAIL PROTECTED]; php-general@lists.php.net Sent: Friday, June 08, 2007 6:42 PM Subject: Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs Sukhwinder Singh wrote: PHP's handling of uploaded files is confusing. I have added logging to the script to which jupload posts. Logging suggests that php file is only accessed after file is completely uploaded by jupload. I also checked apache access logs. Entry in apache log appears only after file is completely uploaded. Absolutely correct - the data sent with an HTTP POST will be completely uploaded before the PHP script is started. Likewise, the entry in the apache log will only be done once the request is completed otherwise it doesn't know all the information it needs. Another thing that came up with this jupload is that as soon as I hit upload button, a file named phpxx.tmp (xx is any number) is created in temporary directory and its size starts growing as upload happens. If applet doesn't post to php even at start then how come the temporary file is named phpxx.tmp. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). I tried to upload file on internet using jupload and it seems to me script is timing out even when I have specified 1 hour as timeout. set_time_limit(216000); 137 MB upload seemed to time out. and the set_time_limit doesn't seem to have any affect. Now the even if I accept php's 1.99 GB limit. What to do about this timeout? Do I have to set it in php.ini? Why woudn't it work through the above function? The timeout is likely coming from Apache. It has a limits similar to PHP. Stut is suggesting it has to be custom made as this kind of solution doesn't seem to already exit. A solution where some applet etc. only passes that information to php which is needed to move the file and update the database. Indeed, that is what I'm suggesting. However, I *do* think the solution exists, but I've only ever seen it on specific websites. I've never seen a product that can do this sort of thing. If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
I think you need to read up on how an HTTP POST request works when it's handled by PHP because some of what you've written there indicates that you don't quite have it. When you hit the upload button the browser makes a connection to the server and the PHP engine gets invoked to handle the request. The engine then reads the entire HTTP request, including the file data that is being uploaded. That temporary file is created by the PHP engine to temporarily store the incoming data. Only when the entire request has been read does the PHP engine start running your PHP script. Yes I didn't know. But I learned by actully experimenting with it and discussing on this mailing list. It is not everyday that you have to deal with such issues. Sites like these calim that you can use your web browser to upload 10 gb files. http://www.sendyourfiles.com/features/?web-mail But clients forget that they are using some kind of plugins. That's the PHP timeout kicking in. There is another setting in php.ini (I forget what it's called) that tells it how long it is allowed to wait for the request to be read, but looking at that error I'm not convinced that's what's causing that timeout. I think what you wanted to write was max_input_time. It's very difficult to tell without seeing your script. My script does nothing more than get the file information, move it and update the database. Looks like I have to post that php's upload limit of 1.99 gb to php internal mailing list. Definitely worth doing, but don't be surprised if you get the same reaction. Yes I expect the same response :-). Don't upload 4 gb files over http Thank you very much Stut! - Original Message - From: Stut [EMAIL PROTECTED] To: Sukhwinder Singh [EMAIL PROTECTED] Cc: Jim Moseby [EMAIL PROTECTED]; php-general@lists.php.net Sent: Friday, June 08, 2007 8:01 PM Subject: Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs Sukhwinder Singh wrote: Stut, thank you very much for explaining things and trying to help. That's no problem, it's what I'm here for (I knew there was a reason). The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). So it seems applet posts complete data to that php file handling script (rather php engine),*as soon as upload button is clicked*, rather than uploading first and then posting to php. That is the reason for name phpxx.tmp of temporary file. As that seems to be the case; there is no real benefit in using applet (in case of http) other than that progress bar which is already supported by php 5.2 It also seems that php engine only hands over control to php file after the file is completely uploaded that is why my custom logging doesn't start as soon as upload starts. It only starts when file is already uploaded. I think you need to read up on how an HTTP POST request works when it's handled by PHP because some of what you've written there indicates that you don't quite have it. When you hit the upload button the browser makes a connection to the server and the PHP engine gets invoked to handle the request. The engine then reads the entire HTTP request, including the file data that is being uploaded. That temporary file is created by the PHP engine to temporarily store the incoming data. Only when the entire request has been read does the PHP engine start running your PHP script. The timeout is likely coming from Apache. It has a limits similar to PHP. Tested on two servers. Both cases it seems to fail if it takes more than 30 seconds. I get an error: [08-Jun-2007 20:15:06] PHP Fatal error: Maximum execution time of 30 seconds exceeded in \handle_upload.php on line 2 That's the PHP timeout kicking in. There is another setting in php.ini (I forget what it's called) that tells it how long it is allowed to wait for the request to be read, but looking at that error I'm not convinced that's what's causing that timeout. It's very difficult to tell without seeing your script. If you don't mind me asking, what are people uploading? 4gig is a lot for a single file of any type except maybe DVD images. Is there no way the user can upload several (read lots) smaller files instead of one big one? It is not people that are uploading things. It is admin, and admin is local to server. I won't want to try uploading 4 GB over internet. And yes these are dvd files. They are going to be uploaded for streaming. But along with uploading of files som other fields also have to be uploaded. Client is showing examples like http://www.attachmore.com/Whysubscribe.aspx which claim unlimited upload size. I assume they are using custom activex controls. If you look at the screenshots on that site you will see that they install something on your machine that provides the upload capability. Look specifically
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
On Fri, June 8, 2007 8:58 am, Sukhwinder Singh wrote: I had a look at that. My problem is that along with uploading file I also have to post some other information as well. File names are randomly generated. Jupload seems to do the work other than that POST problem. Client is asking for web based solution. I cannot ask him to do ftp. You are never going to be happy with such large uploads over HTTP, imho. Nor are your users, for several reasons: No way to continue aborted/failed upload, unlike FTP Browser timeout completely outside your control Too many users will think they have to sit and wait to finish PHP's handling of uploaded files is confusing. I have added logging to the script to which jupload posts. Logging suggests that php file is only accessed after file is completely uploaded by jupload. I also checked apache access logs. Entry in apache log appears only after file is completely uploaded. Sort of... You'd have to read Apache and PHP sources to see how it all works for sure, but... Another thing that came up with this jupload is that as soon as I hit upload button, a file named phpxx.tmp (xx is any number) is created in temporary directory and its size starts growing as upload happens. It's entirely possible that Apache and PHP conspire to handle the upload, and do things about the upload, but you won't have any access in php script to it, most likely, until it's all finished. That said, check out Rasmus' recent patch to PHP which allows for an upload progress meter -- It's entirely possible you could glean something useful from that... If applet doesn't post to php even at start then how come the temporary file is named phpxx.tmp. It's posting to Apache which is talking to PHP to handle the upload, but it's a team process, and your PHP script has zero access until it's done. I tried to upload file on internet using jupload and it seems to me script is timing out even when I have specified 1 hour as timeout. set_time_limit(216000); You have to define timing out more clearly... Often-times, a browser will give up, even if the upload is proceeding normally. We cannot fix broken web browsers. No matter how much we'd like to. :-) 137 MB upload seemed to time out. and the set_time_limit doesn't seem to have any affect. Then the timeout isn't coming from PHP, but from another source. Apache may be timing out, a buggy browser may be timing out, the Internet connection may simply not be stable enough for a sustained transfer... Now the even if I accept php's 1.99 GB limit. What to do about this timeout? A 1.99 G limit is probably file-system based... Do I have to set it in php.ini? Why woudn't it work through the above function? You cannot affect file upload in a PHP script, because the upload occurs (or fails to occur) long before your PHP script begins to run -- You can only do it in php.ini or in .htaccess Stut is suggesting it has to be custom made as this kind of solution doesn't seem to already exit. A solution where some applet etc. only passes that information to php which is needed to move the file and update the database. Perhaps you should look at Flikr's application that they implemented for this purpose, or read Building Scalable Web Sites by Cal Henderson (of Flikr) -- Some people have a gift link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
On Fri, June 8, 2007 10:17 am, Sukhwinder Singh wrote: Stut, thank you very much for explaining things and trying to help. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). So it seems applet posts complete data to that php file handling script (rather php engine),*as soon as upload button is clicked*, rather than uploading first and then posting to php. That is the reason for name phpxx.tmp of temporary file. If it tried to upload and then do a separate POST, then you'd have no way to connect the two events, as HTTP is stateless. Or, rather, you'd need to overlay some kind of protocol on top of HTTP that would maintain the state to tie the upload to the POST. It's not impossible but nobody has bothered to do it because HTTP file upload works okay for normal uses, and if you have ginormous files, you'll be better off using FTP anyway. But do feel free to write and release an Open Source solution to this problem. Tested on two servers. Both cases it seems to fail if it takes more than 30 seconds. I get an error: [08-Jun-2007 20:15:06] PHP Fatal error: Maximum execution time of 30 seconds exceeded in \handle_upload.php on line 2 That's php timeout message. You have NOT set the tmeout as you think. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
- Original Message - From: Richard Lynch [EMAIL PROTECTED] To: Sukhwinder Singh [EMAIL PROTECTED] Cc: Jim Moseby [EMAIL PROTECTED]; php-general@lists.php.net; Stut [EMAIL PROTECTED] Sent: Friday, June 08, 2007 11:32 PM Subject: Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs On Fri, June 8, 2007 8:58 am, Sukhwinder Singh wrote: I had a look at that. My problem is that along with uploading file I also have to post some other information as well. File names are randomly generated. Jupload seems to do the work other than that POST problem. Client is asking for web based solution. I cannot ask him to do ftp. You are never going to be happy with such large uploads over HTTP, imho. Nor are your users, for several reasons: No way to continue aborted/failed upload, unlike FTP Browser timeout completely outside your control Too many users will think they have to sit and wait to finish Actually only admin will upload files and that to locally. That is why I am not worried about wasting 15 minutes to upload it through browser. PHP's handling of uploaded files is confusing. I have added logging to the script to which jupload posts. Logging suggests that php file is only accessed after file is completely uploaded by jupload. I also checked apache access logs. Entry in apache log appears only after file is completely uploaded. Sort of... You'd have to read Apache and PHP sources to see how it all works for sure, but... Only if I was a C programmer :-). Not even java programmer anymore. Another thing that came up with this jupload is that as soon as I hit upload button, a file named phpxx.tmp (xx is any number) is created in temporary directory and its size starts growing as upload happens. It's entirely possible that Apache and PHP conspire to handle the upload, and do things about the upload, but you won't have any access in php script to it, most likely, until it's all finished. Yes, that is what is happening. That said, check out Rasmus' recent patch to PHP which allows for an upload progress meter -- It's entirely possible you could glean something useful from that... I have my own ajax script. which uses php 5.2 to monitor progress. The problem is php's limit of around 1.99 GB. You have missed few posts. http://www.nabble.com/RE:-File-Upload---post_max_size-and-upload_max_filesize-in--GBs-t3884697.html In my first email I had specified that when I set php limit to 4G it starts complaining about maximum post length (as it becomes negative). It seems to store values in integer or something. Also even apache 2.0 complains about invalid content length when I try to post 4 gb file. I have checked logs. May be apache 2.2 solves this problem. If applet doesn't post to php even at start then how come the temporary file is named phpxx.tmp. It's posting to Apache which is talking to PHP to handle the upload, but it's a team process, and your PHP script has zero access until it's done. I tried to upload file on internet using jupload and it seems to me script is timing out even when I have specified 1 hour as timeout. set_time_limit(216000); You have to define timing out more clearly... Often-times, a browser will give up, even if the upload is proceeding normally. We cannot fix broken web browsers. No matter how much we'd like to. :-) 137 MB upload seemed to time out. and the set_time_limit doesn't seem to have any affect. Then the timeout isn't coming from PHP, but from another source. Apache may be timing out, a buggy browser may be timing out, the Internet connection may simply not be stable enough for a sustained transfer... Now the even if I accept php's 1.99 GB limit. What to do about this timeout? A 1.99 G limit is probably file-system based... Not file system. I have tried on linux, windows (NTFS). As I explain above about negative numbers of post_max_size. Do I have to set it in php.ini? Why woudn't it work through the above function? You cannot affect file upload in a PHP script, because the upload occurs (or fails to occur) long before your PHP script begins to run -- You can only do it in php.ini or in .htaccess Stut is suggesting it has to be custom made as this kind of solution doesn't seem to already exit. A solution where some applet etc. only passes that information to php which is needed to move the file and update the database. Perhaps you should look at Flikr's application that they implemented for this purpose, or read Building Scalable Web Sites by Cal Henderson (of Flikr) Right. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Tested on two servers. Both cases it seems to fail if it takes more
than 30
seconds.
I get an error:
[08-Jun-2007 20:15:06] PHP Fatal error: Maximum execution time of 30
seconds exceeded in \handle_upload.php on line 2
That's php timeout message. You have NOT set the tmeout as you think.
Yes it is.
I had tried setting it using ini_set and also set_time_limit didn't take
into consideration any.
ini_set('max_execution_time ', 216000);
set_time_limit(216000);
I had to change in php.ini and it seems to work.
Sukhwinder Singh
- Original Message -
From: Richard Lynch [EMAIL PROTECTED]
To: Sukhwinder Singh [EMAIL PROTECTED]
Cc: Stut [EMAIL PROTECTED]; Jim Moseby [EMAIL PROTECTED];
php-general@lists.php.net
Sent: Friday, June 08, 2007 11:39 PM
Subject: Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize
in GBs
On Fri, June 8, 2007 10:17 am, Sukhwinder Singh wrote:
Stut, thank you very much for explaining things and trying to help.
The applet *does* POST to PHP. However, it has the option to upload
to an
FTP server the choice of using HTTP, HTTPS or FTP connections
for
your transfer (from the website).
So it seems applet posts complete data to that php file handling
script
(rather php engine),*as soon as upload button is clicked*, rather than
uploading first and then posting to php. That is the reason for name
phpxx.tmp of temporary file.
If it tried to upload and then do a separate POST, then you'd have no
way to connect the two events, as HTTP is stateless.
Or, rather, you'd need to overlay some kind of protocol on top of HTTP
that would maintain the state to tie the upload to the POST.
It's not impossible but nobody has bothered to do it because HTTP
file upload works okay for normal uses, and if you have ginormous
files, you'll be better off using FTP anyway.
But do feel free to write and release an Open Source solution to this
problem.
Tested on two servers. Both cases it seems to fail if it takes more
than 30
seconds.
I get an error:
[08-Jun-2007 20:15:06] PHP Fatal error: Maximum execution time of 30
seconds exceeded in \handle_upload.php on line 2
That's php timeout message. You have NOT set the tmeout as you think.
--
Some people have a gift link here.
Know what I want?
I want you to buy a CD from some indie artist.
http://cdbaby.com/browse/from/lynch
Yeah, I get a buck. So?
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Richard Lynch wrote: On Fri, June 8, 2007 10:17 am, Sukhwinder Singh wrote: Stut, thank you very much for explaining things and trying to help. The applet *does* POST to PHP. However, it has the option to upload to an FTP server the choice of using HTTP, HTTPS or FTP connections for your transfer (from the website). So it seems applet posts complete data to that php file handling script (rather php engine),*as soon as upload button is clicked*, rather than uploading first and then posting to php. That is the reason for name phpxx.tmp of temporary file. If it tried to upload and then do a separate POST, then you'd have no way to connect the two events, as HTTP is stateless. The name of the uploaded file is all it would need to connect the two events. Or, rather, you'd need to overlay some kind of protocol on top of HTTP that would maintain the state to tie the upload to the POST. If by protocol you mean pass an identifier in the POST to tell it what file it relates to, then yes. And that identifier would be the filename. -Stut -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Sukhwinder Singh wrote: I will again suggest http://radinks.com/upload/ . It does not use POST to transfer huge files. Thanks everyone for replying. I had a look at that. My problem is that along with uploading file I also have to post some other information as well. File names are randomly generated. Jupload seems to do the work other than that POST problem. Client is asking for web based solution. I cannot ask him to do ftp. PHP's handling of uploaded files is confusing. I have added logging to the script to which jupload posts. Logging suggests that php file is only accessed after file is completely uploaded by jupload. I also checked apache access logs. Entry in apache log appears only after file is completely uploaded. Another thing that came up with this jupload is that as soon as I hit upload button, a file named phpxx.tmp (xx is any number) is created in temporary directory and its size starts growing as upload happens. If applet doesn't post to php even at start then how come the temporary file is named phpxx.tmp. I tried to upload file on internet using jupload and it seems to me script is timing out even when I have specified 1 hour as timeout. set_time_limit(216000); The php script that you are posting to does not get parsed until PHP has control of the upload. There for, you cannot override the timeout with uploads. afaik, you are using the stock php.ini setting while you are uploading. look here http://us.php.net/set_time_limit#33462 and this http://us.php.net/set_time_limit#54765 I think you are running into an Apache time limit, not php 137 MB upload seemed to time out. and the set_time_limit doesn't seem to have any affect. Now the even if I accept php's 1.99 GB limit. What to do about this timeout? Do I have to set it in php.ini? Why woudn't it work through the above function? Stut is suggesting it has to be custom made as this kind of solution doesn't seem to already exit. A solution where some applet etc. only passes that information to php which is needed to move the file and update the database. Thanks, Sukhwinder Singh - Original Message - From: Jim Moseby [EMAIL PROTECTED] To: 'Sukhwinder Singh' [EMAIL PROTECTED]; Abdullah Ramazanoglu [EMAIL PROTECTED] Cc: php-general@lists.php.net Sent: Friday, June 08, 2007 5:40 PM Subject: RE: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs Any suggestion to use some other applet. Any freeware etc. Or one not so expensive and which overcomes this php's 1.99 gb limit. I will again suggest http://radinks.com/upload/ . It does not use POST to transfer huge files. JM -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- Jim Lucas Some men are born to greatness, some achieve greatness, and some have greatness thrust upon them. Twelfth Night, Act II, Scene V by William Shakespeare -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
On Fri, June 8, 2007 4:03 pm, Jim Lucas wrote: afaik, you are using the stock php.ini setting while you are uploading. This should not be taken to mean that if one changes php.ini, that the change does not affect file upload -- it definitely does matter... Not sure Jim meant that Sukhwinder was using stock php.ini, or everybody, but it's definitely not the case that everybody is, like it or not -- If that were true, upload_max_filesize would be meaningless. You CANNOT change the file upload time out with set_time_limit, however, because the file upload happens BEFORE your PHP script begins execution. The time limits affecting file upload can only be set: a. in http.conf, for various Apache settings, if any (off-topic) b. in php.ini or .htaccess b1. you'll need to crank up max_input_time, I think b2. max_execution_time for sure b3. memory_limit while you are at it, probably You of course have to crank up upload_max_filesise to way beyond what you expect for an uploaded file. -- Some people have a gift link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Richard Lynch wrote: On Fri, June 8, 2007 4:03 pm, Jim Lucas wrote: afaik, you are using the stock php.ini setting while you are uploading. This should not be taken to mean that if one changes php.ini, that the change does not affect file upload -- it definitely does matter... Not sure Jim meant that Sukhwinder was using stock php.ini, or everybody, but it's definitely not the case that everybody is, like it or not -- If that were true, upload_max_filesize would be meaningless. You CANNOT change the file upload time out with set_time_limit, however, because the file upload happens BEFORE your PHP script begins execution. The time limits affecting file upload can only be set: a. in http.conf, for various Apache settings, if any (off-topic) b. in php.ini or .htaccess b1. you'll need to crank up max_input_time, I think b2. max_execution_time for sure b3. memory_limit while you are at it, probably You of course have to crank up upload_max_filesise to way beyond what you expect for an uploaded file. Don't forget about other php.ini limiters. These are all that I think one would have to adjust to make things work. max_execution_time max_input_time memory_limit post_max_size file_uploads upload_max_filesize Granted, you need to take into account limitations outside of php. ie... available hard drive space any possible hard disk quota limit single file size limit of the OS file size limit of Apache ( mentioned by another replier ) Apache 2.x is less than 2g Apache 2.x is greater than 2g And there might be more that I am not aware of... -- Jim Lucas Some men are born to greatness, some achieve greatness, and some have greatness thrust upon them. Twelfth Night, Act II, Scene V by William Shakespeare -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
On 6/9/07, Jim Lucas [EMAIL PROTECTED] wrote: Richard Lynch wrote: available hard drive space any possible hard disk quota limit single file size limit of the OS file size limit of Apache ( mentioned by another replier ) Apache 2.x is less than 2g Apache 2.x is greater than 2g Stut pointed out that it is in 2.2.x , not 2.x If you want to upload files larger than 2 gigabytes, then you can't use Apache 1.3.x or 2.0.x. You need to use Apache 2.2.x, as explained on the New Features in Apache 2.2 page: Large File Support httpd is now built with support for files larger than 2GB on modern 32-bit Unix systems. Support for handling 2GB request bodies has also been added. Tijnema -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
I think the problem occurred as I specified size in php.ini using 4096M or 4G. Then just now I specified as 4294967294 around 4(GB) and I am getting the POSTed values. *Locally, on windows xp* this is working but when I set these values as as 4294967294 on Mandriva with php 5.1.6 and Apache/2.2.3. I get nothing in post as post_max_size turns negative and it won't allow any content greater than negative value. I am still trying. I Installed Apache 2.2.4 on windows. Now file is being uploaded properly but file was still not moved. I had added a check if ($file_details['size'] 0) // do moving, update databases and it was being returned negative. The logged it and $_FILES showed this for a 3.28 gb file Array ( [File0] = Array ( [name] = FC-6-i386-DVD.iso [type] = application/octet-stream [tmp_name] = C:\DOCUME~1\ADMINI~1.COM\LOCALS~1\Temp\php77.tmp [error] = 0 [size] = -769771520 ) ) I have removed this file size check and file *Uploaded Successfully*. I haven't changed memory limit at all. Script time out I had to specify in php.ini file as one hour. Even though time of around 6 minutes should be enough to do the move_uploaded_file and update database but set_time_limit had no effect. Any guess upgrading to php 5.2.1 on mandriva will solve problem as it already has apache 2.2.3? Sukhwinder Singh - Original Message - From: Jim Lucas [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: Sukhwinder Singh [EMAIL PROTECTED]; Jim Moseby [EMAIL PROTECTED]; php-general@lists.php.net; Stut [EMAIL PROTECTED] Sent: Saturday, June 09, 2007 3:55 AM Subject: Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs Richard Lynch wrote: On Fri, June 8, 2007 4:03 pm, Jim Lucas wrote: afaik, you are using the stock php.ini setting while you are uploading. This should not be taken to mean that if one changes php.ini, that the change does not affect file upload -- it definitely does matter... Not sure Jim meant that Sukhwinder was using stock php.ini, or everybody, but it's definitely not the case that everybody is, like it or not -- If that were true, upload_max_filesize would be meaningless. You CANNOT change the file upload time out with set_time_limit, however, because the file upload happens BEFORE your PHP script begins execution. The time limits affecting file upload can only be set: a. in http.conf, for various Apache settings, if any (off-topic) b. in php.ini or .htaccess b1. you'll need to crank up max_input_time, I think b2. max_execution_time for sure b3. memory_limit while you are at it, probably You of course have to crank up upload_max_filesise to way beyond what you expect for an uploaded file. Don't forget about other php.ini limiters. These are all that I think one would have to adjust to make things work. max_execution_time max_input_time memory_limit post_max_size file_uploads upload_max_filesize Granted, you need to take into account limitations outside of php. ie... available hard drive space any possible hard disk quota limit single file size limit of the OS file size limit of Apache ( mentioned by another replier ) Apache 2.x is less than 2g Apache 2.x is greater than 2g And there might be more that I am not aware of... -- Jim Lucas Some men are born to greatness, some achieve greatness, and some have greatness thrust upon them. Twelfth Night, Act II, Scene V by William Shakespeare -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs
Sorry but I couldn't follow. If I understood correctly, there's 50M (or 4G, for that matter) data, and there's say 5K metadata. First, java applet uploads the bulk data over ftp to a temp directory on the server (employing the ftp service running on the server - not apache/php). If/when the bulk data transfer completes successfully, then the java applet needs to POST the 5K metadata over http to php. Up to this point I can follow. But in your example, the java applet transfers the bulk data itself -not metadata- a second time, POSTing it to php? Yes you are right. I seems to post complete data. Actually I want 4GB uploads as uploads will be local so it is not going to take so much time. The example of 50 MB I provided was to prove that applet was actually POSTing to php and php's post and upload limits came into effect. I am using JUpload. http://jupload.sourceforge.net/ And it POSTs the data to a php script after it uploads. PHP doesn't seem to allow more than 1.9... GB If so, then the applet you use (or the parameters it's been passed) should be broken, I guess. Any suggestion to use some other applet. Any freeware etc. Or one not so expensive and which overcomes this php's 1.99 gb limit. Sukwhinder Singh - Original Message - From: Abdullah Ramazanoglu [EMAIL PROTECTED] To: php-general@lists.php.net Sent: Friday, June 08, 2007 3:37 AM Subject: [PHP] Re: File Upload - post_max_size and upload_max_filesize in GBs Sukhwinder Singh dedi ki: --8-- I have read. I have been trying to find out way for last two days. I can say about jupload and how it seems to work. It uploads file to server in a temporary directory. It has postURL parameter. Then it POSTS the data to php file. Because it POSTS, the php configuration values comes into question. Here is an example: Settings in php.ini are 50M for upload_max_fisesize and 50M for post_max_size. I used japplet and uploaded a 51.89 MB file. This is what is in error log. [08-Jun-2007 03:06:29] PHP Warning: POST Content-Length of 54414946 [bytes exceeds the limit of 52428800 bytes in Unknown on line 0 Sorry but I couldn't follow. If I understood correctly, there's 50M (or 4G, for that matter) data, and there's say 5K metadata. First, java applet uploads the bulk data over ftp to a temp directory on the server (employing the ftp service running on the server - not apache/php). If/when the bulk data transfer completes successfully, then the java applet needs to POST the 5K metadata over http to php. Up to this point I can follow. But in your example, the java applet transfers the bulk data itself -not metadata- a second time, POSTing it to php? If so, then the applet you use (or the parameters it's been passed) should be broken, I guess. Kind regards, -- Abdullah Ramazanoglu aramazan ÄT myrealbox D0T cöm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
