#2070: [patch] Large streaming uploads
----------------------------------------+-----------------------------------
Reporter: [EMAIL PROTECTED] | Owner: jacob
Status: assigned | Component: Core framework
Version: | Resolution:
Keywords: streaming, upload, large | Stage: Accepted
Has_patch: 1 | Needs_docs: 1
Needs_tests: 0 | Needs_better_patch: 1
----------------------------------------+-----------------------------------
Comment (by Joakim Sernbrant <[EMAIL PROTECTED]>):
Replying to [comment:98 Ivan Sagalaev <[EMAIL PROTECTED]>]:
> A couple of comments from quick reading of 4459-streaming-file-
upload.diff
>
> * It streams uploads to files only if FILE_UPLOAD_DIR is set. But it
looks that it stores files in temp directory anyway. What's the point of
FILE_UPLOAD_DIR then?
A StringIO() buffer is used in that case so no files are used.
> * Instead of FILE_UPLOAD_DIR it would be nice to have a setting like
MAX_FILE_UPLOAD_BUFFER with the maximum number of bytes stored in memory
during an upload. And only if the file is bigger it is streamed to disc.
This will have a nice effect of not touching disc when dealing with small
files which is faster
The problem is that you do not know in advance how large a file is.
However you know how large the POST is in total so you could enable file
streaming based on that. I did not want to create too many settings
variables but I agree that it might be a good idea. Another option would
be to have a setting where you name the form fields what you want to have
streamed.
> * It looks like any Exception during the parsing is swallowed and
returned as a dict of errors that is then stored in a POST data. This
actually breaks the whole exceptions paradigm :-).
Yes, I did that to avoid too many changes in wsgi.py/modpython.py since
you have to take care not to try to read the input stream again in case of
an exception. Meaning that I always wanted to return something. Bad choice
maybe but I tried to make as few changes as possible so that the powers
that be could start looking at the patch without too much confusion ;)
> * All the save_FIELD_file now accept raw_field. This is backward-
incompatible but worse it limits this method to work only with uploaded
files while now save_FIELD_file can accept any content not matter how it
was created in a program. I understand that just renaming streamed upload
is a fast feature but I believe it deserves a new method. Like
'save_FIELD_from_file' or something...
Aren't those methods invoked automatically for you when you do .save()? I
didnt think they were public but I dont understand exacly how they work so
I take your word for it...
--
Ticket URL: <http://code.djangoproject.com/ticket/2070#comment:99>
Django Code <http://code.djangoproject.com/>
The web framework for perfectionists with deadlines
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en
-~----------~----~----~----~------~----~------~--~---
