#29599: chunk_size for InMemoryUploadedFile is not used
-------------------------------------+-------------------------------------
Reporter: Ali Aliyev | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: File | Version: master
uploads/storage |
Severity: Normal | Resolution: invalid
Keywords: chunks | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Ali Aliyev):
Thanks for the quick response!
I have my own file storage class where `_save` method is called:
https://github.com/django/django/blob/master/django/core/files/storage.py#L49
The problem is that I have to upload chunks (each chunks should not be
more than 4mb) but `content` is instance of the `InMemoryUploadedFile` in
which case I had to implement my own `chunks` method:
{{{
class AzureStorage(Storage):
...
def _read_in_chunks(self, file_object, chunk_size=1024):
while True:
data = file_object.read(chunk_size)
if not data:
break
yield data
def _save(self, name, content):
if hasattr(content.file, 'content_type'):
content_type = content.file.content_type
else:
content_type = mimetypes.guess_type(name)[0]
chunks = self._read_in_chunks(
content,
settings.AZURE_CHUNK_SIZE * 1024 * 1024
)
blocks_list = []
for chunk in chunks:
block_id = uuid.uuid4()
self.connection.put_block(self.azure_container, name, chunk,
block_id)
blocks_list.append(str(block_id))
self.connection.put_block_list(
self.azure_container,
name,
blocks_list,
x_ms_blob_content_type=content_type
)
return name
}}}
so `content.cunks` will not work here
Replying to [comment:1 Carlton Gibson]:
> > is it okay...
>
> Yes.
>
> In general, why do you handle a file in chunks? So that you can control
how much is in memory at one time.
>
> With `InMemoryUploadedFile` you already decided you'd handle the whole
thing in memory at once.
> So the idea of handling it in `chunks` doesn't really make sense.
>
> (See
[https://github.com/django/django/blob/1c05fe65f280cedaddf5e5f308e5e45449b02e93/django/core/files/uploadedfile.py#L95
the comment a few lines below].)
--
Ticket URL: <https://code.djangoproject.com/ticket/29599#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/063.b8f115dc9155b58fa9e5c51bb92fe836%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
