Thanks for the great explanation Timur. I have bookmarked both and will
look into it shortly.

From:   Timur Sufiev <>
To:     "OpenStack Development Mailing List (not for usage questions)"
Date:   05/17/2016 10:38 AM
Subject:        Re: [openstack-dev] [horizon] [javascript] Async file uploads
            in a        new JS-centric UI

Since 10 lines of code > 1000 words, here are references to 2 patch chains:

* New Swift UI file upload
* New Angular Create Image file upload

I like Create Image solution more because it doesn't use Django csrf_exempt
and single FileField form hacks just to accept a binary stream on Django
side. So CORS offers us a shorter and more elegant solution to the task of
file uploads.

I got an off-ML feedback that the question / intention of original mail is
not clear. My intention / proposal is to adopt the approach used for
uploading files in Create Image workflow as the standard way for other
interactions (which include file uploads) between Web Clients and OpenStack

On Sat, May 14, 2016 at 2:48 AM Timur Sufiev <> wrote:
  Hello, fellow web developers!

  I'd like to share some of my thoughts and findings that I made during
  playing with ng-file-upload [1] for file uploads in Swift UI and Create
  Image wizard in Horizon (both Angular). Sorry for a long letter, couldn't
  put it shorter (TL;DR => go to CONCLUSION section).

  As a foreword: it's a really useful library, both for customizing
  stubborn <input type="file"> widget appearance (hello, themability!) and
  behavior and for the actual file transfer. More on the file transfer
  below, since it's the topic that really interests me.


  First, in modern browsers (by modern I mean HTML5 support and
  particularly FileReader API) you don't need a single-purposed library to
  upload file asynchronously, both jQuery $.ajax() and vanilla Angular
  $http calls support it - just pass File()/Blob() object as data (no
  wrapping {} please) and it works - browser transparently reads data chunk
  by chunk  from a local file system and sends it to the server. There is
  even a working solution for Django and jQuery-based 'Create Image' form
  [2]. There are a few subtleties though. Most importantly, there should be
  no other data (like other key-value pairs from form fields), just the
  file blob - and then the server endpoint must support raw byte stream as
  well. This rules out Django views which expect certain headers and body

  (Second,) What ng-file-upload offers us to solve the challenge of file
  transfers? There are 2 methods in Upload service: .http() and .upload().
  First one is a very thin wrapper around Angular $http, with one
  difference that it allows to notify() browser of file upload progress
  (when just a single file blob is passed in .http(), as in case of $http()
  as well). The second method offers more features, like
  abortable/resumable uploads and transparent handling of data like {key1:
  value1, key2: value2, file: FileBlob}. Uploading such data is implemented
  using standard multipart/form-data content type, so actually, it's just a
  convenient wrapper around facilities we've already seen. Anyways it's
  better to just feed the data into Upload.upload() than to construct
  FormData() on your own (still the same is happening under the bonnet).

  Third, and most important point, we still have to couple Upload.http() /
  Upload.upload() with a server-side machinery. If it's a single file
  upload with Upload.http(), then the server must be able to work with raw
  binary stream (I'm repeating myself here). If it's a form data including
  file blob, it's easily handled on front-end with Upload.upload(), then
  the server must be able to parse multipart/form-data (Django perfectly
  does that). What's bad in this situation is that it also needs to store
  any sufficiently sized file in a web server's file system - which is both
  bug-prone [4] and suboptimal from performance POV. First we need to send
  a file (possibly GB-sized) from browser to web server, then from web
  server to the Glance/Swift/any other service host. So, blindly using
  Upload.upload() won't solve our _real_ problems with file uploads.


  What can be done here to help JS UI to handle really large uploads? Split
  any API calls / views / whatever server things we have into 2 parts:
  lightweight JSON metadata + heavyweight binary stream. Moreover, use CORS
  for the second part to send binary streams directly to that data
  consumers (I know of 2 services atm - Glance & Swift, maybe there are
  more?). That will reduce the processing time, increasing the possibility
  that an operation will complete successfully before Keystone token
  expires :). IMO any new Angular wizard in Horizon should be designed with
  this thing in mind: a separate API call for binary stream transfer.

  Thoughts, suggestions?

  P.S. None of above means that we shouldn't use ng-file-upload, it's still
  a very convenient tool.


  OpenStack Development Mailing List (not for usage questions)

OpenStack Development Mailing List (not for usage questions)

Reply via email to