Hi Mathias,
I think you are suggesting pretty much exactly what I did, but I did it in such
a way that I didn’t have to make each body part (corresponding to a form
parameter) manually (I used mapping).
However, AWS/S3 will not accept the Content-Type: text/plain in the body parts.
It thinks the request is trying to “upload” multiple files. I had to use
ContentTypes.NoContentType.
See below for my approach (where m is a map of the parameter/value pairs):
val strictFormDataEntityFuture = m.map { case (k, v) => (k,
HttpEntity(ContentTypes.NoContentType, v).toStrict(timeout)) }
That produces a Map[String, Future[HttpEntity.Strict]], which is then converted
to a Future[Map[String,HttpEntity.Strict]] with:
for {
strictEntityMap <- Future.sequence(strictFormDataEntityFuture.map(entry
=> entry._2.map(i => (entry._1, i)))).map(_.toMap)
…
AWS/S3 doesn’t want the form data parts, nor the whole HTTP POST to be chunked.
So those entities have to be made “strict”. So I had to make each MIME part be
strict, as well as the final Multipart.FormData entity.
The point of my original post was that it seems the kaka-http library has made
a policy decision that the only way to convert a FormData to an entity (in a
multi-part MIME body) is to create a single part, using www-url-encoding, and
include all fields in query string syntax. This doesn’t work for AWS and
therefore it seems a limiting policy decision. It should be equally
possible/easy to convert a FormData to a collection of parts. The requirement
that there should be a Content-Type header in these parts, also should not be
dictated by the framework.
— Eric
> On Oct 14, 2015, at 5:21 AM, Mathias <[email protected]> wrote:
>
> Eric,
>
> you can create a multipart/formdata request like this, for example:
>
> val formData = {
> def utf8TextEntity(content: String) = {
> val bytes = ByteString(content)
> HttpEntity.Default(ContentTypes.`text/plain(UTF-8)`, bytes.length,
> Source.single(bytes))
> }
>
> import Multipart._
> FormData(Source(
> FormData.BodyPart("foo", utf8TextEntity("FOO")) ::
> FormData.BodyPart("bar", utf8TextEntity("BAR")) :: Nil))
> }
>
> val request = HttpRequest(POST, entity = formData.toEntity())
>
> This will render a chunked request that looks something like this:
>
> POST / HTTP/1.1
> Host: example.com
> User-Agent: akka-http/test
> Transfer-Encoding: chunked
> Content-Type: multipart/form-data; boundary=o5bjIygPrPrCCYUp8FyfvsQe
>
> 71
> --o5bjIygPrPrCCYUp8FyfvsQe
> Content-Type: text/plain; charset=UTF-8
> Content-Disposition: form-data; name=foo
>
>
> 3
> FOO
> 73
>
> --o5bjIygPrPrCCYUp8FyfvsQe
> Content-Type: text/plain; charset=UTF-8
> Content-Disposition: form-data; name=bar
>
>
> 3
> BAR
> 1e
>
> --o5bjIygPrPrCCYUp8FyfvsQe--
> 0
>
> If you want to force the rendering of an unchunked request you can call
> `toStrict` on the `formData` first.
> This would create a request looking something like this:
>
> POST / HTTP/1.1
> Host: example.com
> User-Agent: akka-http/test
> Content-Type: multipart/form-data; boundary=1QipXPrd9L25eJwcQYgiNbfA
> Content-Length: 264
>
> --1QipXPrd9L25eJwcQYgiNbfA
> Content-Type: text/plain; charset=UTF-8
> Content-Disposition: form-data; name=foo
>
> FOO
> --1QipXPrd9L25eJwcQYgiNbfA
> Content-Type: text/plain; charset=UTF-8
> Content-Disposition: form-data; name=bar
>
> BAR
> --1QipXPrd9L25eJwcQYgiNbfA--
>
> There is no www-url-encoding happening anywhere.
>
> Cheers,
> Mathias
>
>
> On Tuesday, October 13, 2015 at 11:33:43 PM UTC+2, Eric Swenson wrote:
> I have not been able to find any way to marshall a set of HTTP Post form
> parameters AND a file using akka-http, where the form data parameters are NOT
> www-url-encoded, but rather each placed in a separate MIME part, with no
> Content-Type header. This appears to be the format required in order to do an
> HTTP POST to AWS S3 with a policy. Furthermore, it appears AWS/S3 is not
> happy with Chunked HTTP POST requests, and therefore I've had to use toStrict
> on all my entities. I've resorted to some hacky code, which works, but I
> find it surprising that there is no built-in way to generate a multi-part
> MIME message where post parameters are not combined into one www-url-encoded
> query string.
>
> Here is my (admittedly ugly) code:
>
> val m = Map(
>
> "key" -> "upload/quux.txt",
>
> "acl" -> acl,
>
> "policy" -> policy,
>
> "X-amz-algorithm" -> "AWS4-HMAC-SHA256",
>
> "X-amz-credential" ->
> s"$awsAccessKeyId/$shortDate/$region/$service/aws4_request",
>
> "X-amz-date" -> date,
>
> "X-amz-expires" -> expires.toString,
>
> "X-amz-signature" -> signature
>
> )
>
>
> implicit val timeout: FiniteDuration = 10.seconds
>
>
> val uri = s"http://$bucket.$service-$region.amazonaws.com
> <http://amazonaws.com/>"
>
>
> // create request entities for each of the POST parameters. Note that
> generating Strict entities is an asynchronous
>
> // operation (returns a future). Strict entities are NOT chunked. It
> appears AWS is not happy with chunked MIME
>
> // body parts.
>
> val strictFormDataEntityFuture = m.map { case (k, v) => (k,
> HttpEntity(ContentTypes.NoContentType, v).toStrict(timeout)) }
>
>
> val responseFuture = for {
>
> strictEntityMap <- Future.sequence(strictFormDataEntityFuture.map(entry
> => entry._2.map(i => (entry._1, i)))).map(_.toMap) recoverWith { case t:
> Throwable => throw t }
>
> formDataBodyPartList = strictEntityMap.map {e =>
> Multipart.FormData.BodyPart.Strict(e._1, e._2)}.toList
>
> fileEntity <- HttpEntity(ContentTypes.`application/octet-stream`, 3,
> Source.single(ByteString("foo"))).toStrict(timeout) recoverWith { case t:
> Throwable => throw t }
>
> multipartForm = Multipart.FormData(Source(formDataBodyPartList
> ++List(Multipart.FormData.BodyPart("file", fileEntity, Map("filename" ->
> "foo.bin")))))
>
> requestEntity <- Marshal(multipartForm).to[RequestEntity] recoverWith {
> case t: Throwable => throw t }
>
> strictRequestEntity <- requestEntity.toStrict(timeout) recoverWith {
> case t: Throwable => throw t }
>
> response <- Http().singleRequest(HttpRequest(method = HttpMethods.POST,
> uri = uri, entity = strictRequestEntity)) recoverWith { case t: Throwable =>
> throw t }
>
> responseEntity <- Unmarshal(response.entity).to[String] recoverWith {
> case t: Throwable => throw t }
>
> } yield (responseEntity, response)
>
>
>
> If anyone can suggest a better approach, I'd appreciate it. Thanks.
>
>
> On Sunday, October 11, 2015 at 10:45:37 PM UTC-7, Eric Swenson wrote:
> Hey Scott!
>
> When you get a chance, if you can point me to the right one, I'd appreciate
> it. I've been looking at the sources, tests, and examples, and I can't find
> how to do it. What I'm after is a way to take a Form, and emit a multi-part
> MIME message with one part for each field in the form and one part for the
> single file I need to upload. Using code like this:
>
> val formBodyPartFuture = for {
> httpEntity <- Marshal(form).to[HttpEntity]
>
> strictEntity <- httpEntity.toStrict(timeout)
>
> } yield Multipart.FormData.BodyPart("foo", strictEntity)
>
>
>
> gives me one body part with Content-Type x-www-url-encoded, and the set of
> all fields are encoded as in a query string.If you can point me to the
> "other" way of marshalling a form to a set of mime body parts, I'd appreciate
> it. I had pretty much given up (and googling and using stack overflow have
> turned up nothing) and was going to generate the parts manually. i'm sure
> there must be a correct way to do this. Thanks. -- Eric
>
>
>
>
>
>
> On Sunday, October 11, 2015 at 9:43:20 PM UTC-7, Scott Maher wrote:
> I can't test this as I am not at home but there are multiple form
> marshallers, one for url encoding and one for multipart called, I think,
> Multipart.FormData. sorry if you already know this.
>
> Hi Eric! :P
>
> On Oct 11, 2015 9:18 PM, "Eric Swenson" <[email protected] <>> wrote:
> AWS/S3 HTTP Post with a policy requires an HTTP post with multipart-mime --
> one part for the file and one part, each, for various form parameters. When I
> use akka.http and attempt to marshal FormData to an Entity and use that to
> create a multi-part mime message, all the form data go into one mime-part
> using the query string format (foo=bar&baz=quux). AWS rejects such a request
> as it doesn't want all form parameters in a single www-url-encoded-form-data
> part, but rather separate mime parts for each parameter. How to I cause a
> Form to be martialled in this way using akka.http?
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
> >>>>>>>>>> Check the FAQ:
> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> >>>>>>>>>> <https://groups.google.com/group/akka-user>
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <>.
> To post to this group, send email to [email protected] <>.
> Visit this group at http://groups.google.com/group/akka-user
> <http://groups.google.com/group/akka-user>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/>
> >>>>>>>>>> Check the FAQ:
> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html>
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> >>>>>>>>>> <https://groups.google.com/group/akka-user>
> ---
> You received this message because you are subscribed to a topic in the Google
> Groups "Akka User List" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/akka-user/ChTzhtmTvyU/unsubscribe
> <https://groups.google.com/d/topic/akka-user/ChTzhtmTvyU/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to
> [email protected]
> <mailto:[email protected]>.
> To post to this group, send email to [email protected]
> <mailto:[email protected]>.
> Visit this group at http://groups.google.com/group/akka-user
> <http://groups.google.com/group/akka-user>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
