Re: Constructing multipart/mixed requests
Thanks, I thought that GitHub was just a mirror for an Apache repo somewhere! I have opened a PR here - https://github.com/apache/httpcomponents-client/pull/151 On Sun, 2 Jun 2019 at 15:43, Oleg Kalnichevski wrote: > > On June 2, 2019 4:30:44 PM GMT+02:00, Adam Retter wrote: > >Oleg, > > > >Attached is a first draft for a patch against 5.0 to make the > >Multipart stuff more applicable for those people who don't want > >multipart/form-data. What would be the next stage to progress this? > > > > Could you please submit this change-set as a PR at Github? > > Oleg > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. -- Adam Retter eXist Core Developer { United Kingdom } a...@exist-db.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Constructing multipart/mixed requests
On June 2, 2019 4:30:44 PM GMT+02:00, Adam Retter wrote: >Oleg, > >Attached is a first draft for a patch against 5.0 to make the >Multipart stuff more applicable for those people who don't want >multipart/form-data. What would be the next stage to progress this? > Could you please submit this change-set as a PR at Github? Oleg -- Sent from my Android device with K-9 Mail. Please excuse my brevity. - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Constructing multipart/mixed requests
Oleg, Attached is a first draft for a patch against 5.0 to make the Multipart stuff more applicable for those people who don't want multipart/form-data. What would be the next stage to progress this? On Sun, 2 Jun 2019 at 11:23, Oleg Kalnichevski wrote: > > On Sun, 2019-06-02 at 09:47 +0100, Adam Retter wrote: > > > > On my system that produces a HTTP Request like: > > > > > > > > POST / HTTP/1.1 > > > > Content-Length: 456 > > > > Content-Type: multipart/form-data; > > > > boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb > > > > > > I can change this content-type by calling .setContentType() on the > > > entity builder, > > > > Ah yes! I forgot to include that, sorry. > > > > > > > however: > > > > > > > Content-Disposition: form-data; name="part1" > > > > > > Every part gets this content-disposition which is clearly bogus for > > > non-form multipart messages. > > > > I missed that somehow when I was checking the output. The closest I > > think you could get is to override the Content-Disposition in each > > part by using: > > > > addField("Content-Disposition", "inline") > > > > Whether you should use `inline` or `attachment` I cannot say, as I > > don't know enough about your use-case. > > > > Another option, which would be cleaner would be to to derive your own > > builder class from org.apache.http.entity.mime.FormBodyPartBuilder. > > Having studied the code of that class, it looks to me like it already > > does 95% of what you need, and you could just modify your version of > > the `build` method, to not do the Content-Disposition stuff. > > > > Whether that class stays stable over time I cannot say, I agree with > > you that it looks like the HTTP Client is missing an easy way to > > cleanly do multipart. It would seem to me that a > > org.apache.http.entity.mime.MultiPartBuilder would make sense, from > > which org.apache.http.entity.mime.FormBodyPartBuilder is subclassed. > > > > Cheers Adam. > > > > Adam, Norman, et al. > > 12 years ago the plan was to use Apache Mime4j once its APIs got > frozen. The existing multipart code was initially intended as a throw- > away stop-gap fix. > > It looks increasingly likely Mime4j 1.0 will never get released in our > life span, but it is still infinitely more useful and flexible than > what HttpClient has to offer. > > Having said all that, there is still time to get onboard and fix > whatever you deem in need of fixing in 5.0. Time is running out, > though, as we are looking at freezing 5.0 APIs soon. > > By the way, MIME spec refers to MIME headers as `header fields`, so the > choice of method names was not completely random. > > Cheers > > Oleg > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > -- Adam Retter eXist Core Developer { United Kingdom } a...@exist-db.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Constructing multipart/mixed requests
On Sun, 2019-06-02 at 09:47 +0100, Adam Retter wrote: > > > On my system that produces a HTTP Request like: > > > > > > POST / HTTP/1.1 > > > Content-Length: 456 > > > Content-Type: multipart/form-data; > > > boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb > > > > I can change this content-type by calling .setContentType() on the > > entity builder, > > Ah yes! I forgot to include that, sorry. > > > > however: > > > > > Content-Disposition: form-data; name="part1" > > > > Every part gets this content-disposition which is clearly bogus for > > non-form multipart messages. > > I missed that somehow when I was checking the output. The closest I > think you could get is to override the Content-Disposition in each > part by using: > > addField("Content-Disposition", "inline") > > Whether you should use `inline` or `attachment` I cannot say, as I > don't know enough about your use-case. > > Another option, which would be cleaner would be to to derive your own > builder class from org.apache.http.entity.mime.FormBodyPartBuilder. > Having studied the code of that class, it looks to me like it already > does 95% of what you need, and you could just modify your version of > the `build` method, to not do the Content-Disposition stuff. > > Whether that class stays stable over time I cannot say, I agree with > you that it looks like the HTTP Client is missing an easy way to > cleanly do multipart. It would seem to me that a > org.apache.http.entity.mime.MultiPartBuilder would make sense, from > which org.apache.http.entity.mime.FormBodyPartBuilder is subclassed. > > Cheers Adam. > Adam, Norman, et al. 12 years ago the plan was to use Apache Mime4j once its APIs got frozen. The existing multipart code was initially intended as a throw- away stop-gap fix. It looks increasingly likely Mime4j 1.0 will never get released in our life span, but it is still infinitely more useful and flexible than what HttpClient has to offer. Having said all that, there is still time to get onboard and fix whatever you deem in need of fixing in 5.0. Time is running out, though, as we are looking at freezing 5.0 APIs soon. By the way, MIME spec refers to MIME headers as `header fields`, so the choice of method names was not completely random. Cheers Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Constructing multipart/mixed requests
On Sat, 2019-06-01 at 23:27 +0100, Adam Retter wrote: > Hi Norm, > > I think this might be what you are looking for: > > package norm1; > > import org.apache.http.HttpEntity; > import org.apache.http.client.methods.HttpPost; > import org.apache.http.entity.ContentType; > import org.apache.http.entity.mime.FormBodyPart; > import org.apache.http.entity.mime.FormBodyPartBuilder; > import org.apache.http.entity.mime.MultipartEntityBuilder; > import org.apache.http.entity.mime.content.StringBody; > import org.apache.http.impl.client.CloseableHttpClient; > import org.apache.http.impl.client.HttpClientBuilder; > > import java.io.IOException; > > public class MultipartExample { > > public static void main(final String args[]) throws IOException { > > try(final CloseableHttpClient client = > HttpClientBuilder.create().build()) { > > final FormBodyPart part1 = FormBodyPartBuilder.create() > .setName("part1") > .addField("X-HELLO", "adam") > .addField("X-HELLO", "norm") > .setBody(new StringBody("", > ContentType.TEXT_XML)) > .build(); > > final FormBodyPart part2 = FormBodyPartBuilder.create() > .setName("part2") > .addField("X-BYE", "adam") > .addField("X-BYE", "norm") > .setBody(new StringBody("{\"some\": \"json\"}", > ContentType.APPLICATION_JSON)) > .build(); > > final HttpEntity entity = MultipartEntityBuilder.create() > .addPart(part1) > .addPart(part2) > .build(); > > > final HttpPost post = new HttpPost("http://localhost:8080 > "); > post.setEntity(entity); > > client.execute(post); > } > } > } > > > On my system that produces a HTTP Request like: > > POST / HTTP/1.1 > Content-Length: 456 > Content-Type: multipart/form-data; > boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb > Host: localhost:8080 > Connection: Keep-Alive > User-Agent: Apache-HttpClient/4.5.8 (Java/1.8.0_202) > Accept-Encoding: gzip,deflate > > --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb > X-HELLO: adam > X-HELLO: norm > Content-Disposition: form-data; name="part1" > Content-Type: text/xml; charset=ISO-8859-1 > Content-Transfer-Encoding: 8bit > > > --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb > X-BYE: adam > X-BYE: norm > Content-Disposition: form-data; name="part2" > Content-Type: application/json; charset=UTF-8 > Content-Transfer-Encoding: 8bit > > {"some": "json"} > --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb-- > > > Personally I find the naming of some of these things > inaccurate/misleading... i.e. the class `FormBodyPart`, and the > method > `addField` (which appears to actually add a header). Regardless it > seems to "just about work". > > Hope that helps? > > Kind Regards. Adam. > Here's something similar based on Apache Mime4j --- StorageBodyFactory bodyFactory = new StorageBodyFactory(); Message message = Message.Builder.of() .setFrom("John Doe ") .setTo("Mary Smith ") .setSubject("An image for you") .setDate(new Date()) .generateMessageId(InetAddress.getLocalHost().getCanonicalHostName()) .setBody(MultipartBuilder.create("mixed") .use(bodyFactory) .setPreamble("This is a multi-part message in MIME format.") .addBodyPart(BodyPartBuilder.create() .use(bodyFactory) .setBody("Why so serious?", Charsets.UTF_8) .setContentTransferEncoding("quoted-printable") .build()) .addBodyPart(BodyPartBuilder.create() .use(bodyFactory) .setBody(new byte[] { 1, 2, 3}, "application/octet-stream") .setContentTransferEncoding("base64") .setContentDisposition("attachment", "smiley.png") .build()) .build()) .build(); EntityTemplate entityTemplate = new EntityTemplate(outstream -> { try { MessageWriter writer = new DefaultMessageWriter(); writer.writeMessage(message, outstream); } finally { message.dispose(); } }); entityTemplate.setContentType(message.getMimeType()); --- Oleg > On Sat, 1 Jun 2019 at 18:58, Norman Walsh wrote: > > > > Hi all, > > > > Apologies if I’ve been down this road before on this list. The > > HttpClient 4.5 library has changed the way multipart works (from > > some > > earlier 4.x where I had it working). > > > > Near as I can tell from looking at the multipart examples in 4.5, > > they’re all geared towards file uploading. I’m not interested in > > file > > uploading, I want to construct a payload for a web service that’s > > expecting a multipart/mixed request: I want complete control over >
Re: Constructing multipart/mixed requests
> > On my system that produces a HTTP Request like: > > > > POST / HTTP/1.1 > > Content-Length: 456 > > Content-Type: multipart/form-data; > > boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb > > I can change this content-type by calling .setContentType() on the entity > builder, Ah yes! I forgot to include that, sorry. > however: > > > Content-Disposition: form-data; name="part1" > > Every part gets this content-disposition which is clearly bogus for > non-form multipart messages. I missed that somehow when I was checking the output. The closest I think you could get is to override the Content-Disposition in each part by using: addField("Content-Disposition", "inline") Whether you should use `inline` or `attachment` I cannot say, as I don't know enough about your use-case. Another option, which would be cleaner would be to to derive your own builder class from org.apache.http.entity.mime.FormBodyPartBuilder. Having studied the code of that class, it looks to me like it already does 95% of what you need, and you could just modify your version of the `build` method, to not do the Content-Disposition stuff. Whether that class stays stable over time I cannot say, I agree with you that it looks like the HTTP Client is missing an easy way to cleanly do multipart. It would seem to me that a org.apache.http.entity.mime.MultiPartBuilder would make sense, from which org.apache.http.entity.mime.FormBodyPartBuilder is subclassed. Cheers Adam. -- Adam Retter eXist Core Developer { United Kingdom } a...@exist-db.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Constructing multipart/mixed requests
Adam Retter writes: > Hi Norm, Hi Adam. Thanks for the example; you’ve threaded the needle better than I did. Maybe it’s good enough, but it’s still not exactly what I was hoping for. > final FormBodyPart part1 = FormBodyPartBuilder.create() > .setName("part1") The requirement that parts have a name is a form requirement, not a multipart requirement. > On my system that produces a HTTP Request like: > > POST / HTTP/1.1 > Content-Length: 456 > Content-Type: multipart/form-data; boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb I can change this content-type by calling .setContentType() on the entity builder, however: > Content-Disposition: form-data; name="part1" Every part gets this content-disposition which is clearly bogus for non-form multipart messages. Bogus but perhaps harmless. I’m going to run with it and see what happens. I’ll fail all of the XProc multipart tests of course, but… Be seeing you, norm -- Norman Walsh | So, are you working on finding that bug http://nwalsh.com/| now, or are you leaving it until later? | Yes. signature.asc Description: PGP signature
Re: Constructing multipart/mixed requests
Hi Norm, I think this might be what you are looking for: package norm1; import org.apache.http.HttpEntity; import org.apache.http.client.methods.HttpPost; import org.apache.http.entity.ContentType; import org.apache.http.entity.mime.FormBodyPart; import org.apache.http.entity.mime.FormBodyPartBuilder; import org.apache.http.entity.mime.MultipartEntityBuilder; import org.apache.http.entity.mime.content.StringBody; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClientBuilder; import java.io.IOException; public class MultipartExample { public static void main(final String args[]) throws IOException { try(final CloseableHttpClient client = HttpClientBuilder.create().build()) { final FormBodyPart part1 = FormBodyPartBuilder.create() .setName("part1") .addField("X-HELLO", "adam") .addField("X-HELLO", "norm") .setBody(new StringBody("", ContentType.TEXT_XML)) .build(); final FormBodyPart part2 = FormBodyPartBuilder.create() .setName("part2") .addField("X-BYE", "adam") .addField("X-BYE", "norm") .setBody(new StringBody("{\"some\": \"json\"}", ContentType.APPLICATION_JSON)) .build(); final HttpEntity entity = MultipartEntityBuilder.create() .addPart(part1) .addPart(part2) .build(); final HttpPost post = new HttpPost("http://localhost:8080";); post.setEntity(entity); client.execute(post); } } } On my system that produces a HTTP Request like: POST / HTTP/1.1 Content-Length: 456 Content-Type: multipart/form-data; boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb Host: localhost:8080 Connection: Keep-Alive User-Agent: Apache-HttpClient/4.5.8 (Java/1.8.0_202) Accept-Encoding: gzip,deflate --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb X-HELLO: adam X-HELLO: norm Content-Disposition: form-data; name="part1" Content-Type: text/xml; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb X-BYE: adam X-BYE: norm Content-Disposition: form-data; name="part2" Content-Type: application/json; charset=UTF-8 Content-Transfer-Encoding: 8bit {"some": "json"} --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb-- Personally I find the naming of some of these things inaccurate/misleading... i.e. the class `FormBodyPart`, and the method `addField` (which appears to actually add a header). Regardless it seems to "just about work". Hope that helps? Kind Regards. Adam. On Sat, 1 Jun 2019 at 18:58, Norman Walsh wrote: > > Hi all, > > Apologies if I’ve been down this road before on this list. The > HttpClient 4.5 library has changed the way multipart works (from some > earlier 4.x where I had it working). > > Near as I can tell from looking at the multipart examples in 4.5, > they’re all geared towards file uploading. I’m not interested in file > uploading, I want to construct a payload for a web service that’s > expecting a multipart/mixed request: I want complete control over the > parts, the headers associated with those parts, their content types, > etc. > > I stumbled from HttpClient to MIME4J, prehaps on the advice of someone > from this list. I got a little further that way, at least in as much > as I now believe I’ve constructed a mime4j.message that is logically > what I want to send. > > But I cannot see how to get from that back to an HttpEntity that I can > set as the entity for an HttpClient request. > > Does anyone have an example of sending a fully general multipart/mixed > example? Or is anyone familiar enough with both HttpClient and MIME4J > to point me to an explanation of how to turn one of those into a > request Entity? > > (Multipart is only one possibility so I’d really, really rather not > have to have two entirely different code paths where I use HttpClient > for some requests and use direct serialization of MIME4J payloads over > a URLConnection for the other.) > > Help and advice most humbly solicited. > > Be seeing you, > norm > > -- > Norman Walsh | Why do writers write? Because it isn't > http://nwalsh.com/| there.--Thomas Berger -- Adam Retter eXist Core Developer { United Kingdom } a...@exist-db.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Constructing multipart/mixed requests
Hi all, Apologies if I’ve been down this road before on this list. The HttpClient 4.5 library has changed the way multipart works (from some earlier 4.x where I had it working). Near as I can tell from looking at the multipart examples in 4.5, they’re all geared towards file uploading. I’m not interested in file uploading, I want to construct a payload for a web service that’s expecting a multipart/mixed request: I want complete control over the parts, the headers associated with those parts, their content types, etc. I stumbled from HttpClient to MIME4J, prehaps on the advice of someone from this list. I got a little further that way, at least in as much as I now believe I’ve constructed a mime4j.message that is logically what I want to send. But I cannot see how to get from that back to an HttpEntity that I can set as the entity for an HttpClient request. Does anyone have an example of sending a fully general multipart/mixed example? Or is anyone familiar enough with both HttpClient and MIME4J to point me to an explanation of how to turn one of those into a request Entity? (Multipart is only one possibility so I’d really, really rather not have to have two entirely different code paths where I use HttpClient for some requests and use direct serialization of MIME4J payloads over a URLConnection for the other.) Help and advice most humbly solicited. Be seeing you, norm -- Norman Walsh | Why do writers write? Because it isn't http://nwalsh.com/| there.--Thomas Berger signature.asc Description: PGP signature