Re: Apache HttpClient include boundary in multipart/form-data

[Bindul Bhowmik]

On Mon, Feb 26, 2018 at 5:27 PM, Arya F  wrote:
> I made a post about this on Stackoverflow and never received an answer.
> Here is the link to the post
> https://stackoverflow.com/questions/48994951/apache-httpclient-include-boundary-in-multipart-form-data
>
> Here is what I am stuck on
>
> I have the following POST request which is supposed to upload a file. But I
> can not figure out how to include the boundary in the request for the
> "Content-Type" header.
>
>
> HttpPost request = new HttpPost(url);
> request.setConfig(config);
>
> StringEntity params = new StringEntity("");
>
> HttpEntity entity = MultipartEntityBuilder.create()
> .addBinaryBody("blob", file,
> ContentType.create("application/octet-stream"), "filename").build();
>
> request.addHeader("Host", "upload.twitter.com");
> request.addHeader("Connection", "keep-alive");
> request.addHeader("User-Agent", userAgent);
> request.addHeader("Content-Type",  );
> request.addHeader("Accept", "*/*");
> request.addHeader("Accept-Encoding", "gzip, deflate, br");
> request.addHeader("Accept-Language", "en-US,en;q=0.9");
> request.addHeader("Cookie", cookies);
>
> request.setEntity(entity);
> response = httpClient.execute(request);

Arya,

I am not sure why you are adding all the request headers manually.
Things like the Content-Type should be handled from the entity
automatically for you.

A simple example code is available on the site at
https://hc.apache.org/httpcomponents-client-ga/httpmime/examples/org/apache/http/examples/entity/mime/ClientMultipartFormPost.java

Bindul

>
> int responseCode = response.getStatusLine().getStatusCode();
>
> System.out.println("upload response code: " + responseCode);
>
>
>
> any idea how this is done?

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Trying to use HttpClient in lieu of HttpsUrlConnection

[Bindul Bhowmik]

On Mon, Feb 12, 2018 at 6:48 PM, Murat Cetin  wrote:
> Hi,
>
> I am having issues with the keep-alive in HttpsUrlConnection in some legacy
> code and considering the HttpClient as an alternative.
>
> My question is, essentially, I have a URLCursor class definition as follows:
>
>public URLCursor(String[] urls, ClientMetadata clientMetadata) {
> this.urls = urls;
> this.urlIdx = 0;
> this.clientMetadata = clientMetadata;
> // Custom trust manager to ignore certification
> TrustManager[] customTrustManager = new TrustManager[]{
> new X509TrustManager() {
> public X509Certificate[] getAcceptedIssuers() {
> return null;
> }
> public void checkClientTrusted(X509Certificate[]
> certs, String authType) {
> }
> public void checkServerTrusted(X509Certificate[]
> certs, String authType) {
> }
> }
> };
> // Custom host verifier to accept all hosts.
> HostnameVerifier allHostsValid = new HostnameVerifier() {
> public boolean verify(String hostname, SSLSession session) {
> return true;
> }
> };
>
> // Setup custom SSL trust manager that ignores SSL certificate
> validation =
> try {
> SSLContext sc = SSLContext.getInstance("SSL");
> sc.init(null, customTrustManager, new 
> java.security.SecureRandom());
> 
> HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
> HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);
> } catch (Exception e) {
> System.err.println("Error: Failed to establish https with
> no cert verification");
> }
> }
>
> I have a subsequent next() method that essentially creates a new URL, opens
> a http connection using url.openConnection(), gets a BufferedReader from
> the input stream and then reads lines out of this stream
>
> How can I achieve the same using HttpClient, especially the constructor
> logic that ignores the certification?

Murat,

>From what I see, what you are doing is disabling hostname and SSL
certificate verification. You can achieve both using a
NoopHostnameVerifier and a TrustAllStrategy for certificates.

You can initialize your HttpClient something like:

SSLContext sslContext = SSLContexts.custom().loadTrustMaterial( new
TrustAllStrategy() ).build();
CloseableHttpClient httpClient =
HttpClients.custom().setSSLHostnameVerifier(
NoopHostnameVerifier.INSTANCE ).setSSLContext( sslContext ).build();

Depending on your use case, unless you are running requests across
multiple threads, you should be able to share the http client instance
created for all your requests.

Disclaimer: it is not a good idea to have any of those verifications
turned off in production.

Bindul

>
> thanks,
> Murat
>

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Advantages of using DNSResolver

[Bindul Bhowmik]

On Fri, Nov 3, 2017 at 10:11 AM, Guru Prashanth Thanakodi
 wrote:
> Hi
> I saw that connectionManager constructor  is taking DNSresolver as
> parameter . There are two types of DNSresolver namely
> SystemDefaultDNSResolver and InMemoryDNSresolver. When does Apache
> httpClient uses these class? If my request url has IPinstead of hostname
> will this DNSresolver class still be used?Do I get optimization if I load
> the InetAddress once and reuse them?

HttpClient uses the SytemDefaultDNSResolver by default. If you need to
customize your dns lookups, you can of course create your own
DNSResolver or use the InMemoryDNSResolver. The
SystemDefaultDNSResolver uses the underlying JRE DNS resolver, which
provides caching in most implementations. Someone with deeper
information into the code can correct me, but as far as I understand
even if you use an IP address, the DNSResolver will be called, but the
method the system default DNS resolver calls -
InetAddress.getAllByName [1] knows if the value passed is an IP
address rather than a hostname, and actually does not do any lookup.

You can of course do pro-active or cache backed DNS resolution and
hook in your own implementation, and again I do not know the details
of your implementation, but you may be missing out on an advantage of
the DNS system - for hosts to be able to move or even provide load
balancing.

>
> Thanks,
> Guru

Regards,
Bindul


[1] 
https://docs.oracle.com/javase/8/docs/api/java/net/InetAddress.html#getAllByName-java.lang.String-

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: SSL Certificate exceptions and defensive coding

[Bindul Bhowmik]

Mohan,

On Sat, Nov 4, 2017 at 6:01 AM, Mohan Radhakrishnan
 wrote:
> Hi,
>
>

Let me start by mentioning that Http Client does not implement its own
cyrpto algorithms or SSL/TLS protocols. It relies on the underlying
Java JSSE implementation to create secure sockets. And unless you have
plugged in an alternate JSSE provider, you are using the JSSE
implementation packaged with your JRE. Having said that, Http Client
does allow some customization to the secure socket creation process
using SSLConnectionSocketFactory [1]. This allows you to use an
alternate trust store, host name verifier, etc.. The latter being
especially valuable during development/testing, but should not be used
in production IMO. To see how you can customize the SSL socket
creation, please read the 'Connection management' chapter in the Http
Client tutorial [2].

>I am invoking a HTTPS SOAP service and this is what I think is
> happening. It is one-way SSL. The JSSE implementation automagically adds it
> to the client's truststore and ensures that the SOAP call is successful.

I am not sure that is accurate. If the server you are targeting uses a
certificate signed by one of the trusted CAs (or a CA signed by a
trusted CA) in your trust store, the secure socket will be
established. The client does not alter the trust store automagically,
unless you have done it yourself, or are using a custom trust store
(other than say the cacerts that ships with Oracle JRE).

>
> This question is based on this assumption.
> When I read
> https://www.ssl.com/guide/ssl-best-practices-a-quick-and-dirty-guide/ there
> are various SSL exceptions and checks that
> are required. I code the client. How do I trap the various exceptions ?
> Which list of exceptions should I use for SSL ?

I am not sure I understand the question, but if you need to catch all
exceptions, see the JSSE Reference Guide [3] on exceptions that can
come from the SSL context. I guess the most important one will be
javax.net.ssl.SSLException and its sub-classes. As far as I can tell,
HC adds org.apache.http.conn.ssl.SSLInitializationException and of
course most connect methods can throw IOException.


>
> How do I know the the HttpClient uses the latest security patches in my JDK
> 8 ? It should automatically be secure. Right ?

As I noted above, HC is using the configured or default JSSE
implementation in your JRE. If that is patched, you will be as secure
as that.

>
> Thanks,
> Mohan

Regards,
Bindul

[1] 
https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/conn/ssl/SSLConnectionSocketFactory.html
[2] http://hc.apache.org/httpcomponents-client-ga/tutorial/html/

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Accept-Encoding:gzip

[Bindul Bhowmik]

On Thu, May 25, 2017 at 11:13 AM, Shaun Elliott  wrote:
> The following code works in v4.2.x
>
> public class RestClientTest {
> private static final Executor EXECUTOR = Executor.newInstance();
>
> public static void main(String[] args) throws IOException {
> Request request =
> Request.Get("http://www.google.com/;).addHeader("Accept-Encoding",
> "gzip");
> Response response = request.execute();//Response
> response = EXECUTOR.execute(request);
> Header[] headers = response.returnResponse().getAllHeaders();
> for (Header header : headers) {
> System.out.println(header);
> }
> }
> }
>
>
> Returned in response headers:
>
> Content-Encoding: gzip
>
>
> But, if I update my version, to 4.3.x+ it no longer works. It appears based
> on this comment:
>
> https://issues.apache.org/jira/browse/HTTPCLIENT-816?focusedCommentId=13998604=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13998604
>
> That gzip encoding was only available in 4.2.x. Is this the case, or is
> there another explanation? Can we get gzip encoding in versions later than
> 4.2.x?

Shaun,

You can achieve that by using the RequestAcceptEncoding request
interceptor, and the ResponseContentEncoding response interceptor. To
use interceptors, see the tutorial section 1.4 [1]. Note that the
ResponseContentEncoding provides 'transparent' content decompression,
and after the content is decompressed you will not see the
Content-Encoding header even though the entity may be compressed on
the wire.

If you are using HttpClients or HttpClientBuilder, the two
interceptors should automatically be added for you, unless you
explicitly call disableContentCompression or add custom compression
handling using setContentDecoderRegistry [2] method.

- Bindul

[1] 
http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/fundamentals.html#protocol_interceptors
[2] 
http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/HttpClientBuilder.html#setContentDecoderRegistry(java.util.Map)

>
> Thanks,
> Shaun Elliott

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: https digest request failing with security exception

[Bindul Bhowmik]

David,

On Fri, Mar 31, 2017 at 1:23 PM, David Thielen  wrote:
> Hi;
>
> Second question. Is there a way to tell it I trust any certificate? That way 
> I can connect to self-signed servers.

You will need a variation of TrustStrategy [1] like
TrustSelfSignedStrategy [2]. The ClientCustomSSL example [3] shows
(among other customizations) how to use a non-default TrustStrategy.

Of course, standard advise to not use that in production code.

- Bindul

[1] 
https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/conn/ssl/TrustStrategy.html
[2] 
https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/conn/ssl/TrustSelfSignedStrategy.html
[3] 
http://hc.apache.org/httpcomponents-client-4.5.x/httpclient/examples/org/apache/http/examples/client/ClientCustomSSL.java

>
> ??? - thanks - dave
>
>
> -Original Message-
> From: Bhowmik, Bindul [mailto:bindulbhow...@gmail.com]
> Sent: Friday, March 31, 2017 11:31 AM
> To: HttpClient User Discussion 
> Subject: Re: https digest request failing with security exception
>
> Dave,
>
> On Fri, Mar 31, 2017 at 11:21 AM, David Thielen  wrote:
>> I have some simple code that does what I think is a correct & basic request 
>> to read the url https://httpbin.org/digest-auth/auth/user/passwd
>>
>> Reading that url in a browser works fine. But the code throws:
>> Exception in thread "main" javax.net.ssl.SSLHandshakeException: 
>> sun.security.validator.ValidatorException: PKIX path building failed: 
>> sun.security.provider.certpath.SunCertPathBuilderException: unable to find 
>> valid certification path to requested target at 
>> sun.security.ssl.Alerts.getSSLException
>>
>> I have the code and full exception stack at 
>> http://stackoverflow.com/questions/43146218/https-digest-request-failing-with-security-exception
>>  (not repeated here to keep this email short).
>>
>> Any ideas?
>
> httpbin.org uses SSL cert from Let's Encrypt, and you are most likely
> hitting an issue with your JRE missing their CA certs. See
> https://community.letsencrypt.org/t/certificate-error-sun-security-validator-validatorexception-pkix-path-building-failed-sun-security-provider-certpath-suncertpathbuilderexception-unable-to-find-valid-certification-path-to-requested-target/28283
>
>>
>> Thanks - dave
>>
>
> - Bindul
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Wire Logging

[Bindul Bhowmik]

Danny,

On Wed, Jun 3, 2009 at 13:53, Danny Gallagher
danny.gallag...@generatedsystems.com wrote:
 Using 3.1



 I cannot seem to get wire logging to produce anything in my log.



 Here is the code:



 FileHandler defaultHandler = new FileHandler(path + BBD%u.log, 5,
 1, true);

 Logger.getLogger().addHandler(defaultHandler);

 Logger.getLogger(org.apache.commons.httpclient).setLevel(Level.ALL);



 Logger.getLogger(org.apache.http.wire.level).setLevel(Level.ALL);

 Logger.getLogger(org.apache.http.level).setLevel(Level.ALL);


You might want to change the two lines above to:
Logger.getLogger(org.apache.http.wire).setLevel(Level.ALL);
Logger.getLogger(org.apache.http).setLevel(Level.ALL);

Basically remove the '.level' from the logger names.





 This produces log entries for the classes under
 org.apache.commons.httpclient, but nothing

 for data going over the wire.



 I can't seem to figure out what I am missing / not doing correctly.



 Any help is appreciated.



 Thanks


HTH
Bindul

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org