[
https://issues.apache.org/jira/browse/HTTPCLIENT-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18029840#comment-18029840
]
Oleg Kalnichevski commented on HTTPCLIENT-2401:
-----------------------------------------------
[~Laxika] Exactly what part of the request message generated by the client you
think is wrong or bad?
{noformat}
2025-10-14 20:16:40,129 DEBUG
[main][org.apache.hc.client5.http.impl.classic.MainClientExec] ex-0000000001
executing GET http://crawler-test.com/
2025-10-14 20:16:40,134 DEBUG
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient]
ep-0000000001 start execution ex-0000000001
2025-10-14 20:16:40,134 DEBUG
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager]
ep-0000000001 executing exchange ex-0000000001 over http-outgoing-0
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 >> GET http://crawler-test.com/ HTTP/1.1
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 >> Proxy-Authorization: Basic xyz
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 >> Accept-Encoding: gzip, deflate, lz4-framed, lz4-block,
bzip2, pack200, deflate64, x-gzip
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 >> Host: crawler-test.com
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 >> User-Agent: Apache-HttpClient/5.6-alpha1-SNAPSHOT
(Java/1.8.0_462)
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 >> Upgrade: TLS/1.2
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 >> Connection: Upgrade
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 >> "GET http://crawler-test.com/ HTTP/1.1[\r][\n]"
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 >> "Proxy-Authorization: Basic xyz[\r][\n]"
2025-10-14 20:16:40,135 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 >> "Accept-Encoding: gzip, deflate, lz4-framed, lz4-block,
bzip2, pack200, deflate64, x-gzip[\r][\n]"
2025-10-14 20:16:40,136 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 >> "Host: crawler-test.com[\r][\n]"
2025-10-14 20:16:40,136 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 >> "User-Agent: Apache-HttpClient/5.6-alpha1-SNAPSHOT
(Java/1.8.0_462)[\r][\n]"
2025-10-14 20:16:40,136 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 >> "Upgrade: TLS/1.2[\r][\n]"
2025-10-14 20:16:40,136 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 >> "Connection: Upgrade[\r][\n]"
2025-10-14 20:16:40,136 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 >> "[\r][\n]"
2025-10-14 20:16:40,202 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 << "HTTP/1.1 400 Bad Request[\r][\n]"
2025-10-14 20:16:40,202 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 << "Connection: close[\r][\n]"
2025-10-14 20:16:40,202 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 << "Content-type: text/html[\r][\n]"
2025-10-14 20:16:40,202 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 << "Content-Length: 11[\r][\n]"
2025-10-14 20:16:40,202 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 << "[\r][\n]"
2025-10-14 20:16:40,202 DEBUG [main][org.apache.hc.client5.http.wire]
http-outgoing-0 << "Bad Request"
2025-10-14 20:16:40,204 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 << HTTP/1.1 400 Bad Request
2025-10-14 20:16:40,205 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 << Connection: close
2025-10-14 20:16:40,205 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 << Content-type: text/html
2025-10-14 20:16:40,205 DEBUG [main][org.apache.hc.client5.http.headers]
http-outgoing-0 << Content-Length: 11
2025-10-14 20:16:40,211 DEBUG
[main][org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection]
http-outgoing-0 set socket timeout to 1 SECONDS
2025-10-14 20:16:40,211 DEBUG
[main][org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection]
http-outgoing-0 close connection IMMEDIATE
2025-10-14 20:16:40,212 DEBUG
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient]
ep-0000000001 endpoint closed
2025-10-14 20:16:40,212 DEBUG
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient]
ep-0000000001 discarding endpoint
2025-10-14 20:16:40,212 DEBUG
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager]
ep-0000000001 releasing endpoint
2025-10-14 20:16:40,212 DEBUG
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager]
ep-0000000001 connection is not kept alive
2025-10-14 20:16:40,212 DEBUG
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager]
ep-0000000001 connection released [route:
{}->http://proxy.scrape.do:8080->[http://crawler-test.com:80]][total available:
0; route allocated: 0 of 5; total allocated: 0 of 25]
Bad Request
2025-10-14 20:16:40,212 DEBUG
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager]
Shutdown connection pool GRACEFUL
2025-10-14 20:16:40,213 DEBUG
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager]
Connection pool shut down
{noformat}
> RequestUpgrade interceptor tries to upgrade the connection when using proxies
> -----------------------------------------------------------------------------
>
> Key: HTTPCLIENT-2401
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2401
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
> Affects Versions: 5.5.1
> Reporter: Lakatos Gyula
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{RequestUpgrade}} interceptor tries to upgrade the connection to TLS 1.2
> when proxies are being used. Most proxies don't support HTTPS connections,
> only HTTP tunneling (via {{{}CONNECT{}}}).
> Here is a trivial example that reproduces the error:
> {code:java}
> package com.example.proxy.apachehttpclient5;
> import org.apache.hc.client5.http.classic.methods.HttpGet;
> import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
> import org.apache.hc.client5.http.impl.classic.HttpClients;
> import org.apache.hc.core5.http.HttpHost;
> import org.apache.hc.core5.http.io.entity.EntityUtils;
> import java.io.IOException;
> public class Main {
> public static void main(String[] args) throws IOException {
> try (final CloseableHttpClient httpclient = HttpClients.custom()
> .setProxy(new HttpHost("proxy.scrape.do", 8080))
> .build()) {
> final HttpGet request = new HttpGet("http://crawler-test.com/";);
> request.setHeader("Proxy-Authorization", "Basic xyz");
> String responseContent = httpclient.execute(request, response ->
> EntityUtils.toString(response.getEntity()));
> System.out.println(responseContent);
> }
> } {code}
> The proxy service returns with a 400 Bad Request.
> As a workaround, the logic in the {{RequestUpgrade}} interceptor can be
> disabled with a {{RequestConfig}} that sets {{protocolUpgradeEnabled}} to
> false. However, I think the default behavior is still incorrect in this case
> and is highly unexpected. It took me a significant amount of digging around
> to figure out what's going on in the background.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
