Am 15. Dezember 2018 13:38:37 MEZ schrieb Philippe Mouawad
<[email protected]>:
>Great you have a simple reproducer.
>On my side , looking inti HC4 I notice issue triggers when we enter
>proxy
>tunnel mode
Right, but we enter that only on https requests.
>
>http://httpcomponents.10934.n7.nabble.com/Strange-behaviour-when-using-Proxy-td34776.html
>
>
>did you submit a bug or not yet ?
No. I haven't done that.
Felix
>
>Thanks
>On Saturday, December 15, 2018, Felix Schumacher <
>[email protected]> wrote:
>
>>
>> Am 14.12.18 um 17:37 schrieb Philippe Mouawad:
>>
>>> Hi Felix,
>>> Thanks for investigations !
>>> Is it an issue due to our code or a bug in hc4?
>>>
>>
>> It looks like a bug in hc4 to me (or we haven't understood that
>feature :)
>>
>> If you change your test code to use https for the request, you will
>get
>> the same behaviour that we see in our bug report.
>>
>> @Test
>> public void checkThatHeadersAreNotHidden() throws Exception {
>> TrustStrategy trustStrategy = new TrustAllStrategy();
>> SSLContext sslContext = new SSLContextBuilder()
>> .loadTrustMaterial(null, trustStrategy).build();
>> SSLConnectionSocketFactory socketFactory = new
>> SSLConnectionSocketFactory(
>> sslContext);
>> CloseableHttpClient httpclient = HttpClients.custom()
>> .setSSLSocketFactory(socketFactory).build();
>> try {
>>
>> HttpHost target = new HttpHost("jmeter.apache.org", 443,
>> "https");
>> HttpHost proxy = new HttpHost("localhost", 8888, "http");
>>
>> RequestConfig config =
>RequestConfig.custom().setProxy(proxy)
>> .build();
>> HttpGet request = new HttpGet("/");
>> request.addHeader("X-sleep", "5");
>> request.setConfig(config);
>>
>> HttpContext localContext = new BasicHttpContext();
>> CloseableHttpResponse response =
>httpclient.execute(target,
>> request,
>> localContext);
>> final HttpRequest httpRequestFromLocalContext =
>(HttpRequest)
>> localContext
>> .getAttribute(HttpCoreContext.HTTP_REQUEST);
>> try {
>> Assert.assertThat(httpRequestFromLocalContext.getRequestLine()
>> .getMethod(), CoreMatchers.is("CONNECT"));
>> Assert.assertThat(response.getStatusLine().getStatusCode(),
>> CoreMatchers.is(200));
>> Assert.assertThat(
>> Arrays.asList(request.getAllHeaders()).toString(),
>> CoreMatchers.containsString("X-sleep"));
>> Assert.assertThat(
>> Arrays.asList(
>> httpRequestFromLocalContext.getAllHeaders())
>> .toString(),
>> CoreMatchers
>> .not(CoreMatchers.containsString("X-sleep")));
>> } finally {
>> response.close();
>> }
>> } finally {
>> httpclient.close();
>> }
>> }
>>
>> I have add a trust all ssl context and changed the class to be a
>JUnit
>> test case.
>>
>> Regards,
>>
>> Felix
>>
>>
>> ps: didn’t look deeply yet qt your findings
>>>
>>> Regards
>>>
>>> On Friday, December 14, 2018, Felix Schumacher <
>>> [email protected]> wrote:
>>>
>>> Am 14.12.18 um 13:17 schrieb Felix Schumacher:
>>>>
>>>> Am 13.12.18 um 23:28 schrieb Philippe Mouawad:
>>>>>
>>>>> Hello,
>>>>>>
>>>>>> We have a confirmed bug report :
>>>>>>
>>>>>> - https://bz.apache.org/bugzilla/show_bug.cgi?id=62852
>>>>>>
>>>>>> It appears that when using proxy for a request, the method
>>>>>> request.getRequestHeaders() called on the request returned
>>>>>> HttpContext#getAttribute(HttpCoreContext.HTTP_REQUEST) returns
>only
>>>>>> part of
>>>>>> the headers.
>>>>>>
>>>>>> While it returns them all if not using proxy.
>>>>>>
>>>>>> The headers (context) gets changed on line 189 in
>>>>> org.apache.http.impl.execchain.ProtcolExec.
>>>>>
>>>>> final CloseableHttpResponse response =
>>>>> this.requestExecutor.execute(route, request,
>>>>> context, execAware);
>>>>> try {
>>>>> // Run response protocol interceptors
>>>>> context.setAttribute(HttpCoreContext.HTTP_RESPONSE,
>>>>> response); <-- Here
>>>>> this.httpProcessor.process(response, context);
>>>>> return response;
>>>>> } catch (final RuntimeException ex) {
>>>>>
>>>>> Will debug further.
>>>>>
>>>>> Thread [Thread Group 1-1] (Suspended)
>>>> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$5(org.
>>>> apache.http.protocol.HttpRequestExecutor).preProcess(org.
>>>> apache.http.HttpRequest,
>>>> org.apache.http.protocol.HttpProcessor,
>org.apache.http.protocol.HttpC
>>>> ontext)
>>>> line: 163
>>>> org.apache.http.impl.execchain.MainClientExec.createTunnelTo
>>>> Target(org.apache.http.auth.AuthState,
>org.apache.http.HttpClientConn
>>>> ection,
>>>> org.apache.http.conn.routing.HttpRoute,
>org.apache.http.HttpRequest,
>>>> org.apache.http.client.protocol.HttpClientContext) line: 472
>>>> org.apache.http.impl.execchain.MainClientExec.establishRoute
>>>> (org.apache.http.auth.AuthState,
>org.apache.http.HttpClientConnection,
>>>> org.apache.http.conn.routing.HttpRoute,
>org.apache.http.HttpRequest,
>>>> org.apache.http.client.protocol.HttpClientContext) line: 411
>>>> org.apache.http.impl.execchain.MainClientExec.execute(org.
>>>> apache.http.conn.routing.HttpRoute, org.apache.http.client.methods
>>>> .HttpRequestWrapper,
>>>> org.apache.http.client.protocol.HttpClientContext,
>>>> org.apache.http.client.methods.HttpExecutionAware) line: 237
>>>> org.apache.http.impl.execchain.ProtocolExec.execute(org.
>>>> apache.http.conn.routing.HttpRoute, org.apache.http.client.methods
>>>> .HttpRequestWrapper,
>>>> org.apache.http.client.protocol.HttpClientContext,
>>>> org.apache.http.client.methods.HttpExecutionAware) line: 185
>>>> org.apache.http.impl.execchain.RetryExec.execute(org.apache.
>>>> http.conn.routing.HttpRoute,
>>>> org.apache.http.client.methods.HttpRequestWrapper,
>>>> org.apache.http.client.protocol.HttpClientContext,
>>>> org.apache.http.client.methods.HttpExecutionAware) line: 89
>>>> org.apache.http.impl.execchain.RedirectExec.execute(org.
>>>> apache.http.conn.routing.HttpRoute, org.apache.http.client.methods
>>>> .HttpRequestWrapper,
>>>> org.apache.http.client.protocol.HttpClientContext,
>>>> org.apache.http.client.methods.HttpExecutionAware) line: 110
>>>> org.apache.http.impl.client.InternalHttpClient.doExecute(org
>>>> .apache.http.HttpHost,
>>>> org.apache.http.HttpRequest, org.apache.http.protocol.HttpContext)
>line:
>>>> 185
>>>> org.apache.http.impl.client.InternalHttpClient(org.apache.ht
>>>> tp.impl.client.CloseableHttpClient).execute(org.apache.http.
>>>> client.methods.HttpUriRequest,
>>>> org.apache.http.protocol.HttpContext) line: 83
>>>> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeR
>>>> equest(org.apache.http.impl.client.CloseableHttpClient,
>>>> org.apache.http.client.methods.HttpRequestBase,
>>>> org.apache.http.protocol.HttpContext, java.net.URL) line: 840
>>>>
>org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(java.net.URL,
>>>> java.lang.String, boolean, int) line: 576
>>>> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sam
>>>> ple(java.net.URL,
>>>> java.lang.String, boolean, int) line: 67
>>>> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy(org
>>>> .apache.jmeter.protocol.http.sampler.HTTPSamplerBase).sample()
>line:
>>>> 1231
>>>> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy(org
>>>> .apache.jmeter.protocol.http.sampler.HTTPSamplerBase).sampl
>>>> e(org.apache.jmeter.samplers.Entry) line: 1220
>>>> org.apache.jmeter.threads.JMeterThread.doSampling(org.apache
>>>> .jmeter.threads.JMeterContext,
>>>> org.apache.jmeter.samplers.Sampler) line: 622
>>>> org.apache.jmeter.threads.JMeterThread.executeSamplePackage(
>>>> org.apache.jmeter.samplers.Sampler, org.apache.jmeter.control.Tran
>>>> sactionSampler,
>>>> org.apache.jmeter.threads.SamplePackage,
>org.apache.jmeter.threads.JMet
>>>> erContext)
>>>> line: 546
>>>> org.apache.jmeter.threads.JMeterThread.processSampler(org.
>>>> apache.jmeter.samplers.Sampler, org.apache.jmeter.samplers.Sampler,
>>>> org.apache.jmeter.threads.JMeterContext) line: 486
>>>> org.apache.jmeter.threads.JMeterThread.run() line: 253
>>>> java.lang.Thread.run() line: 748
>>>>
>>>> It seems that httpclient initiates the replacement of the "correct"
>>>> request. I have managed to revert that behaviour by adding a
>save/restore
>>>> functionality in our REQUEST_EXECUTOR that is implemented in
>>>> HTTPHC4Impl. I
>>>> implemented the method preProcess and saved the old request before
>>>> calling
>>>> the super method and restoring the request after the call.
>>>>
>>>> That looks quite ugly and I have no idea, if it is correct. But it
>works
>>>> for the simple test plan.
>>>>
>>>> Felix
>>>>
>>>>
>>>> Felix
>>>>>
>>>>>
>>>>> See:
>>>>>>
>>>>>> -
>>>>>> https://github.com/apache/jmeter/blob/trunk/src/protocol/
>>>>>>
>http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java#L580
>>>>>>
>>>>>>
>>>>>> I tried to reproduce behavior with HTTPClient only but couldn't ,
>so it
>>>>>> seems issue is somewhere in our code possibly due to interceptors
>or
>>>>>> request executor.
>>>>>>
>>>>>> /*
>>>>>> * ============================================================
>>>>>> ========
>>>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>>>> * or more contributor license agreements. See the NOTICE file
>>>>>> * distributed with this work for additional information
>>>>>> * regarding copyright ownership. The ASF licenses this file
>>>>>> * to you under the Apache License, Version 2.0 (the
>>>>>> * "License"); you may not use this file except in compliance
>>>>>> * with the License. You may obtain a copy of the License at
>>>>>> *
>>>>>> * http://www.apache.org/licenses/LICENSE-2.0
>>>>>> *
>>>>>> * Unless required by applicable law or agreed to in writing,
>>>>>> * software distributed under the License is distributed on an
>>>>>> * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
>>>>>> * KIND, either express or implied. See the License for the
>>>>>> * specific language governing permissions and limitations
>>>>>> * under the License.
>>>>>> * ============================================================
>>>>>> ========
>>>>>> *
>>>>>> * This software consists of voluntary contributions made by
>many
>>>>>> * individuals on behalf of the Apache Software Foundation. For
>more
>>>>>> * information on the Apache Software Foundation, please see
>>>>>> * <http://www.apache.org/>.
>>>>>> *
>>>>>> */
>>>>>>
>>>>>> package org.apache.jmeter.protocol.http.proxy;
>>>>>>
>>>>>> import java.util.Arrays;
>>>>>>
>>>>>> import org.apache.http.HttpHost;
>>>>>> import org.apache.http.HttpRequest;
>>>>>> import org.apache.http.client.config.RequestConfig;
>>>>>> import org.apache.http.client.methods.CloseableHttpResponse;
>>>>>> import org.apache.http.client.methods.HttpGet;
>>>>>> import org.apache.http.impl.client.CloseableHttpClient;
>>>>>> import org.apache.http.impl.client.HttpClients;
>>>>>> import org.apache.http.protocol.BasicHttpContext;
>>>>>> import org.apache.http.protocol.HttpContext;
>>>>>> import org.apache.http.protocol.HttpCoreContext;
>>>>>>
>>>>>> /**
>>>>>> * How to send a request via proxy.
>>>>>> *
>>>>>> * @since 4.0
>>>>>> */
>>>>>> public class TestProxy {
>>>>>>
>>>>>> public static void main(String[] args)throws Exception {
>>>>>> CloseableHttpClient httpclient =
>HttpClients.createDefault();
>>>>>> try {
>>>>>>
>>>>>> HttpHost target = new HttpHost("jmeter.apache.org",
>80,
>>>>>> "http");
>>>>>> HttpHost proxy = new HttpHost("localhost", 8888,
>"http");
>>>>>>
>>>>>> RequestConfig config = RequestConfig.custom()
>>>>>> .setProxy(proxy)
>>>>>> .build();
>>>>>> HttpGet request = new HttpGet("/");
>>>>>> request.addHeader("X-sleep", "5");
>>>>>> request.setConfig(config);
>>>>>>
>>>>>> System.out.println("Executing request " +
>>>>>> request.getRequestLine() + " to " + target + " via " + proxy);
>>>>>> HttpContext localContext = new BasicHttpContext();
>>>>>> CloseableHttpResponse response =
>>>>>> httpclient.execute(target,
>>>>>> request, localContext);
>>>>>> final HttpRequest httpRequestFromLocalContext =
>>>>>> (HttpRequest)
>>>>>> localContext.getAttribute(HttpCoreContext.HTTP_REQUEST);
>>>>>> try {
>>>>>>
>>>>>> System.out.println("----------------------------------------");
>>>>>> System.out.println(response.getStatusLine());
>>>>>> System.out.println(Arrays.asList(request.getAllHeaders()));
>>>>>>
>>>>>> System.out.println(Arrays.asList(httpRequestFromLocalContext
>>>>>> .getAllHeaders()));
>>>>>>
>>>>>> System.out.println(Arrays.asList(response.getAllHeaders()));
>>>>>>
>>>>>> //System.out.println(EntityUtils.toString(response.getEntity()));
>>>>>> } finally {
>>>>>> response.close();
>>>>>> }
>>>>>> } finally {
>>>>>> httpclient.close();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> OUTPUT:
>>>>>> ------------------------------------------------------------
>>>>>> ----------------------------------------
>>>>>> Executing request GET / HTTP/1.1 to http://jmeter.apache.org:80
>via
>>>>>> http://localhost:8888
>>>>>> ----------------------------------------
>>>>>> HTTP/1.1 200 OK
>>>>>> [X-sleep: 5]
>>>>>> [X-sleep: 5, Host: jmeter.apache.org:80, Proxy-Connection:
>Keep-Alive,
>>>>>> User-Agent: Apache-HttpClient/4.5.6 (Java/1.8.0_161),
>Accept-Encoding:
>>>>>> gzip,deflate]
>>>>>> [Date: Thu, 13 Dec 2018 22:23:23 GMT, Server: Apache/2.4.18
>(Ubuntu),
>>>>>> Last-Modified: Sun, 09 Sep 2018 15:25:54 GMT, ETag:
>>>>>> "3159-57571db50c67a-gzip", Accept-Ranges: bytes, Vary:
>Accept-Encoding,
>>>>>> Keep-Alive: timeout=30, max=100, Connection: Keep-Alive,
>Content-Type:
>>>>>> text/html, Content-Length: 12633]
>>>>>> ------------------------------------------------------------
>>>>>> ----------------------------------------
>>>>>>
>>>>>>
>>>>>> Regards
>>>>>> Philippe M.
>>>>>> <https://www.openstreetmap.org/#map=18/50.69454/3.16455>
>>>>>>
>>>>>>
>>>>>>
