Re: turn off debug logging from org.apache.http.wire

[larry mccay]

+1 to Kevin's point.
Ambari rewrites all configs on server restart.

On Sun, Apr 15, 2018 at 1:16 PM, Kevin Risden  wrote:

> Are you using Ambari or deploying Knox manually?
>
> If you using Ambari, then Ambari will force overwrite the log4j configs
> during a restart. You must update the log4j settings in Ambari. Another
> option if using Ambari is to find the debug setting and set it to false (I
> don't have a cluster in front of me so can't look up the setting).
>
> Kevin Risden
>
> On Sun, Apr 15, 2018 at 10:56 AM, Wei Han  wrote:
>
>> Interesting. Thanks Larry. I'll dig more on my side.
>>
>> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay  wrote:
>>
>>> No, I cannot reproduce it.
>>> If you are modifying the correct gateway-log4j.properties and restarting
>>> the server you should not see that.
>>>
>>> In fact, turning on DEBUG for wire via:
>>> log4j.logger.org.apache.http.wire=DEBUG
>>>
>>> Doesn't result in output in gateway.out for me but instead gateway.log
>>> and turning it on and off certainly works for me.
>>>
>>> You may have enabled TRACE logging if you are seeing body content -
>>> those settings are like the following:
>>>
>>> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
>>> #log4j.additivity.org.apache.knox.gateway.access=false
>>>
>>> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
>>> #log4j.additivity.org.apache.knox.gateway.http=false
>>> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>>>
>>> I suggest you back up to the gateway-log4j.properties from the original
>>> install and remove any other log4j config that you may have elsewhere.
>>>
>>> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>>>
 Hi Larry - Thanks a lot for getting back to me.

 Yes I made sure all DEBUG level is turned off in my 
 gateway-log4j.properties
 file, but that doesn't seem to be working. I also tried to explicitly
 set log4j.logger.httpclient.wire.header to WARN (as suggested in post
 ),
 but that also didn't help.

 Actually it's very easy to reproduce this(at least on my side). If you
 call knox with a webhdfs request (like 
 webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
 you should be able to see a bunch of below logs in gateway.out. In fact it
 outputs the actual content on the wire(security hole?)

   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
 http-outgoing-2 << "[0x0][0x0

 Let me know if you're able to repro this.

 Thanks.

 On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:

> Hi Wei -
>
> If you look at your gateway-log4j.properties file, you should see
> something like the following near the top:
>
> app.log.dir=${launcher.dir}/../logs
> app.log.file=${launcher.name}.log
> app.audit.file=${launcher.name}-audit.log
>
> log4j.rootLogger=ERROR, drfa
>
> log4j.logger.org.apache.knox.gateway=INFO
> #log4j.logger.org.apache.knox.gateway=DEBUG
>
> #log4j.logger.org.eclipse.jetty=DEBUG
> #log4j.logger.org.apache.shiro=DEBUG
> #log4j.logger.org.apache.http=DEBUG
> #log4j.logger.org.apache.http.client=DEBUG
> #log4j.logger.org.apache.http.headers=DEBUG
> #log4j.logger.org.apache.http.wire=DEBUG
>
> Note that all of the DEBUG settings are commented out.
> Also note that the rootLogger is set to ERROR and not DEBUG.
>
> Can you compare and share with us what yours are set to?
>
> thanks,
>
> --larry
>
> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:
>
>> Hi Knox experts -
>>
>> Has anyone successfully turn off the debug loggings
>> from org.apache.http.wire, like below?
>>
>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>> http-outgoing-2 << "[0x0][0x0
>>
>> I've tried almost everything in this stackoverflow post
>> ,
>> but my gateway.out is still filled with all the debug logs from 
>> httpclient.
>>
>> Any tip would be appreciated.
>>
>
>

>>>
>>
>

Re: turn off debug logging from org.apache.http.wire

[Kevin Risden]

Are you using Ambari or deploying Knox manually?

If you using Ambari, then Ambari will force overwrite the log4j configs
during a restart. You must update the log4j settings in Ambari. Another
option if using Ambari is to find the debug setting and set it to false (I
don't have a cluster in front of me so can't look up the setting).

Kevin Risden

On Sun, Apr 15, 2018 at 10:56 AM, Wei Han  wrote:

> Interesting. Thanks Larry. I'll dig more on my side.
>
> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay  wrote:
>
>> No, I cannot reproduce it.
>> If you are modifying the correct gateway-log4j.properties and restarting
>> the server you should not see that.
>>
>> In fact, turning on DEBUG for wire via:
>> log4j.logger.org.apache.http.wire=DEBUG
>>
>> Doesn't result in output in gateway.out for me but instead gateway.log
>> and turning it on and off certainly works for me.
>>
>> You may have enabled TRACE logging if you are seeing body content - those
>> settings are like the following:
>>
>> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
>> #log4j.additivity.org.apache.knox.gateway.access=false
>>
>> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
>> #log4j.additivity.org.apache.knox.gateway.http=false
>> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>>
>> I suggest you back up to the gateway-log4j.properties from the original
>> install and remove any other log4j config that you may have elsewhere.
>>
>> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>>
>>> Hi Larry - Thanks a lot for getting back to me.
>>>
>>> Yes I made sure all DEBUG level is turned off in my gateway-log4j.properties
>>> file, but that doesn't seem to be working. I also tried to explicitly
>>> set log4j.logger.httpclient.wire.header to WARN (as suggested in post
>>> ),
>>> but that also didn't help.
>>>
>>> Actually it's very easy to reproduce this(at least on my side). If you
>>> call knox with a webhdfs request (like 
>>> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
>>> you should be able to see a bunch of below logs in gateway.out. In fact it
>>> outputs the actual content on the wire(security hole?)
>>>
>>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>>> http-outgoing-2 << "[0x0][0x0
>>>
>>> Let me know if you're able to repro this.
>>>
>>> Thanks.
>>>
>>> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:
>>>
 Hi Wei -

 If you look at your gateway-log4j.properties file, you should see
 something like the following near the top:

 app.log.dir=${launcher.dir}/../logs
 app.log.file=${launcher.name}.log
 app.audit.file=${launcher.name}-audit.log

 log4j.rootLogger=ERROR, drfa

 log4j.logger.org.apache.knox.gateway=INFO
 #log4j.logger.org.apache.knox.gateway=DEBUG

 #log4j.logger.org.eclipse.jetty=DEBUG
 #log4j.logger.org.apache.shiro=DEBUG
 #log4j.logger.org.apache.http=DEBUG
 #log4j.logger.org.apache.http.client=DEBUG
 #log4j.logger.org.apache.http.headers=DEBUG
 #log4j.logger.org.apache.http.wire=DEBUG

 Note that all of the DEBUG settings are commented out.
 Also note that the rootLogger is set to ERROR and not DEBUG.

 Can you compare and share with us what yours are set to?

 thanks,

 --larry

 On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:

> Hi Knox experts -
>
> Has anyone successfully turn off the debug loggings
> from org.apache.http.wire, like below?
>
>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
> http-outgoing-2 << "[0x0][0x0
>
> I've tried almost everything in this stackoverflow post
> ,
> but my gateway.out is still filled with all the debug logs from 
> httpclient.
>
> Any tip would be appreciated.
>


>>>
>>
>

Re: turn off debug logging from org.apache.http.wire

[Wei Han]

Interesting. Thanks Larry. I'll dig more on my side.

On Sun, Apr 15, 2018 at 4:54 AM, larry mccay  wrote:

> No, I cannot reproduce it.
> If you are modifying the correct gateway-log4j.properties and restarting
> the server you should not see that.
>
> In fact, turning on DEBUG for wire via:
> log4j.logger.org.apache.http.wire=DEBUG
>
> Doesn't result in output in gateway.out for me but instead gateway.log and
> turning it on and off certainly works for me.
>
> You may have enabled TRACE logging if you are seeing body content - those
> settings are like the following:
>
> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
> #log4j.additivity.org.apache.knox.gateway.access=false
>
> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
> #log4j.additivity.org.apache.knox.gateway.http=false
> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>
> I suggest you back up to the gateway-log4j.properties from the original
> install and remove any other log4j config that you may have elsewhere.
>
> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>
>> Hi Larry - Thanks a lot for getting back to me.
>>
>> Yes I made sure all DEBUG level is turned off in my gateway-log4j.properties
>> file, but that doesn't seem to be working. I also tried to explicitly
>> set log4j.logger.httpclient.wire.header to WARN (as suggested in post
>> ),
>> but that also didn't help.
>>
>> Actually it's very easy to reproduce this(at least on my side). If you
>> call knox with a webhdfs request (like 
>> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
>> you should be able to see a bunch of below logs in gateway.out. In fact it
>> outputs the actual content on the wire(security hole?)
>>
>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>> http-outgoing-2 << "[0x0][0x0
>>
>> Let me know if you're able to repro this.
>>
>> Thanks.
>>
>> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:
>>
>>> Hi Wei -
>>>
>>> If you look at your gateway-log4j.properties file, you should see
>>> something like the following near the top:
>>>
>>> app.log.dir=${launcher.dir}/../logs
>>> app.log.file=${launcher.name}.log
>>> app.audit.file=${launcher.name}-audit.log
>>>
>>> log4j.rootLogger=ERROR, drfa
>>>
>>> log4j.logger.org.apache.knox.gateway=INFO
>>> #log4j.logger.org.apache.knox.gateway=DEBUG
>>>
>>> #log4j.logger.org.eclipse.jetty=DEBUG
>>> #log4j.logger.org.apache.shiro=DEBUG
>>> #log4j.logger.org.apache.http=DEBUG
>>> #log4j.logger.org.apache.http.client=DEBUG
>>> #log4j.logger.org.apache.http.headers=DEBUG
>>> #log4j.logger.org.apache.http.wire=DEBUG
>>>
>>> Note that all of the DEBUG settings are commented out.
>>> Also note that the rootLogger is set to ERROR and not DEBUG.
>>>
>>> Can you compare and share with us what yours are set to?
>>>
>>> thanks,
>>>
>>> --larry
>>>
>>> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:
>>>
 Hi Knox experts -

 Has anyone successfully turn off the debug loggings
 from org.apache.http.wire, like below?

   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
 http-outgoing-2 << "[0x0][0x0

 I've tried almost everything in this stackoverflow post
 ,
 but my gateway.out is still filled with all the debug logs from httpclient.

 Any tip would be appreciated.

>>>
>>>
>>
>

Re: turn off debug logging from org.apache.http.wire

[larry mccay]

No, I cannot reproduce it.
If you are modifying the correct gateway-log4j.properties and restarting
the server you should not see that.

In fact, turning on DEBUG for wire via:
log4j.logger.org.apache.http.wire=DEBUG

Doesn't result in output in gateway.out for me but instead gateway.log and
turning it on and off certainly works for me.

You may have enabled TRACE logging if you are seeing body content - those
settings are like the following:

#log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
#log4j.additivity.org.apache.knox.gateway.access=false

#log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
#log4j.additivity.org.apache.knox.gateway.http=false
##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
##log4j.logger.org.apache.knox.gateway.http.response.body=OFF

I suggest you back up to the gateway-log4j.properties from the original
install and remove any other log4j config that you may have elsewhere.

On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:

> Hi Larry - Thanks a lot for getting back to me.
>
> Yes I made sure all DEBUG level is turned off in my gateway-log4j.properties
> file, but that doesn't seem to be working. I also tried to explicitly set
> log4j.logger.httpclient.wire.header to WARN (as suggested in post
> ),
> but that also didn't help.
>
> Actually it's very easy to reproduce this(at least on my side). If you
> call knox with a webhdfs request (like 
> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
> you should be able to see a bunch of below logs in gateway.out. In fact it
> outputs the actual content on the wire(security hole?)
>
>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
> http-outgoing-2 << "[0x0][0x0
>
> Let me know if you're able to repro this.
>
> Thanks.
>
> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:
>
>> Hi Wei -
>>
>> If you look at your gateway-log4j.properties file, you should see
>> something like the following near the top:
>>
>> app.log.dir=${launcher.dir}/../logs
>> app.log.file=${launcher.name}.log
>> app.audit.file=${launcher.name}-audit.log
>>
>> log4j.rootLogger=ERROR, drfa
>>
>> log4j.logger.org.apache.knox.gateway=INFO
>> #log4j.logger.org.apache.knox.gateway=DEBUG
>>
>> #log4j.logger.org.eclipse.jetty=DEBUG
>> #log4j.logger.org.apache.shiro=DEBUG
>> #log4j.logger.org.apache.http=DEBUG
>> #log4j.logger.org.apache.http.client=DEBUG
>> #log4j.logger.org.apache.http.headers=DEBUG
>> #log4j.logger.org.apache.http.wire=DEBUG
>>
>> Note that all of the DEBUG settings are commented out.
>> Also note that the rootLogger is set to ERROR and not DEBUG.
>>
>> Can you compare and share with us what yours are set to?
>>
>> thanks,
>>
>> --larry
>>
>> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:
>>
>>> Hi Knox experts -
>>>
>>> Has anyone successfully turn off the debug loggings
>>> from org.apache.http.wire, like below?
>>>
>>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>>> http-outgoing-2 << "[0x0][0x0
>>>
>>> I've tried almost everything in this stackoverflow post
>>> ,
>>> but my gateway.out is still filled with all the debug logs from httpclient.
>>>
>>> Any tip would be appreciated.
>>>
>>
>>
>