Re: turn off debug logging from org.apache.http.wire

2018-09-06 Thread Guang Yang
Ok guys, I finally figure out the reason.

It's because we introduced logback from one of our internal packages, so
Knox was using logback instead of log4j underlying, and that's why no
matter how we modify gateway-log4j.properties, nothing changed. By simply
removing that package, Knox is using log4j now and all the DEBUG logs go
away.

Thanks for all the help guys, and this page

basically has all the possible reasons of failing to turn off DEBUG logs.

On Wed, Sep 5, 2018 at 11:34 AM, larry mccay  wrote:

> Change it back to this and see what happens.
> Another thing to check is that - if you are in a kerberized environment is
> whether any of the kerberos DEBUG settings are set via environment
> variables or krb5.conf, etc.
>
> Lastly, make sure that you are actually using the log4j.propoerties file
> that you think that you are.
> If you are using init.d to start the gateway make sure that you haven't
> changed where the gateway-log4j.properties is being found.
>
> # Licensed to the Apache Software Foundation (ASF) under one
> # or more contributor license agreements.  See the NOTICE file
> # distributed with this work for additional information
> # regarding copyright ownership.  The ASF licenses this file
> # to you under the Apache License, Version 2.0 (the
> # "License"); you may not use this file except in compliance
> # with the License.  You may obtain a copy of the License at
> #
> # http://www.apache.org/licenses/LICENSE-2.0
> #
> # Unless required by applicable law or agreed to in writing, software
> # distributed under the License is distributed on an "AS IS" BASIS,
> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> # See the License for the specific language governing permissions and
> # limitations under the License.
>
> app.log.dir=${launcher.dir}/../logs
> app.log.file=${launcher.name}.log
> app.audit.file=${launcher.name}-audit.log
>
> log4j.rootLogger=ERROR, drfa
>
> log4j.logger.org.apache.knox.gateway=INFO
> #log4j.logger.org.apache.knox.gateway=DEBUG
>
> #log4j.logger.org.eclipse.jetty=DEBUG
> #log4j.logger.org.apache.shiro=DEBUG
> #log4j.logger.org.apache.http=DEBUG
> #log4j.logger.org.apache.http.client=DEBUG
> #log4j.logger.org.apache.http.headers=DEBUG
> #log4j.logger.org.apache.http.wire=DEBUG
>
> log4j.appender.stdout=org.apache.log4j.ConsoleAppender
> log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
> log4j.appender.stdout.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
> %c{2}: %m%n
>
> log4j.appender.drfa=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.drfa.File=${app.log.dir}/${app.log.file}
> log4j.appender.drfa.DatePattern=.-MM-dd
> log4j.appender.drfa.layout=org.apache.log4j.PatternLayout
> log4j.appender.drfa.layout.ConversionPattern=%d{ISO8601} %-5p %c{2}
> (%F:%M(%L)) - %m%n
>
> log4j.logger.audit=INFO, auditfile
> log4j.appender.auditfile=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.auditfile.File=${app.log.dir}/${app.audit.file}
> log4j.appender.auditfile.Append = true
> log4j.appender.auditfile.DatePattern = '.'-MM-dd
> log4j.appender.auditfile.layout = org.apache.knox.gateway.audit.
> log4j.layout.AuditLayout
>
> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
> #log4j.additivity.org.apache.knox.gateway.access=false
>
> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
> #log4j.additivity.org.apache.knox.gateway.http=false
> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>
> #log4j.logger.org.apache.http.wire=DEBUG,httpclient
> #log4j.additivity.org.apache.http.wire=false
>
> #log4j.appender.httpaccess=org.apache.log4j.DailyRollingFileAppender
> #log4j.appender.httpaccess.File=${app.log.dir}/${launcher.name
> }-http-access.log
> #log4j.appender.httpaccess.DatePattern=.-MM-dd
> #log4j.appender.httpaccess.layout=org.apache.log4j.PatternLayout
> #log4j.appender.httpaccess.layout.ConversionPattern=%d{ISO8601}|%t|%m%n
>
> #log4j.appender.httpserver=org.apache.log4j.DailyRollingFileAppender
> #log4j.appender.httpserver.File=${app.log.dir}/${launcher.name
> }-http-server.log
> #log4j.appender.httpserver.DatePattern=.-MM-dd
> #log4j.appender.httpserver.layout=org.apache.log4j.PatternLayout
> #log4j.appender.httpserver.layout.ConversionPattern=%d{ISO8601}|%t|%m%n
>
> #log4j.appender.httpclient=org.apache.log4j.DailyRollingFileAppender
> #log4j.appender.httpclient.File=${app.log.dir}/${launcher.name
> }-http-client.log
> #log4j.appender.httpclient.DatePattern=.-MM-dd
> #log4j.appender.httpclient.layout=org.apache.log4j.PatternLayout
> #log4j.appender.httpclient.layout.ConversionPattern=%d{ISO8601}|%t|%m%n
>
> # Apache Shiro Related logging - KNOX-757
> 

Re: turn off debug logging from org.apache.http.wire

2018-09-05 Thread larry mccay
Change it back to this and see what happens.
Another thing to check is that - if you are in a kerberized environment is
whether any of the kerberos DEBUG settings are set via environment
variables or krb5.conf, etc.

Lastly, make sure that you are actually using the log4j.propoerties file
that you think that you are.
If you are using init.d to start the gateway make sure that you haven't
changed where the gateway-log4j.properties is being found.

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

app.log.dir=${launcher.dir}/../logs
app.log.file=${launcher.name}.log
app.audit.file=${launcher.name}-audit.log

log4j.rootLogger=ERROR, drfa

log4j.logger.org.apache.knox.gateway=INFO
#log4j.logger.org.apache.knox.gateway=DEBUG

#log4j.logger.org.eclipse.jetty=DEBUG
#log4j.logger.org.apache.shiro=DEBUG
#log4j.logger.org.apache.http=DEBUG
#log4j.logger.org.apache.http.client=DEBUG
#log4j.logger.org.apache.http.headers=DEBUG
#log4j.logger.org.apache.http.wire=DEBUG

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{2}: %m%n

log4j.appender.drfa=org.apache.log4j.DailyRollingFileAppender
log4j.appender.drfa.File=${app.log.dir}/${app.log.file}
log4j.appender.drfa.DatePattern=.-MM-dd
log4j.appender.drfa.layout=org.apache.log4j.PatternLayout
log4j.appender.drfa.layout.ConversionPattern=%d{ISO8601} %-5p %c{2}
(%F:%M(%L)) - %m%n

log4j.logger.audit=INFO, auditfile
log4j.appender.auditfile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.auditfile.File=${app.log.dir}/${app.audit.file}
log4j.appender.auditfile.Append = true
log4j.appender.auditfile.DatePattern = '.'-MM-dd
log4j.appender.auditfile.layout =
org.apache.knox.gateway.audit.log4j.layout.AuditLayout

#log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
#log4j.additivity.org.apache.knox.gateway.access=false

#log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
#log4j.additivity.org.apache.knox.gateway.http=false
##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
##log4j.logger.org.apache.knox.gateway.http.response.body=OFF

#log4j.logger.org.apache.http.wire=DEBUG,httpclient
#log4j.additivity.org.apache.http.wire=false

#log4j.appender.httpaccess=org.apache.log4j.DailyRollingFileAppender
#log4j.appender.httpaccess.File=${app.log.dir}/${launcher.name
}-http-access.log
#log4j.appender.httpaccess.DatePattern=.-MM-dd
#log4j.appender.httpaccess.layout=org.apache.log4j.PatternLayout
#log4j.appender.httpaccess.layout.ConversionPattern=%d{ISO8601}|%t|%m%n

#log4j.appender.httpserver=org.apache.log4j.DailyRollingFileAppender
#log4j.appender.httpserver.File=${app.log.dir}/${launcher.name
}-http-server.log
#log4j.appender.httpserver.DatePattern=.-MM-dd
#log4j.appender.httpserver.layout=org.apache.log4j.PatternLayout
#log4j.appender.httpserver.layout.ConversionPattern=%d{ISO8601}|%t|%m%n

#log4j.appender.httpclient=org.apache.log4j.DailyRollingFileAppender
#log4j.appender.httpclient.File=${app.log.dir}/${launcher.name
}-http-client.log
#log4j.appender.httpclient.DatePattern=.-MM-dd
#log4j.appender.httpclient.layout=org.apache.log4j.PatternLayout
#log4j.appender.httpclient.layout.ConversionPattern=%d{ISO8601}|%t|%m%n

# Apache Shiro Related logging - KNOX-757
#log4j.logger.org.springframework=DEBUG
#log4j.logger.net.sf.ehcache=DEBUG
#log4j.logger.org.apache.shiro.util.ThreadContext=DEBUG


On Wed, Sep 5, 2018 at 2:07 PM Guang Yang  wrote:

> Hi Larry,
>
> Here is my gateway-log4j.properties.
>
> *app.log.dir=/var/log/knox*
> *app.log.file=${launcher.name }.log*
> *app.audit.file=${launcher.name }-audit.log*
>
> *log4j.rootLogger=ERROR*
>
> *log4j.logger.org.apache.knox.gateway=ERROR*
>
> *log4j.logger.org.eclipse.jetty=ERROR*
>
> *log4j.logger.org.apache.shiro=ERROR*
> *log4j.logger.org.apache.http=ERROR*
> *log4j.logger.org.apache.http.client=ERROR*
> *log4j.logger.org.apache.http.headers=ERROR*
> *log4j.logger.org.apache.http.wire=ERROR*
>
> Even I changed it like this, I can still see lots of 

Re: turn off debug logging from org.apache.http.wire

2018-09-05 Thread Guang Yang
Hi Larry,

Here is my gateway-log4j.properties.

*app.log.dir=/var/log/knox*
*app.log.file=${launcher.name }.log*
*app.audit.file=${launcher.name }-audit.log*

*log4j.rootLogger=ERROR*

*log4j.logger.org.apache.knox.gateway=ERROR*

*log4j.logger.org.eclipse.jetty=ERROR*

*log4j.logger.org.apache.shiro=ERROR*
*log4j.logger.org.apache.http=ERROR*
*log4j.logger.org.apache.http.client=ERROR*
*log4j.logger.org.apache.http.headers=ERROR*
*log4j.logger.org.apache.http.wire=ERROR*

Even I changed it like this, I can still see lots of DEBUG log in
*gateway.out*. Seems it only affects* gateway.log*, not *gateway.out.*

On Wed, Sep 5, 2018 at 10:48 AM, larry mccay  wrote:

> Hi Guang -
>
> This certainly sounds frustrating.
> I have never had trouble turning it off.
> Can you share your gatway-log4j.properties file - just make sure there
> isn't anything sensitive in there?
>
> thanks,
>
> --larry
>
> On Wed, Sep 5, 2018 at 1:40 PM Guang Yang  wrote:
>
>> And we're not using Ambari. We just deploy manually.
>>
>> On Tue, Sep 4, 2018 at 11:02 PM, Guang Yang  wrote:
>>
>>> Hi guys,
>>>
>>> I'm working with Wei and we still don't figure it out. Let me clarify
>>> the question.
>>>
>>> Currently, we're seeing lots of DEBUG logs in file *gateway.out*, which
>>> is from here https://github.com/apache/knox/blob/master/gateway-
>>> release/home/bin/gateway.sh#L127. On the one hand, it prints the file
>>> content just like Wei talked about before, on the other hand we suspect it
>>> might be related to the performance issue when download a file through
>>> WEBHDFS. So we're trying to disable all these DEBUG logs. We tried simply
>>> removing this part *>>$APP_OUT_FILE*, although there is no such output
>>> file, but actually Knox still prints logs to console. So what we want to do
>>> is to disable all the DEBUG log thoroughly, so the service won't print logs
>>> to anywhere.
>>>
>>> We almost tried everything in *gateway-log4j.properties*, but it seems
>>> it only affects app.log.file=${launcher.name}.*log* instead of
>>> *gateway.out*. So, any idea guys?
>>>
>>> Thanks,
>>> Guang
>>>
>>> On Sun, Apr 15, 2018 at 11:08 AM, larry mccay  wrote:
>>>
 +1 to Kevin's point.
 Ambari rewrites all configs on server restart.

 On Sun, Apr 15, 2018 at 1:16 PM, Kevin Risden 
 wrote:

> Are you using Ambari or deploying Knox manually?
>
> If you using Ambari, then Ambari will force overwrite the log4j
> configs during a restart. You must update the log4j settings in Ambari.
> Another option if using Ambari is to find the debug setting and set it to
> false (I don't have a cluster in front of me so can't look up the 
> setting).
>
> Kevin Risden
>
> On Sun, Apr 15, 2018 at 10:56 AM, Wei Han  wrote:
>
>> Interesting. Thanks Larry. I'll dig more on my side.
>>
>> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay 
>> wrote:
>>
>>> No, I cannot reproduce it.
>>> If you are modifying the correct gateway-log4j.properties and
>>> restarting the server you should not see that.
>>>
>>> In fact, turning on DEBUG for wire via:
>>> log4j.logger.org.apache.http.wire=DEBUG
>>>
>>> Doesn't result in output in gateway.out for me but instead
>>> gateway.log and turning it on and off certainly works for me.
>>>
>>> You may have enabled TRACE logging if you are seeing body content -
>>> those settings are like the following:
>>>
>>> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
>>> #log4j.additivity.org.apache.knox.gateway.access=false
>>>
>>> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
>>> #log4j.additivity.org.apache.knox.gateway.http=false
>>> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>>>
>>> I suggest you back up to the gateway-log4j.properties from the
>>> original install and remove any other log4j config that you may have
>>> elsewhere.
>>>
>>> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>>>
 Hi Larry - Thanks a lot for getting back to me.

 Yes I made sure all DEBUG level is turned off in my 
 gateway-log4j.properties
 file, but that doesn't seem to be working. I also tried to
 explicitly set log4j.logger.httpclient.wire.header to WARN (as
 suggested in post
 ),
 but that also didn't help.

 Actually it's very easy to reproduce this(at least on my side). If
 you call knox with a webhdfs request (like 
 webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
 you should be able to see a 

Re: turn off debug logging from org.apache.http.wire

2018-09-05 Thread larry mccay
Hi Guang -

This certainly sounds frustrating.
I have never had trouble turning it off.
Can you share your gatway-log4j.properties file - just make sure there
isn't anything sensitive in there?

thanks,

--larry

On Wed, Sep 5, 2018 at 1:40 PM Guang Yang  wrote:

> And we're not using Ambari. We just deploy manually.
>
> On Tue, Sep 4, 2018 at 11:02 PM, Guang Yang  wrote:
>
>> Hi guys,
>>
>> I'm working with Wei and we still don't figure it out. Let me clarify the
>> question.
>>
>> Currently, we're seeing lots of DEBUG logs in file *gateway.out*, which
>> is from here
>> https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh#L127.
>> On the one hand, it prints the file content just like Wei talked about
>> before, on the other hand we suspect it might be related to the performance
>> issue when download a file through WEBHDFS. So we're trying to disable all
>> these DEBUG logs. We tried simply removing this part *>>$APP_OUT_FILE*,
>> although there is no such output file, but actually Knox still prints logs
>> to console. So what we want to do is to disable all the DEBUG log
>> thoroughly, so the service won't print logs to anywhere.
>>
>> We almost tried everything in *gateway-log4j.properties*, but it seems
>> it only affects app.log.file=${launcher.name}.*log* instead of
>> *gateway.out*. So, any idea guys?
>>
>> Thanks,
>> Guang
>>
>> On Sun, Apr 15, 2018 at 11:08 AM, larry mccay  wrote:
>>
>>> +1 to Kevin's point.
>>> Ambari rewrites all configs on server restart.
>>>
>>> On Sun, Apr 15, 2018 at 1:16 PM, Kevin Risden 
>>> wrote:
>>>
 Are you using Ambari or deploying Knox manually?

 If you using Ambari, then Ambari will force overwrite the log4j configs
 during a restart. You must update the log4j settings in Ambari. Another
 option if using Ambari is to find the debug setting and set it to false (I
 don't have a cluster in front of me so can't look up the setting).

 Kevin Risden

 On Sun, Apr 15, 2018 at 10:56 AM, Wei Han  wrote:

> Interesting. Thanks Larry. I'll dig more on my side.
>
> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay 
> wrote:
>
>> No, I cannot reproduce it.
>> If you are modifying the correct gateway-log4j.properties and
>> restarting the server you should not see that.
>>
>> In fact, turning on DEBUG for wire via:
>> log4j.logger.org.apache.http.wire=DEBUG
>>
>> Doesn't result in output in gateway.out for me but instead
>> gateway.log and turning it on and off certainly works for me.
>>
>> You may have enabled TRACE logging if you are seeing body content -
>> those settings are like the following:
>>
>> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
>> #log4j.additivity.org.apache.knox.gateway.access=false
>>
>> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
>> #log4j.additivity.org.apache.knox.gateway.http=false
>> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>>
>> I suggest you back up to the gateway-log4j.properties from the
>> original install and remove any other log4j config that you may have
>> elsewhere.
>>
>> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>>
>>> Hi Larry - Thanks a lot for getting back to me.
>>>
>>> Yes I made sure all DEBUG level is turned off in my 
>>> gateway-log4j.properties
>>> file, but that doesn't seem to be working. I also tried to
>>> explicitly set log4j.logger.httpclient.wire.header to WARN (as 
>>> suggested in
>>> post
>>> ),
>>> but that also didn't help.
>>>
>>> Actually it's very easy to reproduce this(at least on my side). If
>>> you call knox with a webhdfs request
>>> (like webhdfs/v1/tmp/weihan/small.txt?op=OPEN), you should be able to 
>>> see a
>>> bunch of below logs in gateway.out. In fact it outputs the actual 
>>> content
>>> on the wire(security hole?)
>>>
>>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>>> http-outgoing-2 << "[0x0][0x0
>>>
>>> Let me know if you're able to repro this.
>>>
>>> Thanks.
>>>
>>> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay 
>>> wrote:
>>>
 Hi Wei -

 If you look at your gateway-log4j.properties file, you should see
 something like the following near the top:

 app.log.dir=${launcher.dir}/../logs
 app.log.file=${launcher.name}.log
 app.audit.file=${launcher.name}-audit.log

 log4j.rootLogger=ERROR, drfa

 log4j.logger.org.apache.knox.gateway=INFO

Re: turn off debug logging from org.apache.http.wire

2018-09-05 Thread Guang Yang
And we're not using Ambari. We just deploy manually.

On Tue, Sep 4, 2018 at 11:02 PM, Guang Yang  wrote:

> Hi guys,
>
> I'm working with Wei and we still don't figure it out. Let me clarify the
> question.
>
> Currently, we're seeing lots of DEBUG logs in file *gateway.out*, which
> is from here https://github.com/apache/knox/blob/master/gateway-
> release/home/bin/gateway.sh#L127. On the one hand, it prints the file
> content just like Wei talked about before, on the other hand we suspect it
> might be related to the performance issue when download a file through
> WEBHDFS. So we're trying to disable all these DEBUG logs. We tried simply
> removing this part *>>$APP_OUT_FILE*, although there is no such output
> file, but actually Knox still prints logs to console. So what we want to do
> is to disable all the DEBUG log thoroughly, so the service won't print logs
> to anywhere.
>
> We almost tried everything in *gateway-log4j.properties*, but it seems it
> only affects app.log.file=${launcher.name}.*log* instead of *gateway.out*.
> So, any idea guys?
>
> Thanks,
> Guang
>
> On Sun, Apr 15, 2018 at 11:08 AM, larry mccay  wrote:
>
>> +1 to Kevin's point.
>> Ambari rewrites all configs on server restart.
>>
>> On Sun, Apr 15, 2018 at 1:16 PM, Kevin Risden  wrote:
>>
>>> Are you using Ambari or deploying Knox manually?
>>>
>>> If you using Ambari, then Ambari will force overwrite the log4j configs
>>> during a restart. You must update the log4j settings in Ambari. Another
>>> option if using Ambari is to find the debug setting and set it to false (I
>>> don't have a cluster in front of me so can't look up the setting).
>>>
>>> Kevin Risden
>>>
>>> On Sun, Apr 15, 2018 at 10:56 AM, Wei Han  wrote:
>>>
 Interesting. Thanks Larry. I'll dig more on my side.

 On Sun, Apr 15, 2018 at 4:54 AM, larry mccay  wrote:

> No, I cannot reproduce it.
> If you are modifying the correct gateway-log4j.properties and
> restarting the server you should not see that.
>
> In fact, turning on DEBUG for wire via:
> log4j.logger.org.apache.http.wire=DEBUG
>
> Doesn't result in output in gateway.out for me but instead gateway.log
> and turning it on and off certainly works for me.
>
> You may have enabled TRACE logging if you are seeing body content -
> those settings are like the following:
>
> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
> #log4j.additivity.org.apache.knox.gateway.access=false
>
> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
> #log4j.additivity.org.apache.knox.gateway.http=false
> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>
> I suggest you back up to the gateway-log4j.properties from the
> original install and remove any other log4j config that you may have
> elsewhere.
>
> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>
>> Hi Larry - Thanks a lot for getting back to me.
>>
>> Yes I made sure all DEBUG level is turned off in my 
>> gateway-log4j.properties
>> file, but that doesn't seem to be working. I also tried to
>> explicitly set log4j.logger.httpclient.wire.header to WARN (as
>> suggested in post
>> ),
>> but that also didn't help.
>>
>> Actually it's very easy to reproduce this(at least on my side). If
>> you call knox with a webhdfs request (like 
>> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
>> you should be able to see a bunch of below logs in gateway.out. In fact 
>> it
>> outputs the actual content on the wire(security hole?)
>>
>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>> http-outgoing-2 << "[0x0][0x0
>>
>> Let me know if you're able to repro this.
>>
>> Thanks.
>>
>> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay 
>> wrote:
>>
>>> Hi Wei -
>>>
>>> If you look at your gateway-log4j.properties file, you should see
>>> something like the following near the top:
>>>
>>> app.log.dir=${launcher.dir}/../logs
>>> app.log.file=${launcher.name}.log
>>> app.audit.file=${launcher.name}-audit.log
>>>
>>> log4j.rootLogger=ERROR, drfa
>>>
>>> log4j.logger.org.apache.knox.gateway=INFO
>>> #log4j.logger.org.apache.knox.gateway=DEBUG
>>>
>>> #log4j.logger.org.eclipse.jetty=DEBUG
>>> #log4j.logger.org.apache.shiro=DEBUG
>>> #log4j.logger.org.apache.http=DEBUG
>>> #log4j.logger.org.apache.http.client=DEBUG
>>> #log4j.logger.org.apache.http.headers=DEBUG
>>> #log4j.logger.org.apache.http.wire=DEBUG
>>>
>>> Note that all of the DEBUG settings are commented 

Re: turn off debug logging from org.apache.http.wire

2018-09-05 Thread Guang Yang
Hi guys,

I'm working with Wei and we still don't figure it out. Let me clarify the
question.

Currently, we're seeing lots of DEBUG logs in file *gateway.out*, which is
from here
https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh#L127.
On the one hand, it prints the file content just like Wei talked about
before, on the other hand we suspect it might be related to the performance
issue when download a file through WEBHDFS. So we're trying to disable all
these DEBUG logs. We tried simply removing this part *>>$APP_OUT_FILE*,
although there is no such output file, but actually Knox still prints logs
to console. So what we want to do is to disable all the DEBUG log
thoroughly, so the service won't print logs to anywhere.

We almost tried everything in *gateway-log4j.properties*, but it seems it
only affects app.log.file=${launcher.name}.*log* instead of *gateway.out*.
So, any idea guys?

Thanks,
Guang

On Sun, Apr 15, 2018 at 11:08 AM, larry mccay  wrote:

> +1 to Kevin's point.
> Ambari rewrites all configs on server restart.
>
> On Sun, Apr 15, 2018 at 1:16 PM, Kevin Risden  wrote:
>
>> Are you using Ambari or deploying Knox manually?
>>
>> If you using Ambari, then Ambari will force overwrite the log4j configs
>> during a restart. You must update the log4j settings in Ambari. Another
>> option if using Ambari is to find the debug setting and set it to false (I
>> don't have a cluster in front of me so can't look up the setting).
>>
>> Kevin Risden
>>
>> On Sun, Apr 15, 2018 at 10:56 AM, Wei Han  wrote:
>>
>>> Interesting. Thanks Larry. I'll dig more on my side.
>>>
>>> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay  wrote:
>>>
 No, I cannot reproduce it.
 If you are modifying the correct gateway-log4j.properties and
 restarting the server you should not see that.

 In fact, turning on DEBUG for wire via:
 log4j.logger.org.apache.http.wire=DEBUG

 Doesn't result in output in gateway.out for me but instead gateway.log
 and turning it on and off certainly works for me.

 You may have enabled TRACE logging if you are seeing body content -
 those settings are like the following:

 #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
 #log4j.additivity.org.apache.knox.gateway.access=false

 #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
 #log4j.additivity.org.apache.knox.gateway.http=false
 ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
 ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
 ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
 ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF

 I suggest you back up to the gateway-log4j.properties from the original
 install and remove any other log4j config that you may have elsewhere.

 On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:

> Hi Larry - Thanks a lot for getting back to me.
>
> Yes I made sure all DEBUG level is turned off in my 
> gateway-log4j.properties
> file, but that doesn't seem to be working. I also tried to explicitly
> set log4j.logger.httpclient.wire.header to WARN (as suggested in post
> ),
> but that also didn't help.
>
> Actually it's very easy to reproduce this(at least on my side). If you
> call knox with a webhdfs request (like 
> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
> you should be able to see a bunch of below logs in gateway.out. In fact it
> outputs the actual content on the wire(security hole?)
>
>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
> http-outgoing-2 << "[0x0][0x0
>
> Let me know if you're able to repro this.
>
> Thanks.
>
> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay 
> wrote:
>
>> Hi Wei -
>>
>> If you look at your gateway-log4j.properties file, you should see
>> something like the following near the top:
>>
>> app.log.dir=${launcher.dir}/../logs
>> app.log.file=${launcher.name}.log
>> app.audit.file=${launcher.name}-audit.log
>>
>> log4j.rootLogger=ERROR, drfa
>>
>> log4j.logger.org.apache.knox.gateway=INFO
>> #log4j.logger.org.apache.knox.gateway=DEBUG
>>
>> #log4j.logger.org.eclipse.jetty=DEBUG
>> #log4j.logger.org.apache.shiro=DEBUG
>> #log4j.logger.org.apache.http=DEBUG
>> #log4j.logger.org.apache.http.client=DEBUG
>> #log4j.logger.org.apache.http.headers=DEBUG
>> #log4j.logger.org.apache.http.wire=DEBUG
>>
>> Note that all of the DEBUG settings are commented out.
>> Also note that the rootLogger is set to ERROR and not DEBUG.
>>
>> Can you compare and share with us what yours are set to?
>>
>> thanks,
>>
>> --larry
>>
>> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:
>>
>>> Hi 

Re: turn off debug logging from org.apache.http.wire

2018-04-15 Thread larry mccay
+1 to Kevin's point.
Ambari rewrites all configs on server restart.

On Sun, Apr 15, 2018 at 1:16 PM, Kevin Risden  wrote:

> Are you using Ambari or deploying Knox manually?
>
> If you using Ambari, then Ambari will force overwrite the log4j configs
> during a restart. You must update the log4j settings in Ambari. Another
> option if using Ambari is to find the debug setting and set it to false (I
> don't have a cluster in front of me so can't look up the setting).
>
> Kevin Risden
>
> On Sun, Apr 15, 2018 at 10:56 AM, Wei Han  wrote:
>
>> Interesting. Thanks Larry. I'll dig more on my side.
>>
>> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay  wrote:
>>
>>> No, I cannot reproduce it.
>>> If you are modifying the correct gateway-log4j.properties and restarting
>>> the server you should not see that.
>>>
>>> In fact, turning on DEBUG for wire via:
>>> log4j.logger.org.apache.http.wire=DEBUG
>>>
>>> Doesn't result in output in gateway.out for me but instead gateway.log
>>> and turning it on and off certainly works for me.
>>>
>>> You may have enabled TRACE logging if you are seeing body content -
>>> those settings are like the following:
>>>
>>> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
>>> #log4j.additivity.org.apache.knox.gateway.access=false
>>>
>>> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
>>> #log4j.additivity.org.apache.knox.gateway.http=false
>>> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
>>> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>>>
>>> I suggest you back up to the gateway-log4j.properties from the original
>>> install and remove any other log4j config that you may have elsewhere.
>>>
>>> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>>>
 Hi Larry - Thanks a lot for getting back to me.

 Yes I made sure all DEBUG level is turned off in my 
 gateway-log4j.properties
 file, but that doesn't seem to be working. I also tried to explicitly
 set log4j.logger.httpclient.wire.header to WARN (as suggested in post
 ),
 but that also didn't help.

 Actually it's very easy to reproduce this(at least on my side). If you
 call knox with a webhdfs request (like 
 webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
 you should be able to see a bunch of below logs in gateway.out. In fact it
 outputs the actual content on the wire(security hole?)

   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
 http-outgoing-2 << "[0x0][0x0

 Let me know if you're able to repro this.

 Thanks.

 On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:

> Hi Wei -
>
> If you look at your gateway-log4j.properties file, you should see
> something like the following near the top:
>
> app.log.dir=${launcher.dir}/../logs
> app.log.file=${launcher.name}.log
> app.audit.file=${launcher.name}-audit.log
>
> log4j.rootLogger=ERROR, drfa
>
> log4j.logger.org.apache.knox.gateway=INFO
> #log4j.logger.org.apache.knox.gateway=DEBUG
>
> #log4j.logger.org.eclipse.jetty=DEBUG
> #log4j.logger.org.apache.shiro=DEBUG
> #log4j.logger.org.apache.http=DEBUG
> #log4j.logger.org.apache.http.client=DEBUG
> #log4j.logger.org.apache.http.headers=DEBUG
> #log4j.logger.org.apache.http.wire=DEBUG
>
> Note that all of the DEBUG settings are commented out.
> Also note that the rootLogger is set to ERROR and not DEBUG.
>
> Can you compare and share with us what yours are set to?
>
> thanks,
>
> --larry
>
> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:
>
>> Hi Knox experts -
>>
>> Has anyone successfully turn off the debug loggings
>> from org.apache.http.wire, like below?
>>
>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>> http-outgoing-2 << "[0x0][0x0
>>
>> I've tried almost everything in this stackoverflow post
>> ,
>> but my gateway.out is still filled with all the debug logs from 
>> httpclient.
>>
>> Any tip would be appreciated.
>>
>
>

>>>
>>
>


Re: turn off debug logging from org.apache.http.wire

2018-04-15 Thread Kevin Risden
Are you using Ambari or deploying Knox manually?

If you using Ambari, then Ambari will force overwrite the log4j configs
during a restart. You must update the log4j settings in Ambari. Another
option if using Ambari is to find the debug setting and set it to false (I
don't have a cluster in front of me so can't look up the setting).

Kevin Risden

On Sun, Apr 15, 2018 at 10:56 AM, Wei Han  wrote:

> Interesting. Thanks Larry. I'll dig more on my side.
>
> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay  wrote:
>
>> No, I cannot reproduce it.
>> If you are modifying the correct gateway-log4j.properties and restarting
>> the server you should not see that.
>>
>> In fact, turning on DEBUG for wire via:
>> log4j.logger.org.apache.http.wire=DEBUG
>>
>> Doesn't result in output in gateway.out for me but instead gateway.log
>> and turning it on and off certainly works for me.
>>
>> You may have enabled TRACE logging if you are seeing body content - those
>> settings are like the following:
>>
>> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
>> #log4j.additivity.org.apache.knox.gateway.access=false
>>
>> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
>> #log4j.additivity.org.apache.knox.gateway.http=false
>> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
>> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>>
>> I suggest you back up to the gateway-log4j.properties from the original
>> install and remove any other log4j config that you may have elsewhere.
>>
>> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>>
>>> Hi Larry - Thanks a lot for getting back to me.
>>>
>>> Yes I made sure all DEBUG level is turned off in my gateway-log4j.properties
>>> file, but that doesn't seem to be working. I also tried to explicitly
>>> set log4j.logger.httpclient.wire.header to WARN (as suggested in post
>>> ),
>>> but that also didn't help.
>>>
>>> Actually it's very easy to reproduce this(at least on my side). If you
>>> call knox with a webhdfs request (like 
>>> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
>>> you should be able to see a bunch of below logs in gateway.out. In fact it
>>> outputs the actual content on the wire(security hole?)
>>>
>>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>>> http-outgoing-2 << "[0x0][0x0
>>>
>>> Let me know if you're able to repro this.
>>>
>>> Thanks.
>>>
>>> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:
>>>
 Hi Wei -

 If you look at your gateway-log4j.properties file, you should see
 something like the following near the top:

 app.log.dir=${launcher.dir}/../logs
 app.log.file=${launcher.name}.log
 app.audit.file=${launcher.name}-audit.log

 log4j.rootLogger=ERROR, drfa

 log4j.logger.org.apache.knox.gateway=INFO
 #log4j.logger.org.apache.knox.gateway=DEBUG

 #log4j.logger.org.eclipse.jetty=DEBUG
 #log4j.logger.org.apache.shiro=DEBUG
 #log4j.logger.org.apache.http=DEBUG
 #log4j.logger.org.apache.http.client=DEBUG
 #log4j.logger.org.apache.http.headers=DEBUG
 #log4j.logger.org.apache.http.wire=DEBUG

 Note that all of the DEBUG settings are commented out.
 Also note that the rootLogger is set to ERROR and not DEBUG.

 Can you compare and share with us what yours are set to?

 thanks,

 --larry

 On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:

> Hi Knox experts -
>
> Has anyone successfully turn off the debug loggings
> from org.apache.http.wire, like below?
>
>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
> http-outgoing-2 << "[0x0][0x0
>
> I've tried almost everything in this stackoverflow post
> ,
> but my gateway.out is still filled with all the debug logs from 
> httpclient.
>
> Any tip would be appreciated.
>


>>>
>>
>


Re: turn off debug logging from org.apache.http.wire

2018-04-15 Thread Wei Han
Interesting. Thanks Larry. I'll dig more on my side.

On Sun, Apr 15, 2018 at 4:54 AM, larry mccay  wrote:

> No, I cannot reproduce it.
> If you are modifying the correct gateway-log4j.properties and restarting
> the server you should not see that.
>
> In fact, turning on DEBUG for wire via:
> log4j.logger.org.apache.http.wire=DEBUG
>
> Doesn't result in output in gateway.out for me but instead gateway.log and
> turning it on and off certainly works for me.
>
> You may have enabled TRACE logging if you are seeing body content - those
> settings are like the following:
>
> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
> #log4j.additivity.org.apache.knox.gateway.access=false
>
> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
> #log4j.additivity.org.apache.knox.gateway.http=false
> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>
> I suggest you back up to the gateway-log4j.properties from the original
> install and remove any other log4j config that you may have elsewhere.
>
> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:
>
>> Hi Larry - Thanks a lot for getting back to me.
>>
>> Yes I made sure all DEBUG level is turned off in my gateway-log4j.properties
>> file, but that doesn't seem to be working. I also tried to explicitly
>> set log4j.logger.httpclient.wire.header to WARN (as suggested in post
>> ),
>> but that also didn't help.
>>
>> Actually it's very easy to reproduce this(at least on my side). If you
>> call knox with a webhdfs request (like 
>> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
>> you should be able to see a bunch of below logs in gateway.out. In fact it
>> outputs the actual content on the wire(security hole?)
>>
>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>> http-outgoing-2 << "[0x0][0x0
>>
>> Let me know if you're able to repro this.
>>
>> Thanks.
>>
>> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:
>>
>>> Hi Wei -
>>>
>>> If you look at your gateway-log4j.properties file, you should see
>>> something like the following near the top:
>>>
>>> app.log.dir=${launcher.dir}/../logs
>>> app.log.file=${launcher.name}.log
>>> app.audit.file=${launcher.name}-audit.log
>>>
>>> log4j.rootLogger=ERROR, drfa
>>>
>>> log4j.logger.org.apache.knox.gateway=INFO
>>> #log4j.logger.org.apache.knox.gateway=DEBUG
>>>
>>> #log4j.logger.org.eclipse.jetty=DEBUG
>>> #log4j.logger.org.apache.shiro=DEBUG
>>> #log4j.logger.org.apache.http=DEBUG
>>> #log4j.logger.org.apache.http.client=DEBUG
>>> #log4j.logger.org.apache.http.headers=DEBUG
>>> #log4j.logger.org.apache.http.wire=DEBUG
>>>
>>> Note that all of the DEBUG settings are commented out.
>>> Also note that the rootLogger is set to ERROR and not DEBUG.
>>>
>>> Can you compare and share with us what yours are set to?
>>>
>>> thanks,
>>>
>>> --larry
>>>
>>> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:
>>>
 Hi Knox experts -

 Has anyone successfully turn off the debug loggings
 from org.apache.http.wire, like below?

   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
 http-outgoing-2 << "[0x0][0x0

 I've tried almost everything in this stackoverflow post
 ,
 but my gateway.out is still filled with all the debug logs from httpclient.

 Any tip would be appreciated.

>>>
>>>
>>
>


Re: turn off debug logging from org.apache.http.wire

2018-04-15 Thread larry mccay
No, I cannot reproduce it.
If you are modifying the correct gateway-log4j.properties and restarting
the server you should not see that.

In fact, turning on DEBUG for wire via:
log4j.logger.org.apache.http.wire=DEBUG

Doesn't result in output in gateway.out for me but instead gateway.log and
turning it on and off certainly works for me.

You may have enabled TRACE logging if you are seeing body content - those
settings are like the following:

#log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
#log4j.additivity.org.apache.knox.gateway.access=false

#log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
#log4j.additivity.org.apache.knox.gateway.http=false
##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
##log4j.logger.org.apache.knox.gateway.http.response.body=OFF

I suggest you back up to the gateway-log4j.properties from the original
install and remove any other log4j config that you may have elsewhere.

On Sun, Apr 15, 2018 at 1:58 AM, Wei Han  wrote:

> Hi Larry - Thanks a lot for getting back to me.
>
> Yes I made sure all DEBUG level is turned off in my gateway-log4j.properties
> file, but that doesn't seem to be working. I also tried to explicitly set
> log4j.logger.httpclient.wire.header to WARN (as suggested in post
> ),
> but that also didn't help.
>
> Actually it's very easy to reproduce this(at least on my side). If you
> call knox with a webhdfs request (like 
> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
> you should be able to see a bunch of below logs in gateway.out. In fact it
> outputs the actual content on the wire(security hole?)
>
>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
> http-outgoing-2 << "[0x0][0x0
>
> Let me know if you're able to repro this.
>
> Thanks.
>
> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:
>
>> Hi Wei -
>>
>> If you look at your gateway-log4j.properties file, you should see
>> something like the following near the top:
>>
>> app.log.dir=${launcher.dir}/../logs
>> app.log.file=${launcher.name}.log
>> app.audit.file=${launcher.name}-audit.log
>>
>> log4j.rootLogger=ERROR, drfa
>>
>> log4j.logger.org.apache.knox.gateway=INFO
>> #log4j.logger.org.apache.knox.gateway=DEBUG
>>
>> #log4j.logger.org.eclipse.jetty=DEBUG
>> #log4j.logger.org.apache.shiro=DEBUG
>> #log4j.logger.org.apache.http=DEBUG
>> #log4j.logger.org.apache.http.client=DEBUG
>> #log4j.logger.org.apache.http.headers=DEBUG
>> #log4j.logger.org.apache.http.wire=DEBUG
>>
>> Note that all of the DEBUG settings are commented out.
>> Also note that the rootLogger is set to ERROR and not DEBUG.
>>
>> Can you compare and share with us what yours are set to?
>>
>> thanks,
>>
>> --larry
>>
>> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:
>>
>>> Hi Knox experts -
>>>
>>> Has anyone successfully turn off the debug loggings
>>> from org.apache.http.wire, like below?
>>>
>>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>>> http-outgoing-2 << "[0x0][0x0
>>>
>>> I've tried almost everything in this stackoverflow post
>>> ,
>>> but my gateway.out is still filled with all the debug logs from httpclient.
>>>
>>> Any tip would be appreciated.
>>>
>>
>>
>


Re: turn off debug logging from org.apache.http.wire

2018-04-14 Thread Wei Han
Hi Larry - Thanks a lot for getting back to me.

Yes I made sure all DEBUG level is turned off in my gateway-log4j.properties
file, but that doesn't seem to be working. I also tried to explicitly set
log4j.logger.httpclient.wire.header to WARN (as suggested in post
),
but that also didn't help.

Actually it's very easy to reproduce this(at least on my side). If you call
knox with a webhdfs request (like webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
you should be able to see a bunch of below logs in gateway.out. In fact it
outputs the actual content on the wire(security hole?)

  06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
http-outgoing-2 << "[0x0][0x0

Let me know if you're able to repro this.

Thanks.

On Sat, Apr 14, 2018 at 7:11 AM, larry mccay  wrote:

> Hi Wei -
>
> If you look at your gateway-log4j.properties file, you should see
> something like the following near the top:
>
> app.log.dir=${launcher.dir}/../logs
> app.log.file=${launcher.name}.log
> app.audit.file=${launcher.name}-audit.log
>
> log4j.rootLogger=ERROR, drfa
>
> log4j.logger.org.apache.knox.gateway=INFO
> #log4j.logger.org.apache.knox.gateway=DEBUG
>
> #log4j.logger.org.eclipse.jetty=DEBUG
> #log4j.logger.org.apache.shiro=DEBUG
> #log4j.logger.org.apache.http=DEBUG
> #log4j.logger.org.apache.http.client=DEBUG
> #log4j.logger.org.apache.http.headers=DEBUG
> #log4j.logger.org.apache.http.wire=DEBUG
>
> Note that all of the DEBUG settings are commented out.
> Also note that the rootLogger is set to ERROR and not DEBUG.
>
> Can you compare and share with us what yours are set to?
>
> thanks,
>
> --larry
>
> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:
>
>> Hi Knox experts -
>>
>> Has anyone successfully turn off the debug loggings
>> from org.apache.http.wire, like below?
>>
>>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>> http-outgoing-2 << "[0x0][0x0
>>
>> I've tried almost everything in this stackoverflow post
>> ,
>> but my gateway.out is still filled with all the debug logs from httpclient.
>>
>> Any tip would be appreciated.
>>
>
>


Re: turn off debug logging from org.apache.http.wire

2018-04-14 Thread larry mccay
Hi Wei -

If you look at your gateway-log4j.properties file, you should see something
like the following near the top:

app.log.dir=${launcher.dir}/../logs
app.log.file=${launcher.name}.log
app.audit.file=${launcher.name}-audit.log

log4j.rootLogger=ERROR, drfa

log4j.logger.org.apache.knox.gateway=INFO
#log4j.logger.org.apache.knox.gateway=DEBUG

#log4j.logger.org.eclipse.jetty=DEBUG
#log4j.logger.org.apache.shiro=DEBUG
#log4j.logger.org.apache.http=DEBUG
#log4j.logger.org.apache.http.client=DEBUG
#log4j.logger.org.apache.http.headers=DEBUG
#log4j.logger.org.apache.http.wire=DEBUG

Note that all of the DEBUG settings are commented out.
Also note that the rootLogger is set to ERROR and not DEBUG.

Can you compare and share with us what yours are set to?

thanks,

--larry

On Sat, Apr 14, 2018 at 2:56 AM, Wei Han  wrote:

> Hi Knox experts -
>
> Has anyone successfully turn off the debug loggings
> from org.apache.http.wire, like below?
>
>   06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
> http-outgoing-2 << "[0x0][0x0
>
> I've tried almost everything in this stackoverflow post
> ,
> but my gateway.out is still filled with all the debug logs from httpclient.
>
> Any tip would be appreciated.
>