Hi Larry,
Here is my gateway-log4j.properties.
*app.log.dir=/var/log/knox*
*app.log.file=${launcher.name <http://launcher.name>}.log*
*app.audit.file=${launcher.name <http://launcher.name>}-audit.log*
*log4j.rootLogger=ERROR*
*log4j.logger.org.apache.knox.gateway=ERROR*
*log4j.logger.org.eclipse.jetty=ERROR*
*log4j.logger.org.apache.shiro=ERROR*
*log4j.logger.org.apache.http=ERROR*
*log4j.logger.org.apache.http.client=ERROR*
*log4j.logger.org.apache.http.headers=ERROR*
*log4j.logger.org.apache.http.wire=ERROR*
Even I changed it like this, I can still see lots of DEBUG log in
*gateway.out*. Seems it only affects* gateway.log*, not *gateway.out.*
On Wed, Sep 5, 2018 at 10:48 AM, larry mccay <lmc...@apache.org> wrote:
> Hi Guang -
>
> This certainly sounds frustrating.
> I have never had trouble turning it off.
> Can you share your gatway-log4j.properties file - just make sure there
> isn't anything sensitive in there?
>
> thanks,
>
> --larry
>
> On Wed, Sep 5, 2018 at 1:40 PM Guang Yang <k...@uber.com> wrote:
>
>> And we're not using Ambari. We just deploy manually.
>>
>> On Tue, Sep 4, 2018 at 11:02 PM, Guang Yang <k...@uber.com> wrote:
>>
>>> Hi guys,
>>>
>>> I'm working with Wei and we still don't figure it out. Let me clarify
>>> the question.
>>>
>>> Currently, we're seeing lots of DEBUG logs in file *gateway.out*, which
>>> is from here https://github.com/apache/knox/blob/master/gateway-
>>> release/home/bin/gateway.sh#L127. On the one hand, it prints the file
>>> content just like Wei talked about before, on the other hand we suspect it
>>> might be related to the performance issue when download a file through
>>> WEBHDFS. So we're trying to disable all these DEBUG logs. We tried simply
>>> removing this part *>>$APP_OUT_FILE*, although there is no such output
>>> file, but actually Knox still prints logs to console. So what we want to do
>>> is to disable all the DEBUG log thoroughly, so the service won't print logs
>>> to anywhere.
>>>
>>> We almost tried everything in *gateway-log4j.properties*, but it seems
>>> it only affects app.log.file=${launcher.name}.*log* instead of
>>> *gateway.out*. So, any idea guys?
>>>
>>> Thanks,
>>> Guang
>>>
>>> On Sun, Apr 15, 2018 at 11:08 AM, larry mccay <lmc...@apache.org> wrote:
>>>
>>>> +1 to Kevin's point.
>>>> Ambari rewrites all configs on server restart.
>>>>
>>>> On Sun, Apr 15, 2018 at 1:16 PM, Kevin Risden <kris...@apache.org>
>>>> wrote:
>>>>
>>>>> Are you using Ambari or deploying Knox manually?
>>>>>
>>>>> If you using Ambari, then Ambari will force overwrite the log4j
>>>>> configs during a restart. You must update the log4j settings in Ambari.
>>>>> Another option if using Ambari is to find the debug setting and set it to
>>>>> false (I don't have a cluster in front of me so can't look up the
>>>>> setting).
>>>>>
>>>>> Kevin Risden
>>>>>
>>>>> On Sun, Apr 15, 2018 at 10:56 AM, Wei Han <wei...@uber.com> wrote:
>>>>>
>>>>>> Interesting. Thanks Larry. I'll dig more on my side.
>>>>>>
>>>>>> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay <lmc...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> No, I cannot reproduce it.
>>>>>>> If you are modifying the correct gateway-log4j.properties and
>>>>>>> restarting the server you should not see that.
>>>>>>>
>>>>>>> In fact, turning on DEBUG for wire via:
>>>>>>> log4j.logger.org.apache.http.wire=DEBUG
>>>>>>>
>>>>>>> Doesn't result in output in gateway.out for me but instead
>>>>>>> gateway.log and turning it on and off certainly works for me.
>>>>>>>
>>>>>>> You may have enabled TRACE logging if you are seeing body content -
>>>>>>> those settings are like the following:
>>>>>>>
>>>>>>> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess
>>>>>>> #log4j.additivity.org.apache.knox.gateway.access=false
>>>>>>>
>>>>>>> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver
>>>>>>> #log4j.additivity.org.apache.knox.gateway.http=false
>>>>>>> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF
>>>>>>> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF
>>>>>>> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF
>>>>>>> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF
>>>>>>>
>>>>>>> I suggest you back up to the gateway-log4j.properties from the
>>>>>>> original install and remove any other log4j config that you may have
>>>>>>> elsewhere.
>>>>>>>
>>>>>>> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han <wei...@uber.com> wrote:
>>>>>>>
>>>>>>>> Hi Larry - Thanks a lot for getting back to me.
>>>>>>>>
>>>>>>>> Yes I made sure all DEBUG level is turned off in my
>>>>>>>> gateway-log4j.properties
>>>>>>>> file, but that doesn't seem to be working. I also tried to
>>>>>>>> explicitly set log4j.logger.httpclient.wire.header to WARN (as
>>>>>>>> suggested in post
>>>>>>>> <https://stackoverflow.com/questions/4915414/disable-httpclient-logging>),
>>>>>>>> but that also didn't help.
>>>>>>>>
>>>>>>>> Actually it's very easy to reproduce this(at least on my side). If
>>>>>>>> you call knox with a webhdfs request (like
>>>>>>>> webhdfs/v1/tmp/weihan/small.txt?op=OPEN),
>>>>>>>> you should be able to see a bunch of below logs in gateway.out. In
>>>>>>>> fact it
>>>>>>>> outputs the actual content on the wire(security hole?)
>>>>>>>>
>>>>>>>> 06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire -
>>>>>>>> http-outgoing-2 << "[0x0][0x0
>>>>>>>>
>>>>>>>> Let me know if you're able to repro this.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay <lmc...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Wei -
>>>>>>>>>
>>>>>>>>> If you look at your gateway-log4j.properties file, you should see
>>>>>>>>> something like the following near the top:
>>>>>>>>>
>>>>>>>>> app.log.dir=${launcher.dir}/../logs
>>>>>>>>> app.log.file=${launcher.name}.log
>>>>>>>>> app.audit.file=${launcher.name}-audit.log
>>>>>>>>>
>>>>>>>>> log4j.rootLogger=ERROR, drfa
>>>>>>>>>
>>>>>>>>> log4j.logger.org.apache.knox.gateway=INFO
>>>>>>>>> #log4j.logger.org.apache.knox.gateway=DEBUG
>>>>>>>>>
>>>>>>>>> #log4j.logger.org.eclipse.jetty=DEBUG
>>>>>>>>> #log4j.logger.org.apache.shiro=DEBUG
>>>>>>>>> #log4j.logger.org.apache.http=DEBUG
>>>>>>>>> #log4j.logger.org.apache.http.client=DEBUG
>>>>>>>>> #log4j.logger.org.apache.http.headers=DEBUG
>>>>>>>>> #log4j.logger.org.apache.http.wire=DEBUG
>>>>>>>>>
>>>>>>>>> Note that all of the DEBUG settings are commented out.
>>>>>>>>> Also note that the rootLogger is set to ERROR and not DEBUG.
>>>>>>>>>
>>>>>>>>> Can you compare and share with us what yours are set to?
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>>
>>>>>>>>> --larry
>>>>>>>>>
>>>>>>>>> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han <wei...@uber.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Knox experts -
>>>>>>>>>>
>>>>>>>>>> Has anyone successfully turn off the debug loggings
>>>>>>>>>> from org.apache.http.wire, like below?
>>>>>>>>>>
>>>>>>>>>> 06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire
>>>>>>>>>> - http-outgoing-2 << "[0x0][0x0
>>>>>>>>>>
>>>>>>>>>> I've tried almost everything in this stackoverflow post
>>>>>>>>>> <https://stackoverflow.com/questions/4915414/disable-httpclient-logging>,
>>>>>>>>>> but my gateway.out is still filled with all the debug logs from
>>>>>>>>>> httpclient.
>>>>>>>>>>
>>>>>>>>>> Any tip would be appreciated.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
