Hi guys, I'm working with Wei and we still don't figure it out. Let me clarify the question.
Currently, we're seeing lots of DEBUG logs in file *gateway.out*, which is from here https://github.com/apache/knox/blob/master/gateway-release/home/bin/gateway.sh#L127. On the one hand, it prints the file content just like Wei talked about before, on the other hand we suspect it might be related to the performance issue when download a file through WEBHDFS. So we're trying to disable all these DEBUG logs. We tried simply removing this part *>>$APP_OUT_FILE*, although there is no such output file, but actually Knox still prints logs to console. So what we want to do is to disable all the DEBUG log thoroughly, so the service won't print logs to anywhere. We almost tried everything in *gateway-log4j.properties*, but it seems it only affects app.log.file=${launcher.name}.*log* instead of *gateway.out*. So, any idea guys? Thanks, Guang On Sun, Apr 15, 2018 at 11:08 AM, larry mccay <lmc...@apache.org> wrote: > +1 to Kevin's point. > Ambari rewrites all configs on server restart. > > On Sun, Apr 15, 2018 at 1:16 PM, Kevin Risden <kris...@apache.org> wrote: > >> Are you using Ambari or deploying Knox manually? >> >> If you using Ambari, then Ambari will force overwrite the log4j configs >> during a restart. You must update the log4j settings in Ambari. Another >> option if using Ambari is to find the debug setting and set it to false (I >> don't have a cluster in front of me so can't look up the setting). >> >> Kevin Risden >> >> On Sun, Apr 15, 2018 at 10:56 AM, Wei Han <wei...@uber.com> wrote: >> >>> Interesting. Thanks Larry. I'll dig more on my side. >>> >>> On Sun, Apr 15, 2018 at 4:54 AM, larry mccay <lmc...@apache.org> wrote: >>> >>>> No, I cannot reproduce it. >>>> If you are modifying the correct gateway-log4j.properties and >>>> restarting the server you should not see that. >>>> >>>> In fact, turning on DEBUG for wire via: >>>> log4j.logger.org.apache.http.wire=DEBUG >>>> >>>> Doesn't result in output in gateway.out for me but instead gateway.log >>>> and turning it on and off certainly works for me. >>>> >>>> You may have enabled TRACE logging if you are seeing body content - >>>> those settings are like the following: >>>> >>>> #log4j.logger.org.apache.knox.gateway.access=TRACE,httpaccess >>>> #log4j.additivity.org.apache.knox.gateway.access=false >>>> >>>> #log4j.logger.org.apache.knox.gateway.http=TRACE,httpserver >>>> #log4j.additivity.org.apache.knox.gateway.http=false >>>> ##log4j.logger.org.apache.knox.gateway.http.request.headers=OFF >>>> ##log4j.logger.org.apache.knox.gateway.http.response.headers=OFF >>>> ##log4j.logger.org.apache.knox.gateway.http.request.body=OFF >>>> ##log4j.logger.org.apache.knox.gateway.http.response.body=OFF >>>> >>>> I suggest you back up to the gateway-log4j.properties from the original >>>> install and remove any other log4j config that you may have elsewhere. >>>> >>>> On Sun, Apr 15, 2018 at 1:58 AM, Wei Han <wei...@uber.com> wrote: >>>> >>>>> Hi Larry - Thanks a lot for getting back to me. >>>>> >>>>> Yes I made sure all DEBUG level is turned off in my >>>>> gateway-log4j.properties >>>>> file, but that doesn't seem to be working. I also tried to explicitly >>>>> set log4j.logger.httpclient.wire.header to WARN (as suggested in post >>>>> <https://stackoverflow.com/questions/4915414/disable-httpclient-logging>), >>>>> but that also didn't help. >>>>> >>>>> Actually it's very easy to reproduce this(at least on my side). If you >>>>> call knox with a webhdfs request (like >>>>> webhdfs/v1/tmp/weihan/small.txt?op=OPEN), >>>>> you should be able to see a bunch of below logs in gateway.out. In fact it >>>>> outputs the actual content on the wire(security hole?) >>>>> >>>>> 06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire - >>>>> http-outgoing-2 << "[0x0][0x0 >>>>> >>>>> Let me know if you're able to repro this. >>>>> >>>>> Thanks. >>>>> >>>>> On Sat, Apr 14, 2018 at 7:11 AM, larry mccay <lmc...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi Wei - >>>>>> >>>>>> If you look at your gateway-log4j.properties file, you should see >>>>>> something like the following near the top: >>>>>> >>>>>> app.log.dir=${launcher.dir}/../logs >>>>>> app.log.file=${launcher.name}.log >>>>>> app.audit.file=${launcher.name}-audit.log >>>>>> >>>>>> log4j.rootLogger=ERROR, drfa >>>>>> >>>>>> log4j.logger.org.apache.knox.gateway=INFO >>>>>> #log4j.logger.org.apache.knox.gateway=DEBUG >>>>>> >>>>>> #log4j.logger.org.eclipse.jetty=DEBUG >>>>>> #log4j.logger.org.apache.shiro=DEBUG >>>>>> #log4j.logger.org.apache.http=DEBUG >>>>>> #log4j.logger.org.apache.http.client=DEBUG >>>>>> #log4j.logger.org.apache.http.headers=DEBUG >>>>>> #log4j.logger.org.apache.http.wire=DEBUG >>>>>> >>>>>> Note that all of the DEBUG settings are commented out. >>>>>> Also note that the rootLogger is set to ERROR and not DEBUG. >>>>>> >>>>>> Can you compare and share with us what yours are set to? >>>>>> >>>>>> thanks, >>>>>> >>>>>> --larry >>>>>> >>>>>> On Sat, Apr 14, 2018 at 2:56 AM, Wei Han <wei...@uber.com> wrote: >>>>>> >>>>>>> Hi Knox experts - >>>>>>> >>>>>>> Has anyone successfully turn off the debug loggings >>>>>>> from org.apache.http.wire, like below? >>>>>>> >>>>>>> 06:52:49.751 [qtp1473205473-61] DEBUG org.apache.http.wire - >>>>>>> http-outgoing-2 << "[0x0][0x0 >>>>>>> >>>>>>> I've tried almost everything in this stackoverflow post >>>>>>> <https://stackoverflow.com/questions/4915414/disable-httpclient-logging>, >>>>>>> but my gateway.out is still filled with all the debug logs from >>>>>>> httpclient. >>>>>>> >>>>>>> Any tip would be appreciated. >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
