If it is Nutch logging, change its level in conf/log4j.properties. It can also
be Hadoop logging.
M.
-----Original message-----
> From:shubham.gupta <[email protected]>
> Sent: Tuesday 23rd August 2016 8:15
> To: [email protected]
> Subject: Application creating huge amount of logs : Nutch 2.3.1 + Hadoop 2.7.1
>
> Hey
>
> I have integrated Nutch 2.3.1 with Hadoop 2.7.1, and the fetcher.parse
> property is set TRUE and the database used is MongoDB. While the map job
> of nutch runs, it creates a huge size of nodelogs over 13GB in size. And
> the cause of such huge amount of files in unknown. Any suggestion would
> help.
>
> Thanks in advance.
>
> Shubham Gupta
>
