Jerome, thanks a lot. This is helpful.
-AJ
Jérôme Charron wrote:
Following the tutorial, I redirect the log messages to a log file. But,
when crawling 1 million pages, this log file can become hugh and writing
log messages to a huge file can slow down the fetching process. Is
there a better way to manage the log? maybe saving it to a series of
smaller files? appreciate your suggestions.
Change your JDK logging behaviors in $JAVA_HOME/jre/lib/logging.properties
and add a FileHandler with limit greater than 0.
Thus, the logs will be rotated in many log files (with a maximum size =
limit).
See also
http://java.sun.com/j2se/1.4.2/docs/api/java/util/logging/package-summary.html
http://java.sun.com/j2se/1.4.2/docs/api/java/util/logging/FileHandler.html
Regards
Jérôme
