Hi,
I also agree and would love to see things changed.
In general I would love to be able to be able to write log files also
in custom storages types.
For example it would be great in case it would be possibe to write
log files into the ndfs or into a database.
Especially for smaller scaled horizontal / intranet setups having
logs in a database would be nice to see what pages throw errors.
I suggest you open a improvement request in the nutch jira.
Stefan
Am 03.01.2006 um 15:22 schrieb Christopher Burkey:
H Daniel,
I agree with you. Using the LogFormatter is a bad design and is
causing us problems as well. We use Spring and set the error
handler to any objects that want it.
Daniel Feinstein wrote:
Hi,
This is my first email to the community. In our (rawsugar.com)
project we integrated multiple lucene indexes and a nutch package.
We use the packages for about two years. Lucene is working great
and we could use it as a "black box". Unfortunately, we had to
modify Nutch package for our usage because of some leaks and a
design problem in LogFormatter class. This fact force us to use a
customized version of Nutch and avoid to update versions. At the
last week we decided to move to the latest version of Nutch but
discovered that the LogFormatter still have the same problem
(leaks I believe has been fixed).
The problem:
In Nutch project LogFormatter has duplicated functionality:
1) Logger records format and
2) Severe error handler
The first usage is standard and usually could be overwritten by a
user of the package by modifying logging.properties file.
The second usage is much more problematic because it affects the
behavior of the whole application (not only Nutch package). To
support the error handling LogFormatter enforce usage of the
formatter class by all classes of the whole application which uses
Nutch package. This is done by overwriting all the system handlers
(class java.util.logging.Handler). This operation prevents the
application to use its own log formatter. Also this cause
LogFormatter.hasLoggedSevere() to be sensitive to all severe
records in the big system but not only to relevant. More than that
this flag, LogFormatter.loggedSevere is never cleaned what means
if an application had one, even unrelated severe record, tools
like Fetcher will never run until the application will be restarted.
I would like to suggest the following solutions:
1) To separate the functionality of log formatting and error
handling or
2) Change LogFormatter class to be affected only by nutch package
functions
For my opinion the first solution is much better especially if
error handling will be encapsulated for each task. I have found
the following usages of LogFormatter.hasLoggedSevere():
- Fetcher
- URLFilterChecker
- ParseSegment
Unfortunately I'm not familiar enough with the usages above to
implement this solution that why I suggest the second one. The
attached file is our implementation of LogFormatter class which we
use for more than a year. I hope this change will be accepted by
the community.
Daniel
_____________________
*/Daniel Feinstein/*
Lead Software Engineer
www.rawsugar.com <http://www.rawsugar.com/>
__________________________
CONFIDENTIALITY NOTICE
This email may contain confidential and privileged material for
the sole use of the intended recipient. Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient please contact the sender and delete all copies.
--
Christopher Burkey
513-542-3401
[EMAIL PROTECTED]
http://www.openedit.org
