On 12/22/2016 8:10 AM, Webster Homer wrote:
> While testing CDCR I found that it is writing tons of log messages per
> second. Example:
> 2016-12-21 23:24:41.652 INFO (qtp110456297-13) [c:sial-catalog-material
> s:shard1 r:core_node1 x:sial-catalog-material_shard1_replica1]
> o.a.s.c.S.Request [sial-catalog-material_shard1_replica1] webapp=/solr
> path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2}
> status=0 QTime=0
> 2016-12-21 23:24:41.653 INFO (qtp110456297-18) [c:sial-catalog-material
> s:shard1 r:core_node1 x:sial-catalog-material_shard1_replica1]
> o.a.s.c.S.Request [sial-catalog-material_shard1_replica1] webapp=/solr
> path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2}
> status=0 QTime=0
I hadn't looked closely at the messages you were seeing in your logs
until now.
These messages are *request* logging. This is the same code path that
logs every query -- it's not specific to CDCR. It's just logging all
the requests that Solr is receiving. If this log message were changed
to DEBUG, then Solr would not log queries by default. A large number of
Solr users want that logging.
I think that you could probably avoid seeing these logs by configuring
log4j to not log things tagged
asorg.apache.solr.core.SolrCore.Request(even though it's not a real
class, I think log4j can still configure it) ... but then you wouldn't
get your queries logged either.
In order to not log these particular messages, but still log queries and
other requests, the request logging code will need to have a way to
specify that certain messages should not be logged. This might be
something thatcould beconfigurable at the request handler definition
level -- put something in the requestHandler configuration (for /cdcr in
this case) that tells it to skip logging. That seems like a good
feature to have.
After looking at the CDCR configuration page in the reference guide, I
might have a little more insight. You're getting one of these logs
every 1-2 milliseconds ... so it sounds like you have configured the
CDCR with a schedule of one millisecond. The default value for the
replicator schedule is is ten milliseconds, and the update log
synchronizer defaults to a full minute. I'm guessing that CDCR is not
designed to have such a low schedule value. I would personally
configure the replicator schedule even higher than the default --
network latency between Internet locations is often longer than ten
milliseconds.
Thanks,
Shawn
