Hi Lewis,
for me (nutch use case) a lower value is better, because of 3 main reasons:
a) load is better distributed for the db backend
b) when running the nutch fetcherJob, towards the end of the job you
don't have to wait for gora flushing all data to backend, because it was
mostly done during the fetching
c) during debugging you'll get gora/cassandra flushing errors much earlier
I'm running with 1k write buffer for cassandra.
--Roland
Am 01.03.2013 02:01, schrieb Lewis John Mcgibbney:
Hi,
We use the above class for write operations in the Nutch InjectorJob.
I am writing large URL lists to Cassandra using Gora and wonder if I
can get it working better.
Currently I am getting around 10000 writes per 90 seconds. Don't get
me wrong, I am working from a very primitive laptop and right now I am
merely attempting to push the software.
What I want to know, is what is the consequence of altering the
BUFFER_LIMIT_WRITE_VALUE?
Currently we set a default value of 10K for the limit on this value,
meaning that Gora batches flushes to reflect this value.
Is a higher or lower value better? Is there any evidence of better
performance by changing this value.
I see it a pretty critical so I am wanting to understand more about this.
Thanks
Lewis
--
/Lewis/
