Hi,
What do you mean by "accumulo's ingest rate affects accumulo's insertion
performance"? Ingest *is* insertion into the database. Please describe
what you mean by "insertion performance". Are you comparing some custom
code you have written to the Continuous Ingest client?
60K entries/sec per tabletserver seems to be a reasonable ingest rate
for the hardware you have described for continuous ingest.
However, you should *definitely* tweak the default configurations. The
provided configuration are meant to operate Accumulo in less than 3GB of
resident memory. I would imagine that this is a bottleneck.
A non-exhaustive list of things to check ...
* Increase TabletServer JVM heap size (4-8G)
* Enable the native maps [1]
* Increase tserver.total.mutation.queue.max=256M [2]
* Reduce table durability if your use-case allows it [3]
[1] http://accumulo.apache.org/1.7/accumulo_user_manual.html#_native_map
[2]
http://accumulo.apache.org/1.7/accumulo_user_manual.html#_tserver_total_mutation_queue_max
[3] http://accumulo.apache.org/blog/2016/11/02/durability-performance.html
hjs19890 wrote:
hi,
I'm testing the accumulo insertion performance and found that accumulo's
ingest rate affects accumulo's insertion performance. In the access to relevant
information when I found the accumulo itself comes with the test suite.
Therefore, I ran accumulo-1.7.1 / test / system / continuous / start-ingest.sh
to test the ingest rate of my cluster. Test results averaged 180,000 (entry /
s) ingest rate and I tested the results of the insertion performance is
similar, although these tests are not comparable. This result is not very
satisfactory.
My cluster has 4 servers (1 master, 3 tservers), the computer
configuration is i7-4700, 4cores, 32g Mem. The accumulo version is 1.7.1. The
software is the default configuration. Therefore, I would like to ask a few
questions:
1, insert performance bottleneck?
2, my cluster of intake reached the bottleneck it?
3, if not reached the bottleneck, how can I tune my cluster?
thanks,
