The tserver.compaction.major.thread.files.open.max=10 and I have a maximum of 30 files per tablet (table.file.max=30). What property are you referring to for the max open files? Thanks, Max
From: "Marc P." <marc.par...@gmail.com> To: user@accumulo.apache.org Date: 23/03/2017 16:37 Subject: Re: tserver.compaction.*.concurrent.max behavior in Accumulo 1.8.1 Max, Additionally, what are your max open files? If you are under heavy ingest you may exceed this with compactions if they cannot keep up. Additionally, how many files per compaction are you allowing? On Thu, Mar 23, 2017 at 10:27 AM, Michael Wall <mjw...@gmail.com> wrote: Max, On you 3 node cluster, how many tables are you ingesting into? How many tablets are in each table? Are the tablets equally spread amongst the 3 tablet servers? Mike On Thu, Mar 23, 2017 at 10:13 AM Massimilian Mattetti <massi...@il.ibm.com > wrote: With the configuration I presented before the concurrent major compactions are never more than 3 per tablet server while the minor are under the 4 per node. Can one of the other configurations be the cause of this behavior? Regards, Max From: Dave Marion <dlmar...@comcast.net> To: user@accumulo.apache.org, Massimilian Mattetti/Haifa/IBM@IBMIL Date: 23/03/2017 14:55 Subject: Re: tserver.compaction.*.concurrent.max behavior in Accumulo 1.8.1 Can you explain more what you mean by "My problem is that both the minor and major compactions do not overcome their default max values?" I have done some testing with 1.8.1 and specifically modifying tserver.compaction.major.concurrent.max to a higher number and I have seen it take effect. On March 23, 2017 at 7:54 AM Massimilian Mattetti <massi...@il.ibm.com> wrote: Hi All, I am running some heavy ingestion process on a 3 nodes cluster of Accumulo 1.8.1, using the following configuration: table.compaction.minor.logs.threshold=10 table.durability=flush table.file.max=30 tserver.wal.blocksize=2G tserver.walog.max.size=4G tserver.mutation.queue.max=2M tserver.memory.maps.max=4G tserver.compaction.minor.concurrent.max=50 tserver.compaction.major.concurrent.max=8 My problem is that both the minor and major compactions do not overcome their default max values. I checked the config from the shell and it looks fine to me: default | tserver.compaction.major.concurrent.max ................ | 3 system | @override ........................................... | 8 default | tserver.compaction.minor.concurrent.max ............... | 4 system | @override ........................................... | 50 Is something changed from 1.8.0? I haven't seen such behavior with the previous version. Thanks. Regards, Max
