The number of concurrent major compactions per tserver should be directly related to the number of tablets hosted on each tablet server. During normal operations, major compactions for a tablet should be initiated by the applying the value of `table.compaction.major.ratio` to `table.file.max` and the actual number of files. Additionally, the major compaction process in the tablet server is controlled by the `tserver.compaction.major.*` properties. If you want to test whether or not your `tserver.compaction.major.concurrent.max` property is being honored, go into the shell and compact the table manually using the compact command. This will perform a full compaction and you should see 8 concurrent MajC's occurring on each tserver. If this is the case, but you are not seeing 8 concurrent MajC's at ingest time, then I would think that you are not pushing it hard enough.
I am less familiar with the minor compaction initiation process, but in general if you are not seeing hold times, push harder. You could also test your `tserver.compaction.minor.concurrent.max` setting by performing a flush command on the table from the shell. > On March 23, 2017 at 10:13 AM Massimilian Mattetti <massi...@il.ibm.com> > wrote: > > With the configuration I presented before the concurrent major > compactions are never more than 3 per tablet server while the minor are under > the 4 per node. Can one of the other configurations be the cause of this > behavior? > > Regards, > Max > > > > From: Dave Marion <dlmar...@comcast.net> > To: user@accumulo.apache.org, Massimilian Mattetti/Haifa/IBM@IBMIL > Date: 23/03/2017 14:55 > Subject: Re: tserver.compaction.*.concurrent.max behavior in > Accumulo 1.8.1 > > --------------------------------------------- > > > > Can you explain more what you mean by "My problem is that both the minor > and major compactions do not overcome their default max values?" I have done > some testing with 1.8.1 and specifically modifying > tserver.compaction.major.concurrent.max to a higher number and I have seen it > take effect. > > On March 23, 2017 at 7:54 AM Massimilian Mattetti <massi...@il.ibm.com> > wrote: > > Hi All, > > I am running some heavy ingestion process on a 3 nodes cluster of > Accumulo 1.8.1, using the following configuration: > > table.compaction.minor.logs.threshold=10 > table.durability=flush > table.file.max=30 > > tserver.wal.blocksize=2G > tserver.walog.max.size=4G > tserver.mutation.queue.max=2M > tserver.memory.maps.max=4G > tserver.compaction.minor.concurrent.max=50 > tserver.compaction.major.concurrent.max=8 > > My problem is that both the minor and major compactions do not overcome > their default max values. I checked the config from the shell and it looks > fine to me: > > default | tserver.compaction.major.concurrent.max ................ | 3 > system | @override > ........................................... | 8 > > default | tserver.compaction.minor.concurrent.max ............... | 4 > system | @override > ........................................... | 50 > > Is something changed from 1.8.0? I haven't seen such behavior with the > previous version. > Thanks. > > Regards, > Max > > > > >
