[
https://issues.apache.org/jira/browse/ACCUMULO-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984144#comment-14984144
]
Josh Elser commented on ACCUMULO-3797:
--------------------------------------
[~ecn] [~phrocker], didn't you all observe this and make a fix for something
similar? Might this be a duplicate issue now?
> Tablet not major compacting for >30minutes
> ------------------------------------------
>
> Key: ACCUMULO-3797
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3797
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Reporter: Josh Elser
> Labels: 1.7.0_QA
> Fix For: 1.7.1, 1.8.0
>
>
> Noticed a case during RandomWalk where scans against a table were blocked for
> a long time. There were too many files for the tablet, and batch scans were
> blocked.
> Eventually, the major compaction _did_ start, but I'm not sure why it didn't
> start in order seconds to minutes. Eventually, it was noticed that the trace
> table (1) had a ton of WALs and needed a minor compaction. Shortly after
> that, a major compaction for our table (h6) was started.
> {noformat}
> 2015-05-11 12:02:17,159 [tserver.TabletServer] DEBUG: Initiating minor
> compaction for 1<< because it has 35 write ahead logs
> 2015-05-11 12:02:17,160 [tserver.NativeMap] DEBUG: Allocated native map
> 0x000000000e198b30
> 2015-05-11 12:02:17,160 [tablet.Tablet] DEBUG: MinC initiate lock 0.00 secs
> 2015-05-11 12:02:17,160 [zookeeper.DistributedWorkQueue] DEBUG: Looking for
> work in /accumulo/374feac3-96ff-4ba1-99e6-8c1ab5f9d7c2/replication/workqueue
> 2015-05-11 12:02:17,172 [tablet.Tablet] DEBUG: Major compaction plan:
> [hdfs://cn020:8020/accumulo/tables/h6/t-0000hz8/C0000i04.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzh/C0000i0b.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hz
> e/C0000i05.rf, hdfs://cn020:8020/accumulo/tables/h6/t-0000hzj/C0000i10.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hz3/C0000i0c.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzm/C0000i0x.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-00
> 00hzk/C0000i0v.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hza/C0000i0j.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzc/C0000i0f.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzi/C0000i0q.rf] propogate deletes
> : true
> 2015-05-11 12:02:17,172 [tablet.Tablet] DEBUG: MajC initiate lock 0.00 secs,
> wait 0.00 secs
> 2015-05-11 12:02:17,173 [tablet.Tablet] DEBUG: Starting MajC h6<< (NORMAL)
> [hdfs://cn020:8020/accumulo/tables/h6/t-0000hz3/C0000i0c.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hze/C0000i05.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0
> 000hzm/C0000i0x.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hza/C0000i0j.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzi/C0000i0q.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzh/C0000i0b.rf,
> hdfs://cn020:8020/accumulo/tables/h6
> /t-0000hz8/C0000i04.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzk/C0000i0v.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzj/C0000i10.rf,
> hdfs://cn020:8020/accumulo/tables/h6/t-0000hzc/C0000i0f.rf] -->
> hdfs://cn020:8020/accumulo/t
> ables/h6/default_tablet/C0000i2j.rf_tmp []
> 2015-05-11 12:02:17,219 [tablet.MinorCompactor] DEBUG: Begin minor compaction
> hdfs://cn020:8020/accumulo/tables/1/default_tablet/F0000i2i.rf_tmp 1<<
> 2015-05-11 12:02:17,290 [tabletserver.LargestFirstMemoryManager] DEBUG:
> BEFORE compactionThreshold = 0.851 maxObserved = 11,134,533
> 2015-05-11 12:02:17,290 [tabletserver.LargestFirstMemoryManager] DEBUG: AFTER
> compactionThreshold = 0.851
> 2015-05-11 12:02:17,337 [tserver.TabletServer] DEBUG: ScanSess tid
> 172.18.128.20:35527 +rep 0 entries in 0.00 secs, nbTimes = [1 1 1.00 1]
> 2015-05-11 12:02:17,833 [tablet.Compactor] DEBUG: Compaction h6<< 54,721 read
> | 54,721 written | 238,956 entries/sec | 0.229 secs
> 2015-05-11 12:02:17,841 [tserver.TabletServer] DEBUG: ScanSess tid
> 172.18.128.20:37393 +r 1 entries in 0.00 secs, nbTimes = [1 1 1.00 1]
> 2015-05-11 12:02:17,841 [tserver.TabletServer] INFO : Adding 1 logs for
> extent !0<< as alias 230
> 2015-05-11 12:02:17,842 [tserver.TabletServer] INFO : Adding 1 logs for
> extent +r<< as alias 1
> 2015-05-11 12:02:17,941 [tablet.Compactor] DEBUG: Compaction 1<< 47,890 read
> | 47,713 written | 164,006 entries/sec | 0.292 secs
> {noformat}
> After the major compactions completed, things got back to normal. It seems
> highly unusual to sit there blocked for so long.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
