Please take a look at the following JIRAs: HBASE-4552 HBASE-4716 Cheers
On Sat, May 17, 2014 at 11:30 PM, Frank Chow <[email protected]>wrote: > Hi guys, > > Do bulkload Hfiles sometimes failed on my cluster because rpc time out. > I found that when doing bulkload hfiles, it require wirtelock on the > region when there are multi-families. But the compact operation also > require read lock on the region. When the compacting files are big, it need > more time to finish (some times cost half an hour or more). So when the > region is compacting, the bulkloadHfiles operation will not be able to get > the writelock, and causing the bulkloading hfiles timeout. > > Code is below: > HRegion: > public boolean bulkLoadHFiles(List<Pair<byte[], String>> familyPaths, > BulkLoadListener bulkLoadListener) throws IOException { > Preconditions.checkNotNull(familyPaths); > // we need writeLock for multi-family bulk load > startBulkRegionOperation(hasMultipleColumnFamilies(familyPaths)); > > private void startBulkRegionOperation(boolean writeLockNeeded) > throws NotServingRegionException, RegionTooBusyException, > InterruptedIOException { > if (this.closing.get()) { > throw new > NotServingRegionException(regionInfo.getRegionNameAsString() + > " is closing"); > } > if (writeLockNeeded) lock(lock.writeLock()); > else lock(lock.readLock()); > > My question is: why write lock is needed when there is multi-families ? > Can readlock also work? If it only need a readlock, there will be no > conflict and loading hfiles will not timeout. > > Thanks.
