[jira] [Commented] (HBASE-15456) CreateTableProcedure/ModifyTableProcedure needs to fail when there is no family in table descriptor
[
https://issues.apache.org/jira/browse/HBASE-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201798#comment-15201798
]
Hudson commented on HBASE-15456:
SUCCESS: Integrated in HBase-1.4 #29 (See
[https://builds.apache.org/job/HBase-1.4/29/])
HBASE-15456 CreateTableProcedure/ModifyTableProcedure needs to fail when
(tedyu: rev eda2656b88edc829d4d520ea5fc39cf73aca270f)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java
*
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java
> CreateTableProcedure/ModifyTableProcedure needs to fail when there is no
> family in table descriptor
> ---
>
> Key: HBASE-15456
> URL: https://issues.apache.org/jira/browse/HBASE-15456
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15456-001_branch-1.patch,
> HBASE-15456-branch-1.patch, HBASE-15456-branch-1.patch,
> HBASE-15456-branch-1_v002.patch, HBASE-15456-v001.patch,
> HBASE-15456-v002.patch, HBASE-15456-v002.patch, HBASE-15456-v003.patch,
> HBASE-15456-v004.patch
>
>
> If there is only one family in the table, DeleteColumnFamilyProcedure will
> fail.
> Currently, when hbase.table.sanity.checks is set to false, hbase master logs
> a warning and CreateTableProcedure/ModifyTableProcedure will succeed.
> This behavior is not consistent with DeleteColumnFamilyProcedure's.
> Another point, before HBASE-13145, PeriodicMemstoreFlusher will run into the
> following exception. lastStoreFlushTimeMap is populated for families, if
> there is no family in the table, there is no entry in lastStoreFlushTimeMap.
> {code}
> 16/02/01 11:14:26 ERROR regionserver.HRegionServer$PeriodicMemstoreFlusher:
> Caught exception
> java.util.NoSuchElementException
> at
> java.util.concurrent.ConcurrentHashMap$HashIterator.nextEntry(ConcurrentHashMap.java:1354)
>
> at
> java.util.concurrent.ConcurrentHashMap$ValueIterator.next(ConcurrentHashMap.java:1384)
>
> at java.util.Collections.min(Collections.java:628)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getEarliestFlushTimeForAllStores(HRegion.java:1572)
>
> at
> org.apache.hadoop.hbase.regionserver.HRegion.shouldFlush(HRegion.java:1904)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher.chore(HRegionServer.java:1509)
>
> at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15456) CreateTableProcedure/ModifyTableProcedure needs to fail when there is no family in table descriptor
[
https://issues.apache.org/jira/browse/HBASE-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198157#comment-15198157
]
huaxiang sun commented on HBASE-15456:
--
Hi [~te...@apache.org] I attached a patch for branch-1, thanks!
> CreateTableProcedure/ModifyTableProcedure needs to fail when there is no
> family in table descriptor
> ---
>
> Key: HBASE-15456
> URL: https://issues.apache.org/jira/browse/HBASE-15456
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15456-001_branch-1.patch, HBASE-15456-v001.patch,
> HBASE-15456-v002.patch, HBASE-15456-v002.patch, HBASE-15456-v003.patch
>
>
> If there is only one family in the table, DeleteColumnFamilyProcedure will
> fail.
> Currently, when hbase.table.sanity.checks is set to false, hbase master logs
> a warning and CreateTableProcedure/ModifyTableProcedure will succeed.
> This behavior is not consistent with DeleteColumnFamilyProcedure's.
> Another point, before HBASE-13145, PeriodicMemstoreFlusher will run into the
> following exception. lastStoreFlushTimeMap is populated for families, if
> there is no family in the table, there is no entry in lastStoreFlushTimeMap.
> {code}
> 16/02/01 11:14:26 ERROR regionserver.HRegionServer$PeriodicMemstoreFlusher:
> Caught exception
> java.util.NoSuchElementException
> at
> java.util.concurrent.ConcurrentHashMap$HashIterator.nextEntry(ConcurrentHashMap.java:1354)
>
> at
> java.util.concurrent.ConcurrentHashMap$ValueIterator.next(ConcurrentHashMap.java:1384)
>
> at java.util.Collections.min(Collections.java:628)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getEarliestFlushTimeForAllStores(HRegion.java:1572)
>
> at
> org.apache.hadoop.hbase.regionserver.HRegion.shouldFlush(HRegion.java:1904)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher.chore(HRegionServer.java:1509)
>
> at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15389) Write out multiple files when compaction
[ https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198668#comment-15198668 ] Ted Yu commented on HBASE-15389: +1, assuming tests pass. > Write out multiple files when compaction > > > Key: HBASE-15389 > URL: https://issues.apache.org/jira/browse/HBASE-15389 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Affects Versions: 2.0.0, 1.3.0, 0.98.19 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: HBASE-15389-0.98.patch, HBASE-15389-uc.patch, > HBASE-15389-v1.patch, HBASE-15389-v10.patch, HBASE-15389-v11.patch, > HBASE-15389-v2.patch, HBASE-15389-v3.patch, HBASE-15389-v4.patch, > HBASE-15389-v5.patch, HBASE-15389-v6.patch, HBASE-15389-v7.patch, > HBASE-15389-v8.patch, HBASE-15389-v9.patch, HBASE-15389.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15475) Allow TimestampsFilter to provide a seek hint
[ https://issues.apache.org/jira/browse/HBASE-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15475: -- Attachment: HBASE-15475-v4.patch > Allow TimestampsFilter to provide a seek hint > - > > Key: HBASE-15475 > URL: https://issues.apache.org/jira/browse/HBASE-15475 > Project: HBase > Issue Type: Improvement > Components: Client, Filters, regionserver >Affects Versions: 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15475-v1.patch, HBASE-15475-v2.patch, > HBASE-15475-v3.patch, HBASE-15475-v4.patch, HBASE-15475.patch > > > If a user wants to read specific timestamps currently it's a very linear > scan. This is so that all deletes can be respected. However if the user > doesn't care about deletes it can dramatically speed up the request to seek. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14963) Remove use of Guava Stopwatch from HBase client code
[ https://issues.apache.org/jira/browse/HBASE-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-14963: - Status: Patch Available (was: Reopened) > Remove use of Guava Stopwatch from HBase client code > > > Key: HBASE-14963 > URL: https://issues.apache.org/jira/browse/HBASE-14963 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Devaraj Das >Assignee: Devaraj Das > Labels: needs_releasenote > Fix For: 2.0.0 > > Attachments: HBASE-14963-branch-1.patch, no-stopwatch.txt > > > We ran into an issue where an application bundled its own Guava (and that > happened to be in the classpath first) and HBase's MetaTableLocator threw an > exception due to the fact that Stopwatch's constructor wasn't compatible... > Might be better to not depend on Stopwatch at all in MetaTableLocator since > the functionality is easily doable without. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15475) Allow TimestampsFilter to provide a seek hint
[ https://issues.apache.org/jira/browse/HBASE-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15475: -- Attachment: HBASE-15475.patch > Allow TimestampsFilter to provide a seek hint > - > > Key: HBASE-15475 > URL: https://issues.apache.org/jira/browse/HBASE-15475 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-15475.patch > > > If a user wants to read specific timestamps currently it's a very linear > scan. This is so that all deletes can be respected. However if the user > doesn't care about deletes it can dramatically speed up the request to seek. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201887#comment-15201887
]
Ted Yu commented on HBASE-14123:
{code}
+conf.setBoolean("hbase.loadincremental.validate.hfile", false);
{code}
Describe in release notes that hfile validation is turned off.
{code}
+ master.createTable(backupHTD, null, HConstants.NO_NONCE,
HConstants.NO_NONCE);
+ LOG.info("Created "+ BackupSystemTable.getTableName()+" table");
+} catch(TableExistsException e) {
+ e.printStackTrace();
{code}
Better check the existence of table before issuing table creation request.
{code}
-Job job = Job.getInstance(conf, conf.get(JOB_NAME_CONF_KEY, NAME + "_" +
inputDir));
+Job job = Job.getInstance(conf, conf.get(JOB_NAME_CONF_KEY, NAME + "_" +
System.currentTimeMillis()));
{code}
Keep the input dir in job name.
Please use EnvironmentEdgeManager.
> HBase Backup/Restore Phase 2
>
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
> Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-14123-v1.patch, HBASE-14123-v10.patch,
> HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v2.patch,
> HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch,
> HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15474) Exception in HConnectionImplementation's constructor cause Zookeeper connnections leak
[
https://issues.apache.org/jira/browse/HBASE-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15474:
---
Resolution: Duplicate
Fix Version/s: (was: 1.1.0)
Status: Resolved (was: Patch Available)
Dup of HBASE-14485
> Exception in HConnectionImplementation's constructor cause Zookeeper
> connnections leak
> ---
>
> Key: HBASE-15474
> URL: https://issues.apache.org/jira/browse/HBASE-15474
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Allan Yang
>Assignee: Allan Yang
> Labels: patch
> Attachments: HBASE-15474-branch-1.v3.patch, HBASE-15474v2.patch
>
>
> HConnectionImplementation creates a ZooKeeperKeepAliveConnection during
> construction, but if the constructor throw a exception, the zookeeper
> connection is not properly closed.
> {code}
> HConnectionImplementation(Configuration conf, boolean managed,
> ExecutorService pool, User user) throws IOException {
> this(conf);
> this.user = user;
> this.batchPool = pool;
> this.managed = managed;
> this.registry = setupRegistry();
> retrieveClusterId(); //here is the zookeeper connection created
> this.rpcClient = RpcClientFactory.createClient(this.conf,
> this.clusterId);// In our case, the exception happens here, so the zookeeper
> connection never closes
> this.rpcControllerFactory = RpcControllerFactory.instantiate(conf);
> ..
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15490) Two CompactionThroughputControllerFactory co-exist in branch-1
[
https://issues.apache.org/jira/browse/HBASE-15490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203095#comment-15203095
]
Ted Yu commented on HBASE-15490:
lgtm
> Two CompactionThroughputControllerFactory co-exist in branch-1
> --
>
> Key: HBASE-15490
> URL: https://issues.apache.org/jira/browse/HBASE-15490
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15490.branch-1.patch, HBASE-15490.patch
>
>
> Currently there're two {{CompactionThroughputControllerFactory}} in our
> branch-1 code base (one in {{o.a.h.h.regionserver.compactions}} package, the
> other in {{o.a.h.h.regionserver.throttle}}) and both are in use.
> This is a regression of HBASE-14969 and only exists in branch-1. We should
> remove the one in {{o.a.h.h.regionserver.compactions}}, and change the
> default compaction throughput controller back to
> {{NoLimitThroughputController}} to keep compatible with previous branch-1
> version
> Thanks [~ghelmling] for pointing out the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15479) No more garbage or beware of autoboxing
[ https://issues.apache.org/jira/browse/HBASE-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15479: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.1.5 1.4.0 0.98.19 1.2.1 1.3.0 Status: Resolved (was: Patch Available) Thanks for the patch, Vlad. > No more garbage or beware of autoboxing > --- > > Key: HBASE-15479 > URL: https://issues.apache.org/jira/browse/HBASE-15479 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0, 1.3.0, 1.2.1, 0.98.19, 1.4.0, 1.1.5 > > Attachments: HBASE-15479-v1.patch, HBASE-15479-v2.patch > > > Quick journey with JMC in a profile mode revealed very interesting and > unexpected heap polluter on a client side. Patch will shortly follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200977#comment-15200977 ] Lars Hofhansl commented on HBASE-12148: --- Do we see actual contention, as in threads waiting for each other? Or is this slow down due to memory barrier implied by a synchronized (and lock, and CAS, and volatile, and AtomicXXX, etc)? [~stack], [~wkoetke], where does this show as a problem? It is at the time when write a store file? Or is it at the time when we insert into the memstore? > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148-V3.patch, HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot > 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15484) Correct the semantic of batch and partial
[ https://issues.apache.org/jira/browse/HBASE-15484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-15484: -- Status: Patch Available (was: Open) > Correct the semantic of batch and partial > - > > Key: HBASE-15484 > URL: https://issues.apache.org/jira/browse/HBASE-15484 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.3, 1.2.0 >Reporter: Phil Yang >Assignee: Phil Yang > > Follow-up to HBASE-15325, as discussed, the meaning of setBatch and > setAllowPartialResults should not be same. We should not regard setBatch as > setAllowPartialResults. > And isPartial should be define accurately. > (Considering getBatch==MaxInt if we don't setBatch.) If > result.rawcells.length row, isPartial==true, otherwise isPartial == false. So if user don't > setAllowPartialResults(true), isPartial should always be false. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201088#comment-15201088 ] Lars Hofhansl commented on HBASE-12148: --- I see. So this is at _read_ time from a StoreFile. Is that the main problem? Not contention at write time. Is the problem that readers lock each other out? That we can solve with a simple ReadWrite lock. (but that still takes a memory fence) Or we can have two TimeRangeTracker implementations. One that mutable (and synchronized) for the memstore, and one that is immutable (and not synchronized at all) for StoreFiles. > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148-V3.patch, HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot > 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png, > TimeRangeTracker.tiff > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15334) Add avro support for spark hbase connector
[ https://issues.apache.org/jira/browse/HBASE-15334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15334: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch, Zhan. > Add avro support for spark hbase connector > -- > > Key: HBASE-15334 > URL: https://issues.apache.org/jira/browse/HBASE-15334 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Fix For: 2.0.0 > > Attachments: HBASE-15334-1.patch, HBASE-15334-2.patch, > HBASE-15334-3.patch, HBASE-15334-4.patch, HBASE-15334-5.patch > > > Avro is a popular format for hbase storage. User may want the support > natively in the connector. > With the support, user can save serialized avro into hbase table, and then > query on top of it using spark sql. The conversion between avro and catalyst > datatype will be handled automatically. This is one way of support complex > data types. Otherwise, user has to define their own customized Serdes to > support complex data types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15392) Single Cell Get reads two HFileBlocks
[
https://issues.apache.org/jira/browse/HBASE-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203085#comment-15203085
]
Lars Hofhansl commented on HBASE-15392:
---
[~stack], let's also change the compare condition to > rather than >= (as
[~ram_krish] suggests). It won't make any difference but it will be easier to
understand in the future I feel.
> Single Cell Get reads two HFileBlocks
> -
>
> Key: HBASE-15392
> URL: https://issues.apache.org/jira/browse/HBASE-15392
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
>Reporter: stack
>Assignee: stack
> Attachments: 15392-0.98-looksee.txt, 15392.wip.patch,
> 15392v2.wip.patch, 15392v3.wip.patch, 15392v4.patch, 15392v5.patch,
> 15392v6.patch, 15392v7 (1).patch, 15392v7.patch, 15392v7.patch,
> 15392v7.patch, 15392v7.patch, 15392v7.patch, HBASE-15392_suggest.patch,
> gc.png, gc.png, io.png, no_optimize.patch, no_optimize.patch, reads.png,
> reads.png, two_seeks.txt
>
>
> As found by Daniel "SystemTap" Pol, a simple Get results in our reading two
> HFileBlocks, the one that contains the wanted Cell, and the block that
> follows.
> Here is a bit of custom logging that logs a stack trace on each HFileBlock
> read so you can see the call stack responsible:
> {code}
> 2016-03-03 22:20:30,191 INFO
> [B.defaultRpcServer.handler=20,queue=2,port=16020] regionserver.StoreScanner:
> START LOOP
> 2016-03-03 22:20:30,192 INFO
> [B.defaultRpcServer.handler=20,queue=2,port=16020] regionserver.StoreScanner:
> QCODE SEEK_NEXT_COL
> 2016-03-03 22:20:30,192 INFO
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileBlockIndex:
> STARTED WHILE
> 2016-03-03 22:20:30,192 INFO
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.CombinedBlockCache:
> OUT OF L2
> 2016-03-03 22:20:30,192 TRACE
> [B.defaultRpcServer.handler=20,queue=2,port=16020] bucket.BucketCache: Read
> offset=31409152, len=2103
> 2016-03-03 22:20:30,192 TRACE
> [B.defaultRpcServer.handler=20,queue=2,port=16020] bucket.FileIOEngine:
> offset=31409152, length=2103
> 2016-03-03 22:20:30,193 TRACE
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileReaderImpl:
> From Cache [blockType=DATA, fileOffset=2055421, headerSize=33,
> onDiskSizeWithoutHeader=2024, uncompressedSizeWithoutHeader=2020,
> prevBlockOffset=2053364, isUseHBaseChecksum=true, checksumType=CRC32C,
> bytesPerChecksum=16384, onDiskDataSizeWithHeader=2053,
> getOnDiskSizeWithHeader=2057, totalChecksumBytes=4, isUnpacked=true,
> buf=[org.apache.hadoop.hbase.nio.SingleByteBuff@e19fbd54],
> dataBeginsWith=\x00\x00\x00)\x00\x00\x01`\x00\x16user995139035672819231,
> fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
> bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
> includesTags=false, compressAlgo=NONE, compressTags=false,
> cryptoContext=[cipher=NONE keyHash=NONE]]]
> 2016-03-03 22:20:30,193 TRACE
> [B.defaultRpcServer.handler=20,queue=2,port=16020] hfile.HFileReaderImpl:
> Cache hit return [blockType=DATA, fileOffset=2055421, headerSize=33,
> onDiskSizeWithoutHeader=2024, uncompressedSizeWithoutHeader=2020,
> prevBlockOffset=2053364, isUseHBaseChecksum=true, checksumType=CRC32C,
> bytesPerChecksum=16384, onDiskDataSizeWithHeader=2053,
> getOnDiskSizeWithHeader=2057, totalChecksumBytes=4, isUnpacked=true,
> buf=[org.apache.hadoop.hbase.nio.SingleByteBuff@e19fbd54],
> dataBeginsWith=\x00\x00\x00)\x00\x00\x01`\x00\x16user995139035672819231,
> fileContext=[usesHBaseChecksum=true, checksumType=CRC32C,
> bytesPerChecksum=16384, blocksize=65536, encoding=NONE, includesMvcc=true,
> includesTags=false, compressAlgo=NONE, compressTags=false,
> cryptoContext=[cipher=NONE keyHash=NONE]]]
> java.lang.Throwable
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1515)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$CellBasedKeyBlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:324)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.seekTo(HFileReaderImpl.java:831)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.reseekTo(HFileReaderImpl.java:812)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:288)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:198)
> at
> org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:54)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:321)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:279)
> at
>
[jira] [Commented] (HBASE-15135) Add metrics for storefile age
[ https://issues.apache.org/jira/browse/HBASE-15135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197838#comment-15197838 ] Hudson commented on HBASE-15135: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1186 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1186/]) HBASE-15135 Add metrics for storefile age (apurtell: rev f0ddd3c066f10aa05e07433d91ee3d356658ec75) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapper.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegion.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java * hbase-hadoop1-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionSourceImpl.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperImpl.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperStub.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java > Add metrics for storefile age > - > > Key: HBASE-15135 > URL: https://issues.apache.org/jira/browse/HBASE-15135 > Project: HBase > Issue Type: New Feature >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0, 0.98.18 > > Attachments: HBASE-15135-branch-1.v4.1.patch, HBASE-15135-v2.patch, > HBASE-15135-v3.patch, HBASE-15135-v4.1.patch, HBASE-15135-v4.patch, > HBASE-15135.patch > > > In order to make sure that compactions are fully up to date it would be nice > to have metrics on: > * Max storefile age > * Min storefile age > * Average storefile age > * Number of reference files > If we could have those metrics per region and per regionserver it would be > awesome. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15481) Add pre/post roll to WALObserver
[ https://issues.apache.org/jira/browse/HBASE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15481: -- Attachment: HBASE-15481-v1.patch Retry to see if timeouts this time around. +1 on patch. We should commit HBASE-15228 too? > Add pre/post roll to WALObserver > > > Key: HBASE-15481 > URL: https://issues.apache.org/jira/browse/HBASE-15481 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.3.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15481-v0.patch, HBASE-15481-v1.patch, > HBASE-15481-v1.patch > > > currently the WALObserver has only a pre/post Write. It will be useful to > have a pre/post Roll too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15412) Add average region size metric
[
https://issues.apache.org/jira/browse/HBASE-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199606#comment-15199606
]
Hadoop QA commented on HBASE-15412:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 35s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 51s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 43s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 10s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 37s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
9s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 19s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 15s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 17s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 22s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 42s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 25s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
48s
[jira] [Commented] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat
[ https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201910#comment-15201910 ] Nick Dimiduk commented on HBASE-15482: -- Seems like this would apply for all the InputFormat implementations, not just SnapshotInputFormat. I have that right [~liyin]? > Provide an option to skip calculating block locations for SnapshotInputFormat > - > > Key: HBASE-15482 > URL: https://issues.apache.org/jira/browse/HBASE-15482 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Liyin Tang >Priority: Minor > > When a MR job is reading from SnapshotInputFormat, it needs to calculate the > splits based on the block locations in order to get best locality. However, > this process may take a long time for large snapshots. > In some setup, the computing layer, Spark, Hive or Presto could run out side > of HBase cluster. In these scenarios, the block locality doesn't matter. > Therefore, it will be great to have an option to skip calculating the block > locations for every job. That will super useful for the Hive/Presto/Spark > connectors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15456) CreateTableProcedure/ModifyTableProcedure needs to fail when there is no family in table descriptor
[
https://issues.apache.org/jira/browse/HBASE-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201363#comment-15201363
]
Hudson commented on HBASE-15456:
SUCCESS: Integrated in HBase-1.3 #609 (See
[https://builds.apache.org/job/HBase-1.3/609/])
HBASE-15456 Revert due to unit test failure in thrift module (tedyu: rev
944d9f6b46b9d576f2dfc9f52eb7acb1f41453b2)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestModifyTableProcedure.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCreateTableProcedure.java
> CreateTableProcedure/ModifyTableProcedure needs to fail when there is no
> family in table descriptor
> ---
>
> Key: HBASE-15456
> URL: https://issues.apache.org/jira/browse/HBASE-15456
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15456-001_branch-1.patch,
> HBASE-15456-branch-1.patch, HBASE-15456-branch-1.patch,
> HBASE-15456-branch-1_v002.patch, HBASE-15456-v001.patch,
> HBASE-15456-v002.patch, HBASE-15456-v002.patch, HBASE-15456-v003.patch,
> HBASE-15456-v004.patch
>
>
> If there is only one family in the table, DeleteColumnFamilyProcedure will
> fail.
> Currently, when hbase.table.sanity.checks is set to false, hbase master logs
> a warning and CreateTableProcedure/ModifyTableProcedure will succeed.
> This behavior is not consistent with DeleteColumnFamilyProcedure's.
> Another point, before HBASE-13145, PeriodicMemstoreFlusher will run into the
> following exception. lastStoreFlushTimeMap is populated for families, if
> there is no family in the table, there is no entry in lastStoreFlushTimeMap.
> {code}
> 16/02/01 11:14:26 ERROR regionserver.HRegionServer$PeriodicMemstoreFlusher:
> Caught exception
> java.util.NoSuchElementException
> at
> java.util.concurrent.ConcurrentHashMap$HashIterator.nextEntry(ConcurrentHashMap.java:1354)
>
> at
> java.util.concurrent.ConcurrentHashMap$ValueIterator.next(ConcurrentHashMap.java:1384)
>
> at java.util.Collections.min(Collections.java:628)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getEarliestFlushTimeForAllStores(HRegion.java:1572)
>
> at
> org.apache.hadoop.hbase.regionserver.HRegion.shouldFlush(HRegion.java:1904)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher.chore(HRegionServer.java:1509)
>
> at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndPut() and checkAndDelete()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200126#comment-15200126 ] Esther Kundin commented on HBASE-8458: -- I was testing with HDP HBase 0.98. > Support for batch version of checkAndPut() and checkAndDelete() > --- > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: Improvement > Components: Client, regionserver >Affects Versions: 0.95.0 >Reporter: Hari Mankude > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndPut() instead of a put(). However, doing a checkAndPut() rpc for > every key update is non optimal. It would be good to have a batch version of > checkAndPut() similar to batch put(). The client can partition the keys on > region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndPut() > Batch version of checkAndDelete() is a similar requirement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12940) Expose listPeerConfigs and getPeerConfig to the HBase shell
[ https://issues.apache.org/jira/browse/HBASE-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12940: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Expose listPeerConfigs and getPeerConfig to the HBase shell > --- > > Key: HBASE-12940 > URL: https://issues.apache.org/jira/browse/HBASE-12940 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Kevin Risden >Assignee: Geoffrey Jacoby > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-12940-v1.patch, HBASE-12940.patch > > > In HBASE-12867 found that listPeerConfigs and getPeerConfig from > ReplicationAdmin are not exposed to the HBase shell. This makes looking at > details for custom replication endpoints and testing of add_peer from > HBASE-12867 impossible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15326) NPE in TestHRegion.testBatchPut_whileNoRowLocksHeld
[
https://issues.apache.org/jira/browse/HBASE-15326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-15326.
---
Resolution: Duplicate
Yeah, duplicate.
> NPE in TestHRegion.testBatchPut_whileNoRowLocksHeld
> ---
>
> Key: HBASE-15326
> URL: https://issues.apache.org/jira/browse/HBASE-15326
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.metrics2.lib.MutableHistogram.updateSnapshotMetrics(MutableHistogram.java:72)
> at
> org.apache.hadoop.metrics2.lib.MutableRangeHistogram.snapshot(MutableRangeHistogram.java:59)
> at
> org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry.snapshot(DynamicMetricsRegistry.java:391)
> at
> org.apache.hadoop.hbase.metrics.BaseSourceImpl.getMetrics(BaseSourceImpl.java:146)
> at
> org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.getMetrics(MetricsAssertHelperImpl.java:243)
> at
> org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.getCounter(MetricsAssertHelperImpl.java:201)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testBatchPut_whileNoRowLocksHeld(TestHRegion.java:1498)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
> {code}
> It seems to be introduced after HBASE-15222, [~eclark]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15390) Unnecessary MetaCache evictions cause elevated number of requests to meta
[ https://issues.apache.org/jira/browse/HBASE-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197815#comment-15197815 ] Sean Busbey commented on HBASE-15390: - precommit only tests modules that have changed. if you'd like to have the tests in other modules run, you'll have to touch something in that module. > Unnecessary MetaCache evictions cause elevated number of requests to meta > - > > Key: HBASE-15390 > URL: https://issues.apache.org/jira/browse/HBASE-15390 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0, 1.2.1 > > Attachments: HBASE-15390.branch-1.3.v1.patch > > > - In ClientExceptionsUtil#findException() we don't unwrap correctly all > remote exceptions but a few selected ones; >more specifically, we don't unwrap correctly all remote exceptions where > getCause() exception doesn't have a constructor (String message) > - In AsyncProcess#receiveGlobalFailure() we don't check for > #isMetaClearingException() if tableName is null > - We don't have metric on client side to track number of cache drops -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15389) Write out multiple files when compaction
[ https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clara Xiong updated HBASE-15389: Assignee: Duo Zhang > Write out multiple files when compaction > > > Key: HBASE-15389 > URL: https://issues.apache.org/jira/browse/HBASE-15389 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Affects Versions: 2.0.0, 1.3.0, 0.98.19 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: HBASE-15389-0.98.patch, HBASE-15389-uc.patch, > HBASE-15389-v1.patch, HBASE-15389-v10.patch, HBASE-15389-v11.patch, > HBASE-15389-v12.patch, HBASE-15389-v2.patch, HBASE-15389-v3.patch, > HBASE-15389-v4.patch, HBASE-15389-v5.patch, HBASE-15389-v6.patch, > HBASE-15389-v7.patch, HBASE-15389-v8.patch, HBASE-15389-v9.patch, > HBASE-15389.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15406) Split / merge switch left disabled after early termination of hbck
[
https://issues.apache.org/jira/browse/HBASE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201103#comment-15201103
]
Hadoop QA commented on HBASE-15406:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 23s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 48s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 5s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 10s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 8s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 10m 41s
{color} | {color:red} hbase-server: patch generated 1 new + 87 unchanged - 1
fixed = 88 total (was 88) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
81m 24s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 101m 51s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 246m 29s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
|
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.TestCorruptedRegionStoreFile |
| | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.regionserver.TestCompaction |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.TestHBaseTestingUtility |
| | org.apache.hadoop.hbase.TestFullLogReconstruction |
| |
[jira] [Updated] (HBASE-15478) add comments to FSHLog explaining why syncRunnerIndex won't overflow
[ https://issues.apache.org/jira/browse/HBASE-15478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15478: Status: Patch Available (was: Open) > add comments to FSHLog explaining why syncRunnerIndex won't overflow > > > Key: HBASE-15478 > URL: https://issues.apache.org/jira/browse/HBASE-15478 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 1.1.3, 1.0.3, 1.2.0, 2.0.0, 1.3.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15478.1.patch > > > A comment near the fix for HBASE-14759 will make the code easier for folks to > follow later down the road. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15064) BufferUnderflowException after last Cell fetched from an HFile Block served from L2 offheap cache
[
https://issues.apache.org/jira/browse/HBASE-15064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198794#comment-15198794
]
deepankar commented on HBASE-15064:
---
pressed add too early i mean like this
{code}
public MultiByteBuff limit(int limit) {
this.limit = limit;
// Normally the limit will try to limit within the last BB item
int limitedIndexBegin = this.itemBeginPos[this.limitedItemIndex];
if (limit > limitedIndexBegin && limit <
this.itemBeginPos[this.limitedItemIndex + 1]) {
this.items[this.limitedItemIndex].limit(limit - limitedIndexBegin);
return this;
}
int itemIndex = getItemIndex(limit - 1);
{code}
> BufferUnderflowException after last Cell fetched from an HFile Block served
> from L2 offheap cache
> -
>
> Key: HBASE-15064
> URL: https://issues.apache.org/jira/browse/HBASE-15064
> Project: HBase
> Issue Type: Bug
> Components: io
>Affects Versions: 2.0.0
>Reporter: deepankar
>Assignee: Anoop Sam John
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15064.patch
>
>
> While running the newer patches on our production system, I saw this error
> come couple of times
> {noformat}
> ipc.RpcServer: Unexpected throwable object
> 2016-01-01 16:42:56,090 ERROR
> [B.defaultRpcServer.handler=20,queue=20,port=60020] ipc.RpcServer: Unexpected
> throwable object
> java.nio.BufferUnderflowException
> at java.nio.Buffer.nextGetIndex(Buffer.java:500)
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
> at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:494)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decode(FastDiffDeltaEncoder.java:402)
>
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decodeNext(FastDiffDeltaEncoder.java:517)
>
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.next(BufferedDataBlockEncoder.java:815)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:138)
> {noformat}
> Looking at the get code
> {code}
> if (this.curItem.remaining() == 0) {
> if (items.length - 1 == this.curItemIndex) {
> // means cur item is the last one and we wont be able to read a long.
> Throw exception
> throw new BufferUnderflowException();
> }
> this.curItemIndex++;
> this.curItem = this.items[this.curItemIndex];
> }
> return this.curItem.get();
> {code}
> Can the new currentItem have zero elements (position == limit), does it make
> sense to change the {{if}} to {{while}} ? {{while (this.curItem.remaining()
> == 0)}}. This logic is repeated may make sense abstract to a new function if
> we plan to change to {{if}} to {{while}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15478) add comments to FSHLog explaining why syncRunnerIndex won't overflow
Sean Busbey created HBASE-15478: --- Summary: add comments to FSHLog explaining why syncRunnerIndex won't overflow Key: HBASE-15478 URL: https://issues.apache.org/jira/browse/HBASE-15478 Project: HBase Issue Type: Improvement Components: wal Affects Versions: 1.1.3, 1.0.3, 1.2.0, 2.0.0, 1.3.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 2.0.0 A comment near the fix for HBASE-14759 will make the code easier for folks to follow later down the road. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15479) No more garbage or beware of autoboxing
[ https://issues.apache.org/jira/browse/HBASE-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203063#comment-15203063 ] Hudson commented on HBASE-15479: SUCCESS: Integrated in HBase-1.3-IT #567 (See [https://builds.apache.org/job/HBase-1.3-IT/567/]) HBASE-15479 No more garbage or beware of autoboxing (Vladimir Rodionov) (tedyu: rev 050c13e83f8483c5a44d543a842eb51e35644a9f) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java > No more garbage or beware of autoboxing > --- > > Key: HBASE-15479 > URL: https://issues.apache.org/jira/browse/HBASE-15479 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-15479-v1.patch, HBASE-15479-v2.patch > > > Quick journey with JMC in a profile mode revealed very interesting and > unexpected heap polluter on a client side. Patch will shortly follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15451) Remove unnecessary wait in MVCC
[
https://issues.apache.org/jira/browse/HBASE-15451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203062#comment-15203062
]
Hudson commented on HBASE-15451:
SUCCESS: Integrated in HBase-1.3-IT #567 (See
[https://builds.apache.org/job/HBase-1.3-IT/567/])
HBASE-15451 Remove unnecessary wait in MVCC (liyu: rev
463374ecfa770645c7098b4648cb805933eb7aab)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConcurrencyControl.java
> Remove unnecessary wait in MVCC
> ---
>
> Key: HBASE-15451
> URL: https://issues.apache.org/jira/browse/HBASE-15451
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15451.patch, HBASE-15451.test
>
>
> Currently the return value of MVCC#complete indicates whether readPoint
> already advanced over write number of the given WriteEntry:
> {code}
> return readPoint.get() >= writeEntry.getWriteNumber();
> {code}
> While in MVCC#checkAndWait we never take usage of this but always call
> waitForRead which will acquire and release lock on {{readWaiters}} and cause
> additional context switch. This JIRA will improve this logic and remove the
> unnecessary wait.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14963) Remove use of Guava Stopwatch from HBase client code
[ https://issues.apache.org/jira/browse/HBASE-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203064#comment-15203064 ] Hudson commented on HBASE-14963: SUCCESS: Integrated in HBase-1.3-IT #567 (See [https://builds.apache.org/job/HBase-1.3-IT/567/]) HBASE-14963 Remove use of Guava Stopwatch from HBase client code (jerryjch: rev 8151503fa075bb7388019770bccb28703d23935d) * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaTableLocator.java > Remove use of Guava Stopwatch from HBase client code > > > Key: HBASE-14963 > URL: https://issues.apache.org/jira/browse/HBASE-14963 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Devaraj Das >Assignee: Devaraj Das > Labels: needs_releasenote > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14963-branch-1.patch, no-stopwatch.txt > > > We ran into an issue where an application bundled its own Guava (and that > happened to be in the classpath first) and HBase's MetaTableLocator threw an > exception due to the fact that Stopwatch's constructor wasn't compatible... > Might be better to not depend on Stopwatch at all in MetaTableLocator since > the functionality is easily doable without. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14963) Remove use of Guava Stopwatch from HBase client code
[ https://issues.apache.org/jira/browse/HBASE-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203061#comment-15203061 ] Hudson commented on HBASE-14963: SUCCESS: Integrated in HBase-1.3 #611 (See [https://builds.apache.org/job/HBase-1.3/611/]) HBASE-14963 Remove use of Guava Stopwatch from HBase client code (jerryjch: rev c2cd23fee98f34b846b4c43a9141f9bab598ae6a) * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaTableLocator.java > Remove use of Guava Stopwatch from HBase client code > > > Key: HBASE-14963 > URL: https://issues.apache.org/jira/browse/HBASE-14963 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Devaraj Das >Assignee: Devaraj Das > Labels: needs_releasenote > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14963-branch-1.patch, no-stopwatch.txt > > > We ran into an issue where an application bundled its own Guava (and that > happened to be in the classpath first) and HBase's MetaTableLocator threw an > exception due to the fact that Stopwatch's constructor wasn't compatible... > Might be better to not depend on Stopwatch at all in MetaTableLocator since > the functionality is easily doable without. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15479) No more garbage or beware of autoboxing
[ https://issues.apache.org/jira/browse/HBASE-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203060#comment-15203060 ] Hudson commented on HBASE-15479: FAILURE: Integrated in HBase-Trunk_matrix #791 (See [https://builds.apache.org/job/HBase-Trunk_matrix/791/]) HBASE-15479 No more garbage or beware of autoboxing (Vladimir Rodionov) (tedyu: rev dc6cd0bb7412b8746e28886d42f6708b0c5f2322) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java > No more garbage or beware of autoboxing > --- > > Key: HBASE-15479 > URL: https://issues.apache.org/jira/browse/HBASE-15479 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-15479-v1.patch, HBASE-15479-v2.patch > > > Quick journey with JMC in a profile mode revealed very interesting and > unexpected heap polluter on a client side. Patch will shortly follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15489) Improve handling of hbase.rpc.protection configuration mismatch when using replication
Esteban Gutierrez created HBASE-15489:
-
Summary: Improve handling of hbase.rpc.protection configuration
mismatch when using replication
Key: HBASE-15489
URL: https://issues.apache.org/jira/browse/HBASE-15489
Project: HBase
Issue Type: Improvement
Reporter: Esteban Gutierrez
This probably should be a sub-task for major a major revamp of how we report
our replication metrics in HBase in the UI.
After switching {{hbase.rpc.protection}} to {{privacy}} in one cluster I didn't
noticed immediately there was a mismatch across my other clusters which caused
replication to stop. Ideally if this happens we should have a better log
message and show this mismatch in the RegionServer and Master UIs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14945) Backport HBASE-13153 (Bulk loaded HFile replication) to 0.98
[ https://issues.apache.org/jira/browse/HBASE-14945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14945: --- Fix Version/s: (was: 0.98.18) 0.98.19 > Backport HBASE-13153 (Bulk loaded HFile replication) to 0.98 > > > Key: HBASE-14945 > URL: https://issues.apache.org/jira/browse/HBASE-14945 > Project: HBase > Issue Type: New Feature >Reporter: Andrew Purtell > Fix For: 0.98.19 > > > HBASE-13153 (Bulk loaded HFile replication) looks like a good candidate to > backport to 0.98. It would address an important functional gap in replication > (bulk loads don't replicate) for 0.98 users, is a new feature toggled with a > new configuration parameter that is by default off, and does not change any > interfaces not marked as Private. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14857) add region server groups in master UI
[ https://issues.apache.org/jira/browse/HBASE-14857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-14857: Labels: hbase-6721 (was: ) > add region server groups in master UI > - > > Key: HBASE-14857 > URL: https://issues.apache.org/jira/browse/HBASE-14857 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu > Labels: hbase-6721 > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14030) HBase Backup/Restore Phase 1
[ https://issues.apache.org/jira/browse/HBASE-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14030: --- Attachment: HBASE-14030.v42.patch Patch v42 drops parameters backupRootPath and backupId from non-static HBackupFileSystem methods (Thanks to Vlad's review) > HBase Backup/Restore Phase 1 > > > Key: HBASE-14030 > URL: https://issues.apache.org/jira/browse/HBASE-14030 > Project: HBase > Issue Type: Umbrella >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-14030-v0.patch, HBASE-14030-v1.patch, > HBASE-14030-v10.patch, HBASE-14030-v11.patch, HBASE-14030-v12.patch, > HBASE-14030-v13.patch, HBASE-14030-v14.patch, HBASE-14030-v15.patch, > HBASE-14030-v17.patch, HBASE-14030-v18.patch, HBASE-14030-v2.patch, > HBASE-14030-v20.patch, HBASE-14030-v21.patch, HBASE-14030-v22.patch, > HBASE-14030-v23.patch, HBASE-14030-v24.patch, HBASE-14030-v25.patch, > HBASE-14030-v26.patch, HBASE-14030-v27.patch, HBASE-14030-v28.patch, > HBASE-14030-v3.patch, HBASE-14030-v30.patch, HBASE-14030-v35.patch, > HBASE-14030-v37.patch, HBASE-14030-v4.patch, HBASE-14030-v5.patch, > HBASE-14030-v6.patch, HBASE-14030-v7.patch, HBASE-14030-v8.patch, > HBASE-14030.v38.patch, HBASE-14030.v39.patch, HBASE-14030.v40.patch, > HBASE-14030.v41.patch, HBASE-14030.v42.patch, hbase-14030_v36.patch > > > This is the umbrella ticket for Backup/Restore Phase 1. See HBASE-7912 design > doc for the phase description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14963) Remove use of Guava Stopwatch from HBase client code
[ https://issues.apache.org/jira/browse/HBASE-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203055#comment-15203055 ] Hudson commented on HBASE-14963: SUCCESS: Integrated in HBase-1.4 #34 (See [https://builds.apache.org/job/HBase-1.4/34/]) HBASE-14963 Remove use of Guava Stopwatch from HBase client code (jerryjch: rev 8151503fa075bb7388019770bccb28703d23935d) * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaTableLocator.java > Remove use of Guava Stopwatch from HBase client code > > > Key: HBASE-14963 > URL: https://issues.apache.org/jira/browse/HBASE-14963 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Devaraj Das >Assignee: Devaraj Das > Labels: needs_releasenote > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-14963-branch-1.patch, no-stopwatch.txt > > > We ran into an issue where an application bundled its own Guava (and that > happened to be in the classpath first) and HBase's MetaTableLocator threw an > exception due to the fact that Stopwatch's constructor wasn't compatible... > Might be better to not depend on Stopwatch at all in MetaTableLocator since > the functionality is easily doable without. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15475) Allow TimestampsFilter to provide a seek hint
[ https://issues.apache.org/jira/browse/HBASE-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15475: -- Attachment: HBASE-15475-v3.patch A couple of code comments from phabricator, and some extra code comments. > Allow TimestampsFilter to provide a seek hint > - > > Key: HBASE-15475 > URL: https://issues.apache.org/jira/browse/HBASE-15475 > Project: HBase > Issue Type: Improvement > Components: Client, Filters, regionserver >Affects Versions: 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15475-v1.patch, HBASE-15475-v2.patch, > HBASE-15475-v3.patch, HBASE-15475.patch > > > If a user wants to read specific timestamps currently it's a very linear > scan. This is so that all deletes can be respected. However if the user > doesn't care about deletes it can dramatically speed up the request to seek. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15441) Fix WAL splitting when region has moved multiple times
[ https://issues.apache.org/jira/browse/HBASE-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15441: -- Resolution: Fixed Fix Version/s: 1.4.0 1.2.1 Status: Resolved (was: Patch Available) > Fix WAL splitting when region has moved multiple times > -- > > Key: HBASE-15441 > URL: https://issues.apache.org/jira/browse/HBASE-15441 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.0, 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Blocker > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.4.0 > > Attachments: HBASE-15441-v1.patch, HBASE-15441-v2.patch, > HBASE-15441-v3.patch, HBASE-15441-v4.patch, HBASE-15441-v5.patch, > HBASE-15441.patch > > > Currently WAL splitting is broken when a region has been opened multiple > times in recent minutes. > Region open and region close write event markers to the wal. These markers > should have the sequence id in them. However it is currently getting 1. That > means that if a region has moved multiple times in the last few mins then > multiple split log workers will try and create the recovered edits file for > sequence id 1. One of the workers will fail and on failing they will delete > the recovered edits. Causing all split wal attempts to fail. > We need to: > # make sure that close get the correct sequence id for open. > # Filter all region events from recovered edits > It appears that the close event with a sequence id of one is coming from > region warm up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15398) Cells loss or disorder when using family essential filter and partial scanning protocol
[ https://issues.apache.org/jira/browse/HBASE-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197333#comment-15197333 ] stack commented on HBASE-15398: --- bq. Is it a good idea that adding a new flag in protocol of Result? If we have this flag and it is true, we sort all cells when createCompleteResult, otherwise we needn't. You think it not possible for server to deliver results sorted always? You saw the comment above where it is suggested that adding to sorting to Result is a break with what we have assumed up to this and the question that the essential filter is broken in the manner in which it does its work making two heaps where everywhere else we do one so the results come out ordered? Is your patch trying to work around this fundamental flaw? Thank you [~yangzhe1991] > Cells loss or disorder when using family essential filter and partial > scanning protocol > --- > > Key: HBASE-15398 > URL: https://issues.apache.org/jira/browse/HBASE-15398 > Project: HBase > Issue Type: Bug > Components: dataloss, Scanners >Affects Versions: 1.2.0, 1.1.3 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Critical > Attachments: 15398-test.txt, HBASE-15398-v2.patch, > HBASE-15398-v3.patch, HBASE-15398-v4.patch, HBASE-15398.v1.txt > > > In RegionScannerImpl, we have two heaps, storeHeap and joinedHeap. If we have > a filter and it doesn't apply to all cf, the stores whose families needn't be > filtered will be in joinedHeap. We scan storeHeap first, then joinedHeap, > and merge the results and sort and return to client. We need sort because the > order of Cell is rowkey/cf/cq/ts and a smaller cf may be in the joinedHeap. > However, after HBASE-11544 we may transfer partial results when we get > SIZE_LIMIT_REACHED_MID_ROW or other similar states. We may return a larger cf > first because it is in storeHeap and then a smaller cf because it is in > joinedHeap. Server won't hold all cells in a row and client doesn't have a > sorting logic. The order of cf in Result for user is wrong. > And a more critical bug is, if we get a LIMIT_REACHED_MID_ROW on the last > cell of a row in storeHeap, we will break scanning in RegionScannerImpl and > in populateResult we will change the state to SIZE_LIMIT_REACHED because next > peeked cell is next row. But this is only the last cell of one and we have > two... And SIZE_LIMIT_REACHED means this Result is not partial (by > ScannerContext.partialResultFormed), client will see it and merge them and > return to user with losing data of joinedHeap. On next scan we will read next > row of storeHeap and joinedHeap is forgotten and never be read... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke reassigned HBASE-12148: - Assignee: Walter Koetke (was: John Leach) > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.18 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot 2014-10-01 at 3.39.46 PM.png, > Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15405) Fix PE logging and wrong defaults in help message
[
https://issues.apache.org/jira/browse/HBASE-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198042#comment-15198042
]
Hadoop QA commented on HBASE-15405:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
46m 44s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 96m 47s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 111m 32s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
47s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 295m 20s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0 Failed junit tests |
hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| JDK v1.8.0 Timed out junit tests |
org.apache.hadoop.hbase.regionserver.TestCorruptedRegionStoreFile |
| | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.regionserver.TestCompaction |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.wal.TestWALFiltering |
| | org.apache.hadoop.hbase.coprocessor.TestRegionServerObserver |
| |
[jira] [Updated] (HBASE-14856) document region servers groups in the book
[ https://issues.apache.org/jira/browse/HBASE-14856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-14856: Labels: hbase-6721 (was: ) > document region servers groups in the book > -- > > Key: HBASE-14856 > URL: https://issues.apache.org/jira/browse/HBASE-14856 > Project: HBase > Issue Type: Task >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15477) Do not save 'next block header' when we cache hfileblocks
[ https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202361#comment-15202361 ] stack commented on HBASE-15477: --- RB won't let me update. The difference from RB is adding to FileContextBuilder a constructor that takes a FileContext so I can make new context based off a previous changing just the few items I want to change rather than have to copy all. This fixed unit tests (was using the checksum type from default filecontext rather than the one gotten from the block read from hdfs/cache). > Do not save 'next block header' when we cache hfileblocks > - > > Key: HBASE-15477 > URL: https://issues.apache.org/jira/browse/HBASE-15477 > Project: HBase > Issue Type: Sub-task > Components: BlockCache, Performance >Reporter: stack >Assignee: stack > Attachments: 15366v4.patch, 15477.patch, 15477v2.patch, 15477v3.patch > > > When we read from HDFS, we overread to pick up the next blocks header. > Doing this saves a seek as we move through the hfile; we save having to > do an explicit seek just to read the block header every time we need to > read the body. We used to read in the next header as part of the > current blocks buffer. This buffer was then what got persisted to > blockcache; so we were over-persisting wrtiting out our block plus the > next blocks' header (overpersisting 33 bytes). Parse of HFileBlock > complicated by this extra tail. Fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15382) Expose regionserver metadata (ie groups, tables, servers) via JMX
[ https://issues.apache.org/jira/browse/HBASE-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-15382: Issue Type: New Feature (was: Sub-task) Parent: (was: HBASE-6721) > Expose regionserver metadata (ie groups, tables, servers) via JMX > - > > Key: HBASE-15382 > URL: https://issues.apache.org/jira/browse/HBASE-15382 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > > This feature was removed from the base patch. So we can come up with a proper > interface for other components to use as well, as directly accessing jmx is > not an option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15406) Split / merge switch left disabled after early termination of hbck
[
https://issues.apache.org/jira/browse/HBASE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201230#comment-15201230
]
Hadoop QA commented on HBASE-15406:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 6s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 34s
{color} | {color:red} hbase-server: patch generated 1 new + 87 unchanged - 1
fixed = 88 total (was 88) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 5s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 59s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 130m 19s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.client.TestReplicasClient |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface |
| | org.apache.hadoop.hbase.mapreduce.TestWALPlayer |
| | org.apache.hadoop.hbase.coprocessor.TestHTableWrapper |
| | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.mapreduce.TestTableInputFormat |
| | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.client.TestMultipleTimestamps |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| |
[jira] [Updated] (HBASE-15456) CreateTableProcedure/ModifyTableProcedure needs to fail when there is no family in table descriptor
[
https://issues.apache.org/jira/browse/HBASE-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huaxiang sun updated HBASE-15456:
-
Attachment: HBASE-15456-v004.patch
The latest diff to address thrift unitest failure. The only difference is to
add column families for create tableB, which addresses all thrift unittest
failure. I got a green hbase unittest run locally.
> CreateTableProcedure/ModifyTableProcedure needs to fail when there is no
> family in table descriptor
> ---
>
> Key: HBASE-15456
> URL: https://issues.apache.org/jira/browse/HBASE-15456
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15456-001_branch-1.patch,
> HBASE-15456-branch-1.patch, HBASE-15456-branch-1.patch,
> HBASE-15456-v001.patch, HBASE-15456-v002.patch, HBASE-15456-v002.patch,
> HBASE-15456-v003.patch, HBASE-15456-v004.patch
>
>
> If there is only one family in the table, DeleteColumnFamilyProcedure will
> fail.
> Currently, when hbase.table.sanity.checks is set to false, hbase master logs
> a warning and CreateTableProcedure/ModifyTableProcedure will succeed.
> This behavior is not consistent with DeleteColumnFamilyProcedure's.
> Another point, before HBASE-13145, PeriodicMemstoreFlusher will run into the
> following exception. lastStoreFlushTimeMap is populated for families, if
> there is no family in the table, there is no entry in lastStoreFlushTimeMap.
> {code}
> 16/02/01 11:14:26 ERROR regionserver.HRegionServer$PeriodicMemstoreFlusher:
> Caught exception
> java.util.NoSuchElementException
> at
> java.util.concurrent.ConcurrentHashMap$HashIterator.nextEntry(ConcurrentHashMap.java:1354)
>
> at
> java.util.concurrent.ConcurrentHashMap$ValueIterator.next(ConcurrentHashMap.java:1384)
>
> at java.util.Collections.min(Collections.java:628)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getEarliestFlushTimeForAllStores(HRegion.java:1572)
>
> at
> org.apache.hadoop.hbase.regionserver.HRegion.shouldFlush(HRegion.java:1904)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher.chore(HRegionServer.java:1509)
>
> at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15406) Split / merge switch left disabled after early termination of hbck
[
https://issues.apache.org/jira/browse/HBASE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201516#comment-15201516
]
Heng Chen commented on HBASE-15406:
---
{quote}
Can you identify the users in steps 1 and 3 above ?
Are they different users ?
{quote}
Is there any difference whether they are the same user? we can't protect our
switch be changed without lock.
{quote}
There is lock mechanism in place to prevent two hbck instances from running at
the same time. You don't need to add extra code covering this.
{quote}
But as you said above, if the hbck can't run at the same time, current patch
v1 will has no problem, right?
> Split / merge switch left disabled after early termination of hbck
> --
>
> Key: HBASE-15406
> URL: https://issues.apache.org/jira/browse/HBASE-15406
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15406.patch, HBASE-15406.v1.patch,
> HBASE-15406_v1.patch, test.patch, wip.patch
>
>
> This was what I did on cluster with 1.4.0-SNAPSHOT built Thursday:
> Run 'hbase hbck -disableSplitAndMerge' on gateway node of the cluster
> Terminate hbck early
> Enter hbase shell where I observed:
> {code}
> hbase(main):001:0> splitormerge_enabled 'SPLIT'
> false
> 0 row(s) in 0.3280 seconds
> hbase(main):002:0> splitormerge_enabled 'MERGE'
> false
> 0 row(s) in 0.0070 seconds
> {code}
> Expectation is that the split / merge switches should be restored to default
> value after hbck exits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15390) Unnecessary MetaCache evictions cause elevated number of requests to meta
[ https://issues.apache.org/jira/browse/HBASE-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198047#comment-15198047 ] Sean Busbey commented on HBASE-15390: - yep! everything except the per-branch post-commit checks. > Unnecessary MetaCache evictions cause elevated number of requests to meta > - > > Key: HBASE-15390 > URL: https://issues.apache.org/jira/browse/HBASE-15390 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0, 1.2.1 > > Attachments: HBASE-15390.branch-1.3.v1.patch, HBASE-15390.v1.patch > > > - In ClientExceptionsUtil#findException() we don't unwrap correctly all > remote exceptions but a few selected ones; >more specifically, we don't unwrap correctly all remote exceptions where > getCause() exception doesn't have a constructor (String message) > - In AsyncProcess#receiveGlobalFailure() we don't check for > #isMetaClearingException() if tableName is null > - We don't have metric on client side to track number of cache drops -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15474) Exception in HConnectionImplementation's constructor cause Zookeeper connnections leak
[
https://issues.apache.org/jira/browse/HBASE-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203038#comment-15203038
]
Allan Yang commented on HBASE-15474:
still not apply
> Exception in HConnectionImplementation's constructor cause Zookeeper
> connnections leak
> ---
>
> Key: HBASE-15474
> URL: https://issues.apache.org/jira/browse/HBASE-15474
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Allan Yang
>Assignee: Allan Yang
> Labels: patch
> Fix For: 1.1.0
>
> Attachments: HBASE-15474-branch-1.v3.patch, HBASE-15474v2.patch
>
>
> HConnectionImplementation creates a ZooKeeperKeepAliveConnection during
> construction, but if the constructor throw a exception, the zookeeper
> connection is not properly closed.
> {code}
> HConnectionImplementation(Configuration conf, boolean managed,
> ExecutorService pool, User user) throws IOException {
> this(conf);
> this.user = user;
> this.batchPool = pool;
> this.managed = managed;
> this.registry = setupRegistry();
> retrieveClusterId(); //here is the zookeeper connection created
> this.rpcClient = RpcClientFactory.createClient(this.conf,
> this.clusterId);// In our case, the exception happens here, so the zookeeper
> connection never closes
> this.rpcControllerFactory = RpcControllerFactory.instantiate(conf);
> ..
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15474) Exception in HConnectionImplementation's constructor cause Zookeeper connnections leak
[
https://issues.apache.org/jira/browse/HBASE-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203037#comment-15203037
]
Hadoop QA commented on HBASE-15474:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HBASE-15474 does not apply to branch-1. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12794381/HBASE-15474-branch-1.v3.patch
|
| JIRA Issue | HBASE-15474 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1091/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> Exception in HConnectionImplementation's constructor cause Zookeeper
> connnections leak
> ---
>
> Key: HBASE-15474
> URL: https://issues.apache.org/jira/browse/HBASE-15474
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Allan Yang
>Assignee: Allan Yang
> Labels: patch
> Fix For: 1.1.0
>
> Attachments: HBASE-15474-branch-1.v3.patch, HBASE-15474v2.patch
>
>
> HConnectionImplementation creates a ZooKeeperKeepAliveConnection during
> construction, but if the constructor throw a exception, the zookeeper
> connection is not properly closed.
> {code}
> HConnectionImplementation(Configuration conf, boolean managed,
> ExecutorService pool, User user) throws IOException {
> this(conf);
> this.user = user;
> this.batchPool = pool;
> this.managed = managed;
> this.registry = setupRegistry();
> retrieveClusterId(); //here is the zookeeper connection created
> this.rpcClient = RpcClientFactory.createClient(this.conf,
> this.clusterId);// In our case, the exception happens here, so the zookeeper
> connection never closes
> this.rpcControllerFactory = RpcControllerFactory.instantiate(conf);
> ..
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15369) Handle NPE in region.jsp
[
https://issues.apache.org/jira/browse/HBASE-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199388#comment-15199388
]
Hadoop QA commented on HBASE-15369:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 7 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 96m 39s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 130m 35s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12793940/HBASE-15369_v0.patch |
| JIRA Issue | HBASE-15369 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a2c99b1 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/1043/artifact/patchprocess/whitespace-tabs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/1043/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1043/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> Handle NPE in region.jsp
>
>
> Key: HBASE-15369
> URL: https://issues.apache.org/jira/browse/HBASE-15369
> Project: HBase
> Issue Type: Sub-task
> Components: UI
>Affects Versions: 2.0.0
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15369_v0.patch
>
>
> If empty or non-existing region name parameter is passed to "region.jsp
> ?name=" it will cause 500 NPE error. It is not big deal but looks ugly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15470) Add a setting for Priority queue length
[ https://issues.apache.org/jira/browse/HBASE-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15470: -- Affects Version/s: 1.2.0 > Add a setting for Priority queue length > --- > > Key: HBASE-15470 > URL: https://issues.apache.org/jira/browse/HBASE-15470 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, Region Assignment >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > > Meta can have very different request rates than any other region. We > shouldn't use the same call queue length. > For example: > If a normal request takes 100ms ( long scan or large get ) > but a call to meta is all in memory so it takes < 1ms > So a call queue length of 100 represents multiple seconds of work for normal > requests, but less than a second for meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15474) Exception in HConnectionImplementation's constructor cause Zookeeper connnections leak
[
https://issues.apache.org/jira/browse/HBASE-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-15474:
---
Attachment: HBASE-15474-branch-1.v3.patch
patch for branch 1
> Exception in HConnectionImplementation's constructor cause Zookeeper
> connnections leak
> ---
>
> Key: HBASE-15474
> URL: https://issues.apache.org/jira/browse/HBASE-15474
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Allan Yang
>Assignee: Allan Yang
> Labels: patch
> Fix For: 1.1.0
>
> Attachments: HBASE-15474-branch-1.v3.patch, HBASE-15474v2.patch
>
>
> HConnectionImplementation creates a ZooKeeperKeepAliveConnection during
> construction, but if the constructor throw a exception, the zookeeper
> connection is not properly closed.
> {code}
> HConnectionImplementation(Configuration conf, boolean managed,
> ExecutorService pool, User user) throws IOException {
> this(conf);
> this.user = user;
> this.batchPool = pool;
> this.managed = managed;
> this.registry = setupRegistry();
> retrieveClusterId(); //here is the zookeeper connection created
> this.rpcClient = RpcClientFactory.createClient(this.conf,
> this.clusterId);// In our case, the exception happens here, so the zookeeper
> connection never closes
> this.rpcControllerFactory = RpcControllerFactory.instantiate(conf);
> ..
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15470) Add a setting for Priority queue length
[ https://issues.apache.org/jira/browse/HBASE-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199129#comment-15199129 ] Hudson commented on HBASE-15470: ABORTED: Integrated in HBase-1.3 #606 (See [https://builds.apache.org/job/HBase-1.3/606/]) HBASE-15470 Add a setting for Priority queue length (eclark: rev 9c13e0daffcf489f91755b1f93aff0fa23b51aff) * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoRpcScheduler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcScheduler.java > Add a setting for Priority queue length > --- > > Key: HBASE-15470 > URL: https://issues.apache.org/jira/browse/HBASE-15470 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, Region Assignment >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15470.patch > > > Meta can have very different request rates than any other region. We > shouldn't use the same call queue length. > For example: > If a normal request takes 100ms ( long scan or large get ) > but a call to meta is all in memory so it takes < 1ms > So a call queue length of 100 represents multiple seconds of work for normal > requests, but less than a second for meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Leach updated HBASE-12148: --- Attachment: TimeRangeTracker.tiff Here are the blocked CPU cycles via JProfiler... > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148-V3.patch, HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot > 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png, > TimeRangeTracker.tiff > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14985) TimeRange constructors should set allTime when appropriate
[ https://issues.apache.org/jira/browse/HBASE-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-14985: Affects Version/s: (was: 0.98.16.1) (was: 1.1.2) 1.2.0 1.1.3 0.98.17 Status: Patch Available (was: Open) > TimeRange constructors should set allTime when appropriate > -- > > Key: HBASE-14985 > URL: https://issues.apache.org/jira/browse/HBASE-14985 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.98.17, 1.1.3, 1.2.0 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Minor > Attachments: HBASE-14985-v1.patch, HBASE-14985.patch > > > The default TimeRange constructor creates a range from 0 to Long.MAX_VALUE > and sets an allTime flag to true. This flag allows some performance > optimizations when comparing or using TimeRanges. > This flag is not set, however, if you call "new TimeRange(0L)" or "new > TimeRange(0L, Long.MAX_VALUE)", even though both of these create a logically > equivalent TimeRange to "new TimeRange()". Since TimeRanges are immutable and > detecting this condition is trivial, we should set the flag automatically in > the explicit constructors when appropriate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15390) Unnecessary MetaCache evictions cause elevated number of requests to meta
[ https://issues.apache.org/jira/browse/HBASE-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15390: Status: Open (was: Patch Available) > Unnecessary MetaCache evictions cause elevated number of requests to meta > - > > Key: HBASE-15390 > URL: https://issues.apache.org/jira/browse/HBASE-15390 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0, 1.2.1 > > Attachments: HBASE-15390.branch-1.3.v1.patch, HBASE-15390.v1.patch > > > - In ClientExceptionsUtil#findException() we don't unwrap correctly all > remote exceptions but a few selected ones; >more specifically, we don't unwrap correctly all remote exceptions where > getCause() exception doesn't have a constructor (String message) > - In AsyncProcess#receiveGlobalFailure() we don't check for > #isMetaClearingException() if tableName is null > - We don't have metric on client side to track number of cache drops -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15226) HBase Backup/Restore Phase 3: Add FSM to Backup state
[ https://issues.apache.org/jira/browse/HBASE-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200120#comment-15200120 ] Vladimir Rodionov commented on HBASE-15226: --- Moved to phase 3 > HBase Backup/Restore Phase 3: Add FSM to Backup state > - > > Key: HBASE-15226 > URL: https://issues.apache.org/jira/browse/HBASE-15226 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > > Final State Machine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15390) Unnecessary MetaCache evictions cause elevated number of requests to meta
[ https://issues.apache.org/jira/browse/HBASE-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197754#comment-15197754 ] Mikhail Antonov commented on HBASE-15390: - [~busbey] it seems like the build is marked as failed because of checkstyle errors before most of the tests have a chance to run? > Unnecessary MetaCache evictions cause elevated number of requests to meta > - > > Key: HBASE-15390 > URL: https://issues.apache.org/jira/browse/HBASE-15390 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 1.3.0, 1.2.1 > > Attachments: HBASE-15390.branch-1.3.v1.patch > > > - In ClientExceptionsUtil#findException() we don't unwrap correctly all > remote exceptions but a few selected ones; >more specifically, we don't unwrap correctly all remote exceptions where > getCause() exception doesn't have a constructor (String message) > - In AsyncProcess#receiveGlobalFailure() we don't check for > #isMetaClearingException() if tableName is null > - We don't have metric on client side to track number of cache drops -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15469) Take snapshot by family
[
https://issues.apache.org/jira/browse/HBASE-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200847#comment-15200847
]
Jianwei Cui commented on HBASE-15469:
-
Good question! Yes, the current path will create all families when cloning or
restoring. This could be optional for user. For most cases, it is more
reasonable to only retain the requested families when taking snapshot? Users
can add other needed families after cloning or restoring. What do you think?
[~mbertozzi]. Thanks.
> Take snapshot by family
> ---
>
> Key: HBASE-15469
> URL: https://issues.apache.org/jira/browse/HBASE-15469
> Project: HBase
> Issue Type: Improvement
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15469-v1.patch
>
>
> In our production environment, there are some 'wide' tables in offline
> cluster. The 'wide' table has a number of families, different applications
> will access different families of the table through MapReduce. When some
> application starting to provide online service, we need to copy needed
> families from offline cluster to online cluster. For future write, the
> inter-cluster replication supports setting families for table, we can use it
> to copy future edits for needed families. For existed data, we can take
> snapshot of the table on offline cluster, then exploit {{ExportSnapshot}} to
> copy snapshot to online cluster and clone the snapshot. However, we can only
> take snapshot for the whole table in which many families are not needed for
> the application, this will lead unnecessary data copy. I think it is useful
> to support taking snapshot by family, so that we can only copy needed data.
> Possible solution to support such function:
> 1. Add family names field to the protobuf definition of
> {{SnapshotDescription}}
> 2. Allow to set families when taking snapshot in hbase shell, such as:
> {code}
>snapshot 'tableName', 'snapshotName', 'FamilyA', 'FamilyB', {SKIP_FLUSH =>
> true}
> {code}
> 3. Add family names to {{SnapshotDescription}} in client side
> 4. Read family names from {{SnapshotDescription}} in Master/Regionserver,
> keep only requested families when taking snapshot for region.
> Discussions and suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15456) CreateTableProcedure/ModifyTableProcedure needs to fail when there is no family in table descriptor
[
https://issues.apache.org/jira/browse/HBASE-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200793#comment-15200793
]
huaxiang sun commented on HBASE-15456:
--
[~te...@apache.org] I am looking at this. I thought I run the whole unittest
job, but it seems not. Sorry about it.
> CreateTableProcedure/ModifyTableProcedure needs to fail when there is no
> family in table descriptor
> ---
>
> Key: HBASE-15456
> URL: https://issues.apache.org/jira/browse/HBASE-15456
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15456-001_branch-1.patch,
> HBASE-15456-branch-1.patch, HBASE-15456-branch-1.patch,
> HBASE-15456-v001.patch, HBASE-15456-v002.patch, HBASE-15456-v002.patch,
> HBASE-15456-v003.patch
>
>
> If there is only one family in the table, DeleteColumnFamilyProcedure will
> fail.
> Currently, when hbase.table.sanity.checks is set to false, hbase master logs
> a warning and CreateTableProcedure/ModifyTableProcedure will succeed.
> This behavior is not consistent with DeleteColumnFamilyProcedure's.
> Another point, before HBASE-13145, PeriodicMemstoreFlusher will run into the
> following exception. lastStoreFlushTimeMap is populated for families, if
> there is no family in the table, there is no entry in lastStoreFlushTimeMap.
> {code}
> 16/02/01 11:14:26 ERROR regionserver.HRegionServer$PeriodicMemstoreFlusher:
> Caught exception
> java.util.NoSuchElementException
> at
> java.util.concurrent.ConcurrentHashMap$HashIterator.nextEntry(ConcurrentHashMap.java:1354)
>
> at
> java.util.concurrent.ConcurrentHashMap$ValueIterator.next(ConcurrentHashMap.java:1384)
>
> at java.util.Collections.min(Collections.java:628)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getEarliestFlushTimeForAllStores(HRegion.java:1572)
>
> at
> org.apache.hadoop.hbase.regionserver.HRegion.shouldFlush(HRegion.java:1904)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher.chore(HRegionServer.java:1509)
>
> at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15456) CreateTableProcedure/ModifyTableProcedure needs to fail when there is no family in table descriptor
[
https://issues.apache.org/jira/browse/HBASE-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199810#comment-15199810
]
huaxiang sun commented on HBASE-15456:
--
Thanks [~te...@apache.org], I will rename the patch and reattach.
> CreateTableProcedure/ModifyTableProcedure needs to fail when there is no
> family in table descriptor
> ---
>
> Key: HBASE-15456
> URL: https://issues.apache.org/jira/browse/HBASE-15456
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15456-001_branch-1.patch, HBASE-15456-v001.patch,
> HBASE-15456-v002.patch, HBASE-15456-v002.patch, HBASE-15456-v003.patch
>
>
> If there is only one family in the table, DeleteColumnFamilyProcedure will
> fail.
> Currently, when hbase.table.sanity.checks is set to false, hbase master logs
> a warning and CreateTableProcedure/ModifyTableProcedure will succeed.
> This behavior is not consistent with DeleteColumnFamilyProcedure's.
> Another point, before HBASE-13145, PeriodicMemstoreFlusher will run into the
> following exception. lastStoreFlushTimeMap is populated for families, if
> there is no family in the table, there is no entry in lastStoreFlushTimeMap.
> {code}
> 16/02/01 11:14:26 ERROR regionserver.HRegionServer$PeriodicMemstoreFlusher:
> Caught exception
> java.util.NoSuchElementException
> at
> java.util.concurrent.ConcurrentHashMap$HashIterator.nextEntry(ConcurrentHashMap.java:1354)
>
> at
> java.util.concurrent.ConcurrentHashMap$ValueIterator.next(ConcurrentHashMap.java:1384)
>
> at java.util.Collections.min(Collections.java:628)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getEarliestFlushTimeForAllStores(HRegion.java:1572)
>
> at
> org.apache.hadoop.hbase.regionserver.HRegion.shouldFlush(HRegion.java:1904)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher.chore(HRegionServer.java:1509)
>
> at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15430) Failed taking snapshot - Manifest proto-message too large
[
https://issues.apache.org/jira/browse/HBASE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198497#comment-15198497
]
Hudson commented on HBASE-15430:
SUCCESS: Integrated in HBase-1.4 #23 (See
[https://builds.apache.org/job/HBase-1.4/23/])
HBASE-15430 Failed taking snapshot - Manifest proto-message too large
(matteo.bertozzi: rev 103c313b0d0fe4a40d33bb4a81b75ba66a6c6f3e)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotManifest.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
> Failed taking snapshot - Manifest proto-message too large
> -
>
> Key: HBASE-15430
> URL: https://issues.apache.org/jira/browse/HBASE-15430
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.11
>Reporter: JunHo Cho
>Assignee: JunHo Cho
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4, 0.98.18
>
> Attachments: HBASE-15430-v4.patch, HBASE-15430-v5.patch,
> HBASE-15430-v6.patch, hbase-15430-v1.patch, hbase-15430-v2.patch,
> hbase-15430-v3.branch.0.98.patch, hbase-15430.patch
>
>
> the size of a protobuf message is 64MB (default). but the size of snapshot
> meta is over 64MB.
> Caused by: com.google.protobuf.InvalidProtocolBufferException via Failed
> taking snapshot { ss=snapshot_xxx table=xxx type=FLUSH } due to
> exception:Protocol message was too large. May be malicious. Use
> CodedInputStream.setSizeLimit() to increase the size
> limit.:com.google.protobuf.InvalidProtocolBufferException: Protocol message
> was too large. May be malicious. Use CodedInputStream.setSizeLimit() to
> increase the size limit.
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:307)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:341)
> ... 10 more
> Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
> message was too large. May be malicious. Use
> CodedInputStream.setSizeLimit() to increase the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.readRawBytes(CodedInputStream.java:811)
> at
> com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:329)
> at
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo.(HBaseProtos.java:3767)
> at
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo.(HBaseProtos.java:3699)
> at
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo$1.parsePartialFrom(HBaseProtos.java:3815)
> at
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo$1.parsePartialFrom(HBaseProtos.java:3810)
> at
> com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.(SnapshotProtos.java:1152)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.(SnapshotProtos.java:1094)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest$1.parsePartialFrom(SnapshotProtos.java:1201)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest$1.parsePartialFrom(SnapshotProtos.java:1196)
> at
> com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.(SnapshotProtos.java:3858)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.(SnapshotProtos.java:3792)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest$1.parsePartialFrom(SnapshotProtos.java:3894)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest$1.parsePartialFrom(SnapshotProtos.java:3889)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:223)
> at
>
[jira] [Updated] (HBASE-14918) In-Memory MemStore Flush and Compaction
[ https://issues.apache.org/jira/browse/HBASE-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14918: --- Fix Version/s: (was: 0.98.18) > In-Memory MemStore Flush and Compaction > --- > > Key: HBASE-14918 > URL: https://issues.apache.org/jira/browse/HBASE-14918 > Project: HBase > Issue Type: Umbrella >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: CellBlocksSegmentDesign.pdf, MSLABMove.patch > > > A memstore serves as the in-memory component of a store unit, absorbing all > updates to the store. From time to time these updates are flushed to a file > on disk, where they are compacted (by eliminating redundancies) and > compressed (i.e., written in a compressed format to reduce their storage > size). > We aim to speed up data access, and therefore suggest to apply in-memory > memstore flush. That is to flush the active in-memory segment into an > intermediate buffer where it can be accessed by the application. Data in the > buffer is subject to compaction and can be stored in any format that allows > it to take up smaller space in RAM. The less space the buffer consumes the > longer it can reside in memory before data is flushed to disk, resulting in > better performance. > Specifically, the optimization is beneficial for workloads with > medium-to-high key churn which incur many redundant cells, like persistent > messaging. > We suggest to structure the solution as 4 subtasks (respectively, patches). > (1) Infrastructure - refactoring of the MemStore hierarchy, introducing > segment (StoreSegment) as first-class citizen, and decoupling memstore > scanner from the memstore implementation; > (2) Adding StoreServices facility at the region level to allow memstores > update region counters and access region level synchronization mechanism; > (3) Implementation of a new memstore (CompactingMemstore) with non-optimized > immutable segment representation, and > (4) Memory optimization including compressed format representation and off > heap allocations. > This Jira continues the discussion in HBASE-13408. > Design documents, evaluation results and previous patches can be found in > HBASE-13408. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15478) add comments to FSHLog explaining why syncRunnerIndex won't overflow
[
https://issues.apache.org/jira/browse/HBASE-15478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200761#comment-15200761
]
Hadoop QA commented on HBASE-15478:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 30s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 9s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 7m 13s
{color} | {color:red} hbase-server: patch generated 3 new + 60 unchanged - 0
fixed = 63 total (was 60) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
48m 11s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 90m 56s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 181m 19s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.TestCorruptedRegionStoreFile |
| | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.regionserver.TestCompaction |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.wal.TestWALFiltering |
| | org.apache.hadoop.hbase.coprocessor.TestRegionServerObserver |
| | org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey |
| |
[jira] [Commented] (HBASE-15135) Add metrics for storefile age
[ https://issues.apache.org/jira/browse/HBASE-15135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197220#comment-15197220 ] Hudson commented on HBASE-15135: ABORTED: Integrated in HBase-0.98-matrix #312 (See [https://builds.apache.org/job/HBase-0.98-matrix/312/]) HBASE-15135 Add metrics for storefile age (apurtell: rev f0ddd3c066f10aa05e07433d91ee3d356658ec75) * hbase-hadoop1-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionSourceImpl.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperStub.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java * hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionSourceImpl.java > Add metrics for storefile age > - > > Key: HBASE-15135 > URL: https://issues.apache.org/jira/browse/HBASE-15135 > Project: HBase > Issue Type: New Feature >Reporter: Elliott Clark >Assignee: Mikhail Antonov > Fix For: 2.0.0, 1.3.0, 0.98.18 > > Attachments: HBASE-15135-branch-1.v4.1.patch, HBASE-15135-v2.patch, > HBASE-15135-v3.patch, HBASE-15135-v4.1.patch, HBASE-15135-v4.patch, > HBASE-15135.patch > > > In order to make sure that compactions are fully up to date it would be nice > to have metrics on: > * Max storefile age > * Min storefile age > * Average storefile age > * Number of reference files > If we could have those metrics per region and per regionserver it would be > awesome. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15449) HBase Backup Phase 3: Support physical table layout change
[ https://issues.apache.org/jira/browse/HBASE-15449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200136#comment-15200136 ] Vladimir Rodionov commented on HBASE-15449: --- Moved to Phase 3. > HBase Backup Phase 3: Support physical table layout change > --- > > Key: HBASE-15449 > URL: https://issues.apache.org/jira/browse/HBASE-15449 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > > Table operation such as add column family, delete column family, truncate , > delete table may result in subsequent backup restore failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15353) Add metric for number of CallQueueTooBigException's
[ https://issues.apache.org/jira/browse/HBASE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15353: -- Affects Version/s: 2.0.0 1.2.0 > Add metric for number of CallQueueTooBigException's > --- > > Key: HBASE-15353 > URL: https://issues.apache.org/jira/browse/HBASE-15353 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, metrics >Affects Versions: 2.0.0, 1.2.0 >Reporter: Elliott Clark > Labels: easy, noob > > This exception is being thrown more. We should add a metric for this one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15470) Add a setting for Priority queue length
[ https://issues.apache.org/jira/browse/HBASE-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15470: -- Attachment: HBASE-15470.patch https://reviews.facebook.net/D55575 > Add a setting for Priority queue length > --- > > Key: HBASE-15470 > URL: https://issues.apache.org/jira/browse/HBASE-15470 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, Region Assignment >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15470.patch > > > Meta can have very different request rates than any other region. We > shouldn't use the same call queue length. > For example: > If a normal request takes 100ms ( long scan or large get ) > but a call to meta is all in memory so it takes < 1ms > So a call queue length of 100 represents multiple seconds of work for normal > requests, but less than a second for meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15484) Correct the semantic of batch and partial
Phil Yang created HBASE-15484: - Summary: Correct the semantic of batch and partial Key: HBASE-15484 URL: https://issues.apache.org/jira/browse/HBASE-15484 Project: HBase Issue Type: Bug Affects Versions: 1.1.3, 1.2.0 Reporter: Phil Yang Assignee: Phil Yang Follow-up to HBASE-15325, as discussed, the meaning of setBatch and setAllowPartialResults should not be same. We should not regard setBatch as setAllowPartialResults. And isPartial should be define accurately. (Considering getBatch==MaxInt if we don't setBatch.) If result.rawcells.length
[jira] [Updated] (HBASE-15412) Add average region size metric
[ https://issues.apache.org/jira/browse/HBASE-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alicia Ying Shu updated HBASE-15412: Status: Patch Available (was: Open) > Add average region size metric > -- > > Key: HBASE-15412 > URL: https://issues.apache.org/jira/browse/HBASE-15412 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alicia Ying Shu > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15412.patch, HBASE-15412.patch, HBASE-15412.patch > > > We have several metrics related to region store file size, num regions, etc > per regionserver, but we do not have a single metric to track the average > region size per regionserver. > Avg region size is important to look at for deciding on the split policy, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15479) No more garbage or beware of autoboxing
[
https://issues.apache.org/jira/browse/HBASE-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200803#comment-15200803
]
Enis Soztutar commented on HBASE-15479:
---
Can we change the {{Map regionsIncluded}} map to be
{{Map regionsIncluded}}?
> No more garbage or beware of autoboxing
> ---
>
> Key: HBASE-15479
> URL: https://issues.apache.org/jira/browse/HBASE-15479
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-15479-v1.patch
>
>
> Quick journey with JMC in a profile mode revealed very interesting and
> unexpected heap polluter on a client side. Patch will shortly follow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2
[ https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199964#comment-15199964 ] Ted Yu commented on HBASE-14123: Is it possible to upload the patch onto review board ? > HBase Backup/Restore Phase 2 > > > Key: HBASE-14123 > URL: https://issues.apache.org/jira/browse/HBASE-14123 > Project: HBase > Issue Type: Umbrella >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-14123-v1.patch, HBASE-14123-v10.patch, > HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v2.patch, > HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, > HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch > > > Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15477) Do not save 'next block header' when we cache hfileblocks
[
https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201355#comment-15201355
]
Hadoop QA commented on HBASE-15477:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 11 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 53s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 48s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 1s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 7s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 39s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
14s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 8s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 2s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 10s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 8s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 12s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 57s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 22m 2s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s
{color} | {color:green} hbase-external-blockcache in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s
{color} | {color:green} hbase-external-blockcache in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s {color} |
[jira] [Commented] (HBASE-15479) No more garbage or beware of autoboxing
[ https://issues.apache.org/jira/browse/HBASE-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203016#comment-15203016 ] Lars Hofhansl commented on HBASE-15479: --- Ahh... The call is not in the patch but the parameter change is. Cool. Seems good to commit to all branches. > No more garbage or beware of autoboxing > --- > > Key: HBASE-15479 > URL: https://issues.apache.org/jira/browse/HBASE-15479 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-15479-v1.patch, HBASE-15479-v2.patch > > > Quick journey with JMC in a profile mode revealed very interesting and > unexpected heap polluter on a client side. Patch will shortly follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201427#comment-15201427 ] Ted Yu commented on HBASE-15302: Please look at checkstyle warnings in WALSplitter.java which is modified by the patch. > Reenable the other tests disabled by HBASE-14678 > > > Key: HBASE-15302 > URL: https://issues.apache.org/jira/browse/HBASE-15302 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0, 1.2.1 > > Attachments: HBASE-15302-branch-1-v1.patch, HBASE-15302-v1.txt, > HBASE-15302-v1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15412) Add average region size metric
[ https://issues.apache.org/jira/browse/HBASE-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alicia Ying Shu updated HBASE-15412: Attachment: HBASE-15412.patch > Add average region size metric > -- > > Key: HBASE-15412 > URL: https://issues.apache.org/jira/browse/HBASE-15412 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Alicia Ying Shu > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15412.patch, HBASE-15412.patch, HBASE-15412.patch > > > We have several metrics related to region store file size, num regions, etc > per regionserver, but we do not have a single metric to track the average > region size per regionserver. > Avg region size is important to look at for deciding on the split policy, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15064) BufferUnderflowException after last Cell fetched from an HFile Block served from L2 offheap cache
[
https://issues.apache.org/jira/browse/HBASE-15064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198777#comment-15198777
]
Anoop Sam John commented on HBASE-15064:
I see I get now what you are saying.. Let me see
> BufferUnderflowException after last Cell fetched from an HFile Block served
> from L2 offheap cache
> -
>
> Key: HBASE-15064
> URL: https://issues.apache.org/jira/browse/HBASE-15064
> Project: HBase
> Issue Type: Bug
> Components: io
>Affects Versions: 2.0.0
>Reporter: deepankar
>Assignee: Anoop Sam John
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15064.patch
>
>
> While running the newer patches on our production system, I saw this error
> come couple of times
> {noformat}
> ipc.RpcServer: Unexpected throwable object
> 2016-01-01 16:42:56,090 ERROR
> [B.defaultRpcServer.handler=20,queue=20,port=60020] ipc.RpcServer: Unexpected
> throwable object
> java.nio.BufferUnderflowException
> at java.nio.Buffer.nextGetIndex(Buffer.java:500)
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
> at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:494)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decode(FastDiffDeltaEncoder.java:402)
>
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decodeNext(FastDiffDeltaEncoder.java:517)
>
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.next(BufferedDataBlockEncoder.java:815)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:138)
> {noformat}
> Looking at the get code
> {code}
> if (this.curItem.remaining() == 0) {
> if (items.length - 1 == this.curItemIndex) {
> // means cur item is the last one and we wont be able to read a long.
> Throw exception
> throw new BufferUnderflowException();
> }
> this.curItemIndex++;
> this.curItem = this.items[this.curItemIndex];
> }
> return this.curItem.get();
> {code}
> Can the new currentItem have zero elements (position == limit), does it make
> sense to change the {{if}} to {{while}} ? {{while (this.curItem.remaining()
> == 0)}}. This logic is repeated may make sense abstract to a new function if
> we plan to change to {{if}} to {{while}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15474) Exception in HConnectionImplementation's constructor cause Zookeeper connnections leak
[
https://issues.apache.org/jira/browse/HBASE-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-15474:
---
Description:
HConnectionImplementation creates a ZooKeeperKeepAliveConnection during
construction, but if the constructor throw a exception, the zookeeper
connection is not properly closed.
{code}
HConnectionImplementation(Configuration conf, boolean managed,
ExecutorService pool, User user) throws IOException {
this(conf);
this.user = user;
this.batchPool = pool;
this.managed = managed;
this.registry = setupRegistry();
retrieveClusterId(); //here is the zookeeper connection created
this.rpcClient = RpcClientFactory.createClient(this.conf,
this.clusterId);// In our case, the exception happens here, so the zookeeper
connection never closes
this.rpcControllerFactory = RpcControllerFactory.instantiate(conf);
..
{code}
was:
HConnectionImplementation creates a ZooKeeperKeepAliveConnection during
construction, but if the constructor throw a exception, the zookeeper
connection is not properly closed.
{code}
HConnectionImplementation(Configuration conf, boolean managed,
ExecutorService pool, User user) throws IOException {
this(conf);
this.user = user;
this.batchPool = pool;
this.managed = managed;
this.registry = setupRegistry();
retrieveClusterId(); //here is the zookeeper connection created
this.rpcClient = RpcClientFactory.createClient(this.conf,
this.clusterId);
this.rpcControllerFactory = RpcControllerFactory.instantiate(conf);//
In our case, the exception happens here, so the zookeeper connection never close
..
{code}
> Exception in HConnectionImplementation's constructor cause Zookeeper
> connnections leak
> ---
>
> Key: HBASE-15474
> URL: https://issues.apache.org/jira/browse/HBASE-15474
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Allan Yang
>Assignee: Allan Yang
> Labels: patch
> Fix For: 1.1.0
>
> Attachments: HBASE-15474.patch
>
>
> HConnectionImplementation creates a ZooKeeperKeepAliveConnection during
> construction, but if the constructor throw a exception, the zookeeper
> connection is not properly closed.
> {code}
> HConnectionImplementation(Configuration conf, boolean managed,
> ExecutorService pool, User user) throws IOException {
> this(conf);
> this.user = user;
> this.batchPool = pool;
> this.managed = managed;
> this.registry = setupRegistry();
> retrieveClusterId(); //here is the zookeeper connection created
> this.rpcClient = RpcClientFactory.createClient(this.conf,
> this.clusterId);// In our case, the exception happens here, so the zookeeper
> connection never closes
> this.rpcControllerFactory = RpcControllerFactory.instantiate(conf);
> ..
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15470) Add a setting for Priority queue length
[ https://issues.apache.org/jira/browse/HBASE-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198825#comment-15198825 ] Hudson commented on HBASE-15470: SUCCESS: Integrated in HBase-1.3-IT #557 (See [https://builds.apache.org/job/HBase-1.3-IT/557/]) HBASE-15470 Add a setting for Priority queue length (eclark: rev f1bb26c2235e231c6326cd5c78932f492844b8aa) * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcScheduler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoRpcScheduler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java > Add a setting for Priority queue length > --- > > Key: HBASE-15470 > URL: https://issues.apache.org/jira/browse/HBASE-15470 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, Region Assignment >Affects Versions: 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15470.patch > > > Meta can have very different request rates than any other region. We > shouldn't use the same call queue length. > For example: > If a normal request takes 100ms ( long scan or large get ) > but a call to meta is all in memory so it takes < 1ms > So a call queue length of 100 represents multiple seconds of work for normal > requests, but less than a second for meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9393) Hbase does not closing a closed socket resulting in many CLOSE_WAIT
[
https://issues.apache.org/jira/browse/HBASE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197886#comment-15197886
]
Hadoop QA commented on HBASE-9393:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 32s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 109m 1s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 82m 34s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 244m 30s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0 Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| | org.apache.hadoop.hbase.master.TestMasterFailover |
| | org.apache.hadoop.hbase.util.TestHBaseFsckOneRS |
| | org.apache.hadoop.hbase.mapred.TestTableInputFormat |
| | org.apache.hadoop.hbase.wal.TestWALFiltering |
| | org.apache.hadoop.hbase.mapred.TestTableMapReduceUtil |
| |
[jira] [Commented] (HBASE-14450) HBase Backup/Restore Phase 3: Multiwal support
[ https://issues.apache.org/jira/browse/HBASE-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200142#comment-15200142 ] Vladimir Rodionov commented on HBASE-14450: --- Moved to Phase 3. > HBase Backup/Restore Phase 3: Multiwal support > -- > > Key: HBASE-14450 > URL: https://issues.apache.org/jira/browse/HBASE-14450 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > > We need to support multiwal configurations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14857) add region server groups in master UI
[ https://issues.apache.org/jira/browse/HBASE-14857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-14857: Issue Type: New Feature (was: Sub-task) Parent: (was: HBASE-6721) > add region server groups in master UI > - > > Key: HBASE-14857 > URL: https://issues.apache.org/jira/browse/HBASE-14857 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu > Labels: hbase-6721 > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15411) Rewrite backup with Procedure V2
[ https://issues.apache.org/jira/browse/HBASE-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15411: --- Status: Patch Available (was: Open) > Rewrite backup with Procedure V2 > > > Key: HBASE-15411 > URL: https://issues.apache.org/jira/browse/HBASE-15411 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 15411-v1.txt, 15411-v11.txt, 15411-v12.txt, > 15411-v13.txt, 15411-v3.txt, 15411-v5.txt, 15411-v6.txt, 15411-v7.txt, > 15411-v9.txt, FullTableBackupProcedure.java > > > Currently full / incremental backup is driven by BackupHandler (see call() > method for flow). > This issue is to rewrite the flow using Procedure V2. > States (enum) for full / incremental backup would be introduced in > Backup.proto which correspond to the steps performed in BackupHandler#call(). > executeFromState() would pace the backup based on the current state. > serializeStateData() / deserializeStateData() would be used to persist state > into procedure WAL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14450) HBase Backup/Restore Phase 2: Multiwal support
[ https://issues.apache.org/jira/browse/HBASE-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14450: -- Description: We need to support multiwal configurations. (was: We need support multiwal configurations.) > HBase Backup/Restore Phase 2: Multiwal support > -- > > Key: HBASE-14450 > URL: https://issues.apache.org/jira/browse/HBASE-14450 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > > We need to support multiwal configurations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15472) replication_admin_test creates a table it doesn't use
Geoffrey Jacoby created HBASE-15472: --- Summary: replication_admin_test creates a table it doesn't use Key: HBASE-15472 URL: https://issues.apache.org/jira/browse/HBASE-15472 Project: HBase Issue Type: Bug Components: Replication, shell Affects Versions: 1.2.0, 2.0.0 Reporter: Geoffrey Jacoby Assignee: Geoffrey Jacoby Priority: Minor I noticed while adding tests to replication_admin_test.rb for HBASE-12940 that it creates an HBase table "hbase_shell_tests_table" that is never used in any of the suite's tests. Removing the table creation statements speeds up the suite locally from 1min 10s to 2s. Note that until HBASE-14562 is worked, this test suite doesn't run as part of the automatic test runs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15390) Unnecessary MetaCache evictions cause elevated number of requests to meta
[ https://issues.apache.org/jira/browse/HBASE-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198446#comment-15198446 ] Mikhail Antonov commented on HBASE-15390: - Thanks [~busbey], that puzzled me when I saw it. > Unnecessary MetaCache evictions cause elevated number of requests to meta > - > > Key: HBASE-15390 > URL: https://issues.apache.org/jira/browse/HBASE-15390 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.2.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Blocker > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15390.branch-1.3.v1.patch, HBASE-15390.v1.patch > > > - In ClientExceptionsUtil#findException() we don't unwrap correctly all > remote exceptions but a few selected ones; >more specifically, we don't unwrap correctly all remote exceptions where > getCause() exception doesn't have a constructor (String message) > - In AsyncProcess#receiveGlobalFailure() we don't check for > #isMetaClearingException() if tableName is null > - We don't have metric on client side to track number of cache drops -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15064) BufferUnderflowException after last Cell fetched from an HFile Block served from L2 offheap cache
[
https://issues.apache.org/jira/browse/HBASE-15064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198793#comment-15198793
]
deepankar commented on HBASE-15064:
---
What if we remove the equality in the if clause in the limit and in the limit
method modify the new limitIndex to search for limit - 1 will that not work
seamlessly ?
> BufferUnderflowException after last Cell fetched from an HFile Block served
> from L2 offheap cache
> -
>
> Key: HBASE-15064
> URL: https://issues.apache.org/jira/browse/HBASE-15064
> Project: HBase
> Issue Type: Bug
> Components: io
>Affects Versions: 2.0.0
>Reporter: deepankar
>Assignee: Anoop Sam John
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15064.patch
>
>
> While running the newer patches on our production system, I saw this error
> come couple of times
> {noformat}
> ipc.RpcServer: Unexpected throwable object
> 2016-01-01 16:42:56,090 ERROR
> [B.defaultRpcServer.handler=20,queue=20,port=60020] ipc.RpcServer: Unexpected
> throwable object
> java.nio.BufferUnderflowException
> at java.nio.Buffer.nextGetIndex(Buffer.java:500)
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
> at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:494)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decode(FastDiffDeltaEncoder.java:402)
>
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decodeNext(FastDiffDeltaEncoder.java:517)
>
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.next(BufferedDataBlockEncoder.java:815)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:138)
> {noformat}
> Looking at the get code
> {code}
> if (this.curItem.remaining() == 0) {
> if (items.length - 1 == this.curItemIndex) {
> // means cur item is the last one and we wont be able to read a long.
> Throw exception
> throw new BufferUnderflowException();
> }
> this.curItemIndex++;
> this.curItem = this.items[this.curItemIndex];
> }
> return this.curItem.get();
> {code}
> Can the new currentItem have zero elements (position == limit), does it make
> sense to change the {{if}} to {{while}} ? {{while (this.curItem.remaining()
> == 0)}}. This logic is repeated may make sense abstract to a new function if
> we plan to change to {{if}} to {{while}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15479) No more garbage or beware of autoboxing
[
https://issues.apache.org/jira/browse/HBASE-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203001#comment-15203001
]
Vladimir Rodionov commented on HBASE-15479:
---
Yes, it is complete, Lars
{code}
- Map regionIncluded = new HashMap();
+ Map regionIncluded = new HashMap();
skip
protected boolean canTakeOperation(HRegionLocation loc,
- Map regionsIncluded,
+ Map regionsIncluded,
Map serversIncluded)
{
{code}
> No more garbage or beware of autoboxing
> ---
>
> Key: HBASE-15479
> URL: https://issues.apache.org/jira/browse/HBASE-15479
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-15479-v1.patch, HBASE-15479-v2.patch
>
>
> Quick journey with JMC in a profile mode revealed very interesting and
> unexpected heap polluter on a client side. Patch will shortly follow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15430) Failed taking snapshot - Manifest proto-message too large
[
https://issues.apache.org/jira/browse/HBASE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198505#comment-15198505
]
Hudson commented on HBASE-15430:
FAILURE: Integrated in HBase-1.3 #605 (See
[https://builds.apache.org/job/HBase-1.3/605/])
HBASE-15430 Failed taking snapshot - Manifest proto-message too large
(matteo.bertozzi: rev 725a9cb4c352541a386e0b4e2c162422b5cb4079)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotManifest.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
> Failed taking snapshot - Manifest proto-message too large
> -
>
> Key: HBASE-15430
> URL: https://issues.apache.org/jira/browse/HBASE-15430
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.11
>Reporter: JunHo Cho
>Assignee: JunHo Cho
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4, 0.98.18
>
> Attachments: HBASE-15430-v4.patch, HBASE-15430-v5.patch,
> HBASE-15430-v6.patch, hbase-15430-v1.patch, hbase-15430-v2.patch,
> hbase-15430-v3.branch.0.98.patch, hbase-15430.patch
>
>
> the size of a protobuf message is 64MB (default). but the size of snapshot
> meta is over 64MB.
> Caused by: com.google.protobuf.InvalidProtocolBufferException via Failed
> taking snapshot { ss=snapshot_xxx table=xxx type=FLUSH } due to
> exception:Protocol message was too large. May be malicious. Use
> CodedInputStream.setSizeLimit() to increase the size
> limit.:com.google.protobuf.InvalidProtocolBufferException: Protocol message
> was too large. May be malicious. Use CodedInputStream.setSizeLimit() to
> increase the size limit.
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:307)
> at
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:341)
> ... 10 more
> Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
> message was too large. May be malicious. Use
> CodedInputStream.setSizeLimit() to increase the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.readRawBytes(CodedInputStream.java:811)
> at
> com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:329)
> at
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo.(HBaseProtos.java:3767)
> at
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo.(HBaseProtos.java:3699)
> at
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo$1.parsePartialFrom(HBaseProtos.java:3815)
> at
> org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo$1.parsePartialFrom(HBaseProtos.java:3810)
> at
> com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.(SnapshotProtos.java:1152)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.(SnapshotProtos.java:1094)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest$1.parsePartialFrom(SnapshotProtos.java:1201)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest$1.parsePartialFrom(SnapshotProtos.java:1196)
> at
> com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.(SnapshotProtos.java:3858)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.(SnapshotProtos.java:3792)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest$1.parsePartialFrom(SnapshotProtos.java:3894)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest$1.parsePartialFrom(SnapshotProtos.java:3889)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:223)
> at
>
[jira] [Updated] (HBASE-15456) CreateTableProcedure/ModifyTableProcedure needs to fail when there is no family in table descriptor
[
https://issues.apache.org/jira/browse/HBASE-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15456:
---
Status: Patch Available (was: Reopened)
> CreateTableProcedure/ModifyTableProcedure needs to fail when there is no
> family in table descriptor
> ---
>
> Key: HBASE-15456
> URL: https://issues.apache.org/jira/browse/HBASE-15456
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15456-001_branch-1.patch, HBASE-15456-v001.patch,
> HBASE-15456-v002.patch, HBASE-15456-v002.patch, HBASE-15456-v003.patch
>
>
> If there is only one family in the table, DeleteColumnFamilyProcedure will
> fail.
> Currently, when hbase.table.sanity.checks is set to false, hbase master logs
> a warning and CreateTableProcedure/ModifyTableProcedure will succeed.
> This behavior is not consistent with DeleteColumnFamilyProcedure's.
> Another point, before HBASE-13145, PeriodicMemstoreFlusher will run into the
> following exception. lastStoreFlushTimeMap is populated for families, if
> there is no family in the table, there is no entry in lastStoreFlushTimeMap.
> {code}
> 16/02/01 11:14:26 ERROR regionserver.HRegionServer$PeriodicMemstoreFlusher:
> Caught exception
> java.util.NoSuchElementException
> at
> java.util.concurrent.ConcurrentHashMap$HashIterator.nextEntry(ConcurrentHashMap.java:1354)
>
> at
> java.util.concurrent.ConcurrentHashMap$ValueIterator.next(ConcurrentHashMap.java:1384)
>
> at java.util.Collections.min(Collections.java:628)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getEarliestFlushTimeForAllStores(HRegion.java:1572)
>
> at
> org.apache.hadoop.hbase.regionserver.HRegion.shouldFlush(HRegion.java:1904)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$PeriodicMemstoreFlusher.chore(HRegionServer.java:1509)
>
> at org.apache.hadoop.hbase.Chore.run(Chore.java:87)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15411) Rewrite backup with Procedure V2
[
https://issues.apache.org/jira/browse/HBASE-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198611#comment-15198611
]
Hadoop QA commented on HBASE-15411:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.2.0/precommit-patchnames for
instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15411 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12793882/15411-v14.txt |
| JIRA Issue | HBASE-15411 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1029/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> Rewrite backup with Procedure V2
>
>
> Key: HBASE-15411
> URL: https://issues.apache.org/jira/browse/HBASE-15411
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15411-v1.txt, 15411-v11.txt, 15411-v12.txt,
> 15411-v13.txt, 15411-v14.txt, 15411-v3.txt, 15411-v5.txt, 15411-v6.txt,
> 15411-v7.txt, 15411-v9.txt, FullTableBackupProcedure.java
>
>
> Currently full / incremental backup is driven by BackupHandler (see call()
> method for flow).
> This issue is to rewrite the flow using Procedure V2.
> States (enum) for full / incremental backup would be introduced in
> Backup.proto which correspond to the steps performed in BackupHandler#call().
> executeFromState() would pace the backup based on the current state.
> serializeStateData() / deserializeStateData() would be used to persist state
> into procedure WAL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15475) Allow TimestampsFilter to provide a seek hint
[ https://issues.apache.org/jira/browse/HBASE-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15475: -- Attachment: HBASE-15475-v2.patch Cleaned up a couple of un-used imports before checkstyle got mad at me. > Allow TimestampsFilter to provide a seek hint > - > > Key: HBASE-15475 > URL: https://issues.apache.org/jira/browse/HBASE-15475 > Project: HBase > Issue Type: Improvement > Components: Client, Filters, regionserver >Affects Versions: 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15475-v1.patch, HBASE-15475-v2.patch, > HBASE-15475.patch > > > If a user wants to read specific timestamps currently it's a very linear > scan. This is so that all deletes can be respected. However if the user > doesn't care about deletes it can dramatically speed up the request to seek. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15441) Fix WAL splitting when region has moved multiple times
[ https://issues.apache.org/jira/browse/HBASE-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198351#comment-15198351 ] Sean Busbey commented on HBASE-15441: - I presume the H* machines are in need of some attention. I've noticed a tons of spurious failed tests over the last week or so, just haven't had time to dig in and see what changed on the jenkins config. > Fix WAL splitting when region has moved multiple times > -- > > Key: HBASE-15441 > URL: https://issues.apache.org/jira/browse/HBASE-15441 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.0, 1.2.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Blocker > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15441-v1.patch, HBASE-15441-v2.patch, > HBASE-15441-v3.patch, HBASE-15441-v4.patch, HBASE-15441-v5.patch, > HBASE-15441.patch > > > Currently WAL splitting is broken when a region has been opened multiple > times in recent minutes. > Region open and region close write event markers to the wal. These markers > should have the sequence id in them. However it is currently getting 1. That > means that if a region has moved multiple times in the last few mins then > multiple split log workers will try and create the recovered edits file for > sequence id 1. One of the workers will fail and on failing they will delete > the recovered edits. Causing all split wal attempts to fail. > We need to: > # make sure that close get the correct sequence id for open. > # Filter all region events from recovered edits > It appears that the close event with a sequence id of one is coming from > region warm up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15064) BufferUnderflowException after last Cell fetched from an HFile Block served from L2 offheap cache
[
https://issues.apache.org/jira/browse/HBASE-15064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198872#comment-15198872
]
ramkrishna.s.vasudevan commented on HBASE-15064:
{code}
for(int i = 0; i < 12; i++) {
assertTrue(mbb1.hasRemaining());
mbb1.get();
}
{code}
Ya that is true. Good point.
> BufferUnderflowException after last Cell fetched from an HFile Block served
> from L2 offheap cache
> -
>
> Key: HBASE-15064
> URL: https://issues.apache.org/jira/browse/HBASE-15064
> Project: HBase
> Issue Type: Bug
> Components: io
>Affects Versions: 2.0.0
>Reporter: deepankar
>Assignee: Anoop Sam John
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15064.patch, MBB_hasRemaining.patch
>
>
> While running the newer patches on our production system, I saw this error
> come couple of times
> {noformat}
> ipc.RpcServer: Unexpected throwable object
> 2016-01-01 16:42:56,090 ERROR
> [B.defaultRpcServer.handler=20,queue=20,port=60020] ipc.RpcServer: Unexpected
> throwable object
> java.nio.BufferUnderflowException
> at java.nio.Buffer.nextGetIndex(Buffer.java:500)
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
> at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:494)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decode(FastDiffDeltaEncoder.java:402)
>
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decodeNext(FastDiffDeltaEncoder.java:517)
>
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.next(BufferedDataBlockEncoder.java:815)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:138)
> {noformat}
> Looking at the get code
> {code}
> if (this.curItem.remaining() == 0) {
> if (items.length - 1 == this.curItemIndex) {
> // means cur item is the last one and we wont be able to read a long.
> Throw exception
> throw new BufferUnderflowException();
> }
> this.curItemIndex++;
> this.curItem = this.items[this.curItemIndex];
> }
> return this.curItem.get();
> {code}
> Can the new currentItem have zero elements (position == limit), does it make
> sense to change the {{if}} to {{while}} ? {{while (this.curItem.remaining()
> == 0)}}. This logic is repeated may make sense abstract to a new function if
> we plan to change to {{if}} to {{while}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception
[
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201414#comment-15201414
]
Ted Yu commented on HBASE-15433:
Jianwei:
I looked at the scenario (region counts of 8 and 10 in snapshots) above where
retrieving from region locator may produce inaccurate end result.
We can go with path v4.
Since branch-1 is among the Fix versions, can you attach patch for branch-1 ?
branch-1 tests are more stable, hopefully we can get higher confidence.
Thanks
> SnapshotManager#restoreSnapshot not update table and region count quota
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.4.0, 1.1.5
>
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch,
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be
> checked and updated as:
> {code}
> try {
> // Table already exist. Check and update the region quota for this
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
> } catch (IOException e) {
>
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " +
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
> }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot
> make the region count quota exceeded, then, the table will be removed in the
> 'catch' block. This will make the current table count and region count
> decrease, following table creation or region split will succeed even if the
> actual quota is exceeded.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15406) Split / merge switch left disabled after early termination of hbck
[
https://issues.apache.org/jira/browse/HBASE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201458#comment-15201458
]
Heng Chen commented on HBASE-15406:
---
{quote}
Since the key can be ignored, it is not that useful. You can remove the code
related to key.
{quote}
So we do not need to protect the switch NOT be changed during hbck?
> Split / merge switch left disabled after early termination of hbck
> --
>
> Key: HBASE-15406
> URL: https://issues.apache.org/jira/browse/HBASE-15406
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15406.patch, HBASE-15406.v1.patch,
> HBASE-15406_v1.patch, test.patch, wip.patch
>
>
> This was what I did on cluster with 1.4.0-SNAPSHOT built Thursday:
> Run 'hbase hbck -disableSplitAndMerge' on gateway node of the cluster
> Terminate hbck early
> Enter hbase shell where I observed:
> {code}
> hbase(main):001:0> splitormerge_enabled 'SPLIT'
> false
> 0 row(s) in 0.3280 seconds
> hbase(main):002:0> splitormerge_enabled 'MERGE'
> false
> 0 row(s) in 0.0070 seconds
> {code}
> Expectation is that the split / merge switches should be restored to default
> value after hbck exits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
