[jira] [Updated] (HBASE-17994) Add async client test to Performance Evaluation tool
[ https://issues.apache.org/jira/browse/HBASE-17994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17994: --- Attachment: HBASE-17994.master.002.patch > Add async client test to Performance Evaluation tool > > > Key: HBASE-17994 > URL: https://issues.apache.org/jira/browse/HBASE-17994 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-17994.master.001.patch, > HBASE-17994.master.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106832#comment-16106832 ] Hudson commented on HBASE-18481: FAILURE: Integrated in Jenkins build HBase-2.0 #265 (See [https://builds.apache.org/job/HBase-2.0/265/]) HBASE-18481 The autoFlush flag was not used in PE tool (zghao: rev 8fc010636a265cd4dc435d0905583af8e06a4600) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4
[
https://issues.apache.org/jira/browse/HBASE-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106828#comment-16106828
]
Anoop Sam John commented on HBASE-18431:
+1 on ur proposal Andy
> Mitigate compatibility concerns between branch-1.3 and branch-1.4
> -
>
> Key: HBASE-18431
> URL: https://issues.apache.org/jira/browse/HBASE-18431
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0, 1.5.0
>
> Attachments: HBASE-18431-branch-1.4.patch
>
>
> There are compatibility concerns with branch-1.4.
> {noformat}
> Library Name HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject Binary Compatibility
> Compatibility - 89.9%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
> High - 23
> Medium - 9
> Low - 21
> {noformat}
> {noformat}
> Library Name HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject Source Compatibility
> Compatibility- 86.5%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
> High - 88
> Medium - 0
> Low - 0
> Other Changes in Data Types- 25
> {noformat}
> This report includes HBASE-15816 which hasn't been committed yet. Otherwise
> it's current.
> I'm not generally concerned with added methods.
> The following methods have been added to Public/Evolving interface Table.
> Pointing them out in case it merits review.
> \\
> * Abstract method Table.getReadRpcTimeout ( ) has been added to this
> interface. No effect.
> * Abstract method Table.getWriteRpcTimeout ( ) has been added to this
> interface. No effect.
> * Abstract method Table.setReadRpcTimeout ( int ) has been added to this
> interface. No effect.
> * Abstract method Table.setWriteRpcTimeout ( int ) has been added to this
> interface.
> The Public/Evolving interface Admin has some signature changes equating to
> removed methods. I don't think this is allowed in a minor release.
> \\
> * Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription
> ) has been removed from Admin.
> * Abstract method Admin.snapshot ( String, TableName,
> HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
> * Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been
> removed from Admin.
> * Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription
> ) has been removed from Admin.
> The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This
> change is debatable but I think we can allow it.
> \\
> * AsyncRpcClient has been removed
> The Public/Evolving class FastLongHistogram has been removed. I don't believe
> this change is allowed in a minor release.
> \\
> * FastLongHistogram has been removed
> Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and
> RegionObserver have changed, equating to removed methods. The first set of
> changes is due to move of SnapshotDescription from HBaseProtos to
> SnapshotProtos:
> \\
> * Abstract method MasterObserver.postCloneSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.postDeleteSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postListSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postRestoreSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.postSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.preCloneSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.preDeleteSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preListSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preRestoreSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.preSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> Here, maybe DeleteTracker moved packages?
> \\
> * Abstract method
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106823#comment-16106823 ] Hudson commented on HBASE-18481: FAILURE: Integrated in Jenkins build HBase-1.4 #828 (See [https://builds.apache.org/job/HBase-1.4/828/]) HBASE-18481 The autoFlush flag was not used in PE tool (zghao: rev 59cad7d3e35f0093a391ab56926d6d99be2ea328) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106820#comment-16106820 ] Hudson commented on HBASE-18481: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1978 (See [https://builds.apache.org/job/HBase-1.1-JDK8/1978/]) HBASE-18481 The autoFlush flag was not used in PE tool (zghao: rev f946c0b02d31dd0f8428cc6672d6bccbb1118995) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18374) RegionServer Metrics improvements
[ https://issues.apache.org/jira/browse/HBASE-18374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106816#comment-16106816 ] Abhishek Singh Chouhan commented on HBASE-18374: Sure [~carp84], will fill out the release notes. Will also be pushing to branch-1.4. > RegionServer Metrics improvements > - > > Key: HBASE-18374 > URL: https://issues.apache.org/jira/browse/HBASE-18374 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-18374.branch-1.001.patch, > HBASE-18374.branch-1.001.patch, HBASE-18374.branch-1.002.patch, > HBASE-18374.branch-1.003.patch, HBASE-18374.master.001.patch, > HBASE-18374.master.002.patch, HBASE-18374.master.003.patch, > HBASE-18374.master.004.patch, HBASE-18374.master.005.patch > > > At the RS level we have latency metrics for mutate/puts and deletes that are > updated per batch (ie. at the end of entire batchop if it contains put/delete > update the respective metric) in contrast with append/increment/get metrics > that are updated per op. This is a bit ambiguous since the delete and put > metrics are updated for multi row mutations that happen to contain a > put/delete. We should rename the metric(eg. delete_batch)/add better > description. Also we should add metrics for single delete client operations > that come through RSRpcServer.mutate path. We should also add metrics for > checkAndPut and checkAndDelete. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15248) BLOCKSIZE 4k should result in 4096 bytes on disk; i.e. fit inside a BucketCache 'block' of 4k
[ https://issues.apache.org/jira/browse/HBASE-15248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106811#comment-16106811 ] Anoop Sam John commented on HBASE-15248: This block header we know it is 33 bytes and there will be 13 bytes meta data. And the CRCs we know(?) how many bytes? May be yes. These are all extra other than the block size what a user configured (4KB). But we know how many bytes extra. The biggest issue is our block size is NOT an upper cap. This is not the max size a block will have. This is the min size a block will have. The check for ending cur block and starting new is with out considering cur cell size. Also one more thing is we dont want to write duplicate cells (Same key but diff mvcc numbers) in diff blocks. IMO this the main concern area for making sure the block size is very predictable. Because of this unpredictability what we do is all block sizes user will end up adding some extra size. Remember the default values for BC block sizes as (5 KB, 9 KB etc) for block sizes 4KB, 8 KB. Extra 1 KB per block is considering all these extra things. > BLOCKSIZE 4k should result in 4096 bytes on disk; i.e. fit inside a > BucketCache 'block' of 4k > - > > Key: HBASE-15248 > URL: https://issues.apache.org/jira/browse/HBASE-15248 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Reporter: stack > > Chatting w/ a gentleman named Daniel Pol who is messing w/ bucketcache, he > wants blocks to be the size specified in the configuration and no bigger. His > hardware set ups fetches pages of 4k and so a block that has 4k of payload > but has then a header and the header of the next block (which helps figure > whats next when scanning) ends up being 4203 bytes or something, and this > then then translates into two seeks per block fetch. > This issue is about what it would take to stay inside our configured size > boundary writing out blocks. > If not possible, give back better signal on what to do so you could fit > inside a particular constraint. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18448) Added support for refreshing HFiles through API and shell
[ https://issues.apache.org/jira/browse/HBASE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106793#comment-16106793 ] ramkrishna.s.vasudevan commented on HBASE-18448: bq.My only concern here is that we are already exposing this as an API. Yes as I said in my previous comment as this is exposed in shell you may have to provide CP endpoint way and also the old way until you find ways to remove the API support. > Added support for refreshing HFiles through API and shell > - > > Key: HBASE-18448 > URL: https://issues.apache.org/jira/browse/HBASE-18448 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.1 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav >Priority: Minor > Fix For: 1.4.0 > > Attachments: HBASE-18448.branch-1.001.patch, > HBASE-18448.branch-1.002.patch > > > In the case where multiple HBase clusters are sharing a common rootDir, even > after flushing the data from > one cluster doesn't mean that other clusters (replicas) will automatically > pick the new HFile. Through this patch, > we are exposing the refresh HFiles API which when issued from a replica will > update the in-memory file handle list > with the newly added file. > This allows replicas to be consistent with the data written through the > primary cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18469) Correct RegionServer metric of totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li reassigned HBASE-18469: - Assignee: Yu Li Assign to myself to move this forward. > Correct RegionServer metric of totalRequestCount > -- > > Key: HBASE-18469 > URL: https://issues.apache.org/jira/browse/HBASE-18469 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Shibin Zhang >Assignee: Yu Li >Priority: Critical > > when i get the metric ,i found this three metric may be have some error as > follow : > "totalRequestCount" : 17541, > "readRequestCount" : 17483, > "writeRequestCount" : 1633, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18469) Correct RegionServer metric of totalRequestCount
[
https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106788#comment-16106788
]
Yu Li commented on HBASE-18469:
---
bq. I do not think we should count a request with caching 10 as 10 requests.
Also for multi. It may be called requestRowsCount?
I like the idea of introducing requestRowsCount, which could clearly separate
the RS metrics into two categories:
1. requestCount: The rpc workload, which only count in rpc requests, count as
only one no matter it's single or multi.
2. requestRowsCount: The real workload, which counts per region action, and
will count as many for caching-scan or multi.
While currently for multi we have below codes in {{RsRpcServices#multi}}:
{code}
for (RegionAction regionAction : request.getRegionActionList()) {
this.requestCount.add(regionAction.getActionCount());
{code}
So there would be some changes required to do the metrics separation. Let's
wait for others' opinion.
btw, I could see relative efforts/discussion in HBASE-18374.
> Correct RegionServer metric of totalRequestCount
> --
>
> Key: HBASE-18469
> URL: https://issues.apache.org/jira/browse/HBASE-18469
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Shibin Zhang
>Priority: Critical
>
> when i get the metric ,i found this three metric may be have some error as
> follow :
> "totalRequestCount" : 17541,
> "readRequestCount" : 17483,
> "writeRequestCount" : 1633,
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18374) RegionServer Metrics improvements
[ https://issues.apache.org/jira/browse/HBASE-18374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106786#comment-16106786 ] Yu Li commented on HBASE-18374: --- I could see some incompatible changes on metrics names (such as regionserver.server.mutate -> regionserver.server.put), which will cause existing monitoring on those metrics not working anymore after upgrading. The metrics compatibility is included in "Operational Compatibility" in our [ref guid|http://hbase.apache.org/book.html#hbase.versioning] and we're allowed to break it in major/minor versions but not in patch versions. So I think it's ok to go into branch-1/branch-2/master but please fill in the release notes. Thanks. > RegionServer Metrics improvements > - > > Key: HBASE-18374 > URL: https://issues.apache.org/jira/browse/HBASE-18374 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-18374.branch-1.001.patch, > HBASE-18374.branch-1.001.patch, HBASE-18374.branch-1.002.patch, > HBASE-18374.branch-1.003.patch, HBASE-18374.master.001.patch, > HBASE-18374.master.002.patch, HBASE-18374.master.003.patch, > HBASE-18374.master.004.patch, HBASE-18374.master.005.patch > > > At the RS level we have latency metrics for mutate/puts and deletes that are > updated per batch (ie. at the end of entire batchop if it contains put/delete > update the respective metric) in contrast with append/increment/get metrics > that are updated per op. This is a bit ambiguous since the delete and put > metrics are updated for multi row mutations that happen to contain a > put/delete. We should rename the metric(eg. delete_batch)/add better > description. Also we should add metrics for single delete client operations > that come through RSRpcServer.mutate path. We should also add metrics for > checkAndPut and checkAndDelete. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18469) Correct RegionServer metric of totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106782#comment-16106782 ] Duo Zhang commented on HBASE-18469: --- I do not think we should count a request with caching 10 as 10 requests. Also for multi. It may be called requestRowsCount? > Correct RegionServer metric of totalRequestCount > -- > > Key: HBASE-18469 > URL: https://issues.apache.org/jira/browse/HBASE-18469 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Shibin Zhang >Priority: Critical > > when i get the metric ,i found this three metric may be have some error as > follow : > "totalRequestCount" : 17541, > "readRequestCount" : 17483, > "writeRequestCount" : 1633, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18469) Correct RegionServer metric of totalRequestCount
[
https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106779#comment-16106779
]
Yu Li commented on HBASE-18469:
---
[~zhangshibin] Thanks for filing this JIRA and let me try to explain.
The difference mainly comes from scan with caching. Each scan request will be
counted as one in {{RsRpcServices#requestCount}} no matter with caching or not,
while the {{readRequestCount}} is counted inside {{HRegion}} and will increment
every time {{next(List outResults, ScannerContext scannerContext)}} is
called. So if scan with caching 10, then readRequestCount will be counted 9
more than requestCount, thus finally totalRequestCount doesn't equal to the sum
of readRequestCount and writeRequestCount.
Considering how we're dealing with the {{multi}} call (we will increase
requestCount per region action), I think it makes sense to also count caching
in for {{RsRpcServices#requestCount}}. And if we all agree on this, I could
supply a patch (should be straight forward). Thanks.
cc more people for more suggestions (besides those already watching)
[~yangzhe1991] [~Apache9] [~busbey] [~ghelmling] [~anoop.hbase] [~stack]
> Correct RegionServer metric of totalRequestCount
> --
>
> Key: HBASE-18469
> URL: https://issues.apache.org/jira/browse/HBASE-18469
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Shibin Zhang
>Priority: Critical
>
> when i get the metric ,i found this three metric may be have some error as
> follow :
> "totalRequestCount" : 17541,
> "readRequestCount" : 17483,
> "writeRequestCount" : 1633,
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106778#comment-16106778 ] Hudson commented on HBASE-18481: FAILURE: Integrated in Jenkins build HBase-1.2-JDK7 #177 (See [https://builds.apache.org/job/HBase-1.2-JDK7/177/]) HBASE-18481 The autoFlush flag was not used in PE tool (zghao: rev bd86094f3c9337d88597bc01d20e775631188f12) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106776#comment-16106776 ] Hudson commented on HBASE-18481: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #911 (See [https://builds.apache.org/job/HBase-1.2-IT/911/]) HBASE-18481 The autoFlush flag was not used in PE tool (zghao: rev bd86094f3c9337d88597bc01d20e775631188f12) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106775#comment-16106775 ] Hudson commented on HBASE-18481: FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #211 (See [https://builds.apache.org/job/HBase-1.3-JDK7/211/]) HBASE-18481 The autoFlush flag was not used in PE tool (zghao: rev d666ffc7ad014f21b32cd5d5f7666506aede30b7) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106773#comment-16106773 ] Hudson commented on HBASE-18481: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #149 (See [https://builds.apache.org/job/HBase-1.3-IT/149/]) HBASE-18481 The autoFlush flag was not used in PE tool (zghao: rev d666ffc7ad014f21b32cd5d5f7666506aede30b7) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18481: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.1.12 2.0.0-alpha-2 1.2.7 1.5.0 1.3.2 1.4.0 3.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2 and branch-1.1. Thanks [~stack] for reviewing. > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18469) Correct RegionServer metric of totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106726#comment-16106726 ] Shibin Zhang commented on HBASE-18469: -- [~abhishek.chouhan] the totalRequestCount of a RegionServer should be large than the sum of readRequestcount and writeRequestcount > Correct RegionServer metric of totalRequestCount > -- > > Key: HBASE-18469 > URL: https://issues.apache.org/jira/browse/HBASE-18469 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Shibin Zhang >Priority: Critical > > when i get the metric ,i found this three metric may be have some error as > follow : > "totalRequestCount" : 17541, > "readRequestCount" : 17483, > "writeRequestCount" : 1633, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18469) Correct RegionServer metric of totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shibin Zhang updated HBASE-18469: - Priority: Critical (was: Minor) > Correct RegionServer metric of totalRequestCount > -- > > Key: HBASE-18469 > URL: https://issues.apache.org/jira/browse/HBASE-18469 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Shibin Zhang >Priority: Critical > > when i get the metric ,i found this three metric may be have some error as > follow : > "totalRequestCount" : 17541, > "readRequestCount" : 17483, > "writeRequestCount" : 1633, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18465) [AMv2] remove old split region code that is no longer needed
[ https://issues.apache.org/jira/browse/HBASE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106715#comment-16106715 ] Hudson commented on HBASE-18465: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3464 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3464/]) HBASE-18465: [AMv2] remove old split region code that is no longer (stack: rev 00c1b566658aa2fecde4ce2dbde2464f5771430e) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicator.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AdminProtos.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AnnotationReadingPriorityFunction.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java * (edit) hbase-protocol-shaded/src/main/protobuf/Admin.proto * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java > [AMv2] remove old split region code that is no longer needed > > > Key: HBASE-18465 > URL: https://issues.apache.org/jira/browse/HBASE-18465 > Project: HBase > Issue Type: Task > Components: Admin, rpc >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-18465-master-v1.patch > > > After HBASE-18229, Admin.split go to MasterRpcServices directly, so need to > clean up the old logic going through RSRpcServices. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18465) [AMv2] remove old split region code that is no longer needed
[ https://issues.apache.org/jira/browse/HBASE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106708#comment-16106708 ] Yi Liang commented on HBASE-18465: -- Thanks for reviewing and pushing. [~stack] > [AMv2] remove old split region code that is no longer needed > > > Key: HBASE-18465 > URL: https://issues.apache.org/jira/browse/HBASE-18465 > Project: HBase > Issue Type: Task > Components: Admin, rpc >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-18465-master-v1.patch > > > After HBASE-18229, Admin.split go to MasterRpcServices directly, so need to > clean up the old logic going through RSRpcServices. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4
[
https://issues.apache.org/jira/browse/HBASE-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106693#comment-16106693
]
Andrew Purtell commented on HBASE-18431:
Thanks [~anoop.hbase]. Then I think it would be reasonable for branch-1.4 and
up to add exactly the 2.0 admin APIs for snapshot and attach deprecation tags
to the others. If there are no objections to this approach I will add this to
the current patch on this issue, commit to branch-1.4 and branch-1, and call it
a day.
> Mitigate compatibility concerns between branch-1.3 and branch-1.4
> -
>
> Key: HBASE-18431
> URL: https://issues.apache.org/jira/browse/HBASE-18431
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0, 1.5.0
>
> Attachments: HBASE-18431-branch-1.4.patch
>
>
> There are compatibility concerns with branch-1.4.
> {noformat}
> Library Name HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject Binary Compatibility
> Compatibility - 89.9%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
> High - 23
> Medium - 9
> Low - 21
> {noformat}
> {noformat}
> Library Name HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject Source Compatibility
> Compatibility- 86.5%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
> High - 88
> Medium - 0
> Low - 0
> Other Changes in Data Types- 25
> {noformat}
> This report includes HBASE-15816 which hasn't been committed yet. Otherwise
> it's current.
> I'm not generally concerned with added methods.
> The following methods have been added to Public/Evolving interface Table.
> Pointing them out in case it merits review.
> \\
> * Abstract method Table.getReadRpcTimeout ( ) has been added to this
> interface. No effect.
> * Abstract method Table.getWriteRpcTimeout ( ) has been added to this
> interface. No effect.
> * Abstract method Table.setReadRpcTimeout ( int ) has been added to this
> interface. No effect.
> * Abstract method Table.setWriteRpcTimeout ( int ) has been added to this
> interface.
> The Public/Evolving interface Admin has some signature changes equating to
> removed methods. I don't think this is allowed in a minor release.
> \\
> * Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription
> ) has been removed from Admin.
> * Abstract method Admin.snapshot ( String, TableName,
> HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
> * Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been
> removed from Admin.
> * Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription
> ) has been removed from Admin.
> The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This
> change is debatable but I think we can allow it.
> \\
> * AsyncRpcClient has been removed
> The Public/Evolving class FastLongHistogram has been removed. I don't believe
> this change is allowed in a minor release.
> \\
> * FastLongHistogram has been removed
> Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and
> RegionObserver have changed, equating to removed methods. The first set of
> changes is due to move of SnapshotDescription from HBaseProtos to
> SnapshotProtos:
> \\
> * Abstract method MasterObserver.postCloneSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.postDeleteSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postListSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postRestoreSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.postSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.preCloneSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.preDeleteSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preListSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preRestoreSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription,
[jira] [Comment Edited] (HBASE-18474) HRegion#doMiniBatchMutation is acquiring read row locks
[
https://issues.apache.org/jira/browse/HBASE-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106682#comment-16106682
]
Andrew Purtell edited comment on HBASE-18474 at 7/30/17 10:34 PM:
--
Thanks Stack. I filed this for discussion because we ran into problems because
exclusive row locking for some coprocessor upcalls in 0.98 went away with this
change, and doMiniBatchMutation with all of its change history is a beast.
Information provided here has been very helpful, thank you.
I've resolved this as Not A Problem.
was (Author: apurtell):
Thanks Stack. I filed this for discussion because I don't claim to understand
everything that has happened to doMiniBatchMutation since 0.98. Information
provided here has been very helpful, thank you.
I've resolved this as Not A Problem.
> HRegion#doMiniBatchMutation is acquiring read row locks
> ---
>
> Key: HBASE-18474
> URL: https://issues.apache.org/jira/browse/HBASE-18474
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>
> Looking at 1.3, HRegion#doMiniBatchMutation is acquiring read row locks in
> step 1.
> {code}
> // If we haven't got any rows in our batch, we should block to
>
> // get the next one.
>
> RowLock rowLock = null;
> try {
> rowLock = getRowLockInternal(mutation.getRow(), true);
> } catch (TimeoutIOException e) {
> // We will retry when other exceptions, but we should stop if we
> timeout .
> throw e;
> } catch (IOException ioe) {
> LOG.warn("Failed getting lock in batch put, row="
> + Bytes.toStringBinary(mutation.getRow()), ioe);
> }
> if (rowLock == null) {
> // We failed to grab another lock
>
> break; // stop acquiring more rows for this batch
>
> } else {
> acquiredRowLocks.add(rowLock);
> }
> {code}
> Other code paths that apply mutations are acquiring write locks.
> In HRegion#append
> {code}
> try {
> rowLock = getRowLockInternal(row, false);
> assert rowLock != null;
> ...
> {code}
> In HRegion#doIn
> {code}
> try {
> rowLock = getRowLockInternal(increment.getRow(), false);
> ...
> {code}
> In HRegion#checkAndMutate
> {code}
> // Lock row - note that doBatchMutate will relock this row if called
>
> RowLock rowLock = getRowLockInternal(get.getRow(), false);
> // wait for all previous transactions to complete (with lock held)
>
> mvcc.await();
> {code}
> In HRegion#processRowsWithLocks
> {code}
> // 2. Acquire the row lock(s)
>
> acquiredRowLocks = new ArrayList(rowsToLock.size());
> for (byte[] row : rowsToLock) {
> // Attempt to lock all involved rows, throw if any lock times out
>
> // use a writer lock for mixed reads and writes
>
> acquiredRowLocks.add(getRowLockInternal(row, false));
> }
> {code}
> and so on.
> What doMiniBatchMutation is doing looks wrong.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18474) HRegion#doMiniBatchMutation is acquiring read row locks
[
https://issues.apache.org/jira/browse/HBASE-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106682#comment-16106682
]
Andrew Purtell commented on HBASE-18474:
Thanks Stack. I filed this for discussion because I don't claim to understand
everything that has happened to doMiniBatchMutation since 0.98. Information
provided here has been very helpful, thank you.
I've resolved this as Not A Problem.
> HRegion#doMiniBatchMutation is acquiring read row locks
> ---
>
> Key: HBASE-18474
> URL: https://issues.apache.org/jira/browse/HBASE-18474
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>
> Looking at 1.3, HRegion#doMiniBatchMutation is acquiring read row locks in
> step 1.
> {code}
> // If we haven't got any rows in our batch, we should block to
>
> // get the next one.
>
> RowLock rowLock = null;
> try {
> rowLock = getRowLockInternal(mutation.getRow(), true);
> } catch (TimeoutIOException e) {
> // We will retry when other exceptions, but we should stop if we
> timeout .
> throw e;
> } catch (IOException ioe) {
> LOG.warn("Failed getting lock in batch put, row="
> + Bytes.toStringBinary(mutation.getRow()), ioe);
> }
> if (rowLock == null) {
> // We failed to grab another lock
>
> break; // stop acquiring more rows for this batch
>
> } else {
> acquiredRowLocks.add(rowLock);
> }
> {code}
> Other code paths that apply mutations are acquiring write locks.
> In HRegion#append
> {code}
> try {
> rowLock = getRowLockInternal(row, false);
> assert rowLock != null;
> ...
> {code}
> In HRegion#doIn
> {code}
> try {
> rowLock = getRowLockInternal(increment.getRow(), false);
> ...
> {code}
> In HRegion#checkAndMutate
> {code}
> // Lock row - note that doBatchMutate will relock this row if called
>
> RowLock rowLock = getRowLockInternal(get.getRow(), false);
> // wait for all previous transactions to complete (with lock held)
>
> mvcc.await();
> {code}
> In HRegion#processRowsWithLocks
> {code}
> // 2. Acquire the row lock(s)
>
> acquiredRowLocks = new ArrayList(rowsToLock.size());
> for (byte[] row : rowsToLock) {
> // Attempt to lock all involved rows, throw if any lock times out
>
> // use a writer lock for mixed reads and writes
>
> acquiredRowLocks.add(getRowLockInternal(row, false));
> }
> {code}
> and so on.
> What doMiniBatchMutation is doing looks wrong.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-18474) HRegion#doMiniBatchMutation is acquiring read row locks
[
https://issues.apache.org/jira/browse/HBASE-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-18474.
Resolution: Not A Problem
> HRegion#doMiniBatchMutation is acquiring read row locks
> ---
>
> Key: HBASE-18474
> URL: https://issues.apache.org/jira/browse/HBASE-18474
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>
> Looking at 1.3, HRegion#doMiniBatchMutation is acquiring read row locks in
> step 1.
> {code}
> // If we haven't got any rows in our batch, we should block to
>
> // get the next one.
>
> RowLock rowLock = null;
> try {
> rowLock = getRowLockInternal(mutation.getRow(), true);
> } catch (TimeoutIOException e) {
> // We will retry when other exceptions, but we should stop if we
> timeout .
> throw e;
> } catch (IOException ioe) {
> LOG.warn("Failed getting lock in batch put, row="
> + Bytes.toStringBinary(mutation.getRow()), ioe);
> }
> if (rowLock == null) {
> // We failed to grab another lock
>
> break; // stop acquiring more rows for this batch
>
> } else {
> acquiredRowLocks.add(rowLock);
> }
> {code}
> Other code paths that apply mutations are acquiring write locks.
> In HRegion#append
> {code}
> try {
> rowLock = getRowLockInternal(row, false);
> assert rowLock != null;
> ...
> {code}
> In HRegion#doIn
> {code}
> try {
> rowLock = getRowLockInternal(increment.getRow(), false);
> ...
> {code}
> In HRegion#checkAndMutate
> {code}
> // Lock row - note that doBatchMutate will relock this row if called
>
> RowLock rowLock = getRowLockInternal(get.getRow(), false);
> // wait for all previous transactions to complete (with lock held)
>
> mvcc.await();
> {code}
> In HRegion#processRowsWithLocks
> {code}
> // 2. Acquire the row lock(s)
>
> acquiredRowLocks = new ArrayList(rowsToLock.size());
> for (byte[] row : rowsToLock) {
> // Attempt to lock all involved rows, throw if any lock times out
>
> // use a writer lock for mixed reads and writes
>
> acquiredRowLocks.add(getRowLockInternal(row, false));
> }
> {code}
> and so on.
> What doMiniBatchMutation is doing looks wrong.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18127) Allow regionobserver to optionally skip postPut/postDelete when postBatchMutate was called
[ https://issues.apache.org/jira/browse/HBASE-18127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106681#comment-16106681 ] Andrew Purtell commented on HBASE-18127: I think making OperationContext a class and avoiding unnecessary Simple* or Default* implementation is better, because that is what we do with ObserverContext already. Just my opinion. > Allow regionobserver to optionally skip postPut/postDelete when > postBatchMutate was called > -- > > Key: HBASE-18127 > URL: https://issues.apache.org/jira/browse/HBASE-18127 > Project: HBase > Issue Type: New Feature >Reporter: Lars Hofhansl >Assignee: Abhishek Singh Chouhan > Attachments: HBASE-18127.master.001.patch > > > Right now a RegionObserver can only statically implement one or the other. In > scenarios where we need to work sometimes on the single postPut and > postDelete hooks and sometimes on the batchMutate hooks, there is currently > no place to convey this information to the single hooks. I.e. the work has > been done in the batch, skip the single hooks. > There are various solutions: > 1. Allow some state to be passed _per operation_. > 2. Remove the single hooks and always only call batch hooks (with a default > wrapper for the single hooks). > 3. more? > [~apurtell], what we had discussed a few days back. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18482) Enable state to be passed between the batch mutate coprocessors
[
https://issues.apache.org/jira/browse/HBASE-18482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106678#comment-16106678
]
Andrew Purtell commented on HBASE-18482:
On HBASE-18127 [~abhishek.chouhan] is already pursuing something along these
lines. His preliminary patch introduces {{OperationCoprocessorContext}} which
is passed among all the coprocessor hooks which might be invoked during the
processing of a batch mutation op.
> Enable state to be passed between the batch mutate coprocessors
> ---
>
> Key: HBASE-18482
> URL: https://issues.apache.org/jira/browse/HBASE-18482
> Project: HBase
> Issue Type: Improvement
>Reporter: James Taylor
>
> For secondary indexing, Phoenix leverages the coprocessors that fire during
> the processing of a batch mutate: preBatchMutate, postBatchMutate, and
> postBatchMutateIndispensably. It would be very helpful if we had a way of
> passing state between these calls. One solution would be to provide a member
> variable in the MiniBatchOperationInProgress class that gets passed from
> invocation to invocation. Within doMiniBatchMutation, the instantiation of
> the MiniBatchOperationInProgress would simply set this context member
> variable based on the value from the previous MiniBatchOperationInProgress
> instantiation. Another solution would be to allow the a context object to be
> set on the ObserverContext which would then be available to the other calls.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18465) [AMv2] remove old split region code that is no longer needed
[ https://issues.apache.org/jira/browse/HBASE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106675#comment-16106675 ] Hudson commented on HBASE-18465: FAILURE: Integrated in Jenkins build HBase-2.0 #263 (See [https://builds.apache.org/job/HBase-2.0/263/]) HBASE-18465: [AMv2] remove old split region code that is no longer (stack: rev 928137c1ce15af8896a6d0329bfebec51c1912f3) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AnnotationReadingPriorityFunction.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicator.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AdminProtos.java * (edit) hbase-protocol-shaded/src/main/protobuf/Admin.proto > [AMv2] remove old split region code that is no longer needed > > > Key: HBASE-18465 > URL: https://issues.apache.org/jira/browse/HBASE-18465 > Project: HBase > Issue Type: Task > Components: Admin, rpc >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-18465-master-v1.patch > > > After HBASE-18229, Admin.split go to MasterRpcServices directly, so need to > clean up the old logic going through RSRpcServices. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14414) HBase Backup/Restore Phase 3
[ https://issues.apache.org/jira/browse/HBASE-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1610#comment-1610 ] stack commented on HBASE-14414: --- How do I figure what of backup/restore feature is going to be in hbase-2.0.0? Thanks [~vrodionov]. > HBase Backup/Restore Phase 3 > > > Key: HBASE-14414 > URL: https://issues.apache.org/jira/browse/HBASE-14414 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > > Umbrella ticket for Phase 3 of development -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106665#comment-16106665 ] stack commented on HBASE-17614: --- This issue is a blocker for 2.0.0. We can't ship a 2.0.0 w/ hbase-server bulked up w/ backup classes. > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0, HBASE-7912 > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17614: -- Fix Version/s: 2.0.0 > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0, HBASE-7912 > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17614: -- Priority: Blocker (was: Major) > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0, HBASE-7912 > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14414) HBase Backup/Restore Phase 3
[ https://issues.apache.org/jira/browse/HBASE-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106662#comment-16106662 ] stack commented on HBASE-14414: --- Shouldn't there be a link here to phase 2 and phase 1 if only for easier navigation. I'm trying to see what is going on in here from a high-level. Thanks. > HBase Backup/Restore Phase 3 > > > Key: HBASE-14414 > URL: https://issues.apache.org/jira/browse/HBASE-14414 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > > Umbrella ticket for Phase 3 of development -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106658#comment-16106658 ] stack commented on HBASE-17442: --- Good stuff. Let me take on starting a DISCUSS thread on dev. Yeah, I think it too much to expect folks to move to latest branch-1. I think it ok that we not allow update from 0.98 say to 2.0 w/o going through branch-1 but let me post this on dev list and see what others think. > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.v1.patch, HBASE-17442.v2.patch, HBASE-17442.v2.patch, > HBASE-17442.v3.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106657#comment-16106657 ] stack commented on HBASE-18366: --- Why this change sir: optional ServerName destination_server = 3; It was required. A move w/o a destination doesn't make sense? I like the change that foists assignment on to the balancer (by passing null) presuming balancer will favor regionservers with higher versions High-level, how would you describe the change [~uagashe]? I like the bit of having assign conscious of favoring system tables... Patch looking good. What else is to be done? Thanks boss. > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > Attachments: hbase-18366.fix1.patch, hbase-18366.fix2.patch > > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18366) Fix flaky test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
[ https://issues.apache.org/jira/browse/HBASE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106652#comment-16106652 ] stack commented on HBASE-18366: --- Nice writeup @umesh. Added a few comments. Let me look at patches. > Fix flaky test > hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta > - > > Key: HBASE-18366 > URL: https://issues.apache.org/jira/browse/HBASE-18366 > Project: HBase > Issue Type: Bug >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > Attachments: hbase-18366.fix1.patch, hbase-18366.fix2.patch > > > It worked for a few days after enabling it with HBASE-18278. But started > failing after commits: > 6786b2b > 68436c9 > 75d2eca > 50bb045 > df93c13 > It works with one commit before: c5abb6c. Need to see what changed with those > commits. > Currently it fails with TableNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18475) MasterProcedureScheduler incorrectly passes null Procedure to table locking
[
https://issues.apache.org/jira/browse/HBASE-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106647#comment-16106647
]
stack commented on HBASE-18475:
---
+1
> MasterProcedureScheduler incorrectly passes null Procedure to table locking
> ---
>
> Key: HBASE-18475
> URL: https://issues.apache.org/jira/browse/HBASE-18475
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18475.0.patch
>
>
> Occasionally I get a series of these in tests during shutdown
> {code}
> 2017-07-27 16:24:26,774 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=37201]
> master.MasterRpcServices(1011): Checking to see if procedure is done
> procId=506
> 2017-07-27 16:24:26,777 INFO [main] client.HBaseAdmin$TableFuture(3591):
> Operation: CREATE, Table Name: default:foo failed with foo
> 2017-07-27 16:24:26,782 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=37201]
> master.HMaster$4(1755): Client=jenkins//172.26.21.67 create 'foo', {NAME =>
> 'family_1', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', CACHE_DATA_IN_L1
> => 'false', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
> 2017-07-27 16:24:26,884 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=37201]
> procedure2.ProcedureExecutor(792): Stored pid=507,
> state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=foo
> 2017-07-27 16:24:26,887 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=37201]
> master.MasterRpcServices(1011): Checking to see if procedure is done
> procId=507
> 2017-07-27 16:24:26,890 INFO [ProcExecWrkr-5]
> procedure2.ProcedureExecutor(1261): Rolled back pid=507, state=ROLLEDBACK,
> exception=org.apache.hadoop.hbase.TableExistsException via
> master-create-table:org.apache.hadoop.hbase.TableExistsException: foo;
> CreateTableProcedure table=foo exec-time=106msec
> 2017-07-27 16:24:26,890 WARN [ProcExecWrkr-5]
> procedure2.ProcedureExecutor$WorkerThread(1668): Worker terminating
> UNNATURALLY null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.requireTableExclusiveLock(MasterProcedureScheduler.java:590)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.access$300(MasterProcedureScheduler.java:106)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler$TableQueue.requireExclusiveLock(MasterProcedureScheduler.java:582)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.doPoll(MasterProcedureScheduler.java:215)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.dequeue(MasterProcedureScheduler.java:203)
> at
> org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:145)
> at
> org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:119)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1641)
> 2017-07-27 16:24:26,890 DEBUG [ProcExecWrkr-5]
> procedure2.ProcedureExecutor$WorkerThread(1670): Worker terminated.
> {code}
> Eventually all the workers are done.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18465) [AMv2] remove old split region code that is no longer needed
[ https://issues.apache.org/jira/browse/HBASE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18465: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 3.0.0) Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thanks for the lovely cleanup [~easyliangjob] > [AMv2] remove old split region code that is no longer needed > > > Key: HBASE-18465 > URL: https://issues.apache.org/jira/browse/HBASE-18465 > Project: HBase > Issue Type: Task > Components: Admin, rpc >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-18465-master-v1.patch > > > After HBASE-18229, Admin.split go to MasterRpcServices directly, so need to > clean up the old logic going through RSRpcServices. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18478) Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
[ https://issues.apache.org/jira/browse/HBASE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106643#comment-16106643 ] stack commented on HBASE-18478: --- Patch seems fine. This only use of RegionFinder in the code base? [~zyork] > Allow users to remove RegionFinder from LoadBalancer calculations if no > locality possible > - > > Key: HBASE-18478 > URL: https://issues.apache.org/jira/browse/HBASE-18478 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18478.branch-1.001.patch, > HBASE-18478.master.001.patch > > > BaseLoadBalancer should have the option to remove RegionFinder from load > balancing. This provides significant cluster start time reduction for > FileSystems that do not surface locality such as Amazon S3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18465) [AMv2] remove old split region code that is no longer needed
[
https://issues.apache.org/jira/browse/HBASE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106642#comment-16106642
]
Hadoop QA commented on HBASE-18465:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
58s{color} | {color:red} hbase-protocol-shaded in master has 27 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 15s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}111m
39s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}193m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18465 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879538/HBASE-18465-master-v1.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile cc hbaseprotoc |
| uname | Linux 69da41e60c84 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106639#comment-16106639 ] stack commented on HBASE-18142: --- [~chia7712] I made you a JIRA admin for HBASE project so you can do this stuff yourself going forward. > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumns(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106637#comment-16106637 ] stack commented on HBASE-18480: --- [~chia7712] where is the 'undo' happening sir? > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.0, 1.2.6, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.branch-1.v0.patch, HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18482) Enable state to be passed between the batch mutate coprocessors
[ https://issues.apache.org/jira/browse/HBASE-18482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106636#comment-16106636 ] James Taylor commented on HBASE-18482: -- Any memory allocated would go out of scope when doMiniBatchMutation is completed. It really be no different than other memory allocation done within the coprocessor hook calls. In general, you always have to be aware of allocated huge amounts of memory. > Enable state to be passed between the batch mutate coprocessors > --- > > Key: HBASE-18482 > URL: https://issues.apache.org/jira/browse/HBASE-18482 > Project: HBase > Issue Type: Improvement >Reporter: James Taylor > > For secondary indexing, Phoenix leverages the coprocessors that fire during > the processing of a batch mutate: preBatchMutate, postBatchMutate, and > postBatchMutateIndispensably. It would be very helpful if we had a way of > passing state between these calls. One solution would be to provide a member > variable in the MiniBatchOperationInProgress class that gets passed from > invocation to invocation. Within doMiniBatchMutation, the instantiation of > the MiniBatchOperationInProgress would simply set this context member > variable based on the value from the previous MiniBatchOperationInProgress > instantiation. Another solution would be to allow the a context object to be > set on the ObserverContext which would then be available to the other calls. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106635#comment-16106635 ] stack commented on HBASE-18481: --- +1 > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17056) Remove checked in PB generated files
[ https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106634#comment-16106634 ] stack commented on HBASE-17056: --- .004 is rebase after guava patch went in. Lets see how we do. Might break this into smaller pieces if it don't go in easy. > Remove checked in PB generated files > - > > Key: HBASE-17056 > URL: https://issues.apache.org/jira/browse/HBASE-17056 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Enis Soztutar >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: > 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, > HBASE-17056.master.001.patch, HBASE-17056.master.002.patch, > HBASE-17056.master.003.patch, HBASE-17056.master.004.patch > > > Now that we have the new PB maven plugin, there is no need to have the PB > files checked in to the repo. The reason we did that was to ease up developer > env setup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18465) [AMv2] remove old split region code that is no longer needed
[ https://issues.apache.org/jira/browse/HBASE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106633#comment-16106633 ] stack commented on HBASE-18465: --- +1 Beautiful patch full of purged code. Will commit after unit tests look better. > [AMv2] remove old split region code that is no longer needed > > > Key: HBASE-18465 > URL: https://issues.apache.org/jira/browse/HBASE-18465 > Project: HBase > Issue Type: Task > Components: Admin, rpc >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18465-master-v1.patch > > > After HBASE-18229, Admin.split go to MasterRpcServices directly, so need to > clean up the old logic going through RSRpcServices. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18482) Enable state to be passed between the batch mutate coprocessors
[ https://issues.apache.org/jira/browse/HBASE-18482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106624#comment-16106624 ] Ted Yu commented on HBASE-18482: For #1, what if the member variable happens to have large memory footprint ? > Enable state to be passed between the batch mutate coprocessors > --- > > Key: HBASE-18482 > URL: https://issues.apache.org/jira/browse/HBASE-18482 > Project: HBase > Issue Type: Improvement >Reporter: James Taylor > > For secondary indexing, Phoenix leverages the coprocessors that fire during > the processing of a batch mutate: preBatchMutate, postBatchMutate, and > postBatchMutateIndispensably. It would be very helpful if we had a way of > passing state between these calls. One solution would be to provide a member > variable in the MiniBatchOperationInProgress class that gets passed from > invocation to invocation. Within doMiniBatchMutation, the instantiation of > the MiniBatchOperationInProgress would simply set this context member > variable based on the value from the previous MiniBatchOperationInProgress > instantiation. Another solution would be to allow the a context object to be > set on the ObserverContext which would then be available to the other calls. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15806) An endpoint-based export tool
[ https://issues.apache.org/jira/browse/HBASE-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106620#comment-16106620 ] stack commented on HBASE-15806: --- Oh, I have reviewed it up on rb... FYI [~chia7712] > An endpoint-based export tool > - > > Key: HBASE-15806 > URL: https://issues.apache.org/jira/browse/HBASE-15806 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 2.0.0 > > Attachments: Experiment.png, HBASE-15806.patch, HBASE-15806-v1.patch, > HBASE-15806-v2.patch, HBASE-15806-v3.patch, HBASE-15806.v4.patch, > HBASE-15806.v5.patch, HBASE-15806.v6.patch, HBASE-15806.v7.patch > > > The time for exporting table can be reduced, if we use the endpoint technique > to export the hdfs files by the region server rather than by hbase client. > In my experiments, the elapsed time of endpoint-based export can be less than > half of current export tool (enable the hdfs compression) > But the shortcomings is we need to alter table for deploying the endpoint > any comments about this? thanks -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15806) An endpoint-based export tool
[ https://issues.apache.org/jira/browse/HBASE-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106617#comment-16106617 ] stack commented on HBASE-15806: --- [~chia7712] I tried to apply the patch but it fails now... sorry, I let it rot. Let me review... > An endpoint-based export tool > - > > Key: HBASE-15806 > URL: https://issues.apache.org/jira/browse/HBASE-15806 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 2.0.0 > > Attachments: Experiment.png, HBASE-15806.patch, HBASE-15806-v1.patch, > HBASE-15806-v2.patch, HBASE-15806-v3.patch, HBASE-15806.v4.patch, > HBASE-15806.v5.patch, HBASE-15806.v6.patch, HBASE-15806.v7.patch > > > The time for exporting table can be reduced, if we use the endpoint technique > to export the hdfs files by the region server rather than by hbase client. > In my experiments, the elapsed time of endpoint-based export can be less than > half of current export tool (enable the hdfs compression) > But the shortcomings is we need to alter table for deploying the endpoint > any comments about this? thanks -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18406) In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op
[ https://issues.apache.org/jira/browse/HBASE-18406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106589#comment-16106589 ] Hudson commented on HBASE-18406: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3462 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3462/]) HBASE-18406 Remove NO-OP Method (stack: rev 0e9390bd6d8a4ad41e48015556786dda9fbdb73b) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java > In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op > - > > Key: HBASE-18406 > URL: https://issues.apache.org/jira/browse/HBASE-18406 > Project: HBase > Issue Type: Bug >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18406.master.001.patch, > HBASE-18406.master.002.patch > > > The comments above this method explain that it exists to set configs and > return, however, no configs are set in the method. > As you can see here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L210-L214 > > It is only ever called here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L142 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18482) Enable state to be passed between the batch mutate coprocessors
James Taylor created HBASE-18482: Summary: Enable state to be passed between the batch mutate coprocessors Key: HBASE-18482 URL: https://issues.apache.org/jira/browse/HBASE-18482 Project: HBase Issue Type: Improvement Reporter: James Taylor For secondary indexing, Phoenix leverages the coprocessors that fire during the processing of a batch mutate: preBatchMutate, postBatchMutate, and postBatchMutateIndispensably. It would be very helpful if we had a way of passing state between these calls. One solution would be to provide a member variable in the MiniBatchOperationInProgress class that gets passed from invocation to invocation. Within doMiniBatchMutation, the instantiation of the MiniBatchOperationInProgress would simply set this context member variable based on the value from the previous MiniBatchOperationInProgress instantiation. Another solution would be to allow the a context object to be set on the ObserverContext which would then be available to the other calls. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18465) [AMv2] remove old split region code that is no longer needed
[ https://issues.apache.org/jira/browse/HBASE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18465: - Attachment: HBASE-18465-master-v1.patch > [AMv2] remove old split region code that is no longer needed > > > Key: HBASE-18465 > URL: https://issues.apache.org/jira/browse/HBASE-18465 > Project: HBase > Issue Type: Task > Components: Admin, rpc >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18465-master-v1.patch > > > After HBASE-18229, Admin.split go to MasterRpcServices directly, so need to > clean up the old logic going through RSRpcServices. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18465) [AMv2] remove old split region code that is no longer needed
[ https://issues.apache.org/jira/browse/HBASE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18465: - Attachment: (was: HBASE-18465-master-v1.patch) > [AMv2] remove old split region code that is no longer needed > > > Key: HBASE-18465 > URL: https://issues.apache.org/jira/browse/HBASE-18465 > Project: HBase > Issue Type: Task > Components: Admin, rpc >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0, 3.0.0 > > > After HBASE-18229, Admin.split go to MasterRpcServices directly, so need to > clean up the old logic going through RSRpcServices. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17056) Remove checked in PB generated files
[ https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17056: -- Attachment: HBASE-17056.master.004.patch > Remove checked in PB generated files > - > > Key: HBASE-17056 > URL: https://issues.apache.org/jira/browse/HBASE-17056 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Enis Soztutar >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: > 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, > HBASE-17056.master.001.patch, HBASE-17056.master.002.patch, > HBASE-17056.master.003.patch, HBASE-17056.master.004.patch > > > Now that we have the new PB maven plugin, there is no need to have the PB > files checked in to the repo. The reason we did that was to ease up developer > env setup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18406) In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op
[ https://issues.apache.org/jira/browse/HBASE-18406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106534#comment-16106534 ] Hudson commented on HBASE-18406: FAILURE: Integrated in Jenkins build HBase-2.0 #261 (See [https://builds.apache.org/job/HBase-2.0/261/]) HBASE-18406 Remove NO-OP Method (stack: rev dbc8ea64862935c27414beaae73f12c25b56431c) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java > In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op > - > > Key: HBASE-18406 > URL: https://issues.apache.org/jira/browse/HBASE-18406 > Project: HBase > Issue Type: Bug >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18406.master.001.patch, > HBASE-18406.master.002.patch > > > The comments above this method explain that it exists to set configs and > return, however, no configs are set in the method. > As you can see here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L210-L214 > > It is only ever called here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L142 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106533#comment-16106533
]
Hudson commented on HBASE-18473:
FAILURE: Integrated in Jenkins build HBase-2.0 #261 (See
[https://builds.apache.org/job/HBase-2.0/261/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev d776a3caae935f3e08d8a8a5d2ffdaf23cb9b41f)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Reopened] (HBASE-18259) HBase book link to "beginner" issues includes resolved issues
[ https://issues.apache.org/jira/browse/HBASE-18259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened HBASE-18259: > HBase book link to "beginner" issues includes resolved issues > - > > Key: HBASE-18259 > URL: https://issues.apache.org/jira/browse/HBASE-18259 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Mike Drob >Assignee: Peter Somogyi > Labels: beginner > Fix For: 3.0.0 > > Attachments: HBASE-18259.master.001.patch > > > The link at http://hbase.apache.org/book.html#getting.involved for beginner > issues is > https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20labels%20in%20(beginner) > but this includes resolved issues as well, which is not useful to folks > looking for new issues to cut their teeth on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17658) Bookkeeping issue in BaseLoadBalancer for Table Skew
[ https://issues.apache.org/jira/browse/HBASE-17658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106519#comment-16106519 ] Chia-Ping Tsai commented on HBASE-17658: [~yuzhih...@gmail.com] Why not push this to the other active 1.y branches? > Bookkeeping issue in BaseLoadBalancer for Table Skew > > > Key: HBASE-17658 > URL: https://issues.apache.org/jira/browse/HBASE-17658 > Project: HBase > Issue Type: Bug > Components: Balancer >Reporter: Tim Brown >Assignee: Tim Brown > Fix For: 2.0.0, 1.4.0 > > Attachments: JIRA-17658.master.001.patch > > > Currently when numMaxRegionsPerTable of a given table increases above the > current maximum, the value is not properly updated. This means that the cost > for Table Skew cannot get worse for a given region move during the balancer > process leading to an imbalanced cluster with respect to Table Skew. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18213) Add documentation about the new async client
[ https://issues.apache.org/jira/browse/HBASE-18213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18213: -- Fix Version/s: (was: 2.0.0-alpha-2) > Add documentation about the new async client > > > Key: HBASE-18213 > URL: https://issues.apache.org/jira/browse/HBASE-18213 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, documentation >Affects Versions: 3.0.0, 2.0.0-alpha-1 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 3.0.0 > > Attachments: HBASE-18213.patch, HBASE-18213-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18481) The autoFlush flag was not used in PE tool
[
https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106513#comment-16106513
]
Hadoop QA commented on HBASE-18481:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}133m
55s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}179m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18481 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879527/HBASE-18481.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux bec8ee849a7a 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5c47cb5 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7844/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7844/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> The autoFlush flag was not used in PE tool
> --
>
> Key: HBASE-18481
> URL: https://issues.apache.org/jira/browse/HBASE-18481
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-18481.patch
>
>
> After HBASE-12728, PE used the BufferedMutator for random/sequential
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106495#comment-16106495
]
Hudson commented on HBASE-18473:
SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #225 (See
[https://builds.apache.org/job/HBase-1.3-JDK8/225/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev 3e450f7a077632902fafa43dc7d9afa462125060)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-6322) Unnecessary creation of finalizers in HTablePool
[ https://issues.apache.org/jira/browse/HBASE-6322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6322. -- Resolution: Invalid Resolving as no longer valid (assigning to Chia...) > Unnecessary creation of finalizers in HTablePool > > > Key: HBASE-6322 > URL: https://issues.apache.org/jira/browse/HBASE-6322 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.92.0, 0.92.1, 0.94.0 >Reporter: Ryan Brush > Attachments: HBASE-6322-0.92.1.patch, HBASE-6322-trunk.1.patch > > > From a mailing list question: > While generating some load against a library that makes extensive use of > HTablePool in 0.92, I noticed that the largest heap consumer was > java.lang.ref.Finalizer. Digging in, I discovered that HTablePool's internal > PooledHTable extends HTable, which instantiates a ThreadPoolExecutor and > supporting objects every time a pooled HTable is retrieved. Since > ThreadPoolExecutor has a finalizer, it and its dependencies can't get garbage > collected until the finalizer runs. The result is by using HTablePool, we're > creating a ton of objects to be finalized that are stuck on the heap longer > than they should be, creating our largest source of pressure on the garbage > collector. It looks like this will also be a problem in 0.94 and trunk. > The easy fix is just to have PooledHTable implement HTableInterface (rather > than subclass HTable), but this does break a unit test that explicitly checks > that PooledHTable implements HTable -- I can only assume this test is there > for some historical passivity reason. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-6903) HLog.Entry implements Writable; change to pb
[ https://issues.apache.org/jira/browse/HBASE-6903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6903. -- Resolution: Invalid Assignee: Chia-Ping Tsai Resolving as no longer relevant. Assigning to Chia > HLog.Entry implements Writable; change to pb > > > Key: HBASE-6903 > URL: https://issues.apache.org/jira/browse/HBASE-6903 > Project: HBase > Issue Type: Task > Components: wal >Reporter: stack >Assignee: Chia-Ping Tsai > > Can we do this in a way that makes it so even after 0.96, we can read old > WALs? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106491#comment-16106491
]
Hudson commented on HBASE-18473:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3461 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3461/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev 5c47cb5d491f78c00df0b09ed3899c9334c7fd85)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18406) In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op
[ https://issues.apache.org/jira/browse/HBASE-18406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18406: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-2. Thanks for cleanup @alex > In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op > - > > Key: HBASE-18406 > URL: https://issues.apache.org/jira/browse/HBASE-18406 > Project: HBase > Issue Type: Bug >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18406.master.001.patch, > HBASE-18406.master.002.patch > > > The comments above this method explain that it exists to set configs and > return, however, no configs are set in the method. > As you can see here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L210-L214 > > It is only ever called here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L142 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106488#comment-16106488
]
Hudson commented on HBASE-18473:
SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #171 (See
[https://builds.apache.org/job/HBase-1.2-JDK8/171/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev 79c8031ebf2d11214f60a8fd0bf32195cd44c273)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[
https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106485#comment-16106485
]
Hadoop QA commented on HBASE-18480:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
27s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
24s{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_131.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_131. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
28s{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_131.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 28s{color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
52s{color} | {color:red} The patch causes 31 errors with Hadoop v2.4.0. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
44s{color} | {color:red} The patch causes 31 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
35s{color} | {color:red} The patch causes 31 errors with Hadoop v2.5.0. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
26s{color} | {color:red} The patch causes 31 errors with Hadoop v2.5.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
16s{color} | {color:red} The patch causes 31 errors with Hadoop v2.5.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
8s{color} | {color:red} The patch causes 31 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
1s{color} | {color:red} The patch causes 31 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
52s{color} | {color:red} The patch causes 31 errors with Hadoop v2.6.3. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m
45s{color} | {color:red} The patch causes 31 errors with Hadoop v2.7.1. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
21s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK
[jira] [Commented] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106478#comment-16106478 ] Ted Yu commented on HBASE-18480: Lgtm > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.0, 1.2.6, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.branch-1.v0.patch, HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106475#comment-16106475 ] Chia-Ping Tsai commented on HBASE-18480: The branch-1.v0.patch is a mistaken patch...Will re-submit it after QA > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.0, 1.2.6, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.branch-1.v0.patch, HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations
[ https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106474#comment-16106474 ] stack commented on HBASE-15816: --- [~churromorales] Hey boss, a nice change like this needs a release note... say how to enable/use -- high-level. Thanks. > Provide client with ability to set priority on Operations > -- > > Key: HBASE-15816 > URL: https://issues.apache.org/jira/browse/HBASE-15816 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, > HBASE-15816-v1.patch, HBASE-15816.v2.patch > > > First round will just be to expose the ability to set priorities for client > operations. For more background: > http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E > Next step would be to remove AnnotationReadingPriorityFunction and have the > client send priorities explicitly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18480: --- Status: Open (was: Patch Available) > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-1, 1.2.6, 1.3.0, 3.0.0 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.branch-1.v0.patch, HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18480: --- Attachment: HBASE-18480.branch-1.v0.patch > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.0, 1.2.6, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.branch-1.v0.patch, HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18480: --- Status: Patch Available (was: Open) > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-1, 1.2.6, 1.3.0, 3.0.0 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.branch-1.v0.patch, HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18480: --- Affects Version/s: (was: 1.4.0) 1.3.0 1.2.6 > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.3.0, 1.2.6, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.branch-1.v0.patch, HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18480: --- Fix Version/s: 1.5.0 1.4.0 > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.4.0, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18480: --- Fix Version/s: 1.2.7 1.3.2 > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.4.0, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[ https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18480: --- Affects Version/s: 1.4.0 > The cost of BaseLoadBalancer.cluster is changed even if the rollback is done > > > Key: HBASE-18480 > URL: https://issues.apache.org/jira/browse/HBASE-18480 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.4.0, 2.0.0-alpha-1 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18480.v0.patch > > > If the cost of cluster isn't restored after the undo action, > StochasticLoadBalancer will be hard to generate the "good" action to balance > the cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15248) BLOCKSIZE 4k should result in 4096 bytes on disk; i.e. fit inside a BucketCache 'block' of 4k
[ https://issues.apache.org/jira/browse/HBASE-15248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106470#comment-16106470 ] stack commented on HBASE-15248: --- That is right [~anoop.hbase] There is tail data too... the CRCs. This issue is about "...what it would take to stay inside our configured size boundary writing out blocks " or ".give back better signal on what to do so you could fit inside a particular constraint." ... so if user says 4k, then we'd only write 4k blocks (the 4k would include metadata and CRCs...). As [~ndimiduk] says, the 4k doesn't count compression + encryption.. but maybe a first pass would fix case when not compression or encoding? > BLOCKSIZE 4k should result in 4096 bytes on disk; i.e. fit inside a > BucketCache 'block' of 4k > - > > Key: HBASE-15248 > URL: https://issues.apache.org/jira/browse/HBASE-15248 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Reporter: stack > > Chatting w/ a gentleman named Daniel Pol who is messing w/ bucketcache, he > wants blocks to be the size specified in the configuration and no bigger. His > hardware set ups fetches pages of 4k and so a block that has 4k of payload > but has then a header and the header of the next block (which helps figure > whats next when scanning) ends up being 4203 bytes or something, and this > then then translates into two seeks per block fetch. > This issue is about what it would take to stay inside our configured size > boundary writing out blocks. > If not possible, give back better signal on what to do so you could fit > inside a particular constraint. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18480) The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
[
https://issues.apache.org/jira/browse/HBASE-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106469#comment-16106469
]
Hadoop QA commented on HBASE-18480:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 47s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}112m
52s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}159m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18480 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879525/HBASE-18480.v0.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 15ff5f94bf7e 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5c47cb5 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7843/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7843/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
>
>
> Key: HBASE-18480
> URL: https://issues.apache.org/jira/browse/HBASE-18480
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 3.0.0,
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106468#comment-16106468
]
Hudson commented on HBASE-18473:
FAILURE: Integrated in Jenkins build HBase-1.4 #827 (See
[https://builds.apache.org/job/HBase-1.4/827/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev cef9fffa3906eb8b01f0f2d76a39d9832a3d6fe6)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-2256) Delete row, followed quickly to put of the same row will sometimes fail.
[ https://issues.apache.org/jira/browse/HBASE-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106463#comment-16106463 ] stack commented on HBASE-2256: -- HBASE-15968 has been resolved. It adds option to enable change in comparator which should fix this issue. > Delete row, followed quickly to put of the same row will sometimes fail. > > > Key: HBASE-2256 > URL: https://issues.apache.org/jira/browse/HBASE-2256 > Project: HBase > Issue Type: Bug >Affects Versions: 0.20.3 >Reporter: Clint Morgan > Attachments: hbase-2256.patch > > > Doing a Delete of a whole row, followed immediately by a put to that row will > sometimes miss a cell. Attached is a test to provoke the issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-9879) Can't undelete a KeyValue
[ https://issues.apache.org/jira/browse/HBASE-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106462#comment-16106462 ] stack commented on HBASE-9879: -- HBASE-15968 adds an option on a table to change the compare in hbase-2. Enabling it addresses this issue. > Can't undelete a KeyValue > - > > Key: HBASE-9879 > URL: https://issues.apache.org/jira/browse/HBASE-9879 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0, 2.0.0 >Reporter: Benoit Sigoure > > Test scenario: > put(KV, timestamp=100) > put(KV, timestamp=200) > delete(KV, timestamp=200, with MutationProto.DeleteType.DELETE_ONE_VERSION) > get(KV) => returns value at timestamp=100 (OK) > put(KV, timestamp=200) > get(KV) => returns value at timestamp=100 (but not the one at timestamp=200 > that was "reborn" by the previous put) > Is that normal? > I ran into this bug while running the integration tests at > https://github.com/OpenTSDB/asynchbase/pull/60 – the first time you run it, > it passes, but after that, it keeps failing. Sorry I don't have the > corresponding HTable-based code but that should be fairly easy to write. > I only tested this with 0.96.0, dunno yet how this behaved in prior releases. > My hunch is that the tombstone added by the DELETE_ONE_VERSION keeps > shadowing the value even after it's reborn. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15968) New behavior of versions considering mvcc and ts rather than ts only
[
https://issues.apache.org/jira/browse/HBASE-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106461#comment-16106461
]
stack commented on HBASE-15968:
---
This is excellent.
> New behavior of versions considering mvcc and ts rather than ts only
>
>
> Key: HBASE-15968
> URL: https://issues.apache.org/jira/browse/HBASE-15968
> Project: HBase
> Issue Type: New Feature
>Reporter: Phil Yang
>Assignee: Phil Yang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-15968.v05.patch, HBASE-15968.v06.patch,
> HBASE-15968.v07.patch, HBASE-15968.v07.patch, HBASE-15968-v1.patch,
> HBASE-15968-v2.patch, HBASE-15968-v3.patch, HBASE-15968-v4.patch
>
>
> In HBase book, we have a section in Versions called "Current Limitations" see
> http://hbase.apache.org/book.html#_current_limitations
> {quote}
> 28.3. Current Limitations
> 28.3.1. Deletes mask Puts
> Deletes mask puts, even puts that happened after the delete was entered. See
> HBASE-2256. Remember that a delete writes a tombstone, which only disappears
> after then next major compaction has run. Suppose you do a delete of
> everything ⇐ T. After this you do a new put with a timestamp ⇐ T. This put,
> even if it happened after the delete, will be masked by the delete tombstone.
> Performing the put will not fail, but when you do a get you will notice the
> put did have no effect. It will start working again after the major
> compaction has run. These issues should not be a problem if you use
> always-increasing versions for new puts to a row. But they can occur even if
> you do not care about time: just do delete and put immediately after each
> other, and there is some chance they happen within the same millisecond.
> 28.3.2. Major compactions change query results
> …create three cell versions at t1, t2 and t3, with a maximum-versions
> setting of 2. So when getting all versions, only the values at t2 and t3 will
> be returned. But if you delete the version at t2 or t3, the one at t1 will
> appear again. Obviously, once a major compaction has run, such behavior will
> not be the case anymore… (See Garbage Collection in Bending time in HBase.)
> {quote}
> These limitations result from the current implementation on multi-versions:
> we only consider timestamp, no matter when it comes; we will not remove old
> version immediately if there are enough number of new versions.
> So we can get a stronger semantics of versions by two guarantees:
> 1, Delete will not mask Put that comes after it.
> 2, If a version is masked by enough number of higher versions (VERSIONS in
> cf's conf), it will never be seen any more.
> Some examples for understanding:
> (delete t<=3 means use Delete.addColumns to delete all versions whose ts is
> not greater than 3, and delete t3 means use Delete.addColumn to delete the
> version whose ts=3)
> case 1: put t2 -> put t3 -> delete t<=3 -> put t1, and we will get t1 because
> the put is after delete.
> case 2: maxversion=2, put t1 -> put t2 -> put t3 -> delete t3, and we will
> always get t2 no matter if there is a major compaction, because t1 is masked
> when we put t3 so t1 will never be seen.
> case 3: maxversion=2, put t1 -> put t2 -> put t3 -> delete t2 -> delete t3,
> and we will get nothing.
> case 4: maxversion=3, put t1 -> put t2 -> put t3 -> delete t2 -> delete t3,
> and we will get t1 because it is not masked.
> case 5: maxversion=2, put t1 -> put t2 -> put t3 -> delete t3 -> put t1, and
> we can get t3+t1 because when we put t1 at second time it is the 2nd latest
> version and it can be read.
> case 6:maxversion=2, put t3->put t2->put t1, and we will get t3+t2 just like
> what we can get now, ts is still the key of versions.
> Different VERSIONS may result in different results even the size of result is
> smaller than VERSIONS(see case 3 and 4). So Get/Scan.setMaxVersions will be
> handled at end after we read correct data according to CF's VERSIONS setting.
> The semantics is different from the current HBase, and we may need more logic
> to support the new semantic, so it is configurable and default is disabled.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-10092) Move up on to log4j2
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106456#comment-16106456 ] stack commented on HBASE-10092: --- Unscheduling from 2.0.0 for now. My guess is that our users are not up for the 'surprise' of a new logging format, at least in 2.0.0 timeframe. > Move up on to log4j2 > > > Key: HBASE-10092 > URL: https://issues.apache.org/jira/browse/HBASE-10092 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: Alex Newman >Priority: Critical > Fix For: 2.0.0 > > Attachments: 10092.txt, 10092v2.txt, HBASE-10092.patch, > HBASE-10092-preview-v0.patch > > > Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ > This rather radical transition can be done w/ minor change given they have an > adapter for apache's logging, the one we use. They also have and adapter for > slf4j so we likely can remove at least some of the 4 versions of this module > our dependencies make use of. > I made a start in attached patch but am currently stuck in maven dependency > resolve hell courtesy of our slf4j. Fixing will take some concentration and > a good net connection, an item I currently lack. Other TODOs are that will > need to fix our little log level setting jsp page -- will likely have to undo > our use of hadoop's tool here -- and the config system changes a little. > I will return to this project soon. Will bring numbers. > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John reassigned HBASE-18142: -- Assignee: ChunHao (was: Chia-Ping Tsai) > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumns(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18481: --- Status: Patch Available (was: Open) > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-18481: -- Assignee: Guanghao Zhang > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18481) The autoFlush flag was not used in PE tool
[ https://issues.apache.org/jira/browse/HBASE-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18481: --- Attachment: HBASE-18481.patch > The autoFlush flag was not used in PE tool > -- > > Key: HBASE-18481 > URL: https://issues.apache.org/jira/browse/HBASE-18481 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Minor > Attachments: HBASE-18481.patch > > > After HBASE-12728, PE used the BufferedMutator for random/sequential write > test and the autoFlush flag was not used. So all write test will buffered the > write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18481) The autoFlush flag was not used in PE tool
Guanghao Zhang created HBASE-18481: -- Summary: The autoFlush flag was not used in PE tool Key: HBASE-18481 URL: https://issues.apache.org/jira/browse/HBASE-18481 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang Priority: Minor After HBASE-12728, PE used the BufferedMutator for random/sequential write test and the autoFlush flag was not used. So all write test will buffered the write request and send as a batch request when the buffer has filled. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106446#comment-16106446 ] Chia-Ping Tsai commented on HBASE-18142: [~anoop.hbase] Would you please provide [~chunhao] the right to assign issues in JIRA ? > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API >Reporter: Karthick >Assignee: Chia-Ping Tsai > Labels: beginner > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumns(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18259) HBase book link to "beginner" issues includes resolved issues
[
https://issues.apache.org/jira/browse/HBASE-18259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106443#comment-16106443
]
Chia-Ping Tsai commented on HBASE-18259:
We overlook a link.
{noformat}
To check for existing issues which you can tackle as a beginner, search for
link:https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20labels%20in%20(beginner)[issues
in JIRA tagged with the label 'beginner'].
{noformat}
[~psomogyi] Would you please fix it?
> HBase book link to "beginner" issues includes resolved issues
> -
>
> Key: HBASE-18259
> URL: https://issues.apache.org/jira/browse/HBASE-18259
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Mike Drob
>Assignee: Peter Somogyi
> Labels: beginner
> Fix For: 3.0.0
>
> Attachments: HBASE-18259.master.001.patch
>
>
> The link at http://hbase.apache.org/book.html#getting.involved for beginner
> issues is
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20labels%20in%20(beginner)
> but this includes resolved issues as well, which is not useful to folks
> looking for new issues to cut their teeth on.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106445#comment-16106445 ] ChunHao commented on HBASE-18142: - Hi, Chia-Ping Tsai： I want to try to solve this issue. Could you assign this issue to me? thanks. > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API >Reporter: Karthick >Assignee: Chia-Ping Tsai > Labels: beginner > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumns(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106441#comment-16106441
]
Hudson commented on HBASE-18473:
SUCCESS: Integrated in Jenkins build HBase-1.2-IT #910 (See
[https://builds.apache.org/job/HBase-1.2-IT/910/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev 79c8031ebf2d11214f60a8fd0bf32195cd44c273)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106438#comment-16106438
]
Hudson commented on HBASE-18473:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #148 (See
[https://builds.apache.org/job/HBase-1.3-IT/148/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev 3e450f7a077632902fafa43dc7d9afa462125060)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106436#comment-16106436
]
Hudson commented on HBASE-18473:
FAILURE: Integrated in Jenkins build HBase-1.2-JDK7 #176 (See
[https://builds.apache.org/job/HBase-1.2-JDK7/176/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev 79c8031ebf2d11214f60a8fd0bf32195cd44c273)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18374) RegionServer Metrics improvements
[ https://issues.apache.org/jira/browse/HBASE-18374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106434#comment-16106434 ] Chia-Ping Tsai commented on HBASE-18374: +1 > RegionServer Metrics improvements > - > > Key: HBASE-18374 > URL: https://issues.apache.org/jira/browse/HBASE-18374 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-18374.branch-1.001.patch, > HBASE-18374.branch-1.001.patch, HBASE-18374.branch-1.002.patch, > HBASE-18374.branch-1.003.patch, HBASE-18374.master.001.patch, > HBASE-18374.master.002.patch, HBASE-18374.master.003.patch, > HBASE-18374.master.004.patch, HBASE-18374.master.005.patch > > > At the RS level we have latency metrics for mutate/puts and deletes that are > updated per batch (ie. at the end of entire batchop if it contains put/delete > update the respective metric) in contrast with append/increment/get metrics > that are updated per op. This is a bit ambiguous since the delete and put > metrics are updated for multi row mutations that happen to contain a > put/delete. We should rename the metric(eg. delete_batch)/add better > description. Also we should add metrics for single delete client operations > that come through RSRpcServer.mutate path. We should also add metrics for > checkAndPut and checkAndDelete. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106424#comment-16106424
]
Hudson commented on HBASE-18473:
FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #210 (See
[https://builds.apache.org/job/HBase-1.3-JDK7/210/])
HBASE-18473 VC.listLabels() erroneously closes any connection. (anoopsamjohn:
rev 3e450f7a077632902fafa43dc7d9afa462125060)
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18473) VC.listLabels() erroneously closes any connection
[
https://issues.apache.org/jira/browse/HBASE-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-18473:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the quick reviews. Pushed to all applicable branches
> VC.listLabels() erroneously closes any connection
> -
>
> Key: HBASE-18473
> URL: https://issues.apache.org/jira/browse/HBASE-18473
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.2.0, 2.0.0-alpha-1
>Reporter: Lars George
>Assignee: Anoop Sam John
> Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18473.patch
>
>
> In HBASE-13358 the {{VisibilityClient.listLabels()}} was amended to take in a
> connection from the caller, which totally makes sense. But the patch forgot
> to remove the unconditional call to {{connection.close()}} in the {{finally}}
> block:
> {code}
> finally {
> if (table != null) {
> table.close();
> }
> if (connection != null) {
> connection.close();
> }
> }
> {code}
> Remove the second {{if}} completely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18476) HTable#put should call RS#mutate rather than RS#multi
[
https://issues.apache.org/jira/browse/HBASE-18476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106411#comment-16106411
]
Chia-Ping Tsai commented on HBASE-18476:
bq. was there any particular reason of using the multi path rather than mutate?
non-auto-flush and write buffer.
{code:title=HTableInterface.java}
@Deprecated
void setAutoFlush(boolean autoFlush);
{code}
{code:title=Table.java}
@Deprecated
void setWriteBufferSize(long writeBufferSize) throws IOException;
{code}
It seems to me that the HTableInterface can be removed because it is IA.private
and deprecated in 0.99. But the method of "write buffer" can't be removed in
branch-1...
> HTable#put should call RS#mutate rather than RS#multi
> -
>
> Key: HBASE-18476
> URL: https://issues.apache.org/jira/browse/HBASE-18476
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2
>
>
> The HBASE-18374 separates the metric for a single put and batched puts so the
> HTable#put shouldn't use the RS#mulit. It messes up the metrics.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
