[jira] [Commented] (HBASE-18489) Expose scan cursor in RawScanResultConsumer
[
https://issues.apache.org/jira/browse/HBASE-18489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117891#comment-16117891
]
Hadoop QA commented on HBASE-18489:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
57s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}130m 24s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}197m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestScannerHeartbeatMessages |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18489 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880772/HBASE-18489-v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux f63473d53a56 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7e7461e |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/7970/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7970/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
[jira] [Commented] (HBASE-18528) Support to modify TableDescriptor/ColumnFamilyDescriptor through MasterObserver; Or disable that.
[ https://issues.apache.org/jira/browse/HBASE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117888#comment-16117888 ] Anoop Sam John commented on HBASE-18528: What we need is change the param type from deprecated HTableDescriptor to TableDescriptor right? Which deprecated methods u want to remove? Ya may be we can remove deprecated methods as we are any way breaking CP for 2.0 So lets do all cleanup. If you see RegionObserver there may be so many APIs ready for removal then. (But that is any way another issue) > Support to modify TableDescriptor/ColumnFamilyDescriptor through > MasterObserver; Or disable that. > - > > Key: HBASE-18528 > URL: https://issues.apache.org/jira/browse/HBASE-18528 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, master >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > > We are replacing the HTableDescriptor by TableDescriptor from code base. The > TableDescriptor is designed to be a read-only object so user can't modifiy it > through MasterObserver. HBASE-18502 change many methods of MasterObserver to > use TableDescriptor but some deprecated methods still accept the > HTableDescriptor. User may be confused by why some methods can't modify the > table descriptor. > In short, Should we allow user to modify the passed table descriptor? > # if yes, we should introduce a mechanism that user can return a modified > table descripror > # if no, we should pass ImmutableHTableDescriptor to user. Or we just remove > all methods accepting the HTableDescriptor > Ditto for HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18528) Support to modify TableDescriptor/ColumnFamilyDescriptor through MasterObserver; Or disable that.
[
https://issues.apache.org/jira/browse/HBASE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117885#comment-16117885
]
Chia-Ping Tsai commented on HBASE-18528:
Personally, I prefer removeing the deprecated methods which use the
HTableDescriptor. Reserving the deprecated APIs puts some ugly code. Also, we
need to create a ImmutableHTableDescriptor for the deprecated API...
{code}
public void preCreateTableAction(final HTableDescriptor htd, final
HRegionInfo[] regions,
final User user)
throws IOException {
execOperation(coprocessors.isEmpty() ? null : new
CoprocessorOperation(user) {
@Override
public void call(MasterObserver oserver,
ObserverContext ctx)
throws IOException {
oserver.preCreateTableHandler(ctx, htd, regions);
oserver.preCreateTableAction(ctx, htd, regions);
}
});
}
{code}
If it is legal to break the CP in major version, why not remove the deprecated
APIs?
> Support to modify TableDescriptor/ColumnFamilyDescriptor through
> MasterObserver; Or disable that.
> -
>
> Key: HBASE-18528
> URL: https://issues.apache.org/jira/browse/HBASE-18528
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors, master
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
>
> We are replacing the HTableDescriptor by TableDescriptor from code base. The
> TableDescriptor is designed to be a read-only object so user can't modifiy it
> through MasterObserver. HBASE-18502 change many methods of MasterObserver to
> use TableDescriptor but some deprecated methods still accept the
> HTableDescriptor. User may be confused by why some methods can't modify the
> table descriptor.
> In short, Should we allow user to modify the passed table descriptor?
> # if yes, we should introduce a mechanism that user can return a modified
> table descripror
> # if no, we should pass ImmutableHTableDescriptor to user. Or we just remove
> all methods accepting the HTableDescriptor
> Ditto for HColumnDescriptor.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18517) limit max log message width in log4j
[
https://issues.apache.org/jira/browse/HBASE-18517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117884#comment-16117884
]
Vikas Vishwakarma commented on HBASE-18517:
---
thanks [~stack] !
> limit max log message width in log4j
>
>
> Key: HBASE-18517
> URL: https://issues.apache.org/jira/browse/HBASE-18517
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 2.0.0-alpha-2
>Reporter: Vikas Vishwakarma
>Assignee: Vikas Vishwakarma
> Fix For: 1.5.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18517.branch-1.001.patch,
> HBASE-18517.master.001.patch
>
>
> We had two cases now in our prod / pilot setups which is leading to humongous
> log lines in RegionServer logs.
> In first case, one of the phoenix user had constructed a query with a really
> large list of Id filters (61 MB) that translated into HBase scan that was
> running slow which lead to responseTooSlow messages in the logs with the
> entire filter list being printed in the logs, example
> ipc.RpcServer - (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","starttimems":1501457864417,"responsesize":11,"method":"Scan","param":"region
> { type: REGION_NAME value: .
> org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter\\022\351\\200\\036\\n(org.apache.phoenix.filter.SkipScanFilter
> ... ...
> There was another case where a use case had created a table with really large
> key/region names. This was causing humongous log lines for flush and
> compaction on these regions filling up the RS logs
> These large logs usually cause issues with disk I/O load, loading the splunk
> servers, even machine perf degradations. With 61 MB log lines basic log
> processing commands like vim, scrolling the logs, wc -l , etc were getting
> stuck. High GC activity was also noted on this cluster although not 100% sure
> if it was related to above issue.
> We should consider limiting the message size in logs which can be easily done
> by adding a maximum width format modifier on the message conversion character
> in log4j.properties
> log4j.appender.console.layout.ConversionPattern=...: %m%n
> to
> log4j.appender.console.layout.ConversionPattern=...: %.1m%n
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-15511) ClusterStatus should be able to return responses by scope
[
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117863#comment-16117863
]
Reid Chan edited comment on HBASE-15511 at 8/8/17 5:11 AM:
---
Thanks for reviews [~chia7712].
Most of the codes is modified as suggestions, the getter/setter i reserves for
literal comprehensive, but you remind me of the reusability of the Options, so
i add a method to reset defaults{code} public Options reset() {code}
was (Author: reidchan):
Thanks for reviews [~chia7712].
Most of the codes is modified as suggestions, the getter/setter i reserves for
literal comprehensive, but you remind me of the reusability of the Options, so
i add a method {code} public Options reset() {code}
> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
> Issue Type: Improvement
>Reporter: Esteban Gutierrez
>Assignee: Reid Chan
> Attachments: HBASE-15511.master.001.patch,
> HBASE-15511.master.002.patch, HBASE-15511.master.003.patch,
> HBASE-15511.master.004.patch, HBASE-15511.master.005.patch,
> HBASE-15511.master.006.patch, HBASE-15511.master.007.patch,
> HBASE-15511.master.008.patch, HBASE-15511.master.009.patch,
> HBASE-15511.master.010.patch
>
>
> The current ClusterStatus response returns too much information about the
> load per region and replication cluster wide. Sometimes that response can be
> quite large (10s or 100s of MBs) and methods like getServerSize() or
> getRegionsCount() don't really need the full response. One possibility is to
> provide a scope (or filter) for the ClusterStatus requests to limit the
> response back to the client.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15511) ClusterStatus should be able to return responses by scope
[
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117863#comment-16117863
]
Reid Chan commented on HBASE-15511:
---
Thanks for reviews [~chia7712].
Most of the codes is modified as suggestions, the getter/setter i reserves for
literal comprehensive, but you remind me of the reusability of the Options, so
i add a method {code} public Options reset() {code}
> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
> Issue Type: Improvement
>Reporter: Esteban Gutierrez
>Assignee: Reid Chan
> Attachments: HBASE-15511.master.001.patch,
> HBASE-15511.master.002.patch, HBASE-15511.master.003.patch,
> HBASE-15511.master.004.patch, HBASE-15511.master.005.patch,
> HBASE-15511.master.006.patch, HBASE-15511.master.007.patch,
> HBASE-15511.master.008.patch, HBASE-15511.master.009.patch,
> HBASE-15511.master.010.patch
>
>
> The current ClusterStatus response returns too much information about the
> load per region and replication cluster wide. Sometimes that response can be
> quite large (10s or 100s of MBs) and methods like getServerSize() or
> getRegionsCount() don't really need the full response. One possibility is to
> provide a scope (or filter) for the ClusterStatus requests to limit the
> response back to the client.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18528) Support to modify TableDescriptor/ColumnFamilyDescriptor through MasterObserver; Or disable that.
[ https://issues.apache.org/jira/browse/HBASE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117861#comment-16117861 ] Anoop Sam John commented on HBASE-18528: We might need the desc to be passed to CPs. This can give many details (getters).. ImmutableHTableDescriptor is again an impl class and we want to pass interfaces. TableDesc or ColumnDesc interface type only we can pass. Ya the object passed to be ImmutableHTableDescriptor (I believe that only u mean here). That object any way throw Exception while calling setters on it right. > Support to modify TableDescriptor/ColumnFamilyDescriptor through > MasterObserver; Or disable that. > - > > Key: HBASE-18528 > URL: https://issues.apache.org/jira/browse/HBASE-18528 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, master >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > > We are replacing the HTableDescriptor by TableDescriptor from code base. The > TableDescriptor is designed to be a read-only object so user can't modifiy it > through MasterObserver. HBASE-18502 change many methods of MasterObserver to > use TableDescriptor but some deprecated methods still accept the > HTableDescriptor. User may be confused by why some methods can't modify the > table descriptor. > In short, Should we allow user to modify the passed table descriptor? > # if yes, we should introduce a mechanism that user can return a modified > table descripror > # if no, we should pass ImmutableHTableDescriptor to user. Or we just remove > all methods accepting the HTableDescriptor > Ditto for HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18398) Snapshot operation fails with FileNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117858#comment-16117858
]
Ted Yu commented on HBASE-18398:
{code}
+ public void preFsSnapshot() {
{code}
Can Fs in the method name be removed ? There is only one snapshot operation in
hbase.
TestSnapshot needs to carry test category.
I only see testAddRegionWithCompactions in the class. Can you make the class
name more specific ? There are some snapshot test classes already.
Can you measure the impact of this change on snapshot performance ?
> Snapshot operation fails with FileNotFoundException
> ---
>
> Key: HBASE-18398
> URL: https://issues.apache.org/jira/browse/HBASE-18398
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Ashu Pachauri
>Assignee: Ashu Pachauri
> Fix For: 1.3.2
>
> Attachments: HBASE-18398.master.001.patch
>
>
> Failing to take snapshot due to FileNotFoundException
> * FlushSnapshotSubprocedure.RegionSnapshotTask takes a region level read
> lock
> * Call to HRegion#addRegionToSnapshot.
> * Call to SnapshotManifest#addRegion. This gets the current list of store
> files.
> * RACE → File is marked as compacted away and HFileArchiver moves the
> file to archive under store level lock.
> * SnapshotManifest#addRegion visits the stale list of store files one by
> one. It does a file.getStatus() call to get length of each file. Since the
> file object still points to the original file, file.getStatus() fails with
> FileNotFoundException.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope
[ https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-15511: -- Status: Patch Available (was: Open) > ClusterStatus should be able to return responses by scope > - > > Key: HBASE-15511 > URL: https://issues.apache.org/jira/browse/HBASE-15511 > Project: HBase > Issue Type: Improvement >Reporter: Esteban Gutierrez >Assignee: Reid Chan > Attachments: HBASE-15511.master.001.patch, > HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, > HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, > HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, > HBASE-15511.master.008.patch, HBASE-15511.master.009.patch, > HBASE-15511.master.010.patch > > > The current ClusterStatus response returns too much information about the > load per region and replication cluster wide. Sometimes that response can be > quite large (10s or 100s of MBs) and methods like getServerSize() or > getRegionsCount() don't really need the full response. One possibility is to > provide a scope (or filter) for the ClusterStatus requests to limit the > response back to the client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope
[ https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-15511: -- Attachment: HBASE-15511.master.010.patch > ClusterStatus should be able to return responses by scope > - > > Key: HBASE-15511 > URL: https://issues.apache.org/jira/browse/HBASE-15511 > Project: HBase > Issue Type: Improvement >Reporter: Esteban Gutierrez >Assignee: Reid Chan > Attachments: HBASE-15511.master.001.patch, > HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, > HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, > HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, > HBASE-15511.master.008.patch, HBASE-15511.master.009.patch, > HBASE-15511.master.010.patch > > > The current ClusterStatus response returns too much information about the > load per region and replication cluster wide. Sometimes that response can be > quite large (10s or 100s of MBs) and methods like getServerSize() or > getRegionsCount() don't really need the full response. One possibility is to > provide a scope (or filter) for the ClusterStatus requests to limit the > response back to the client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope
[ https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-15511: -- Status: Open (was: Patch Available) > ClusterStatus should be able to return responses by scope > - > > Key: HBASE-15511 > URL: https://issues.apache.org/jira/browse/HBASE-15511 > Project: HBase > Issue Type: Improvement >Reporter: Esteban Gutierrez >Assignee: Reid Chan > Attachments: HBASE-15511.master.001.patch, > HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, > HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, > HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, > HBASE-15511.master.008.patch, HBASE-15511.master.009.patch > > > The current ClusterStatus response returns too much information about the > load per region and replication cluster wide. Sometimes that response can be > quite large (10s or 100s of MBs) and methods like getServerSize() or > getRegionsCount() don't really need the full response. One possibility is to > provide a scope (or filter) for the ClusterStatus requests to limit the > response back to the client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18500) Performance issue: Don't use BufferedMutator for HTable's put method
[
https://issues.apache.org/jira/browse/HBASE-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117857#comment-16117857
]
Hadoop QA commented on HBASE-18500:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 17 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
43s{color} | {color:red} hbase-rest in master has 3 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 26s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
1s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
59s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}176m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18500 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880767/HBASE-18500-v5.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 6ed0a3a3ed0e 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7e7461e |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7969/artifact/patchprocess/branch-findbugs-hbase-rest-warnings.html
|
| Test Results |
[jira] [Commented] (HBASE-18511) Default no regions on master
[ https://issues.apache.org/jira/browse/HBASE-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117852#comment-16117852 ] stack commented on HBASE-18511: --- I've set TABLES_ON_MASTER to a boolean for now. This issue is a little interesting. Trying to figure how to do this tables on master (system, any, or none). Let me see what tests fail now w/ these few fixes. > Default no regions on master > > > Key: HBASE-18511 > URL: https://issues.apache.org/jira/browse/HBASE-18511 > Project: HBase > Issue Type: Task > Components: master >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18511.master.001.patch, > HBASE-18511.master.002.patch > > > Let this be umbrella issue for no-regions-on-master as default deploy (as it > was in branch-1). > Also need to make sure we can run WITH regions on master; in particular > system tables with RPC short-circuit as it is now in hbase master. > Background is that master branch carried a change that allowed Master carry > regions. On top of this improvement on branch-1, Master defaulted to carry > system tables only. No release was made with this configuration. Now we are > going to cut the 2.0.0 release, the decision is that hbase-2 should have the > same layout as hbase-1 so this issue implements the undoing of Master > carrying system tables by default (though the capability remains). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18511) Default no regions on master
[
https://issues.apache.org/jira/browse/HBASE-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117842#comment-16117842
]
Hadoop QA commented on HBASE-18511:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-18511 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18511 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880782/HBASE-18511.master.002.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7972/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Default no regions on master
>
>
> Key: HBASE-18511
> URL: https://issues.apache.org/jira/browse/HBASE-18511
> Project: HBase
> Issue Type: Task
> Components: master
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-18511.master.001.patch,
> HBASE-18511.master.002.patch
>
>
> Let this be umbrella issue for no-regions-on-master as default deploy (as it
> was in branch-1).
> Also need to make sure we can run WITH regions on master; in particular
> system tables with RPC short-circuit as it is now in hbase master.
> Background is that master branch carried a change that allowed Master carry
> regions. On top of this improvement on branch-1, Master defaulted to carry
> system tables only. No release was made with this configuration. Now we are
> going to cut the 2.0.0 release, the decision is that hbase-2 should have the
> same layout as hbase-1 so this issue implements the undoing of Master
> carrying system tables by default (though the capability remains).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18511) Default no regions on master
[ https://issues.apache.org/jira/browse/HBASE-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18511: -- Attachment: HBASE-18511.master.002.patch > Default no regions on master > > > Key: HBASE-18511 > URL: https://issues.apache.org/jira/browse/HBASE-18511 > Project: HBase > Issue Type: Task > Components: master >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18511.master.001.patch, > HBASE-18511.master.002.patch > > > Let this be umbrella issue for no-regions-on-master as default deploy (as it > was in branch-1). > Also need to make sure we can run WITH regions on master; in particular > system tables with RPC short-circuit as it is now in hbase master. > Background is that master branch carried a change that allowed Master carry > regions. On top of this improvement on branch-1, Master defaulted to carry > system tables only. No release was made with this configuration. Now we are > going to cut the 2.0.0 release, the decision is that hbase-2 should have the > same layout as hbase-1 so this issue implements the undoing of Master > carrying system tables by default (though the capability remains). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18517) limit max log message width in log4j
[
https://issues.apache.org/jira/browse/HBASE-18517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18517:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> limit max log message width in log4j
>
>
> Key: HBASE-18517
> URL: https://issues.apache.org/jira/browse/HBASE-18517
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 2.0.0-alpha-2
>Reporter: Vikas Vishwakarma
>Assignee: Vikas Vishwakarma
> Fix For: 1.5.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18517.branch-1.001.patch,
> HBASE-18517.master.001.patch
>
>
> We had two cases now in our prod / pilot setups which is leading to humongous
> log lines in RegionServer logs.
> In first case, one of the phoenix user had constructed a query with a really
> large list of Id filters (61 MB) that translated into HBase scan that was
> running slow which lead to responseTooSlow messages in the logs with the
> entire filter list being printed in the logs, example
> ipc.RpcServer - (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","starttimems":1501457864417,"responsesize":11,"method":"Scan","param":"region
> { type: REGION_NAME value: .
> org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter\\022\351\\200\\036\\n(org.apache.phoenix.filter.SkipScanFilter
> ... ...
> There was another case where a use case had created a table with really large
> key/region names. This was causing humongous log lines for flush and
> compaction on these regions filling up the RS logs
> These large logs usually cause issues with disk I/O load, loading the splunk
> servers, even machine perf degradations. With 61 MB log lines basic log
> processing commands like vim, scrolling the logs, wc -l , etc were getting
> stuck. High GC activity was also noted on this cluster although not 100% sure
> if it was related to above issue.
> We should consider limiting the message size in logs which can be easily done
> by adding a maximum width format modifier on the message conversion character
> in log4j.properties
> log4j.appender.console.layout.ConversionPattern=...: %m%n
> to
> log4j.appender.console.layout.ConversionPattern=...: %.1m%n
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18517) limit max log message width in log4j
[
https://issues.apache.org/jira/browse/HBASE-18517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18517:
--
Affects Version/s: (was: 3.0.0)
2.0.0-alpha-2
Hadoop Flags: Incompatible change,Reviewed
Release Note: Sets a log length max of 1k.
Fix Version/s: (was: 3.0.0)
2.0.0-alpha-2
Pushed to branch-1 and branch-2 as well as master. Thanks [~vik.karma]
> limit max log message width in log4j
>
>
> Key: HBASE-18517
> URL: https://issues.apache.org/jira/browse/HBASE-18517
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 2.0.0-alpha-2
>Reporter: Vikas Vishwakarma
>Assignee: Vikas Vishwakarma
> Fix For: 1.5.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18517.branch-1.001.patch,
> HBASE-18517.master.001.patch
>
>
> We had two cases now in our prod / pilot setups which is leading to humongous
> log lines in RegionServer logs.
> In first case, one of the phoenix user had constructed a query with a really
> large list of Id filters (61 MB) that translated into HBase scan that was
> running slow which lead to responseTooSlow messages in the logs with the
> entire filter list being printed in the logs, example
> ipc.RpcServer - (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","starttimems":1501457864417,"responsesize":11,"method":"Scan","param":"region
> { type: REGION_NAME value: .
> org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter\\022\351\\200\\036\\n(org.apache.phoenix.filter.SkipScanFilter
> ... ...
> There was another case where a use case had created a table with really large
> key/region names. This was causing humongous log lines for flush and
> compaction on these regions filling up the RS logs
> These large logs usually cause issues with disk I/O load, loading the splunk
> servers, even machine perf degradations. With 61 MB log lines basic log
> processing commands like vim, scrolling the logs, wc -l , etc were getting
> stuck. High GC activity was also noted on this cluster although not 100% sure
> if it was related to above issue.
> We should consider limiting the message size in logs which can be easily done
> by adding a maximum width format modifier on the message conversion character
> in log4j.properties
> log4j.appender.console.layout.ConversionPattern=...: %m%n
> to
> log4j.appender.console.layout.ConversionPattern=...: %.1m%n
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18522) Add RowMutations support to Batch
[ https://issues.apache.org/jira/browse/HBASE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117819#comment-16117819 ] Jerry He commented on HBASE-18522: -- I looked at the master. The patch should be similar. But additionally AsyncTable is master branch (and branch-2). I could add the same support for AsyncTable as well. Or I could have a separate JIRA, if it is agreed to add it. > Add RowMutations support to Batch > - > > Key: HBASE-18522 > URL: https://issues.apache.org/jira/browse/HBASE-18522 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.6 >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18522-branch-1.patch > > > RowMutations is multiple Puts and/or Deletes atomically on a single row. > Current Batch call does not support RowMutations as part of the batch. > We should add this missing part. We should be able to batch RowMutations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18522) Add RowMutations support to Batch
[
https://issues.apache.org/jira/browse/HBASE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117816#comment-16117816
]
Jerry He commented on HBASE-18522:
--
I ran the failed tests with the patch multiple times locally. The same tests
fail and pass in different times.
{noformat}
Failed tests:
TestRSKilledWhenInitializing.testRSTerminationAfterRegisteringToMasterBeforeCreatingEphemeralNode:142
null
Tests in error:
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitFailure:462->TestLoadIncrementalHFilesSplitRecovery.setupTable:136->Object.wait:-2
» TestTimedOut
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitPresplit:389->TestLoadIncrementalHFilesSplitRecovery.setupTable:136->Object.wait:-2
» TestTimedOut
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta:502->TestLoadIncrementalHFilesSplitRecovery.setupTableWithSplitkeys:159->Object.wait:-2
» TestTimedOut
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testSplitTmpFileCleanUp:431->TestLoadIncrementalHFilesSplitRecovery.setupTableWithSplitkeys:159->Object.wait:-2
» TestTimedOu
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase:346->TestLoadIncrementalHFilesSplitRecovery.setupTable:136->Object.wait:-2
» TestTimedOut
Tests run: 32, Failures: 1, Errors: 5, Skipped: 0
{noformat}
And
{noformat}
Failed tests:
TestRSKilledWhenInitializing.testRSTerminationAfterRegisteringToMasterBeforeCreatingEphemeralNode:142
null
Tests in error:
TestZooKeeper.testMasterZKSessionRecoveryFailure:243->testSanity:258->Object.wait:502->Object.wait:-2
»TestTimedOut
TestClientScannerRPCTimeout.testScannerNextRPCTimesout:87 » TableExists
testSc...
Tests run: 32, Failures: 1, Errors: 2, Skipped: 0
{noformat}
I ran a few times without the patch as well. The same tests pass and fail from
time to time.
{noformat}
Tests in error:
TestClientScannerRPCTimeout.testScannerNextRPCTimesout:87 » TableExists
testSc...
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitFailure:462->TestLoadIncrementalHFilesSplitRecovery.setupTable:136->Object.wait:-2
» TestTimedOut
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitPresplit:389->TestLoadIncrementalHFilesSplitRecovery.setupTable:136->Object.wait:-2
» TestTimedOut
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta:502->TestLoadIncrementalHFilesSplitRecovery.setupTableWithSplitkeys:159->Object.wait:-2
» TestTimedOut
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testSplitTmpFileCleanUp:431->TestLoadIncrementalHFilesSplitRecovery.setupTableWithSplitkeys:159->Object.wait:-2
» TestTimedOu
TestSecureLoadIncrementalHFilesSplitRecovery>TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase:346->TestLoadIncrementalHFilesSplitRecovery.setupTable:136->Object.wait:-2
» TestTimedOut
Tests run: 32, Failures: 0, Errors: 6, Skipped: 0
{noformat}
And
{noformat}
Failed tests:
TestRSKilledWhenInitializing.testRSTerminationAfterRegisteringToMasterBeforeCreatingEphemeralNode:142
null
Tests in error:
TestZooKeeper.testMasterSessionExpired:227->testSanity:258->Object.wait:-2 »
TestTimedOut
TestClientScannerRPCTimeout.testScannerNextRPCTimesout:87 » TableExists
testSc...
{noformat}
> Add RowMutations support to Batch
> -
>
> Key: HBASE-18522
> URL: https://issues.apache.org/jira/browse/HBASE-18522
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.6
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18522-branch-1.patch
>
>
> RowMutations is multiple Puts and/or Deletes atomically on a single row.
> Current Batch call does not support RowMutations as part of the batch.
> We should add this missing part. We should be able to batch RowMutations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18469) Correct RegionServer metric of totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-18469: -- Status: Patch Available (was: Open) An early check on HadoopQA > Correct RegionServer metric of totalRequestCount > -- > > Key: HBASE-18469 > URL: https://issues.apache.org/jira/browse/HBASE-18469 > Project: HBase > Issue Type: Improvement > Components: metrics, regionserver >Affects Versions: 1.2.0 >Reporter: Shibin Zhang >Assignee: Yu Li >Priority: Critical > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18469.patch, HBASE-18469.v2.patch > > > when i get the metric ,i found this three metric may be have some error as > follow : > "totalRequestCount" : 17541, > "readRequestCount" : 17483, > "writeRequestCount" : 1633, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18533) Expose BucketCache values to be configured
[
https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117785#comment-16117785
]
Hadoop QA commented on HBASE-18533:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}138m
20s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}189m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18533 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880742/HBASE-18533.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e79e8f907905 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7e7461e |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7966/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7966/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Expose BucketCache values to be configured
> --
>
> Key: HBASE-18533
> URL: https://issues.apache.org/jira/browse/HBASE-18533
> Project: HBase
> Issue Type: Improvement
> Components: BucketCache
>Reporter: Zach York
>Assignee: Zach York
> Attachments: HBASE-18533.branch-1.001.patch,
> HBASE-18533.master.001.patch
>
>
>
[jira] [Updated] (HBASE-18469) Correct RegionServer metric of totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HBASE-18469: -- Attachment: HBASE-18469.v2.patch Update patch to address review comments. > Correct RegionServer metric of totalRequestCount > -- > > Key: HBASE-18469 > URL: https://issues.apache.org/jira/browse/HBASE-18469 > Project: HBase > Issue Type: Improvement > Components: metrics, regionserver >Affects Versions: 1.2.0 >Reporter: Shibin Zhang >Assignee: Yu Li >Priority: Critical > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18469.patch, HBASE-18469.v2.patch > > > when i get the metric ,i found this three metric may be have some error as > follow : > "totalRequestCount" : 17541, > "readRequestCount" : 17483, > "writeRequestCount" : 1633, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18528) Support to modify TableDescriptor/ColumnFamilyDescriptor through MasterObserver; Or disable that.
[ https://issues.apache.org/jira/browse/HBASE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117775#comment-16117775 ] Chia-Ping Tsai commented on HBASE-18528: bq. We should NOT allow this IMO. Should we pass ImmutableHTableDescriptor to user ? Or remove all methods accepting the HTableDescriptor? > Support to modify TableDescriptor/ColumnFamilyDescriptor through > MasterObserver; Or disable that. > - > > Key: HBASE-18528 > URL: https://issues.apache.org/jira/browse/HBASE-18528 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, master >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 3.0.0, 2.0.0-alpha-2 > > > We are replacing the HTableDescriptor by TableDescriptor from code base. The > TableDescriptor is designed to be a read-only object so user can't modifiy it > through MasterObserver. HBASE-18502 change many methods of MasterObserver to > use TableDescriptor but some deprecated methods still accept the > HTableDescriptor. User may be confused by why some methods can't modify the > table descriptor. > In short, Should we allow user to modify the passed table descriptor? > # if yes, we should introduce a mechanism that user can return a modified > table descripror > # if no, we should pass ImmutableHTableDescriptor to user. Or we just remove > all methods accepting the HTableDescriptor > Ditto for HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18266) Eliminate the warnings from the spotbugs
[ https://issues.apache.org/jira/browse/HBASE-18266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-18266. Resolution: Fixed All sub-tasks are resolved. > Eliminate the warnings from the spotbugs > > > Key: HBASE-18266 > URL: https://issues.apache.org/jira/browse/HBASE-18266 > Project: HBase > Issue Type: Umbrella >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2 > > > It is hard to get +1 from QA currently because spotbugs is always unhappy... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18315) Eliminate the findbugs warnings for hbase-rest
[ https://issues.apache.org/jira/browse/HBASE-18315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18315: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Push this to master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2 Thanks for the reviews. [~zghaobac] > Eliminate the findbugs warnings for hbase-rest > -- > > Key: HBASE-18315 > URL: https://issues.apache.org/jira/browse/HBASE-18315 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18315.branch-1.2.v0.patch, > HBASE-18315.branch-1.3.v0.patch, HBASE-18315.branch-1.v0.patch, > HBASE-18315.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18142: -- Hadoop Flags: Incompatible change,Reviewed > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumn(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117758#comment-16117758 ] stack commented on HBASE-18142: --- bq. Should we push this to 1.x ? As I see it, its a bug fix so yes. > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumn(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117758#comment-16117758 ] stack edited comment on HBASE-18142 at 8/8/17 2:33 AM: --- bq. Should we push this to 1.x ? As I see it, its a bug fix so yes. I marked it as an incompatible change. Release note would be good too. Thanks lads. was (Author: stack): bq. Should we push this to 1.x ? As I see it, its a bug fix so yes. > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumn(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18315) Eliminate the findbugs warnings for hbase-rest
[ https://issues.apache.org/jira/browse/HBASE-18315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117757#comment-16117757 ] Chia-Ping Tsai commented on HBASE-18315: Will commit it later. > Eliminate the findbugs warnings for hbase-rest > -- > > Key: HBASE-18315 > URL: https://issues.apache.org/jira/browse/HBASE-18315 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18315.branch-1.2.v0.patch, > HBASE-18315.branch-1.3.v0.patch, HBASE-18315.branch-1.v0.patch, > HBASE-18315.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18437) Revoke access permissions of a user from a table does not work as expected
[
https://issues.apache.org/jira/browse/HBASE-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117755#comment-16117755
]
Ashish Singhi commented on HBASE-18437:
---
bq. The permsList is obtained for this user and why again user check? Sorry not
getting. Or u have to check the table details?
The permlist is obtained for acl table, the row key in acl table is the
tablename, it will return all the global user names.
Below is the acl table scan output for better understanding,
{noformat}
hbase(main):011:0> scan 'hbase:acl'
ROW COLUMN+CELL
hbase:acl column=l:ashish,
timestamp=1502156949293, value=RWXCA
hbase:acl column=l:singhi,
timestamp=1502159481193, value=RW
t1 column=l:hbase,
timestamp=1501849980130, value=RWXCA
t12 column=l:hbase,
timestamp=1502156979137, value=RWXCA
3 row(s) in 0.0140 seconds
{noformat}
> Revoke access permissions of a user from a table does not work as expected
> --
>
> Key: HBASE-18437
> URL: https://issues.apache.org/jira/browse/HBASE-18437
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 1.1.12
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
> Attachments: HBASE-18437.patch
>
>
> A table for which a user was granted 'RW' permission. Now when we want to
> revoke its 'W' permission only, code removes the user itself from that table
> permissions.
> Below is the test code which reproduces the issue.
> {noformat}
> @Test(timeout = 18)
> public void testRevokeOnlySomePerms() throws Throwable {
> TableName name = TableName.valueOf("testAgain");
> HTableDescriptor htd = new HTableDescriptor(name);
> HColumnDescriptor hcd = new HColumnDescriptor("cf");
> htd.addFamily(hcd);
> createTable(TEST_UTIL, htd);
> TEST_UTIL.waitUntilAllRegionsAssigned(name);
> try (Connection conn = ConnectionFactory.createConnection(conf)) {
> AccessControlClient.grant(conn, name, USER_RO.getShortName(), null,
> null, Action.READ, Action.WRITE);
> ListMultimap tablePermissions =
> AccessControlLists.getTablePermissions(conf, name);
> // hbase user and USER_RO has permis
> assertEquals(2, tablePermissions.size());
> AccessControlClient.revoke(conn, name, USER_RO.getShortName(), null,
> null, Action.WRITE);
> tablePermissions = AccessControlLists.getTablePermissions(conf, name);
> List userPerm =
> tablePermissions.get(USER_RO.getShortName());
> assertEquals(1, userPerm.size());
> } finally {
> deleteTable(TEST_UTIL, name);
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18398) Snapshot operation fails with FileNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117756#comment-16117756
]
Hadoop QA commented on HBASE-18398:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 22m 45s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestCheckTestClasses |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18398 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880760/HBASE-18398.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b66e01e7e4d3 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7e7461e |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/7968/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7968/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7968/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Snapshot operation fails with FileNotFoundException
> ---
>
> Key: HBASE-18398
> URL: https://issues.apache.org/jira/browse/HBASE-18398
> Project: HBase
> Issue Type: Sub-task
>
[jira] [Resolved] (HBASE-18078) [C++] Harden RPC by handling various communication abnormalities
[ https://issues.apache.org/jira/browse/HBASE-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-18078. --- Resolution: Fixed Fix Version/s: HBASE-14850 I have committed the v8 version which already addresses the review comments. HBASE-18204 builds on top of this. Thanks [~xiaobingo] for the patch. > [C++] Harden RPC by handling various communication abnormalities > > > Key: HBASE-18078 > URL: https://issues.apache.org/jira/browse/HBASE-18078 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Fix For: HBASE-14850 > > Attachments: HBASE-18078.000.patch, HBASE-18078.001.patch, > HBASE-18078.002.patch, HBASE-18078.003.patch, HBASE-18078.004.patch, > HBASE-18078.005.patch, HBASE-18078.006.patch, HBASE-18078.007.patch, > HBASE-18078.008.patch > > > RPC layer should handle various communication abnormalities (e.g. connection > timeout, server aborted connection, and so on). Ideally, the corresponding > exceptions should be raised and propagated through handlers of pipeline in > client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18489) Expose scan cursor in RawScanResultConsumer
[ https://issues.apache.org/jira/browse/HBASE-18489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-18489: -- Attachment: HBASE-18489-v1.patch Do a copy every time when create a scan cursor in StoreScanner. Store the last scan cursor in RegionScannerHolder and set it into the newly created ScannerContext to avoid setting a less scan cursor. > Expose scan cursor in RawScanResultConsumer > --- > > Key: HBASE-18489 > URL: https://issues.apache.org/jira/browse/HBASE-18489 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, scan >Affects Versions: 2.0.0-alpha-1 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-2 > > Attachments: HBASE-18489.patch, HBASE-18489-v1.patch > > > The first step of supporting scan cursor for async client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18437) Revoke access permissions of a user from a table does not work as expected
[
https://issues.apache.org/jira/browse/HBASE-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117750#comment-16117750
]
Ashish Singhi commented on HBASE-18437:
---
I will try to give a patch by tonight (China time) if not able to then anyone
can feel free assign the jira to them self and take it forward.
> Revoke access permissions of a user from a table does not work as expected
> --
>
> Key: HBASE-18437
> URL: https://issues.apache.org/jira/browse/HBASE-18437
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 1.1.12
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
> Attachments: HBASE-18437.patch
>
>
> A table for which a user was granted 'RW' permission. Now when we want to
> revoke its 'W' permission only, code removes the user itself from that table
> permissions.
> Below is the test code which reproduces the issue.
> {noformat}
> @Test(timeout = 18)
> public void testRevokeOnlySomePerms() throws Throwable {
> TableName name = TableName.valueOf("testAgain");
> HTableDescriptor htd = new HTableDescriptor(name);
> HColumnDescriptor hcd = new HColumnDescriptor("cf");
> htd.addFamily(hcd);
> createTable(TEST_UTIL, htd);
> TEST_UTIL.waitUntilAllRegionsAssigned(name);
> try (Connection conn = ConnectionFactory.createConnection(conf)) {
> AccessControlClient.grant(conn, name, USER_RO.getShortName(), null,
> null, Action.READ, Action.WRITE);
> ListMultimap tablePermissions =
> AccessControlLists.getTablePermissions(conf, name);
> // hbase user and USER_RO has permis
> assertEquals(2, tablePermissions.size());
> AccessControlClient.revoke(conn, name, USER_RO.getShortName(), null,
> null, Action.WRITE);
> tablePermissions = AccessControlLists.getTablePermissions(conf, name);
> List userPerm =
> tablePermissions.get(USER_RO.getShortName());
> assertEquals(1, userPerm.size());
> } finally {
> deleteTable(TEST_UTIL, name);
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18536) [C++] Add RPC fault injection infra
[ https://issues.apache.org/jira/browse/HBASE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-18536: -- Resolution: Fixed Fix Version/s: HBASE-14850 Status: Resolved (was: Patch Available) Thanks [~xiaobingo]. Committed this one. > [C++] Add RPC fault injection infra > --- > > Key: HBASE-18536 > URL: https://issues.apache.org/jira/browse/HBASE-18536 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Fix For: HBASE-14850 > > Attachments: HBASE-18536.000.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18537) [C++] Improvements to load-client
[ https://issues.apache.org/jira/browse/HBASE-18537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-18537: -- Attachment: hbase-18537.patch v1 patch. > [C++] Improvements to load-client > - > > Key: HBASE-18537 > URL: https://issues.apache.org/jira/browse/HBASE-18537 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: HBASE-14850 > > Attachments: hbase-18537.patch > > > A couple of improvements to the load-client after spending some time with > testing: > - better log messages > - support for progress > - minor bug fixes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18537) [C++] Improvements to load-client
Enis Soztutar created HBASE-18537: - Summary: [C++] Improvements to load-client Key: HBASE-18537 URL: https://issues.apache.org/jira/browse/HBASE-18537 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: HBASE-14850 A couple of improvements to the load-client after spending some time with testing: - better log messages - support for progress - minor bug fixes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117729#comment-16117729 ] Chia-Ping Tsai commented on HBASE-18142: bq. But that is not correct. addColumn (Pls see there is no 's') is what delete exactly one version. Thanks for the reminder. I have fixed the description bq. Here we are changing the behave of an existing Shell API? Can u pls write the details? Ya, this change break the operational compatibility. Should we push this to 1.x ? > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumn(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18142: --- Description: When I tried to delete a cell using it's timestamp in the Hbase Shell, the previous versions of the same cell also got deleted. But when I tried the same using the Java API, then the previous versions are not deleted and I can retrive the previous values. https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java see this file to fix the issue. This method (public Delete addColumn(final byte [] family, final byte [] qualifier, final long timestamp)) only deletes the current version of the cell. The previous versions are not deleted. was: When I tried to delete a cell using it's timestamp in the Hbase Shell, the previous versions of the same cell also got deleted. But when I tried the same using the Java API, then the previous versions are not deleted and I can retrive the previous values. https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java see this file to fix the issue. This method (public Delete addColumns(final byte [] family, final byte [] qualifier, final long timestamp)) only deletes the current version of the cell. The previous versions are not deleted. > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumn(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117723#comment-16117723 ] Anoop Sam John commented on HBASE-18142: Here we are changing the behave of an existing Shell API? Can u pls write the details? > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumns(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117721#comment-16117721 ] Anoop Sam John commented on HBASE-18142: The description says bq.This method (public Delete addColumns(final byte [] family, final byte [] qualifier, final long timestamp)) only deletes the current version of the cell. The previous versions are not deleted. But that is not correct. addColumn (Pls see there is no 's') is what delete exactly one version. > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumns(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18500) Performance issue: Don't use BufferedMutator for HTable's put method
[
https://issues.apache.org/jira/browse/HBASE-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-18500:
---
Attachment: HBASE-18500-v5.patch
Fix the javadoc issue. And findbugs problem was addressed by HBASE-18315.
> Performance issue: Don't use BufferedMutator for HTable's put method
>
>
> Key: HBASE-18500
> URL: https://issues.apache.org/jira/browse/HBASE-18500
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18500-v1.patch, HBASE-18500-v2.patch,
> HBASE-18500-v3.patch, HBASE-18500-v4.patch, HBASE-18500-v5.patch
>
>
> Copied the test result from HBASE-17994.
> Run start-hbase.sh in my local computer and use the default config to test
> with PE tool.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --autoFlush=True randomWrite 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --autoFlush=True asyncRandomWrite 1
> {code}
> Mean latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 164.39 | 161.22 | 164.78 | 140.61 | 151.69 |
> | asyncRandomWrite | 122.29 | 125.58 | 122.23 | 113.18 | 123.02 |
> 50th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 130.00 | 125.00 | 123.00 | 112.00 | 121.00 |
> | asyncRandomWrite | 95.00 | 97.00 | 95.00 | 88.00 | 95.00 |
> 99th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 600.00 | 600.00 | 650.00 | 404.00 | 425.00 |
> | asyncRandomWrite | 339.00 | 327.00 | 297.00 | 311.00 | 318.00 |
> In our internal 0.98 branch, the PE test result shows the async write has the
> almost same latency with the blocking write. But for master branch, the
> result shows the async write has better latency than the blocking client.
> Take a look about the code, I thought the difference is the BufferedMutator.
> For master branch, HTable don't have a write buffer and all write request
> will be flushed directly. And user can use BufferedMutator when user want to
> perform client-side buffering of writes. For the performance issue
> (autoFlush=True), I thought we can use rpc caller directly in HTable's put
> method. Thanks.
> Review: https://reviews.apache.org/r/61454/
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18315) Eliminate the findbugs warnings for hbase-rest
[ https://issues.apache.org/jira/browse/HBASE-18315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117707#comment-16117707 ] Guanghao Zhang commented on HBASE-18315: +1. > Eliminate the findbugs warnings for hbase-rest > -- > > Key: HBASE-18315 > URL: https://issues.apache.org/jira/browse/HBASE-18315 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18315.branch-1.2.v0.patch, > HBASE-18315.branch-1.3.v0.patch, HBASE-18315.branch-1.v0.patch, > HBASE-18315.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17125) Inconsistent result when use filter to read data
[
https://issues.apache.org/jira/browse/HBASE-17125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117703#comment-16117703
]
Guanghao Zhang commented on HBASE-17125:
Hadoop QA passed. [~anoop.hbase] [~Apache9] [~tedyu] [~chia7712] Any more
concerns?
> Inconsistent result when use filter to read data
>
>
> Key: HBASE-17125
> URL: https://issues.apache.org/jira/browse/HBASE-17125
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: 17125-slack-13.txt, example.diff,
> HBASE-17125.master.001.patch, HBASE-17125.master.002.patch,
> HBASE-17125.master.002.patch, HBASE-17125.master.003.patch,
> HBASE-17125.master.004.patch, HBASE-17125.master.005.patch,
> HBASE-17125.master.006.patch, HBASE-17125.master.007.patch,
> HBASE-17125.master.008.patch, HBASE-17125.master.009.patch,
> HBASE-17125.master.009.patch, HBASE-17125.master.010.patch,
> HBASE-17125.master.011.patch, HBASE-17125.master.011.patch,
> HBASE-17125.master.012.patch, HBASE-17125.master.013.patch,
> HBASE-17125.master.014.patch, HBASE-17125.master.015.patch,
> HBASE-17125.master.016.patch, HBASE-17125.master.017.patch,
> HBASE-17125.master.018.patch, HBASE-17125.master.019.patch,
> HBASE-17125.master.020.patch, HBASE-17125.master.020.patch,
> HBASE-17125.master.021.patch, HBASE-17125.master.checkReturnedVersions.patch,
> HBASE-17125.master.no-specified-filter.patch
>
>
> Assume a cloumn's max versions is 3, then we write 4 versions of this column.
> The oldest version doesn't remove immediately. But from the user view, the
> oldest version has gone. When user use a filter to query, if the filter skip
> a new version, then the oldest version will be seen again. But after compact
> the region, then the oldest version will never been seen. So it is weird for
> user. The query will get inconsistent result before and after region
> compaction.
> The reason is matchColumn method of UserScanQueryMatcher. It first check the
> cell by filter, then check the number of versions needed. So if the filter
> skip the new version, then the oldest version will be seen again when it is
> not removed.
> Have a discussion offline with [~Apache9] and [~fenghh], now we have two
> solution for this problem. The first idea is check the number of versions
> first, then check the cell by filter. As the comment of setFilter, the filter
> is called after all tests for ttl, column match, deletes and max versions
> have been run.
> {code}
> /**
>* Apply the specified server-side filter when performing the Query.
>* Only {@link Filter#filterKeyValue(Cell)} is called AFTER all tests
>* for ttl, column match, deletes and max versions have been run.
>* @param filter filter to run on the server
>* @return this for invocation chaining
>*/
> public Query setFilter(Filter filter) {
> this.filter = filter;
> return this;
> }
> {code}
> But this idea has another problem, if a column's max version is 5 and the
> user query only need 3 versions. It first check the version's number, then
> check the cell by filter. So the cells number of the result may less than 3.
> But there are 2 versions which don't read anymore.
> So the second idea has three steps.
> 1. check by the max versions of this column
> 2. check the kv by filter
> 3. check the versions which user need.
> But this will lead the ScanQueryMatcher more complicated. And this will break
> the javadoc of Query.setFilter.
> Now we don't have a final solution for this problem. Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18398) Snapshot operation fails with FileNotFoundException
[ https://issues.apache.org/jira/browse/HBASE-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri updated HBASE-18398: -- Status: Patch Available (was: Open) > Snapshot operation fails with FileNotFoundException > --- > > Key: HBASE-18398 > URL: https://issues.apache.org/jira/browse/HBASE-18398 > Project: HBase > Issue Type: Sub-task > Components: snapshots >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 1.3.2 > > Attachments: HBASE-18398.master.001.patch > > > Failing to take snapshot due to FileNotFoundException > * FlushSnapshotSubprocedure.RegionSnapshotTask takes a region level read > lock > * Call to HRegion#addRegionToSnapshot. > * Call to SnapshotManifest#addRegion. This gets the current list of store > files. > * RACE → File is marked as compacted away and HFileArchiver moves the > file to archive under store level lock. > * SnapshotManifest#addRegion visits the stale list of store files one by > one. It does a file.getStatus() call to get length of each file. Since the > file object still points to the original file, file.getStatus() fails with > FileNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18398) Snapshot operation fails with FileNotFoundException
[ https://issues.apache.org/jira/browse/HBASE-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117689#comment-16117689 ] Ashu Pachauri commented on HBASE-18398: --- Attaching patch for master. The patch adds another region operation called SNAPSHOT. Since the unit of snapshot operation (the unit of manifest that's written to filesystem) is a region, the operation takes locks for all stores for the region that is being processed. The locks that are taken are 1. Region level read lock, to avoid state changes to the region. 2. Archive lock for all stores, to prevent movement for compacted files to the archive while the snapshot operation is in progress. > Snapshot operation fails with FileNotFoundException > --- > > Key: HBASE-18398 > URL: https://issues.apache.org/jira/browse/HBASE-18398 > Project: HBase > Issue Type: Sub-task > Components: snapshots >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 1.3.2 > > Attachments: HBASE-18398.master.001.patch > > > Failing to take snapshot due to FileNotFoundException > * FlushSnapshotSubprocedure.RegionSnapshotTask takes a region level read > lock > * Call to HRegion#addRegionToSnapshot. > * Call to SnapshotManifest#addRegion. This gets the current list of store > files. > * RACE → File is marked as compacted away and HFileArchiver moves the > file to archive under store level lock. > * SnapshotManifest#addRegion visits the stale list of store files one by > one. It does a file.getStatus() call to get length of each file. Since the > file object still points to the original file, file.getStatus() fails with > FileNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18398) Snapshot operation fails with FileNotFoundException
[ https://issues.apache.org/jira/browse/HBASE-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri updated HBASE-18398: -- Attachment: HBASE-18398.master.001.patch > Snapshot operation fails with FileNotFoundException > --- > > Key: HBASE-18398 > URL: https://issues.apache.org/jira/browse/HBASE-18398 > Project: HBase > Issue Type: Sub-task > Components: snapshots >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 1.3.2 > > Attachments: HBASE-18398.master.001.patch > > > Failing to take snapshot due to FileNotFoundException > * FlushSnapshotSubprocedure.RegionSnapshotTask takes a region level read > lock > * Call to HRegion#addRegionToSnapshot. > * Call to SnapshotManifest#addRegion. This gets the current list of store > files. > * RACE → File is marked as compacted away and HFileArchiver moves the > file to archive under store level lock. > * SnapshotManifest#addRegion visits the stale list of store files one by > one. It does a file.getStatus() call to get length of each file. Since the > file object still points to the original file, file.getStatus() fails with > FileNotFoundException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18437) Revoke access permissions of a user from a table does not work as expected
[
https://issues.apache.org/jira/browse/HBASE-18437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117679#comment-16117679
]
Andrew Purtell commented on HBASE-18437:
[~ashish singhi] [~anoop.hbase] This looks almost ready to go. What do you
think about finishing it? Or should someone else take it up to finish it?
> Revoke access permissions of a user from a table does not work as expected
> --
>
> Key: HBASE-18437
> URL: https://issues.apache.org/jira/browse/HBASE-18437
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 1.1.12
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
> Attachments: HBASE-18437.patch
>
>
> A table for which a user was granted 'RW' permission. Now when we want to
> revoke its 'W' permission only, code removes the user itself from that table
> permissions.
> Below is the test code which reproduces the issue.
> {noformat}
> @Test(timeout = 18)
> public void testRevokeOnlySomePerms() throws Throwable {
> TableName name = TableName.valueOf("testAgain");
> HTableDescriptor htd = new HTableDescriptor(name);
> HColumnDescriptor hcd = new HColumnDescriptor("cf");
> htd.addFamily(hcd);
> createTable(TEST_UTIL, htd);
> TEST_UTIL.waitUntilAllRegionsAssigned(name);
> try (Connection conn = ConnectionFactory.createConnection(conf)) {
> AccessControlClient.grant(conn, name, USER_RO.getShortName(), null,
> null, Action.READ, Action.WRITE);
> ListMultimap tablePermissions =
> AccessControlLists.getTablePermissions(conf, name);
> // hbase user and USER_RO has permis
> assertEquals(2, tablePermissions.size());
> AccessControlClient.revoke(conn, name, USER_RO.getShortName(), null,
> null, Action.WRITE);
> tablePermissions = AccessControlLists.getTablePermissions(conf, name);
> List userPerm =
> tablePermissions.get(USER_RO.getShortName());
> assertEquals(1, userPerm.size());
> } finally {
> deleteTable(TEST_UTIL, name);
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18536) [C++] Add RPC fault injection infra
[
https://issues.apache.org/jira/browse/HBASE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117676#comment-16117676
]
Hadoop QA commented on HBASE-18536:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-18536 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18536 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880758/HBASE-18536.000.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7967/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> [C++] Add RPC fault injection infra
> ---
>
> Key: HBASE-18536
> URL: https://issues.apache.org/jira/browse/HBASE-18536
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Attachments: HBASE-18536.000.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18536) [C++] Add RPC fault injection infra
[ https://issues.apache.org/jira/browse/HBASE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18536: -- Status: Patch Available (was: Open) > [C++] Add RPC fault injection infra > --- > > Key: HBASE-18536 > URL: https://issues.apache.org/jira/browse/HBASE-18536 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18536.000.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18536) [C++] Add RPC fault injection infra
[ https://issues.apache.org/jira/browse/HBASE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18536: -- Attachment: HBASE-18536.000.patch > [C++] Add RPC fault injection infra > --- > > Key: HBASE-18536 > URL: https://issues.apache.org/jira/browse/HBASE-18536 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18536.000.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18536) [C++] Add RPC fault injection infra
[ https://issues.apache.org/jira/browse/HBASE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18536: -- Summary: [C++] Add RPC fault injection infra (was: [C++] Add fault injection infra) > [C++] Add RPC fault injection infra > --- > > Key: HBASE-18536 > URL: https://issues.apache.org/jira/browse/HBASE-18536 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18078) [C++] Harden RPC by handling various communication abnormalities
[ https://issues.apache.org/jira/browse/HBASE-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18078: -- Attachment: HBASE-18078.008.patch > [C++] Harden RPC by handling various communication abnormalities > > > Key: HBASE-18078 > URL: https://issues.apache.org/jira/browse/HBASE-18078 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18078.000.patch, HBASE-18078.001.patch, > HBASE-18078.002.patch, HBASE-18078.003.patch, HBASE-18078.004.patch, > HBASE-18078.005.patch, HBASE-18078.006.patch, HBASE-18078.007.patch, > HBASE-18078.008.patch > > > RPC layer should handle various communication abnormalities (e.g. connection > timeout, server aborted connection, and so on). Ideally, the corresponding > exceptions should be raised and propagated through handlers of pipeline in > client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18353) Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
[
https://issues.apache.org/jira/browse/HBASE-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117632#comment-16117632
]
Duo Zhang commented on HBASE-18353:
---
I can not follow... What's your point here?
{code}
/**
* Unassign a region from current hosting regionserver. Region will then be
assigned to a
* regionserver chosen at random. Region could be reassigned back to the
same server. Use {@link
* #move(byte[], byte[])} if you want to control the region movement.
*
* @param regionName Region to unassign. Will clear any existing RegionPlan
if one found.
* @param force If true, force unassign (Will remove region from
regions-in-transition too if
* present. If results in double assignment use hbck -fix to resolve. To be
used by experts).
*/
void unassign(final byte[] regionName, final boolean force)
throws IOException;
{code}
This is the comment of Admin.unassign. It will reassign the region
automatically.
> Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in
> HBASE-14614
> ---
>
> Key: HBASE-18353
> URL: https://issues.apache.org/jira/browse/HBASE-18353
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0-alpha-1
>Reporter: Stephen Yuan Jiang
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18353-v1.patch, HBASE-18353-v2.patch
>
>
> HBASE-14614 disabled TestCorruptedRegionStoreFile, as it depends on a
> half-implemented reopen of a region when a store file goes missing.
> This JIRA tracks the work to fix/enable the test.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18078) [C++] Harden RPC by handling various communication abnormalities
[ https://issues.apache.org/jira/browse/HBASE-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18078: -- Attachment: (was: HBASE-18078.008.patch) > [C++] Harden RPC by handling various communication abnormalities > > > Key: HBASE-18078 > URL: https://issues.apache.org/jira/browse/HBASE-18078 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18078.000.patch, HBASE-18078.001.patch, > HBASE-18078.002.patch, HBASE-18078.003.patch, HBASE-18078.004.patch, > HBASE-18078.005.patch, HBASE-18078.006.patch, HBASE-18078.007.patch > > > RPC layer should handle various communication abnormalities (e.g. connection > timeout, server aborted connection, and so on). Ideally, the corresponding > exceptions should be raised and propagated through handlers of pipeline in > client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18424) Fix TestAsyncTableGetMultiThreaded
[
https://issues.apache.org/jira/browse/HBASE-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117629#comment-16117629
]
Duo Zhang commented on HBASE-18424:
---
{code}
TEST_UTIL.getConfiguration().set(TABLES_ON_MASTER, "none");
{code}
At least for now there is a {{BaseLoadBalancer.TABLES_ON_MASTER}} config to
control whether master can carry any regions, so this is not a problem.
> Fix TestAsyncTableGetMultiThreaded
> --
>
> Key: HBASE-18424
> URL: https://issues.apache.org/jira/browse/HBASE-18424
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18424-v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18536) [C++] Add fault injection infra
[ https://issues.apache.org/jira/browse/HBASE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18536: -- Summary: [C++] Add fault injection infra (was: [C++] ) > [C++] Add fault injection infra > --- > > Key: HBASE-18536 > URL: https://issues.apache.org/jira/browse/HBASE-18536 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18536) [C++] Add fault injection infra
[ https://issues.apache.org/jira/browse/HBASE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou reassigned HBASE-18536: - Assignee: Xiaobing Zhou > [C++] Add fault injection infra > --- > > Key: HBASE-18536 > URL: https://issues.apache.org/jira/browse/HBASE-18536 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18536) [C++]
Xiaobing Zhou created HBASE-18536: - Summary: [C++] Key: HBASE-18536 URL: https://issues.apache.org/jira/browse/HBASE-18536 Project: HBase Issue Type: Sub-task Reporter: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18535) [C++] make RPC test mode transparent to initialization of RpcPipeline
[
https://issues.apache.org/jira/browse/HBASE-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobing Zhou updated HBASE-18535:
--
Description:
This is a follow up work of HBASE-18338.
In RpcPipelineFactory::newPipeline, the HBASE_CLIENT_RPC_TEST_MODE is used to
exclude SaslHandler which otherwise will cause RpcTestServer not receiving any
requests.
{code}
+ if (!conf_->GetBool(
+ Configuration::HBASE_CLIENT_RPC_TEST_MODE,
+ Configuration::DEFAULT_HBASE_CLIENT_RPC_TEST_MODE)) {
+secure = security::User::IsSecurityEnabled(*conf_);
+pipeline->addBack(SaslHandler{user_util_.user_name(secure), conf_});
+ }
{code}
This is not clean. Handlers should be added to pipeline regardless of test or
not, instead, every handler can choose to discriminate test or not. Taking
ClientHandler as an example:
{code}
folly::Future ClientHandler::write(Context *ctx,
std::unique_ptr r) {
/* for RPC test, there's no need to send connection header */
if (!conf_->GetBool(RpcSerde::HBASE_CLIENT_RPC_TEST_MODE,
RpcSerde::DEFAULT_HBASE_CLIENT_RPC_TEST_MODE)) {
// We need to send the header once.
// So use call_once to make sure that only one thread wins this.
std::call_once((*once_flag_), [ctx, this]() {
VLOG(3) << "Writing RPC Header to server: " << server_;
auto header = serde_.Header(user_name_);
ctx->fireWrite(std::move(header));
});
}
{code}
> [C++] make RPC test mode transparent to initialization of RpcPipeline
> -
>
> Key: HBASE-18535
> URL: https://issues.apache.org/jira/browse/HBASE-18535
> Project: HBase
> Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>
> This is a follow up work of HBASE-18338.
> In RpcPipelineFactory::newPipeline, the HBASE_CLIENT_RPC_TEST_MODE is used to
> exclude SaslHandler which otherwise will cause RpcTestServer not receiving
> any requests.
> {code}
> + if (!conf_->GetBool(
> + Configuration::HBASE_CLIENT_RPC_TEST_MODE,
> + Configuration::DEFAULT_HBASE_CLIENT_RPC_TEST_MODE)) {
> +secure = security::User::IsSecurityEnabled(*conf_);
> +pipeline->addBack(SaslHandler{user_util_.user_name(secure), conf_});
> + }
> {code}
> This is not clean. Handlers should be added to pipeline regardless of test or
> not, instead, every handler can choose to discriminate test or not. Taking
> ClientHandler as an example:
> {code}
> folly::Future ClientHandler::write(Context *ctx,
> std::unique_ptr r) {
> /* for RPC test, there's no need to send connection header */
> if (!conf_->GetBool(RpcSerde::HBASE_CLIENT_RPC_TEST_MODE,
> RpcSerde::DEFAULT_HBASE_CLIENT_RPC_TEST_MODE)) {
> // We need to send the header once.
> // So use call_once to make sure that only one thread wins this.
> std::call_once((*once_flag_), [ctx, this]() {
> VLOG(3) << "Writing RPC Header to server: " << server_;
> auto header = serde_.Header(user_name_);
> ctx->fireWrite(std::move(header));
> });
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18535) [C++] make RPC test mode transparent to initialization of RpcPipeline
Xiaobing Zhou created HBASE-18535: - Summary: [C++] make RPC test mode transparent to initialization of RpcPipeline Key: HBASE-18535 URL: https://issues.apache.org/jira/browse/HBASE-18535 Project: HBase Issue Type: Sub-task Reporter: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18534) [C++] Support timeout in Rpc
Xiaobing Zhou created HBASE-18534: - Summary: [C++] Support timeout in Rpc Key: HBASE-18534 URL: https://issues.apache.org/jira/browse/HBASE-18534 Project: HBase Issue Type: Sub-task Reporter: Xiaobing Zhou Assignee: Xiaobing Zhou -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18510: --- Attachment: HBASE-18510.HBASE-14070.HLC.002.patch > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18510.HBASE-14070.HLC.001.patch, > HBASE-18510.HBASE-14070.HLC.002.patch > > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4
[
https://issues.apache.org/jira/browse/HBASE-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117602#comment-16117602
]
Andrew Purtell commented on HBASE-18431:
I plan to commit the branch-1 patch to branch-1 and branch-1.4 tomorrow and
then resolve this. Let me know if you have any concerns or objections. We can
follow up for branch-2 (if desired) on another issue.
> Mitigate compatibility concerns between branch-1.3 and branch-1.4
> -
>
> Key: HBASE-18431
> URL: https://issues.apache.org/jira/browse/HBASE-18431
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0, 1.5.0
>
> Attachments: HBASE-18431-branch-1.4.patch,
> HBASE-18431-branch-1.patch, HBASE-18431-branch-2-WIP.patch
>
>
> There are compatibility concerns with branch-1.4.
> {noformat}
> Library Name HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject Binary Compatibility
> Compatibility - 89.9%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
> High - 23
> Medium - 9
> Low - 21
> {noformat}
> {noformat}
> Library Name HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject Source Compatibility
> Compatibility- 86.5%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
> High - 88
> Medium - 0
> Low - 0
> Other Changes in Data Types- 25
> {noformat}
> This report includes HBASE-15816 which hasn't been committed yet. Otherwise
> it's current.
> I'm not generally concerned with added methods.
> The following methods have been added to Public/Evolving interface Table.
> Pointing them out in case it merits review.
> \\
> * Abstract method Table.getReadRpcTimeout ( ) has been added to this
> interface. No effect.
> * Abstract method Table.getWriteRpcTimeout ( ) has been added to this
> interface. No effect.
> * Abstract method Table.setReadRpcTimeout ( int ) has been added to this
> interface. No effect.
> * Abstract method Table.setWriteRpcTimeout ( int ) has been added to this
> interface.
> The Public/Evolving interface Admin has some signature changes equating to
> removed methods. I don't think this is allowed in a minor release.
> \\
> * Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription
> ) has been removed from Admin.
> * Abstract method Admin.snapshot ( String, TableName,
> HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
> * Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been
> removed from Admin.
> * Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription
> ) has been removed from Admin.
> The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This
> change is debatable but I think we can allow it.
> \\
> * AsyncRpcClient has been removed
> The Public/Evolving class FastLongHistogram has been removed. I don't believe
> this change is allowed in a minor release.
> \\
> * FastLongHistogram has been removed
> Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and
> RegionObserver have changed, equating to removed methods. The first set of
> changes is due to move of SnapshotDescription from HBaseProtos to
> SnapshotProtos:
> \\
> * Abstract method MasterObserver.postCloneSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.postDeleteSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postListSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postRestoreSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.postSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.preCloneSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
> MasterObserver.
> * Abstract method MasterObserver.preDeleteSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preListSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preRestoreSnapshot (
> ObserverContext,
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from
>
[jira] [Commented] (HBASE-18248) Warn if monitored RPC task has been tied up beyond a configurable threshold
[ https://issues.apache.org/jira/browse/HBASE-18248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117601#comment-16117601 ] Andrew Purtell commented on HBASE-18248: If no more comment or concerns I'll commit the current patch. > Warn if monitored RPC task has been tied up beyond a configurable threshold > --- > > Key: HBASE-18248 > URL: https://issues.apache.org/jira/browse/HBASE-18248 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 3.0.0, 1.4.0 > > Attachments: HBASE-18248-branch-1.patch, HBASE-18248-branch-1.patch, > HBASE-18248.patch, HBASE-18248.patch > > > Warn if monitored task has been tied up beyond a configurable threshold. We > especially want to do this for RPC tasks. Use a separate threshold for > warning about stuck RPC tasks versus other types of tasks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18533: -- Status: Patch Available (was: Open) > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18533.branch-1.001.patch, > HBASE-18533.master.001.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18533: -- Attachment: HBASE-18533.master.001.patch > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18533.branch-1.001.patch, > HBASE-18533.master.001.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18533: -- Attachment: HBASE-18533.branch-1.001.patch > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18533.branch-1.001.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18533) Expose BucketCache values to be configured
Zach York created HBASE-18533: - Summary: Expose BucketCache values to be configured Key: HBASE-18533 URL: https://issues.apache.org/jira/browse/HBASE-18533 Project: HBase Issue Type: Improvement Components: BucketCache Reporter: Zach York Assignee: Zach York BucketCache always uses the default values for all cache configuration. However, this doesn't work for all use cases. In particular, users want to be able to configure the percentage of the cache that is single access, multi access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18231) Deprecate and throw unsupported operation when Admin#closeRegion is called.
[ https://issues.apache.org/jira/browse/HBASE-18231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117577#comment-16117577 ] stack commented on HBASE-18231: --- Admin will be broken for a few reasons. I like your concern though @appy wondering about how it fails. Let's try it and see. Let fill in info in a while. > Deprecate and throw unsupported operation when Admin#closeRegion is called. > --- > > Key: HBASE-18231 > URL: https://issues.apache.org/jira/browse/HBASE-18231 > Project: HBase > Issue Type: Sub-task > Components: Admin >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Appy >Priority: Critical > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18231.master.001.patch, > HBASE-18231.master.002.patch, HBASE-18231.master.003.patch > > > [~uagashe] tripped over this today. Admin#closeRegion which we used to use in > branch-1 will cause damage in AMv2 cluster. Instead you need to call unassign > -- i.e. all cluster ops must go via the Master; no more going direct to > RegionServer closing regions behind the Master's back. > Review all Admin ops to see what else skirts Master and deprecate and throw > unsupported if called. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18078) [C++] Harden RPC by handling various communication abnormalities
[ https://issues.apache.org/jira/browse/HBASE-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18078: -- Attachment: (was: HBASE-18078.009.patch) > [C++] Harden RPC by handling various communication abnormalities > > > Key: HBASE-18078 > URL: https://issues.apache.org/jira/browse/HBASE-18078 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18078.000.patch, HBASE-18078.001.patch, > HBASE-18078.002.patch, HBASE-18078.003.patch, HBASE-18078.004.patch, > HBASE-18078.005.patch, HBASE-18078.006.patch, HBASE-18078.007.patch, > HBASE-18078.008.patch > > > RPC layer should handle various communication abnormalities (e.g. connection > timeout, server aborted connection, and so on). Ideally, the corresponding > exceptions should be raised and propagated through handlers of pipeline in > client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18078) [C++] Harden RPC by handling various communication abnormalities
[ https://issues.apache.org/jira/browse/HBASE-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117573#comment-16117573 ] Xiaobing Zhou commented on HBASE-18078: --- Since the behavior of closing socket before sending request and after getting connection is beyond expectation (i.e.Broken promise for type unique_ptr), I've decided to move this case to separate ticket. v8 is the work to be committed, v9 is dropped. > [C++] Harden RPC by handling various communication abnormalities > > > Key: HBASE-18078 > URL: https://issues.apache.org/jira/browse/HBASE-18078 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18078.000.patch, HBASE-18078.001.patch, > HBASE-18078.002.patch, HBASE-18078.003.patch, HBASE-18078.004.patch, > HBASE-18078.005.patch, HBASE-18078.006.patch, HBASE-18078.007.patch, > HBASE-18078.008.patch > > > RPC layer should handle various communication abnormalities (e.g. connection > timeout, server aborted connection, and so on). Ideally, the corresponding > exceptions should be raised and propagated through handlers of pipeline in > client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18353) Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117558#comment-16117558 ] Vladimir Rodionov commented on HBASE-18353: --- Its RegionReassigner, [~syuanjiang] > Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in > HBASE-14614 > --- > > Key: HBASE-18353 > URL: https://issues.apache.org/jira/browse/HBASE-18353 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > Attachments: HBASE-18353-v1.patch, HBASE-18353-v2.patch > > > HBASE-14614 disabled TestCorruptedRegionStoreFile, as it depends on a > half-implemented reopen of a region when a store file goes missing. > This JIRA tracks the work to fix/enable the test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18519) Substitute IndividualBytesFieldCell for CellUtil.createCell
[
https://issues.apache.org/jira/browse/HBASE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117555#comment-16117555
]
Hadoop QA commented on HBASE-18519:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
15s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}140m
9s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
47s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}200m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18519 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880697/HBASE-18519.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 232c521fab88 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7e7461e |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7965/testReport/ |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output |
[jira] [Commented] (HBASE-18353) Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
[
https://issues.apache.org/jira/browse/HBASE-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117525#comment-16117525
]
Stephen Yuan Jiang commented on HBASE-18353:
[~vrodionov], reading HBASE-17712, I am unsure whether your approach is the
correct action. Let us wait for Duo's comment on the change. For the patch, I
think you should at least rename the {{RegionUnassigner.java}} file to
{{RegionReassigner.java}}
> Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in
> HBASE-14614
> ---
>
> Key: HBASE-18353
> URL: https://issues.apache.org/jira/browse/HBASE-18353
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0-alpha-1
>Reporter: Stephen Yuan Jiang
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18353-v1.patch, HBASE-18353-v2.patch
>
>
> HBASE-14614 disabled TestCorruptedRegionStoreFile, as it depends on a
> half-implemented reopen of a region when a store file goes missing.
> This JIRA tracks the work to fix/enable the test.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18353) Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
[
https://issues.apache.org/jira/browse/HBASE-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117522#comment-16117522
]
Stephen Yuan Jiang commented on HBASE-18353:
[~Apache9], in HBASE-17712, you mentioned that {{"Unassign region
asynchronously when hitting FNFE. Can pass TestCorruptedRegionStoreFile. And
also fix a problem in TestCorruptedRegionStoreFile that the already opened
DFSInputStream may still work after we deleted the storefile because the block
replica deletion is asynchronous. We should wait until all the replicas have
been removed from DNs."}} You also talked about implementing some reassign
logic.
[~vrodionov] implemented this by doing unassign/assign of the region if FNFE
happens. How do you think?
> Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in
> HBASE-14614
> ---
>
> Key: HBASE-18353
> URL: https://issues.apache.org/jira/browse/HBASE-18353
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 2.0.0-alpha-1
>Reporter: Stephen Yuan Jiang
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18353-v1.patch, HBASE-18353-v2.patch
>
>
> HBASE-14614 disabled TestCorruptedRegionStoreFile, as it depends on a
> half-implemented reopen of a region when a store file goes missing.
> This JIRA tracks the work to fix/enable the test.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18520) Add jmx value to determine true Master Start time
[ https://issues.apache.org/jira/browse/HBASE-18520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117513#comment-16117513 ] Zach York commented on HBASE-18520: --- How do I find who the release managers are? Is there a list? [~stack] I think you are the RM for branch-2. Any reservations on putting this in branch-2? > Add jmx value to determine true Master Start time > - > > Key: HBASE-18520 > URL: https://issues.apache.org/jira/browse/HBASE-18520 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Zach York >Assignee: Zach York >Priority: Minor > Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-2 > > Attachments: HBASE-18520.branch-1.001.patch, > HBASE-18520.branch-1.002.patch, HBASE-18520.master.001.patch, > HBASE-18520.master.002.patch, HBASE-18520.master.003.patch > > > The masterActiveTime is being set before regions are assigned. This patch > adds a new jmx metric to expose the final time when the master has become the > active master (All regions are assigned, etc.). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18204) [C++] Rpc connection close and reconnecting
[ https://issues.apache.org/jira/browse/HBASE-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-18204: -- Attachment: hbase-18204_v4.patch > [C++] Rpc connection close and reconnecting > --- > > Key: HBASE-18204 > URL: https://issues.apache.org/jira/browse/HBASE-18204 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-18204_v1.patch, hbase-18204_v2.patch, > hbase-18204_v3.patch, hbase-18204_v4.patch > > > Our client-dispatcher maintains a map of outgoing RPCs per server with the > promises. > In case the server goes down, or TCP connection is closed, we should complete > the outgoing RPCs with exceptions so that higher level waiters can unblock > and retry. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117435#comment-16117435 ] Ted Yu commented on HBASE-16179: My response above is based on the assumption that any future patch would be a rehash of one of my posted patches. Please keep the assignee as myself. Unless there is significant change in the implementation in which case we need to all agree on the approach. It is fine to mention both names at the time of commit. Again, what's the plan for proceeding ? How many versions of Spark are we supporting ? Please don't change the assignee without my consent. > Fix compilation errors when building hbase-spark against Spark 2.0 > -- > > Key: HBASE-16179 > URL: https://issues.apache.org/jira/browse/HBASE-16179 > Project: HBase > Issue Type: Bug > Components: spark >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Labels: build > Fix For: 2.0.0 > > Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, > 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, 16179.v15.txt, > 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, 16179.v1.txt, > 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, > 16179.v25.txt, 16179.v26.txt, 16179.v27.txt, 16179.v4.txt, 16179.v5.txt, > 16179.v7.txt, 16179.v8.txt, 16179.v9.txt > > > I tried building hbase-spark module against Spark-2.0 snapshot and got the > following compilation errors: > http://pastebin.com/bg3w247a > Some Spark classes such as DataTypeParser and Logging are no longer > accessible to downstream projects. > hbase-spark module should not depend on such classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18424) Fix TestAsyncTableGetMultiThreaded
[
https://issues.apache.org/jira/browse/HBASE-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117434#comment-16117434
]
Vladimir Rodionov commented on HBASE-18424:
---
That is the issue.
{quote}
admin.move(HRegionInfo.FIRST_META_REGIONINFO.getEncodedNameAsBytes(),
Bytes.toBytes(newMetaServer.getServerName()));
{quote}
2.0 does not allow to re-assign meta. It is always hosted by Master.
> Fix TestAsyncTableGetMultiThreaded
> --
>
> Key: HBASE-18424
> URL: https://issues.apache.org/jira/browse/HBASE-18424
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18424-v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18353) Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117424#comment-16117424 ] Vladimir Rodionov commented on HBASE-18353: --- [~syuanjiang] any feedback? > Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in > HBASE-14614 > --- > > Key: HBASE-18353 > URL: https://issues.apache.org/jira/browse/HBASE-18353 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > Attachments: HBASE-18353-v1.patch, HBASE-18353-v2.patch > > > HBASE-14614 disabled TestCorruptedRegionStoreFile, as it depends on a > half-implemented reopen of a region when a store file goes missing. > This JIRA tracks the work to fix/enable the test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18522) Add RowMutations support to Batch
[ https://issues.apache.org/jira/browse/HBASE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116111#comment-16116111 ] Ted Yu edited comment on HBASE-18522 at 8/7/17 10:12 PM: - I was talking about 'Integer rowMutationsIndex' which can be declared as int. However, since we don't know whether the map contains i or not, you can keep the current formation. Please check failed tests (they should fail without patch). was (Author: yuzhih...@gmail.com): I was talking about 'Integer rowMutationsIndex' which can be declared as int. Please check failed tests (they should fail without patch). > Add RowMutations support to Batch > - > > Key: HBASE-18522 > URL: https://issues.apache.org/jira/browse/HBASE-18522 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.6 >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18522-branch-1.patch > > > RowMutations is multiple Puts and/or Deletes atomically on a single row. > Current Batch call does not support RowMutations as part of the batch. > We should add this missing part. We should be able to batch RowMutations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18529) Do not delete the tmp jars dir when load the coprocessor jar
[
https://issues.apache.org/jira/browse/HBASE-18529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117398#comment-16117398
]
Biju Nair commented on HBASE-18529:
---
Will it be better by having different directories for the multiple RSes running
on the same node so that they are well isolated?
> Do not delete the tmp jars dir when load the coprocessor jar
>
>
> Key: HBASE-18529
> URL: https://issues.apache.org/jira/browse/HBASE-18529
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Reporter: Yun Zhao
>Assignee: Yun Zhao
> Attachments: HBASE-18529.master.001.patch,
> HBASE-18529.master.002.patch
>
>
> When multi regionserver is deployed on a single server, used default
> hbase.local.dir . The tmp jars dir will deleted when one of them is restarted.
> Also when multi regionserver start at the same time, the jar in the
> copyToLocalFile process may be deleted, causing the coprocessor load failed.
> {code}
> 2017-08-06 20:02:15,326 ERROR [RS_OPEN_REGION--2]
> regionserver.RegionCoprocessorHost: Failed to load coprocessor
> ENOENT: No such file or directory
> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native
> Method)
> at
> org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:226)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:629)
> at
> org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:467)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:424)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:887)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:784)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:365)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:338)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
> at
> org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1968)
> at
> org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1937)
> at
> org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1913)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.init(CoprocessorClassLoader.java:168)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.getClassLoader(CoprocessorClassLoader.java:250)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18244) org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups hangs/fails
[
https://issues.apache.org/jira/browse/HBASE-18244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117340#comment-16117340
]
stack commented on HBASE-18244:
---
[~elserj] Thanks.
> org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups hangs/fails
>
>
> Key: HBASE-18244
> URL: https://issues.apache.org/jira/browse/HBASE-18244
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Josh Elser
>Assignee: Stephen Yuan Jiang
> Fix For: 2.0.0
>
> Attachments: HBASE-18244.001.patch
>
>
> Sometime in the past couple of weeks, TestShellRSGroups has started
> timing-out/failing for me.
> It will get stuck on a call to moveTables()
> {noformat}
> "main" #1 prio=5 os_prio=31 tid=0x7ff012004800 nid=0x1703 in
> Object.wait() [0x7020d000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at
> org.apache.hadoop.hbase.ipc.BlockingRpcCallback.get(BlockingRpcCallback.java:62)
> - locked <0x00078d1003f0> (a
> org.apache.hadoop.hbase.ipc.BlockingRpcCallback)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:328)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$200(AbstractRpcClient.java:94)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:567)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$BlockingStub.execMasterService(MasterProtos.java)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation$3.execMasterService(ConnectionImplementation.java:1500)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$67$1.rpcCall(HBaseAdmin.java:2991)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$67$1.rpcCall(HBaseAdmin.java:2986)
> at
> org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:98)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$67.callExecService(HBaseAdmin.java:2997)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:69)
> at
> org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveTables(RSGroupAdminProtos.java:13171)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveTables(RSGroupAdminClient.java:117)
> {noformat}
> The server-side end of the RPC is waiting on a procedure to finish:
> {noformat}
> "RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=64242" #289 daemon
> prio=5 os_prio=31 tid=0x7ff015b7c000 nid=0x1e603 waiting on condition
> [0x7dbc9000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(ProcedureSyncWait.java:184)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(ProcedureSyncWait.java:171)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitForProcedureToComplete(ProcedureSyncWait.java:141)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitForProcedureToCompleteIOE(ProcedureSyncWait.java:130)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitAndWaitProcedure(ProcedureSyncWait.java:123)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.unassign(AssignmentManager.java:478)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.unassign(AssignmentManager.java:465)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:432)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveTables(RSGroupAdminEndpoint.java:174)
> at
> org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:12786)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:673)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:278)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:258)
>Locked ownable synchronizers:
> - None
> {noformat}
> I don't see anything
[jira] [Updated] (HBASE-18204) [C++] Rpc connection close and reconnecting
[ https://issues.apache.org/jira/browse/HBASE-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-18204: -- Attachment: hbase-18204_v3.patch > [C++] Rpc connection close and reconnecting > --- > > Key: HBASE-18204 > URL: https://issues.apache.org/jira/browse/HBASE-18204 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: hbase-18204_v1.patch, hbase-18204_v2.patch, > hbase-18204_v3.patch > > > Our client-dispatcher maintains a map of outgoing RPCs per server with the > promises. > In case the server goes down, or TCP connection is closed, we should complete > the outgoing RPCs with exceptions so that higher level waiters can unblock > and retry. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18514) Backport space quota "phase2" work to branch-2
[ https://issues.apache.org/jira/browse/HBASE-18514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18514: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Didn't force someone else to re-review these changes as 2.0 hasn't really changed over 3.0 when these were reviewed (a thousand apologies if I step on some toes because of that). > Backport space quota "phase2" work to branch-2 > -- > > Key: HBASE-18514 > URL: https://issues.apache.org/jira/browse/HBASE-18514 > Project: HBase > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18514.001.branch-2.patch > > > People generally seem to be in favor of backporting the phase 2 work > (includes the size of hbase snapshots in quota rules) for the hbase-2.0 > release. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18387) [Thrift] Make principal configurable in DemoClient.java
[
https://issues.apache.org/jira/browse/HBASE-18387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117298#comment-16117298
]
Josh Elser commented on HBASE-18387:
{code}
+System.out.println("Usage: DemoClient host port [secure=false]
[server-principal=hbase]");
{code}
I think the parsing could be simplified if instead the arguments were
{{DemoClient host port \[secure=false \[server-principal=hbase\] \]}} (the
server principal option can only be provided if the secure option is also
provided). Trying to do optional-positional arguments in the manner you tried
to implement is super confusing, IMO (named args are a bit more clear in that
regard, but I digress).
Any chance I could convince you to tweak this? :)
> [Thrift] Make principal configurable in DemoClient.java
> ---
>
> Key: HBASE-18387
> URL: https://issues.apache.org/jira/browse/HBASE-18387
> Project: HBase
> Issue Type: Improvement
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-18387.master.001.patch,
> HBASE-18387.master.002.patch
>
>
> In the Thrift1 demo client we have this code:
> {code}
> transport = new TSaslClientTransport("GSSAPI", null,
> "hbase", // Thrift server user name, should be an authorized proxy user.
> host, // Thrift server domain
> saslProperties, null, transport);
> {code}
> This will only work when the Thrift server is started with the {{hbase}}
> principal. Often this may deviate, for example I am using {{hbase-thrift}} to
> separate the names from those of backend servers.
> What we need is either an additional command line option to specify the name,
> or a property that can be set with -D and can be passed at runtime. I prefer
> the former, as the latter is making this a little convoluted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117284#comment-16117284 ] Sean Busbey commented on HBASE-16179: - Feel free to attach stuff if you have time before I get to it, [~easyliangjob]. I'll hold off assigning to myself until I have time blocked out. If you assign the issue to yourself first, then I'll be happy to review whatever you have. > Fix compilation errors when building hbase-spark against Spark 2.0 > -- > > Key: HBASE-16179 > URL: https://issues.apache.org/jira/browse/HBASE-16179 > Project: HBase > Issue Type: Bug > Components: spark >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Labels: build > Fix For: 2.0.0 > > Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, > 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, 16179.v15.txt, > 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, 16179.v1.txt, > 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, > 16179.v25.txt, 16179.v26.txt, 16179.v27.txt, 16179.v4.txt, 16179.v5.txt, > 16179.v7.txt, 16179.v8.txt, 16179.v9.txt > > > I tried building hbase-spark module against Spark-2.0 snapshot and got the > following compilation errors: > http://pastebin.com/bg3w247a > Some Spark classes such as DataTypeParser and Logging are no longer > accessible to downstream projects. > hbase-spark module should not depend on such classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117262#comment-16117262 ] Yi Liang commented on HBASE-16179: -- Hi [~busbey] I used to write a patch that make hbase-spark only support spark 2.0, my patch is based on v7 or v8 of above Ted's patch(do not remember it clearly), if you need help, I can contribute it to this jira. > Fix compilation errors when building hbase-spark against Spark 2.0 > -- > > Key: HBASE-16179 > URL: https://issues.apache.org/jira/browse/HBASE-16179 > Project: HBase > Issue Type: Bug > Components: spark >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Labels: build > Fix For: 2.0.0 > > Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, > 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, 16179.v15.txt, > 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, 16179.v1.txt, > 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, > 16179.v25.txt, 16179.v26.txt, 16179.v27.txt, 16179.v4.txt, 16179.v5.txt, > 16179.v7.txt, 16179.v8.txt, 16179.v9.txt > > > I tried building hbase-spark module against Spark-2.0 snapshot and got the > following compilation errors: > http://pastebin.com/bg3w247a > Some Spark classes such as DataTypeParser and Logging are no longer > accessible to downstream projects. > hbase-spark module should not depend on such classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18244) org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups hangs/fails
[
https://issues.apache.org/jira/browse/HBASE-18244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117242#comment-16117242
]
Josh Elser commented on HBASE-18244:
Just noticed that this didn't make it to branch-2 (somehow? branch-snafu I
suppose). Going to push that over there.
> org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups hangs/fails
>
>
> Key: HBASE-18244
> URL: https://issues.apache.org/jira/browse/HBASE-18244
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Josh Elser
>Assignee: Stephen Yuan Jiang
> Fix For: 2.0.0
>
> Attachments: HBASE-18244.001.patch
>
>
> Sometime in the past couple of weeks, TestShellRSGroups has started
> timing-out/failing for me.
> It will get stuck on a call to moveTables()
> {noformat}
> "main" #1 prio=5 os_prio=31 tid=0x7ff012004800 nid=0x1703 in
> Object.wait() [0x7020d000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at
> org.apache.hadoop.hbase.ipc.BlockingRpcCallback.get(BlockingRpcCallback.java:62)
> - locked <0x00078d1003f0> (a
> org.apache.hadoop.hbase.ipc.BlockingRpcCallback)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:328)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$200(AbstractRpcClient.java:94)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:567)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$BlockingStub.execMasterService(MasterProtos.java)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation$3.execMasterService(ConnectionImplementation.java:1500)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$67$1.rpcCall(HBaseAdmin.java:2991)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$67$1.rpcCall(HBaseAdmin.java:2986)
> at
> org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:98)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$67.callExecService(HBaseAdmin.java:2997)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:69)
> at
> org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveTables(RSGroupAdminProtos.java:13171)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveTables(RSGroupAdminClient.java:117)
> {noformat}
> The server-side end of the RPC is waiting on a procedure to finish:
> {noformat}
> "RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=64242" #289 daemon
> prio=5 os_prio=31 tid=0x7ff015b7c000 nid=0x1e603 waiting on condition
> [0x7dbc9000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(ProcedureSyncWait.java:184)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(ProcedureSyncWait.java:171)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitForProcedureToComplete(ProcedureSyncWait.java:141)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitForProcedureToCompleteIOE(ProcedureSyncWait.java:130)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitAndWaitProcedure(ProcedureSyncWait.java:123)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.unassign(AssignmentManager.java:478)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.unassign(AssignmentManager.java:465)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:432)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveTables(RSGroupAdminEndpoint.java:174)
> at
> org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:12786)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:673)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:278)
> at
>
[jira] [Updated] (HBASE-18519) Substitute IndividualBytesFieldCell for CellUtil.createCell
[ https://issues.apache.org/jira/browse/HBASE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18519: --- Status: Patch Available (was: Open) Review board: https://reviews.apache.org/r/61478 > Substitute IndividualBytesFieldCell for CellUtil.createCell > --- > > Key: HBASE-18519 > URL: https://issues.apache.org/jira/browse/HBASE-18519 > Project: HBase > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18519.v0.patch, HBASE-18519.v1.patch > > > The IndividualBytesFieldCell, which is introduced by HBASE-14882, carries the > input arguments without copying. We can substitute IndividualBytesFieldCell > for CellUtil.createCell. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18519) Substitute IndividualBytesFieldCell for CellUtil.createCell
[ https://issues.apache.org/jira/browse/HBASE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18519: --- Attachment: HBASE-18519.v1.patch v1 # address ted's comment # introduce the cell builder which creates cell with shadow/deep copy > Substitute IndividualBytesFieldCell for CellUtil.createCell > --- > > Key: HBASE-18519 > URL: https://issues.apache.org/jira/browse/HBASE-18519 > Project: HBase > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18519.v0.patch, HBASE-18519.v1.patch > > > The IndividualBytesFieldCell, which is introduced by HBASE-14882, carries the > input arguments without copying. We can substitute IndividualBytesFieldCell > for CellUtil.createCell. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18519) Substitute IndividualBytesFieldCell for CellUtil.createCell
[ https://issues.apache.org/jira/browse/HBASE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18519: --- Status: Open (was: Patch Available) > Substitute IndividualBytesFieldCell for CellUtil.createCell > --- > > Key: HBASE-18519 > URL: https://issues.apache.org/jira/browse/HBASE-18519 > Project: HBase > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18519.v0.patch > > > The IndividualBytesFieldCell, which is introduced by HBASE-14882, carries the > input arguments without copying. We can substitute IndividualBytesFieldCell > for CellUtil.createCell. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18532: --- Component/s: UI > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 1.1.2 >Reporter: Biju Nair > Attachments: COMBINED-STATS.PNG, L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18500) Performance issue: Don't use BufferedMutator for HTable's put method
[
https://issues.apache.org/jira/browse/HBASE-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117087#comment-16117087
]
Chia-Ping Tsai commented on HBASE-18500:
bq. Was it before 1.3 itself?
Yes, see HBASE-2898
> Performance issue: Don't use BufferedMutator for HTable's put method
>
>
> Key: HBASE-18500
> URL: https://issues.apache.org/jira/browse/HBASE-18500
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18500-v1.patch, HBASE-18500-v2.patch,
> HBASE-18500-v3.patch, HBASE-18500-v4.patch
>
>
> Copied the test result from HBASE-17994.
> Run start-hbase.sh in my local computer and use the default config to test
> with PE tool.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --autoFlush=True randomWrite 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --autoFlush=True asyncRandomWrite 1
> {code}
> Mean latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 164.39 | 161.22 | 164.78 | 140.61 | 151.69 |
> | asyncRandomWrite | 122.29 | 125.58 | 122.23 | 113.18 | 123.02 |
> 50th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 130.00 | 125.00 | 123.00 | 112.00 | 121.00 |
> | asyncRandomWrite | 95.00 | 97.00 | 95.00 | 88.00 | 95.00 |
> 99th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 600.00 | 600.00 | 650.00 | 404.00 | 425.00 |
> | asyncRandomWrite | 339.00 | 327.00 | 297.00 | 311.00 | 318.00 |
> In our internal 0.98 branch, the PE test result shows the async write has the
> almost same latency with the blocking write. But for master branch, the
> result shows the async write has better latency than the blocking client.
> Take a look about the code, I thought the difference is the BufferedMutator.
> For master branch, HTable don't have a write buffer and all write request
> will be flushed directly. And user can use BufferedMutator when user want to
> perform client-side buffering of writes. For the performance issue
> (autoFlush=True), I thought we can use rpc caller directly in HTable's put
> method. Thanks.
> Review: https://reviews.apache.org/r/61454/
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117071#comment-16117071 ] Chia-Ping Tsai edited comment on HBASE-18142 at 8/7/17 7:02 PM: bq. Please fix the warnings. Sorry for the misunderstanding. Please ignore the comment. was (Author: chia7712): bq. Please fix the warnings. Sorry for the misunderstanding. Please ignore this comment. > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumns(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[ https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117071#comment-16117071 ] Chia-Ping Tsai commented on HBASE-18142: bq. Please fix the warnings. Sorry for the misunderstanding. Please ignore this comment. > Deletion of a cell deletes the previous versions too > > > Key: HBASE-18142 > URL: https://issues.apache.org/jira/browse/HBASE-18142 > Project: HBase > Issue Type: Bug > Components: API, shell >Affects Versions: 3.0.0 >Reporter: Karthick >Assignee: ChunHao > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18142.master.v0.patch, > HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, > HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, > HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, > HBASE-18142.master.v7.patch > > > When I tried to delete a cell using it's timestamp in the Hbase Shell, the > previous versions of the same cell also got deleted. But when I tried the > same using the Java API, then the previous versions are not deleted and I can > retrive the previous values. > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java > see this file to fix the issue. This method (public Delete addColumns(final > byte [] family, final byte [] qualifier, final long timestamp)) only deletes > the current version of the cell. The previous versions are not deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair updated HBASE-18532: -- Attachment: L2-STATS.PNG COMBINED-STATS.PNG L1-STATS.PNG > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.1.2 >Reporter: Biju Nair > Attachments: COMBINED-STATS.PNG, L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18532) Improve cache related stats rendered on RS UI
Biju Nair created HBASE-18532: - Summary: Improve cache related stats rendered on RS UI Key: HBASE-18532 URL: https://issues.apache.org/jira/browse/HBASE-18532 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 1.1.2 Reporter: Biju Nair The stats currently rendered for L1 and L2 cache are incorrect. Refer to the attached screenshots of stats from a cluster showing the combined cache stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used for cache while if we sum size of L1 and L2 cache the value is way less. One way we can improve this is to use the same stats used to populate the combined stats to render the values of L1 & L2 cache. Also for usability we can remove the table with details with BucketCache buckets from the L2 cache stats since this is going to be long for any installation using L2 cache. This will help in understanding the cache usage better. Thoughts? If there are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope
[
https://issues.apache.org/jira/browse/HBASE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Rodionov updated HBASE-18526:
--
Attachment: HBASE-18526-v1.patch
Small patch. [~larsgeorge], can you take a look? Thanks.
> FIFOCompactionPolicy pre-check uses wrong scope
> ---
>
> Key: HBASE-18526
> URL: https://issues.apache.org/jira/browse/HBASE-18526
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.3.1
>Reporter: Lars George
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18526-v1.patch
>
>
> See https://issues.apache.org/jira/browse/HBASE-14468
> It adds this check to {{HMaster.checkCompactionPolicy()}}:
> {code}
> // 1. Check TTL
> if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
> message = "Default TTL is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 2. Check min versions
> if (hcd.getMinVersions() > 0) {
> message = "MIN_VERSION > 0 is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 3. blocking file count
> String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
> if (sbfc != null) {
> blockingFileCount = Integer.parseInt(sbfc);
> }
> if (blockingFileCount < 1000) {
> message =
> "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
> + blockingFileCount
> + " is below recommended minimum of 1000";
> throw new IOException(message);
> }
> {code}
> Why does it only check the blocking file count on the HTD level, while
> others are check on the HCD level? Doing this for example fails
> because of it:
> {noformat}
> hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
> CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
> => 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
> 'hbase.hstore.blockingStoreFiles' => 2000 } }
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
> count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
> minimum of 1000 Set hbase.table.sanity.checks to false at conf or
> table descriptor if you want to bypass sanity checks
> at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.io.IOException: blocking file count
> 'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
> 1000
> at
> org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
> ... 7 more
> {noformat}
> The check should be performed on the column family level instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
