[jira] [Commented] (HBASE-17237) Override the correct compact method in HMobStore
[ https://issues.apache.org/jira/browse/HBASE-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741242#comment-15741242 ] Hudson commented on HBASE-17237: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2117 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2117/]) HBASE-17237 Override the correct compact method in HMobStore (jingchengdu: rev 1615f45b31f4a8ae2344c1b3b2534f151141fa5d) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHMobStore.java > Override the correct compact method in HMobStore > > > Key: HBASE-17237 > URL: https://issues.apache.org/jira/browse/HBASE-17237 > Project: HBase > Issue Type: Bug > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Fix For: 2.0.0 > > Attachments: HBASE-17237-V2.patch, HBASE-17237.patch > > > The method Store#compact(CompactionContext compaction, ThroughputController > throughputController) is deprecated now, and HMobStore overrides it. > HMobStore needs to override compact(CompactionContext compaction, > ThroughputController throughputController, User user) instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17254) Do alignment in heapOverhead() for ExtendedCell implementations
[ https://issues.apache.org/jira/browse/HBASE-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-17254: - Description: In KeyValue (and maybe other ExtendedCell implementations), heapOverhead() does not do alignment(padding gap). I think alignment is required when calculating heapOverhead, because the space losses due to alignment can not be used by other objects in heap. (was: In KeyValue(and there might be other ExtendedCell implementations), heapOverhead() does not do alignment(padding gap). I think alignment is required when calculating heapOverhead, because the space losses due to alignment can not be used by other objects in heap.) > Do alignment in heapOverhead() for ExtendedCell implementations > --- > > Key: HBASE-17254 > URL: https://issues.apache.org/jira/browse/HBASE-17254 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Critical > > In KeyValue (and maybe other ExtendedCell implementations), heapOverhead() > does not do alignment(padding gap). I think alignment is required when > calculating heapOverhead, because the space losses due to alignment can not > be used by other objects in heap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[ https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17292: --- Status: Patch Available (was: Open) > Add observer notification before bulk loaded hfile is moved to region > directory > --- > > Key: HBASE-17292 > URL: https://issues.apache.org/jira/browse/HBASE-17292 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17292.v1.txt > > > Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded > hfiles. > However, if bulk load fails after hfile is moved to region directory but > before postBulkLoadHFile() hook is called, there is no way for pluggable > components (replication, backup / restore) to know which hfile(s) have been > moved to region directory. > This issue adds a preCommitStoreFile() hook which notifies path of to be > committed hfile before bulk loaded hfile is moved to region directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[ https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17292: --- Attachment: 17292.v1.txt > Add observer notification before bulk loaded hfile is moved to region > directory > --- > > Key: HBASE-17292 > URL: https://issues.apache.org/jira/browse/HBASE-17292 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17292.v1.txt > > > Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded > hfiles. > However, if bulk load fails after hfile is moved to region directory but > before postBulkLoadHFile() hook is called, there is no way for pluggable > components (replication, backup / restore) to know which hfile(s) have been > moved to region directory. > This issue adds a preCommitStoreFile() hook which notifies path of to be > committed hfile before bulk loaded hfile is moved to region directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
Ted Yu created HBASE-17292: -- Summary: Add observer notification before bulk loaded hfile is moved to region directory Key: HBASE-17292 URL: https://issues.apache.org/jira/browse/HBASE-17292 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Ted Yu Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded hfiles. However, if bulk load fails after hfile is moved to region directory but before postBulkLoadHFile() hook is called, there is no way for pluggable components (replication, backup / restore) to know which hfile(s) have been moved to region directory. This issue adds a preCommitStoreFile() hook which notifies path of to be committed hfile before bulk loaded hfile is moved to region directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16033) Add more details in logging of responseTooSlow/TooLarge
[
https://issues.apache.org/jira/browse/HBASE-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741112#comment-15741112
]
Yu Li commented on HBASE-16033:
---
I guess you meant HBASE-16972 instead of this one sir? [~ndimiduk]
For HBASE-16972, since {{RpcServerInterface}} is marked as {{IA.Evolving}}, I
guess adding a new method is ok? Maybe adding some release note there is
better? Let me know if any action to take, thanks.
> Add more details in logging of responseTooSlow/TooLarge
> ---
>
> Key: HBASE-16033
> URL: https://issues.apache.org/jira/browse/HBASE-16033
> Project: HBase
> Issue Type: Improvement
> Components: Operability
>Affects Versions: 1.2.3, 1.1.7
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21, 1.2.4, 1.1.8
>
> Attachments: HBASE-16033.patch, HBASE-16033.patch, HBASE-16033.patch
>
>
> Currently the log message when responseTooSlow/TooLarge is like:
> {noformat}
> 2016-06-08 12:18:04,363 WARN
> [B.defaultRpcServer.handler=127,queue=10,port=16020]
> ipc.RpcServer: (responseTooSlow):
> {"processingtimems":13125,"call":"Multi(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$MultiRequest)",
> "client":"11.251.158.22:36331","starttimems":1465359471238,"queuetimems":1540116,
> "class":"HRegionServer","responsesize":17,"method":"Multi"}
> {noformat}
> which is kind of helpless for debugging since we don't know on which
> table/region/row the request is against.
> What's more, we could see some if-else check in the {{RpcServer#logResponse}}
> method which trying to do sth different when the {{param}} includes instance
> of {{Operation}}, but there's only one place invoking {{logResponse}} and the
> {{param}} is always an instance of {{Message}}. Checking the change history,
> I believe this is a left-over cleanup in work of HBASE-8214
> We will address the above issues, do some cleanup and improve the log just
> like {{RpcServer$Call#toString}} does to include table/region/row information
> of the request
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-17291) Remove ImmutableSegment#getKeyValueScanner
ramkrishna.s.vasudevan created HBASE-17291: -- Summary: Remove ImmutableSegment#getKeyValueScanner Key: HBASE-17291 URL: https://issues.apache.org/jira/browse/HBASE-17291 Project: HBase Issue Type: Improvement Components: Scanners Affects Versions: 2.0.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 This is based on a discussion over [~anastas]'s patch. The MemstoreSnapshot uses a KeyValueScanner which actually seems redundant considering we already have a SegmentScanner. The idea is that the snapshot scanner should be a simple iterator type of scanner but it lacks the capability to do the reference counting on that segment that is now used in snapshot. With snapshot having mulitple segments in the latest impl it is better we hold on to the segment by doing ref counting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17230) port HBASE-17112 to 1.3.1
[ https://issues.apache.org/jira/browse/HBASE-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-17230: Affects Version/s: 1.3.0 > port HBASE-17112 to 1.3.1 > - > > Key: HBASE-17230 > URL: https://issues.apache.org/jira/browse/HBASE-17230 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 1.3.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17230) port HBASE-17112 to 1.3.1
[ https://issues.apache.org/jira/browse/HBASE-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-17230: Fix Version/s: 1.3.1 > port HBASE-17112 to 1.3.1 > - > > Key: HBASE-17230 > URL: https://issues.apache.org/jira/browse/HBASE-17230 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 1.3.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17175) backport HBASE-17127 to 1.3.1
[ https://issues.apache.org/jira/browse/HBASE-17175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-17175: Fix Version/s: 1.3.1 > backport HBASE-17127 to 1.3.1 > - > > Key: HBASE-17175 > URL: https://issues.apache.org/jira/browse/HBASE-17175 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Yu Li >Assignee: Yu Li > Fix For: 1.3.1 > > > branch-1.3 is currently freeze for 1.3.0 release, need to backport > HBASE-17127 later -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16398) optimize HRegion computeHDFSBlocksDistribution
[
https://issues.apache.org/jira/browse/HBASE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741053#comment-15741053
]
Hadoop QA commented on HBASE-16398:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 25s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m 14s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 126m 27s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12842749/HBASE-16398_v4.patch |
| JIRA Issue | HBASE-16398 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b13abe192ac4 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c24a055 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4877/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4877/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> optimize HRegion computeHDFSBlocksDistribution
> --
>
> Key: HBASE-16398
> URL: https://issues.apache.org/jira/browse/HBASE-16398
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Affects Versions: 2.0.0
>Reporter:
[jira] [Commented] (HBASE-17237) Override the correct compact method in HMobStore
[
https://issues.apache.org/jira/browse/HBASE-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741023#comment-15741023
]
Hadoop QA commented on HBASE-17237:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m 19s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 136m 32s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12842748/HBASE-17237-V2.patch |
| JIRA Issue | HBASE-17237 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 40f108177b53 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c24a055 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4876/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4876/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Override the correct compact method in HMobStore
>
>
> Key: HBASE-17237
> URL: https://issues.apache.org/jira/browse/HBASE-17237
> Project: HBase
> Issue Type: Bug
> Components: mob
>Reporter: Jingcheng Du
>Assignee: Jingcheng Du
> Fix For: 2.0.0
>
> Attachments: HBASE-17237-V2.patch, HBASE-17237.patch
>
>
> The method
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740924#comment-15740924 ] Ted Yu commented on HBASE-17289: HBASE-17289-branch-1.patch looks good. > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.1.patch, > HBASE-17289-branch-1.2.patch, HBASE-17289-branch-1.3.patch, > HBASE-17289-branch-1.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17237) Override the correct compact method in HMobStore
[ https://issues.apache.org/jira/browse/HBASE-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-17237: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) > Override the correct compact method in HMobStore > > > Key: HBASE-17237 > URL: https://issues.apache.org/jira/browse/HBASE-17237 > Project: HBase > Issue Type: Bug > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Fix For: 2.0.0 > > Attachments: HBASE-17237-V2.patch, HBASE-17237.patch > > > The method Store#compact(CompactionContext compaction, ThroughputController > throughputController) is deprecated now, and HMobStore overrides it. > HMobStore needs to override compact(CompactionContext compaction, > ThroughputController throughputController, User user) instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17237) Override the correct compact method in HMobStore
[ https://issues.apache.org/jira/browse/HBASE-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740897#comment-15740897 ] Jingcheng Du commented on HBASE-17237: -- Pushed to master. Thanks [~tedyu] and [~anoop.hbase] for review. > Override the correct compact method in HMobStore > > > Key: HBASE-17237 > URL: https://issues.apache.org/jira/browse/HBASE-17237 > Project: HBase > Issue Type: Bug > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Attachments: HBASE-17237-V2.patch, HBASE-17237.patch > > > The method Store#compact(CompactionContext compaction, ThroughputController > throughputController) is deprecated now, and HMobStore overrides it. > HMobStore needs to override compact(CompactionContext compaction, > ThroughputController throughputController, User user) instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[
https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740858#comment-15740858
]
Hadoop QA commented on HBASE-17289:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 55s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 34s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 55s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 83m 23s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 121m 34s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestSerialReplication |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:e01ee2f |
| JIRA Patch URL |
[jira] [Updated] (HBASE-16398) optimize HRegion computeHDFSBlocksDistribution
[ https://issues.apache.org/jira/browse/HBASE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-16398: - Attachment: HBASE-16398_v4.patch > optimize HRegion computeHDFSBlocksDistribution > -- > > Key: HBASE-16398 > URL: https://issues.apache.org/jira/browse/HBASE-16398 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: binlijin >Assignee: binlijin > Fix For: 2.0.0 > > Attachments: HBASE-16398.patch, HBASE-16398_v2.patch, > HBASE-16398_v3.patch, HBASE-16398_v4.patch, LocatedBlockStatusComparison.java > > > First i assume there is no reference and link in a region family's directory. > Without the patch to computeHDFSBlocksDistribution for a region family, there > is 1+2*N rpc call, N is hfile numbers, The first rpc call is to > DistributedFileSystem#listStatus to get hfiles, for every hfile there is two > rpc call DistributedFileSystem#getFileStatus(path) and then > DistributedFileSystem#getFileBlockLocations(status, start, length). > With the patch to computeHDFSBlocksDistribution for a region family, there is > 2 rpc call, they are DistributedFileSystem#getFileStatus(path) and > DistributedFileSystem#listLocatedStatus(final Path p, final PathFilter > filter). > So if there is at least one hfile, with the patch, the rpc call will less. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16398) optimize HRegion computeHDFSBlocksDistribution
[ https://issues.apache.org/jira/browse/HBASE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740815#comment-15740815 ] binlijin commented on HBASE-16398: -- The fail related to this patch, let me fix it..Thanks ted! > optimize HRegion computeHDFSBlocksDistribution > -- > > Key: HBASE-16398 > URL: https://issues.apache.org/jira/browse/HBASE-16398 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: binlijin >Assignee: binlijin > Fix For: 2.0.0 > > Attachments: HBASE-16398.patch, HBASE-16398_v2.patch, > HBASE-16398_v3.patch, LocatedBlockStatusComparison.java > > > First i assume there is no reference and link in a region family's directory. > Without the patch to computeHDFSBlocksDistribution for a region family, there > is 1+2*N rpc call, N is hfile numbers, The first rpc call is to > DistributedFileSystem#listStatus to get hfiles, for every hfile there is two > rpc call DistributedFileSystem#getFileStatus(path) and then > DistributedFileSystem#getFileBlockLocations(status, start, length). > With the patch to computeHDFSBlocksDistribution for a region family, there is > 2 rpc call, they are DistributedFileSystem#getFileStatus(path) and > DistributedFileSystem#listLocatedStatus(final Path p, final PathFilter > filter). > So if there is at least one hfile, with the patch, the rpc call will less. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17237) Override the correct compact method in HMobStore
[ https://issues.apache.org/jira/browse/HBASE-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-17237: - Attachment: HBASE-17237-V2.patch Rebase the code and upload a new patch V2. > Override the correct compact method in HMobStore > > > Key: HBASE-17237 > URL: https://issues.apache.org/jira/browse/HBASE-17237 > Project: HBase > Issue Type: Bug > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Attachments: HBASE-17237-V2.patch, HBASE-17237.patch > > > The method Store#compact(CompactionContext compaction, ThroughputController > throughputController) is deprecated now, and HMobStore overrides it. > HMobStore needs to override compact(CompactionContext compaction, > ThroughputController throughputController, User user) instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17276) Reduce log spam from WrongRegionException in large multi()'s
[
https://issues.apache.org/jira/browse/HBASE-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740768#comment-15740768
]
Hudson commented on HBASE-17276:
SUCCESS: Integrated in Jenkins build HBase-1.4 #562 (See
[https://builds.apache.org/job/HBase-1.4/562/])
HBASE-17276 Only log stacktraces for exceptions once for updates in a (elserj:
rev f13283bc05e5306a9c57e4b42fe801d9b2a4a2f1)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestObservedExceptionsInBatch.java
> Reduce log spam from WrongRegionException in large multi()'s
>
>
> Key: HBASE-17276
> URL: https://issues.apache.org/jira/browse/HBASE-17276
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 0.98.24
>
> Attachments: HBASE-17276.001.patch, HBASE-17276.002.patch
>
>
> The following spam drives me up a wall in the regionserver log:
> {noformat}
> 2016-12-05 05:53:05,085 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
> getEndKey()='Y\x99\x99\x99\x99\x99\x99\x94',
> row='\x0C\xD2\xA5\xA3\x99\xC7\xE0Q!\x15^\xA6\x90\x1E\xA3\xAD'
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:5211)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkAndPrepareMutation(HRegion.java:3879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2933)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2875)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:717)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:679)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2056)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32303)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> 2016-12-05 05:53:05,086 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
> getEndKey()='Y\x99\x99\x99\x99\x99\x99\x94',
> row='\x0E\xE7\xFA[\x8D\x93;\xF4\xC7F\xF9\x85\x84\x85\xF3\x0E'
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:5211)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkAndPrepareMutation(HRegion.java:3879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2933)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2875)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:717)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:679)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2056)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32303)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> 2016-12-05 05:53:05,087 WARN
>
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740766#comment-15740766 ] Guanghao Zhang commented on HBASE-17289: I check all unit tests about addPeer method and it is suitable to add ut in TestReplicationAdmin. Thanks. > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.1.patch, > HBASE-17289-branch-1.2.patch, HBASE-17289-branch-1.3.patch, > HBASE-17289-branch-1.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740758#comment-15740758 ] Guanghao Zhang commented on HBASE-17289: Yes, I can do it by adding new ut... But I thought it is ok to be public. Because this is a final String as constant. And in branch-1, it can be used on ReplicationZKNodeCleaner and it is public too. > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.1.patch, > HBASE-17289-branch-1.2.patch, HBASE-17289-branch-1.3.patch, > HBASE-17289-branch-1.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740748#comment-15740748 ] Ted Yu commented on HBASE-17289: Is it possible to add the test in the same package ? > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.1.patch, > HBASE-17289-branch-1.2.patch, HBASE-17289-branch-1.3.patch, > HBASE-17289-branch-1.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-17142) Implement multi get
[
https://issues.apache.org/jira/browse/HBASE-17142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740715#comment-15740715
]
Duo Zhang edited comment on HBASE-17142 at 12/12/16 1:53 AM:
-
{quote}
Looks clean to me. It is an alternative to AsyncProcess
{quote}
I hope it could be. There are several differences with AsyncProcess. First is
that the operationTimeout is the total time limit now, which is the same with
other single operations. Second is that the maxRetries is a limit for each get,
not for RS.
Thanks.
was (Author: apache9):
{quote}
Looks clean to me. It is an alternative to AsyncProcess
{quote}
I hope it could be. There are several differences with AsyncProcess. First is
that the operationTimeout is the total time limit now, which is the same with
other single operations. Second is that the maxRetries is a limit for each get
operation, not for RS.
Thanks.
> Implement multi get
> ---
>
> Key: HBASE-17142
> URL: https://issues.apache.org/jira/browse/HBASE-17142
> Project: HBase
> Issue Type: Sub-task
> Components: asyncclient, Client
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17142.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17142) Implement multi get
[
https://issues.apache.org/jira/browse/HBASE-17142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740715#comment-15740715
]
Duo Zhang commented on HBASE-17142:
---
{quote}
Looks clean to me. It is an alternative to AsyncProcess
{quote}
I hope it could be. There are several differences with AsyncProcess. First is
that the operationTimeout is the total time limit now, which is the same with
other single operations. Second is that the maxRetries is a limit for each get
operation, not for RS.
Thanks.
> Implement multi get
> ---
>
> Key: HBASE-17142
> URL: https://issues.apache.org/jira/browse/HBASE-17142
> Project: HBase
> Issue Type: Sub-task
> Components: asyncclient, Client
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17142.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740679#comment-15740679 ] Guanghao Zhang commented on HBASE-17289: Thanks. This variable can be used in test, too. It is a different package org.apache.hadoop.hbase.client.replication. > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.1.patch, > HBASE-17289-branch-1.2.patch, HBASE-17289-branch-1.3.patch, > HBASE-17289-branch-1.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17289: --- Attachment: HBASE-17289-branch-1.patch Attach patch for branch-1. > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.1.patch, > HBASE-17289-branch-1.2.patch, HBASE-17289-branch-1.3.patch, > HBASE-17289-branch-1.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17285) Misconfiguration of JVM GC options in HADOOP_CLIENT_OPTS may break `bin/hbase`
[
https://issues.apache.org/jira/browse/HBASE-17285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740383#comment-15740383
]
Allen Wittenauer commented on HBASE-17285:
--
Unfortunately, all of the _OPTS handling in most of the Hadoop ecosystem
scripts I've looked at do very bad things and are pretty much dependent upon
using space delimiters. This means no, folks can't properly quote it in
scripts and there are some limitations on these values. This obviously causes
other problems (the biggest one probably being the inability to use directory
paths with spaces) which is why shellcheck is throwing a fit.
The only real solution I've found is to convert them all to arrays. This can
be done in a somewhat backward compatible change, but it's massive amount of
work, even for the rewritten scripts. See HADOOP-13365 for what I've started
doing in Hadoop.
> Misconfiguration of JVM GC options in HADOOP_CLIENT_OPTS may break `bin/hbase`
> --
>
> Key: HBASE-17285
> URL: https://issues.apache.org/jira/browse/HBASE-17285
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17285.001.patch
>
>
> Had the great fun of digging through this one. Had a user reporting that
> hiveserver2 was no longer finding HBase jars on the classpath. This is
> supposed to happen via {{hbase mapredcp}}.
> It turned out that they had configured hive-env.sh to set
> {{HADOOP_CLIENT_OPTS="-XX:+PrintGCDetails"}} (among other things), which
> creates a big multi-line string instead of just a directory. Because of poor
> quoting in {{bin/hbase}}, this gives you a wonderfully intuitive error:
> {noformat}
> Error: Could not find or load main class Heap
> {noformat}
> That {{Heap}} is actually from the JVM GC details that it was told to print.
> While I don't expect this to be a common problem people run into, it's one
> that we can address with better quoting. e.g.
> {noformat}
> + exec
> /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/bin/java
> -Dproc_mapredcp '-XX:OnOutOfMemoryError=kill -9 %p' -XX:+UseConcMarkSweepGC
> -Dhbase.log.dir=/usr/local/lib/hbase//logs -Dhbase.log.file=hbase.log
> -Dhbase.home.dir=/usr/local/lib/hbase/ -Dhbase.id.str=
> -Dhbase.root.logger=INFO,console
> '-Djava.library.path='\''/usr/local/lib/hadoop//lib/native' Heap PSYoungGen
> total 76800K, used 7942K '[0x0007f550,' 0x0007faa8,
> '0x0008)' eden space 66048K, 12% used
> '[0x0007f550,0x0007f5cc19c0,0x0007f958)' from space
> 10752K, 0% used '[0x0007fa00,0x0007fa00,0x0007faa8)'
> to space 10752K, 0% used
> '[0x0007f958,0x0007f958,0x0007fa00)' ParOldGen total
> 174592K, used 0K '[0x0007e000,' 0x0007eaa8,
> '0x0007f550)' object space 174592K, 0% used
> '[0x0007e000,0x0007e000,0x0007eaa8)' PSPermGen total
> 21504K, used 2756K '[0x0007dae0,' 0x0007dc30,
> '0x0007e000)' object space 21504K, 12% used
> '[0x0007dae0,0x0007db0b11b8,0x0007dc30)'\'''
> -Dhbase.security.logger=INFO,NullAppender
> org.apache.hadoop.hbase.util.MapreduceDependencyClasspathTool
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14160) backport hbase-spark module to branch-1
[ https://issues.apache.org/jira/browse/HBASE-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740377#comment-15740377 ] Ted Yu commented on HBASE-14160: Sean: Are you still working on this ? > backport hbase-spark module to branch-1 > --- > > Key: HBASE-14160 > URL: https://issues.apache.org/jira/browse/HBASE-14160 > Project: HBase > Issue Type: New Feature > Components: spark >Affects Versions: 1.3.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 1.4.0 > > > Once the hbase-spark module gets polished a bit, we should backport to > branch-1 so we can publish it sooner. > needs (per previous discussion): > * examples refactored into different module > * user facing documentation > * defined public API -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17276) Reduce log spam from WrongRegionException in large multi()'s
[
https://issues.apache.org/jira/browse/HBASE-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740333#comment-15740333
]
Josh Elser commented on HBASE-17276:
Done.
> Reduce log spam from WrongRegionException in large multi()'s
>
>
> Key: HBASE-17276
> URL: https://issues.apache.org/jira/browse/HBASE-17276
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 0.98.24
>
> Attachments: HBASE-17276.001.patch, HBASE-17276.002.patch
>
>
> The following spam drives me up a wall in the regionserver log:
> {noformat}
> 2016-12-05 05:53:05,085 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
> getEndKey()='Y\x99\x99\x99\x99\x99\x99\x94',
> row='\x0C\xD2\xA5\xA3\x99\xC7\xE0Q!\x15^\xA6\x90\x1E\xA3\xAD'
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:5211)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkAndPrepareMutation(HRegion.java:3879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2933)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2875)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:717)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:679)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2056)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32303)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> 2016-12-05 05:53:05,086 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
> getEndKey()='Y\x99\x99\x99\x99\x99\x99\x94',
> row='\x0E\xE7\xFA[\x8D\x93;\xF4\xC7F\xF9\x85\x84\x85\xF3\x0E'
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:5211)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkAndPrepareMutation(HRegion.java:3879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2933)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2875)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:717)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:679)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2056)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32303)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> 2016-12-05 05:53:05,087 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
>
[jira] [Updated] (HBASE-17276) Reduce log spam from WrongRegionException in large multi()'s
[
https://issues.apache.org/jira/browse/HBASE-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-17276:
---
Fix Version/s: 0.98.24
1.4.0
> Reduce log spam from WrongRegionException in large multi()'s
>
>
> Key: HBASE-17276
> URL: https://issues.apache.org/jira/browse/HBASE-17276
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 0.98.24
>
> Attachments: HBASE-17276.001.patch, HBASE-17276.002.patch
>
>
> The following spam drives me up a wall in the regionserver log:
> {noformat}
> 2016-12-05 05:53:05,085 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
> getEndKey()='Y\x99\x99\x99\x99\x99\x99\x94',
> row='\x0C\xD2\xA5\xA3\x99\xC7\xE0Q!\x15^\xA6\x90\x1E\xA3\xAD'
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:5211)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkAndPrepareMutation(HRegion.java:3879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2933)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2875)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:717)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:679)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2056)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32303)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> 2016-12-05 05:53:05,086 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
> getEndKey()='Y\x99\x99\x99\x99\x99\x99\x94',
> row='\x0E\xE7\xFA[\x8D\x93;\xF4\xC7F\xF9\x85\x84\x85\xF3\x0E'
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:5211)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkAndPrepareMutation(HRegion.java:3879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2933)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2875)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:717)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:679)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2056)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32303)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> 2016-12-05 05:53:05,087 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
>
[jira] [Commented] (HBASE-17276) Reduce log spam from WrongRegionException in large multi()'s
[
https://issues.apache.org/jira/browse/HBASE-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740219#comment-15740219
]
Josh Elser commented on HBASE-17276:
[~apurtell], happy to! I'll take care of it [~stack] ;).
> Reduce log spam from WrongRegionException in large multi()'s
>
>
> Key: HBASE-17276
> URL: https://issues.apache.org/jira/browse/HBASE-17276
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17276.001.patch, HBASE-17276.002.patch
>
>
> The following spam drives me up a wall in the regionserver log:
> {noformat}
> 2016-12-05 05:53:05,085 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
> getEndKey()='Y\x99\x99\x99\x99\x99\x99\x94',
> row='\x0C\xD2\xA5\xA3\x99\xC7\xE0Q!\x15^\xA6\x90\x1E\xA3\xAD'
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:5211)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkAndPrepareMutation(HRegion.java:3879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2933)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2875)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:717)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:679)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2056)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32303)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> 2016-12-05 05:53:05,086 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
> startKey='L\xCC\xCC\xCC\xCC\xCC\xCC\xC8',
> getEndKey()='Y\x99\x99\x99\x99\x99\x99\x94',
> row='\x0E\xE7\xFA[\x8D\x93;\xF4\xC7F\xF9\x85\x84\x85\xF3\x0E'
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:5211)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkAndPrepareMutation(HRegion.java:3879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2933)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2875)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:717)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:679)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2056)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32303)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> 2016-12-05 05:53:05,087 WARN
> [RpcServer.FifoWFPBQ.default.handler=29,queue=2,port=16020]
> regionserver.HRegion: Batch mutation had a row that does not belong to this
> region
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
> of range for doMiniBatchMutation on HRegion
> IntegrationTestReplicationSinkRestart,L\xCC\xCC\xCC\xCC\xCC\xCC\xC8,1480916713541.caab3310166699287b54b72b35b29431.,
>
[jira] [Commented] (HBASE-17233) See if we should replace System.arrayCopy with Arrays.copyOfRange
[
https://issues.apache.org/jira/browse/HBASE-17233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740131#comment-15740131
]
ChiaPing Tsai commented on HBASE-17233:
---
The big difference may be caused by JIT optimization. We should use static and
final in the res1 and b1 so that JIT can leverage the memory for both methods.
For example, the data can be prefetched because the input is never null.
> See if we should replace System.arrayCopy with Arrays.copyOfRange
> -
>
> Key: HBASE-17233
> URL: https://issues.apache.org/jira/browse/HBASE-17233
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>
> Just saw this interesting comment in PB code. Since we deal with byte[]
> extensively (when we are onheap) we do lot of copies too.
> {code}
> * One of the noticeable costs of copying a byte[] into a new array using
>* {@code System.arraycopy} is nullification of a new buffer before the
> copy. It has been shown
>* the Hotspot VM is capable to intrisicfy {@code Arrays.copyOfRange}
> operation to avoid this
>* expensive nullification and provide substantial performance gain.
> Unfortunately this does not
>* hold on Android runtimes and could make the copy slightly slower due to
> additional code in
>* the {@code Arrays.copyOfRange}.
> {code}
> So since we are hotspot VM we could see if the places we use System.arrayCopy
> can be replaced with Arrays.copyOfRange.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16336) Removing peers seems to be leaving spare queues
[ https://issues.apache.org/jira/browse/HBASE-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740120#comment-15740120 ] Hudson commented on HBASE-16336: SUCCESS: Integrated in Jenkins build HBase-1.4 #561 (See [https://builds.apache.org/job/HBase-1.4/561/]) HBASE-16336 Removing peers seems to be leaving spare queues (Guanghao (tedyu: rev d87b05f040608e7ffa7908246d914d5ff1353943) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/ReplicationZKNodeCleanerChore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/ReplicationChecker.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationZKNodeCleaner.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/ReplicationZKNodeCleaner.java > Removing peers seems to be leaving spare queues > --- > > Key: HBASE-16336 > URL: https://issues.apache.org/jira/browse/HBASE-16336 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Guanghao Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: 16336.v4.diff, HBASE-15867-v1.patch, > HBASE-15867-v2.patch, HBASE-16336-branch-1-v1.patch, > HBASE-16336-branch-1.patch, HBASE-16636-v3.patch > > > I have been running IntegrationTestReplication repeatedly with the backported > Replication Table changes. Every other iteration of the test fails with, but > these queues should have been deleted when we removed the peers. I believe > this may be related to HBASE-16096, HBASE-16208, or HBASE-16081. > 16/08/02 08:36:07 ERROR util.AbstractHBaseTool: Error running command-line > tool > org.apache.hadoop.hbase.replication.ReplicationException: undeleted queue for > peerId: TestPeer, replicator: > hbase4124.ash2.facebook.com,16020,1470150251042, queueId: TestPeer > at > org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.checkQueuesDeleted(ReplicationPeersZKImpl.java:544) > at > org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.addPeer(ReplicationPeersZKImpl.java:127) > at > org.apache.hadoop.hbase.client.replication.ReplicationAdmin.addPeer(ReplicationAdmin.java:200) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication$VerifyReplicationLoop.setupTablesAndReplication(IntegrationTestReplication.java:239) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication$VerifyReplicationLoop.run(IntegrationTestReplication.java:325) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication.runTestFromCommandLine(IntegrationTestReplication.java:418) > at > org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:134) > at > org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication.main(IntegrationTestReplication.java:424) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17290) Potential loss of data for replication of bulk loaded hfiles
Ted Yu created HBASE-17290: -- Summary: Potential loss of data for replication of bulk loaded hfiles Key: HBASE-17290 URL: https://issues.apache.org/jira/browse/HBASE-17290 Project: HBase Issue Type: Bug Reporter: Ted Yu Currently the support for replication of bulk loaded hfiles relies on bulk load marker written in the WAL. The move of bulk loaded hfile(s) (into region directory) may succeed but the write of bulk load marker may fail. This means that although bulk loaded hfile is being served in source cluster, the replication wouldn't happen. Normally operator is supposed to retry the bulk load. But relying on human retry is not robust solution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[
https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15740010#comment-15740010
]
Ted Yu commented on HBASE-17289:
Looks good overall.
{code}
71public final static String RS_LOCK_ZNODE = "lock";
{code}
The variable can be package private.
> Avoid adding a replication peer named "lock"
>
>
> Key: HBASE-17289
> URL: https://issues.apache.org/jira/browse/HBASE-17289
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-17289-branch-1.1.patch,
> HBASE-17289-branch-1.2.patch, HBASE-17289-branch-1.3.patch
>
>
> When zk based replication queue is used and useMulti is false, the steps of
> transfer replication queues are first add a lock, then copy nodes, finally
> clean old queue and the lock. And the default lock znode's name is "lock". So
> we should avoid adding a peer named "lock".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[
https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739949#comment-15739949
]
Hadoop QA commented on HBASE-17289:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 20s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
19s {color} | {color:green} branch-1.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} branch-1.1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} branch-1.1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s {color} | {color:green} branch-1.1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s {color} | {color:green} branch-1.1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 58s
{color} | {color:red} hbase-client in branch-1.1 has 15 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 47s
{color} | {color:red} hbase-server in branch-1.1 has 81 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 18s
{color} | {color:red} hbase-client in branch-1.1 failed with JDK v1.8.0_111.
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 23s
{color} | {color:red} hbase-server in branch-1.1 failed with JDK v1.8.0_111.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} branch-1.1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 22s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
10s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 14s
{color} | {color:red} hbase-client in the patch failed with JDK v1.8.0_111.
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 24s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_111.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 35s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 83m 35s
{color} | {color:green} hbase-server in the patch passed. {color} |
|
[jira] [Updated] (HBASE-16336) Removing peers seems to be leaving spare queues
[ https://issues.apache.org/jira/browse/HBASE-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16336: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.4.0 2.0.0 Status: Resolved (was: Patch Available) Thanks for the patch, Guanghao > Removing peers seems to be leaving spare queues > --- > > Key: HBASE-16336 > URL: https://issues.apache.org/jira/browse/HBASE-16336 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Guanghao Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: 16336.v4.diff, HBASE-15867-v1.patch, > HBASE-15867-v2.patch, HBASE-16336-branch-1-v1.patch, > HBASE-16336-branch-1.patch, HBASE-16636-v3.patch > > > I have been running IntegrationTestReplication repeatedly with the backported > Replication Table changes. Every other iteration of the test fails with, but > these queues should have been deleted when we removed the peers. I believe > this may be related to HBASE-16096, HBASE-16208, or HBASE-16081. > 16/08/02 08:36:07 ERROR util.AbstractHBaseTool: Error running command-line > tool > org.apache.hadoop.hbase.replication.ReplicationException: undeleted queue for > peerId: TestPeer, replicator: > hbase4124.ash2.facebook.com,16020,1470150251042, queueId: TestPeer > at > org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.checkQueuesDeleted(ReplicationPeersZKImpl.java:544) > at > org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.addPeer(ReplicationPeersZKImpl.java:127) > at > org.apache.hadoop.hbase.client.replication.ReplicationAdmin.addPeer(ReplicationAdmin.java:200) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication$VerifyReplicationLoop.setupTablesAndReplication(IntegrationTestReplication.java:239) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication$VerifyReplicationLoop.run(IntegrationTestReplication.java:325) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication.runTestFromCommandLine(IntegrationTestReplication.java:418) > at > org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:134) > at > org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication.main(IntegrationTestReplication.java:424) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[
https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739874#comment-15739874
]
Hadoop QA commented on HBASE-17289:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 5m 8s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
11s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
43s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 8s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 32s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 72m 25s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 112m 19s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:d6626eb |
| JIRA Patch URL |
[jira] [Updated] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17289: --- Attachment: HBASE-17289-branch-1.1.patch > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.1.patch, > HBASE-17289-branch-1.2.patch, HBASE-17289-branch-1.3.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17289: --- Attachment: HBASE-17289-branch-1.2.patch > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.2.patch, > HBASE-17289-branch-1.3.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17289) Avoid adding a replication peer named "lock"
[
https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739663#comment-15739663
]
Hadoop QA commented on HBASE-17289:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 12s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
59s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} branch-1.3 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} branch-1.3 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} branch-1.3 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} branch-1.3 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 4s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 32s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 17s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 120m 24s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:66fbe99 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-17288) Add warn log for huge keyvalue and huge row
[
https://issues.apache.org/jira/browse/HBASE-17288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739562#comment-15739562
]
Hadoop QA commented on HBASE-17288:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 41s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m 4s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 131m 20s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12842701/HBASE-17288-v1.patch |
| JIRA Issue | HBASE-17288 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 01aff8d5ce50 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 67420fe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4871/testReport/ |
| modules | C: hbase-common hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4871/console |
| Powered by | Apache Yetus 0.3.0
[jira] [Commented] (HBASE-16336) Removing peers seems to be leaving spare queues
[
https://issues.apache.org/jira/browse/HBASE-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739508#comment-15739508
]
Hadoop QA commented on HBASE-16336:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 56s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 42s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 50s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 81m 47s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 118m 20s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:e01ee2f |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12842699/HBASE-16336-branch-1-v1.patch
|
| JIRA Issue | HBASE-16336 |
| Optional
[jira] [Updated] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17289: --- Assignee: Guanghao Zhang Status: Patch Available (was: Open) > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.2.4, 0.98.23, 1.1.7, 1.3.0, 1.4.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.3.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17289) Avoid adding a replication peer named "lock"
[ https://issues.apache.org/jira/browse/HBASE-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17289: --- Attachment: HBASE-17289-branch-1.3.patch > Avoid adding a replication peer named "lock" > > > Key: HBASE-17289 > URL: https://issues.apache.org/jira/browse/HBASE-17289 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0, 1.4.0, 1.1.7, 0.98.23, 1.2.4 >Reporter: Guanghao Zhang >Priority: Minor > Attachments: HBASE-17289-branch-1.3.patch > > > When zk based replication queue is used and useMulti is false, the steps of > transfer replication queues are first add a lock, then copy nodes, finally > clean old queue and the lock. And the default lock znode's name is "lock". So > we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17289) Avoid adding a replication peer named "lock"
Guanghao Zhang created HBASE-17289: -- Summary: Avoid adding a replication peer named "lock" Key: HBASE-17289 URL: https://issues.apache.org/jira/browse/HBASE-17289 Project: HBase Issue Type: Bug Components: Replication Affects Versions: 1.2.4, 0.98.23, 1.1.7, 1.3.0, 1.4.0 Reporter: Guanghao Zhang Priority: Minor When zk based replication queue is used and useMulti is false, the steps of transfer replication queues are first add a lock, then copy nodes, finally clean old queue and the lock. And the default lock znode's name is "lock". So we should avoid adding a peer named "lock". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17288) Add warn log for huge keyvalue and huge row
[
https://issues.apache.org/jira/browse/HBASE-17288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739387#comment-15739387
]
Guanghao Zhang commented on HBASE-17288:
Attach a v1 patch keep the behavior same with before.
> Add warn log for huge keyvalue and huge row
> ---
>
> Key: HBASE-17288
> URL: https://issues.apache.org/jira/browse/HBASE-17288
> Project: HBase
> Issue Type: Improvement
> Components: scan
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-17288-v1.patch, HBASE-17288.patch
>
>
> Some log examples from our production cluster.
> {code}
> 2016-12-10,17:08:11,478 WARN
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into
> result list, kv size:1253360,
> kv:10567114001-1-c/R:r1/1481360887152/Put/vlen=1253245/ts=923099, from
> table X
> 2016-12-10,17:08:16,724 WARN
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into
> result list, kv size:1048680,
> kv:0220459/I:i_0/1481360889551/Put/vlen=1048576/ts=13642, from table XX
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17288) Add warn log for huge keyvalue and huge row
[
https://issues.apache.org/jira/browse/HBASE-17288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-17288:
---
Attachment: HBASE-17288-v1.patch
> Add warn log for huge keyvalue and huge row
> ---
>
> Key: HBASE-17288
> URL: https://issues.apache.org/jira/browse/HBASE-17288
> Project: HBase
> Issue Type: Improvement
> Components: scan
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-17288-v1.patch, HBASE-17288.patch
>
>
> Some log examples from our production cluster.
> {code}
> 2016-12-10,17:08:11,478 WARN
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into
> result list, kv size:1253360,
> kv:10567114001-1-c/R:r1/1481360887152/Put/vlen=1253245/ts=923099, from
> table X
> 2016-12-10,17:08:16,724 WARN
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into
> result list, kv size:1048680,
> kv:0220459/I:i_0/1481360889551/Put/vlen=1048576/ts=13642, from table XX
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16336) Removing peers seems to be leaving spare queues
[ https://issues.apache.org/jira/browse/HBASE-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-16336: --- Attachment: HBASE-16336-branch-1-v1.patch Attach a v1 patch to fix the ut. > Removing peers seems to be leaving spare queues > --- > > Key: HBASE-16336 > URL: https://issues.apache.org/jira/browse/HBASE-16336 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Guanghao Zhang > Attachments: 16336.v4.diff, HBASE-15867-v1.patch, > HBASE-15867-v2.patch, HBASE-16336-branch-1-v1.patch, > HBASE-16336-branch-1.patch, HBASE-16636-v3.patch > > > I have been running IntegrationTestReplication repeatedly with the backported > Replication Table changes. Every other iteration of the test fails with, but > these queues should have been deleted when we removed the peers. I believe > this may be related to HBASE-16096, HBASE-16208, or HBASE-16081. > 16/08/02 08:36:07 ERROR util.AbstractHBaseTool: Error running command-line > tool > org.apache.hadoop.hbase.replication.ReplicationException: undeleted queue for > peerId: TestPeer, replicator: > hbase4124.ash2.facebook.com,16020,1470150251042, queueId: TestPeer > at > org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.checkQueuesDeleted(ReplicationPeersZKImpl.java:544) > at > org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.addPeer(ReplicationPeersZKImpl.java:127) > at > org.apache.hadoop.hbase.client.replication.ReplicationAdmin.addPeer(ReplicationAdmin.java:200) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication$VerifyReplicationLoop.setupTablesAndReplication(IntegrationTestReplication.java:239) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication$VerifyReplicationLoop.run(IntegrationTestReplication.java:325) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication.runTestFromCommandLine(IntegrationTestReplication.java:418) > at > org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:134) > at > org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.test.IntegrationTestReplication.main(IntegrationTestReplication.java:424) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
