[jira] [Commented] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338387#comment-15338387
]
Mikhail Antonov commented on HBASE-15600:
-
please open follow up jiras if backports are needed
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Labels: phoenix
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch, hbase-15600_v3.patch, hbase-15600_v4.patch,
> hbase-15600_v5.patch, hbase-15600_v6.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15600:
Resolution: Fixed
Fix Version/s: (was: 0.98.21)
Status: Resolved (was: Patch Available)
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Labels: phoenix
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch, hbase-15600_v3.patch, hbase-15600_v4.patch,
> hbase-15600_v5.patch, hbase-15600_v6.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338386#comment-15338386
]
Mikhail Antonov commented on HBASE-15600:
-
[~enis] [~rajeshbabu] [~busbey] I'm marking it as resolved as it seems to be
committed where it should be
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Labels: phoenix
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch, hbase-15600_v3.patch, hbase-15600_v4.patch,
> hbase-15600_v5.patch, hbase-15600_v6.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13981) Fix ImportTsv spelling and usage issues
[
https://issues.apache.org/jira/browse/HBASE-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338383#comment-15338383
]
Hadoop QA commented on HBASE-13981:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color}
| {color:red} HBASE-13981 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12745366/HBASE-13981.4.patch |
| JIRA Issue | HBASE-13981 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2296/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Fix ImportTsv spelling and usage issues
> ---
>
> Key: HBASE-13981
> URL: https://issues.apache.org/jira/browse/HBASE-13981
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 1.1.0.1
>Reporter: Lars George
>Assignee: Gabor Liptak
> Labels: beginner
> Fix For: 2.0.0, 1.3.1
>
> Attachments: HBASE-13981.1.patch, HBASE-13981.2.patch,
> HBASE-13981.3.patch, HBASE-13981.4.patch
>
>
> The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
> In code:
> {noformat}
> public final static String ATTRIBUTE_SEPERATOR_CONF_KEY =
> "attributes.seperator";
> {noformat}
> It is "separator".
> In usage text:
> {noformat}
> "input data. Another special columnHBASE_TS_KEY designates that this column
> should be"
> {noformat}
> Space missing.
> {noformat}
> "Record with invalid timestamps (blank, non-numeric) will be treated as bad
> record."
> {noformat}
> "Records ... as bad records" - plural missing twice.
> {noformat}
> "HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
> Should be specified as key=>value where -1 is used
> as the seperator. Note that more than one OperationAttributes can be
> specified."
> {noformat}
> - Remove line wraps and indentation.
> - Fix "separator".
> - Fix wrong separator being output, it is not "-1" (wrong constant use in
> code)
> - General wording/style could be better (eg. last sentence now uses
> OperationAttributes without a space).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13860) Remove units from ServerMetricsTmpl.jamon since values are formatted human readable
[ https://issues.apache.org/jira/browse/HBASE-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13860: Fix Version/s: (was: 1.3.0) 1.3.1 > Remove units from ServerMetricsTmpl.jamon since values are formatted human > readable > --- > > Key: HBASE-13860 > URL: https://issues.apache.org/jira/browse/HBASE-13860 > Project: HBase > Issue Type: Bug > Components: regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Gabor Liptak >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.3.1 > > Attachments: HBASE-13860.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13844) Move static helper methods from KeyValue into CellUtils
[
https://issues.apache.org/jira/browse/HBASE-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338382#comment-15338382
]
Hadoop QA commented on HBASE-13844:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-13844 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12743178/HBASE-13844.3.patch |
| JIRA Issue | HBASE-13844 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2295/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Move static helper methods from KeyValue into CellUtils
> ---
>
> Key: HBASE-13844
> URL: https://issues.apache.org/jira/browse/HBASE-13844
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Lars George
>Assignee: Gabor Liptak
> Labels: beginner
> Fix For: 2.0.0, 1.3.1
>
> Attachments: HBASE-13844.1.patch, HBASE-13844.2.patch,
> HBASE-13844.3.patch
>
>
> Add KeyValue.parseColumn() to CellUtils (also any other public static helper)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13860) Remove units from ServerMetricsTmpl.jamon since values are formatted human readable
[
https://issues.apache.org/jira/browse/HBASE-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338384#comment-15338384
]
Hadoop QA commented on HBASE-13860:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color}
| {color:red} HBASE-13860 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12738475/HBASE-13860.1.patch |
| JIRA Issue | HBASE-13860 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2298/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Remove units from ServerMetricsTmpl.jamon since values are formatted human
> readable
> ---
>
> Key: HBASE-13860
> URL: https://issues.apache.org/jira/browse/HBASE-13860
> Project: HBase
> Issue Type: Bug
> Components: regionserver, UI
>Affects Versions: 1.1.0
>Reporter: Lars George
>Assignee: Gabor Liptak
>Priority: Trivial
> Labels: beginner
> Fix For: 2.0.0, 1.3.1
>
> Attachments: HBASE-13860.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13860) Remove units from ServerMetricsTmpl.jamon since values are formatted human readable
[ https://issues.apache.org/jira/browse/HBASE-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338381#comment-15338381 ] Mikhail Antonov commented on HBASE-13860: - can we get it in, any update? > Remove units from ServerMetricsTmpl.jamon since values are formatted human > readable > --- > > Key: HBASE-13860 > URL: https://issues.apache.org/jira/browse/HBASE-13860 > Project: HBase > Issue Type: Bug > Components: regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Gabor Liptak >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13860.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13147) Load actual META table descriptor, don't use statically defined one.
[ https://issues.apache.org/jira/browse/HBASE-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13147: Fix Version/s: (was: 1.3.0) 1.4.0 > Load actual META table descriptor, don't use statically defined one. > > > Key: HBASE-13147 > URL: https://issues.apache.org/jira/browse/HBASE-13147 > Project: HBase > Issue Type: Bug > Components: master, regionserver >Affects Versions: 2.0.0 >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-13147-branch-1.patch, > HBASE-13147-branch-1.v2.patch, HBASE-13147.patch, HBASE-13147.v2.patch, > HBASE-13147.v3.patch, HBASE-13147.v4.patch, HBASE-13147.v4.patch, > HBASE-13147.v5.patch, HBASE-13147.v6.patch, HBASE-13147.v7.patch > > > In HBASE-13087 stumbled on the fact, that region servers don't see actual > meta descriptor, they use their own, statically compiled. > Need to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6562) Fake KVs are sometimes passed to filters
[ https://issues.apache.org/jira/browse/HBASE-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-6562: --- Fix Version/s: (was: 1.3.0) 1.4.0 > Fake KVs are sometimes passed to filters > > > Key: HBASE-6562 > URL: https://issues.apache.org/jira/browse/HBASE-6562 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: 6562-0.94-v1.txt, 6562-0.96-v1.txt, 6562-v2.txt, > 6562-v3.txt, 6562-v4.txt, 6562-v5.txt, 6562.txt, minimalTest.java > > > In internal tests at Salesforce we found that fake row keys sometimes are > passed to filters (Filter.filterRowKey(...) specifically). > The KVs are eventually filtered by the StoreScanner/ScanQueryMatcher, but the > row key is passed to filterRowKey in RegionScannImpl *before* that happens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13844) Move static helper methods from KeyValue into CellUtils
[ https://issues.apache.org/jira/browse/HBASE-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13844: Fix Version/s: (was: 1.3.0) 1.3.1 > Move static helper methods from KeyValue into CellUtils > --- > > Key: HBASE-13844 > URL: https://issues.apache.org/jira/browse/HBASE-13844 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Gabor Liptak > Labels: beginner > Fix For: 2.0.0, 1.3.1 > > Attachments: HBASE-13844.1.patch, HBASE-13844.2.patch, > HBASE-13844.3.patch > > > Add KeyValue.parseColumn() to CellUtils (also any other public static helper) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13841) Master UI: Region server table compactions percentage less useful over time
[ https://issues.apache.org/jira/browse/HBASE-13841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13841: Fix Version/s: (was: 1.3.0) 1.3.1 > Master UI: Region server table compactions percentage less useful over time > --- > > Key: HBASE-13841 > URL: https://issues.apache.org/jira/browse/HBASE-13841 > Project: HBase > Issue Type: Bug > Components: master, UI >Affects Versions: 1.1.0 >Reporter: Lars George > Labels: beginner > Fix For: 2.0.0, 1.3.1 > > > The problem is that the percentage is computed as "current KVs to > compact"/"total KVs to compact", and the template accumulating the counts > over its lifetime. Initially it is OK (for the very first run), but after a > short amount of time the percentage idles around 99-100% always. > Not sure how to fix, since we do not have the previews value (unless we save > it somewhere in the page, which is messy during reloads etc.). We may have to > simply drop the percentage? Or better would be not to accumulate the counts, > but keep the current/total number as well, which makes the percentage > computation always correct. I vote for the latter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13840) Server UIs should rename column labels from KVs to Cell
[ https://issues.apache.org/jira/browse/HBASE-13840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13840: Fix Version/s: (was: 1.3.0) 1.3.1 > Server UIs should rename column labels from KVs to Cell > --- > > Key: HBASE-13840 > URL: https://issues.apache.org/jira/browse/HBASE-13840 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George > Fix For: 2.0.0, 1.3.1 > > > Currently the master UI still refers to KVs in some of the tables. We should > do a sweep and rename to Cell. > Also do for RS templates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6581) Build with hadoop.profile=3.0
[ https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-6581: --- Fix Version/s: (was: 1.3.0) 1.4.0 > Build with hadoop.profile=3.0 > - > > Key: HBASE-6581 > URL: https://issues.apache.org/jira/browse/HBASE-6581 > Project: HBase > Issue Type: Bug >Reporter: Eric Charles >Assignee: Eric Charles > Fix For: 2.0.0, 1.4.0 > > Attachments: 6581-009.txt, 6581-010.txt, 6581.txt, 6581v2.txt, > HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581-20130821.patch, > HBASE-6581-3.patch, HBASE-6581-4.patch, HBASE-6581-5.patch, > HBASE-6581-6.patch, HBASE-6581-7.patch, HBASE-6581-8-pom.patch, > HBASE-6581.diff, HBASE-6581.diff > > > Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to > change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT > instead of 3.0.0-SNAPSHOT in hbase-common). > I can provide a patch that would move most of hadoop dependencies in their > respective profiles and will define the correct hadoop deps in the 3.0 > profile. > Please tell me if that's ok to go this way. > Thx, Eric > [1] > $ mvn clean install -Dhadoop.profile=3.0 > [INFO] Scanning for projects... > [ERROR] The build could not read 3 projects -> [Help 1] > [ERROR] > [ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT > (/d/hbase.svn/hbase-server/pom.xml) has 3 errors > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21 > [ERROR] > [ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT > (/d/hbase.svn/hbase-common/pom.xml) has 3 errors > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21 > [ERROR] > [ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT > (/d/hbase.svn/hbase-it/pom.xml) has 3 errors > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21 > [ERROR] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[ https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338380#comment-15338380 ] Mikhail Antonov commented on HBASE-6581: any update? > Build with hadoop.profile=3.0 > - > > Key: HBASE-6581 > URL: https://issues.apache.org/jira/browse/HBASE-6581 > Project: HBase > Issue Type: Bug >Reporter: Eric Charles >Assignee: Eric Charles > Fix For: 2.0.0, 1.4.0 > > Attachments: 6581-009.txt, 6581-010.txt, 6581.txt, 6581v2.txt, > HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581-20130821.patch, > HBASE-6581-3.patch, HBASE-6581-4.patch, HBASE-6581-5.patch, > HBASE-6581-6.patch, HBASE-6581-7.patch, HBASE-6581-8-pom.patch, > HBASE-6581.diff, HBASE-6581.diff > > > Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to > change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT > instead of 3.0.0-SNAPSHOT in hbase-common). > I can provide a patch that would move most of hadoop dependencies in their > respective profiles and will define the correct hadoop deps in the 3.0 > profile. > Please tell me if that's ok to go this way. > Thx, Eric > [1] > $ mvn clean install -Dhadoop.profile=3.0 > [INFO] Scanning for projects... > [ERROR] The build could not read 3 projects -> [Help 1] > [ERROR] > [ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT > (/d/hbase.svn/hbase-server/pom.xml) has 3 errors > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21 > [ERROR] > [ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT > (/d/hbase.svn/hbase-common/pom.xml) has 3 errors > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21 > [ERROR] > [ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT > (/d/hbase.svn/hbase-it/pom.xml) has 3 errors > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21 > [ERROR] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13981) Fix ImportTsv spelling and usage issues
[
https://issues.apache.org/jira/browse/HBASE-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-13981:
Fix Version/s: (was: 1.3.0)
1.3.1
> Fix ImportTsv spelling and usage issues
> ---
>
> Key: HBASE-13981
> URL: https://issues.apache.org/jira/browse/HBASE-13981
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 1.1.0.1
>Reporter: Lars George
>Assignee: Gabor Liptak
> Labels: beginner
> Fix For: 2.0.0, 1.3.1
>
> Attachments: HBASE-13981.1.patch, HBASE-13981.2.patch,
> HBASE-13981.3.patch, HBASE-13981.4.patch
>
>
> The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
> In code:
> {noformat}
> public final static String ATTRIBUTE_SEPERATOR_CONF_KEY =
> "attributes.seperator";
> {noformat}
> It is "separator".
> In usage text:
> {noformat}
> "input data. Another special columnHBASE_TS_KEY designates that this column
> should be"
> {noformat}
> Space missing.
> {noformat}
> "Record with invalid timestamps (blank, non-numeric) will be treated as bad
> record."
> {noformat}
> "Records ... as bad records" - plural missing twice.
> {noformat}
> "HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
> Should be specified as key=>value where -1 is used
> as the seperator. Note that more than one OperationAttributes can be
> specified."
> {noformat}
> - Remove line wraps and indentation.
> - Fix "separator".
> - Fix wrong separator being output, it is not "-1" (wrong constant use in
> code)
> - General wording/style could be better (eg. last sentence now uses
> OperationAttributes without a space).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13605) RegionStates should not keep its list of dead servers
[
https://issues.apache.org/jira/browse/HBASE-13605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-13605:
Fix Version/s: (was: 1.3.0)
1.4.0
> RegionStates should not keep its list of dead servers
> -
>
> Key: HBASE-13605
> URL: https://issues.apache.org/jira/browse/HBASE-13605
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: hbase-13605_v1.patch, hbase-13605_v3-branch-1.1.patch,
> hbase-13605_v4-branch-1.1.patch, hbase-13605_v4-master.patch
>
>
> As mentioned in
> https://issues.apache.org/jira/browse/HBASE-9514?focusedCommentId=13769761=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13769761
> and HBASE-12844 we should have only 1 source of cluster membership.
> The list of dead server and RegionStates doing it's own liveliness check
> (ServerManager.isServerReachable()) has caused an assignment problem again in
> a test cluster where the region states "thinks" that the server is dead and
> SSH will handle the region assignment. However the RS is not dead at all,
> living happily, and never gets zk expiry or YouAreDeadException or anything.
> This leaves the list of regions unassigned in OFFLINE state.
> master assigning the region:
> {code}
> 15-04-20 09:02:25,780 DEBUG [AM.ZK.Worker-pool3-t330] master.RegionStates:
> Onlined 77dddcd50c22e56bfff133c0e1f9165b on
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268 {ENCODED =>
> 77dddcd50c
> {code}
> Master then disabled the table, and unassigned the region:
> {code}
> 2015-04-20 09:02:27,158 WARN [ProcedureExecutorThread-1]
> zookeeper.ZKTableStateManager: Moving table loadtest_d1 state from DISABLING
> to DISABLING
> Starting unassign of
> loadtest_d1,,1429520544378.77dddcd50c22e56bfff133c0e1f9165b. (offlining),
> current state: {77dddcd50c22e56bfff133c0e1f9165b state=OPEN,
> ts=1429520545780,
> server=os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268}
> bleProcedure$BulkDisabler-0] master.AssignmentManager: Sent CLOSE to
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268 for region
> loadtest_d1,,1429520544378.77dddcd50c22e56bfff133c0e1f9165b.
> 2015-04-20 09:02:27,414 INFO [AM.ZK.Worker-pool3-t316] master.RegionStates:
> Offlined 77dddcd50c22e56bfff133c0e1f9165b from
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268
> {code}
> On table re-enable, AM does not assign the region:
> {code}
> 2015-04-20 09:02:30,415 INFO [ProcedureExecutorThread-3]
> balancer.BaseLoadBalancer: Reassigned 25 regions. 25 retained the pre-restart
> assignment.·
> 2015-04-20 09:02:30,415 INFO [ProcedureExecutorThread-3]
> procedure.EnableTableProcedure: Bulk assigning 25 region(s) across 5
> server(s), retainAssignment=true
> l,16000,1429515659726-GeneralBulkAssigner-4] master.RegionStates: Couldn't
> reach online server
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268
> l,16000,1429515659726-GeneralBulkAssigner-4] master.AssignmentManager:
> Updating the state to OFFLINE to allow to be reassigned by SSH
> nmentManager: Skip assigning
> loadtest_d1,,1429520544378.77dddcd50c22e56bfff133c0e1f9165b., it is on a dead
> but not processed yet server:
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14223) Meta WALs are not cleared if meta region was closed and RS aborts
[
https://issues.apache.org/jira/browse/HBASE-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-14223:
Fix Version/s: (was: 1.3.0)
1.3.1
> Meta WALs are not cleared if meta region was closed and RS aborts
> -
>
> Key: HBASE-14223
> URL: https://issues.apache.org/jira/browse/HBASE-14223
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.0.4, 1.1.6, 1.3.1, 1.2.3
>
> Attachments: HBASE-14223logs, hbase-14223_v0.patch,
> hbase-14223_v1-branch-1.patch, hbase-14223_v2-branch-1.patch,
> hbase-14223_v3-branch-1.patch, hbase-14223_v3-branch-1.patch,
> hbase-14223_v3-master.patch
>
>
> When an RS opens meta, and later closes it, the WAL(FSHlog) is not closed.
> The last WAL file just sits there in the RS WAL directory. If RS stops
> gracefully, the WAL file for meta is deleted. Otherwise if RS aborts, WAL for
> meta is not cleaned. It is also not split (which is correct) since master
> determines that the RS no longer hosts meta at the time of RS abort.
> From a cluster after running ITBLL with CM, I see a lot of {{-splitting}}
> directories left uncleaned:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs
> Found 31 items
> drwxr-xr-x - hbase hadoop 0 2015-06-05 01:14
> /apps/hbase/data/WALs/hregion-58203265
> drwxr-xr-x - hbase hadoop 0 2015-06-05 07:54
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433489308745-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 09:28
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433494382959-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 10:01
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433498252205-splitting
> ...
> {code}
> The directories contain WALs from meta:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting
> Found 2 items
> -rw-r--r-- 3 hbase hadoop 201608 2015-06-05 03:15
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> -rw-r--r-- 3 hbase hadoop 44420 2015-06-05 04:36
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> The RS hosted the meta region for some time:
> {code}
> 2015-06-05 03:14:28,692 INFO [PostOpenDeployTasks:1588230740]
> zookeeper.MetaTableLocator: Setting hbase:meta region location in ZooKeeper
> as os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285
> ...
> 2015-06-05 03:15:17,302 INFO
> [RS_CLOSE_META-os-enis-dal-test-jun-4-5:16020-0] regionserver.HRegion: Closed
> hbase:meta,,1.1588230740
> {code}
> In between, a WAL is created:
> {code}
> 2015-06-05 03:15:11,707 INFO
> [RS_OPEN_META-os-enis-dal-test-jun-4-5:16020-0-MetaLogRoller] wal.FSHLog:
> Rolled WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> with entries=385, filesize=196.88 KB; new WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> When CM killed the region server later master did not see these WAL files:
> {code}
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:46,075
> INFO [MASTER_SERVER_OPERATIONS-os-enis-dal-test-jun-4-3:16000-0]
> master.SplitLogManager: started splitting 2 logs in
> [hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting]
> for [os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285]
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:47,300
> INFO [main-EventThread] wal.WALSplitter: Archived processed log
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> to
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/oldWALs/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:50,497
> INFO [main-EventThread]
[jira] [Updated] (HBASE-13160) SplitLogWorker does not pick up the task immediately
[
https://issues.apache.org/jira/browse/HBASE-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-13160:
Fix Version/s: (was: 1.3.0)
1.3.1
> SplitLogWorker does not pick up the task immediately
>
>
> Key: HBASE-13160
> URL: https://issues.apache.org/jira/browse/HBASE-13160
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.1
>
> Attachments: hbase-13160_v1.patch
>
>
> We were reading some code with Jeffrey, and we realized that the
> SplitLogWorker's internal task loop is weird. It does {{ls}} every second and
> sleeps, but have another mechanism to learn about new tasks, but does not
> make affective use of the zk notification.
> I have a simple patch which might improve this area.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15302: Fix Version/s: (was: 1.3.0) 1.3.1 > Reenable the other tests disabled by HBASE-14678 > > > Key: HBASE-15302 > URL: https://issues.apache.org/jira/browse/HBASE-15302 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.1 > > Attachments: HBASE-15302-branch-1-v1.patch, HBASE-15302-v1.txt, > HBASE-15302-v1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13160) SplitLogWorker does not pick up the task immediately
[
https://issues.apache.org/jira/browse/HBASE-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338369#comment-15338369
]
Mikhail Antonov commented on HBASE-13160:
-
Moving to 1.3.1. [~enis] ?
> SplitLogWorker does not pick up the task immediately
>
>
> Key: HBASE-13160
> URL: https://issues.apache.org/jira/browse/HBASE-13160
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.3.1
>
> Attachments: hbase-13160_v1.patch
>
>
> We were reading some code with Jeffrey, and we realized that the
> SplitLogWorker's internal task loop is weird. It does {{ls}} every second and
> sleeps, but have another mechanism to learn about new tasks, but does not
> make affective use of the zk notification.
> I have a simple patch which might improve this area.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14938) Limit the number of znodes for ZK in bulk loaded hfile replication
[ https://issues.apache.org/jira/browse/HBASE-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14938: Fix Version/s: (was: 1.3.0) 1.4.0 > Limit the number of znodes for ZK in bulk loaded hfile replication > -- > > Key: HBASE-14938 > URL: https://issues.apache.org/jira/browse/HBASE-14938 > Project: HBase > Issue Type: Improvement >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-14938(1).patch, HBASE-14938-v1.patch, > HBASE-14938-v2(1).patch, HBASE-14938-v2.patch, HBASE-14938.patch > > > In ZK the maximum allowable size of the data array is 1 MB. Until we have > fixed HBASE-10295 we need to handle this. > Approach to this problem will be discussed in the comments section. > Note: We have done internal testing with more than 3k nodes in ZK yet to be > replicated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15387) Make HalfStoreFileReader configurable in LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-15387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15387: Fix Version/s: (was: 1.3.0) 1.3.1 > Make HalfStoreFileReader configurable in LoadIncrementalHFiles > -- > > Key: HBASE-15387 > URL: https://issues.apache.org/jira/browse/HBASE-15387 > Project: HBase > Issue Type: Improvement >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla > Fix For: 2.0.0, 1.1.6, 1.3.1, 1.2.3 > > > Currently we are initializing HalfStoreFileReader to split the HFile but we > might have different implementation for splitting. So we can make it > configurable. It's needed for local indexing in Phoenix(PHOENIX-2736). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15075) Allow region split request to carry identification information
[ https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15075: Fix Version/s: (was: 1.3.0) 1.4.0 > Allow region split request to carry identification information > -- > > Key: HBASE-15075 > URL: https://issues.apache.org/jira/browse/HBASE-15075 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.4.0 > > Attachments: 15075-v0.txt, 15075-v1.txt, 15075-v2.txt, > HBASE-15075.v10.patch, HBASE-15075.v2.patch, HBASE-15075.v3.patch, > HBASE-15075.v4.patch, HBASE-15075.v5.patch, HBASE-15075.v6.patch, > HBASE-15075.v7.patch, HBASE-15075.v8.patch, HBASE-15075.v9.patch > > > During the process of improving region normalization feature, I found that if > region split request triggered by the execution of SplitNormalizationPlan > fails, there is no way of knowing whether the failed split originated from > region normalization. > The association of particular split request with outcome of split would give > RegionNormalizer information so that it can make better normalization > decisions in the subsequent invocations. > One approach is to embed metadata, such as a UUID, in SplitRequest which gets > passed through RegionStateTransitionContext when > RegionServerServices#reportRegionStateTransition() is called. > This way, RegionStateListener can be notified with the metadata (id of the > requester). > See discussion on dev mailing list > http://search-hadoop.com/m/YGbbCXdkivihp2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15383) Load distribute across secondary read replicas for meta
[ https://issues.apache.org/jira/browse/HBASE-15383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15383: Fix Version/s: (was: 1.3.0) 1.4.0 > Load distribute across secondary read replicas for meta > --- > > Key: HBASE-15383 > URL: https://issues.apache.org/jira/browse/HBASE-15383 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.4.0 > > > Right now, we always hit the primary replica for meta and fallback to the > secondary replicas in case of a timeout. This can hamper performance in > scenarios where meta becomes a hot region e.g. cluster ramp up..clients > dropping connections etc. > It's good to have a load distribution approach on meta's secondary replicas > with fallback to primary if we read stale data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14937) Make rpc call timeout for replication adaptive
[
https://issues.apache.org/jira/browse/HBASE-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338365#comment-15338365
]
Hadoop QA commented on HBASE-14937:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color}
| {color:red} HBASE-14937 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12778212/HBASE-14937.patch |
| JIRA Issue | HBASE-14937 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2294/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Make rpc call timeout for replication adaptive
> --
>
> Key: HBASE-14937
> URL: https://issues.apache.org/jira/browse/HBASE-14937
> Project: HBase
> Issue Type: Improvement
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
> Labels: replication
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-14937.patch
>
>
> When peer cluster replication is disabled and lot of writes are happening in
> active cluster and later on peer cluster replication is enabled then there
> are chances that replication requests to peer cluster may time out.
> This is possible after HBASE-13153 and it can also happen with many and many
> WAL data replication still pending to replicate.
> Approach to this problem will be discussed in the comments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14937) Make rpc call timeout for replication adaptive
[ https://issues.apache.org/jira/browse/HBASE-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14937: Fix Version/s: (was: 1.3.0) 1.4.0 > Make rpc call timeout for replication adaptive > -- > > Key: HBASE-14937 > URL: https://issues.apache.org/jira/browse/HBASE-14937 > Project: HBase > Issue Type: Improvement >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Labels: replication > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-14937.patch > > > When peer cluster replication is disabled and lot of writes are happening in > active cluster and later on peer cluster replication is enabled then there > are chances that replication requests to peer cluster may time out. > This is possible after HBASE-13153 and it can also happen with many and many > WAL data replication still pending to replicate. > Approach to this problem will be discussed in the comments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14937) Make rpc call timeout for replication adaptive
[ https://issues.apache.org/jira/browse/HBASE-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338363#comment-15338363 ] Mikhail Antonov commented on HBASE-14937: - Any progress here? kicked out to 1.4.0 > Make rpc call timeout for replication adaptive > -- > > Key: HBASE-14937 > URL: https://issues.apache.org/jira/browse/HBASE-14937 > Project: HBase > Issue Type: Improvement >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Labels: replication > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-14937.patch > > > When peer cluster replication is disabled and lot of writes are happening in > active cluster and later on peer cluster replication is enabled then there > are chances that replication requests to peer cluster may time out. > This is possible after HBASE-13153 and it can also happen with many and many > WAL data replication still pending to replicate. > Approach to this problem will be discussed in the comments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13017) Backport HBASE-12035 Keep table state in Meta to branch-1
[ https://issues.apache.org/jira/browse/HBASE-13017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338362#comment-15338362 ] Mikhail Antonov commented on HBASE-13017: - Should we target it for 2.0 only? > Backport HBASE-12035 Keep table state in Meta to branch-1 > - > > Key: HBASE-13017 > URL: https://issues.apache.org/jira/browse/HBASE-13017 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.1.0 >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Labels: backport > Fix For: 1.4.0 > > Attachments: HBASE-13017-branch-1.patch, > HBASE-13017-branch-1.v1.patch, HBASE-13017-branch-1.v1.patch, > HBASE-13017-branch-1.v2.patch, HBASE-13017-branch-1.v3.patch, > HBASE-13017-branch-1.v4.patch, HBASE-13017-branch-1.v5.patch, > HBASE-13017-branch-1.v6.patch > > > Lets backport that feature to branch-1.0 adapting HBASE-12035 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13017) Backport HBASE-12035 Keep table state in Meta to branch-1
[ https://issues.apache.org/jira/browse/HBASE-13017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13017: Fix Version/s: (was: 1.3.0) 1.4.0 > Backport HBASE-12035 Keep table state in Meta to branch-1 > - > > Key: HBASE-13017 > URL: https://issues.apache.org/jira/browse/HBASE-13017 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.1.0 >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Labels: backport > Fix For: 1.4.0 > > Attachments: HBASE-13017-branch-1.patch, > HBASE-13017-branch-1.v1.patch, HBASE-13017-branch-1.v1.patch, > HBASE-13017-branch-1.v2.patch, HBASE-13017-branch-1.v3.patch, > HBASE-13017-branch-1.v4.patch, HBASE-13017-branch-1.v5.patch, > HBASE-13017-branch-1.v6.patch > > > Lets backport that feature to branch-1.0 adapting HBASE-12035 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts
[ https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338360#comment-15338360 ] Anoop Sam John commented on HBASE-16055: +1 for fix in all applicable versions. > PutSortReducer loses any Visibility/acl attribute set on the Puts > -- > > Key: HBASE-16055 > URL: https://issues.apache.org/jira/browse/HBASE-16055 > Project: HBase > Issue Type: Bug > Components: security >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0 > > > Based on a user discussion, I think as the user pointed out rightly, when a > PutSortReducer is used any visibility attribute or external attribute set on > the Put will be lost as we create KVs out of the cells in the puts whereas > the ACL and visibility are all set as Attributes. > In TextSortReducer we tend to read the information we tend to read the > information from the parsed line but here in PutSortReducer we don't do it. I > think this problem should be in all the existing versions where we support > Tags. Correct me if am wrong here. > [~anoop.hbase], [~andrew.purt...@gmail.com]? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14025) Update CHANGES.txt for 1.2
[ https://issues.apache.org/jira/browse/HBASE-14025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338352#comment-15338352 ] Mikhail Antonov commented on HBASE-14025: - Yep, it does - thanks [~busbey]. So far I'm working with merge-base between rel/1.2.1 (should be probably 1.2.0?) and branch-1.3. > Update CHANGES.txt for 1.2 > -- > > Key: HBASE-14025 > URL: https://issues.apache.org/jira/browse/HBASE-14025 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 1.2.0 >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 1.2.0 > > > Since it's more effort than I expected, making a ticket to track actually > updating CHANGES.txt so that new RMs have an idea what to expect. > Maybe will make doc changes if there's enough here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7105) RS throws NPE on forcing compaction from HBase shell on a single bulk imported file.
[
https://issues.apache.org/jira/browse/HBASE-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-7105:
---
Fix Version/s: (was: 1.3.0)
1.4.0
> RS throws NPE on forcing compaction from HBase shell on a single bulk
> imported file.
>
>
> Key: HBASE-7105
> URL: https://issues.apache.org/jira/browse/HBASE-7105
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Karthik Ranganathan
> Fix For: 2.0.0, 1.4.0
>
> Attachments:
> 0001-HBASE-7105-RS-throws-NPE-on-forcing-compaction-from-.patch,
> 0001-HBASE-7105-RS-throws-NPE-on-forcing-compaction-from-.patch
>
>
> In StoreFile, we have:
> private AtomicBoolean majorCompaction = null;
> In StoreFile.open(), we do:
> b = metadataMap.get(MAJOR_COMPACTION_KEY);
> if (b != null) {
> // init majorCompaction variable
> }
> Because the file was bulk imported, this is never initialized. Any subsequent
> call to isMajorCompaction() NPE's.
> Fix is to initialize it to false.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15127) NPE in RpcServer$Call.wrapWithSasl()
[
https://issues.apache.org/jira/browse/HBASE-15127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15127:
Fix Version/s: (was: 1.3.0)
1.3.1
> NPE in RpcServer$Call.wrapWithSasl()
>
>
> Key: HBASE-15127
> URL: https://issues.apache.org/jira/browse/HBASE-15127
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 2.0.0, 1.3.1
>
>
> I saw this in a log file, not sure whether it is important to fix or not:
> {code}
> 2016-01-09 01:30:58,905 WARN [FifoRpcScheduler.handler1-thread-25]
> ipc.RpcServer: FifoRpcScheduler.handler1-thread-25: caught:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:412)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:128)
> at
> org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:74)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> This is 0.98 based code base corresponding to:
> {code}
> synchronized (connection.saslServer) {
> token = connection.saslServer.wrap(responseBytes, 0,
> responseBytes.length);
> }
> {code}
> I believe the saslserver was set to null earlier by {{disposeSasl()}} because
> Connection is closing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15984) Given failure to parse a given WAL that was closed cleanly, replay the WAL.
[ https://issues.apache.org/jira/browse/HBASE-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-15984: Fix Version/s: (was: 1.3.0) 1.3.1 > Given failure to parse a given WAL that was closed cleanly, replay the WAL. > --- > > Key: HBASE-15984 > URL: https://issues.apache.org/jira/browse/HBASE-15984 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 1.0.4, 1.4.0, 1.1.6, 1.3.1, 0.98.21, 1.2.3 > > Attachments: HBASE-15984.1.patch > > > subtask for a general work around for "underlying reader failed / is in a bad > state" just for the case where a WAL 1) was closed cleanly and 2) we can tell > that our current offset ought not be the end of parseable entries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15983) Replication improperly discards data from end-of-wal in some cases.
[ https://issues.apache.org/jira/browse/HBASE-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338282#comment-15338282 ] Mikhail Antonov commented on HBASE-15983: - looking forward to get it in 1.3.1 > Replication improperly discards data from end-of-wal in some cases. > --- > > Key: HBASE-15983 > URL: https://issues.apache.org/jira/browse/HBASE-15983 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.98.0, 1.0.0, 1.1.0, 1.2.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0, 1.0.4, 1.4.0, 1.1.6, 1.3.1, 0.98.21, 1.2.3 > > > In some particular deployments, the Replication code believes it has > reached EOF for a WAL prior to successfully parsing all bytes known to > exist in a cleanly closed file. > The underlying issue is that several different underlying problems with a WAL > reader are all treated as end-of-file by the code in ReplicationSource that > decides if a given WAL is completed or needs to be retried. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14386) Reset MutableHistogram's min/max/sum after snapshot
[ https://issues.apache.org/jira/browse/HBASE-14386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14386: Fix Version/s: (was: 1.3.0) 1.3.1 > Reset MutableHistogram's min/max/sum after snapshot > --- > > Key: HBASE-14386 > URL: https://issues.apache.org/jira/browse/HBASE-14386 > Project: HBase > Issue Type: Bug >Reporter: binlijin >Assignee: Oliver > Fix For: 2.0.0, 1.3.1 > > Attachments: HBASE-14386.patch > > > Current MutableHistogram do not reset min/max/sum after snapshot, so we > affect by historical data. For example when i monitor the QueueCallTime_mean, > i see one host's QueueCallTime_mean metric is high, but when i trace the > host's regionserver log i see the QueueCallTime_mean has been lower, but the > metric is still high. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14386) Reset MutableHistogram's min/max/sum after snapshot
[ https://issues.apache.org/jira/browse/HBASE-14386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338281#comment-15338281 ] Mikhail Antonov commented on HBASE-14386: - is this one still relevant? > Reset MutableHistogram's min/max/sum after snapshot > --- > > Key: HBASE-14386 > URL: https://issues.apache.org/jira/browse/HBASE-14386 > Project: HBase > Issue Type: Bug >Reporter: binlijin >Assignee: Oliver > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-14386.patch > > > Current MutableHistogram do not reset min/max/sum after snapshot, so we > affect by historical data. For example when i monitor the QueueCallTime_mean, > i see one host's QueueCallTime_mean metric is high, but when i trace the > host's regionserver log i see the QueueCallTime_mean has been lower, but the > metric is still high. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13708) PE - Add option to specify the range
[ https://issues.apache.org/jira/browse/HBASE-13708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13708: Fix Version/s: (was: 1.3.0) 1.3.1 > PE - Add option to specify the range > > > Key: HBASE-13708 > URL: https://issues.apache.org/jira/browse/HBASE-13708 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.1 >Reporter: Jean-Marc Spaggiari >Assignee: Matt Warhaftig >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.3.1 > > Attachments: HBASE-13708-master_v1.patch, PE_mapred_output.txt, > PE_nomapred_output.txt, hbase-13708-v2.patch, hbase-13708-v2.patch > > > When specifying --rows=YYY for randomReads in PE, it will read YYY rows but > only betweem 0 and YYY. > We might want to read YYY rows randomly between 0 and ZZZ. > YYY should only be the number of rows we want to ready. Not the high range. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[
https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338279#comment-15338279
]
Matt Warhaftig commented on HBASE-13838:
{quote}
"Any progress here?"
{quote}
[~stack], Can you please take a look at the submitted patch. The associated
Hadoop QA results are at [this
comment|https://issues.apache.org/jira/browse/HBASE-13838?focusedCommentId=14629232=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14629232].
> Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
>
>
> Key: HBASE-13838
> URL: https://issues.apache.org/jira/browse/HBASE-13838
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 1.1.0
>Reporter: Lars George
>Assignee: Matt Warhaftig
> Labels: beginner
> Fix For: 2.0.0, 1.3.1
>
> Attachments: hbase-13838-v1.patch, hbase-13838_post.tiff,
> hbase-13838_post_put_command.txt, hbase-13838_pre.tiff
>
>
> There are a few issues with the shared TaskStatusTmpl:
> - "Client operations" tab is always empty
> For Master this is expected, but for RegionServers there is never anything
> listed either. Fix for RS status page (probably caused by params not
> containing Operation subclass anymore, but some PB generated classes?)
> - Hide “Client Operations” tab for master UI
> Since operations are RS only. Or we fix this and make other calls show here.
> - The "alert-error" for aborted tasks is not set in CSS at all
> When a task was aborted it should be amber or red, but the assigned style is
> not in any of the linked stylesheets (abort-error). Add.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14391) Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
[
https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-14391:
Fix Version/s: (was: 1.3.0)
1.3.1
> Empty regionserver WAL will never be deleted although the coresponding
> regionserver has been stale
> --
>
> Key: HBASE-14391
> URL: https://issues.apache.org/jira/browse/HBASE-14391
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.0.2
>Reporter: Qianxi Zhang
>Assignee: Qianxi Zhang
> Fix For: 2.0.0, 1.1.6, 1.3.1, 1.2.3
>
> Attachments: HBASE-14391-master-v3.patch,
> HBASE_14391_master_v4.patch, HBASE_14391_trunk_v1.patch,
> HBASE_14391_trunk_v2.patch, WALs-leftover-dir.txt
>
>
> When I restarted the hbase cluster in which there was few data, I found there
> are two directories for one host with different timestamp which indicates
> that the old regionserver wal directory is not deleted.
> FHLog#989
> {code}
> @Override
> public void close() throws IOException {
> shutdown();
> final FileStatus[] files = getFiles();
> if (null != files && 0 != files.length) {
> for (FileStatus file : files) {
> Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath());
> // Tell our listeners that a log is going to be archived.
> if (!this.listeners.isEmpty()) {
> for (WALActionsListener i : this.listeners) {
> i.preLogArchive(file.getPath(), p);
> }
> }
> if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) {
> throw new IOException("Unable to rename " + file.getPath() + " to "
> + p);
> }
> // Tell our listeners that a log was archived.
> if (!this.listeners.isEmpty()) {
> for (WALActionsListener i : this.listeners) {
> i.postLogArchive(file.getPath(), p);
> }
> }
> }
> LOG.debug("Moved " + files.length + " WAL file(s) to " +
> FSUtils.getPath(this.fullPathArchiveDir));
> }
> LOG.info("Closed WAL: " + toString());
> }
> {code}
> When regionserver is stopped, the hlog will be archived, so wal/regionserver
> is empty in hdfs.
> MasterFileSystem#252
> {code}
> if (curLogFiles == null || curLogFiles.length == 0) {
> // Empty log folder. No recovery needed
> continue;
> }
> {code}
> The regionserver directory will be not splitted, it makes sense. But it will
> be not deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16048) Tag InternalScanner with LimitedPrivate(HBaseInterfaceAudience.COPROC)
[ https://issues.apache.org/jira/browse/HBASE-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338051#comment-15338051 ] Hudson commented on HBASE-16048: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1229 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1229/]) HBASE-16048 Tag InternalScanner with (apurtell: rev 42de857632557ab810d58aa7191a34afd90a8d9b) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/InternalScanner.java > Tag InternalScanner with LimitedPrivate(HBaseInterfaceAudience.COPROC) > --- > > Key: HBASE-16048 > URL: https://issues.apache.org/jira/browse/HBASE-16048 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21 > > Attachments: 16048.v1.txt > > > Some methods (preCompact, preCompactScannerOpen, preFlush, > preFlushScannerOpen, etc) of BaseRegionObserver take InternalScanner as input > argument as well as the return type. > BaseRegionObserver is tagged with > @InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC) but > InternalScanner is tagged with @InterfaceAudience.Private. > This JIRA is to discuss tagging InternalScanner with > @InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15990) The priority value of subsequent coprocessors in the Coprocessor.Priority.SYSTEM list are not incremented by one
[
https://issues.apache.org/jira/browse/HBASE-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338052#comment-15338052
]
Hudson commented on HBASE-15990:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1229 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1229/])
HBASE-15990 The priority value of subsequent coprocessors in the (apurtell: rev
a35162afff4b99dfbbb85e7b7354031dfb2919a7)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorHost.java
> The priority value of subsequent coprocessors in the
> Coprocessor.Priority.SYSTEM list are not incremented by one
>
>
> Key: HBASE-15990
> URL: https://issues.apache.org/jira/browse/HBASE-15990
> Project: HBase
> Issue Type: Bug
>Affects Versions: master
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-15990.patch
>
>
> {code:title=CoprocessorHost.java|borderStyle=solid}
> // The Coprocessor.PRIORITY_SYSTEM should be replaced by priority
> this.coprocessors.add(loadInstance(implClass,
> Coprocessor.PRIORITY_SYSTEM, conf));
> LOG.info("System coprocessor " + className + " was loaded " +
> "successfully with priority (" + priority++ + ").");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[ https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338050#comment-15338050 ] Mikhail Antonov commented on HBASE-13838: - Any progress here? > Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.) > > > Key: HBASE-13838 > URL: https://issues.apache.org/jira/browse/HBASE-13838 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Matt Warhaftig > Labels: beginner > Fix For: 2.0.0, 1.3.1 > > Attachments: hbase-13838-v1.patch, hbase-13838_post.tiff, > hbase-13838_post_put_command.txt, hbase-13838_pre.tiff > > > There are a few issues with the shared TaskStatusTmpl: > - "Client operations" tab is always empty > For Master this is expected, but for RegionServers there is never anything > listed either. Fix for RS status page (probably caused by params not > containing Operation subclass anymore, but some PB generated classes?) > - Hide “Client Operations” tab for master UI > Since operations are RS only. Or we fix this and make other calls show here. > - The "alert-error" for aborted tasks is not set in CSS at all > When a task was aborted it should be amber or red, but the assigned style is > not in any of the linked stylesheets (abort-error). Add. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[ https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-13838: Fix Version/s: (was: 1.3.0) 1.3.1 > Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.) > > > Key: HBASE-13838 > URL: https://issues.apache.org/jira/browse/HBASE-13838 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Matt Warhaftig > Labels: beginner > Fix For: 2.0.0, 1.3.1 > > Attachments: hbase-13838-v1.patch, hbase-13838_post.tiff, > hbase-13838_post_put_command.txt, hbase-13838_pre.tiff > > > There are a few issues with the shared TaskStatusTmpl: > - "Client operations" tab is always empty > For Master this is expected, but for RegionServers there is never anything > listed either. Fix for RS status page (probably caused by params not > containing Operation subclass anymore, but some PB generated classes?) > - Hide “Client Operations” tab for master UI > Since operations are RS only. Or we fix this and make other calls show here. > - The "alert-error" for aborted tasks is not set in CSS at all > When a task was aborted it should be amber or red, but the assigned style is > not in any of the linked stylesheets (abort-error). Add. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14008) REST - Throw an appropriate error during schema POST
[ https://issues.apache.org/jira/browse/HBASE-14008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14008: Fix Version/s: (was: 1.3.0) 1.4.0 > REST - Throw an appropriate error during schema POST > > > Key: HBASE-14008 > URL: https://issues.apache.org/jira/browse/HBASE-14008 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 0.98.13, 1.1.1 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Minor > Labels: REST > Fix For: 2.0.0, 1.4.0 > > Attachments: 14008.patch, HBASE-14008.patch > > > When an update is done on the schema through REST and an error occurs, the > actual reason is not thrown back to the client. Right now we get a > "javax.ws.rs.WebApplicationException" instead of the actual error message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15990) The priority value of subsequent coprocessors in the Coprocessor.Priority.SYSTEM list are not incremented by one
[
https://issues.apache.org/jira/browse/HBASE-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338022#comment-15338022
]
Hudson commented on HBASE-15990:
FAILURE: Integrated in HBase-0.98-matrix #357 (See
[https://builds.apache.org/job/HBase-0.98-matrix/357/])
HBASE-15990 The priority value of subsequent coprocessors in the (apurtell: rev
a35162afff4b99dfbbb85e7b7354031dfb2919a7)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorHost.java
> The priority value of subsequent coprocessors in the
> Coprocessor.Priority.SYSTEM list are not incremented by one
>
>
> Key: HBASE-15990
> URL: https://issues.apache.org/jira/browse/HBASE-15990
> Project: HBase
> Issue Type: Bug
>Affects Versions: master
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-15990.patch
>
>
> {code:title=CoprocessorHost.java|borderStyle=solid}
> // The Coprocessor.PRIORITY_SYSTEM should be replaced by priority
> this.coprocessors.add(loadInstance(implClass,
> Coprocessor.PRIORITY_SYSTEM, conf));
> LOG.info("System coprocessor " + className + " was loaded " +
> "successfully with priority (" + priority++ + ").");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16048) Tag InternalScanner with LimitedPrivate(HBaseInterfaceAudience.COPROC)
[ https://issues.apache.org/jira/browse/HBASE-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338021#comment-15338021 ] Hudson commented on HBASE-16048: FAILURE: Integrated in HBase-0.98-matrix #357 (See [https://builds.apache.org/job/HBase-0.98-matrix/357/]) HBASE-16048 Tag InternalScanner with (apurtell: rev 42de857632557ab810d58aa7191a34afd90a8d9b) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/InternalScanner.java > Tag InternalScanner with LimitedPrivate(HBaseInterfaceAudience.COPROC) > --- > > Key: HBASE-16048 > URL: https://issues.apache.org/jira/browse/HBASE-16048 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21 > > Attachments: 16048.v1.txt > > > Some methods (preCompact, preCompactScannerOpen, preFlush, > preFlushScannerOpen, etc) of BaseRegionObserver take InternalScanner as input > argument as well as the return type. > BaseRegionObserver is tagged with > @InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC) but > InternalScanner is tagged with @InterfaceAudience.Private. > This JIRA is to discuss tagging InternalScanner with > @InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16009) Restoring an incremental backup should not require -overwrite
[
https://issues.apache.org/jira/browse/HBASE-16009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16009:
---
Description:
When I tried to restore an incremental backup,
hbase restore hdfs://hbase-test-rc-rerun-6:8020/user/hbase backup_1465575766499
t1 t2
I got:
{code}
2016-06-10 19:53:11,317 ERROR [main] impl.RestoreClientImpl: ERROR: restore
failed with error: Existing table found in target while no "-overwrite" option
found
java.io.IOException: Existing table found in target while no "-overwrite"
option found
at
org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.checkTargetTables(RestoreClientImpl.java:186)
at
org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.restore(RestoreClientImpl.java:108)
at
org.apache.hadoop.hbase.backup.RestoreDriver.parseAndRun(RestoreDriver.java:169)
at
org.apache.hadoop.hbase.backup.RestoreDriver.doWork(RestoreDriver.java:198)
{code}
The above check should only be performed for restoring full backup.
was:
When I tried to restore an incremental backup,
hbase restore hdfs://hbase-test-rc-rerun-6:8020/user/hbase backup_1465575766499
t1 t2
I got:
{code}
2016-06-10 19:53:11,317 ERROR [main] impl.RestoreClientImpl: ERROR: restore
failed with error: Existing table found in target while no "-overwrite" option
found
java.io.IOException: Existing table found in target while no "-overwrite"
option found
at
org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.checkTargetTables(RestoreClientImpl.java:186)
at
org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.restore(RestoreClientImpl.java:108)
at
org.apache.hadoop.hbase.backup.RestoreDriver.parseAndRun(RestoreDriver.java:169)
at
org.apache.hadoop.hbase.backup.RestoreDriver.doWork(RestoreDriver.java:198)
{code}
The above check should only be performed for restoring full backup.
> Restoring an incremental backup should not require -overwrite
> -
>
> Key: HBASE-16009
> URL: https://issues.apache.org/jira/browse/HBASE-16009
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Vladimir Rodionov
> Labels: backup
> Attachments: 16009.v1.txt, HBASE-16009-v2.patch
>
>
> When I tried to restore an incremental backup,
> hbase restore hdfs://hbase-test-rc-rerun-6:8020/user/hbase
> backup_1465575766499 t1 t2
> I got:
> {code}
> 2016-06-10 19:53:11,317 ERROR [main] impl.RestoreClientImpl: ERROR: restore
> failed with error: Existing table found in target while no "-overwrite"
> option found
> java.io.IOException: Existing table found in target while no "-overwrite"
> option found
> at
> org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.checkTargetTables(RestoreClientImpl.java:186)
> at
> org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.restore(RestoreClientImpl.java:108)
> at
> org.apache.hadoop.hbase.backup.RestoreDriver.parseAndRun(RestoreDriver.java:169)
> at
> org.apache.hadoop.hbase.backup.RestoreDriver.doWork(RestoreDriver.java:198)
> {code}
> The above check should only be performed for restoring full backup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16062) Improper error handling in WAL Reader/Writer creation
[ https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337880#comment-15337880 ] Ted Yu commented on HBASE-16062: Please attach patch for branch-1 since the classes are different. > Improper error handling in WAL Reader/Writer creation > - > > Key: HBASE-16062 > URL: https://issues.apache.org/jira/browse/HBASE-16062 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-16062-v1.patch, HBASE-16062-v2.patch > > > If creation of WAL reader/ writer fails for some reason RS may leak hanging > socket in CLOSE_WAIT state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16062) Improper error handling in WAL Reader/Writer creation
[ https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337866#comment-15337866 ] Ted Yu commented on HBASE-16062: I ran TestVisibilityLabelsWithDefaultVisLabelService with patch v2 which passed. TestRegionServerMetrics is flaky. Failure not related to patch. > Improper error handling in WAL Reader/Writer creation > - > > Key: HBASE-16062 > URL: https://issues.apache.org/jira/browse/HBASE-16062 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-16062-v1.patch, HBASE-16062-v2.patch > > > If creation of WAL reader/ writer fails for some reason RS may leak hanging > socket in CLOSE_WAIT state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14915) Hanging test : org.apache.hadoop.hbase.mapreduce.TestImportExport
[ https://issues.apache.org/jira/browse/HBASE-14915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337795#comment-15337795 ] Heng Chen commented on HBASE-14915: --- Thanks [~mantonov] for your work. :) > Hanging test : org.apache.hadoop.hbase.mapreduce.TestImportExport > - > > Key: HBASE-14915 > URL: https://issues.apache.org/jira/browse/HBASE-14915 > Project: HBase > Issue Type: Sub-task > Components: hangingTests >Reporter: stack >Assignee: Heng Chen > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14915-branch-1.2.patch, > HBASE-14915_v1-branch-1.2.patch, HBASE-14915_v2-branch-1.2.patch, > HBASE-14915_v2-branch-1.2.patch, HBASE-14915_v3-branch-1.2.patch > > > This test hangs a bunch: > Here is latest: > https://builds.apache.org/job/HBase-1.2/418/jdk=latest1.7,label=Hadoop/consoleText -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16032) Possible memory leak in StoreScanner
[
https://issues.apache.org/jira/browse/HBASE-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337786#comment-15337786
]
Hadoop QA commented on HBASE-16032:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 55s
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 6s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 73m 26s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 114m 38s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811577/HBASE-16032_v4.patch |
| JIRA Issue | HBASE-16032 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6717f0e |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
[jira] [Updated] (HBASE-16032) Possible memory leak in StoreScanner
[
https://issues.apache.org/jira/browse/HBASE-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-16032:
--
Attachment: HBASE-16032_v4.patch
Update patch resolving the issue Enis pointed out, rollback to use the
try-catch way in StoreScanner constructor to avoid missing the StoreFile update
in resetKVHeap when flush/bulkload happens
> Possible memory leak in StoreScanner
>
>
> Key: HBASE-16032
> URL: https://issues.apache.org/jira/browse/HBASE-16032
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.1, 1.1.5, 0.98.20
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.1.6, 1.3.1, 0.98.21, 1.2.3
>
> Attachments: HBASE-16032.patch, HBASE-16032_v2.patch,
> HBASE-16032_v3.patch, HBASE-16032_v4.patch
>
>
> We observed frequent fullGC of RS in our production environment, and after
> analyzing the heapdump, we found large memory occupancy by
> HStore#changedReaderObservers, the map is surprisingly containing 7500w
> objects...
> After some debugging, I located some possible memory leak in StoreScanner
> constructor:
> {code}
> public StoreScanner(Store store, ScanInfo scanInfo, Scan scan, final
> NavigableSet columns,
> long readPt)
> throws IOException {
> this(store, scan, scanInfo, columns, readPt, scan.getCacheBlocks());
> if (columns != null && scan.isRaw()) {
> throw new DoNotRetryIOException("Cannot specify any column for a raw
> scan");
> }
> matcher = new ScanQueryMatcher(scan, scanInfo, columns,
> ScanType.USER_SCAN, Long.MAX_VALUE, HConstants.LATEST_TIMESTAMP,
> oldestUnexpiredTS, now, store.getCoprocessorHost());
> this.store.addChangedReaderObserver(this);
> // Pass columns to try to filter out unnecessary StoreFiles.
> List scanners = getScannersNoCompaction();
> ...
> seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery
> && lazySeekEnabledGlobally, parallelSeekEnabled);
> ...
> resetKVHeap(scanners, store.getComparator());
> }
> {code}
> If there's any Exception thrown after
> {{this.store.addChangedReaderObserver(this)}}, the returned scanner might be
> null and there's no chance to remove the scanner from changedReaderObservers,
> like in {{HRegion#get}}
> {code}
> RegionScanner scanner = null;
> try {
> scanner = getScanner(scan);
> scanner.next(results);
> } finally {
> if (scanner != null)
> scanner.close();
> }
> {code}
> What's more, all exception thrown in the {{HRegion#getScanner}} path will
> cause scanner==null then memory leak, so we also need to handle this part.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16032) Possible memory leak in StoreScanner
[
https://issues.apache.org/jira/browse/HBASE-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337686#comment-15337686
]
Yu Li commented on HBASE-16032:
---
Yes, this is really something I neglected, thanks for pointing this out
[~enis]. Will update the patch soon, and will wait for your +1 before commit
> Possible memory leak in StoreScanner
>
>
> Key: HBASE-16032
> URL: https://issues.apache.org/jira/browse/HBASE-16032
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.1, 1.1.5, 0.98.20
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.1.6, 1.3.1, 0.98.21, 1.2.3
>
> Attachments: HBASE-16032.patch, HBASE-16032_v2.patch,
> HBASE-16032_v3.patch
>
>
> We observed frequent fullGC of RS in our production environment, and after
> analyzing the heapdump, we found large memory occupancy by
> HStore#changedReaderObservers, the map is surprisingly containing 7500w
> objects...
> After some debugging, I located some possible memory leak in StoreScanner
> constructor:
> {code}
> public StoreScanner(Store store, ScanInfo scanInfo, Scan scan, final
> NavigableSet columns,
> long readPt)
> throws IOException {
> this(store, scan, scanInfo, columns, readPt, scan.getCacheBlocks());
> if (columns != null && scan.isRaw()) {
> throw new DoNotRetryIOException("Cannot specify any column for a raw
> scan");
> }
> matcher = new ScanQueryMatcher(scan, scanInfo, columns,
> ScanType.USER_SCAN, Long.MAX_VALUE, HConstants.LATEST_TIMESTAMP,
> oldestUnexpiredTS, now, store.getCoprocessorHost());
> this.store.addChangedReaderObserver(this);
> // Pass columns to try to filter out unnecessary StoreFiles.
> List scanners = getScannersNoCompaction();
> ...
> seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery
> && lazySeekEnabledGlobally, parallelSeekEnabled);
> ...
> resetKVHeap(scanners, store.getComparator());
> }
> {code}
> If there's any Exception thrown after
> {{this.store.addChangedReaderObserver(this)}}, the returned scanner might be
> null and there's no chance to remove the scanner from changedReaderObservers,
> like in {{HRegion#get}}
> {code}
> RegionScanner scanner = null;
> try {
> scanner = getScanner(scan);
> scanner.next(results);
> } finally {
> if (scanner != null)
> scanner.close();
> }
> {code}
> What's more, all exception thrown in the {{HRegion#getScanner}} path will
> cause scanner==null then memory leak, so we also need to handle this part.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15945) Patch for Key Value, Bytes and Cell
[ https://issues.apache.org/jira/browse/HBASE-15945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudeep Sunthankar updated HBASE-15945: -- Attachment: HBASE-15945-HBASE-14850.v2.patch As per the feedback , this patch consists of the Cell Interface for the underlying KeyValue structure, which can be changed based on the requirement. At the moment KeyValue structure is designed as a contiguous array of bytes. A test is also included in the patch. I encountered errors while addition of test in BUCK. Please do let me know if it is fine to add a Makefile which can be used to execute the tests so that the tests can be run > Patch for Key Value, Bytes and Cell > --- > > Key: HBASE-15945 > URL: https://issues.apache.org/jira/browse/HBASE-15945 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar > Attachments: HBASE-15945-HBASE-14850.v2.patch, > HBASE-15945.HBASE-14850.v1.patch > > > This patch contains an implementation of Key Value, Bytes and Cell modeled on > the lines of Java implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16058) TestHRegion fails on 1.4 builds
[ https://issues.apache.org/jira/browse/HBASE-16058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337623#comment-15337623 ] Hudson commented on HBASE-16058: FAILURE: Integrated in HBase-Trunk_matrix #1068 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1068/]) HBASE-16058 TestHRegion fails on 1.4 builds (enis: rev 6717f0e3883577513fef8c6cb8a2d0f4b4f41433) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java > TestHRegion fails on 1.4 builds > --- > > Key: HBASE-16058 > URL: https://issues.apache.org/jira/browse/HBASE-16058 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.4.0 >Reporter: Mikhail Antonov >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.4.0 > > Attachments: hbase-16058-v1.branch-1.patch > > > Results : > Failed tests: > org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushSizeAccounting(org.apache.hadoop.hbase.regionserver.TestHRegion) > Run 1: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368> > Run 2: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368> > Run 3: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368> > org.apache.hadoop.hbase.regionserver.TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate(org.apache.hadoop.hbase.regionserver.TestHRegion) > Run 1: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 > memstoreSize should be incremented expected:<448> but was:<432> > Run 2: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 > memstoreSize should be incremented expected:<448> but was:<432> > Run 3: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 > memstoreSize should be incremented expected:<448> but was:<432> > Tests run: 2567, Failures: 2, Errors: 0, Skipped: 56 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16064) delete backup command shows HDFS permission error when deleting the intended backup
[
https://issues.apache.org/jira/browse/HBASE-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337602#comment-15337602
]
Chris Nauroth commented on HBASE-16064:
---
[~tedyu], thank you for the patch. This looks right to me. Deleting
/user/hbase requires write access on /user. In typical deployments, /user will
be owned by the HDFS super-user, and other users won't have write access to it,
so the delete will fail on permission checks. Even if permissions are relaxed
so that it does succeed, I'm pretty sure the intent was not for this tool to
delete /user/hbase. :-)
Question: Is there anything more needed in HBase tests to verify the change?
> delete backup command shows HDFS permission error when deleting the intended
> backup
> ---
>
> Key: HBASE-16064
> URL: https://issues.apache.org/jira/browse/HBASE-16064
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Ted Yu
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: 16064.v1.txt
>
>
> HBase delete backup command shows error, after successfully deleting the
> intended backup
> {code}
> hbase@cluster-name:~$ hbase backup delete backup_1465950334243
> 2016-06-15 00:36:18,883 INFO [main] util.BackupClientUtil: No data has been
> found in
> hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
> 2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up
> backup data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase
> failed due to Permission denied: user=hbase, access=WRITE,
> inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> .
> {code}
> Backup has been successfully deleted but the backup root dir under
> /user/hbase dir still persists
> {code}
> hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
> Found 6 items
> drwx-- - hbase hbase 0 2016-06-15 00:26 /user/hbase/.staging
> drwxr-xr-x - hbase hbase 0 2016-06-15 00:36
> /user/hbase/backup_1465950334243
> drwxr-xr-x - hbase hbase 0 2016-06-15 00:26
> /user/hbase/hbase-staging
> {code}
> /user/hbase/backup_1465950334243 is now empty though
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16062) Improper error handling in WAL Reader/Writer creation
[
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337598#comment-15337598
]
Hadoop QA commented on HBASE-16062:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 13s
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
43m 39s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 29s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 178m 16s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestRegionServerMetrics |
| |
hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService
|
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811508/HBASE-16062-v2.patch |
| JIRA Issue | HBASE-16062 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
