Re: [VOTE] HBase Thirdparty 2.2.0 RC1
Sure ,I will repost question and logs with new thread . Thanks On Mon, 1 Apr 2019 at 11:59 PM, Stack wrote: > Shekar: > > You put your note on a VOTE thread. Please redo your query in its own > thread to hbase dev. > > Can you paste some log when you redo your message on the dev list? > > Thanks, > S > > On Mon, Apr 1, 2019 at 12:33 AM Shekhar Dhotre < > shekhar.dho...@clairvoyantsoft.com> wrote: > > > Hi All , > > I have a cluster which is setup to replicate to GCP . In the log its > > showing cluster ID null so found this document which shows how to setup > > cluster ID .but that is for a third party product called FusionInsight > > Manager. > > > > > > > http://support-it.huawei.com/docs/en-us/fusioninsight-all/maintenance-guide/en-us_topic_0087315479.html > > > > > > Where can I setup the below in Cloudera ? > > > > HBase Startup Failure Due to cluster.id Not Configured > > > > Region severs starts but as soon as I try to start master ,it brings down > > all RS and master also doesnt come up . > > > > Thanks for your help in advance . > > SD > > Hadoop Admin . > > > -- Thank you ! * Shekhar Dhotre.* senior Hadoop Administrator shekhar.dho...@clairvoyantsoft.com clairvoyantsoft.com/insight
[jira] [Created] (HBASE-22145) windows hbase-env causes hbase cli/etc to ignore HBASE_OPTS
Sergey Shelukhin created HBASE-22145: Summary: windows hbase-env causes hbase cli/etc to ignore HBASE_OPTS Key: HBASE-22145 URL: https://issues.apache.org/jira/browse/HBASE-22145 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22144) MultiRowRangeFilter does not work with reversed scans
Josh Elser created HBASE-22144: -- Summary: MultiRowRangeFilter does not work with reversed scans Key: HBASE-22144 URL: https://issues.apache.org/jira/browse/HBASE-22144 Project: HBase Issue Type: Bug Reporter: Josh Elser Assignee: Josh Elser It appears that MultiRowRangeFilter was never written to function with reverse scans. There is too much logic that operates with the assumption that we are always moving "forward" through increasing ranges. It needs to be rewritten to "traverse" forward or backward, given how the context of the scan being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [DISCUSS] release hbase-connectors
You the man Balazs. No harm in putting up a 1.0.0 I'd say. (There's a bit of a script on HBASE-22136 used building an hbase-thirdparty that might be of use. The README in hbase-thirdparty has some notes on how it was RC'd that might be of help doing hbase-connectors. Shout if I can help). Thanks, S On Mon, Apr 1, 2019 at 4:43 AM Balazs Meszaros wrote: > Hi All, > > Since 2.0 connectors have been moved out from hbase repository to > hbase-connectors. We have several 2.x releases so far, but we haven't > released hbase-connectors. > > 1. Are there any known issues which prevents us releasing it? Currently I > see 5 open issue where component = hbase-connectors [1]. > 2. Should the first release work with Java 11? Or Java 8 compatibility is > enough? > 3. I ran the tests and they succeeded. Currently pom.xml depends on HBase > 2.1.0, so I ran the tests only against that version. What versions should > we test? > 4. How should we release it? I do not have too much experiences with the > releases but I am happy to give helping hand. > > Best regards, > Balazs > > [1] https://s.apache.org/APjq >
Re: [VOTE] HBase Thirdparty 2.2.0 RC1
Shekar: You put your note on a VOTE thread. Please redo your query in its own thread to hbase dev. Can you paste some log when you redo your message on the dev list? Thanks, S On Mon, Apr 1, 2019 at 12:33 AM Shekhar Dhotre < shekhar.dho...@clairvoyantsoft.com> wrote: > Hi All , > I have a cluster which is setup to replicate to GCP . In the log its > showing cluster ID null so found this document which shows how to setup > cluster ID .but that is for a third party product called FusionInsight > Manager. > > > http://support-it.huawei.com/docs/en-us/fusioninsight-all/maintenance-guide/en-us_topic_0087315479.html > > > Where can I setup the below in Cloudera ? > > HBase Startup Failure Due to cluster.id Not Configured > > Region severs starts but as soon as I try to start master ,it brings down > all RS and master also doesnt come up . > > Thanks for your help in advance . > SD > Hadoop Admin . >
Re: [DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
On Mon, Apr 1, 2019 at 9:56 AM Wellington Chevreuil < wellington.chevre...@gmail.com> wrote: > > > > Am interested to hear what's the general opinion on the way to move forward > with hbck2? Should we completely discourage client bound logic patches such > as the one from HBASE-22143 ? > No. If in a bind -- need to change a state but don't have the facility on the master to do it in the kosher way -- then you have no recourse but to do 'dirty tricks' as HBASE-22143 does making use of 'private' Interfaces. What you think Wellington of checking master version in the client-side and doing 'dirty tricks' version if master version is old but taking the master-route when it provides support; i.e. when making a fix, we'd do two versions: one via the ordained route of asking the master to make the change for us, and then a workaround second version for those deploys that don't currently have the master-side support? Thanks for bringing up this topic, S
[DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
My understanding on the motivation of moving hbck2 (and potentially other operation related tools) to a separate project was to allow it to evolve on its own cadence, as new interventions to fix previously unforeseen problems were identified, we would now be able to deliver updated tool versions without requiring a new release on the main projects, therefore avoiding the need of a cluster upgrade. At same time, the current pattern for hbck2 at least is to avoid giving too much control of regions assignment inconsistencies over to the client, favouring an approach where hbck2 client mainly act as a proxy for fix methods exposed at master RPC interface. That way, master is able to keep track of all procedures manipulating regions, where an independent process manipulating regions at same time could cause more problems than fix it (we saw this happening many times with misuse of hbck v1 methods). However, with such restriction we are again limiting the ability to evolve independently of main project releases and cluster upgrades. With growing adoption of branch-2 based releases by our customer base, there's been a surge on assignment issues, which can't be fixed solely with the existing hbck2 commands. One example I had is what motivated HBASE-22143. I would favour the approach of allowing more complex client methods, mainly because it allows for adding new fixes on demand, but there's also value on keeping things owned by master. Am interested to hear what's the general opinion on the way to move forward with hbck2? Should we completely discourage client bound logic patches such as the one from HBASE-22143 ?
[jira] [Resolved] (HBASE-21525) java.io.IOException: java.lang.ClassNotFoundException: clone_snapshot from hbase2.0.0 to hbase1.2.0
[
https://issues.apache.org/jira/browse/HBASE-21525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar resolved HBASE-21525.
--
Resolution: Duplicate
Closing this as a duplicate of HBASE-16189
> java.io.IOException: java.lang.ClassNotFoundException: clone_snapshot from
> hbase2.0.0 to hbase1.2.0
>
>
> Key: HBASE-21525
> URL: https://issues.apache.org/jira/browse/HBASE-21525
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 1.2.0, 2.0.0
>Reporter: huan
>Priority: Minor
>
> When I use
> {code:java}
> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> "wh_scan_issue_monitor_on_snap_1" -copy-from hdfs://:8020/hbase -copy-to
> hdfs://xxx:8020/hbase
> {code}
> to copy snapshot from hbase2.0.0 to hbase 1.2.0 and then use
> {code:java}
> clone_snapshot
> 'wh_scan_issue_monitor_on_snap_1',"wh_scan_issue_monitor_table"{code}
> to restore the table;
> There are no errors on client and hbase master.
> but I can not scan any data in this new table.
> So when I search the log in regionserver which that region running on .I find
> the below error log
>
> {code:java}
> 2018-11-29 13:39:45,826 ERROR
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader: Error reading bloom
> filter meta for GENERAL_BLOOM_META -- proceeding without
> java.io.IOException: java.lang.ClassNotFoundException:
> at
> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.getComparatorClass(FixedFileTrailer.java:581)
> at
> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.createComparator(FixedFileTrailer.java:588)
> at
> org.apache.hadoop.hbase.util.CompoundBloomFilter.(CompoundBloomFilter.java:73)
> at
> org.apache.hadoop.hbase.util.BloomFilterFactory.createFromMeta(BloomFilterFactory.java:109)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.loadBloomfilter(StoreFile.java:1469)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.access$000(StoreFile.java:1088)
> at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:465)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:676)
> at org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:119)
> at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:543)
> at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:540)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.getComparatorClass(FixedFileTrailer.java:578)
> ... 18 more
> {code}
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [DISCUSS] release hbase-connectors
We're not ready to work with Java 11 as a project yet, IMHO, so Java 8 should be fine. On Mon, Apr 1, 2019 at 6:43 AM Balazs Meszaros wrote: > Hi All, > > Since 2.0 connectors have been moved out from hbase repository to > hbase-connectors. We have several 2.x releases so far, but we haven't > released hbase-connectors. > > 1. Are there any known issues which prevents us releasing it? Currently I > see 5 open issue where component = hbase-connectors [1]. > 2. Should the first release work with Java 11? Or Java 8 compatibility is > enough? > 3. I ran the tests and they succeeded. Currently pom.xml depends on HBase > 2.1.0, so I ran the tests only against that version. What versions should > we test? > 4. How should we release it? I do not have too much experiences with the > releases but I am happy to give helping hand. > > Best regards, > Balazs > > [1] https://s.apache.org/APjq >
[jira] [Created] (HBASE-22143) HBCK2 setRegionState command
Wellington Chevreuil created HBASE-22143:
Summary: HBCK2 setRegionState command
Key: HBASE-22143
URL: https://issues.apache.org/jira/browse/HBASE-22143
Project: HBase
Issue Type: New Feature
Components: hbase-operator-tools, hbck2
Reporter: Wellington Chevreuil
Among some of the current AMv2 issues, we faced situation where some regions
had state as OPENING in meta, with an RS startcode that was not valid anymore.
There was no AP running, the region stays permanently being logged as
IN-Transition on master logs, yet no procedure is really trying to bring it
online. Current hbck2 unassigns/assigns commands didn't work either, as per the
exception shown, it expects regions to be in state SPLITTING, SPLIT, MERGING,
OPEN, or CLOSING:
{noformat}
WARN org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
Failed transition, suspend 1secs pid=7093,
state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true; UnassignProcedure
table=rc_accounts, region=db85127b77fa56f7ad44e2c988e53925,
server=server1.example.com,16020,1552682193324; rit=OPENING,
location=server1.example.com,16020,1552682193324; waiting on rectified
condition fixed by other Procedure or operator intervention
org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected
[SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but current
state=OPENING
at
org.apache.hadoop.hbase.master.assignment.RegionStates$RegionStateNode.transitionState(RegionStates.java:166)
at
org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1479)
at
org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:212)
at
org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:369)
at
org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:97)
at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:957)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1835)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1595){noformat}
In this specific case, since we know the region is not actually being operated
by any proc and is not really open anywhere, it's ok to manually set it's state
to one of those assigns/unassigns can operate on, so this jira proposes a new
hbck2 command that allows for arbitrarily set a region to a given state.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [VOTE] HBase Thirdparty 2.2.0 RC1
+1 Checked signature, checksum Built from source Ran Apache Rat check Ran HBase master tests using this RC Checked CHANGES and RELEASENOTES On Sat, Mar 30, 2019 at 10:14 PM Stack wrote: > Please consider the following as the 2.2.0 release of Apache HBase > Thirdparty. It is the second release candidate for your consideration. > > It is a year since our 2.1.0 release. 2.2.0 updates some of our core > dependencies: > > gson 2.8.1 -> 2.8.5 > guava 22.0 -> 27.1-jre > pb 3.5.1 -> 3.7.0 > netty 4.1.17 -> 4.1.34 > commons-collections4 4.1 -> 4.3 > > The coming hbase-2.2.0 RC would like to include an updated > hbase-thirdparty. > > Source artifact, signatures, checksums, and changes are available at: > > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-thirdparty/hbase-thirdparty-2.2.0RC1/ > > The release was made against tag 2.2.0RC0. It was signed with my key > 'st...@duboce.net' which can be found here: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > Maven staging repository is available at: > > https://repository.apache.org/content/repositories/orgapachehbase-1297 > > This vote will remain open for at least 72 hours. > > Yours, > Your HBase Release Manager >
[jira] [Created] (HBASE-22142) Space quota: If table is dropped inside namespace, data size usage is still considered for the drop table.
Shardul Singh created HBASE-22142: - Summary: Space quota: If table is dropped inside namespace, data size usage is still considered for the drop table. Key: HBASE-22142 URL: https://issues.apache.org/jira/browse/HBASE-22142 Project: HBase Issue Type: Bug Affects Versions: 2.1.1 Reporter: Shardul Singh steps to follow: 1.create a quota at namespace level 2.create 2 tables t1 and t2 inside namespace and write data. 3.write 5 mb of data each in both t1 and t2. 4. drop t1. 5. data usage for t2 will be shown 10 mb for 10 minutes. If table is dropped inside namespace, data size usage is still shown for 10 minutes because of the configuration "hbase.master.quotas.region.report.retention.millis". This conf maintains the region report in cache(regionSizes) and old entry is used for 10 minutes. We can remove the entry during the drop command only, so that correct usage is show instantaneously after the drop command. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[DISCUSS] release hbase-connectors
Hi All, Since 2.0 connectors have been moved out from hbase repository to hbase-connectors. We have several 2.x releases so far, but we haven't released hbase-connectors. 1. Are there any known issues which prevents us releasing it? Currently I see 5 open issue where component = hbase-connectors [1]. 2. Should the first release work with Java 11? Or Java 8 compatibility is enough? 3. I ran the tests and they succeeded. Currently pom.xml depends on HBase 2.1.0, so I ran the tests only against that version. What versions should we test? 4. How should we release it? I do not have too much experiences with the releases but I am happy to give helping hand. Best regards, Balazs [1] https://s.apache.org/APjq
[jira] [Created] (HBASE-22141) Fix TestAsyncDecommissionAdminApi
Duo Zhang created HBASE-22141: - Summary: Fix TestAsyncDecommissionAdminApi Key: HBASE-22141 URL: https://issues.apache.org/jira/browse/HBASE-22141 Project: HBase Issue Type: Sub-task Components: test Reporter: Duo Zhang Fix For: 3.0.0, 2.2.0, 2.3.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Hbase master down due to null cluster ID
Hi All , I have a cluster which is setup to replicate to GCP . In the log its showing cluster ID null so found this document which shows how to setup cluster ID .but that is for a third party product called FusionInsight Manager. http://support-it.huawei.com/docs/en-us/fusioninsight-all/maintenance-guide/en-us_topic_0087315479.html Where can I setup the below in Cloudera ? HBase Startup Failure Due to cluster.id Not Configured Region severs starts but as soon as I try to start master ,it brings down all RS and master also doesnt come up . Thanks for your help in advance . SD Hadoop Admin . -- Thank you ! * Shekhar Dhotre.* senior Hadoop Administrator shekhar.dho...@clairvoyantsoft.com clairvoyantsoft.com/insight
Re: [VOTE] HBase Thirdparty 2.2.0 RC1
+1 (binding) Checked sigs and sums: OK Built from src: OK Minor: >From dev_files/make_rc.sh we are using "git archive" instead of "maven package assembly:single", thus producing more files as listed below. Is this on purpose? Thanks. hbase-thirdparty-2.2.0/.gitignore hbase-thirdparty-2.2.0/dev_files/make_rc.sh Best Regards, Yu On Sun, 31 Mar 2019 at 05:14, Stack wrote: > Please consider the following as the 2.2.0 release of Apache HBase > Thirdparty. It is the second release candidate for your consideration. > > It is a year since our 2.1.0 release. 2.2.0 updates some of our core > dependencies: > > gson 2.8.1 -> 2.8.5 > guava 22.0 -> 27.1-jre > pb 3.5.1 -> 3.7.0 > netty 4.1.17 -> 4.1.34 > commons-collections4 4.1 -> 4.3 > > The coming hbase-2.2.0 RC would like to include an updated > hbase-thirdparty. > > Source artifact, signatures, checksums, and changes are available at: > > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-thirdparty/hbase-thirdparty-2.2.0RC1/ > > The release was made against tag 2.2.0RC0. It was signed with my key > 'st...@duboce.net' which can be found here: > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > Maven staging repository is available at: > > https://repository.apache.org/content/repositories/orgapachehbase-1297 > > This vote will remain open for at least 72 hours. > > Yours, > Your HBase Release Manager >
Re: Replacement for CellUtil.setTimestamp()
The later JIRAs like https://issues.apache.org/jira/browse/HBASE-19092 exposed a CellBuilder and the CP has access to a the RawCellBuilder which will allow you to do a setTimeStamp on it. But note that it allows does a deep copy of the Cell bytes when the new cell is created out of the builder. Let us know if this helps or you need some other APIs to make things easier. Regards Ram On Mon, Apr 1, 2019 at 2:33 AM Wellington Chevreuil < wellington.chevre...@gmail.com> wrote: > You could risk use PrivateCellUtil.setTimestamp() method, but it may break > in future releases since PrivateCellUtil is marked > as @InterfaceAudience.Private. Another option, given both Put and Delete > map to KeyValue cell type, is to check if the Cell type is KeyValue, then > do a cast to access setTimestamp method on the KV. > > Em sex, 29 de mar de 2019 às 22:34, Thomas D'Silva > escreveu: > > > In HBase 2.0 since CellUtil.setTimestamp has been deprecated, what is the > > correct way to set the timestamp of a cell from a coprocessor? Phoenix > uses > > this API to set the timestamp of a call in our mutable indexing > coprocessor > > (see https://issues.apache.org/jira/browse/PHOENIX-5219). > > > > Thanks, > > Thomas > > >
Re: [VOTE] HBase Thirdparty 2.2.0 RC1
Hi All , I have a cluster which is setup to replicate to GCP . In the log its showing cluster ID null so found this document which shows how to setup cluster ID .but that is for a third party product called FusionInsight Manager. http://support-it.huawei.com/docs/en-us/fusioninsight-all/maintenance-guide/en-us_topic_0087315479.html Where can I setup the below in Cloudera ? HBase Startup Failure Due to cluster.id Not Configured Region severs starts but as soon as I try to start master ,it brings down all RS and master also doesnt come up . Thanks for your help in advance . SD Hadoop Admin .
[jira] [Created] (HBASE-22140) Fix those flaky tests in master branch
Zheng Hu created HBASE-22140: Summary: Fix those flaky tests in master branch Key: HBASE-22140 URL: https://issues.apache.org/jira/browse/HBASE-22140 Project: HBase Issue Type: Bug Reporter: Zheng Hu Found few flaky tests when running Hadoop QA for HBASE-22005, see: https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/master/lastSuccessfulBuild/artifact/dashboard.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
