Re: The fourth HBase 1.5.0 release candidate (RC3) is available
I have no doubt that you've run the tests locally before announcing a release as you're always a great RM boss. And this shows one value of verifying release, that different voter has different environments. Now I think the failures may be kerberos related, since I possibly has changed some system configuration when doing Flink testing on this env weeks ago. Located one issue (HBASE-22219) which also observed in 1.4.7, will further investigate. Best Regards, Yu On Fri, 12 Apr 2019 at 12:38, Andrew Purtell wrote: > “However it's good to find the issue earlier if there > really is any, before release announced.” > > I run the complete unit test suite before announcing a release candidate. > Just to be clear. > > Totally agree we should get these problems sorted before an actual > release. My policy is to cancel a RC if anyone vetoes for this reason... > want as much coverage and varying environments as we can manage. > > Thank you for your help so far and I hope the failures you see result in > analysis and fixes that lead to better test stability. > > > On Apr 11, 2019, at 9:32 PM, Yu Li wrote: > > > > Confirmed in 1.4.7 source the listed out cases passed (all in the 1st > part > > of hbase-server so the result comes out quickly.)... Also confirmed the > > test ran order are the same... > > > > Will try 1.5.0 again to prevent the environment difference caused by > time. > > If 1.5.0 still fails, will start to do the git bisect to locate the first > > bad commit. > > > > Was also expecting an easy pass and +1 as always to save time and > efforts, > > but obvious no luck. However it's good to find the issue earlier if there > > really is any, before release announced. > > > > Best Regards, > > Yu > > > > > >> On Fri, 12 Apr 2019 at 12:16, Yu Li wrote: > >> > >> Fine, let's focus on verifying whether it's a real problem rather than > >> arguing about wording, after all that's not my intention... > >> > >> As mentioned, I participated in the 1.4.7 release vote[1] and IIRC I was > >> using the same env and all tests passed w/o issue, that's where my > concern > >> lies and the main reason I gave a -1 vote. I'm running against 1.4.7 > source > >> on the same now and let's see the result. > >> > >> [1] https://www.mail-archive.com/dev@hbase.apache.org/msg51380.html > >> > >> Best Regards, > >> Yu > >> > >> > >> On Fri, 12 Apr 2019 at 12:05, Andrew Purtell > >> wrote: > >> > >>> I believe the test execution order matters. We run some tests in > >>> parallel. The ordering of tests is determined by readdir() results and > this > >>> differs from host to host and checkout to checkout. So when you see a > >>> repeatable group of failures, that’s great. And when someone else > doesn’t > >>> see those same tests fail, or they cannot be reproduced when running by > >>> themselves, the commonly accepted term of art for this is “flaky”. > >>> > >>> > On Apr 11, 2019, at 8:52 PM, Yu Li wrote: > > Sorry but I'd call it "possible environment related problem" or "some > feature may not work well in specific environment", rather than a > flaky. > > Will check against 1.4.7 released source package before opening any > >>> JIRA. > > Best Regards, > Yu > > > On Fri, 12 Apr 2019 at 11:37, Andrew Purtell < > andrew.purt...@gmail.com> > wrote: > > > And if they pass in my environment , then what should we call it > then. > >>> I > > have no doubt you are seeing failures. Therefore can you please file > >>> JIRAs > > and attach information that can help identify a fix. Thanks. > > > >> On Apr 11, 2019, at 8:35 PM, Yu Li wrote: > >> > >> I ran the test suite with the -Dsurefire.rerunFailingTestsCount=2 > >>> option > >> and on two different env separately, so it sums up to 6 times stable > >> failure for each case, and from my perspective this is not flaky. > >> > >> IIRC last time when verifying 1.4.7 on the same env no such issue > > observed, > >> will double check. > >> > >> Best Regards, > >> Yu > >> > >> > >> On Fri, 12 Apr 2019 at 00:07, Andrew Purtell < > >>> andrew.purt...@gmail.com> > >> wrote: > >> > >>> There are two failure cases it looks like. And this looks like > >>> flakes. > >>> > >>> The wrong FS assertions are not something I see when I run these > >>> tests > >>> myself. I am not able to investigate something I can’t reproduce. > >>> What I > >>> suggest is since you can reproduce do a git bisect to find the > commit > > that > >>> introduced the problem. Then we can revert it. As an alternative we > >>> can > >>> open a JIRA, report the problem, temporarily @ignore the test, and > >>> continue. This latter option only should be done if we are fairly > > confident > >>> it is a test only problem. > >>> > >>> The connect exceptions are interesting. I see these sometimes when > >>> the > >>> suite is
[jira] [Created] (HBASE-22219) Backport HBASE-19049 to branch-1 to prevent DIRKRB-613
Yu Li created HBASE-22219:
-
Summary: Backport HBASE-19049 to branch-1 to prevent DIRKRB-613
Key: HBASE-22219
URL: https://issues.apache.org/jira/browse/HBASE-22219
Project: HBase
Issue Type: Improvement
Components: test
Affects Versions: 1.4.9, 1.4.7, 1.4.8
Reporter: Yu Li
Assignee: Yu Li
Fix For: 1.5.0
Observed several UT failures when verifying 1.5.0 release, one of which is as
follows:
{noformat}
[ERROR] org.apache.hadoop.hbase.http.TestSpnegoHttpServer Time elapsed: 0.005
s <<< ERROR!
java.lang.RuntimeException: Unable to parse:includedir /etc/krb5.conf.d/
at
org.apache.hadoop.hbase.http.TestSpnegoHttpServer.buildMiniKdc(TestSpnegoHttpServer.java:143)
at
org.apache.hadoop.hbase.http.TestSpnegoHttpServer.setupServer(TestSpnegoHttpServer.java:91)
{noformat}
And this is a known issue of kerby 1.0.0-RC2 as indicated by DIRKRB-613 and
fixed in 1.0.0 release (by [this
commit|https://github.com/apache/directory-kerby/commit/e34b1ef8fec64e89780aec37aac903d4608e215f])
Thus we should backport HBASE-19049 which upgrade kerby dependency from
1.0.0-RC2 to 1.0.1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: The fourth HBase 1.5.0 release candidate (RC3) is available
“However it's good to find the issue earlier if there really is any, before release announced.” I run the complete unit test suite before announcing a release candidate. Just to be clear. Totally agree we should get these problems sorted before an actual release. My policy is to cancel a RC if anyone vetoes for this reason... want as much coverage and varying environments as we can manage. Thank you for your help so far and I hope the failures you see result in analysis and fixes that lead to better test stability. > On Apr 11, 2019, at 9:32 PM, Yu Li wrote: > > Confirmed in 1.4.7 source the listed out cases passed (all in the 1st part > of hbase-server so the result comes out quickly.)... Also confirmed the > test ran order are the same... > > Will try 1.5.0 again to prevent the environment difference caused by time. > If 1.5.0 still fails, will start to do the git bisect to locate the first > bad commit. > > Was also expecting an easy pass and +1 as always to save time and efforts, > but obvious no luck. However it's good to find the issue earlier if there > really is any, before release announced. > > Best Regards, > Yu > > >> On Fri, 12 Apr 2019 at 12:16, Yu Li wrote: >> >> Fine, let's focus on verifying whether it's a real problem rather than >> arguing about wording, after all that's not my intention... >> >> As mentioned, I participated in the 1.4.7 release vote[1] and IIRC I was >> using the same env and all tests passed w/o issue, that's where my concern >> lies and the main reason I gave a -1 vote. I'm running against 1.4.7 source >> on the same now and let's see the result. >> >> [1] https://www.mail-archive.com/dev@hbase.apache.org/msg51380.html >> >> Best Regards, >> Yu >> >> >> On Fri, 12 Apr 2019 at 12:05, Andrew Purtell >> wrote: >> >>> I believe the test execution order matters. We run some tests in >>> parallel. The ordering of tests is determined by readdir() results and this >>> differs from host to host and checkout to checkout. So when you see a >>> repeatable group of failures, that’s great. And when someone else doesn’t >>> see those same tests fail, or they cannot be reproduced when running by >>> themselves, the commonly accepted term of art for this is “flaky”. >>> >>> On Apr 11, 2019, at 8:52 PM, Yu Li wrote: Sorry but I'd call it "possible environment related problem" or "some feature may not work well in specific environment", rather than a flaky. Will check against 1.4.7 released source package before opening any >>> JIRA. Best Regards, Yu On Fri, 12 Apr 2019 at 11:37, Andrew Purtell wrote: > And if they pass in my environment , then what should we call it then. >>> I > have no doubt you are seeing failures. Therefore can you please file >>> JIRAs > and attach information that can help identify a fix. Thanks. > >> On Apr 11, 2019, at 8:35 PM, Yu Li wrote: >> >> I ran the test suite with the -Dsurefire.rerunFailingTestsCount=2 >>> option >> and on two different env separately, so it sums up to 6 times stable >> failure for each case, and from my perspective this is not flaky. >> >> IIRC last time when verifying 1.4.7 on the same env no such issue > observed, >> will double check. >> >> Best Regards, >> Yu >> >> >> On Fri, 12 Apr 2019 at 00:07, Andrew Purtell < >>> andrew.purt...@gmail.com> >> wrote: >> >>> There are two failure cases it looks like. And this looks like >>> flakes. >>> >>> The wrong FS assertions are not something I see when I run these >>> tests >>> myself. I am not able to investigate something I can’t reproduce. >>> What I >>> suggest is since you can reproduce do a git bisect to find the commit > that >>> introduced the problem. Then we can revert it. As an alternative we >>> can >>> open a JIRA, report the problem, temporarily @ignore the test, and >>> continue. This latter option only should be done if we are fairly > confident >>> it is a test only problem. >>> >>> The connect exceptions are interesting. I see these sometimes when >>> the >>> suite is executed, not this particular case, but when the failed >>> test is >>> executed by itself it always passes. It is possible some change to > classes >>> related to the minicluster or startup or shutdown timing are the >>> cause, > but >>> it is test time flaky behavior. I’m not happy about this but it >>> doesn’t >>> actually fail the release because the failure is never repeatable >>> when > the >>> test is run standalone. >>> >>> In general it would be great if some attention was paid to test >>> cleanliness on branch-1. As RM I’m not in a position to insist that >>> everything is perfect or there will never be another 1.x release, > certainly >>> not from branch-1. So, tests which fail repeatedly
Re: The fourth HBase 1.5.0 release candidate (RC3) is available
Confirmed in 1.4.7 source the listed out cases passed (all in the 1st part of hbase-server so the result comes out quickly.)... Also confirmed the test ran order are the same... Will try 1.5.0 again to prevent the environment difference caused by time. If 1.5.0 still fails, will start to do the git bisect to locate the first bad commit. Was also expecting an easy pass and +1 as always to save time and efforts, but obvious no luck. However it's good to find the issue earlier if there really is any, before release announced. Best Regards, Yu On Fri, 12 Apr 2019 at 12:16, Yu Li wrote: > Fine, let's focus on verifying whether it's a real problem rather than > arguing about wording, after all that's not my intention... > > As mentioned, I participated in the 1.4.7 release vote[1] and IIRC I was > using the same env and all tests passed w/o issue, that's where my concern > lies and the main reason I gave a -1 vote. I'm running against 1.4.7 source > on the same now and let's see the result. > > [1] https://www.mail-archive.com/dev@hbase.apache.org/msg51380.html > > Best Regards, > Yu > > > On Fri, 12 Apr 2019 at 12:05, Andrew Purtell > wrote: > >> I believe the test execution order matters. We run some tests in >> parallel. The ordering of tests is determined by readdir() results and this >> differs from host to host and checkout to checkout. So when you see a >> repeatable group of failures, that’s great. And when someone else doesn’t >> see those same tests fail, or they cannot be reproduced when running by >> themselves, the commonly accepted term of art for this is “flaky”. >> >> >> > On Apr 11, 2019, at 8:52 PM, Yu Li wrote: >> > >> > Sorry but I'd call it "possible environment related problem" or "some >> > feature may not work well in specific environment", rather than a flaky. >> > >> > Will check against 1.4.7 released source package before opening any >> JIRA. >> > >> > Best Regards, >> > Yu >> > >> > >> > On Fri, 12 Apr 2019 at 11:37, Andrew Purtell >> > wrote: >> > >> >> And if they pass in my environment , then what should we call it then. >> I >> >> have no doubt you are seeing failures. Therefore can you please file >> JIRAs >> >> and attach information that can help identify a fix. Thanks. >> >> >> >>> On Apr 11, 2019, at 8:35 PM, Yu Li wrote: >> >>> >> >>> I ran the test suite with the -Dsurefire.rerunFailingTestsCount=2 >> option >> >>> and on two different env separately, so it sums up to 6 times stable >> >>> failure for each case, and from my perspective this is not flaky. >> >>> >> >>> IIRC last time when verifying 1.4.7 on the same env no such issue >> >> observed, >> >>> will double check. >> >>> >> >>> Best Regards, >> >>> Yu >> >>> >> >>> >> >>> On Fri, 12 Apr 2019 at 00:07, Andrew Purtell < >> andrew.purt...@gmail.com> >> >>> wrote: >> >>> >> There are two failure cases it looks like. And this looks like >> flakes. >> >> The wrong FS assertions are not something I see when I run these >> tests >> myself. I am not able to investigate something I can’t reproduce. >> What I >> suggest is since you can reproduce do a git bisect to find the commit >> >> that >> introduced the problem. Then we can revert it. As an alternative we >> can >> open a JIRA, report the problem, temporarily @ignore the test, and >> continue. This latter option only should be done if we are fairly >> >> confident >> it is a test only problem. >> >> The connect exceptions are interesting. I see these sometimes when >> the >> suite is executed, not this particular case, but when the failed >> test is >> executed by itself it always passes. It is possible some change to >> >> classes >> related to the minicluster or startup or shutdown timing are the >> cause, >> >> but >> it is test time flaky behavior. I’m not happy about this but it >> doesn’t >> actually fail the release because the failure is never repeatable >> when >> >> the >> test is run standalone. >> >> In general it would be great if some attention was paid to test >> cleanliness on branch-1. As RM I’m not in a position to insist that >> everything is perfect or there will never be another 1.x release, >> >> certainly >> not from branch-1. So, tests which fail repeatedly block a release >> IMHO >> >> but >> flakes do not. >> >> >> > On Apr 10, 2019, at 11:20 PM, Yu Li wrote: >> > >> > -1 >> > >> > Observed many UT failures when checking the source package (tried >> multiple >> > rounds on two different environments, MacOs and Linux, got the same >> > result), including (but not limited to): >> > >> > TestBulkload: >> > >> >> >> >> shouldBulkLoadSingleFamilyHLog(org.apache.hadoop.hbase.regionserver.TestBulkLoad) >> > Time elapsed: 0.083 s <<< ERROR! >> > java.lang.IllegalArgumentException: Wrong FS: >> > >> >> >> >>
Re: The fourth HBase 1.5.0 release candidate (RC3) is available
That sounds good. > On Apr 11, 2019, at 9:16 PM, Yu Li wrote: > > Fine, let's focus on verifying whether it's a real problem rather than > arguing about wording, after all that's not my intention... > > As mentioned, I participated in the 1.4.7 release vote[1] and IIRC I was > using the same env and all tests passed w/o issue, that's where my concern > lies and the main reason I gave a -1 vote. I'm running against 1.4.7 source > on the same now and let's see the result. > > [1] https://www.mail-archive.com/dev@hbase.apache.org/msg51380.html > > Best Regards, > Yu > > > On Fri, 12 Apr 2019 at 12:05, Andrew Purtell > wrote: > >> I believe the test execution order matters. We run some tests in parallel. >> The ordering of tests is determined by readdir() results and this differs >> from host to host and checkout to checkout. So when you see a repeatable >> group of failures, that’s great. And when someone else doesn’t see those >> same tests fail, or they cannot be reproduced when running by themselves, >> the commonly accepted term of art for this is “flaky”. >> >> >>> On Apr 11, 2019, at 8:52 PM, Yu Li wrote: >>> >>> Sorry but I'd call it "possible environment related problem" or "some >>> feature may not work well in specific environment", rather than a flaky. >>> >>> Will check against 1.4.7 released source package before opening any JIRA. >>> >>> Best Regards, >>> Yu >>> >>> >>> On Fri, 12 Apr 2019 at 11:37, Andrew Purtell >>> wrote: >>> And if they pass in my environment , then what should we call it then. I have no doubt you are seeing failures. Therefore can you please file >> JIRAs and attach information that can help identify a fix. Thanks. > On Apr 11, 2019, at 8:35 PM, Yu Li wrote: > > I ran the test suite with the -Dsurefire.rerunFailingTestsCount=2 >> option > and on two different env separately, so it sums up to 6 times stable > failure for each case, and from my perspective this is not flaky. > > IIRC last time when verifying 1.4.7 on the same env no such issue observed, > will double check. > > Best Regards, > Yu > > > On Fri, 12 Apr 2019 at 00:07, Andrew Purtell >> > wrote: > >> There are two failure cases it looks like. And this looks like flakes. >> >> The wrong FS assertions are not something I see when I run these tests >> myself. I am not able to investigate something I can’t reproduce. >> What I >> suggest is since you can reproduce do a git bisect to find the commit that >> introduced the problem. Then we can revert it. As an alternative we >> can >> open a JIRA, report the problem, temporarily @ignore the test, and >> continue. This latter option only should be done if we are fairly confident >> it is a test only problem. >> >> The connect exceptions are interesting. I see these sometimes when the >> suite is executed, not this particular case, but when the failed test >> is >> executed by itself it always passes. It is possible some change to classes >> related to the minicluster or startup or shutdown timing are the >> cause, but >> it is test time flaky behavior. I’m not happy about this but it >> doesn’t >> actually fail the release because the failure is never repeatable when the >> test is run standalone. >> >> In general it would be great if some attention was paid to test >> cleanliness on branch-1. As RM I’m not in a position to insist that >> everything is perfect or there will never be another 1.x release, certainly >> not from branch-1. So, tests which fail repeatedly block a release >> IMHO but >> flakes do not. >> >> >>> On Apr 10, 2019, at 11:20 PM, Yu Li wrote: >>> >>> -1 >>> >>> Observed many UT failures when checking the source package (tried >> multiple >>> rounds on two different environments, MacOs and Linux, got the same >>> result), including (but not limited to): >>> >>> TestBulkload: >>> >> >> shouldBulkLoadSingleFamilyHLog(org.apache.hadoop.hbase.regionserver.TestBulkLoad) >>> Time elapsed: 0.083 s <<< ERROR! >>> java.lang.IllegalArgumentException: Wrong FS: >>> >> >> file:/var/folders/t6/vch4nh357f98y1wlq09lbm7hgn/T/junit1805329913454564189/junit8020757893576011944/data/default/shouldBulkLoadSingleFamilyHLog/8f4a6b584533de2fd1bf3c398dfaac29, >>> expected: hdfs://localhost:55938 >>> at >>> >> >> org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamiliesAndSpecifiedTableName(TestBulkLoad.java:246) >>> at >>> >> >> org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamilies(TestBulkLoad.java:256) >>> at >>> >> >> org.apache.hadoop.hbase.regionserver.TestBulkLoad.shouldBulkLoadSingleFamilyHLog(TestBulkLoad.java:150)
Re: The fourth HBase 1.5.0 release candidate (RC3) is available
Fine, let's focus on verifying whether it's a real problem rather than arguing about wording, after all that's not my intention... As mentioned, I participated in the 1.4.7 release vote[1] and IIRC I was using the same env and all tests passed w/o issue, that's where my concern lies and the main reason I gave a -1 vote. I'm running against 1.4.7 source on the same now and let's see the result. [1] https://www.mail-archive.com/dev@hbase.apache.org/msg51380.html Best Regards, Yu On Fri, 12 Apr 2019 at 12:05, Andrew Purtell wrote: > I believe the test execution order matters. We run some tests in parallel. > The ordering of tests is determined by readdir() results and this differs > from host to host and checkout to checkout. So when you see a repeatable > group of failures, that’s great. And when someone else doesn’t see those > same tests fail, or they cannot be reproduced when running by themselves, > the commonly accepted term of art for this is “flaky”. > > > > On Apr 11, 2019, at 8:52 PM, Yu Li wrote: > > > > Sorry but I'd call it "possible environment related problem" or "some > > feature may not work well in specific environment", rather than a flaky. > > > > Will check against 1.4.7 released source package before opening any JIRA. > > > > Best Regards, > > Yu > > > > > > On Fri, 12 Apr 2019 at 11:37, Andrew Purtell > > wrote: > > > >> And if they pass in my environment , then what should we call it then. I > >> have no doubt you are seeing failures. Therefore can you please file > JIRAs > >> and attach information that can help identify a fix. Thanks. > >> > >>> On Apr 11, 2019, at 8:35 PM, Yu Li wrote: > >>> > >>> I ran the test suite with the -Dsurefire.rerunFailingTestsCount=2 > option > >>> and on two different env separately, so it sums up to 6 times stable > >>> failure for each case, and from my perspective this is not flaky. > >>> > >>> IIRC last time when verifying 1.4.7 on the same env no such issue > >> observed, > >>> will double check. > >>> > >>> Best Regards, > >>> Yu > >>> > >>> > >>> On Fri, 12 Apr 2019 at 00:07, Andrew Purtell > > >>> wrote: > >>> > There are two failure cases it looks like. And this looks like flakes. > > The wrong FS assertions are not something I see when I run these tests > myself. I am not able to investigate something I can’t reproduce. > What I > suggest is since you can reproduce do a git bisect to find the commit > >> that > introduced the problem. Then we can revert it. As an alternative we > can > open a JIRA, report the problem, temporarily @ignore the test, and > continue. This latter option only should be done if we are fairly > >> confident > it is a test only problem. > > The connect exceptions are interesting. I see these sometimes when the > suite is executed, not this particular case, but when the failed test > is > executed by itself it always passes. It is possible some change to > >> classes > related to the minicluster or startup or shutdown timing are the > cause, > >> but > it is test time flaky behavior. I’m not happy about this but it > doesn’t > actually fail the release because the failure is never repeatable when > >> the > test is run standalone. > > In general it would be great if some attention was paid to test > cleanliness on branch-1. As RM I’m not in a position to insist that > everything is perfect or there will never be another 1.x release, > >> certainly > not from branch-1. So, tests which fail repeatedly block a release > IMHO > >> but > flakes do not. > > > > On Apr 10, 2019, at 11:20 PM, Yu Li wrote: > > > > -1 > > > > Observed many UT failures when checking the source package (tried > multiple > > rounds on two different environments, MacOs and Linux, got the same > > result), including (but not limited to): > > > > TestBulkload: > > > > >> > shouldBulkLoadSingleFamilyHLog(org.apache.hadoop.hbase.regionserver.TestBulkLoad) > > Time elapsed: 0.083 s <<< ERROR! > > java.lang.IllegalArgumentException: Wrong FS: > > > > >> > file:/var/folders/t6/vch4nh357f98y1wlq09lbm7hgn/T/junit1805329913454564189/junit8020757893576011944/data/default/shouldBulkLoadSingleFamilyHLog/8f4a6b584533de2fd1bf3c398dfaac29, > > expected: hdfs://localhost:55938 > > at > > > > >> > org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamiliesAndSpecifiedTableName(TestBulkLoad.java:246) > > at > > > > >> > org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamilies(TestBulkLoad.java:256) > > at > > > > >> > org.apache.hadoop.hbase.regionserver.TestBulkLoad.shouldBulkLoadSingleFamilyHLog(TestBulkLoad.java:150) > > > > TestStoreFile: > > > > >> > testCacheOnWriteEvictOnClose(org.apache.hadoop.hbase.regionserver.TestStoreFile) > > Time elapsed:
Re: The fourth HBase 1.5.0 release candidate (RC3) is available
And if they pass in my environment , then what should we call it then. I have no doubt you are seeing failures. Therefore can you please file JIRAs and attach information that can help identify a fix. Thanks. > On Apr 11, 2019, at 8:35 PM, Yu Li wrote: > > I ran the test suite with the -Dsurefire.rerunFailingTestsCount=2 option > and on two different env separately, so it sums up to 6 times stable > failure for each case, and from my perspective this is not flaky. > > IIRC last time when verifying 1.4.7 on the same env no such issue observed, > will double check. > > Best Regards, > Yu > > > On Fri, 12 Apr 2019 at 00:07, Andrew Purtell > wrote: > >> There are two failure cases it looks like. And this looks like flakes. >> >> The wrong FS assertions are not something I see when I run these tests >> myself. I am not able to investigate something I can’t reproduce. What I >> suggest is since you can reproduce do a git bisect to find the commit that >> introduced the problem. Then we can revert it. As an alternative we can >> open a JIRA, report the problem, temporarily @ignore the test, and >> continue. This latter option only should be done if we are fairly confident >> it is a test only problem. >> >> The connect exceptions are interesting. I see these sometimes when the >> suite is executed, not this particular case, but when the failed test is >> executed by itself it always passes. It is possible some change to classes >> related to the minicluster or startup or shutdown timing are the cause, but >> it is test time flaky behavior. I’m not happy about this but it doesn’t >> actually fail the release because the failure is never repeatable when the >> test is run standalone. >> >> In general it would be great if some attention was paid to test >> cleanliness on branch-1. As RM I’m not in a position to insist that >> everything is perfect or there will never be another 1.x release, certainly >> not from branch-1. So, tests which fail repeatedly block a release IMHO but >> flakes do not. >> >> >>> On Apr 10, 2019, at 11:20 PM, Yu Li wrote: >>> >>> -1 >>> >>> Observed many UT failures when checking the source package (tried >> multiple >>> rounds on two different environments, MacOs and Linux, got the same >>> result), including (but not limited to): >>> >>> TestBulkload: >>> >> shouldBulkLoadSingleFamilyHLog(org.apache.hadoop.hbase.regionserver.TestBulkLoad) >>> Time elapsed: 0.083 s <<< ERROR! >>> java.lang.IllegalArgumentException: Wrong FS: >>> >> file:/var/folders/t6/vch4nh357f98y1wlq09lbm7hgn/T/junit1805329913454564189/junit8020757893576011944/data/default/shouldBulkLoadSingleFamilyHLog/8f4a6b584533de2fd1bf3c398dfaac29, >>> expected: hdfs://localhost:55938 >>> at >>> >> org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamiliesAndSpecifiedTableName(TestBulkLoad.java:246) >>> at >>> >> org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamilies(TestBulkLoad.java:256) >>> at >>> >> org.apache.hadoop.hbase.regionserver.TestBulkLoad.shouldBulkLoadSingleFamilyHLog(TestBulkLoad.java:150) >>> >>> TestStoreFile: >>> >> testCacheOnWriteEvictOnClose(org.apache.hadoop.hbase.regionserver.TestStoreFile) >>> Time elapsed: 0.083 s <<< ERROR! >>> java.net.ConnectException: Call From localhost/127.0.0.1 to >> localhost:55938 >>> failed on connection exception: java.net.ConnectException: Connection >>> refused; For more details see: >>> http://wiki.apache.org/hadoop/ConnectionRefused >>> at >>> >> org.apache.hadoop.hbase.regionserver.TestStoreFile.writeStoreFile(TestStoreFile.java:1047) >>> at >>> >> org.apache.hadoop.hbase.regionserver.TestStoreFile.testCacheOnWriteEvictOnClose(TestStoreFile.java:908) >>> >>> TestHFile: >>> testEmptyHFile(org.apache.hadoop.hbase.io.hfile.TestHFile) Time elapsed: >>> 0.08 s <<< ERROR! >>> java.net.ConnectException: Call From >>> z05f06378.sqa.zth.tbsite.net/11.163.183.195 to localhost:35529 failed on >>> connection exception: java.net.ConnectException: Connection refused; For >>> more details see: http://wiki.apache.org/hadoop/ConnectionRefused >>> at >>> org.apache.hadoop.hbase.io >> .hfile.TestHFile.testEmptyHFile(TestHFile.java:90) >>> Caused by: java.net.ConnectException: Connection refused >>> at >>> org.apache.hadoop.hbase.io >> .hfile.TestHFile.testEmptyHFile(TestHFile.java:90) >>> >>> TestBlocksScanned: >>> >> testBlocksScannedWithEncoding(org.apache.hadoop.hbase.regionserver.TestBlocksScanned) >>> Time elapsed: 0.069 s <<< ERROR! >>> java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:35529/tmp/ >>> >> hbase-jueding.ly/hbase/data/default/TestBlocksScannedWithEncoding/a4a416cc3060d9820a621c294af0aa08 >> , >>> expected: file:/// >>> at >>> >> org.apache.hadoop.hbase.regionserver.TestBlocksScanned._testBlocksScanned(TestBlocksScanned.java:90) >>> at >>> >>
Re: The fourth HBase 1.5.0 release candidate (RC3) is available
I ran the test suite with the -Dsurefire.rerunFailingTestsCount=2 option and on two different env separately, so it sums up to 6 times stable failure for each case, and from my perspective this is not flaky. IIRC last time when verifying 1.4.7 on the same env no such issue observed, will double check. Best Regards, Yu On Fri, 12 Apr 2019 at 00:07, Andrew Purtell wrote: > There are two failure cases it looks like. And this looks like flakes. > > The wrong FS assertions are not something I see when I run these tests > myself. I am not able to investigate something I can’t reproduce. What I > suggest is since you can reproduce do a git bisect to find the commit that > introduced the problem. Then we can revert it. As an alternative we can > open a JIRA, report the problem, temporarily @ignore the test, and > continue. This latter option only should be done if we are fairly confident > it is a test only problem. > > The connect exceptions are interesting. I see these sometimes when the > suite is executed, not this particular case, but when the failed test is > executed by itself it always passes. It is possible some change to classes > related to the minicluster or startup or shutdown timing are the cause, but > it is test time flaky behavior. I’m not happy about this but it doesn’t > actually fail the release because the failure is never repeatable when the > test is run standalone. > > In general it would be great if some attention was paid to test > cleanliness on branch-1. As RM I’m not in a position to insist that > everything is perfect or there will never be another 1.x release, certainly > not from branch-1. So, tests which fail repeatedly block a release IMHO but > flakes do not. > > > > On Apr 10, 2019, at 11:20 PM, Yu Li wrote: > > > > -1 > > > > Observed many UT failures when checking the source package (tried > multiple > > rounds on two different environments, MacOs and Linux, got the same > > result), including (but not limited to): > > > > TestBulkload: > > > shouldBulkLoadSingleFamilyHLog(org.apache.hadoop.hbase.regionserver.TestBulkLoad) > > Time elapsed: 0.083 s <<< ERROR! > > java.lang.IllegalArgumentException: Wrong FS: > > > file:/var/folders/t6/vch4nh357f98y1wlq09lbm7hgn/T/junit1805329913454564189/junit8020757893576011944/data/default/shouldBulkLoadSingleFamilyHLog/8f4a6b584533de2fd1bf3c398dfaac29, > > expected: hdfs://localhost:55938 > >at > > > org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamiliesAndSpecifiedTableName(TestBulkLoad.java:246) > >at > > > org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamilies(TestBulkLoad.java:256) > >at > > > org.apache.hadoop.hbase.regionserver.TestBulkLoad.shouldBulkLoadSingleFamilyHLog(TestBulkLoad.java:150) > > > > TestStoreFile: > > > testCacheOnWriteEvictOnClose(org.apache.hadoop.hbase.regionserver.TestStoreFile) > > Time elapsed: 0.083 s <<< ERROR! > > java.net.ConnectException: Call From localhost/127.0.0.1 to > localhost:55938 > > failed on connection exception: java.net.ConnectException: Connection > > refused; For more details see: > > http://wiki.apache.org/hadoop/ConnectionRefused > >at > > > org.apache.hadoop.hbase.regionserver.TestStoreFile.writeStoreFile(TestStoreFile.java:1047) > >at > > > org.apache.hadoop.hbase.regionserver.TestStoreFile.testCacheOnWriteEvictOnClose(TestStoreFile.java:908) > > > > TestHFile: > > testEmptyHFile(org.apache.hadoop.hbase.io.hfile.TestHFile) Time elapsed: > > 0.08 s <<< ERROR! > > java.net.ConnectException: Call From > > z05f06378.sqa.zth.tbsite.net/11.163.183.195 to localhost:35529 failed on > > connection exception: java.net.ConnectException: Connection refused; For > > more details see: http://wiki.apache.org/hadoop/ConnectionRefused > >at > > org.apache.hadoop.hbase.io > .hfile.TestHFile.testEmptyHFile(TestHFile.java:90) > > Caused by: java.net.ConnectException: Connection refused > >at > > org.apache.hadoop.hbase.io > .hfile.TestHFile.testEmptyHFile(TestHFile.java:90) > > > > TestBlocksScanned: > > > testBlocksScannedWithEncoding(org.apache.hadoop.hbase.regionserver.TestBlocksScanned) > > Time elapsed: 0.069 s <<< ERROR! > > java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:35529/tmp/ > > > hbase-jueding.ly/hbase/data/default/TestBlocksScannedWithEncoding/a4a416cc3060d9820a621c294af0aa08 > , > > expected: file:/// > >at > > > org.apache.hadoop.hbase.regionserver.TestBlocksScanned._testBlocksScanned(TestBlocksScanned.java:90) > >at > > > org.apache.hadoop.hbase.regionserver.TestBlocksScanned.testBlocksScannedWithEncoding(TestBlocksScanned.java:86) > > > > And please let me know if any known issue I'm not aware of. Thanks. > > > > Best Regards, > > Yu > > > > > >> On Mon, 8 Apr 2019 at 11:38, Yu Li wrote: > >> > >> The performance report LGTM, thanks! (and sorry for the lag due to > >> Qingming Festival Holiday here in China) > >> > >> Still verifying the release,
[jira] [Resolved] (HBASE-8549) Integrate Favored Nodes into StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-8549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-8549. -- Resolution: Implemented I'll take your word for it [~gsbiju]. Implemented by HBASE-16942 > Integrate Favored Nodes into StochasticLoadBalancer > --- > > Key: HBASE-8549 > URL: https://issues.apache.org/jira/browse/HBASE-8549 > Project: HBase > Issue Type: Bug > Components: Balancer >Reporter: Elliott Clark >Priority: Major > Attachments: HBASE-8549-0.patch > > > Right now we have a FavoredNodeLoadBalancer. It would be pretty easy to > integrate the favored node list into the strochastic balancer. Then we would > have the best of both worlds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-16513) Documentation for new RowIndex DataBlockEncoding
[ https://issues.apache.org/jira/browse/HBASE-16513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin resolved HBASE-16513. -- Resolution: Resolved Fix Version/s: (was: 1.5.0) (was: 3.0.0) > Documentation for new RowIndex DataBlockEncoding > > > Key: HBASE-16513 > URL: https://issues.apache.org/jira/browse/HBASE-16513 > Project: HBase > Issue Type: Improvement > Components: documentation, Performance >Reporter: binlijin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] First release candidate for HBase 1.2.12 is available
+1 Checked sums and signatures: ok Apache RAT check: ok (8u172) Build from source: ok (7u80) Unit tests pass: ok (8u172) 1M row LTT: ok (8u172) On Sun, Apr 7, 2019 at 12:03 PM Sean Busbey wrote: > The first release candidate for HBase 1.2.12 is available for download: > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.2.12RC0/ > > Maven artifacts are also available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1301/ > > Artifacts are signed with my key (0D80DB7C) published in our KEYS > file at http://www.apache.org/dist/hbase/KEYS > > The RC corresponds to the signed tag 1.2.12RC0, which currently points > to commit ref > > 91d5ec4c4dcd10ceec984c6e663ea82acf353995 > > HBase 1.2.12 is the twelfth maintenance release in the HBase 1.2 line, > continuing on the theme of bringing a stable, reliable database to the > Hadoop and NoSQL communities. This release includes one bug fix and > an operational improvement done in the month and a half since 1.2.11. > > The detailed source and binary compatibility report vs 1.2.11 has been > published for your review, at: > > https://s.apache.org/hbase-1.2.12-rc0-compat-report > > The report shows no issues. > > Critical changes include: > > * HBASE-22058 - upgrade thrift dependency to 0.9.3.1 (fixes a CVE) > * HBASE-21926 - Profiler servlet > > The full list of fixes included in this release is available at: > > https://s.apache.org/hbase-1.2.12-jira-release-notes > > and in the CHANGES.txt file included in the distribution. > > Please try out this candidate and vote +1/-1 on whether we should > release these artifacts as HBase 1.2.12. > > The VOTE will remain open for at least 72 hours. > > thanks! > > -busbey > > as of this email the posted artifacts have the following SHA512: > > hbase-1.2.12-src.tar.gz: > > F671CA53 19145732 EAC45FB4 A14E716D 827295E5 200F1C5E > A8976D00 55B4D65B D1D61C96 B2455000 F693B53D 6396F643 > 7F8EE4D2 D1E18A18 47FE02F0 423003F0 > > > hbase-1.2.12-bin.tar.gz: > > 75F4EFED 827C9BB9 C8767E86 CE5E8723 8FB0418F 385EFB7A > C02DAFD5 BB37105F EF535A13 6F44708B 46ABA9CF 4D161324 > B086637D F2372299 1EB006FF ACBF02DC > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-22218) Shell throws "Unsupported Java version" when tried with Java 11 (run-time)
Sakthi created HBASE-22218:
--
Summary: Shell throws "Unsupported Java version" when tried with
Java 11 (run-time)
Key: HBASE-22218
URL: https://issues.apache.org/jira/browse/HBASE-22218
Project: HBase
Issue Type: Bug
Reporter: Sakthi
Assignee: Sakthi
Following warning is thrown in the shell.
{noformat}
unsupported Java version "11", defaulting to 1.7{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22217) HBase shell command proposal : "rit assign all"
Xu Cang created HBASE-22217: --- Summary: HBase shell command proposal : "rit assign all" Key: HBASE-22217 URL: https://issues.apache.org/jira/browse/HBASE-22217 Project: HBase Issue Type: New Feature Reporter: Xu Cang HBase shell command proposal : "rit assign all" Currently we have shell command "rit" to list all RITs. It would be handy having a command "rit assign all" to assign all RITs. This equals to getting the list of RITs from 'rit' command and running "assign " one by one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22216) "Waiting on master failover to complete" shows 30 to 40 time per millisecond
Xu Cang created HBASE-22216: --- Summary: "Waiting on master failover to complete" shows 30 to 40 time per millisecond Key: HBASE-22216 URL: https://issues.apache.org/jira/browse/HBASE-22216 Project: HBase Issue Type: Bug Components: proc-v2 Affects Versions: 1.3.0 Reporter: Xu Cang "Waiting on master failover to complete" shows 30 to 40 time per millisecond from one host when master is initializing. This message is too noisy. Need to fix this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-7297) Allow load balancer to accommodate different region server configurations
[
https://issues.apache.org/jira/browse/HBASE-7297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Marc Spaggiari resolved HBASE-7297.
Resolution: Duplicate
Duplicate of HBASE-11780.
> Allow load balancer to accommodate different region server configurations
> -
>
> Key: HBASE-7297
> URL: https://issues.apache.org/jira/browse/HBASE-7297
> Project: HBase
> Issue Type: New Feature
> Components: Balancer
>Reporter: Ted Yu
>Priority: Major
>
> Robert Dyer raised the following scenario under the thread of 'Multiple
> regionservers on a single node':
> {quote}
> I have a very small cluster where all nodes are identical. However, I was
> just given a very powerful node to add into this cluster which effectively
> doubles the total CPUs, RAM, and HDDs in the cluster.
> As such, when I run a MR job half the jobs go to this single, new node yet
> most of the data is not local due to HBase balancing the regions.
> {quote}
> Load balancer should take region server config (total heap in the above case)
> into account when allocating regions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [VOTE] First release candidate for HBase 1.2.12 is available
Sorry Sean I missed this email last week. Let me check now. On Sun, Apr 7, 2019 at 12:03 PM Sean Busbey wrote: > The first release candidate for HBase 1.2.12 is available for download: > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.2.12RC0/ > > Maven artifacts are also available in a staging repository at: > > https://repository.apache.org/content/repositories/orgapachehbase-1301/ > > Artifacts are signed with my key (0D80DB7C) published in our KEYS > file at http://www.apache.org/dist/hbase/KEYS > > The RC corresponds to the signed tag 1.2.12RC0, which currently points > to commit ref > > 91d5ec4c4dcd10ceec984c6e663ea82acf353995 > > HBase 1.2.12 is the twelfth maintenance release in the HBase 1.2 line, > continuing on the theme of bringing a stable, reliable database to the > Hadoop and NoSQL communities. This release includes one bug fix and > an operational improvement done in the month and a half since 1.2.11. > > The detailed source and binary compatibility report vs 1.2.11 has been > published for your review, at: > > https://s.apache.org/hbase-1.2.12-rc0-compat-report > > The report shows no issues. > > Critical changes include: > > * HBASE-22058 - upgrade thrift dependency to 0.9.3.1 (fixes a CVE) > * HBASE-21926 - Profiler servlet > > The full list of fixes included in this release is available at: > > https://s.apache.org/hbase-1.2.12-jira-release-notes > > and in the CHANGES.txt file included in the distribution. > > Please try out this candidate and vote +1/-1 on whether we should > release these artifacts as HBase 1.2.12. > > The VOTE will remain open for at least 72 hours. > > thanks! > > -busbey > > as of this email the posted artifacts have the following SHA512: > > hbase-1.2.12-src.tar.gz: > > F671CA53 19145732 EAC45FB4 A14E716D 827295E5 200F1C5E > A8976D00 55B4D65B D1D61C96 B2455000 F693B53D 6396F643 > 7F8EE4D2 D1E18A18 47FE02F0 423003F0 > > > hbase-1.2.12-bin.tar.gz: > > 75F4EFED 827C9BB9 C8767E86 CE5E8723 8FB0418F 385EFB7A > C02DAFD5 BB37105F EF535A13 6F44708B 46ABA9CF 4D161324 > B086637D F2372299 1EB006FF ACBF02DC > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Resolved] (HBASE-3268) Run balancer when a new node is added
[ https://issues.apache.org/jira/browse/HBASE-3268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-3268. --- Resolution: Incomplete > Run balancer when a new node is added > - > > Key: HBASE-3268 > URL: https://issues.apache.org/jira/browse/HBASE-3268 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Todd Lipcon >Priority: Major > > Right now we only balance the cluster once every 5 minutes by default. This > is likely to confuse new users. When you start a new region server, you > expect it to pick up some load very quickly, but right now you have to wait 5 > minutes for it to start doing anything in the worst case. > We could/should also add a button/shell command to "trigger balance now" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
RC of 1.4.10 next week
Although branch-1 (1.5.0) is stalled on unit test issues I think we can make progress on another, overdue release, the next 1.4.x, 1.4.10. Let me start the work on the first release candidate of 1.4.10 Monday of next week. If you have any pending work targeting branch-1.4 please try to get it in before then. -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-22215) Backport MultiRowRangeFilter does not work with reverse scans
Josh Elser created HBASE-22215: -- Summary: Backport MultiRowRangeFilter does not work with reverse scans Key: HBASE-22215 URL: https://issues.apache.org/jira/browse/HBASE-22215 Project: HBase Issue Type: Sub-task Components: Filters Reporter: Josh Elser Assignee: Josh Elser Fix For: 1.5.0, 1.4.10 See parent. Modify and apply to 1.x lines. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22214) [2.x] Backport missing filter improvements
[ https://issues.apache.org/jira/browse/HBASE-22214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved HBASE-22214. Resolution: Fixed > [2.x] Backport missing filter improvements > -- > > Key: HBASE-22214 > URL: https://issues.apache.org/jira/browse/HBASE-22214 > Project: HBase > Issue Type: Bug > Components: Filters >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.0.6, 2.1.5 > > > HBASE-19008 and HBASE-21129 were never backported beyond branch-2. I can't > find any reason that this was not done. Despite these being public-tagged > classes, no incompatible changes were added. > The lack of these changes prevents HBASE-22144 from being backported cleanly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: The GitHub PR integration basically works
The HBase QA, didn't list out a consolidated listed of failed UTs in a recently submitted patch of mine. Just wanted to put this out here. On Sat, Apr 6, 2019 at 4:37 AM 张铎(Duo Zhang) wrote: > Oh shit, the default operation for merging is create a merge commit... > > Sorry about that, I should select the rebase and merge. > > And is it possible to disable the 'create a merge commit' option, just like > the 'squash and merge'? > > Thanks. > > 张铎(Duo Zhang) 于2019年4月6日周六 下午7:33写道： > > > Seems GitHub will add a merge commit when clicking the merge button on > the > > PR... > > > > Let me check... > > > > Misty Linville 于2019年4月6日周六 下午1:30写道： > > > >> Ah, I see. Thanks for explaining. That makes more sense! > >> > >> On Fri, Apr 5, 2019 at 10:29 PM 张铎(Duo Zhang) > >> wrote: > >> > >> > Oh, Zheng Hu is a committer so he has the permission to merge... What > I > >> > said is that, he should approve first before merging... > >> > > >> > Misty Linville 于2019年4月6日周六 下午1:19写道： > >> > > >> > > Yes, but you were doing the merge, unless they were a committer. I > >> > > understood (perhaps incorrectly) Duo was describing a situation > where > >> a > >> > > chang was merged by someone who shouldn’t have been able to do so > >> > > otherwise. > >> > > > >> > > On Fri, Apr 5, 2019 at 9:52 PM Sean Busbey > wrote: > >> > > > >> > > > > >> > > > This seems like a difference in ease compared to jira and not > >> > > > something wildly different. There have certainly been times where > a > >> > > > committer posted a patch to jira for review and I merged it at a > >> part > >> > > > of giving my +1. > >> > > > > >> > > > We should make sure things default to squash-and-rebase instead of > >> > > > merge for PRs in the UI, but I think we did that already. > >> > > > > >> > > > On Fri, Apr 5, 2019 at 10:54 PM 张铎(Duo Zhang) < > >> palomino...@gmail.com> > >> > > > wrote: > >> > > > > > >> > > > > IIRC we have filed an infra ticket to disable several operations > >> > > related > >> > > > to > >> > > > > PR, and for merging, I think we should only allow committers to > >> merge > >> > > > PRs. > >> > > > > > >> > > > > Misty Linville 于2019年4月6日周六 上午10:11写道： > >> > > > > > >> > > > > > Can we protect the GitHub branches from direct merges? That’s > a > >> > > > repo-level > >> > > > > > setting and we may not be able to change it. It seems > >> potentially > >> > > > dangerous > >> > > > > > for people to be able to merge their own changes especially if > >> it > >> > > only > >> > > > > > takes one successful reviewer. Other communities use > mechanisms > >> > like > >> > > > Prow > >> > > > > > [1] for this kind of thing. I imagine it requires some > >> > infrastructure > >> > > > > > though. > >> > > > > > > >> > > > > > [1] https://github.com/kubernetes/test-infra/tree/master/prow > >> > > > > > > >> > > > > > On Fri, Apr 5, 2019 at 7:04 PM 张铎(Duo Zhang) < > >> > palomino...@gmail.com> > >> > > > > > wrote: > >> > > > > > > >> > > > > > > Yes, at least there should be a relevant JIRA issue. > >> > > > > > > > >> > > > > > > And on the retesting, we need to find a way to re-trigger > the > >> > > > webhook. > >> > > > > > But > >> > > > > > > anyway, we can fall back to use the old pre commit way, just > >> > > > checkout the > >> > > > > > > branch and make a patch and upload it to the jira issue... > >> > > > > > > > >> > > > > > > I'm trying to make use of GitHub in the recent works. And > >> > > yesterday, > >> > > > I > >> > > > > > > added Zheng Hu as a reviewer for the addendum of > HBASE-22152, > >> and > >> > > he > >> > > > > > posted > >> > > > > > > a LGTM and then just merged the PR... In fact I just want > him > >> to > >> > > > approve > >> > > > > > > the PR, this is the correct way to '+1' on GitHub. So I > think > >> we > >> > > > need to > >> > > > > > > write something done in the tell committers how to make use > of > >> > the > >> > > > GitHub > >> > > > > > > PR... > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > Sean Busbey 于2019年4月6日周六 上午9:43写道： > >> > > > > > > > >> > > > > > > > Excellent to see Duo! > >> > > > > > > > > >> > > > > > > > Do we have any guidelines for committers in the ref > guide? I > >> > > think > >> > > > we > >> > > > > > had > >> > > > > > > > previously discussed calling out that they should make > sure > >> > > > there's a > >> > > > > > > JIRA > >> > > > > > > > for anything merged? > >> > > > > > > > > >> > > > > > > > Does retesting work from the github UI or is it like > before > >> > where > >> > > > one > >> > > > > > > > resubmits the jenkins job? > >> > > > > > > > > >> > > > > > > > On 2019/04/04 06:15:39, 张铎(Duo Zhang) < > >> palomino...@gmail.com> > >> > > > wrote: > >> > > > > > > > > Please see here > >> > > > > > > > > > >> > > > > > > > > https://github.com/apache/hbase/pull/110 > >> > > > > > > > > > >> > > > > > > > > Still need to polish the jenkinsfile so we can keep the > >> same > >> > > > > > experience > >> > > > > > > > > with
[jira] [Created] (HBASE-22214) [2.x] Backport missing filter improvements
Josh Elser created HBASE-22214: -- Summary: [2.x] Backport missing filter improvements Key: HBASE-22214 URL: https://issues.apache.org/jira/browse/HBASE-22214 Project: HBase Issue Type: Bug Components: Filters Reporter: Josh Elser Assignee: Josh Elser Fix For: 1.5.0, 1.4.10, 2.0.6, 2.1.5 HBASE-19008 and HBASE-21129 were never backported beyond branch-2. I can't find any reason that this was not done. Despite these being public-tagged classes, no incompatible changes were added. The lack of these changes prevents HBASE-22144 from being backported cleanly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] First release candidate for HBASE 2.2.0 is available
Ok. Let me pull the last 2.2 branch, build that, deploy and retry... I might be able to let you know tomorrow morning. Le jeu. 11 avr. 2019 à 14:19, Sean Busbey a écrit : > Maybe worth trying out the HEAD of branch-2.2 prior to the next RC? That > way if you hit a blocker like your current ones we can fix them before RC > generation happens. > > On Thu, Apr 11, 2019, 13:08 Jean-Marc Spaggiari > wrote: > > > Thanks Guanghao. > > > > I'm having consistant issues with RC0. I had some issues with a table > this > > morning. Dropped it, recreated it, made it clean, restart my application, > > and I have again some inconsistencies on the regions. There is something > > wrong somewhere. I'm not able to identify where, but I can easily get my > > tables corrupted by running my client application. So I'm eager to try > the > > next RC to see if it helps. > > > > JMS > > > > JMS > > > > Le jeu. 11 avr. 2019 à 13:33, Guanghao Zhang a > écrit > > : > > > > > Sorry for late... I am testing the ITBLL for 2.2.0 and it passed > > > yesterday. See https://issues.apache.org/jira/browse/HBASE-21886. I > > will > > > generate RC1 later. Thanks. > > > > > > Jean-Marc Spaggiari 于2019年4月11日周四 下午10:52写道： > > > > > > > Hi all, > > > > > > > > Any chance to get an updated RC soon? > > > > > > > > JM > > > > > > > > Le mar. 12 mars 2019 à 12:23, Sean Busbey a > écrit > > : > > > > > > > > > quick follow-up here. the full version info is in fact missing the > > > > > revision: > > > > > > > > > > hbase-2.2.0 busbey$ ./bin/hbase version > > > > > HBase 2.2.0 > > > > > Source code repository > > > > > git://hao-OptiPlex-7050/home/hao/open_source/hbase revision= > > > > > Compiled by hao on 2019年 03月 07日 星期四 14:05:34 CST > > > > > From source with checksum 783fee467bb1b28666f0d904437862c4 > > > > > > > > > > I think this is the issue stack ran into on HBASE-21935/HBASE-21999 > > > > > where HBASE-20764 introduced a git cli option that issn't supported > > on > > > > > older versions of git. > > > > > > > > > > Guanghao for the next RC would it be possible to update your local > > git > > > > > version? > > > > > > > > > > On Tue, Mar 12, 2019 at 9:37 AM Sean Busbey > > wrote: > > > > > > > > > > > > locale of the build is up to the RM (this is why, for example, > the > > > 2.1 > > > > > > release line javadocs have chinese for the boilerplate text[1]) > > > > > > > > > > > > however, it does look like that shell output might be missing the > > > > > > build revision information from git or we might not be properly > > > > > > parsing the output from git when a non-english locale is used. > > > > > > > > > > > > [1]: http://hbase.apache.org/2.1/apidocs/index.html > > > > > > > > > > > > > > > > > > On Tue, Mar 12, 2019 at 8:54 AM Jean-Marc Spaggiari > > > > > > wrote: > > > > > > > > > > > > > > Also, in the shell,it displays Asian texte: > > > > > > > > > > > > > > "Version 2.2.0, r, 2019年 03月 07日 星期四 14:05:34 CST" > > > > > > > > > > > > > > Not sure if that's we want. > > > > > > > > > > > > > > JMS > > > > > > > > > > > > > > Le lun. 11 mars 2019 à 21:53, Guanghao Zhang < > zghao...@gmail.com > > > > > > a > > > > > écrit : > > > > > > > > > > > > > > > Let me start a new RC1. HBASE-21970 should be included and > > need a > > > > > release > > > > > > > > note. > > > > > > > > > > > > > > > > Sean Busbey 于2019年3月12日周二 上午8:35写道： > > > > > > > > > > > > > > > > > I'm -1 on RC0 as it is. > > > > > > > > > > > > > > > > > > The current release notes don't include any call out about > > the > > > > > upgrade > > > > > > > > > steps needed. Since we don't usually have minor-version > > > specific > > > > > > > > > upgrade steps and especially since there are things folks > > need > > > to > > > > > do > > > > > > > > > before installing 2.2.0, it's important that they be front > > and > > > > > center. > > > > > > > > > Possibly that should mean a link to the ref guide section > > from > > > > the > > > > > RC > > > > > > > > > instructions and eventual announcement. > > > > > > > > > > > > > > > > > > I think either HBASE-21075 needs to have 2.2.0 included in > > its > > > > fix > > > > > > > > > version or the release note from that issue needs to be > > copied > > > > > over to > > > > > > > > > HBASE-21970 and it needs to have 2.2.0 included in its fix > > > > > version(s). > > > > > > > > > In either case the release notes should link to the ref > guide > > > > > section. > > > > > > > > > > > > > > > > > > On Thu, Mar 7, 2019 at 3:44 AM Guanghao Zhang < > > > > zghao...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Please vote on this release candidate (RC) for Apache > HBase > > > > > 2.2.0. > > > > > > > > > > This is the first release of the branch-2.2 line. > > > > > > > > > > > > > > > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > > > > > > > > > > > > > > > [ ] +1 Release this package as Apache HBase 2.2.0 > > > > > > > > > > [ ] -1 Do not release this package
[jira] [Resolved] (HBASE-22213) Create a Java based BulkLoadPartitioner
[ https://issues.apache.org/jira/browse/HBASE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari resolved HBASE-22213. - Resolution: Later > Create a Java based BulkLoadPartitioner > --- > > Key: HBASE-22213 > URL: https://issues.apache.org/jira/browse/HBASE-22213 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.1.4 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > > We have a scala based partitionner, but not all projects are build in Scala. > We should provide a Java based version of it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Branch-1 releasing paused; HBase community: please help clean up your test code
Pardon, that git ref I failed to include below is commit 677ed9ef89 (HEAD -> branch-1) On Thu, Apr 11, 2019 at 12:14 PM Andrew Purtell wrote: > Thanks for clarifying that this is not a foundation wide policy. It is my > personal policy to cancel a vote whenever there is a veto, if the reason is > sound and technical and, presumably, repeatable, at least in the voter's > environment(s). > > Circling back to the main topic, a unit test run of latest branch-1 (git > ref) completed successfully on my dev host. Based on this result, I would > release it. I'll include details on my environment below. > > It is my theory that the reason some environments observe flaky unit test > failures and others don't is there are occasional unexpected interactions > between various units when surefire executes them concurrently, and the > order in which unit tests are run is dependent on how the underlying OS's > readdir() call orders directory entries, and this will vary from host to > host and even from checkout to checkout. > > My dev host details > > apurtell$ uname -a > Darwin HOSTNAME 17.7.0 Darwin Kernel Version 17.7.0: Thu Dec 20 21:47:19 > PST 2018; root:xnu-4570.71.22~1/RELEASE_X86_64 x86_64 > > apurtell$ mvn -v > Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; > 2018-10-24T11:41:47-07:00) > Maven home: /usr/local/Cellar/maven/3.6.0/libexec > Java version: 1.8.0_172, vendor: Azul Systems, Inc., runtime: > /Users/apurtell/tools/Darwin/jdk/openjdk1.8.0_172_x64/jre > Default locale: en_US, platform encoding: UTF-8 > OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac" > > > On Thu, Apr 11, 2019 at 11:11 AM Sean Busbey wrote: > >> Just as a point of clarification, -1 votes on releases aren't vetos. ASF >> policy requires release votes to be majority. >> >> As release manager it's your perogative to cancel a vote for whatever >> reason, but I don't want future RMs thinking they have to fail the vote >> once there's a -1. >> >> On Thu, Apr 11, 2019, 12:18 Andrew Purtell >> wrote: >> >> > I put back "1.6.0" for TinyLFU backport to branch-1, which I don't think >> > can happen in the near term because it depends on improvements to >> precommit >> > being in place first. >> > >> > >> > On Thu, Apr 11, 2019 at 9:37 AM Andrew Purtell < >> andrew.purt...@gmail.com> >> > wrote: >> > >> > > A helpful view for tracking 1.5.0 issues is >> > > https://issues.apache.org/jira/projects/HBASE/versions/12340316 . >> > > >> > > I deleted fixversions "1.5.1" and "1.6.0" and had JIRA rewrite them >> back >> > > to "1.5.0" so you won't be confused by JIRA thinking 1.5.0 is in >> released >> > > state. All open and pending branch-1 work has the "1.5.0" fix version >> > > again, until we try again for a release at some future time. >> > > >> > > >> > > On Thu, Apr 11, 2019 at 9:10 AM Andrew Purtell < >> andrew.purt...@gmail.com >> > > >> > > wrote: >> > > >> > >> I have tried to release four 1.5.0 candidates from head of branch-1. >> > >> >> > >> Some veto votes were cast for compatibility issues. This was fine. >> > >> >> > >> Others are for unit test results that do not reproduce locally for >> me. I >> > >> do not have the ability or bandwidth to fix tests which do not fail >> > >> reliably for me. So let me appeal to the community. If you find a >> > >> repeatable test failure on branch-1 please file a JIRA and fix it. >> > >> >> > >> As things stand now I am pausing any attempts to make more 1.5.0 >> > release >> > >> candidates until the community steps forward to clean up its code. >> As an >> > >> alternative I may start aggressively disabling unit tests which do >> not >> > fail >> > >> for me but have been reported in on candidate vote vetoes. Otherwise >> it >> > is >> > >> impossible to make forward progress. >> > >> >> > >> Thank you for your attention. >> >
Re: Branch-1 releasing paused; HBase community: please help clean up your test code
Thanks for clarifying that this is not a foundation wide policy. It is my personal policy to cancel a vote whenever there is a veto, if the reason is sound and technical and, presumably, repeatable, at least in the voter's environment(s). Circling back to the main topic, a unit test run of latest branch-1 (git ref) completed successfully on my dev host. Based on this result, I would release it. I'll include details on my environment below. It is my theory that the reason some environments observe flaky unit test failures and others don't is there are occasional unexpected interactions between various units when surefire executes them concurrently, and the order in which unit tests are run is dependent on how the underlying OS's readdir() call orders directory entries, and this will vary from host to host and even from checkout to checkout. My dev host details apurtell$ uname -a Darwin HOSTNAME 17.7.0 Darwin Kernel Version 17.7.0: Thu Dec 20 21:47:19 PST 2018; root:xnu-4570.71.22~1/RELEASE_X86_64 x86_64 apurtell$ mvn -v Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T11:41:47-07:00) Maven home: /usr/local/Cellar/maven/3.6.0/libexec Java version: 1.8.0_172, vendor: Azul Systems, Inc., runtime: /Users/apurtell/tools/Darwin/jdk/openjdk1.8.0_172_x64/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac" On Thu, Apr 11, 2019 at 11:11 AM Sean Busbey wrote: > Just as a point of clarification, -1 votes on releases aren't vetos. ASF > policy requires release votes to be majority. > > As release manager it's your perogative to cancel a vote for whatever > reason, but I don't want future RMs thinking they have to fail the vote > once there's a -1. > > On Thu, Apr 11, 2019, 12:18 Andrew Purtell > wrote: > > > I put back "1.6.0" for TinyLFU backport to branch-1, which I don't think > > can happen in the near term because it depends on improvements to > precommit > > being in place first. > > > > > > On Thu, Apr 11, 2019 at 9:37 AM Andrew Purtell > > > wrote: > > > > > A helpful view for tracking 1.5.0 issues is > > > https://issues.apache.org/jira/projects/HBASE/versions/12340316 . > > > > > > I deleted fixversions "1.5.1" and "1.6.0" and had JIRA rewrite them > back > > > to "1.5.0" so you won't be confused by JIRA thinking 1.5.0 is in > released > > > state. All open and pending branch-1 work has the "1.5.0" fix version > > > again, until we try again for a release at some future time. > > > > > > > > > On Thu, Apr 11, 2019 at 9:10 AM Andrew Purtell < > andrew.purt...@gmail.com > > > > > > wrote: > > > > > >> I have tried to release four 1.5.0 candidates from head of branch-1. > > >> > > >> Some veto votes were cast for compatibility issues. This was fine. > > >> > > >> Others are for unit test results that do not reproduce locally for > me. I > > >> do not have the ability or bandwidth to fix tests which do not fail > > >> reliably for me. So let me appeal to the community. If you find a > > >> repeatable test failure on branch-1 please file a JIRA and fix it. > > >> > > >> As things stand now I am pausing any attempts to make more 1.5.0 > > release > > >> candidates until the community steps forward to clean up its code. As > an > > >> alternative I may start aggressively disabling unit tests which do not > > fail > > >> for me but have been reported in on candidate vote vetoes. Otherwise > it > > is > > >> impossible to make forward progress. > > >> > > >> Thank you for your attention. > > >> > > >> > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > >- A23, Crosstalk > > > > > > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-22213) Create a Java based BulkLoadPartitioner
Jean-Marc Spaggiari created HBASE-22213: --- Summary: Create a Java based BulkLoadPartitioner Key: HBASE-22213 URL: https://issues.apache.org/jira/browse/HBASE-22213 Project: HBase Issue Type: New Feature Affects Versions: 2.1.4 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari We have a scala based partitionner, but not all projects are build in Scala. We should provide a Java based version of it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] First release candidate for HBASE 2.2.0 is available
Maybe worth trying out the HEAD of branch-2.2 prior to the next RC? That way if you hit a blocker like your current ones we can fix them before RC generation happens. On Thu, Apr 11, 2019, 13:08 Jean-Marc Spaggiari wrote: > Thanks Guanghao. > > I'm having consistant issues with RC0. I had some issues with a table this > morning. Dropped it, recreated it, made it clean, restart my application, > and I have again some inconsistencies on the regions. There is something > wrong somewhere. I'm not able to identify where, but I can easily get my > tables corrupted by running my client application. So I'm eager to try the > next RC to see if it helps. > > JMS > > JMS > > Le jeu. 11 avr. 2019 à 13:33, Guanghao Zhang a écrit > : > > > Sorry for late... I am testing the ITBLL for 2.2.0 and it passed > > yesterday. See https://issues.apache.org/jira/browse/HBASE-21886. I > will > > generate RC1 later. Thanks. > > > > Jean-Marc Spaggiari 于2019年4月11日周四 下午10:52写道： > > > > > Hi all, > > > > > > Any chance to get an updated RC soon? > > > > > > JM > > > > > > Le mar. 12 mars 2019 à 12:23, Sean Busbey a écrit > : > > > > > > > quick follow-up here. the full version info is in fact missing the > > > > revision: > > > > > > > > hbase-2.2.0 busbey$ ./bin/hbase version > > > > HBase 2.2.0 > > > > Source code repository > > > > git://hao-OptiPlex-7050/home/hao/open_source/hbase revision= > > > > Compiled by hao on 2019年 03月 07日 星期四 14:05:34 CST > > > > From source with checksum 783fee467bb1b28666f0d904437862c4 > > > > > > > > I think this is the issue stack ran into on HBASE-21935/HBASE-21999 > > > > where HBASE-20764 introduced a git cli option that issn't supported > on > > > > older versions of git. > > > > > > > > Guanghao for the next RC would it be possible to update your local > git > > > > version? > > > > > > > > On Tue, Mar 12, 2019 at 9:37 AM Sean Busbey > wrote: > > > > > > > > > > locale of the build is up to the RM (this is why, for example, the > > 2.1 > > > > > release line javadocs have chinese for the boilerplate text[1]) > > > > > > > > > > however, it does look like that shell output might be missing the > > > > > build revision information from git or we might not be properly > > > > > parsing the output from git when a non-english locale is used. > > > > > > > > > > [1]: http://hbase.apache.org/2.1/apidocs/index.html > > > > > > > > > > > > > > > On Tue, Mar 12, 2019 at 8:54 AM Jean-Marc Spaggiari > > > > > wrote: > > > > > > > > > > > > Also, in the shell,it displays Asian texte: > > > > > > > > > > > > "Version 2.2.0, r, 2019年 03月 07日 星期四 14:05:34 CST" > > > > > > > > > > > > Not sure if that's we want. > > > > > > > > > > > > JMS > > > > > > > > > > > > Le lun. 11 mars 2019 à 21:53, Guanghao Zhang > > > a > > > > écrit : > > > > > > > > > > > > > Let me start a new RC1. HBASE-21970 should be included and > need a > > > > release > > > > > > > note. > > > > > > > > > > > > > > Sean Busbey 于2019年3月12日周二 上午8:35写道： > > > > > > > > > > > > > > > I'm -1 on RC0 as it is. > > > > > > > > > > > > > > > > The current release notes don't include any call out about > the > > > > upgrade > > > > > > > > steps needed. Since we don't usually have minor-version > > specific > > > > > > > > upgrade steps and especially since there are things folks > need > > to > > > > do > > > > > > > > before installing 2.2.0, it's important that they be front > and > > > > center. > > > > > > > > Possibly that should mean a link to the ref guide section > from > > > the > > > > RC > > > > > > > > instructions and eventual announcement. > > > > > > > > > > > > > > > > I think either HBASE-21075 needs to have 2.2.0 included in > its > > > fix > > > > > > > > version or the release note from that issue needs to be > copied > > > > over to > > > > > > > > HBASE-21970 and it needs to have 2.2.0 included in its fix > > > > version(s). > > > > > > > > In either case the release notes should link to the ref guide > > > > section. > > > > > > > > > > > > > > > > On Thu, Mar 7, 2019 at 3:44 AM Guanghao Zhang < > > > zghao...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > Please vote on this release candidate (RC) for Apache HBase > > > > 2.2.0. > > > > > > > > > This is the first release of the branch-2.2 line. > > > > > > > > > > > > > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > > > > > > > > > > > > > [ ] +1 Release this package as Apache HBase 2.2.0 > > > > > > > > > [ ] -1 Do not release this package because ... > > > > > > > > > > > > > > > > > > The tag to be voted on is 2.2.0-RC0 (commit > > > > > > > > > 4ab2dc20f15e9b59477de4bd971c367f3ce342cb): > > > > > > > > > > > > > > > > > > https://github.com/apache/hbase/tree/2.2.0-RC0 > > > > > > > > > > > > > > > > > > The release files, including signatures, digests, etc. can > be > > > > found at: > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC0/ > > > > > > > > > > > > > > > > > > Maven
Re: Branch-1 releasing paused; HBase community: please help clean up your test code
Just as a point of clarification, -1 votes on releases aren't vetos. ASF policy requires release votes to be majority. As release manager it's your perogative to cancel a vote for whatever reason, but I don't want future RMs thinking they have to fail the vote once there's a -1. On Thu, Apr 11, 2019, 12:18 Andrew Purtell wrote: > I put back "1.6.0" for TinyLFU backport to branch-1, which I don't think > can happen in the near term because it depends on improvements to precommit > being in place first. > > > On Thu, Apr 11, 2019 at 9:37 AM Andrew Purtell > wrote: > > > A helpful view for tracking 1.5.0 issues is > > https://issues.apache.org/jira/projects/HBASE/versions/12340316 . > > > > I deleted fixversions "1.5.1" and "1.6.0" and had JIRA rewrite them back > > to "1.5.0" so you won't be confused by JIRA thinking 1.5.0 is in released > > state. All open and pending branch-1 work has the "1.5.0" fix version > > again, until we try again for a release at some future time. > > > > > > On Thu, Apr 11, 2019 at 9:10 AM Andrew Purtell > > > wrote: > > > >> I have tried to release four 1.5.0 candidates from head of branch-1. > >> > >> Some veto votes were cast for compatibility issues. This was fine. > >> > >> Others are for unit test results that do not reproduce locally for me. I > >> do not have the ability or bandwidth to fix tests which do not fail > >> reliably for me. So let me appeal to the community. If you find a > >> repeatable test failure on branch-1 please file a JIRA and fix it. > >> > >> As things stand now I am pausing any attempts to make more 1.5.0 > release > >> candidates until the community steps forward to clean up its code. As an > >> alternative I may start aggressively disabling unit tests which do not > fail > >> for me but have been reported in on candidate vote vetoes. Otherwise it > is > >> impossible to make forward progress. > >> > >> Thank you for your attention. > >> > >> > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
Re: [VOTE] First release candidate for HBASE 2.2.0 is available
Thanks Guanghao. I'm having consistant issues with RC0. I had some issues with a table this morning. Dropped it, recreated it, made it clean, restart my application, and I have again some inconsistencies on the regions. There is something wrong somewhere. I'm not able to identify where, but I can easily get my tables corrupted by running my client application. So I'm eager to try the next RC to see if it helps. JMS JMS Le jeu. 11 avr. 2019 à 13:33, Guanghao Zhang a écrit : > Sorry for late... I am testing the ITBLL for 2.2.0 and it passed > yesterday. See https://issues.apache.org/jira/browse/HBASE-21886. I will > generate RC1 later. Thanks. > > Jean-Marc Spaggiari 于2019年4月11日周四 下午10:52写道： > > > Hi all, > > > > Any chance to get an updated RC soon? > > > > JM > > > > Le mar. 12 mars 2019 à 12:23, Sean Busbey a écrit : > > > > > quick follow-up here. the full version info is in fact missing the > > > revision: > > > > > > hbase-2.2.0 busbey$ ./bin/hbase version > > > HBase 2.2.0 > > > Source code repository > > > git://hao-OptiPlex-7050/home/hao/open_source/hbase revision= > > > Compiled by hao on 2019年 03月 07日 星期四 14:05:34 CST > > > From source with checksum 783fee467bb1b28666f0d904437862c4 > > > > > > I think this is the issue stack ran into on HBASE-21935/HBASE-21999 > > > where HBASE-20764 introduced a git cli option that issn't supported on > > > older versions of git. > > > > > > Guanghao for the next RC would it be possible to update your local git > > > version? > > > > > > On Tue, Mar 12, 2019 at 9:37 AM Sean Busbey wrote: > > > > > > > > locale of the build is up to the RM (this is why, for example, the > 2.1 > > > > release line javadocs have chinese for the boilerplate text[1]) > > > > > > > > however, it does look like that shell output might be missing the > > > > build revision information from git or we might not be properly > > > > parsing the output from git when a non-english locale is used. > > > > > > > > [1]: http://hbase.apache.org/2.1/apidocs/index.html > > > > > > > > > > > > On Tue, Mar 12, 2019 at 8:54 AM Jean-Marc Spaggiari > > > > wrote: > > > > > > > > > > Also, in the shell,it displays Asian texte: > > > > > > > > > > "Version 2.2.0, r, 2019年 03月 07日 星期四 14:05:34 CST" > > > > > > > > > > Not sure if that's we want. > > > > > > > > > > JMS > > > > > > > > > > Le lun. 11 mars 2019 à 21:53, Guanghao Zhang > a > > > écrit : > > > > > > > > > > > Let me start a new RC1. HBASE-21970 should be included and need a > > > release > > > > > > note. > > > > > > > > > > > > Sean Busbey 于2019年3月12日周二 上午8:35写道： > > > > > > > > > > > > > I'm -1 on RC0 as it is. > > > > > > > > > > > > > > The current release notes don't include any call out about the > > > upgrade > > > > > > > steps needed. Since we don't usually have minor-version > specific > > > > > > > upgrade steps and especially since there are things folks need > to > > > do > > > > > > > before installing 2.2.0, it's important that they be front and > > > center. > > > > > > > Possibly that should mean a link to the ref guide section from > > the > > > RC > > > > > > > instructions and eventual announcement. > > > > > > > > > > > > > > I think either HBASE-21075 needs to have 2.2.0 included in its > > fix > > > > > > > version or the release note from that issue needs to be copied > > > over to > > > > > > > HBASE-21970 and it needs to have 2.2.0 included in its fix > > > version(s). > > > > > > > In either case the release notes should link to the ref guide > > > section. > > > > > > > > > > > > > > On Thu, Mar 7, 2019 at 3:44 AM Guanghao Zhang < > > zghao...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > Please vote on this release candidate (RC) for Apache HBase > > > 2.2.0. > > > > > > > > This is the first release of the branch-2.2 line. > > > > > > > > > > > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > > > > > > > > > > > [ ] +1 Release this package as Apache HBase 2.2.0 > > > > > > > > [ ] -1 Do not release this package because ... > > > > > > > > > > > > > > > > The tag to be voted on is 2.2.0-RC0 (commit > > > > > > > > 4ab2dc20f15e9b59477de4bd971c367f3ce342cb): > > > > > > > > > > > > > > > > https://github.com/apache/hbase/tree/2.2.0-RC0 > > > > > > > > > > > > > > > > The release files, including signatures, digests, etc. can be > > > found at: > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC0/ > > > > > > > > > > > > > > > > Maven artifacts are available in a staging repository at: > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1286 > > > > > > > > > > > > > > > > Signatures used for HBase RCs can be found in this file: > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > > > > > > > > > > > > > The list of bug fixes going into 2.2.0 can be found in > included > > > > > > > > CHANGES.md and RELEASENOTES.md available here: > > > > > >
Re: request invite to join slack channel
Done, please check On Thu, 11 Apr 2019, 23:11 Da Zhou, wrote: > Dear HBase Community, > > I saw the channel "https://apache-hbase.slack.com/; shared on the HBase > Book. May I join the HBase Slack channel for easier communication? > > Thanks, > Da >
request invite to join slack channel
Dear HBase Community, I saw the channel "https://apache-hbase.slack.com/; shared on the HBase Book. May I join the HBase Slack channel for easier communication? Thanks, Da
Re: [VOTE] First release candidate for HBASE 2.2.0 is available
Sorry for late... I am testing the ITBLL for 2.2.0 and it passed yesterday. See https://issues.apache.org/jira/browse/HBASE-21886. I will generate RC1 later. Thanks. Jean-Marc Spaggiari 于2019年4月11日周四 下午10:52写道： > Hi all, > > Any chance to get an updated RC soon? > > JM > > Le mar. 12 mars 2019 à 12:23, Sean Busbey a écrit : > > > quick follow-up here. the full version info is in fact missing the > > revision: > > > > hbase-2.2.0 busbey$ ./bin/hbase version > > HBase 2.2.0 > > Source code repository > > git://hao-OptiPlex-7050/home/hao/open_source/hbase revision= > > Compiled by hao on 2019年 03月 07日 星期四 14:05:34 CST > > From source with checksum 783fee467bb1b28666f0d904437862c4 > > > > I think this is the issue stack ran into on HBASE-21935/HBASE-21999 > > where HBASE-20764 introduced a git cli option that issn't supported on > > older versions of git. > > > > Guanghao for the next RC would it be possible to update your local git > > version? > > > > On Tue, Mar 12, 2019 at 9:37 AM Sean Busbey wrote: > > > > > > locale of the build is up to the RM (this is why, for example, the 2.1 > > > release line javadocs have chinese for the boilerplate text[1]) > > > > > > however, it does look like that shell output might be missing the > > > build revision information from git or we might not be properly > > > parsing the output from git when a non-english locale is used. > > > > > > [1]: http://hbase.apache.org/2.1/apidocs/index.html > > > > > > > > > On Tue, Mar 12, 2019 at 8:54 AM Jean-Marc Spaggiari > > > wrote: > > > > > > > > Also, in the shell,it displays Asian texte: > > > > > > > > "Version 2.2.0, r, 2019年 03月 07日 星期四 14:05:34 CST" > > > > > > > > Not sure if that's we want. > > > > > > > > JMS > > > > > > > > Le lun. 11 mars 2019 à 21:53, Guanghao Zhang a > > écrit : > > > > > > > > > Let me start a new RC1. HBASE-21970 should be included and need a > > release > > > > > note. > > > > > > > > > > Sean Busbey 于2019年3月12日周二 上午8:35写道： > > > > > > > > > > > I'm -1 on RC0 as it is. > > > > > > > > > > > > The current release notes don't include any call out about the > > upgrade > > > > > > steps needed. Since we don't usually have minor-version specific > > > > > > upgrade steps and especially since there are things folks need to > > do > > > > > > before installing 2.2.0, it's important that they be front and > > center. > > > > > > Possibly that should mean a link to the ref guide section from > the > > RC > > > > > > instructions and eventual announcement. > > > > > > > > > > > > I think either HBASE-21075 needs to have 2.2.0 included in its > fix > > > > > > version or the release note from that issue needs to be copied > > over to > > > > > > HBASE-21970 and it needs to have 2.2.0 included in its fix > > version(s). > > > > > > In either case the release notes should link to the ref guide > > section. > > > > > > > > > > > > On Thu, Mar 7, 2019 at 3:44 AM Guanghao Zhang < > zghao...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > Please vote on this release candidate (RC) for Apache HBase > > 2.2.0. > > > > > > > This is the first release of the branch-2.2 line. > > > > > > > > > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > > > > > > > > > [ ] +1 Release this package as Apache HBase 2.2.0 > > > > > > > [ ] -1 Do not release this package because ... > > > > > > > > > > > > > > The tag to be voted on is 2.2.0-RC0 (commit > > > > > > > 4ab2dc20f15e9b59477de4bd971c367f3ce342cb): > > > > > > > > > > > > > > https://github.com/apache/hbase/tree/2.2.0-RC0 > > > > > > > > > > > > > > The release files, including signatures, digests, etc. can be > > found at: > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC0/ > > > > > > > > > > > > > > Maven artifacts are available in a staging repository at: > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1286 > > > > > > > > > > > > > > Signatures used for HBase RCs can be found in this file: > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > > > > > > > > > > > The list of bug fixes going into 2.2.0 can be found in included > > > > > > > CHANGES.md and RELEASENOTES.md available here: > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC0/CHANGES.md > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC0/RELEASENOTES.md > > > > > > > > > > > > > > To learn more about Apache HBase, please see > > http://hbase.apache.org/ > > > > > > > > > > > > > > Thanks, > > > > > > > Guanghao Zhang > > > > > > > > > > > > > >
Re: Contributor Permission Request
Done. Add to HBase jira CONTRIBUTORS, too. Manjeet Singh 于2019年4月12日周五 上午12:56写道： > Hi > > Request you to please add me as well for contribute to Apache HBase. > My id is manjeet.chand...@gmail.com > > Thanks > Manjeet singh > > On Thu, 11 Apr 2019, 19:52 Guanghao Zhang, wrote: > > > Done. Add to HBase jira CONTRIBUTORS. Thanks. > > > > fei wen 于2019年4月11日周四 下午7:26写道： > > > > > Hi, > > > I want to contribute to Apache HBase. > > > Would you please give me the contributor permission? > > > My JIRA ID is wen.feiyi. > > > > > >
Re: Branch-1 releasing paused; HBase community: please help clean up your test code
I put back "1.6.0" for TinyLFU backport to branch-1, which I don't think can happen in the near term because it depends on improvements to precommit being in place first. On Thu, Apr 11, 2019 at 9:37 AM Andrew Purtell wrote: > A helpful view for tracking 1.5.0 issues is > https://issues.apache.org/jira/projects/HBASE/versions/12340316 . > > I deleted fixversions "1.5.1" and "1.6.0" and had JIRA rewrite them back > to "1.5.0" so you won't be confused by JIRA thinking 1.5.0 is in released > state. All open and pending branch-1 work has the "1.5.0" fix version > again, until we try again for a release at some future time. > > > On Thu, Apr 11, 2019 at 9:10 AM Andrew Purtell > wrote: > >> I have tried to release four 1.5.0 candidates from head of branch-1. >> >> Some veto votes were cast for compatibility issues. This was fine. >> >> Others are for unit test results that do not reproduce locally for me. I >> do not have the ability or bandwidth to fix tests which do not fail >> reliably for me. So let me appeal to the community. If you find a >> repeatable test failure on branch-1 please file a JIRA and fix it. >> >> As things stand now I am pausing any attempts to make more 1.5.0 release >> candidates until the community steps forward to clean up its code. As an >> alternative I may start aggressively disabling unit tests which do not fail >> for me but have been reported in on candidate vote vetoes. Otherwise it is >> impossible to make forward progress. >> >> Thank you for your attention. >> >> > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Resolved] (HBASE-21758) Update hadoop-three.version on branch-1 to 3.0.3
[
https://issues.apache.org/jira/browse/HBASE-21758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-21758.
Resolution: Won't Fix
Assignee: (was: Andrew Purtell)
Fix Version/s: (was: 1.5.0)
> Update hadoop-three.version on branch-1 to 3.0.3
>
>
> Key: HBASE-21758
> URL: https://issues.apache.org/jira/browse/HBASE-21758
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Priority: Trivial
> Attachments: HBASE-21758-branch-1.patch
>
>
> Sync the branch-1 POM with master and branch-2 with respect to the default
> version of {{hadoop-three.version}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Contributor Permission Request
Hi Request you to please add me as well for contribute to Apache HBase. My id is manjeet.chand...@gmail.com Thanks Manjeet singh On Thu, 11 Apr 2019, 19:52 Guanghao Zhang, wrote: > Done. Add to HBase jira CONTRIBUTORS. Thanks. > > fei wen 于2019年4月11日周四 下午7:26写道： > > > Hi, > > I want to contribute to Apache HBase. > > Would you please give me the contributor permission? > > My JIRA ID is wen.feiyi. > > >
[jira] [Resolved] (HBASE-17762) Add logging to HBaseAdmin for user initiated tasks
[ https://issues.apache.org/jira/browse/HBASE-17762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-17762. Resolution: Incomplete Assignee: (was: churro morales) Fix Version/s: (was: 1.5.0) (was: 3.0.0) > Add logging to HBaseAdmin for user initiated tasks > -- > > Key: HBASE-17762 > URL: https://issues.apache.org/jira/browse/HBASE-17762 > Project: HBase > Issue Type: Task >Reporter: churro morales >Priority: Major > Attachments: HBASE-17762.patch, HBASE-17762.v1.patch > > > Things like auditing a forced major compaction are really useful and right > now there is no logging when this is triggered. Other actions may require > logging as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-16594) ROW_INDEX_V2 DBE
[ https://issues.apache.org/jira/browse/HBASE-16594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-16594. Resolution: Incomplete Assignee: (was: binlijin) Fix Version/s: (was: 2.3.0) (was: 1.5.0) (was: 3.0.0) > ROW_INDEX_V2 DBE > > > Key: HBASE-16594 > URL: https://issues.apache.org/jira/browse/HBASE-16594 > Project: HBase > Issue Type: Improvement > Components: Performance >Reporter: binlijin >Priority: Major > Attachments: HBASE-16594-master_v1.patch, HBASE-16594-master_v2.patch > > > See HBASE-16213, ROW_INDEX_V1 DataBlockEncoding. > ROW_INDEX_V1 is the first version which have no storage optimization, > ROW_INDEX_V2 do storage optimization: store every row only once, store column > family only once in a HFileBlock. > ROW_INDEX_V1 is : > /** > * Store cells following every row's start offset, so we can binary search to > a row's cells. > * > * Format: > * flat cells > * integer: number of rows > * integer: row0's offset > * integer: row1's offset > * > * integer: dataSize > * > */ > ROW_INDEX_V2 is : > * row1 qualifier timestamp type value tag > * qualifier timestamp type value tag > * qualifier timestamp type value tag > * row2 qualifier timestamp type value tag > * row3 qualifier timestamp type value tag > * qualifier timestamp type value tag > * > * integer: number of rows > * integer: row0's offset > * integer: row1's offset > * > * column family > * integer: dataSize -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-15677) FailedServerException shouldn't clear MetaCache
[ https://issues.apache.org/jira/browse/HBASE-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-15677. Resolution: Incomplete Assignee: (was: Mikhail Antonov) Fix Version/s: (was: 1.5.0) (was: 3.0.0) > FailedServerException shouldn't clear MetaCache > --- > > Key: HBASE-15677 > URL: https://issues.apache.org/jira/browse/HBASE-15677 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Priority: Major > > Right now FailedServerException clears meta cache. Seems like it's > unnecessary (if we hit that, someone has already gotten some network/remote > error in the first place and invalidated located cache for us), and seems it > could lead to unnecessary drops, as FailedServers cache has default TTL of 2 > seconds, so we can encounter situation like this: > - thread T1 hit network error and cleared the cache, put server in failed > server list > - thread T2 tries to get it's request in and gets FailedServerException > - thread T1 does meta scan to populate the cache > - thread T2 clears the cache after it's got FSE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21013) Backport "read part" of HBASE-18754 to all active 1.x branches
[ https://issues.apache.org/jira/browse/HBASE-21013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-21013. Resolution: Incomplete Assignee: (was: Mingdao Yang) > Backport "read part" of HBASE-18754 to all active 1.x branches > -- > > Key: HBASE-21013 > URL: https://issues.apache.org/jira/browse/HBASE-21013 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Priority: Critical > Attachments: HBASE-21013-branch-1.4.001.patch > > > The hfiles impacted by HBASE-18754 will have bytes of proto.TimeRangeTracker. > It makes all 1.x branches failed to read the hfile since all 1.x branches > can't deserialize the proto.TimeRangeTracker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Can I get a +1 on HBASE-15560 please?
Latest results look good and discussion points have been addressed. -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: Branch-1 releasing paused; HBase community: please help clean up your test code
A helpful view for tracking 1.5.0 issues is https://issues.apache.org/jira/projects/HBASE/versions/12340316 . I deleted fixversions "1.5.1" and "1.6.0" and had JIRA rewrite them back to "1.5.0" so you won't be confused by JIRA thinking 1.5.0 is in released state. All open and pending branch-1 work has the "1.5.0" fix version again, until we try again for a release at some future time. On Thu, Apr 11, 2019 at 9:10 AM Andrew Purtell wrote: > I have tried to release four 1.5.0 candidates from head of branch-1. > > Some veto votes were cast for compatibility issues. This was fine. > > Others are for unit test results that do not reproduce locally for me. I > do not have the ability or bandwidth to fix tests which do not fail > reliably for me. So let me appeal to the community. If you find a > repeatable test failure on branch-1 please file a JIRA and fix it. > > As things stand now I am pausing any attempts to make more 1.5.0 release > candidates until the community steps forward to clean up its code. As an > alternative I may start aggressively disabling unit tests which do not fail > for me but have been reported in on candidate vote vetoes. Otherwise it is > impossible to make forward progress. > > Thank you for your attention. > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-22212) Backport missing filter improvements
Josh Elser created HBASE-22212: -- Summary: Backport missing filter improvements Key: HBASE-22212 URL: https://issues.apache.org/jira/browse/HBASE-22212 Project: HBase Issue Type: Bug Components: Filters Reporter: Josh Elser Assignee: Josh Elser HBASE-19008 and HBASE-21129 were never backported beyond branch-2. I can't find any reason that this was not done. Despite these being public-tagged classes, no incompatible changes were added. The lack of these changes prevents HBASE-22144 from being backported cleanly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [NOTICE] Renumbering branch-1 from 1.5.0 to 1.6.0-SNAPSHOT
No need to renumber as it turns out because the fourth 1.5.0 RC was just vetoed. I sent another email about the resulting pause on release attempts. Please disregard this thread. > On Apr 10, 2019, at 6:22 PM, Andrew Purtell wrote: > > I am not anticipating a 1.5.1 release unless we need a patch release for some > reason. After 1.5.0 will be 1.6.0. > > If we do need to make a 1.5.1 patch release, we can make a new branch from > rel/1.5.0 named "branch-1.5", but not before this is necessary. Hopefully > this will not be necessary. > > Therefore I plan to renumber branch-1 from 1.5.0 to 1.6.0-SNAPSHOT and also > plan to go through JIRA and retarget any fixVersion of 1.5.1 to 1.6.0. > > Any concerns, please let me know. > > If no concerns, I will do this tomorrow. > > Also, please be advised voting is still open on 1.5.0RC3, please go to the > vote thread on dev@ if you can spare some time to vote on it, thank you. > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk
[REPORT] HBase quarterly report Jan-Mar 2019
All, HBase submits a report to the ASF board once a quarter, to inform the board about project health. I'm sending the report to the user@ and dev@ mailing lists because you are the project, and for transparency. If you have any questions about the report or the running of the project, you can pose them to me or any other PMC member or committer, or send an email to priv...@hbase.apache.org, which every PMC member subscribes to. Thanks, Misty -- ## Description: Apache HBase is an open-source, distributed, versioned, non-relational database. Apache HBase gives you low latency random access to billions of rows with millions of columns atop non-specialized hardware. hbase-thirdparty is a set of internal artifacts used by the project to mitigate the impact of our dependency choices on the wider ecosystem. ## Issues: Board-only information removed from public report. ## Activity: There have been lots of interesting discussions on the dev@ mailing list. Highlights: - We have nearly all the infrastructure ready to complete the move to Gitbox. ( https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E ) - An interesting discussion occurred about the frequency of releases in the 2.1.x line ( https://lists.apache.org/thread.html/222aacbf74d9788287ef5606fe66a06c1a3aa31a299a469d124a5ee0@%3Cdev.hbase.apache.org%3E). The discussion included a sub-discussion about what it will take to separate the hbck took from the main project, and another sub-discussion about time-based releases and release cadence in general. It's worth a read. - A conversation is underway about getting the branch-2 line ready for the "stable" pointer. ( https://lists.apache.org/thread.html/81b2917eba3fce2872b96526635a5ba11535c73641cb3d38271ee353@%3Cdev.hbase.apache.org%3E ) - We asked ourselves what our greatest friction point is when attracting new contributors. We'd love for you to weigh in. ( https://lists.apache.org/thread.html/887a253ccce44b51e4cfc57276545fcea123fa780a73ffc9fa279cd2@%3Cdev.hbase.apache.org%3E ) - Sean Busbey, the release manager for 1.2, proposes that 1.2.12 (currently a release candidate) be the last release in the 1.2 line. ( https://lists.apache.org/thread.html/3cf19458b739a612fbe1f72d17c14124ff536b0cb8678a63d80cf019@%3Cdev.hbase.apache.org%3E ) Planning is underway for the next HBaseCon Asia! Contact Duo Zhang < zhang...@apache.org> to get involved. The HBase project is working toward the goal of more frequent minor releases and fewer unprompted maintenance releases. In total this quarter, HBase had 7 releases over this quarter, across 3 release lines, and hbase-thirdparty had 1 release. Another goal, spanning several quarters, is to solicit more non-binding votes on release candidates, as non-binding votes are one signal of broader community engagement. - HBase 2.1.3 RC1 had 40% non-binding votes ( https://lists.apache.org/thread.html/baef78476204a476b45f7e273b006f19d057c6d9123d8a34178539a9@%3Cdev.hbase.apache.org%3E ) - We voted on and failed several release candidates this quarter. We had less participation from non-binding voters this quarter. As a reminder, anyone on the dev@ list can test and provide feedback on a release candidate, and cast a non-binding vote. One new committer was added this quarter, and one committer joined the PMC. More details below. Thanks to the new committers and PMC members for agreeing to take on more responsibilities in the project. ## Health report: We've bounced back from the holiday dip in activity. Specifically, we have fixed a lot of JIRA issues this quarter! ## PMC changes: - Currently 45 PMC members. - Peter Somogyi was added to the PMC on Mon Jan 21 2019. ## Committer base changes: - Currently 79 committers. - Xu Cang was added as a committer on Fri Feb 01 2019 ## Releases: - 2.0.4 was released on Thu Jan 3 2019 - 2.1.2 was released on Tue Jan 8 2019 - 1.2.10 was released on Wed Jan 16 2019 - 2.1.3 was released on Wed Feb 13 2019 - 1.2.11 was released on Sun Feb 24 2019 - 2.0.5 was released on Sun Mar 24 2019 - 2.1.4 was released on Mon Mar 25 2019 - hbase-thirdparty 2.2.0 was released on Tue Apr 2, 2019 ## Mailing list activity: - dev@hbase.apache.org: - 1051 subscribers (down -5 in the last 3 months): - 1115 emails sent to list (997 in previous quarter) - u...@hbase.apache.org: - 2236 subscribers (down -17 in the last 3 months): - 117 emails sent to list (119 in previous quarter) ## JIRA activity: - 494 JIRA tickets created in the last 3 months (up from 405 last quarter) - 436 JIRA tickets closed/resolved in the last 3 months (up from 332 last quarter)
Branch-1 releasing paused; HBase community: please help clean up your test code
I have tried to release four 1.5.0 candidates from head of branch-1. Some veto votes were cast for compatibility issues. This was fine. Others are for unit test results that do not reproduce locally for me. I do not have the ability or bandwidth to fix tests which do not fail reliably for me. So let me appeal to the community. If you find a repeatable test failure on branch-1 please file a JIRA and fix it. As things stand now I am pausing any attempts to make more 1.5.0 release candidates until the community steps forward to clean up its code. As an alternative I may start aggressively disabling unit tests which do not fail for me but have been reported in on candidate vote vetoes. Otherwise it is impossible to make forward progress. Thank you for your attention.
Re: The fourth HBase 1.5.0 release candidate (RC3) is available
There are two failure cases it looks like. And this looks like flakes. The wrong FS assertions are not something I see when I run these tests myself. I am not able to investigate something I can’t reproduce. What I suggest is since you can reproduce do a git bisect to find the commit that introduced the problem. Then we can revert it. As an alternative we can open a JIRA, report the problem, temporarily @ignore the test, and continue. This latter option only should be done if we are fairly confident it is a test only problem. The connect exceptions are interesting. I see these sometimes when the suite is executed, not this particular case, but when the failed test is executed by itself it always passes. It is possible some change to classes related to the minicluster or startup or shutdown timing are the cause, but it is test time flaky behavior. I’m not happy about this but it doesn’t actually fail the release because the failure is never repeatable when the test is run standalone. In general it would be great if some attention was paid to test cleanliness on branch-1. As RM I’m not in a position to insist that everything is perfect or there will never be another 1.x release, certainly not from branch-1. So, tests which fail repeatedly block a release IMHO but flakes do not. > On Apr 10, 2019, at 11:20 PM, Yu Li wrote: > > -1 > > Observed many UT failures when checking the source package (tried multiple > rounds on two different environments, MacOs and Linux, got the same > result), including (but not limited to): > > TestBulkload: > shouldBulkLoadSingleFamilyHLog(org.apache.hadoop.hbase.regionserver.TestBulkLoad) > Time elapsed: 0.083 s <<< ERROR! > java.lang.IllegalArgumentException: Wrong FS: > file:/var/folders/t6/vch4nh357f98y1wlq09lbm7hgn/T/junit1805329913454564189/junit8020757893576011944/data/default/shouldBulkLoadSingleFamilyHLog/8f4a6b584533de2fd1bf3c398dfaac29, > expected: hdfs://localhost:55938 >at > org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamiliesAndSpecifiedTableName(TestBulkLoad.java:246) >at > org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamilies(TestBulkLoad.java:256) >at > org.apache.hadoop.hbase.regionserver.TestBulkLoad.shouldBulkLoadSingleFamilyHLog(TestBulkLoad.java:150) > > TestStoreFile: > testCacheOnWriteEvictOnClose(org.apache.hadoop.hbase.regionserver.TestStoreFile) > Time elapsed: 0.083 s <<< ERROR! > java.net.ConnectException: Call From localhost/127.0.0.1 to localhost:55938 > failed on connection exception: java.net.ConnectException: Connection > refused; For more details see: > http://wiki.apache.org/hadoop/ConnectionRefused >at > org.apache.hadoop.hbase.regionserver.TestStoreFile.writeStoreFile(TestStoreFile.java:1047) >at > org.apache.hadoop.hbase.regionserver.TestStoreFile.testCacheOnWriteEvictOnClose(TestStoreFile.java:908) > > TestHFile: > testEmptyHFile(org.apache.hadoop.hbase.io.hfile.TestHFile) Time elapsed: > 0.08 s <<< ERROR! > java.net.ConnectException: Call From > z05f06378.sqa.zth.tbsite.net/11.163.183.195 to localhost:35529 failed on > connection exception: java.net.ConnectException: Connection refused; For > more details see: http://wiki.apache.org/hadoop/ConnectionRefused >at > org.apache.hadoop.hbase.io.hfile.TestHFile.testEmptyHFile(TestHFile.java:90) > Caused by: java.net.ConnectException: Connection refused >at > org.apache.hadoop.hbase.io.hfile.TestHFile.testEmptyHFile(TestHFile.java:90) > > TestBlocksScanned: > testBlocksScannedWithEncoding(org.apache.hadoop.hbase.regionserver.TestBlocksScanned) > Time elapsed: 0.069 s <<< ERROR! > java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:35529/tmp/ > hbase-jueding.ly/hbase/data/default/TestBlocksScannedWithEncoding/a4a416cc3060d9820a621c294af0aa08, > expected: file:/// >at > org.apache.hadoop.hbase.regionserver.TestBlocksScanned._testBlocksScanned(TestBlocksScanned.java:90) >at > org.apache.hadoop.hbase.regionserver.TestBlocksScanned.testBlocksScannedWithEncoding(TestBlocksScanned.java:86) > > And please let me know if any known issue I'm not aware of. Thanks. > > Best Regards, > Yu > > >> On Mon, 8 Apr 2019 at 11:38, Yu Li wrote: >> >> The performance report LGTM, thanks! (and sorry for the lag due to >> Qingming Festival Holiday here in China) >> >> Still verifying the release, just some quick feedback: observed some >> incompatible changes in compatibility report including >> HBASE-21492/HBASE-21684 and worth a reminder in ReleaseNote. >> >> Irrelative but noticeable: the 1.4.9 release note URL is invalid on >> https://hbase.apache.org/downloads.html >> >> Best Regards, >> Yu >> >> >>> On Fri, 5 Apr 2019 at 08:45, Andrew Purtell wrote: >>> >>> The difference is basically noise per the usual YCSB evaluation. Small >>> differences in workloads D and F (slightly worse) and workload E (slightly >>> better) that do not indicate
Any bulkload issue with 2.2.0?
Trying to bulkload a single HFile in a single region empty table on a
sleeping 2.2.0 cluster, I get this:
2019-04-11 11:22:40,594 INFO [LoadIncrementalHFiles-0] compress.CodecPool:
Got brand-new decompressor [.snappy]
2019-04-11 11:22:40,632 INFO [LoadIncrementalHFiles-0]
tool.LoadIncrementalHFiles: Trying to load hfile=hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23
first=Optional[\x02\x02\x01\x02\x03\x01\x02\x03\x03\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
last=Optional[\x02\x02\x02\x01\x08\x01\x05\x01\x00\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
2019-04-11 11:22:40,737 WARN [LoadIncrementalHFiles-1]
tool.LoadIncrementalHFiles: Attempt to bulk load region containing into
table stones5 with files [family:A path:hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23]
failed. This is recoverable and they will be retried.
2019-04-11 11:22:40,746 INFO [main] tool.LoadIncrementalHFiles: Split
occurred while grouping HFiles, retry attempt 1 with 1 files remaining to
group or split
2019-04-11 11:22:40,758 INFO [LoadIncrementalHFiles-2]
tool.LoadIncrementalHFiles: Trying to load hfile=hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23
first=Optional[\x02\x02\x01\x02\x03\x01\x02\x03\x03\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
last=Optional[\x02\x02\x02\x01\x08\x01\x05\x01\x00\x00\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F\x7F]
2019-04-11 11:22:40,816 WARN [LoadIncrementalHFiles-3]
tool.LoadIncrementalHFiles: Attempt to bulk load region containing into
table stones5 with files [family:A path:hdfs://
node2.distparser.com:8020/source/A/fd8dd05fbee84a8688733de648eb2a23]
failed. This is recoverable and they will be retried.
2019-04-11 11:22:40,824 INFO [main] tool.LoadIncrementalHFiles: Split
occurred while grouping HFiles, retry attempt 2 with 1 files remaining to
group or split
Bulkload seems to think that the destination table is splitting while
loading.
Destination table:
jmspaggi@node8:~/Othello$ hdfs dfs -ls /hbase/data/default/stones5/*
Found 1 items
-rw-r--r-- 3 hbase supergroup507 2019-04-11 11:22
/hbase/data/default/stones5/.tabledesc/.tableinfo.01
Found 2 items
-rw-r--r-- 3 hbase supergroup 42 2019-04-11 11:22
/hbase/data/default/stones5/052f1fdf28ae75754d28f2ed7fafd6c6/.regioninfo
drwxr-xr-x - hbase supergroup 0 2019-04-11 11:22
/hbase/data/default/stones5/052f1fdf28ae75754d28f2ed7fafd6c6/A
And meta seems to be clean:
stones5 column=table:state, timestamp=1554996138033, value=\x08\x00
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:regioninfo, timestamp=1554996137061, value={ENCODED =>
052f1fdf28ae75754d28f2ed7fafd6c6, NAME =>
'stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.', STARTKEY => '',
ENDKEY => ''}
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:seqnumDuringOpen, timestamp=1554996137061,
value=\x00\x00\x00\x00\x00\x00\x00\x02
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:server, timestamp=1554996137061, value=
node6.distparser.com:16020
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:serverstartcode, timestamp=1554996137061, value=1554893230076
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6. column=info:sn,
timestamp=1554996136579, value=node6.distparser.com,16020,1554893230076
stones5,,1554996135582.052f1fdf28ae75754d28f2ed7fafd6c6.
column=info:state, timestamp=1554996137061, value=OPEN
Looking at the code, it uses some deprecated functions and types.
I went the dirty way and just manually moved all my files into the table
region, but I think there is something to be looked at here :-/
JMS
Re: HBase 2.2.0 overlapping regions
Scanning the tables from the shell returns this: ERROR Java::JavaIo::IOException: Unable to find region for \x03\x02\x03\x01\x01\x02\x01\x01\x01\x01\x04\x00\x04\x02\x00\x00\x02\x01\x00\x00\x01 in stones5 Scanning the meta table returns 22regions (which seems to be normal). Output is there: https://pastebin.com/axbG5C87 I will remove all HFiles and bulkload them back then I will try to reproduce the issue... Le jeu. 11 avr. 2019 à 10:44, Jean-Marc Spaggiari a écrit : > Hi all, > > I don't know how i ended up in this stage, but I cleared /hbase in both ZK > and HDFS 3 days ago. created 3 brand new tables, started to put some data > on them. Twice a day I run a split command, to get a better spread. > > After 3 days I have a lot of regions overlaps. HBCK output is there: > https://pastebin.com/y4awxmN5 > > Now, how can this be repaired? If I understand correctly, -fixHdfsOverlaps > can't be used anymore. I can move the HFiles, drop the table, re-create, > reload and split again, but I have a job running, that's supposed to last 6 > days. I would have liked to keep it running. > > Any idea why there is such overlaps? > > JMS >
Still Failing: HBase Generate Website
Build status: Still Failing
The HBase website has not been updated to incorporate HBase commit
${CURRENT_HBASE_COMMIT}.
See https://builds.apache.org/job/hbase_generate_website/1642/console
Re: [VOTE] First release candidate for HBASE 2.2.0 is available
Hi all, Any chance to get an updated RC soon? JM Le mar. 12 mars 2019 à 12:23, Sean Busbey a écrit : > quick follow-up here. the full version info is in fact missing the > revision: > > hbase-2.2.0 busbey$ ./bin/hbase version > HBase 2.2.0 > Source code repository > git://hao-OptiPlex-7050/home/hao/open_source/hbase revision= > Compiled by hao on 2019年 03月 07日 星期四 14:05:34 CST > From source with checksum 783fee467bb1b28666f0d904437862c4 > > I think this is the issue stack ran into on HBASE-21935/HBASE-21999 > where HBASE-20764 introduced a git cli option that issn't supported on > older versions of git. > > Guanghao for the next RC would it be possible to update your local git > version? > > On Tue, Mar 12, 2019 at 9:37 AM Sean Busbey wrote: > > > > locale of the build is up to the RM (this is why, for example, the 2.1 > > release line javadocs have chinese for the boilerplate text[1]) > > > > however, it does look like that shell output might be missing the > > build revision information from git or we might not be properly > > parsing the output from git when a non-english locale is used. > > > > [1]: http://hbase.apache.org/2.1/apidocs/index.html > > > > > > On Tue, Mar 12, 2019 at 8:54 AM Jean-Marc Spaggiari > > wrote: > > > > > > Also, in the shell,it displays Asian texte: > > > > > > "Version 2.2.0, r, 2019年 03月 07日 星期四 14:05:34 CST" > > > > > > Not sure if that's we want. > > > > > > JMS > > > > > > Le lun. 11 mars 2019 à 21:53, Guanghao Zhang a > écrit : > > > > > > > Let me start a new RC1. HBASE-21970 should be included and need a > release > > > > note. > > > > > > > > Sean Busbey 于2019年3月12日周二 上午8:35写道： > > > > > > > > > I'm -1 on RC0 as it is. > > > > > > > > > > The current release notes don't include any call out about the > upgrade > > > > > steps needed. Since we don't usually have minor-version specific > > > > > upgrade steps and especially since there are things folks need to > do > > > > > before installing 2.2.0, it's important that they be front and > center. > > > > > Possibly that should mean a link to the ref guide section from the > RC > > > > > instructions and eventual announcement. > > > > > > > > > > I think either HBASE-21075 needs to have 2.2.0 included in its fix > > > > > version or the release note from that issue needs to be copied > over to > > > > > HBASE-21970 and it needs to have 2.2.0 included in its fix > version(s). > > > > > In either case the release notes should link to the ref guide > section. > > > > > > > > > > On Thu, Mar 7, 2019 at 3:44 AM Guanghao Zhang > > > > wrote: > > > > > > > > > > > > Please vote on this release candidate (RC) for Apache HBase > 2.2.0. > > > > > > This is the first release of the branch-2.2 line. > > > > > > > > > > > > The VOTE will remain open for at least 72 hours. > > > > > > > > > > > > [ ] +1 Release this package as Apache HBase 2.2.0 > > > > > > [ ] -1 Do not release this package because ... > > > > > > > > > > > > The tag to be voted on is 2.2.0-RC0 (commit > > > > > > 4ab2dc20f15e9b59477de4bd971c367f3ce342cb): > > > > > > > > > > > > https://github.com/apache/hbase/tree/2.2.0-RC0 > > > > > > > > > > > > The release files, including signatures, digests, etc. can be > found at: > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC0/ > > > > > > > > > > > > Maven artifacts are available in a staging repository at: > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehbase-1286 > > > > > > > > > > > > Signatures used for HBase RCs can be found in this file: > > > > > > > > > > > > https://dist.apache.org/repos/dist/release/hbase/KEYS > > > > > > > > > > > > The list of bug fixes going into 2.2.0 can be found in included > > > > > > CHANGES.md and RELEASENOTES.md available here: > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC0/CHANGES.md > > > > > > > https://dist.apache.org/repos/dist/dev/hbase/2.2.0RC0/RELEASENOTES.md > > > > > > > > > > > > To learn more about Apache HBase, please see > http://hbase.apache.org/ > > > > > > > > > > > > Thanks, > > > > > > Guanghao Zhang > > > > > > > > > >
HBase 2.2.0 overlapping regions
Hi all, I don't know how i ended up in this stage, but I cleared /hbase in both ZK and HDFS 3 days ago. created 3 brand new tables, started to put some data on them. Twice a day I run a split command, to get a better spread. After 3 days I have a lot of regions overlaps. HBCK output is there: https://pastebin.com/y4awxmN5 Now, how can this be repaired? If I understand correctly, -fixHdfsOverlaps can't be used anymore. I can move the HFiles, drop the table, re-create, reload and split again, but I have a job running, that's supposed to last 6 days. I would have liked to keep it running. Any idea why there is such overlaps? JMS
Re: Contributor Permission Request
Done. Add to HBase jira CONTRIBUTORS. Thanks. fei wen 于2019年4月11日周四 下午7:26写道： > Hi, > I want to contribute to Apache HBase. > Would you please give me the contributor permission? > My JIRA ID is wen.feiyi. >
[jira] [Created] (HBASE-22211) Remove the returnBlock method in CachingBlockReader because we can just call HFileBlock#release directly
Zheng Hu created HBASE-22211: Summary: Remove the returnBlock method in CachingBlockReader because we can just call HFileBlock#release directly Key: HBASE-22211 URL: https://issues.apache.org/jira/browse/HBASE-22211 Project: HBase Issue Type: Sub-task Reporter: Zheng Hu Assignee: Zheng Hu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22210) Fix hbase-connectors-assembly to include every jar
Balazs Meszaros created HBASE-22210:
---
Summary: Fix hbase-connectors-assembly to include every jar
Key: HBASE-22210
URL: https://issues.apache.org/jira/browse/HBASE-22210
Project: HBase
Issue Type: Task
Components: hbase-connectors
Affects Versions: connector-1.0.0
Reporter: Balazs Meszaros
Assignee: Balazs Meszaros
Fix For: connector-1.0.0
After compiling hbase-connectors, {{bin/hbase-connectors kafkaproxy}} throws
the following exception:
{noformat}
Error: Could not find or load main class
org.apache.hadoop.hbase.kafka.KafkaProxy
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Contributor Permission Request
Hi, I want to contribute to Apache HBase. Would you please give me the contributor permission? My JIRA ID is wen.feiyi.
[jira] [Resolved] (HBASE-17564) Fix remaining calls to deprecated methods of Admin and HBaseAdmin
[ https://issues.apache.org/jira/browse/HBASE-17564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Hentschel resolved HBASE-17564. --- Resolution: Duplicate Fix Version/s: (was: 3.0.0) This was already solved as part of HBASE-18428. > Fix remaining calls to deprecated methods of Admin and HBaseAdmin > - > > Key: HBASE-17564 > URL: https://issues.apache.org/jira/browse/HBASE-17564 > Project: HBase > Issue Type: Improvement >Reporter: Jan Hentschel >Assignee: Jan Hentschel >Priority: Trivial > Attachments: HBASE-17564.master.001.patch > > > Fix the remaining calls to deprecated methods of the *Admin* interface and > the *HBaseAdmin* class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22209) sdf
[ https://issues.apache.org/jira/browse/HBASE-22209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari resolved HBASE-22209. - Resolution: Invalid > sdf > --- > > Key: HBASE-22209 > URL: https://issues.apache.org/jira/browse/HBASE-22209 > Project: HBase > Issue Type: Bug > Components: Admin >Affects Versions: 2.1.4 >Reporter: leonjoe >Priority: Major > Fix For: hbase-6055 > > Original Estimate: 504h > Remaining Estimate: 504h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22209) sdf
leonjoe created HBASE-22209: --- Summary: sdf Key: HBASE-22209 URL: https://issues.apache.org/jira/browse/HBASE-22209 Project: HBase Issue Type: Bug Components: Admin Affects Versions: 2.1.4 Reporter: leonjoe Fix For: hbase-6055 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22189) Remove usage of StoreFile.getModificationTimeStamp
[ https://issues.apache.org/jira/browse/HBASE-22189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Hentschel resolved HBASE-22189. --- Resolution: Fixed Fix Version/s: 2.2.1 2.1.5 2.0.6 2.3.0 3.0.0 > Remove usage of StoreFile.getModificationTimeStamp > -- > > Key: HBASE-22189 > URL: https://issues.apache.org/jira/browse/HBASE-22189 > Project: HBase > Issue Type: Task >Reporter: Jan Hentschel >Assignee: Jan Hentschel >Priority: Trivial > Fix For: 3.0.0, 2.3.0, 2.0.6, 2.1.5, 2.2.1 > > > The method StoreFile.getModificationTimeStamp() was deprecated, but is still > used. The remaining usages should be moved to > StoreFile.getModificationTimestamp(). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22198) Fix flakey TestAsyncTableGetMultiThreaded
[ https://issues.apache.org/jira/browse/HBASE-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-22198. --- Resolution: Fixed Hadoop Flags: Reviewed Pushed to branch-2.1+. > Fix flakey TestAsyncTableGetMultiThreaded > - > > Key: HBASE-22198 > URL: https://issues.apache.org/jira/browse/HBASE-22198 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0, 2.1.5 > > > https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2959/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncTableGetMultiThreaded/test/ > The error is thrown from an admin method, where we do not have any retries if > the region is not online yet. Should be a test issue, let me fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22196) Split TestRestartCluster
[
https://issues.apache.org/jira/browse/HBASE-22196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-22196.
---
Resolution: Fixed
Hadoop Flags: Reviewed
Pushed to branch-2.2+.
> Split TestRestartCluster
>
>
> Key: HBASE-22196
> URL: https://issues.apache.org/jira/browse/HBASE-22196
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
>
> The logs for later tests are messed up with error messages, like
> {noformat}
> 2019-04-09 09:41:11,717 WARN [LeaseRenewer:jenkins.hfs.12@localhost:41108]
> hdfs.LeaseRenewer(468): Failed to renew lease for
> [DFSClient_NONMAPREDUCE_400481390_21] for 55 seconds. Will retry shortly ...
> java.net.ConnectException: Call From asf918.gq1.ygridcore.net/67.195.81.138
> to localhost:41108 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.GeneratedConstructorAccessor79.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
> at org.apache.hadoop.ipc.Client.call(Client.java:1480)
> at org.apache.hadoop.ipc.Client.call(Client.java:1413)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at com.sun.proxy.$Proxy30.renewLease(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.renewLease(ClientNamenodeProtocolTranslatorPB.java:595)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy33.renewLease(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
> at com.sun.proxy.$Proxy34.renewLease(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
> at com.sun.proxy.$Proxy34.renewLease(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:901)
> at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:423)
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:448)
> at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:615)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:713)
> at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:376)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
> at org.apache.hadoop.ipc.Client.call(Client.java:1452)
> ... 26 more
> 2019-04-09 09:41:11,949 WARN [RS_OPEN_REGION-regionserver/asf918:33671-1]
> regionserver.HStore(1062): Failed flushing store file, retrying num=8
> java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:817)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2114)
> at
>
Re: The fourth HBase 1.5.0 release candidate (RC3) is available
-1
Observed many UT failures when checking the source package (tried multiple
rounds on two different environments, MacOs and Linux, got the same
result), including (but not limited to):
TestBulkload:
shouldBulkLoadSingleFamilyHLog(org.apache.hadoop.hbase.regionserver.TestBulkLoad)
Time elapsed: 0.083 s <<< ERROR!
java.lang.IllegalArgumentException: Wrong FS:
file:/var/folders/t6/vch4nh357f98y1wlq09lbm7hgn/T/junit1805329913454564189/junit8020757893576011944/data/default/shouldBulkLoadSingleFamilyHLog/8f4a6b584533de2fd1bf3c398dfaac29,
expected: hdfs://localhost:55938
at
org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamiliesAndSpecifiedTableName(TestBulkLoad.java:246)
at
org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamilies(TestBulkLoad.java:256)
at
org.apache.hadoop.hbase.regionserver.TestBulkLoad.shouldBulkLoadSingleFamilyHLog(TestBulkLoad.java:150)
TestStoreFile:
testCacheOnWriteEvictOnClose(org.apache.hadoop.hbase.regionserver.TestStoreFile)
Time elapsed: 0.083 s <<< ERROR!
java.net.ConnectException: Call From localhost/127.0.0.1 to localhost:55938
failed on connection exception: java.net.ConnectException: Connection
refused; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
at
org.apache.hadoop.hbase.regionserver.TestStoreFile.writeStoreFile(TestStoreFile.java:1047)
at
org.apache.hadoop.hbase.regionserver.TestStoreFile.testCacheOnWriteEvictOnClose(TestStoreFile.java:908)
TestHFile:
testEmptyHFile(org.apache.hadoop.hbase.io.hfile.TestHFile) Time elapsed:
0.08 s <<< ERROR!
java.net.ConnectException: Call From
z05f06378.sqa.zth.tbsite.net/11.163.183.195 to localhost:35529 failed on
connection exception: java.net.ConnectException: Connection refused; For
more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at
org.apache.hadoop.hbase.io.hfile.TestHFile.testEmptyHFile(TestHFile.java:90)
Caused by: java.net.ConnectException: Connection refused
at
org.apache.hadoop.hbase.io.hfile.TestHFile.testEmptyHFile(TestHFile.java:90)
TestBlocksScanned:
testBlocksScannedWithEncoding(org.apache.hadoop.hbase.regionserver.TestBlocksScanned)
Time elapsed: 0.069 s <<< ERROR!
java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:35529/tmp/
hbase-jueding.ly/hbase/data/default/TestBlocksScannedWithEncoding/a4a416cc3060d9820a621c294af0aa08,
expected: file:///
at
org.apache.hadoop.hbase.regionserver.TestBlocksScanned._testBlocksScanned(TestBlocksScanned.java:90)
at
org.apache.hadoop.hbase.regionserver.TestBlocksScanned.testBlocksScannedWithEncoding(TestBlocksScanned.java:86)
And please let me know if any known issue I'm not aware of. Thanks.
Best Regards,
Yu
On Mon, 8 Apr 2019 at 11:38, Yu Li wrote:
> The performance report LGTM, thanks! (and sorry for the lag due to
> Qingming Festival Holiday here in China)
>
> Still verifying the release, just some quick feedback: observed some
> incompatible changes in compatibility report including
> HBASE-21492/HBASE-21684 and worth a reminder in ReleaseNote.
>
> Irrelative but noticeable: the 1.4.9 release note URL is invalid on
> https://hbase.apache.org/downloads.html
>
> Best Regards,
> Yu
>
>
> On Fri, 5 Apr 2019 at 08:45, Andrew Purtell wrote:
>
>> The difference is basically noise per the usual YCSB evaluation. Small
>> differences in workloads D and F (slightly worse) and workload E (slightly
>> better) that do not indicate serious regression.
>>
>> Linux version 4.14.55-62.37.amzn1.x86_64
>> c3.8xlarge x 5
>> OpenJDK Runtime Environment (build 1.8.0_181-shenandoah-b13)
>> -Xms20g -Xmx20g -XX:+UseG1GC -XX:+AlwaysPreTouch -XX:+UseNUMA
>> -XX:-UseBiasedLocking -XX:+ParallelRefProcEnabled
>> Hadoop 2.9.2
>> Init: Load 100 M rows and snapshot
>> Run: Delete table, clone and redeploy from snapshot, run 10 M operations
>> Args: -threads 100 -target 5
>> Test table: {NAME => 'u', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY
>> => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
>> 'ROW_INDEX_V1', TTL => 'FOREVER', COMPRESSION => 'SNAPPY', MIN_VERSIONS =>
>> '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE =>
>> '0'}
>>
>>
>> YCSB Workload A
>>
>> target 50k/op/s 1.4.9 1.5.0
>>
>>
>>
>> [OVERALL], RunTime(ms) 200592 200583
>> [OVERALL], Throughput(ops/sec) 49852 49855
>> [READ], AverageLatency(us) 544 559
>> [READ], MinLatency(us) 267 292
>> [READ], MaxLatency(us) 165631 185087
>> [READ], 95thPercentileLatency(us) 738 742
>> [READ], 99thPercentileLatency(us), 1877 1961
>> [UPDATE], AverageLatency(us) 1370 1181
>> [UPDATE], MinLatency(us) 702 646
>> [UPDATE], MaxLatency(us) 180735 177279
>> [UPDATE], 95thPercentileLatency(us) 1943 1652
>> [UPDATE], 99thPercentileLatency(us) 3257 3085
>>
>> YCSB Workload B
>>
>> target 50k/op/s 1.4.9 1.5.0
>>
>>
>>
>> [OVERALL], RunTime(ms) 200599 200581
>> [OVERALL], Throughput(ops/sec) 49850 49855
>> [READ], AverageLatency(us), 454 471
>> [READ],
