Re: [DISCUSS] 2.1.1 Release Plans?

2018-10-08 Thread Stack
On Mon, Oct 8, 2018 at 4:01 PM Josh Elser  wrote:

> Best place to find hbck2 issue needing review is off of HBASE-19121 or
> somewhere else?
>
>
For 2.1.1 issues, see the 2.1.1 release listing:
https://issues.apache.org/jira/projects/HBASE/versions/12343470 Half these
items are items turned up testing branch-2.1 and trying to use hbck2. Will
link a few others.


> All: please feel free to ping directly if you want/need reviews.
>
> Will do.

Thanks,
S



> On 10/5/18 7:41 PM, 张铎(Duo Zhang) wrote:
> > Stack has a plan on the 2.1.1 release where we want to finish the first
> > version on hbck2. In the real deploy we have met a stuck cluster several
> > times, and lots of users have asked that why hbck can not work any
> more...
> >
> > So the current opening issue is not important, please help reviewing the
> > patches for hbck2 to speed up the release...
> >
> > Thanks for bringing this up
> >
> > Mike Drob 于2018年10月5日 周五23:53写道:
> >
> >> Devs,
> >>
> >> It's been almost 3 months since 2.1.0 was released (Jul 19) and we have
> 150
> >> commits on branch-2.1 in that time. What do folks think of getting a
> >> release going? I know that there's been some discussion around the HBCK2
> >> stuff landing, but I feel like the conversation has gotten a bit lost
> >> without an actual release to relate to.
> >>
> >> Duo, as the 2.1.0 release manager, are you interested in maintaining the
> >> 2.1 branch release cadence? If you've gotten busy, then let's find
> another
> >> volunteer.
> >>
> >> There are 18 issues open or in progress currently. Only one is labelled
> >> blocker, and five more are critical -- let's evaluate these and the
> rest to
> >> figure out what we need for a release to happen. I went ahead and
> created a
> >> 2.1.2 version in Jira so that we have somewhere to move issues that
> aren't
> >> getting done soon.
> >>
> >> Meanwhile, I think we also need to look at test stabilization --
> there's 15
> >> tests on the dashboard that might need attention.
> >>
> >> Mike
> >>
> >
>


Re: [DISCUSS] 2.1.1 Release Plans?

2018-10-08 Thread Josh Elser
Best place to find hbck2 issue needing review is off of HBASE-19121 or 
somewhere else?


All: please feel free to ping directly if you want/need reviews.

On 10/5/18 7:41 PM, 张铎(Duo Zhang) wrote:

Stack has a plan on the 2.1.1 release where we want to finish the first
version on hbck2. In the real deploy we have met a stuck cluster several
times, and lots of users have asked that why hbck can not work any more...

So the current opening issue is not important, please help reviewing the
patches for hbck2 to speed up the release...

Thanks for bringing this up

Mike Drob 于2018年10月5日 周五23:53写道:


Devs,

It's been almost 3 months since 2.1.0 was released (Jul 19) and we have 150
commits on branch-2.1 in that time. What do folks think of getting a
release going? I know that there's been some discussion around the HBCK2
stuff landing, but I feel like the conversation has gotten a bit lost
without an actual release to relate to.

Duo, as the 2.1.0 release manager, are you interested in maintaining the
2.1 branch release cadence? If you've gotten busy, then let's find another
volunteer.

There are 18 issues open or in progress currently. Only one is labelled
blocker, and five more are critical -- let's evaluate these and the rest to
figure out what we need for a release to happen. I went ahead and created a
2.1.2 version in Jira so that we have somewhere to move issues that aren't
getting done soon.

Meanwhile, I think we also need to look at test stabilization -- there's 15
tests on the dashboard that might need attention.

Mike





Re: [DISCUSS] 2.1.1 Release Plans?

2018-10-08 Thread Stack
On Mon, Oct 8, 2018 at 8:19 AM Mike Drob  wrote:

> Thanks for pointing me to that discussion, Duo. I've updated HBASE-19121 to
> have fix version 2.1.1 and 2.0.3 so that it is easier (at least for me) to
> find via Jira.
>
>
I was thinking HBASE-19121 more an umbrella to hang hbck2 stuff off. Would
suggest we unschedule 2.1.1/2.0.3 as target.


> I'll start looking at the hbck2 patches and will leave comments there.
>
> There are a lot of subtasks against that issue, it is unclear to me which
> ones are critical and which ones are not for a release.
>
>
They usually have a target fix but let me give them another pass. Not all
have to land in 2.1.1/2.0.3.


Since hbck2 tool is in a different repo, have we thought about what the
> release actions would look like? Do we need to have two votes going at the
> same time? Can we release core repo with an hbck service in master, but
> release the hbck tool sometime later? This first release is going to be
> more complex I think; would be good if somebody can figure that out in
> parallel so we're not blocked on it later.
>
>
Yeah, the point of hbck2 being a separate repo was so they could have
different release cadences. It would be good if the first hbck2 release
happened around 2.1.1 release time. I can work on this part.

S



> Mike
>
> On Fri, Oct 5, 2018 at 6:41 PM 张铎(Duo Zhang) 
> wrote:
>
> > Stack has a plan on the 2.1.1 release where we want to finish the first
> > version on hbck2. In the real deploy we have met a stuck cluster several
> > times, and lots of users have asked that why hbck can not work any
> more...
> >
> > So the current opening issue is not important, please help reviewing the
> > patches for hbck2 to speed up the release...
> >
> > Thanks for bringing this up
> >
> > Mike Drob 于2018年10月5日 周五23:53写道:
> >
> > > Devs,
> > >
> > > It's been almost 3 months since 2.1.0 was released (Jul 19) and we have
> > 150
> > > commits on branch-2.1 in that time. What do folks think of getting a
> > > release going? I know that there's been some discussion around the
> HBCK2
> > > stuff landing, but I feel like the conversation has gotten a bit lost
> > > without an actual release to relate to.
> > >
> > > Duo, as the 2.1.0 release manager, are you interested in maintaining
> the
> > > 2.1 branch release cadence? If you've gotten busy, then let's find
> > another
> > > volunteer.
> > >
> > > There are 18 issues open or in progress currently. Only one is labelled
> > > blocker, and five more are critical -- let's evaluate these and the
> rest
> > to
> > > figure out what we need for a release to happen. I went ahead and
> > created a
> > > 2.1.2 version in Jira so that we have somewhere to move issues that
> > aren't
> > > getting done soon.
> > >
> > > Meanwhile, I think we also need to look at test stabilization --
> there's
> > 15
> > > tests on the dashboard that might need attention.
> > >
> > > Mike
> > >
> >
>


Re: [DISCUSS] 2.1.1 Release Plans?

2018-10-08 Thread Stack
On Fri, Oct 5, 2018 at 8:53 AM Mike Drob  wrote:

> Devs,
>
> It's been almost 3 months since 2.1.0 was released (Jul 19) and we have 150
> commits on branch-2.1 in that time. What do folks think of getting a
> release going? I know that there's been some discussion around the HBCK2
> stuff landing, but I feel like the conversation has gotten a bit lost
> without an actual release to relate to.
>

Sounds good. I was under the delusion that I was going to help push 2.1.1
out but I went off to work on hbck2 and its been taking a while figuring
what would comprise a first cut at server-side support for a useable hbck2.
I was thinking that 2.1.1 would have our first cut of APIs for a useable
HBCK2.

This project has been coming along with distractions as I run into fun,
dirty bugs in branch-2.1.


>
> There are 18 issues open or in progress currently. Only one is labelled
> blocker, and five more are critical -- let's evaluate these and the rest to
> figure out what we need for a release to happen. I went ahead and created a
> 2.1.2 version in Jira so that we have somewhere to move issues that aren't
> getting done soon.
>
> Meanwhile, I think we also need to look at test stabilization -- there's 15
> tests on the dashboard that might need attention.
>
>
Thanks for doing the above. This is work that needs to be done regardless.
S


> Mike
>


[jira] [Reopened] (HBASE-21185) WALPrettyPrinter: Additional useful info to be printed by wal printer tool, for debugability purposes

2018-10-08 Thread Sean Busbey (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reopened HBASE-21185:
-

reopening to remind myself to push for branch-1.

[~apurtell] opinion on inclusion in 1.4.z? I'll handle the backport work

> WALPrettyPrinter: Additional useful info to be printed by wal printer tool, 
> for debugability purposes
> -
>
> Key: HBASE-21185
> URL: https://issues.apache.org/jira/browse/HBASE-21185
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Trivial
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21185.branch-1.4.001.patch, 
> HBASE-21185.master.001.patch, HBASE-21185.master.002.patch, 
> HBASE-21185.master.003.patch
>
>
> *WALPrettyPrinter* is very useful for troubleshooting wal issues, such as 
> faulty replication sinks. An useful information one might want to track is 
> the size of a single WAL entry edit, as well as size for each edit cell. Am 
> proposing a patch that adds calculations for these two, as well an option to 
> seek straight to a given position on the WAL file being analysed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[RESULT] [VOTE] The first HBase 1.4.8 release candidate (RC0) is available

2018-10-08 Thread Andrew Purtell
With three binding +1s (Ted Yu, Zach York, myself) and three non-binding
+1s, (Balazs Meszaros, Tak-Lon Wu, and Peter Somogyi) and no 0 or -1 votes,
this vote passes. Let me complete the release and send out the
announcement.

Thanks to all who voted on this release candidate!


On Tue, Oct 2, 2018 at 5:57 PM Andrew Purtell  wrote:

> The first HBase 1.4.8 release candidate (RC0) is available for download at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.8RC0/ and Maven
> artifacts are available in the temporary repository
> https://repository.apache.org/content/repositories/orgapachehbase-1233/
>
> The git tag corresponding to the candidate is '1.4.8RC0' (91118ce5f1).
>
> A detailed source and binary compatibility report for this release is
> available for your review at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.8RC0/compat-check-report.html
> . There are no reported compatibility issues.
>
> A list of the 33 issues resolved in this release can be found at
> https://s.apache.org/xpxo .
>
> Please try out the candidate and vote +1/0/-1.
>
> The vote will be open for at least 72 hours. Unless objection I will try
> to close it Monday October 8, 2018 if we have sufficient votes.
>
> Prior to making this announcement I made the following preflight checks:
>
> RAT check passes (7u80)
> Unit test suite passes (7u80, 8u181)
> Opened the UI in a browser, poked around
> LTT load 1M rows with 100% verification and 20% updates (8u181)
> ITBLL 500M rows with serverKilling monkey (8u181)
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk


Re: [DISCUSS] 2.1.1 Release Plans?

2018-10-08 Thread Andrew Purtell
If you need a RM to do ~monthly cadence releases of _one_ 2.x line, I can
volunteer for that. I intend to continue releasing from branch-1 but seeing
how I'm set up to do monthly releases of one set of binaries I can do
another with only a modest increment of effort.

That said, it would be great if we had more people acting as RMs. You only
need commit rights to be a RM, so any committer is welcome to try
releasing. If you are interested, one of us, such as myself, can mentor you
and walk you through the process for the first couple of releases.


On Mon, Oct 8, 2018 at 8:31 AM Mike Drob  wrote:

> Another thought that I realized I forgot to mention -
>
> Previously on branch-1 we've been fairly consistent with time-bound
> maintenance releases happening almost monthly, looking at the great work
> done by our Andrew and Nick especially.
>
> We've fallen off track on branch-2 with maintenance releases on 2.0.x being
> fairly high in patch count. Although not perfect, I think this is a
> reasonable first order approximation for the amount of change going in. If
> we're not releasing often enough (or at all) in this situation, then folks
> that are trying to use it on real clusters don't have access to the fixes.
> Agree that it is bad they don't have hbck to fix some problems, but as they
> run they will keep finding more and more new issues and even running into
> the same issues because they have no better release to upgrade to.
>
> So theoretically we could get some fixes out to folks now to make their
> lives a little bit easier, and then get them another release with hbck2
> when it is ready.
>
> Mike
>
> On Mon, Oct 8, 2018 at 10:19 AM Mike Drob  wrote:
>
> > Thanks for pointing me to that discussion, Duo. I've updated HBASE-19121
> > to have fix version 2.1.1 and 2.0.3 so that it is easier (at least for
> me)
> > to find via Jira.
> >
> > I'll start looking at the hbck2 patches and will leave comments there.
> >
> > There are a lot of subtasks against that issue, it is unclear to me which
> > ones are critical and which ones are not for a release.
> >
> > Since hbck2 tool is in a different repo, have we thought about what the
> > release actions would look like? Do we need to have two votes going at
> the
> > same time? Can we release core repo with an hbck service in master, but
> > release the hbck tool sometime later? This first release is going to be
> > more complex I think; would be good if somebody can figure that out in
> > parallel so we're not blocked on it later.
> >
> > Mike
> >
> > On Fri, Oct 5, 2018 at 6:41 PM 张铎(Duo Zhang) 
> > wrote:
> >
> >> Stack has a plan on the 2.1.1 release where we want to finish the first
> >> version on hbck2. In the real deploy we have met a stuck cluster several
> >> times, and lots of users have asked that why hbck can not work any
> more...
> >>
> >> So the current opening issue is not important, please help reviewing the
> >> patches for hbck2 to speed up the release...
> >>
> >> Thanks for bringing this up
> >>
> >> Mike Drob 于2018年10月5日 周五23:53写道:
> >>
> >> > Devs,
> >> >
> >> > It's been almost 3 months since 2.1.0 was released (Jul 19) and we
> have
> >> 150
> >> > commits on branch-2.1 in that time. What do folks think of getting a
> >> > release going? I know that there's been some discussion around the
> HBCK2
> >> > stuff landing, but I feel like the conversation has gotten a bit lost
> >> > without an actual release to relate to.
> >> >
> >> > Duo, as the 2.1.0 release manager, are you interested in maintaining
> the
> >> > 2.1 branch release cadence? If you've gotten busy, then let's find
> >> another
> >> > volunteer.
> >> >
> >> > There are 18 issues open or in progress currently. Only one is
> labelled
> >> > blocker, and five more are critical -- let's evaluate these and the
> >> rest to
> >> > figure out what we need for a release to happen. I went ahead and
> >> created a
> >> > 2.1.2 version in Jira so that we have somewhere to move issues that
> >> aren't
> >> > getting done soon.
> >> >
> >> > Meanwhile, I think we also need to look at test stabilization --
> >> there's 15
> >> > tests on the dashboard that might need attention.
> >> >
> >> > Mike
> >> >
> >>
> >
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk


Re: [DISCUSS] 2.1.1 Release Plans?

2018-10-08 Thread Mike Drob
Another thought that I realized I forgot to mention -

Previously on branch-1 we've been fairly consistent with time-bound
maintenance releases happening almost monthly, looking at the great work
done by our Andrew and Nick especially.

We've fallen off track on branch-2 with maintenance releases on 2.0.x being
fairly high in patch count. Although not perfect, I think this is a
reasonable first order approximation for the amount of change going in. If
we're not releasing often enough (or at all) in this situation, then folks
that are trying to use it on real clusters don't have access to the fixes.
Agree that it is bad they don't have hbck to fix some problems, but as they
run they will keep finding more and more new issues and even running into
the same issues because they have no better release to upgrade to.

So theoretically we could get some fixes out to folks now to make their
lives a little bit easier, and then get them another release with hbck2
when it is ready.

Mike

On Mon, Oct 8, 2018 at 10:19 AM Mike Drob  wrote:

> Thanks for pointing me to that discussion, Duo. I've updated HBASE-19121
> to have fix version 2.1.1 and 2.0.3 so that it is easier (at least for me)
> to find via Jira.
>
> I'll start looking at the hbck2 patches and will leave comments there.
>
> There are a lot of subtasks against that issue, it is unclear to me which
> ones are critical and which ones are not for a release.
>
> Since hbck2 tool is in a different repo, have we thought about what the
> release actions would look like? Do we need to have two votes going at the
> same time? Can we release core repo with an hbck service in master, but
> release the hbck tool sometime later? This first release is going to be
> more complex I think; would be good if somebody can figure that out in
> parallel so we're not blocked on it later.
>
> Mike
>
> On Fri, Oct 5, 2018 at 6:41 PM 张铎(Duo Zhang) 
> wrote:
>
>> Stack has a plan on the 2.1.1 release where we want to finish the first
>> version on hbck2. In the real deploy we have met a stuck cluster several
>> times, and lots of users have asked that why hbck can not work any more...
>>
>> So the current opening issue is not important, please help reviewing the
>> patches for hbck2 to speed up the release...
>>
>> Thanks for bringing this up
>>
>> Mike Drob 于2018年10月5日 周五23:53写道:
>>
>> > Devs,
>> >
>> > It's been almost 3 months since 2.1.0 was released (Jul 19) and we have
>> 150
>> > commits on branch-2.1 in that time. What do folks think of getting a
>> > release going? I know that there's been some discussion around the HBCK2
>> > stuff landing, but I feel like the conversation has gotten a bit lost
>> > without an actual release to relate to.
>> >
>> > Duo, as the 2.1.0 release manager, are you interested in maintaining the
>> > 2.1 branch release cadence? If you've gotten busy, then let's find
>> another
>> > volunteer.
>> >
>> > There are 18 issues open or in progress currently. Only one is labelled
>> > blocker, and five more are critical -- let's evaluate these and the
>> rest to
>> > figure out what we need for a release to happen. I went ahead and
>> created a
>> > 2.1.2 version in Jira so that we have somewhere to move issues that
>> aren't
>> > getting done soon.
>> >
>> > Meanwhile, I think we also need to look at test stabilization --
>> there's 15
>> > tests on the dashboard that might need attention.
>> >
>> > Mike
>> >
>>
>


Re: [DISCUSS] 2.1.1 Release Plans?

2018-10-08 Thread Mike Drob
Thanks for pointing me to that discussion, Duo. I've updated HBASE-19121 to
have fix version 2.1.1 and 2.0.3 so that it is easier (at least for me) to
find via Jira.

I'll start looking at the hbck2 patches and will leave comments there.

There are a lot of subtasks against that issue, it is unclear to me which
ones are critical and which ones are not for a release.

Since hbck2 tool is in a different repo, have we thought about what the
release actions would look like? Do we need to have two votes going at the
same time? Can we release core repo with an hbck service in master, but
release the hbck tool sometime later? This first release is going to be
more complex I think; would be good if somebody can figure that out in
parallel so we're not blocked on it later.

Mike

On Fri, Oct 5, 2018 at 6:41 PM 张铎(Duo Zhang)  wrote:

> Stack has a plan on the 2.1.1 release where we want to finish the first
> version on hbck2. In the real deploy we have met a stuck cluster several
> times, and lots of users have asked that why hbck can not work any more...
>
> So the current opening issue is not important, please help reviewing the
> patches for hbck2 to speed up the release...
>
> Thanks for bringing this up
>
> Mike Drob 于2018年10月5日 周五23:53写道:
>
> > Devs,
> >
> > It's been almost 3 months since 2.1.0 was released (Jul 19) and we have
> 150
> > commits on branch-2.1 in that time. What do folks think of getting a
> > release going? I know that there's been some discussion around the HBCK2
> > stuff landing, but I feel like the conversation has gotten a bit lost
> > without an actual release to relate to.
> >
> > Duo, as the 2.1.0 release manager, are you interested in maintaining the
> > 2.1 branch release cadence? If you've gotten busy, then let's find
> another
> > volunteer.
> >
> > There are 18 issues open or in progress currently. Only one is labelled
> > blocker, and five more are critical -- let's evaluate these and the rest
> to
> > figure out what we need for a release to happen. I went ahead and
> created a
> > 2.1.2 version in Jira so that we have somewhere to move issues that
> aren't
> > getting done soon.
> >
> > Meanwhile, I think we also need to look at test stabilization -- there's
> 15
> > tests on the dashboard that might need attention.
> >
> > Mike
> >
>


[jira] [Created] (HBASE-21279) Split TestAdminShell into several tests

2018-10-08 Thread Ted Yu (JIRA)
Ted Yu created HBASE-21279:
--

 Summary: Split TestAdminShell into several tests
 Key: HBASE-21279
 URL: https://issues.apache.org/jira/browse/HBASE-21279
 Project: HBase
  Issue Type: Test
Reporter: Ted Yu


In the flaky test board, TestAdminShell often timed out 
(https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html).

I ran the test on Linux with SSD and reproduced the timeout (see attached test 
output).
{code}
2018-10-08 02:36:09,146 DEBUG [main] hbase.HBaseTestingUtility(351): Setting 
hbase.rootdir to 
/mnt/disk2/a/2-hbase/hbase-shell/target/test-data/a103d8e4-695c-a5a9-6690-1ef2580050f9
...
2018-10-08 02:49:09,093 DEBUG 
[RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=7] 
master.MasterRpcServices(1171): Checking to see if procedure is done pid=871
Took 0.7262 seconds2018-10-08 02:49:09,324 DEBUG [PEWorker-1] 
util.FSTableDescriptors(684): Wrote into 
hdfs://localhost:43859/user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-   
d935316c9241/.tmp/data/default/hbase_shell_tests_table/.tabledesc/.tableinfo.01
2018-10-08 02:49:09,328 INFO  
[RegionOpenAndInitThread-hbase_shell_tests_table-1] regionserver.HRegion(7004): 
creating HRegion hbase_shell_tests_table HTD == 
'hbase_shell_tests_table', {NAME => 'x', VERSIONS => '5', EVICT_BLOCKS_ON_CLOSE 
=> 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE', 
  CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', 
TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 
'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', 
CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', 
COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'},  {NAME 
=> 'y', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR 
=> 'false', KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',  
DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', 
REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 
'false', IN_MEMORY => 'false',  CACHE_BLOOMS_ON_WRITE => 'false', 
PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 
'true', BLOCKSIZE => '65536'} RootDir = hdfs://localhost:43859/
user/hbase/test-data/cefc73d9-cc37-d2a6-b92b-d935316c9241/.tmp Table name == 
hbase_shell_tests_table
^[[38;5;226mE^[[0m
===
Error: ^[[48;5;16;38;5;226;1mtest_Get_simple_status(Hbase::StatusTest)^[[0m: 
Java::JavaIo::InterruptedIOException: Interrupt while waiting on Operation: 
CREATE, Table Name:  default:hbase_shell_tests_table, procId: 871
2018-10-08 02:49:09,361 INFO  [Block report processor] 
blockmanagement.BlockManager(2645): BLOCK* addStoredBlock: blockMap updated: 
127.0.0.1:41338 is added to   
blk_1073742193_1369{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-ecc89143-e0a5-4a1c-b552-120be2561334:NORMAL:127.0.0.1:
   41338|RBW]]} size 58
> TEST TIMED OUT. PRINTING THREAD DUMP. <
{code}
We can see that the procedure #871 wasn't stuck - the timeout cut in and 
stopped the test.

We should separate the current test into two (or more) test files (with 
corresponding .rb) so that the execution time consistently would not exceed 
limit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HBASE-21278) TestMergeTableRegionsProcedure is flaky

2018-10-08 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-21278:
-

 Summary: TestMergeTableRegionsProcedure is flaky
 Key: HBASE-21278
 URL: https://issues.apache.org/jira/browse/HBASE-21278
 Project: HBase
  Issue Type: Bug
Reporter: Duo Zhang


https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/

I think the problem is

{noformat}
2018-10-08 03:44:30,315 INFO  [PEWorker-1] 
procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS, 
hasLock=false; TransitRegionStateProcedure 
table=testRollbackAndDoubleExecution, region=9bac7c539ac0cff6dc5706ed375a3bfb, 
UNASSIGN checking lock on 9bac7c539ac0cff6dc5706ed375a3bfb
2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159): 
CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS, 
hasLock=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, 
region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
java.lang.UnsupportedOperationException
at 
org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
at 
org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
at 
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
at 
org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
{noformat}

Typically there is no rollback for TRSP. Need to dig more.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Update compatibility matrices

2018-10-08 Thread Peter Somogyi
Hi,

For the 1.4.8 vote I checked the Hadoop compatibility matrix [1] in the
reference guide which misses HBase-1.4.x release. I assume the supported
versions match with HBase-1.5.x, but I think we should add this explicitly
to the matrix. Are you aware of any differences compared to 1.5 release?

Related to this, would be good to update the JDK matrix as well, add the
missing versions and also include JDK 11 there since it just reached
general availability.

This is partially related to HBASE-21091 but I wanted to bring this up here
to reach wider audience.

Regards,
Peter

[1] https://hbase.apache.org/book.html#hadoop


[jira] [Created] (HBASE-21277) prevent to add same table to two sync replication peer's config

2018-10-08 Thread Guanghao Zhang (JIRA)
Guanghao Zhang created HBASE-21277:
--

 Summary: prevent to add same table to two sync replication peer's 
config
 Key: HBASE-21277
 URL: https://issues.apache.org/jira/browse/HBASE-21277
 Project: HBase
  Issue Type: Sub-task
Reporter: Guanghao Zhang


If a table in two sync replication peer's config, it need write wal to three 
places: local dir and two remote dir. It is not allowed. Need to add check when 
add sync replication peer or modify sync replication peer's config.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HBASE-21276) hbase scan operation cannot scan some rowkey

2018-10-08 Thread xiaolerzheng (JIRA)
xiaolerzheng created HBASE-21276:


 Summary: hbase scan operation  cannot scan some rowkey
 Key: HBASE-21276
 URL: https://issues.apache.org/jira/browse/HBASE-21276
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.1.3
Reporter: xiaolerzheng


the table ZEUS.LOAN_CONSUMER_CONTACT has some row, we can get the row from 
"get",

but we cannot get the row from "scan"  

hbase(main):001:0> get 'ZEUS.LOAN_CONSUMER_CONTACT', '72520206#KB 2 
83#13971083626'
COLUMN CELL
 0:BINLOG_TIME timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 0:CREATE_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:LAST_MOD_TIME timestamp=1537254107291, value=2018-09-18 15:01:46
 0:PHONE_NO timestamp=1537254107291, value=13971083626
 0:SOURCE timestamp=1537254107291, value=\x80\x00\x00\x01
 0:UID timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 0:USER_NAME timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 0:_0 timestamp=1537254107291, value=x
8 row(s) in 0.2280 seconds

 

 

hbase(main):002:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ TIMERANGE => 
[1537254107291, 1537254107293]}
ROW COLUMN+CELL
0 row(s) in 1410.9010 seconds

hbase(main):003:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ TIMERANGE => 
[1537254107280, 1537254107294]}
ROW COLUMN+CELL
0 row(s) in 1410.5480 seconds

 

hbase(main):004:0> scan 'ZEUS.LOAN_CONSUMER_CONTACT', \{ STARTROW => 
'72520206#KB 2 83#13971083626', TIMERANGE => [1537254107280, 1537254107294]}
ROW COLUMN+CELL
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:BINLOG_TIME, 
timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:CREATE_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:LAST_MOD_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:PHONE_NO, 
timestamp=1537254107291, value=13971083626
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:SOURCE, 
timestamp=1537254107291, value=\x80\x00\x00\x01
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:UID, 
timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:USER_NAME, 
timestamp=1537254107291, 
value=KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\xA5\xE9\xB3\x8583
 xA5\xE9\xB3\x8583#13971083626
 72520206#KB\xE6\x9D\x8E\xE6\x80\xBB2\xE6\xB3\ column=0:_0, 
timestamp=1537254107291, value=x
 xA5\xE9\xB3\x8583#13971083626
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:BINLOG_TIME, 
timestamp=1537254107291, value=\x80\x00\x01e\xEB|%I
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:CREATE_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:LAST_MOD_TIME, 
timestamp=1537254107291, value=2018-09-18 15:01:46
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:PHONE_NO, 
timestamp=1537254107291, value=18694056338
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:SOURCE, 
timestamp=1537254107291, value=\x80\x00\x00\x01
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:UID, 
timestamp=1537254107291, value=\x80\x00\x00\x00\x04R\x92\x0E
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:USER_NAME, 
timestamp=1537254107291, value=K\xE6\x96\x8C\xE5\x93\xA5
 8
 72520206#K\xE6\x96\x8C\xE5\x93\xA5#1869405633 column=0:_0, 
timestamp=1537254107291, value=x



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)