Re: [VOTE] First release candidate for HBase 1.1.13 (RC0) is available

[Nick Dimiduk]

+1

- verified tarballs vs public key on people.apache.org.
- extracted bin tgz:
  - inspect structure. look good.
  - with jdk1.8.0_65:
- run LoadTestTool against standalone bin tgz with FAST_DIFF block
encoder and ROWCOL blooms. No issues, logs look good.
- poked around webUI. looks good.
  - load the site, browsed book.
- extracted src tgz:
  - inspect structure. look good.
  - run LoadTestTool against standalone built from src tgz with FAST_DIFF
block encoder and ROWCOL blooms. No issues, logs look good.
  - poked around webUI. looks good.
- ran the hbase-downstreamer project vs. the staged maven repository. tests
pass.

On Thu, Dec 7, 2017 at 1:44 PM, Ted Yu  wrote:

> +1
>
> Checked sums and signatures: ok
> Ran unit tests: passed
> Started standalone cluster and did some basic operations
>
> On Thu, Dec 7, 2017 at 1:14 PM, Andrew Purtell 
> wrote:
>
> > +1
> >
> > Checked sums and signatures: ok
> > Checked compat report: ok
> > RAT check passed: ok (7u80)
> > Built from source: ok (7u80)
> > Unit tests pass: ok (8u131)
> > 1M row LTT: ok (8u131)
> >
> >
> > On Thu, Dec 7, 2017 at 8:40 AM, Nick Dimiduk 
> wrote:
> >
> > > No one has voted a binding -1 with actionable changes, so as far as I'm
> > > concerned this RC remains valid. If people need more time, we can
> extend
> > > this vote.
> > >
> > > Thanks,
> > > Nick
> > >
> > > On Thu, Dec 7, 2017 at 8:07 AM, Ted Yu  wrote:
> > >
> > > > Nick:
> > > > Originally you set tomorrow as deadline.
> > > >
> > > > Is there a new RC coming out (w.r.t. Mike's comment) ?
> > > >
> > > > Cheers
> > > >
> > > > On Mon, Dec 4, 2017 at 8:37 PM, Nick Dimiduk 
> > > wrote:
> > > >
> > > > > Mike:
> > > > >
> > > > > > Do you plan to make a human-readable set of release notes in
> > addition
> > > > to
> > > > > the list of JIRA issues resolved?
> > > > >
> > > > > Not as such. For all branch-1.1 releases, I've written up a little
> > > > > human-friendly summary in the ANNOUNCE email. Basically, expanding
> on
> > > the
> > > > > list of JIRA tickets I highlight in the RC notes to include their
> > full
> > > > > ticket summaries. I haven't followed the details of the branch-1.4
> > > > release
> > > > > line, so I'm not sure what additional information you might be
> hoping
> > > > for.
> > > > >
> > > > > > tar missing hbase-native-client (present in tag)
> > > > >
> > > > > That's been the case since rel/1.1.0. We as a community have never
> > > > shipped
> > > > > a binary native client in this release line and we've never claimed
> > > that
> > > > > the native sources packaged herein are ready for production
> > > consumption.
> > > > > They probably should have been dropped from the branch before
> initial
> > > > > release, but that was not done. I have no objection to dropping
> them
> > > > from a
> > > > > branch-1.1 release; from the git log, I see no commit activity to
> > that
> > > > > module since Jan 2014. I don't see any of this as a blocker for
> this
> > > RC.
> > > > >
> > > > > > WARNING! HBase file layout needs to be upgraded ...
> > > > >
> > > > > When I test these RC's on a Mac, I explicitly set hbase.tmp.dir to
> a
> > > > > location specific to the candidate I've unpacked. This has the
> > benefit
> > > > > avoiding cross-version conflicts and other weirdness of Mac tmp
> > > directory
> > > > > management. For instance,
> > > > >
> > > > > 
> > > > >
> > > > > hbase.tmp.dir/tmp/hbase-1.1.
> > > > > 13/tmp
> > > > > 
> > > > >
> > > > > Peter:
> > > > >
> > > > > > In the logs I saw this line. Source code repository URL looks
> > > > incorrect.
> > > > > > 2017-12-04 10:13:27,028 INFO  [main] util.VersionInfo: Source
> code
> > > > > repository *git://diocles.local/Volumes/hbase-1.1.13/hbase*
> > > > > revision=c64bf8a9f35352cd504f2b8f4b02f9148cf45ab6
> > > > >
> > > > > Looking through the log, HBASE-16538 /
> > > > > 851c89af6ef9a78e2e3bc9ad3153367e85731c81 looks suspicious. It
> looks
> > > like
> > > > > that change first shipped in 1.1.7. Indeed, I see the equivelant
> line
> > > in
> > > > > the binary release of 1.1.12.
> > > > >
> > > > > On Mon, Dec 4, 2017 at 4:12 PM, Stack  wrote:
> > > > >
> > > > > > On Mon, Dec 4, 2017 at 11:52 AM, Andrew Purtell <
> > apurt...@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > > I think this can be fixed by using `git archive` to generate
> > the
> > > > src
> > > > > > tar
> > > > > > > instead of the src assembly.
> > > > > > >
> > > > > > > We could update the make_rc.sh script to create the source
> > tarball
> > > in
> > > > > > this
> > > > > > > way. Would you be willing to make a patch for that?
> > > > > > >
> > > > > > >
> > > > > > (I think you fellows already figured this going by JIRA
> > movement)
> > > > > >
> > > > > > HBASE-19152 "Update refguide 'how to build an RC' and the
> > make_rc.sh
> > > > > > script" changed 

[jira] [Created] (HBASE-19470) Compaction state in Table web UI is not right when table is disabled

[Guanghao Zhang (JIRA)]

Guanghao Zhang created HBASE-19470:
--

 Summary: Compaction state in Table web UI is not right when table 
is disabled
 Key: HBASE-19470
 URL: https://issues.apache.org/jira/browse/HBASE-19470
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.4.0
Reporter: Guanghao Zhang
Priority: Trivial


Table Attributes
Attribute Name  Value   Description
Enabled false   Is the table enabled
Compaction  sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)java.lang.reflect.Constructor.newInstance(Constructor.java:423)org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:95)org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:85)org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:371)org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:330)org.apache.hadoop.hbase.client.HBaseAdmin.getCompactionState(HBaseAdmin.java:3455)org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:283)org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:109)javax.servlet.http.HttpServlet.service(HttpServlet.java:820)org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1432)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)org.mortbay.jetty.Server.handle(Server.java:326)org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
 Unknown 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[VOTE] The second HBase 1.4.0 release candidate (RC1) is available

[Andrew Purtell]

The second HBase 1.4.0​ release candidate (RC1) is available for download
at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC1/ and Maven
artifacts are available in the temporary repository
https://repository.apache.org/content/repositories/orgapachehbase-1186/ .

The git tag corresponding to the candidate is '1.4.0RC1' (10b9b9fae6).

A detailed source and binary compatibility report for this release is
available for your review at
https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC1/hbase-1.3.1-1.4.0RC1_compatibility_report.html
​. All reported compatibility issues should comply with policy but please
review them carefully.

A list of the 660 issues resolved in this release can be found at
https://s.apache.org/OErT .

The changes since RC0 are:

* [HBASE-19180] - Remove unused imports from AlwaysPasses
* [HBASE-19373] - Fix Checkstyle error in hbase-annotations
* [HBASE-19435] - Reopen Files for ClosedChannelException in BucketCache
* [HBASE-19465] - Required httpcore and httpclient jars not included in
binary distribution
* [HBASE-19467] - rsgroups shell commands don't properly display
elapsed time

Please try out the candidate and vote +1/0/-1.

This vote will be open for at least 72 hours. Unless objection I will try
to close it ​Monday December 18, 2017 if we have sufficient votes.

Prior to making this announcement I made the following preflight checks to
1.4.0RC0 (3d571827cb):

RAT check passes (7u80)
Unit test suite passes (8u131)
LTT load 1M rows with 100% verification and 20% updates (8u131)
PE randomWrite, randomRead, scanRange100 (8u131)
100M rows ITBLL (8u131)

and the following preflight checks to 1.4.0RC1 (10b9b9fae6):

RAT check passes (7u80)
Unit test suite passes (8u131)

Between now and when I want to close the vote I'll write up human readable
release notes for the release announcement as promised.

I also have agreed to run a scale ITBLL test, a performance comparison with
1.2 using YCSB, a  performance comparison with 1.2 using PE, and an
analysis of code and allocation hot spot changes from 1.2, all of which I
will publish when available and factor in to my vote.

-- 
Best regards,
Andrew

[jira] [Created] (HBASE-19469) Review Of BackupSystemTable

[BELUGA BEHR (JIRA)]

BELUGA BEHR created HBASE-19469:
---

 Summary: Review Of BackupSystemTable
 Key: HBASE-19469
 URL: https://issues.apache.org/jira/browse/HBASE-19469
 Project: HBase
  Issue Type: Improvement
  Components: hbase
Affects Versions: 3.0.0
Reporter: BELUGA BEHR
Priority: Trivial


* Remove superfluous logging guards
* Fix spelling mistake in log message
* Fixed some DEBUG log messages that were guarded by an _isTraceEnabled_
* Use more Java auto-close functionality
* Some code simplifications
* Use better data structures for merge/disjoin



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: HBase Outage - Drop table operation stuck in "DELETE_TABLE_PRE_OPERATION"

[Ted Yu]

>From the line number of ProcedureSyncWait.java, it seems you are using
1.2.x release.

Can you check master log prior to 2017-12-08 18:59 ?
Pastebin relevant master log snippet (after necessary redaction).

Once we see the master log, we can see what might cause the
DeleteTableProcedure
to be stuck.

bq. Why rebalancing or other rest of operations are stuck?

If there is region in transition, the balancer wouldn't run.

"hbck –repair" combines many fixes. Normally admin is supposed to analyze
the particular inconsistencies before issuing proper fix.

Cheers

On Fri, Dec 8, 2017 at 1:50 PM, Murthy boddu  wrote:

> Hi,
>
>
>
> We recently ran into a production issue, here is the summary of events that
> we went through, in timeline order:
>
>
>
>1. One of the region servers went down (it became inaccessible)
>2. Region transition initiated, some regions of multiple tables were
>stuck in transition status. Most of them are in status “OPEN_FAIILED” or
>“OPENING” or “PENDING”, “CLOSE_FAILED”
>3. Client requests to those tables are still being diverted to lost
>server causing failures/time outs. (Which can we do about it ?)
>4. After waiting for many hours, we ran hbck –repair per table which
>resolved issues with some of them.
>5. One table, whose data can get stale in hours, we planned to recreate
>it to avoid any corruption. Disabling of table went through fine but
>dropping the table stuck at state “DELETE_TABLE_PRE_OPERATION”, it is
>waiting for regions in transition to finish. The regions it is
> complaining
>is in “OPENING” status.
>
> Here is the exception:
>
>
>
> 2017-12-08 18:59:17,975 WARN  [ProcedureExecutor-10]
> procedure.DeleteTableProcedure: Retriable error trying to delete
> table=Queue-SCKAD state=DELETE_TABLE_PRE_OPERATION
>
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timed out while
> waiting on regions
> Queue-SCKAD,B19,1502479054304.15a44cf47634d7d2264eaf00d61f6036. in
> transition
>
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(
> ProcedureSyncWait.java:123)
>
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(
> ProcedureSyncWait.java:103)
>
>
>
>1. This operation has been running for more than 24 hours and doesn’t
>time out (isn't there a 2 hour timeout for client operations at HBase
> level
>? ). Enabling the table back also queues up with no progress.
>2. Because the table is in disable status, running hbck isn’t helping as
>it says regions = 0.
>3. We added new node to the cluster to replace the old one, we see that
>HBase balancer doesn’t kick in at all. So, basically, region movement is
>totally stuck.
>4. No missing data on HDFS, 100% consistent. Hbck detail report on whole
>cluster also returns OK.
>
>
>
> I can provide additional logs if you request, but can you suggest how we
> can resolve this problem with the cluster? Does restarting hbase master
> process would help? We can’t afford another outage on the cluster making
> the situation tricky.
>
>
>
> My questions:
>
>
>
>1. Why drop operation need to wait for regions in transition to finish?
>Is there a way we can abort the on-going region movement or even the
> drop
>operation?
>2. Why rebalancing or other rest of operations are stuck?
>3.  Can you please suggest what action can be taken to resolve this?
>
>
>
> Thank you for your time and help.
>
>
>
> Regards
>

[jira] [Resolved] (HBASE-19467) rsgroups shell commands don't properly display elapsed time

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved HBASE-19467.

Resolution: Fixed

Pushed trivial changes to branch-1.4 and branch-1 after manual verification.

> rsgroups shell commands don't properly display elapsed time
> ---
>
> Key: HBASE-19467
> URL: https://issues.apache.org/jira/browse/HBASE-19467
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 1.4.0
>
> Attachments: HBASE-19467-branch-1.patch
>
>
> hbase(main):001:0> list_rsgroups
> GROUPS
> default
> 1 row(s) in 1512748253.9920 seconds
> hbase(main):002:0> add_rsgroup 'my_group'
> hbase(main):003:0> list_rsgroups
> GROUPS
> default
> my_group
> 2 row(s) in 1512748276.9400 seconds



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19468) FNFE during scans and flushes

[Thiruvel Thirumoolan (JIRA)]

Thiruvel Thirumoolan created HBASE-19468:


 Summary: FNFE during scans and flushes
 Key: HBASE-19468
 URL: https://issues.apache.org/jira/browse/HBASE-19468
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 1.3.1
Reporter: Thiruvel Thirumoolan
Priority: Minor


We see FNFE exceptions on our 1.3 clusters when scans and flushes happen at the 
same time. This causes regionserver to throw a UnknownScannerException and 
client retries.

This happens during the following sequence:

1. Scanner open, client fetched some rows from regionserver and working on it
2. Flush happens and storeScanner is updated with flushed files 
(StoreScanner.updateReaders())
3. Compaction discharger runs and cleans up the newly flushed file as we don't 
have new scanners on it yet.
4. Client issues scan.next and during StoreScanner.resetScannerStack(), we get 
a FNFE. RegionServer throws a UnknownScannerThe client retries in 1.3. With 
branch-1.4, the scan fails with a DoNotRetryIOException.

[~ram_krish], My proposal is to increment the reader count during 
updateReaders() and decrement it during resetScannerStack(), so discharger 
doesn't clean it up. Scan lease expiries also have to be taken care of. Am I 
missing anything? Is there a better approach?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

HBase Outage - Drop table operation stuck in "DELETE_TABLE_PRE_OPERATION"

[Murthy boddu]

Hi,



We recently ran into a production issue, here is the summary of events that
we went through, in timeline order:



   1. One of the region servers went down (it became inaccessible)
   2. Region transition initiated, some regions of multiple tables were
   stuck in transition status. Most of them are in status “OPEN_FAIILED” or
   “OPENING” or “PENDING”, “CLOSE_FAILED”
   3. Client requests to those tables are still being diverted to lost
   server causing failures/time outs. (Which can we do about it ?)
   4. After waiting for many hours, we ran hbck –repair per table which
   resolved issues with some of them.
   5. One table, whose data can get stale in hours, we planned to recreate
   it to avoid any corruption. Disabling of table went through fine but
   dropping the table stuck at state “DELETE_TABLE_PRE_OPERATION”, it is
   waiting for regions in transition to finish. The regions it is complaining
   is in “OPENING” status.

Here is the exception:



2017-12-08 18:59:17,975 WARN  [ProcedureExecutor-10]
procedure.DeleteTableProcedure: Retriable error trying to delete
table=Queue-SCKAD state=DELETE_TABLE_PRE_OPERATION

org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timed out while
waiting on regions
Queue-SCKAD,B19,1502479054304.15a44cf47634d7d2264eaf00d61f6036. in
transition

at
org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(ProcedureSyncWait.java:123)

at
org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitFor(ProcedureSyncWait.java:103)



   1. This operation has been running for more than 24 hours and doesn’t
   time out (isn't there a 2 hour timeout for client operations at HBase level
   ? ). Enabling the table back also queues up with no progress.
   2. Because the table is in disable status, running hbck isn’t helping as
   it says regions = 0.
   3. We added new node to the cluster to replace the old one, we see that
   HBase balancer doesn’t kick in at all. So, basically, region movement is
   totally stuck.
   4. No missing data on HDFS, 100% consistent. Hbck detail report on whole
   cluster also returns OK.



I can provide additional logs if you request, but can you suggest how we
can resolve this problem with the cluster? Does restarting hbase master
process would help? We can’t afford another outage on the cluster making
the situation tricky.



My questions:



   1. Why drop operation need to wait for regions in transition to finish?
   Is there a way we can abort the on-going region movement or even the drop
   operation?
   2. Why rebalancing or other rest of operations are stuck?
   3.  Can you please suggest what action can be taken to resolve this?



Thank you for your time and help.



Regards

[jira] [Resolved] (HBASE-19465) Required httpcore and httpclient jars not included in binary distribution

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved HBASE-19465.

Resolution: Fixed

Pushed trivial change to branch-1 and branch-1.4

> Required httpcore and httpclient jars not included in binary distribution
> -
>
> Key: HBASE-19465
> URL: https://issues.apache.org/jira/browse/HBASE-19465
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0
>
> Attachments: HBASE-19465-branch-1.patch
>
>
> The httpcore and httpclient dependency jars are not included into the binary 
> distribution. This means MR/YARN jobs will fail if launched from the binary 
> tarball as distributed, including those running in "local mode", e.g. ITBLL:
> {noformat}
> 2017-12-08 22:40:36,134 INFO  [LocalJobRunner Map Task Executor #0] 
> mapred.LocalJobRunner: Finishing task: attempt_local936234315_0001_m_03_0
> 2017-12-08 22:40:36,134 INFO  [Thread-23] mapred.LocalJobRunner: map task 
> executor complete.
> 2017-12-08 22:40:36,147 WARN  [Thread-23] mapred.LocalJobRunner: 
> job_local936234315_0001
> java.lang.NoClassDefFoundError: org/apache/http/client/methods/HttpUriRequest
>   at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:546)
> Caused by: java.lang.ClassNotFoundException: 
> org.apache.http.client.methods.HttpUriRequest
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>   ... 1 more
> Exception in thread "Thread-23" java.lang.NoClassDefFoundError: 
> org/apache/http/client/methods/HttpUriRequest
>   at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:562)
> Caused by: java.lang.ClassNotFoundException: 
> org.apache.http.client.methods.HttpUriRequest
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>   ... 1 more
> 2017-12-08 22:40:37,078 INFO  [main] mapreduce.Job: Job 
> job_local936234315_0001 failed with state FAILED due to: NA
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[CANCELLED] [VOTE] The first HBase 1.4.0 release candidate (RC0) is available

[Andrew Purtell]

Based on other feedback I filed HBASE-19467 and will fix it too.

On Fri, Dec 8, 2017 at 2:59 PM, Andrew Purtell  wrote:

> -1, see HBASE-19465
>
> RC1 coming as soon as I fix this.
>
> On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell 
> wrote:
>
>> The first HBase 1.4.0​ release candidate (RC0) is available for download
>> at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and
>> Maven artifacts are available in the temporary repository
>> https://repository.apache.org/content/repositories/orgapachehbase-1185/
>> .
>>
>> The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb).
>>
>> A detailed source and binary compatibility report for this release is
>> available for your review at https://dist.apache.org/repos/
>> dist/dev/hbase/hbase-1.4.0RC0/hbase-1.3.1-1.4.0RC0_compatibi
>> lity_report.html ​. All reported compatibility issues should comply with
>> policy but please review them carefully.
>>
>> A list of the ​652 issues resolved in this release can be found at
>> https://s.apache.org/OErT .
>>
>> Please try out the candidate and vote +1/0/-1.
>>
>> This vote will be open for at least 72 hours. Unless objection I will try
>> to close it ​Monday December 18, 2017 if we have sufficient votes.
>>
>> Prior to making this announcement I made the following preflight checks
>> to 1.4.0RC0 (3d571827cb):
>>
>>- RAT check passes (7u80)
>>- Unit test suite passes (8u131)
>>- LTT load 1M rows with 100% verification and 20% updates (8u131)
>>- PE randomWrite, randomRead, scanRange100 (8u131)
>>- 100M rows ITBLL (8u131)
>>
>> Between now and when I want to close the vote I'll write up human
>> readable release notes for the release announcement as promised. I also
>> have agreed to run a scale ITBLL test, a performance comparison with 1.2
>> using YCSB, a  performance comparison with 1.2 using PE, and an analysis of
>> code and allocation hot spot changes from 1.2, all of which I will publish
>> when available and factor in to my vote.
>>
>>
>> --
>> Best regards,
>> Andrew
>>
>>
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

[jira] [Created] (HBASE-19467) rsgroups shell commands don't properly display elapsed time

[Andrew Purtell (JIRA)]

Andrew Purtell created HBASE-19467:
--

 Summary: rsgroups shell commands don't properly display elapsed 
time
 Key: HBASE-19467
 URL: https://issues.apache.org/jira/browse/HBASE-19467
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.4.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Minor
 Fix For: 1.4.0


hbase(main):001:0> list_rsgroups
GROUPS
default
1 row(s) in 1512748253.9920 seconds

hbase(main):002:0> add_rsgroup 'my_group'

hbase(main):003:0> list_rsgroups
GROUPS
default
my_group
2 row(s) in 1512748276.9400 seconds



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available

[Andrew Purtell]

-1, see HBASE-19465

RC1 coming as soon as I fix this.

On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell  wrote:

> The first HBase 1.4.0​ release candidate (RC0) is available for download
> at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven
> artifacts are available in the temporary repository
> https://repository.apache.org/content/repositories/orgapachehbase-1185/ .
>
> The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb).
>
> A detailed source and binary compatibility report for this release is
> available for your review at https://dist.apache.org/repos/
> dist/dev/hbase/hbase-1.4.0RC0/hbase-1.3.1-1.4.0RC0_
> compatibility_report.html ​. All reported compatibility issues should
> comply with policy but please review them carefully.
>
> A list of the ​652 issues resolved in this release can be found at
> https://s.apache.org/OErT .
>
> Please try out the candidate and vote +1/0/-1.
>
> This vote will be open for at least 72 hours. Unless objection I will try
> to close it ​Monday December 18, 2017 if we have sufficient votes.
>
> Prior to making this announcement I made the following preflight checks to
> 1.4.0RC0 (3d571827cb):
>
>- RAT check passes (7u80)
>- Unit test suite passes (8u131)
>- LTT load 1M rows with 100% verification and 20% updates (8u131)
>- PE randomWrite, randomRead, scanRange100 (8u131)
>- 100M rows ITBLL (8u131)
>
> Between now and when I want to close the vote I'll write up human readable
> release notes for the release announcement as promised. I also have agreed
> to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, a
> performance comparison with 1.2 using PE, and an analysis of code and
> allocation hot spot changes from 1.2, all of which I will publish when
> available and factor in to my vote.
>
>
> --
> Best regards,
> Andrew
>
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

[jira] [Created] (HBASE-19466) Rare failure in TestScannerCursor

[Andrew Purtell (JIRA)]

Andrew Purtell created HBASE-19466:
--

 Summary: Rare failure in TestScannerCursor
 Key: HBASE-19466
 URL: https://issues.apache.org/jira/browse/HBASE-19466
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.4.0
Reporter: Andrew Purtell
Priority: Minor
 Fix For: 1.4.1, 1.5.0


I think we just need to increase the timeout interval to deal with occasional 
slowdowns on test executors. 1998 ms is a pretty short timeout.

By the way "rpcTimetout" in the exception message is a misspelling.

[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 37.412 
s <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestScannerCursor
[ERROR] 
testHeartbeatWithSparseFilter(org.apache.hadoop.hbase.regionserver.TestScannerCursor)
  Time elapsed: 35.604 s  <<< ERROR!
org.apache.hadoop.hbase.client.RetriesExhaustedException: 
Failed after attempts=36, exceptions:
Thu Dec 07 22:27:16 UTC 2017, null, java.net.SocketTimeoutException: 
callTimeout=4000, callDuration=4108: Call to 
ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local 
exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, 
waitTime=2002, rpcTimetout=1998 row '' on table 'TestScannerCursor' at 
region=TestScannerCursor,,1512685598567.1d4e59215a881d6ccbd0b5b5bdec5587., 
hostname=ip-172-31-47-35.us-west-2.compute.internal,35690,1512685593244, 
seqNum=2

at 
org.apache.hadoop.hbase.regionserver.TestScannerCursor.testHeartbeatWithSparseFilter(TestScannerCursor.java:154)
Caused by: java.net.SocketTimeoutException: callTimeout=4000, 
callDuration=4108: Call to 
ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local 
exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, 
waitTime=2002, rpcTimetout=1998 row '' on table 'TestScannerCursor' at 
region=TestScannerCursor,,1512685598567.1d4e59215a881d6ccbd0b5b5bdec5587., 
hostname=ip-172-31-47-35.us-west-2.compute.internal,35690,1512685593244, 
seqNum=2
Caused by: java.io.IOException: Call to 
ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local 
exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, 
waitTime=2002, rpcTimetout=1998
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, 
waitTime=2002, rpcTimetout=1998




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19465) Required httpcore and httpclient jars not included in binary distribution

[Andrew Purtell (JIRA)]

Andrew Purtell created HBASE-19465:
--

 Summary: Required httpcore and httpclient jars not included in 
binary distribution
 Key: HBASE-19465
 URL: https://issues.apache.org/jira/browse/HBASE-19465
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.4.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 1.4.0


The httpcore and httpclient dependency jars are not included into the binary 
distribution. This means MR/YARN jobs will fail if launched from the binary 
tarball as distributed, including those running in "local mode", e.g. ITBLL:

{noformat}
2017-12-08 22:40:36,134 INFO  [LocalJobRunner Map Task Executor #0] 
mapred.LocalJobRunner: Finishing task: attempt_local936234315_0001_m_03_0
2017-12-08 22:40:36,134 INFO  [Thread-23] mapred.LocalJobRunner: map task 
executor complete.
2017-12-08 22:40:36,147 WARN  [Thread-23] mapred.LocalJobRunner: 
job_local936234315_0001
java.lang.NoClassDefFoundError: org/apache/http/client/methods/HttpUriRequest
at 
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:546)
Caused by: java.lang.ClassNotFoundException: 
org.apache.http.client.methods.HttpUriRequest
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
Exception in thread "Thread-23" java.lang.NoClassDefFoundError: 
org/apache/http/client/methods/HttpUriRequest
at 
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:562)
Caused by: java.lang.ClassNotFoundException: 
org.apache.http.client.methods.HttpUriRequest
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
2017-12-08 22:40:37,078 INFO  [main] mapreduce.Job: Job job_local936234315_0001 
failed with state FAILED due to: NA
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Moving To SLF4J and Log4J2

[dam6923 .]

Hello Team,

Maybe it's time to finally realize the dream of HBASE-10092

Once the logging infrastructure is upgraded, I would be happy to
assist in reviewing and improving the log messages.  I've done a bit
of similar work for the Hive project.  It's tedious but doesn't
require deep knowledge of the overall project and can be done
piecemeal.

Commons Logging states that it requires one to use code guards for
logging debug/trace statements.  However, SLF4J recommends using
parameters for logging which is faster than guards, and I personally
think it's a cleaner approach.  The logging parameters are only
resolved if the log level is appropriate and therefore guards are
often unnecessary. It removes code (the guards) and complexity from
the application code.

This seems like a straight-forward path for the project for eking out
some performance.

https://www.slf4j.org/faq.html#logging_performance
https://commons.apache.org/proper/commons-logging/guide.html#Code_Guards

Thanks.

[jira] [Created] (HBASE-19464) Replace StringBuffer with StringBuilder for hbase-common

[BELUGA BEHR (JIRA)]

BELUGA BEHR created HBASE-19464:
---

 Summary: Replace StringBuffer with StringBuilder for hbase-common
 Key: HBASE-19464
 URL: https://issues.apache.org/jira/browse/HBASE-19464
 Project: HBase
  Issue Type: Improvement
  Components: hbase
Affects Versions: 2.0.0
Reporter: BELUGA BEHR
Priority: Trivial


Replace {{StringBuffer}} with non-synchronized version {{StringBuilder}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)

[Stack]

Thanks Andrew. I disabled the job. Use the nightly going forward. The jdk7
builds seem to run fine. The jdk8 has some timeout going on. Need to dig
in. You can see here:

https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.4/

Thanks,
M

On Fri, Dec 8, 2017 at 11:29 AM, Andrew Purtell  wrote:

> Ok with me, Stack. Thanks for asking.
>
>
> On Thu, Nov 30, 2017 at 5:33 PM, Stack  wrote:
>
> > On the move over to nightly test runs:
> >
> > 1.2 nightly had a successful build last night after the branch-1
> > stabilization effort (HBASE-19204) and fixing a few unit test failures.
> See
> > build 150
> > https://builds.apache.org/view/H-L/view/HBase/job/HBase%
> > 20Nightly/job/branch-1.2/
> > It then failed, 151, because of timed out test. Need to dig in. Clean up
> a
> > few more unit tests and branch-1.2 is probably ready for a
> release-cutting.
> >
> > 1.3 has a few flakies. The last build failed because of:
> >
> >   Test Result (1 failure / ±0)
> > org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation.
> > testCFKeyRotation
> >
> > Just a little effort should turn 1.3 green.
> >
> > I was going to disable the 1.4 job,
> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.4/,  in favor
> of
> > the 1.4 nightly,
> > https://builds.apache.org/view/H-L/view/HBase/job/HBase%
> > 20Nightly/job/branch-1.4/,
> > if ok w/ you Andrew Purtell... And move over the branch-1, branch-2, and
> > master too.
> >
> > Thanks,
> > S
> >
> >
> >
> > On Wed, Nov 29, 2017 at 8:06 AM, Stack  wrote:
> >
> > > Example of the new nice reporting: vhttps://builds.apache.org/
> > > view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/
> > > S
> > >
> > > On Wed, Nov 29, 2017 at 8:06 AM, Stack  wrote:
> > >
> > >> Note that I have disabled the HBase-1.2-JDK7, HBase-1.2-JDK8,
> > >> HBase-1.3-JDK7, and HBase-1.3-JDK8 jobs. They have been broken for a
> > good
> > >> while now. In their place, refer to an ongoing Sean "Nightly" project,
> > an
> > >> effort he has been at for a while. It does more checking with pretty
> > >> reports that will help figuring general stability over time. See under
> > >> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/
> > >> See the nightly builds for 1.2 and 1.3. They have some teething issues
> > >> still but are almost there. See the 1.2 build from last night. In
> recent
> > >> days, the 1.2 branch went from trash-can fire to stable. See how all
> > tests
> > >> passed in the last build but then we failed generating the src bundle
> on
> > >> the end (this is what I mean by 'teething' issue). Will work on fixing
> > this
> > >> last step and moving over 1.4, etc., in the next few days.
> > >>
> > >> FYI,
> > >> St.Ack
> > >>
> > >>
> > >> On Tue, Nov 7, 2017 at 7:45 AM, Stack  wrote:
> > >>
> > >>> On Tue, Nov 7, 2017 at 6:10 AM, Sean Busbey 
> wrote:
> > >>>
> >  > Should I be able to see the machine dir when I look at nightlies
> >  output?
> >  > (Was trying to see what else is running).
> > 
> >  Ah. we don't have the same machine sampling on nightly as we do in
> >  precommit. I am 80% on a patch for HBASE-19189 (run test ad-hoc
> >  repeatedly)  that includes pulling that information gathering into a
> >  place where we could also use it in nightly.
> > 
> > 
> > >>> Sweet.
> > >>>
> > >>>
> > >>>
> >  Did we ever figure out how many cores we expect our tests to need?
> It
> >  looks like the Hadoop nodes have 8 cores. (with 2 executors that
> means
> >  4 is our fair share)
> > 
> > 
> > >>> At the end of the thread inquiry I suggested that we don't use enough
> > >>> cores, that we could up our fork counts and tests would complete in
> > less
> > >>> time. I wanted to experiment some w/ high fork counts -- 16 or so --
> > to see
> > >>> if concurrent running brought on  more failure.
> > >>>
> > >>> St.Ack
> > >>>
> > >>>
> > >>>
> > >>>
> >  On Tue, Nov 7, 2017 at 8:05 AM, Sean Busbey 
> > wrote:
> >  > surefire results get zipped up (we were filling the jenkins hosts
> > with
> >  > old test logs previously) and stored in a file called
> > "test_logs.zip"
> >  > for each jvm run. So if that happend in the jdk7 run for
> branch-1.2,
> >  > it'd be in artifacts -> output-jdk7 -> test_logs.zip.
> >  >
> >  > I don't know if the archival process grabs things from surefire
> that
> >  > aren't the surefire XML files, but we can update it to do so if it
> >  > doesn't.
> >  >
> >  > On Mon, Nov 6, 2017 at 11:39 PM, Stack  wrote:
> >  >> I see this in the 1.2 nightly just when it gives up the ghost
> >  >>
> >  >> [WARNING] Corrupted STDOUT by directly writing to native stream
> in
> >  >> forked JVM 2. See FAQ web page and the dump file
> >  >> 

Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)

[Andrew Purtell]

Ok with me, Stack. Thanks for asking.


On Thu, Nov 30, 2017 at 5:33 PM, Stack  wrote:

> On the move over to nightly test runs:
>
> 1.2 nightly had a successful build last night after the branch-1
> stabilization effort (HBASE-19204) and fixing a few unit test failures. See
> build 150
> https://builds.apache.org/view/H-L/view/HBase/job/HBase%
> 20Nightly/job/branch-1.2/
> It then failed, 151, because of timed out test. Need to dig in. Clean up a
> few more unit tests and branch-1.2 is probably ready for a release-cutting.
>
> 1.3 has a few flakies. The last build failed because of:
>
>   Test Result (1 failure / ±0)
> org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation.
> testCFKeyRotation
>
> Just a little effort should turn 1.3 green.
>
> I was going to disable the 1.4 job,
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.4/,  in favor of
> the 1.4 nightly,
> https://builds.apache.org/view/H-L/view/HBase/job/HBase%
> 20Nightly/job/branch-1.4/,
> if ok w/ you Andrew Purtell... And move over the branch-1, branch-2, and
> master too.
>
> Thanks,
> S
>
>
>
> On Wed, Nov 29, 2017 at 8:06 AM, Stack  wrote:
>
> > Example of the new nice reporting: vhttps://builds.apache.org/
> > view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/
> > S
> >
> > On Wed, Nov 29, 2017 at 8:06 AM, Stack  wrote:
> >
> >> Note that I have disabled the HBase-1.2-JDK7, HBase-1.2-JDK8,
> >> HBase-1.3-JDK7, and HBase-1.3-JDK8 jobs. They have been broken for a
> good
> >> while now. In their place, refer to an ongoing Sean "Nightly" project,
> an
> >> effort he has been at for a while. It does more checking with pretty
> >> reports that will help figuring general stability over time. See under
> >> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/
> >> See the nightly builds for 1.2 and 1.3. They have some teething issues
> >> still but are almost there. See the 1.2 build from last night. In recent
> >> days, the 1.2 branch went from trash-can fire to stable. See how all
> tests
> >> passed in the last build but then we failed generating the src bundle on
> >> the end (this is what I mean by 'teething' issue). Will work on fixing
> this
> >> last step and moving over 1.4, etc., in the next few days.
> >>
> >> FYI,
> >> St.Ack
> >>
> >>
> >> On Tue, Nov 7, 2017 at 7:45 AM, Stack  wrote:
> >>
> >>> On Tue, Nov 7, 2017 at 6:10 AM, Sean Busbey  wrote:
> >>>
>  > Should I be able to see the machine dir when I look at nightlies
>  output?
>  > (Was trying to see what else is running).
> 
>  Ah. we don't have the same machine sampling on nightly as we do in
>  precommit. I am 80% on a patch for HBASE-19189 (run test ad-hoc
>  repeatedly)  that includes pulling that information gathering into a
>  place where we could also use it in nightly.
> 
> 
> >>> Sweet.
> >>>
> >>>
> >>>
>  Did we ever figure out how many cores we expect our tests to need? It
>  looks like the Hadoop nodes have 8 cores. (with 2 executors that means
>  4 is our fair share)
> 
> 
> >>> At the end of the thread inquiry I suggested that we don't use enough
> >>> cores, that we could up our fork counts and tests would complete in
> less
> >>> time. I wanted to experiment some w/ high fork counts -- 16 or so --
> to see
> >>> if concurrent running brought on  more failure.
> >>>
> >>> St.Ack
> >>>
> >>>
> >>>
> >>>
>  On Tue, Nov 7, 2017 at 8:05 AM, Sean Busbey 
> wrote:
>  > surefire results get zipped up (we were filling the jenkins hosts
> with
>  > old test logs previously) and stored in a file called
> "test_logs.zip"
>  > for each jvm run. So if that happend in the jdk7 run for branch-1.2,
>  > it'd be in artifacts -> output-jdk7 -> test_logs.zip.
>  >
>  > I don't know if the archival process grabs things from surefire that
>  > aren't the surefire XML files, but we can update it to do so if it
>  > doesn't.
>  >
>  > On Mon, Nov 6, 2017 at 11:39 PM, Stack  wrote:
>  >> I see this in the 1.2 nightly just when it gives up the ghost
>  >>
>  >> [WARNING] Corrupted STDOUT by directly writing to native stream in
>  >> forked JVM 2. See FAQ web page and the dump file
>  >> /testptch/hbase/hbase-server/target/surefire-reports/2017-11
>  -06T20-11-30_219-jvmRun2.dumpstream
>  >>
>  >> .. but the pointed to dumpstream doesn't seem to be around post
>  build.
>  >> I am looking in wrong place?
>  >>
>  >>
>  >> Thanks,
>  >>
>  >> S
>  >>
>  >>
>  >> On Mon, Nov 6, 2017 at 8:20 PM, Stack  wrote:
>  >>
>  >>> On Mon, Nov 6, 2017 at 8:35 AM, Sean Busbey <
> sean.bus...@gmail.com>
>  wrote:
>  >>>
>   Given that all of the old post-commit tests have been posting
> that
>   they're failing 

[jira] [Created] (HBASE-19463) Make CPEnv#getConnection return a facade that throws Unsupported if CP calls #close

[stack (JIRA)]

stack created HBASE-19463:
-

 Summary: Make CPEnv#getConnection return a facade that throws 
Unsupported if CP calls #close
 Key: HBASE-19463
 URL: https://issues.apache.org/jira/browse/HBASE-19463
 Project: HBase
  Issue Type: Improvement
  Components: Coprocessors
Reporter: stack
 Fix For: 2.0.0-beta-2


Follows from HBASE-19301, a suggestion by [~zghaobac].

To prevent a CP accidentally closing the connection returned by 
CpEnv#getConnection -- which returns the hosting Servers Connection -- we 
should throw UnsupportedException if the CP calls #close Do it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available

[Andrew Purtell]

Thanks for taking a look Balazs. Please file a JIRA for that. It may be a 
cosmetic problem but if we have to spin another RC I want to be sure to fix it. 

> On Dec 8, 2017, at 3:09 AM, Balazs Meszaros  
> wrote:
> 
> Hi,
> 
> +1, because:
> 
> Signatures: ok
> Unit tests: ok (8u112)
> RSGroups: worked for me through the shell
> 
> -1, because:
> 
> hbase(main):005:0> list_rsgroups
> GROUPS
> 
> test
> 
> default
> 
> 2 row(s) in *1512730813.6580* seconds
> 
> Trust me, it was faster than 47 years :) Other listings (e.g.
> list_namespace) displayed the elapsed time correctly.
> 
> Best regards,
> Balazs
> 
>> On Fri, Dec 8, 2017 at 4:51 AM, Ted Yu  wrote:
>> 
>> +1
>> 
>> Checked signatures: good
>> Ran test suite: pass
>> 1M row LTT: pass
>> 
>> Cheers
>> 
>> On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell 
>> wrote:
>> 
>>> The first HBase 1.4.0​ release candidate (RC0) is available for download
>> at
>>> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven
>>> artifacts are available in the temporary repository
>>> https://repository.apache.org/content/repositories/orgapachehbase-1185/
>> .
>>> 
>>> The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb).
>>> 
>>> A detailed source and binary compatibility report for this release is
>>> available for your review at
>>> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/
>>> hbase-1.3.1-1.4.0RC0_compatibility_report.html
>>> ​. All reported compatibility issues should comply with policy but please
>>> review them carefully.
>>> 
>>> A list of the ​652 issues resolved in this release can be found at
>>> https://s.apache.org/OErT .
>>> 
>>> Please try out the candidate and vote +1/0/-1.
>>> 
>>> This vote will be open for at least 72 hours. Unless objection I will try
>>> to close it ​Monday December 18, 2017 if we have sufficient votes.
>>> 
>>> Prior to making this announcement I made the following preflight checks
>> to
>>> 1.4.0RC0 (3d571827cb):
>>> 
>>>   - RAT check passes (7u80)
>>>   - Unit test suite passes (8u131)
>>>   - LTT load 1M rows with 100% verification and 20% updates (8u131)
>>>   - PE randomWrite, randomRead, scanRange100 (8u131)
>>>   - 100M rows ITBLL (8u131)
>>> 
>>> Between now and when I want to close the vote I'll write up human
>> readable
>>> release notes for the release announcement as promised. I also have
>> agreed
>>> to run a scale ITBLL test, a performance comparison with 1.2 using YCSB,
>> a
>>> performance comparison with 1.2 using PE, and an analysis of code and
>>> allocation hot spot changes from 1.2, all of which I will publish when
>>> available and factor in to my vote.
>>> 
>>> 
>>> --
>>> Best regards,
>>> Andrew
>>> 
>> 

Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available

[Ted Yu]

I tried a few RS group commands:

--

hbase(main):001:0> list_rsgroups
GROUPS
default
1 row(s) in 1512748253.9920 seconds

hbase(main):002:0> add_rsgroup 'my_group'

hbase(main):003:0> list_rsgroups
GROUPS
default
my_group
2 row(s) in 1512748276.9400 seconds

--
The displayed duration was off but the commands actually returned fast.

I would say this is minor defect.

Cheers

On Fri, Dec 8, 2017 at 3:09 AM, Balazs Meszaros <
balazs.mesza...@cloudera.com> wrote:

> Hi,
>
> +1, because:
>
> Signatures: ok
> Unit tests: ok (8u112)
> RSGroups: worked for me through the shell
>
> -1, because:
>
> hbase(main):005:0> list_rsgroups
> GROUPS
>
> test
>
> default
>
> 2 row(s) in *1512730813.6580* seconds
>
> Trust me, it was faster than 47 years :) Other listings (e.g.
> list_namespace) displayed the elapsed time correctly.
>
> Best regards,
> Balazs
>
> On Fri, Dec 8, 2017 at 4:51 AM, Ted Yu  wrote:
>
> > +1
> >
> > Checked signatures: good
> > Ran test suite: pass
> > 1M row LTT: pass
> >
> > Cheers
> >
> > On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell 
> > wrote:
> >
> > > The first HBase 1.4.0​ release candidate (RC0) is available for
> download
> > at
> > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven
> > > artifacts are available in the temporary repository
> > > https://repository.apache.org/content/repositories/
> orgapachehbase-1185/
> > .
> > >
> > > The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb).
> > >
> > > A detailed source and binary compatibility report for this release is
> > > available for your review at
> > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/
> > > hbase-1.3.1-1.4.0RC0_compatibility_report.html
> > > ​. All reported compatibility issues should comply with policy but
> please
> > > review them carefully.
> > >
> > > A list of the ​652 issues resolved in this release can be found at
> > > https://s.apache.org/OErT .
> > >
> > > Please try out the candidate and vote +1/0/-1.
> > >
> > > This vote will be open for at least 72 hours. Unless objection I will
> try
> > > to close it ​Monday December 18, 2017 if we have sufficient votes.
> > >
> > > Prior to making this announcement I made the following preflight checks
> > to
> > > 1.4.0RC0 (3d571827cb):
> > >
> > >- RAT check passes (7u80)
> > >- Unit test suite passes (8u131)
> > >- LTT load 1M rows with 100% verification and 20% updates (8u131)
> > >- PE randomWrite, randomRead, scanRange100 (8u131)
> > >- 100M rows ITBLL (8u131)
> > >
> > > Between now and when I want to close the vote I'll write up human
> > readable
> > > release notes for the release announcement as promised. I also have
> > agreed
> > > to run a scale ITBLL test, a performance comparison with 1.2 using
> YCSB,
> > a
> > > performance comparison with 1.2 using PE, and an analysis of code and
> > > allocation hot spot changes from 1.2, all of which I will publish when
> > > available and factor in to my vote.
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> >
>

Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available

[Peter Somogyi]

+1

checksum, signature: ok
tests: ok
LTT with 1M rows: ok
basic operations from shell: ok
rat check: ok

On Thu, Dec 7, 2017 at 5:40 AM, Andrew Purtell  wrote:

> The first HBase 1.4.0​ release candidate (RC0) is available for download at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven
> artifacts are available in the temporary repository
> https://repository.apache.org/content/repositories/orgapachehbase-1185/ .
>
> The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb).
>
> A detailed source and binary compatibility report for this release is
> available for your review at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/
> hbase-1.3.1-1.4.0RC0_compatibility_report.html
> ​. All reported compatibility issues should comply with policy but please
> review them carefully.
>
> A list of the ​652 issues resolved in this release can be found at
> https://s.apache.org/OErT .
>
> Please try out the candidate and vote +1/0/-1.
>
> This vote will be open for at least 72 hours. Unless objection I will try
> to close it ​Monday December 18, 2017 if we have sufficient votes.
>
> Prior to making this announcement I made the following preflight checks to
> 1.4.0RC0 (3d571827cb):
>
>- RAT check passes (7u80)
>- Unit test suite passes (8u131)
>- LTT load 1M rows with 100% verification and 20% updates (8u131)
>- PE randomWrite, randomRead, scanRange100 (8u131)
>- 100M rows ITBLL (8u131)
>
> Between now and when I want to close the vote I'll write up human readable
> release notes for the release announcement as promised. I also have agreed
> to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, a
> performance comparison with 1.2 using PE, and an analysis of code and
> allocation hot spot changes from 1.2, all of which I will publish when
> available and factor in to my vote.
>
>
> --
> Best regards,
> Andrew
>

Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available

[Chia-Ping Tsai]

+1

Unit test suite passes (8u131)
hadoop-2.7.4 pass
hadoop-2.7.3 pass
hadoop-2.7.2 pass
hadoop-2.7.1 pass
hadoop-2.6.4 pass
hadoop-2.6.3 pass
hadoop-2.6.2 pass
hadoop-2.6.1 pass

On 2017-12-07 12:40, Andrew Purtell  wrote: 
> The first HBase 1.4.0​ release candidate (RC0) is available for download at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven
> artifacts are available in the temporary repository
> https://repository.apache.org/content/repositories/orgapachehbase-1185/ .
> 
> The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb).
> 
> A detailed source and binary compatibility report for this release is
> available for your review at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/hbase-1.3.1-1.4.0RC0_compatibility_report.html
> ​. All reported compatibility issues should comply with policy but please
> review them carefully.
> 
> A list of the ​652 issues resolved in this release can be found at
> https://s.apache.org/OErT .
> 
> Please try out the candidate and vote +1/0/-1.
> 
> This vote will be open for at least 72 hours. Unless objection I will try
> to close it ​Monday December 18, 2017 if we have sufficient votes.
> 
> Prior to making this announcement I made the following preflight checks to
> 1.4.0RC0 (3d571827cb):
> 
>- RAT check passes (7u80)
>- Unit test suite passes (8u131)
>- LTT load 1M rows with 100% verification and 20% updates (8u131)
>- PE randomWrite, randomRead, scanRange100 (8u131)
>- 100M rows ITBLL (8u131)
> 
> Between now and when I want to close the vote I'll write up human readable
> release notes for the release announcement as promised. I also have agreed
> to run a scale ITBLL test, a performance comparison with 1.2 using YCSB, a
> performance comparison with 1.2 using PE, and an analysis of code and
> allocation hot spot changes from 1.2, all of which I will publish when
> available and factor in to my vote.
> 
> 
> -- 
> Best regards,
> Andrew
> 

Re: [VOTE] The first HBase 1.4.0 release candidate (RC0) is available

[Balazs Meszaros]

Hi,

+1, because:

Signatures: ok
Unit tests: ok (8u112)
RSGroups: worked for me through the shell

-1, because:

hbase(main):005:0> list_rsgroups
GROUPS

test

default

2 row(s) in *1512730813.6580* seconds

Trust me, it was faster than 47 years :) Other listings (e.g.
list_namespace) displayed the elapsed time correctly.

Best regards,
Balazs

On Fri, Dec 8, 2017 at 4:51 AM, Ted Yu  wrote:

> +1
>
> Checked signatures: good
> Ran test suite: pass
> 1M row LTT: pass
>
> Cheers
>
> On Wed, Dec 6, 2017 at 8:40 PM, Andrew Purtell 
> wrote:
>
> > The first HBase 1.4.0​ release candidate (RC0) is available for download
> at
> > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/ and Maven
> > artifacts are available in the temporary repository
> > https://repository.apache.org/content/repositories/orgapachehbase-1185/
> .
> >
> > The git tag corresponding to the candidate is '1.4.0RC0' (3d571827cb).
> >
> > A detailed source and binary compatibility report for this release is
> > available for your review at
> > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.0RC0/
> > hbase-1.3.1-1.4.0RC0_compatibility_report.html
> > ​. All reported compatibility issues should comply with policy but please
> > review them carefully.
> >
> > A list of the ​652 issues resolved in this release can be found at
> > https://s.apache.org/OErT .
> >
> > Please try out the candidate and vote +1/0/-1.
> >
> > This vote will be open for at least 72 hours. Unless objection I will try
> > to close it ​Monday December 18, 2017 if we have sufficient votes.
> >
> > Prior to making this announcement I made the following preflight checks
> to
> > 1.4.0RC0 (3d571827cb):
> >
> >- RAT check passes (7u80)
> >- Unit test suite passes (8u131)
> >- LTT load 1M rows with 100% verification and 20% updates (8u131)
> >- PE randomWrite, randomRead, scanRange100 (8u131)
> >- 100M rows ITBLL (8u131)
> >
> > Between now and when I want to close the vote I'll write up human
> readable
> > release notes for the release announcement as promised. I also have
> agreed
> > to run a scale ITBLL test, a performance comparison with 1.2 using YCSB,
> a
> > performance comparison with 1.2 using PE, and an analysis of code and
> > allocation hot spot changes from 1.2, all of which I will publish when
> > available and factor in to my vote.
> >
> >
> > --
> > Best regards,
> > Andrew
> >
>

[jira] [Created] (HBASE-19462) Deprecate all addImmutable in Put

[Chia-Ping Tsai (JIRA)]

Chia-Ping Tsai created HBASE-19462:
--

 Summary: Deprecate all addImmutable in Put
 Key: HBASE-19462
 URL: https://issues.apache.org/jira/browse/HBASE-19462
 Project: HBase
  Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai


Users are able to use {{CellBuilder}} to build the cell without array copy for 
Put/Delete/Increment/Append, and we always do the copy if user pass the 
{{ByteBuffer}}. Hence, the {{addImmutable}} is unnecessary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19461) TestRSGroups is broke

[stack (JIRA)]

stack created HBASE-19461:
-

 Summary: TestRSGroups is broke
 Key: HBASE-19461
 URL: https://issues.apache.org/jira/browse/HBASE-19461
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
 Fix For: 2.0.0-beta-1


I noticed this. Tried bisect'ing but nought definitive at mo. I see 
testDefaultNamespaceCreateAndAssign failing most of the time. The crazy 
getTable parse in RegionInfo gets a negative offset because the passed in 
region name is nonsense.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)