[jira] [Created] (HBASE-18373) fstat unimplemented error on Rasbian Jessie

[Nathan Green (JIRA)]

Nathan Green created HBASE-18373:


 Summary: fstat unimplemented error on Rasbian Jessie
 Key: HBASE-18373
 URL: https://issues.apache.org/jira/browse/HBASE-18373
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 2.0.0-alpha-1
 Environment: Raspberry Pi 3
Rasbian Jessie (latest version)
Reporter: Nathan Green
Priority: Blocker


I have a Hbase distributed cluster setup on my Raspberry PI cluster (1 x 
Namenode & 4 x Datanode)

The stable version of Hbase shell works fine... unfortunately I had hadoop 
issues (mismatch of jar files gr) so I have installed latest Hbase & 
installed matching version of Hadoop (2.7.1).

HBase itself is up and running and all good.  Hbase shell however gives this 
error:

pi@pidoop1:/opt/hbase/lib/ruby/irb $ hbase shell
2017-07-13 12:25:11,722 WARN  [main] util.NativeCodeLoader: Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
Version 2.0.0-alpha-1, rc830a0f47f58d4892dd3300032c8244d6278aecc, Wed Jun  7 
22:05:17 PDT 2017
Took 0.0300 seconds
NotImplementedError: fstat unimplemented unsupported or native support failed 
to load; see http://wiki.jruby.org/Native-Libraries
  initialize at org/jruby/RubyIO.java:1013
open at org/jruby/RubyIO.java:1154
  initialize at 
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
  initialize at 
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
  initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:426
  initialize at /opt/hbase/lib/ruby/irb/hirb.rb:43
   start at /opt/hbase/bin/hirb.rb:194
   at /opt/hbase/bin/hirb.rb:206




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18372) Potential infinite busy loop in HMaster's ProcedureExecutor

[Benoit Sigoure (JIRA)]

Benoit Sigoure created HBASE-18372:
--

 Summary: Potential infinite busy loop in HMaster's 
ProcedureExecutor
 Key: HBASE-18372
 URL: https://issues.apache.org/jira/browse/HBASE-18372
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 1.3.1
 Environment: Kernel 3.10.0-327.10.1.el7.x86_64
JVM 1.8.0_102
Reporter: Benoit Sigoure


While investigating an issue today with [~timoha] we saw the HMaster 
consistently burning 1.5 cores of CPU cycles.  Upon looking more closely, it 
was actually all 8 threads of {{ProcedureExecutor}} thread pool taking 
constantly ~15% of a CPU core each (I identified this by looking at individual 
threads in {{top}} and cross-referencing the thread IDs with the thread IDs in 
a JVM stack trace).  The HMaster log or output didn't contain anything 
suspicious and it was hard for us to ascertain what exactly was happening.  It 
just looked like these threads were regularly spinning, doing nothing.  We just 
saw a lot of {{futex}} system calls happening all the time, and all the threads 
of the thread pool regularly taking turns in waking up and going back to sleep.

My reading of the code in {{procedure2/ProcedureExecutor.java}} is that this 
can happen if the threads in the thread pool have been interrupted for some 
reason:

{code}
  private void execLoop() {
while (isRunning()) {
  Procedure proc = runnables.poll();
  if (proc == null) continue;
{code}
and then in {master/procedure/MasterProcedureScheduler.java}:
{code}
  @Override
  public Procedure poll() {
return poll(-1);
  }

  @edu.umd.cs.findbugs.annotations.SuppressWarnings("WA_AWAIT_NOT_IN_LOOP")
  Procedure poll(long waitNsec) {
Procedure pollResult = null;
schedLock.lock();
try {
  if (queueSize == 0) {
if (waitNsec < 0) {
  schedWaitCond.await();
[...]
} catch (InterruptedException e) {
  Thread.currentThread().interrupt();
} finally {
  schedLock.unlock();
}
return pollResult;
  }
{code}

so my theory is the threads in the thread pool have all been interrupted (maybe 
by a procedure that ran earlier and left its thread interrupted) and so we are 
perpetually looping in {{execLoop}}, which ends up calling 
{{schedWaitCond.await();}}, which ends up throwing an {{InterruptedException}}, 
which ends up resetting the interrupt status of the thread, and rinse and 
repeat.

But again I wasn't able to get any cold hard evidence that this is what was 
happening.  There was just no other evidence that could explain this behavior, 
and I wasn't able to guess what else could be causing this that was consistent 
with what we saw and what I understood from reading the code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18371) Update folly and wangle dependencies

[Enis Soztutar (JIRA)]

Enis Soztutar created HBASE-18371:
-

 Summary: Update folly and wangle dependencies
 Key: HBASE-18371
 URL: https://issues.apache.org/jira/browse/HBASE-18371
 Project: HBase
  Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: HBASE-14850


We need to update folly and wangle dependency versions. Debugging an issue, I 
realized that we may need a couple of recent patches from wangle. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18370) Master should attempt reassignment of regions in FAILED_OPEN state

[Andrew Purtell (JIRA)]

Andrew Purtell created HBASE-18370:
--

 Summary: Master should attempt reassignment of regions in 
FAILED_OPEN state
 Key: HBASE-18370
 URL: https://issues.apache.org/jira/browse/HBASE-18370
 Project: HBase
  Issue Type: Improvement
Reporter: Andrew Purtell


Currently once a region goes into FAILED_OPEN state this requires operator 
intervention. With some underlying causes, this is necessary. With others, the 
master could eventually successfully deploy the region without humans in the 
loop. The master should optionally attempt automatic resolution of FAILED_OPEN 
states with a strategy of: delay, unassign, reassign. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-17818) On branch-1 in module Server, TestRSKilledWhenInitializing and TestScannerHeartbeatMessages broken

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved HBASE-17818.

   Resolution: Duplicate
Fix Version/s: (was: 1.4.0)

HBASE-16051 and HBASE-18346 now both dup this. Let's follow up there. 

> On branch-1 in module Server, TestRSKilledWhenInitializing and 
> TestScannerHeartbeatMessages broken
> --
>
> Key: HBASE-17818
> URL: https://issues.apache.org/jira/browse/HBASE-17818
> Project: HBase
>  Issue Type: Bug
> Environment: OS: Ubuntu 14.04
> Arch: x86_64
>Reporter: Anup Halarnkar
>
> Branch: branch-1
> Command: mvn clean install -X -fn
> Output:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 2.030 s
> [INFO] Finished at: 2017-03-22T19:17:16+05:30
> [INFO] Final Memory: 15M/304M
> [INFO] 
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache HBase ... SUCCESS [ 24.315 
> s]
> [INFO] Apache HBase - Checkstyle .. SUCCESS [  7.797 
> s]
> [INFO] Apache HBase - Resource Bundle . SUCCESS [  0.263 
> s]
> [INFO] Apache HBase - Annotations . SUCCESS [  2.416 
> s]
> [INFO] Apache HBase - Protocol  SUCCESS [ 12.994 
> s]
> [INFO] Apache HBase - Common .. SUCCESS [02:31 
> min]
> [INFO] Apache HBase - Procedure ... SUCCESS [03:37 
> min]
> [INFO] Apache HBase - Client .. SUCCESS [01:53 
> min]
> [INFO] Apache HBase - Hadoop Compatibility  SUCCESS [  8.560 
> s]
> [INFO] Apache HBase - Hadoop Two Compatibility  SUCCESS [ 10.500 
> s]
> [INFO] Apache HBase - Prefix Tree . SUCCESS [  9.367 
> s]
> [INFO] Apache HBase - Server .. FAILURE [  02:04 
> h]
> [INFO] Apache HBase - Testing Util  SUCCESS [  5.107 
> s]
> [INFO] Apache HBase - Thrift .. SUCCESS [04:52 
> min]
> [INFO] Apache HBase - Rest  SUCCESS [13:56 
> min]
> [INFO] Apache HBase - Shell ... SUCCESS [  3.980 
> s]
> [INFO] Apache HBase - Integration Tests ... SUCCESS [  02:33 
> h]
> [INFO] Apache HBase - Examples  SUCCESS [ 11.129 
> s]
> [INFO] Apache HBase - External Block Cache  SUCCESS [  1.231 
> s]
> [INFO] Apache HBase - Assembly  FAILURE [  4.063 
> s]
> [INFO] Apache HBase - Shaded .. SUCCESS [  0.172 
> s]
> [INFO] Apache HBase - Shaded - Client . SUCCESS [  0.793 
> s]
> [INFO] Apache HBase - Shaded - Server . SUCCESS [  1.721 
> s]
> [INFO] Apache HBase - Archetypes .. SUCCESS [  0.094 
> s]
> [INFO] Apache HBase - Exemplar for hbase-client archetype . SUCCESS [01:24 
> min]
> [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SUCCESS 
> [01:15 min]
> [INFO] Apache HBase - Archetype builder ... SUCCESS [ 31.495 
> s]
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 05:10 h
> [INFO] Finished at: 2017-03-22T19:17:16+05:30
> ---
> Failed tests:
> TestRSKilledWhenInitializing.testRSTerminationAfterRegisteringToMasterBeforeCreatingEphemeralNode:123
>  null
>   
> TestScannerHeartbeatMessages.testScannerHeartbeatMessages:207->testImportanceOfHeartbeats:237
>  Heartbeats messages are disabled, an exception should be thrown. If an 
> exception  is not thrown, the test case is not testing the importance of 
> heartbeat messages
> Tests run: 1698, Failures: 2, Errors: 0, Skipped: 11
> --
> If these tests are passing on your end,then please let me know the possible 
> issue that I could have on my side.
> Thanks in advance,
> Anup



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-13287) Append is missing methods, such as addColumn()

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-13287.

Resolution: Duplicate

> Append is missing methods, such as addColumn()
> --
>
> Key: HBASE-13287
> URL: https://issues.apache.org/jira/browse/HBASE-13287
> Project: HBase
>  Issue Type: Bug
>  Components: API
>Affects Versions: 1.0.0
>Reporter: Lars George
>Assignee: Jan Hentschel
>
> Analog to Put, Delete, Get, etc. Append should have the same common method 
> signature, namely {{addColumn()}} as opposed to the older {{add()}}. Also 
> consider adding the other variant that Put has, including the _immutable_ 
> versions.
> Question related: should those methods not better be in the Mutation class? 
> All derived classes implement them. {{setTTL()}} is in Mutation, and throws 
> an exception when used with {{Delete}}. So why not moved those down too? (cc 
> [~saint@gmail.com])



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18369) hbase thrift web-ui not available

[Peter Somogyi (JIRA)]

Peter Somogyi created HBASE-18369:
-

 Summary: hbase thrift web-ui not available
 Key: HBASE-18369
 URL: https://issues.apache.org/jira/browse/HBASE-18369
 Project: HBase
  Issue Type: Bug
Reporter: Peter Somogyi


In standalone mode hbase thrift web-ui is gives 404 error.

{code}
$ ./bin/start-hbase.sh
running master, logging to 
/Users/peter.somogyi/hbase/bin/../logs/hbase-peter.somogyi-master-psomogyi-MBP.local.out
$ ./bin/hbase thrift start
...
Jul 12, 2017 12:32:03 PM org.apache.jasper.servlet.JspServlet serviceJspFile
SEVERE: PWC6117: File 
"/Users/peter.somogyi/hbase/hbase-server/target/hbase-webapps/thrift/thrift.jsp"
 not found
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18368) Filters with OR does not work

[Peter Somogyi (JIRA)]

Peter Somogyi created HBASE-18368:
-

 Summary: Filters with OR does not work
 Key: HBASE-18368
 URL: https://issues.apache.org/jira/browse/HBASE-18368
 Project: HBase
  Issue Type: Bug
  Components: Filters
Affects Versions: 2.0.0, 3.0.0
Reporter: Peter Somogyi
Priority: Critical


Scan gives back incomplete list if multiple filters are combined with OR / 
MUST_PASS_ONE.
Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give 
back results for only the first Filter.

{code:java|title=Test code}
  @Test
  public void testFiltersWithOr() throws Exception {
TableName tn = TableName.valueOf("MyTest");
Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});

byte[] CF1 = Bytes.toBytes("cf1");
byte[] CF2 = Bytes.toBytes("cf2");

Put put1 = new Put(Bytes.toBytes("0"));
put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
table.put(put1);

Put put2 = new Put(Bytes.toBytes("0"));
put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
table.put(put2);

FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL, 
new BinaryComparator(CF1));
FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL, 
new BinaryComparator(CF2));

FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);

filterList.addFilter(filterCF1);
filterList.addFilter(filterCF2);

Scan scan = new Scan();
scan.setFilter(filterList);

ResultScanner scanner = table.getScanner(scan);

System.out.println(filterList);

for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
  System.out.println(rr);
}
  }
{code}

{noformat:title=Output}
FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18367) Reduce ProcedureInfo usage

[Balazs Meszaros (JIRA)]

Balazs Meszaros created HBASE-18367:
---

 Summary: Reduce ProcedureInfo usage
 Key: HBASE-18367
 URL: https://issues.apache.org/jira/browse/HBASE-18367
 Project: HBase
  Issue Type: Sub-task
Reporter: Balazs Meszaros
Assignee: Balazs Meszaros


If we want to replace ProcedureInfo objects with jsons (HBASE-18106) we have to 
reduce ProcedureInfo usage. Currently it is used several places in the code 
where it could be replaced with Procedure (e.g. ProcedureExecutor). We should 
use ProcedureInfo only for the communication before removing it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Do executing table.get() inside endpoint co-processor invokes a observer's (preGet, postGet etc ) callbacks

[Ted Yu]

Which region would be the recipient of the table.get() call ?

If that is a different region than the one the endpoint is activated on,
you should be aware of the potential deadlock issue.

Please take a look at CoprocessorHConnection.

FYI

On Tue, Jul 11, 2017 at 10:22 PM, Nilkanth Patel 
wrote:

> I have a following test scenario.
>
> Configured and added a co-processor endpoint. Endpoint implementation
> includes a table.get() call. (And this table has observed added as well)
>
> When i execute this end-point co-processor, where table.get() is a part of
> endpoint implementation,* will is invoke preGet() and postGet() observer
> callback as well...?*
>
> Thank you.
> Nilkanth.
>