[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-03-10 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394383#comment-16394383
 ] 

Josh Elser commented on TEPHRA-272:
---

{quote}Thank you so much [~poornachandra], for your time and review. Really 
appreciated!!
{quote}
+1 to that. Big thanks for the Phoenix folks :)

> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-03-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393961#comment-16393961
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-tephra/pull/67


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-03-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393952#comment-16393952
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
@ankitsinghal I spent some time on the tephra-hbase-compat-1.4 compat 
module tests failure. It looks like the 1.4 compat module tests were never run 
earlier. When you changed the travis.xml to build hbase-compat-1.4 module, the 
issue was exposed.

I have filed JIRA https://issues.apache.org/jira/browse/TEPHRA-285 for this 
issue. Since the failure was already present, I will go ahead and merge this 
PR. Thanks for the contribution!



> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-03-08 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390899#comment-16390899
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user ankitsinghal commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
@poornachandra , have squashed all commit in a single one. can you please 
commit this.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-03-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385890#comment-16385890
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user ankitsinghal commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
@poornachandra , I have already made the change to compile HBase-2.0 with 
Java 8. should I rebase and squash the commits for merge now?
It seems HBase 1.x releases are still with Java 7 so We can also support 
Java 7 for the older releases of HBase so that user can still use latest 
releases of Tephra without changing their environment. Though not sure why 1-2 
tests of HBase 1.4 are failing on Java 7, we can fix this in separate JIRA.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-26 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377601#comment-16377601
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
@joshelser There is nothing stopping us from moving Tephra to Java 8. Just 
that we'll have to build consensus on the Tephra dev list, and it will add to 
the delay of this PR. If making the POM changes to work with both Java 7 and 
Java 8 is complicated, then @ankitsinghal can you send out an email to the dev 
list for the Java 8 consensus?


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-26 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377442#comment-16377442
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user joshelser commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
> We can still keep the other modules as Java 7. I haven't tried it out 
myself. But I think it should work.

@poornachandra is there a reason that Tephra is still on Java7? It's quite 
out-dated at this point..


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-26 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377431#comment-16377431
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
@ankitsinghal Since HBase-2.0 needs Java 8, let's try changing the compiler 
for only HBase-2.0 compat module to use Java 8. We can still keep the other 
modules as Java 7. I haven't tried it out myself. But I think it should work.

Right now, we run Travis tests using both Java 7 and Java 8 for all 
modules. We can restrict HBase-2.0 compat module tests to run only for Java 8.



> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-23 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374419#comment-16374419
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user ankitsinghal commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
bq.  Can you update the pom for the 2.0 compat module to use Java 8?
Do we want to make Java8 specific to HBase-2.0 compat module and keep other 
still running on Java7. Will it not affect the release process, as currently 
the release is built with single version JDK 1.7? 



> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-23 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374393#comment-16374393
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user ankitsinghal commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r170260742
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/FilteredInternalScanner.java
 ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.Filter.ReturnCode;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import 
org.apache.tephra.hbase.coprocessor.TransactionProcessor.IncludeInProgressFilter;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Wrapper of InternalScanner to apply Transaction visibility filter for 
flush and compact
+ */
+public class FilteredInternalScanner implements InternalScanner {
+
+  private InternalScanner delegate;
+  private Filter filter;
+
+  public FilteredInternalScanner(InternalScanner internalScanner, 
IncludeInProgressFilter filter) {
+this.delegate = internalScanner;
+this.filter = filter;
+  }
+
+  @Override
+  public void close() throws IOException {
+this.delegate.close();
+  }
+
+  @Override
+  public boolean next(List result, ScannerContext scannerContext) 
throws IOException {
+List outResult = new ArrayList();
+while (true) {
+  boolean next = delegate.next(outResult, scannerContext);
+  for (Cell cell : outResult) {
+ReturnCode code = filter.filterKeyValue(cell);
+switch (code) {
+// included, so we are done
+case INCLUDE:
+case INCLUDE_AND_NEXT_COL:
--- End diff --

Yes @poornachandra , that's a downside, but with HBase-2.0 we don't have 
access to create StoreScanner with the custom scan in pre-compact/pre-flush 
hooks. 
I had a discussion with Anoop(From HBase) on 
https://issues.apache.org/jira/browse/TEPHRA-274 for reviewing the approach, 
probably currently there is no other way.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-20 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370811#comment-16370811
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
@ankitsinghal Just one more question 
(https://github.com/apache/incubator-tephra/pull/67#discussion_r169503415). 

Also the Java 7 build is failing for HBase 2.0 compat module. Looks like 
HBase 2.0 needs Java 8. Can you update the pom for the 2.0 compat module to use 
Java 8? 

Note that the master branch has some changes on how the Travis tests are 
run (TEPHRA-282). You'll have to rebase, and then add an entry to travis.yml 
for the 2.0 compat module.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-20 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370785#comment-16370785
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r169503415
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/FilteredInternalScanner.java
 ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.Filter.ReturnCode;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import 
org.apache.tephra.hbase.coprocessor.TransactionProcessor.IncludeInProgressFilter;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Wrapper of InternalScanner to apply Transaction visibility filter for 
flush and compact
+ */
+public class FilteredInternalScanner implements InternalScanner {
+
+  private InternalScanner delegate;
+  private Filter filter;
+
+  public FilteredInternalScanner(InternalScanner internalScanner, 
IncludeInProgressFilter filter) {
+this.delegate = internalScanner;
+this.filter = filter;
+  }
+
+  @Override
+  public void close() throws IOException {
+this.delegate.close();
+  }
+
+  @Override
+  public boolean next(List result, ScannerContext scannerContext) 
throws IOException {
+List outResult = new ArrayList();
+while (true) {
+  boolean next = delegate.next(outResult, scannerContext);
+  for (Cell cell : outResult) {
+ReturnCode code = filter.filterKeyValue(cell);
+switch (code) {
+// included, so we are done
+case INCLUDE:
+case INCLUDE_AND_NEXT_COL:
--- End diff --

The only downside I see with this approach is that we'll iterate through 
all the cells for a column, even though we could have skipped some cells when 
the filter returns `INCLUDE_AND_NEXT_COL`. 

For example if there are 100 versions of a value in a column, and the 
filter figures out at the 10th version that the rest of the versions have 
reached the TTL, then in the earlier code we would not  iterate through the 
rest of the 90 cells (or at least that's what I think was happening in the 
underlying scanner). Is there anyway we can achieve the same result with the 
new APIs?


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-19 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369614#comment-16369614
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r169195010
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/FilteredInternalScanner.java
 ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.Filter.ReturnCode;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import 
org.apache.tephra.hbase.coprocessor.TransactionProcessor.IncludeInProgressFilter;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Wrapper of InternalScanner to apply Transaction visibility filter for 
flush and compact
+ */
+public class FilteredInternalScanner implements InternalScanner {
+
+  private InternalScanner delegate;
+  private Filter filter;
+
+  public FilteredInternalScanner(InternalScanner internalScanner, 
IncludeInProgressFilter filter) {
+this.delegate = internalScanner;
+this.filter = filter;
+  }
+
+  @Override
+  public void close() throws IOException {
+this.delegate.close();
+  }
+
+  @Override
+  public boolean next(List result, ScannerContext scannerContext) 
throws IOException {
+List outResult = new ArrayList();
+while (true) {
+  boolean next = delegate.next(outResult, scannerContext);
+  for (Cell cell : outResult) {
+ReturnCode code = filter.filterKeyValue(cell);
+switch (code) {
+// included, so we are done
+case INCLUDE:
+case INCLUDE_AND_NEXT_COL:
--- End diff --

  Just confirmed that transaction filter does this


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-19 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368942#comment-16368942
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user ankitsinghal commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r169027005
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/FilteredInternalScanner.java
 ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.Filter.ReturnCode;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import 
org.apache.tephra.hbase.coprocessor.TransactionProcessor.IncludeInProgressFilter;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Wrapper of InternalScanner to apply Transaction visibility filter for 
flush and compact
+ */
+public class FilteredInternalScanner implements InternalScanner {
+
+  private InternalScanner delegate;
+  private Filter filter;
+
+  public FilteredInternalScanner(InternalScanner internalScanner, 
IncludeInProgressFilter filter) {
+this.delegate = internalScanner;
+this.filter = filter;
+  }
+
+  @Override
+  public void close() throws IOException {
+this.delegate.close();
+  }
+
+  @Override
+  public boolean next(List result, ScannerContext scannerContext) 
throws IOException {
+List outResult = new ArrayList();
+while (true) {
+  boolean next = delegate.next(outResult, scannerContext);
+  for (Cell cell : outResult) {
+ReturnCode code = filter.filterKeyValue(cell);
+switch (code) {
+// included, so we are done
+case INCLUDE:
+case INCLUDE_AND_NEXT_COL:
--- End diff --

Yes, so on next cell, we expect the filter to return some NEXT_COL/SKIP (if 
it returned INCLUDE_AND_NEXT_COL for the previous cell) so that we will skip 
them. 


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-16 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368014#comment-16368014
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r168878262
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/TransactionProcessor.java
 ---
@@ -0,0 +1,574 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.CoprocessorEnvironment;
+import org.apache.hadoop.hbase.DoNotRetryIOException;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.OperationWithAttributes;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.FilterBase;
+import org.apache.hadoop.hbase.regionserver.HStore;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.Region;
+import org.apache.hadoop.hbase.regionserver.ScanOptions;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.hadoop.hbase.regionserver.StoreFile;
+import 
org.apache.hadoop.hbase.regionserver.compactions.CompactionLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionCodec;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.CacheSupplier;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.hbase.txprune.CompactionState;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.util.TxUtils;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableSet;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import javax.annotation.Nullable;
+
+/**
+ * {@code org.apache.hadoop.hbase.coprocessor.RegionObserver} coprocessor 
that handles server-side processing
+ * for transactions:
+ * 
+ *   applies filtering to exclude data from invalid and in-progress 
transactions
+ *   overrides the scanner returned for flush and compaction to drop 
data written by invalidated transactions,
+ *   or expired due to TTL.
+ * 
+ *
+ * In order to use this coprocessor for transactions, configure the 
class on any table involved in transactions,
+ * or on all user tables by adding the following to hbase-site.xml:
+ * {@code
+ * 
+ *   

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-16 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368016#comment-16368016
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r16679
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/FilteredInternalScanner.java
 ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.Filter.ReturnCode;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import 
org.apache.tephra.hbase.coprocessor.TransactionProcessor.IncludeInProgressFilter;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Wrapper of InternalScanner to apply Transaction visibility filter for 
flush and compact
+ */
+public class FilteredInternalScanner implements InternalScanner {
+
+  private InternalScanner delegate;
+  private Filter filter;
+
+  public FilteredInternalScanner(InternalScanner internalScanner, 
IncludeInProgressFilter filter) {
+this.delegate = internalScanner;
+this.filter = filter;
+  }
+
+  @Override
+  public void close() throws IOException {
+this.delegate.close();
+  }
+
+  @Override
+  public boolean next(List result, ScannerContext scannerContext) 
throws IOException {
+List outResult = new ArrayList();
--- End diff --

It would be good to make `outResult` as a private variable, so that it is 
not created for every `next` call. We can then just clear it for each call.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-16 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368017#comment-16368017
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r16822
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/FilteredInternalScanner.java
 ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.Filter.ReturnCode;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import 
org.apache.tephra.hbase.coprocessor.TransactionProcessor.IncludeInProgressFilter;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Wrapper of InternalScanner to apply Transaction visibility filter for 
flush and compact
+ */
+public class FilteredInternalScanner implements InternalScanner {
+
+  private InternalScanner delegate;
+  private Filter filter;
--- End diff --

Both the fields can be marked as final since they don't change.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-16 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368015#comment-16368015
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r16423
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/FilteredInternalScanner.java
 ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.Filter.ReturnCode;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.ScannerContext;
+import 
org.apache.tephra.hbase.coprocessor.TransactionProcessor.IncludeInProgressFilter;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Wrapper of InternalScanner to apply Transaction visibility filter for 
flush and compact
+ */
+public class FilteredInternalScanner implements InternalScanner {
+
+  private InternalScanner delegate;
+  private Filter filter;
+
+  public FilteredInternalScanner(InternalScanner internalScanner, 
IncludeInProgressFilter filter) {
+this.delegate = internalScanner;
+this.filter = filter;
+  }
+
+  @Override
+  public void close() throws IOException {
+this.delegate.close();
+  }
+
+  @Override
+  public boolean next(List result, ScannerContext scannerContext) 
throws IOException {
+List outResult = new ArrayList();
+while (true) {
+  boolean next = delegate.next(outResult, scannerContext);
+  for (Cell cell : outResult) {
+ReturnCode code = filter.filterKeyValue(cell);
+switch (code) {
+// included, so we are done
+case INCLUDE:
+case INCLUDE_AND_NEXT_COL:
--- End diff --

When the filter return code is `INCLUDE_AND_NEXT_COL`, this filter needs to 
skip the remaining cells for that column, right?


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-12 Thread Poorna Chandra (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361316#comment-16361316
 ] 

Poorna Chandra commented on TEPHRA-272:
---

[~jamestaylor] [~elserj] I was on vacation. I'm back now and catching up. Will 
continue the review soon.

> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-02-06 Thread Josh Elser (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354192#comment-16354192
 ] 

Josh Elser commented on TEPHRA-272:
---

[~poornachandra], sorry for a second ping, but do you have any cycles to help 
get this committed and a corresponding release of Tephra made?

I'm trying to push for a Phoenix 5.0 but we're blocked on making a stable 
release until these changes land. Thanks in advance.

> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345440#comment-16345440
 ] 

James Taylor commented on TEPHRA-272:
-

Ping [~poornachandra].

> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321815#comment-16321815
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user ankitsinghal commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
Thanks a lot @poornachandra for the review, I have taken care of all the 
current review comments in the last commit. Let me know if any changes required 
with your further review.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321813#comment-16321813
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user ankitsinghal commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160880292
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java
 ---
@@ -0,0 +1,677 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.ImmutableSortedMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ChoreService;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.MockRegionServerServices;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.util.MemorySizeUtil;
+import org.apache.hadoop.hbase.regionserver.ChunkCreator;
+import org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResult;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResultImpl;
+import org.apache.hadoop.hbase.regionserver.HRegionFileSystem;
+import org.apache.hadoop.hbase.regionserver.MemStoreLAB;
+import org.apache.hadoop.hbase.regionserver.MemStoreLABImpl;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionManager;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.manager.InvalidTxList;
+import org.apache.tephra.metrics.TxMetricsCollector;
+import org.apache.tephra.persist.HDFSTransactionStateStorage;
+import org.apache.tephra.persist.TransactionSnapshot;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.snapshot.DefaultSnapshotCodec;
+import org.apache.tephra.snapshot.SnapshotCodecProvider;
+import org.apache.tephra.util.TxUtils;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.management.ManagementFactory;
+import java.net.InetAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321805#comment-16321805
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user ankitsinghal commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160879714
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java
 ---
@@ -0,0 +1,677 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.ImmutableSortedMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ChoreService;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.MockRegionServerServices;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.util.MemorySizeUtil;
+import org.apache.hadoop.hbase.regionserver.ChunkCreator;
+import org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResult;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResultImpl;
+import org.apache.hadoop.hbase.regionserver.HRegionFileSystem;
+import org.apache.hadoop.hbase.regionserver.MemStoreLAB;
+import org.apache.hadoop.hbase.regionserver.MemStoreLABImpl;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionManager;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.manager.InvalidTxList;
+import org.apache.tephra.metrics.TxMetricsCollector;
+import org.apache.tephra.persist.HDFSTransactionStateStorage;
+import org.apache.tephra.persist.TransactionSnapshot;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.snapshot.DefaultSnapshotCodec;
+import org.apache.tephra.snapshot.SnapshotCodecProvider;
+import org.apache.tephra.util.TxUtils;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.management.ManagementFactory;
+import java.net.InetAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318420#comment-16318420
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160405134
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java
 ---
@@ -0,0 +1,677 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.ImmutableSortedMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ChoreService;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.MockRegionServerServices;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.util.MemorySizeUtil;
+import org.apache.hadoop.hbase.regionserver.ChunkCreator;
+import org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResult;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResultImpl;
+import org.apache.hadoop.hbase.regionserver.HRegionFileSystem;
+import org.apache.hadoop.hbase.regionserver.MemStoreLAB;
+import org.apache.hadoop.hbase.regionserver.MemStoreLABImpl;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionManager;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.manager.InvalidTxList;
+import org.apache.tephra.metrics.TxMetricsCollector;
+import org.apache.tephra.persist.HDFSTransactionStateStorage;
+import org.apache.tephra.persist.TransactionSnapshot;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.snapshot.DefaultSnapshotCodec;
+import org.apache.tephra.snapshot.SnapshotCodecProvider;
+import org.apache.tephra.util.TxUtils;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.management.ManagementFactory;
+import java.net.InetAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318416#comment-16318416
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160405760
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java
 ---
@@ -0,0 +1,677 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.ImmutableSortedMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ChoreService;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.MockRegionServerServices;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.util.MemorySizeUtil;
+import org.apache.hadoop.hbase.regionserver.ChunkCreator;
+import org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResult;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResultImpl;
+import org.apache.hadoop.hbase.regionserver.HRegionFileSystem;
+import org.apache.hadoop.hbase.regionserver.MemStoreLAB;
+import org.apache.hadoop.hbase.regionserver.MemStoreLABImpl;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionManager;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.manager.InvalidTxList;
+import org.apache.tephra.metrics.TxMetricsCollector;
+import org.apache.tephra.persist.HDFSTransactionStateStorage;
+import org.apache.tephra.persist.TransactionSnapshot;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.snapshot.DefaultSnapshotCodec;
+import org.apache.tephra.snapshot.SnapshotCodecProvider;
+import org.apache.tephra.util.TxUtils;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.management.ManagementFactory;
+import java.net.InetAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318417#comment-16318417
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160403830
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/AbstractHBaseTableTest.java
 ---
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tephra.hbase;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Coprocessor;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.HBaseAdmin;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.hbase.coprocessor.TransactionProcessor;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * Base class for tests that need a HBase cluster
+ */
+@SuppressWarnings("WeakerAccess")
+public abstract class AbstractHBaseTableTest {
+  protected static HBaseTestingUtility testUtil;
+  protected static HBaseAdmin hBaseAdmin;
+  protected static Configuration conf;
+
+  @BeforeClass
+  public static void startMiniCluster() throws Exception {
+testUtil = conf == null ? new HBaseTestingUtility() : new 
HBaseTestingUtility(conf);
+conf = testUtil.getConfiguration();
+
+// Tune down the connection thread pool size
+conf.setInt("hbase.hconnection.threads.core", 5);
+conf.setInt("hbase.hconnection.threads.max", 10);
+// Tunn down handler threads in regionserver
+conf.setInt("hbase.regionserver.handler.count", 10);
+
+// Set to random port
+conf.setInt("hbase.master.port", 0);
+conf.setInt("hbase.master.info.port", 0);
+conf.setInt("hbase.regionserver.port", 0);
+conf.setInt("hbase.regionserver.info.port", 0);
+
+testUtil.startMiniCluster();
+hBaseAdmin = testUtil.getHBaseAdmin();
+  }
+
+  @AfterClass
+  public static void shutdownMiniCluster() throws Exception {
+try {
+  if (hBaseAdmin != null) {
+hBaseAdmin.close();
+  }
+} finally {
+  testUtil.shutdownMiniCluster();
+}
+  }
+
+  protected static Table createTable(byte[] tableName, byte[][] 
columnFamilies) throws Exception {
+return createTable(tableName, columnFamilies, false,
+   
Collections.singletonList(TransactionProcessor.class.getName()));
+  }
+
+  protected static Table createTable(byte[] tableName, byte[][] 
columnFamilies, boolean existingData,
+  List coprocessors) throws 
Exception {
+HTableDescriptor desc = new 
HTableDescriptor(TableName.valueOf(tableName));
+for (byte[] family : columnFamilies) {
+  HColumnDescriptor columnDesc = new HColumnDescriptor(family);
+  columnDesc.setMaxVersions(Integer.MAX_VALUE);
+  columnDesc.setValue(TxConstants.PROPERTY_TTL, 
String.valueOf(10)); // in millis
+  desc.addFamily(columnDesc);
+}
+if (existingData) {
+  desc.setValue(TxConstants.READ_NON_TX_DATA, "true");
+}
+// Divide individually to prevent any overflow
+int priority = Coprocessor.PRIORITY_USER;
+// order in list is the same order that coprocessors will be invoked
+for (String coprocessor : coprocessors) {
+  desc.addCoprocessor(coprocessor, null, ++priority, null);
+}
+hBaseAdmin.createTable(desc);
+//testUtil.waitTableAvailable(tableName, 5000);
--- 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318422#comment-16318422
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160404092
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/TransactionAwareHTableTest.java
 ---
@@ -0,0 +1,1907 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tephra.hbase;
+
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.DoNotRetryIOException;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.HBaseAdmin;
+import org.apache.hadoop.hbase.client.OperationWithAttributes;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.filter.CompareFilter;
+import org.apache.hadoop.hbase.filter.LongComparator;
+import org.apache.hadoop.hbase.filter.ValueFilter;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionConflictException;
+import org.apache.tephra.TransactionContext;
+import org.apache.tephra.TransactionManager;
+import org.apache.tephra.TransactionSystemClient;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.TxConstants.ConflictDetection;
+import org.apache.tephra.hbase.coprocessor.TransactionProcessor;
+import org.apache.tephra.inmemory.InMemoryTxSystemClient;
+import org.apache.tephra.metrics.TxMetricsCollector;
+import org.apache.tephra.persist.HDFSTransactionStateStorage;
+import org.apache.tephra.persist.TransactionStateStorage;
+import org.apache.tephra.snapshot.SnapshotCodecProvider;
+
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.TimeUnit;
+
+import static org.junit.Assert.assertArrayEquals;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+
+/**
+ * Tests for TransactionAwareHTables.
+ */
+public class TransactionAwareHTableTest extends AbstractHBaseTableTest {
+private 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318418#comment-16318418
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160401932
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/TransactionAwareHTable.java
 ---
@@ -0,0 +1,729 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tephra.hbase;
+
+import com.google.protobuf.Descriptors.MethodDescriptor;
+import com.google.protobuf.Message;
+import com.google.protobuf.Service;
+import com.google.protobuf.ServiceException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Mutation;
+import org.apache.hadoop.hbase.client.OperationWithAttributes;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Row;
+import org.apache.hadoop.hbase.client.RowMutations;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.coprocessor.Batch;
+import org.apache.hadoop.hbase.client.coprocessor.Batch.Callback;
+import org.apache.hadoop.hbase.filter.CompareFilter;
+import org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel;
+import org.apache.tephra.AbstractTransactionAwareTable;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionAware;
+import org.apache.tephra.TxConstants;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableMap;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * A Transaction Aware HTable implementation for HBase 2.0. Operations are 
committed as usual, but
+ * upon a failed or aborted transaction, they are rolled back to the state 
before the transaction
+ * was started.
+ */
+public class TransactionAwareHTable extends AbstractTransactionAwareTable
+implements Table, TransactionAware {
+
+private static final Logger LOG = 
LoggerFactory.getLogger(TransactionAwareHTable.class);
+private final Table hTable;
+
+/**
+ * Create a transactional aware instance of the passed HTable
+ * @param hTable underlying HBase table to use
+ */
+public TransactionAwareHTable(Table hTable) {
+this(hTable, false);
+}
+
+/**
+ * Create a transactional aware instance of the passed HTable
+ * @param hTable underlying HBase table to use
+ * @param conflictLevel level of conflict detection to perform 
(defaults to {@code COLUMN})
+ */
+public TransactionAwareHTable(Table hTable, 
TxConstants.ConflictDetection conflictLevel) {
+this(hTable, conflictLevel, false);
+}
+
+/**
+ * Create a transactional aware instance of the passed HTable, with 
the option of allowing
+ * non-transactional operations.
+ * @param hTable underlying HBase table to use
+ * @param 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318412#comment-16318412
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160400229
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/TransactionAwareHTable.java
 ---
@@ -0,0 +1,729 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tephra.hbase;
+
+import com.google.protobuf.Descriptors.MethodDescriptor;
+import com.google.protobuf.Message;
+import com.google.protobuf.Service;
+import com.google.protobuf.ServiceException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Mutation;
+import org.apache.hadoop.hbase.client.OperationWithAttributes;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Row;
+import org.apache.hadoop.hbase.client.RowMutations;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.coprocessor.Batch;
+import org.apache.hadoop.hbase.client.coprocessor.Batch.Callback;
+import org.apache.hadoop.hbase.filter.CompareFilter;
+import org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel;
+import org.apache.tephra.AbstractTransactionAwareTable;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionAware;
+import org.apache.tephra.TxConstants;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableMap;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * A Transaction Aware HTable implementation for HBase 2.0. Operations are 
committed as usual, but
+ * upon a failed or aborted transaction, they are rolled back to the state 
before the transaction
+ * was started.
+ */
+public class TransactionAwareHTable extends AbstractTransactionAwareTable
+implements Table, TransactionAware {
+
+private static final Logger LOG = 
LoggerFactory.getLogger(TransactionAwareHTable.class);
+private final Table hTable;
+
+/**
+ * Create a transactional aware instance of the passed HTable
+ * @param hTable underlying HBase table to use
+ */
+public TransactionAwareHTable(Table hTable) {
+this(hTable, false);
+}
+
+/**
+ * Create a transactional aware instance of the passed HTable
+ * @param hTable underlying HBase table to use
+ * @param conflictLevel level of conflict detection to perform 
(defaults to {@code COLUMN})
+ */
+public TransactionAwareHTable(Table hTable, 
TxConstants.ConflictDetection conflictLevel) {
+this(hTable, conflictLevel, false);
+}
+
+/**
+ * Create a transactional aware instance of the passed HTable, with 
the option of allowing
+ * non-transactional operations.
+ * @param hTable underlying HBase table to use
+ * @param 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318414#comment-16318414
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160407023
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java
 ---
@@ -0,0 +1,677 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.ImmutableSortedMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ChoreService;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.MockRegionServerServices;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.util.MemorySizeUtil;
+import org.apache.hadoop.hbase.regionserver.ChunkCreator;
+import org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResult;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResultImpl;
+import org.apache.hadoop.hbase.regionserver.HRegionFileSystem;
+import org.apache.hadoop.hbase.regionserver.MemStoreLAB;
+import org.apache.hadoop.hbase.regionserver.MemStoreLABImpl;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionManager;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.manager.InvalidTxList;
+import org.apache.tephra.metrics.TxMetricsCollector;
+import org.apache.tephra.persist.HDFSTransactionStateStorage;
+import org.apache.tephra.persist.TransactionSnapshot;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.snapshot.DefaultSnapshotCodec;
+import org.apache.tephra.snapshot.SnapshotCodecProvider;
+import org.apache.tephra.util.TxUtils;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.management.ManagementFactory;
+import java.net.InetAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318410#comment-16318410
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160400034
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/TransactionAwareHTable.java
 ---
@@ -0,0 +1,729 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tephra.hbase;
+
+import com.google.protobuf.Descriptors.MethodDescriptor;
+import com.google.protobuf.Message;
+import com.google.protobuf.Service;
+import com.google.protobuf.ServiceException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Mutation;
+import org.apache.hadoop.hbase.client.OperationWithAttributes;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Row;
+import org.apache.hadoop.hbase.client.RowMutations;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.coprocessor.Batch;
+import org.apache.hadoop.hbase.client.coprocessor.Batch.Callback;
+import org.apache.hadoop.hbase.filter.CompareFilter;
+import org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel;
+import org.apache.tephra.AbstractTransactionAwareTable;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionAware;
+import org.apache.tephra.TxConstants;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableMap;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * A Transaction Aware HTable implementation for HBase 2.0. Operations are 
committed as usual, but
+ * upon a failed or aborted transaction, they are rolled back to the state 
before the transaction
+ * was started.
+ */
+public class TransactionAwareHTable extends AbstractTransactionAwareTable
+implements Table, TransactionAware {
+
+private static final Logger LOG = 
LoggerFactory.getLogger(TransactionAwareHTable.class);
+private final Table hTable;
+
+/**
+ * Create a transactional aware instance of the passed HTable
+ * @param hTable underlying HBase table to use
+ */
+public TransactionAwareHTable(Table hTable) {
+this(hTable, false);
+}
+
+/**
+ * Create a transactional aware instance of the passed HTable
+ * @param hTable underlying HBase table to use
+ * @param conflictLevel level of conflict detection to perform 
(defaults to {@code COLUMN})
+ */
+public TransactionAwareHTable(Table hTable, 
TxConstants.ConflictDetection conflictLevel) {
+this(hTable, conflictLevel, false);
+}
+
+/**
+ * Create a transactional aware instance of the passed HTable, with 
the option of allowing
+ * non-transactional operations.
+ * @param hTable underlying HBase table to use
+ * @param 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318419#comment-16318419
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160402855
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/coprocessor/TransactionProcessor.java
 ---
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.CoprocessorEnvironment;
+import org.apache.hadoop.hbase.DoNotRetryIOException;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.OperationWithAttributes;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.filter.Filter;
+import org.apache.hadoop.hbase.filter.FilterBase;
+import org.apache.hadoop.hbase.regionserver.HStore;
+import org.apache.hadoop.hbase.regionserver.InternalScanner;
+import org.apache.hadoop.hbase.regionserver.Region;
+import org.apache.hadoop.hbase.regionserver.ScanOptions;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.regionserver.Store;
+import org.apache.hadoop.hbase.regionserver.StoreFile;
+import 
org.apache.hadoop.hbase.regionserver.compactions.CompactionLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionCodec;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.CacheSupplier;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.hbase.txprune.CompactionState;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.util.TxUtils;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableSet;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+import javax.annotation.Nullable;
+
+/**
+ * {@code org.apache.hadoop.hbase.coprocessor.RegionObserver} coprocessor 
that handles server-side processing
+ * for transactions:
+ * 
+ *   applies filtering to exclude data from invalid and in-progress 
transactions
+ *   overrides the scanner returned for flush and compaction to drop 
data written by invalidated transactions,
+ *   or expired due to TTL.
+ * 
+ *
+ * In order to use this coprocessor for transactions, configure the 
class on any table involved in transactions,
+ * or on all user tables by adding the following to hbase-site.xml:
+ * {@code
+ * 
+ *   

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318411#comment-16318411
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160398641
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/TransactionAwareHTable.java
 ---
@@ -0,0 +1,729 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tephra.hbase;
+
+import com.google.protobuf.Descriptors.MethodDescriptor;
+import com.google.protobuf.Message;
+import com.google.protobuf.Service;
+import com.google.protobuf.ServiceException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Mutation;
+import org.apache.hadoop.hbase.client.OperationWithAttributes;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Row;
+import org.apache.hadoop.hbase.client.RowMutations;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.coprocessor.Batch;
+import org.apache.hadoop.hbase.client.coprocessor.Batch.Callback;
+import org.apache.hadoop.hbase.filter.CompareFilter;
+import org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel;
+import org.apache.tephra.AbstractTransactionAwareTable;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionAware;
+import org.apache.tephra.TxConstants;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableMap;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * A Transaction Aware HTable implementation for HBase 2.0. Operations are 
committed as usual, but
+ * upon a failed or aborted transaction, they are rolled back to the state 
before the transaction
+ * was started.
+ */
+public class TransactionAwareHTable extends AbstractTransactionAwareTable
+implements Table, TransactionAware {
+
+private static final Logger LOG = 
LoggerFactory.getLogger(TransactionAwareHTable.class);
--- End diff --

The convention in Tephra is to use 2 spaces for indentation, this class 
uses 4 spaces.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318421#comment-16318421
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160406273
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java
 ---
@@ -0,0 +1,677 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.ImmutableSortedMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ChoreService;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.MockRegionServerServices;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.util.MemorySizeUtil;
+import org.apache.hadoop.hbase.regionserver.ChunkCreator;
+import org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResult;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResultImpl;
+import org.apache.hadoop.hbase.regionserver.HRegionFileSystem;
+import org.apache.hadoop.hbase.regionserver.MemStoreLAB;
+import org.apache.hadoop.hbase.regionserver.MemStoreLABImpl;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionManager;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.manager.InvalidTxList;
+import org.apache.tephra.metrics.TxMetricsCollector;
+import org.apache.tephra.persist.HDFSTransactionStateStorage;
+import org.apache.tephra.persist.TransactionSnapshot;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.snapshot.DefaultSnapshotCodec;
+import org.apache.tephra.snapshot.SnapshotCodecProvider;
+import org.apache.tephra.util.TxUtils;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.management.ManagementFactory;
+import java.net.InetAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318415#comment-16318415
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160406873
  
--- Diff: 
tephra-hbase-compat-2.0/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java
 ---
@@ -0,0 +1,677 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tephra.hbase.coprocessor;
+
+import com.google.common.collect.ImmutableSortedMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.ChoreService;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.KeyValue;
+import org.apache.hadoop.hbase.MockRegionServerServices;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.util.MemorySizeUtil;
+import org.apache.hadoop.hbase.regionserver.ChunkCreator;
+import org.apache.hadoop.hbase.regionserver.FlushLifeCycleTracker;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResult;
+import org.apache.hadoop.hbase.regionserver.HRegion.FlushResultImpl;
+import org.apache.hadoop.hbase.regionserver.HRegionFileSystem;
+import org.apache.hadoop.hbase.regionserver.MemStoreLAB;
+import org.apache.hadoop.hbase.regionserver.MemStoreLABImpl;
+import org.apache.hadoop.hbase.regionserver.RegionScanner;
+import org.apache.hadoop.hbase.regionserver.ScanType;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.apache.tephra.Transaction;
+import org.apache.tephra.TransactionManager;
+import org.apache.tephra.TxConstants;
+import org.apache.tephra.coprocessor.TransactionStateCache;
+import org.apache.tephra.coprocessor.TransactionStateCacheSupplier;
+import org.apache.tephra.manager.InvalidTxList;
+import org.apache.tephra.metrics.TxMetricsCollector;
+import org.apache.tephra.persist.HDFSTransactionStateStorage;
+import org.apache.tephra.persist.TransactionSnapshot;
+import org.apache.tephra.persist.TransactionVisibilityState;
+import org.apache.tephra.snapshot.DefaultSnapshotCodec;
+import org.apache.tephra.snapshot.SnapshotCodecProvider;
+import org.apache.tephra.util.TxUtils;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.management.ManagementFactory;
+import java.net.InetAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import 

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318413#comment-16318413
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160398165
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/SecondaryIndexTable.java
 ---
@@ -0,0 +1,182 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tephra.hbase;
+
+import com.google.common.base.Throwables;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.tephra.TransactionContext;
+import org.apache.tephra.TransactionFailureException;
+import org.apache.tephra.distributed.TransactionServiceClient;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * A Transactional SecondaryIndexTable.
+ */
+public class SecondaryIndexTable implements Closeable {
+  private byte[] secondaryIndex;
+  private TransactionAwareHTable transactionAwareHTable;
+  private TransactionAwareHTable secondaryIndexTable;
+  private TransactionContext transactionContext;
+  private final TableName secondaryIndexTableName;
+  private Connection connection;
+  private static final byte[] secondaryIndexFamily = 
Bytes.toBytes("secondaryIndexFamily");
+  private static final byte[] secondaryIndexQualifier = Bytes.toBytes('r');
+  private static final byte[] DELIMITER  = new byte[] {0};
+
+  public SecondaryIndexTable(TransactionServiceClient 
transactionServiceClient, Table table,
+ byte[] secondaryIndex) throws IOException {
+secondaryIndexTableName = 
TableName.valueOf(table.getName().getNameAsString() + ".idx");
+this.connection = 
ConnectionFactory.createConnection(table.getConfiguration());
+Table secondaryIndexHTable = null;
+try (Admin hBaseAdmin = this.connection.getAdmin()) {
+  if (!hBaseAdmin.tableExists(secondaryIndexTableName)) {
+
hBaseAdmin.createTable(TableDescriptorBuilder.newBuilder(secondaryIndexTableName).build());
+  }
+  secondaryIndexHTable = 
this.connection.getTable(secondaryIndexTableName);
+} catch (Exception e) {
+  Throwables.propagate(e);
+}
+
+this.secondaryIndex = secondaryIndex;
+this.transactionAwareHTable = new TransactionAwareHTable(table);
+this.secondaryIndexTable = new 
TransactionAwareHTable(secondaryIndexHTable);
+this.transactionContext = new 
TransactionContext(transactionServiceClient, transactionAwareHTable,
+ secondaryIndexTable);
+  }
+
+  public Result get(Get get) throws IOException {
+return get(Collections.singletonList(get))[0];
+  }
+
+  public Result[] get(List gets) throws IOException {
+try {
+  transactionContext.start();
+  Result[] result = transactionAwareHTable.get(gets);
+  transactionContext.finish();
+  return result;
+} catch (Exception e) {
+  try {
+

[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318409#comment-16318409
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on a diff in the pull request:

https://github.com/apache/incubator-tephra/pull/67#discussion_r160397436
  
--- Diff: 
tephra-hbase-compat-2.0/src/main/java/org/apache/tephra/hbase/SecondaryIndexTable.java
 ---
@@ -0,0 +1,182 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tephra.hbase;
+
+import com.google.common.base.Throwables;
+
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CellUtil;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.tephra.TransactionContext;
+import org.apache.tephra.TransactionFailureException;
+import org.apache.tephra.distributed.TransactionServiceClient;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * A Transactional SecondaryIndexTable.
+ */
+public class SecondaryIndexTable implements Closeable {
+  private byte[] secondaryIndex;
+  private TransactionAwareHTable transactionAwareHTable;
+  private TransactionAwareHTable secondaryIndexTable;
+  private TransactionContext transactionContext;
+  private final TableName secondaryIndexTableName;
+  private Connection connection;
+  private static final byte[] secondaryIndexFamily = 
Bytes.toBytes("secondaryIndexFamily");
+  private static final byte[] secondaryIndexQualifier = Bytes.toBytes('r');
+  private static final byte[] DELIMITER  = new byte[] {0};
+
+  public SecondaryIndexTable(TransactionServiceClient 
transactionServiceClient, Table table,
+ byte[] secondaryIndex) throws IOException {
+secondaryIndexTableName = 
TableName.valueOf(table.getName().getNameAsString() + ".idx");
+this.connection = 
ConnectionFactory.createConnection(table.getConfiguration());
+Table secondaryIndexHTable = null;
+try (Admin hBaseAdmin = this.connection.getAdmin()) {
+  if (!hBaseAdmin.tableExists(secondaryIndexTableName)) {
+
hBaseAdmin.createTable(TableDescriptorBuilder.newBuilder(secondaryIndexTableName).build());
+  }
+  secondaryIndexHTable = 
this.connection.getTable(secondaryIndexTableName);
+} catch (Exception e) {
+  Throwables.propagate(e);
--- End diff --

It would be good to close the `connection` in case of exception.


> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-05 Thread Ankit Singhal (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312653#comment-16312653
 ] 

Ankit Singhal commented on TEPHRA-272:
--

Thanks [~poornachandra] for taking a look. Actually, Travis is complaining 
about the HBase dependencies as the beta1 release of HBase has not happened 
yet(Vote on RC is going on, so probably we will have it available soon), 
Though I have tested it locally with the SNAPSHOT version of beta1 and all 
tests are passing. 

> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-04 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311387#comment-16311387
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

Github user poornachandra commented on the issue:

https://github.com/apache/incubator-tephra/pull/67
  
@ankitsinghal I'm taking a look at the PR. Meanwhile, the Travis build has 
failed. Looks like there is some issue finding HBase 2.0 dependencies. Can you 
take a look at it?

Also, I'm on vacation this month. So there may be a delay in my replies.



> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2018-01-03 Thread Ankit Singhal (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310814#comment-16310814
 ] 

Ankit Singhal commented on TEPHRA-272:
--

ping [~poornachandra], can you please help here

> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2017-12-18 Thread Ankit Singhal (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294911#comment-16294911
 ] 

Ankit Singhal commented on TEPHRA-272:
--

Raised https://github.com/apache/incubator-tephra/pull/67 for the same. 
[~poornachandra], can you please review.

currently, all the tests are passing but need a cautious look at preFlush and 
preCompact hooks as now we don't have the leverage to create store scanner in 
hooks and also at the TransactionAwareTable where many APIs like commit() etc 
has been removed.

> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Poorna Chandra
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (TEPHRA-272) Add HBase 2.0 compatibility module

2017-12-18 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TEPHRA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294906#comment-16294906
 ] 

ASF GitHub Bot commented on TEPHRA-272:
---

GitHub user ankitsinghal opened a pull request:

https://github.com/apache/incubator-tephra/pull/67

TEPHRA-272 Add HBase 2.0 compatibility module



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ankitsinghal/incubator-tephra master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-tephra/pull/67.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #67


commit fde2186426390913d4c2b7925a20c110071aa3b3
Author: Ankit Singhal 
Date:   2017-12-18T12:25:08Z

TEPHRA-272 Add HBase 2.0 compatibility module




> Add HBase 2.0 compatibility module
> --
>
> Key: TEPHRA-272
> URL: https://issues.apache.org/jira/browse/TEPHRA-272
> Project: Tephra
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Poorna Chandra
>  Labels: HBase-2.0
> Fix For: 0.14.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)