Build failed in Jenkins: Phoenix-4.x-HBase-1.1 #591

[Apache Jenkins Server]

See 


Changes:

[jtaylor] PHOENIX-4189 Introduce a class that wraps the Map of primary key data

[jtaylor] PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN 
set

--
[...truncated 99.40 KB...]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.116 s 
- in org.apache.phoenix.end2end.index.DropMetadataIT
[INFO] Running org.apache.phoenix.end2end.index.IndexIT
[INFO] Running org.apache.phoenix.end2end.index.GlobalIndexOptimizationIT
[INFO] Running org.apache.phoenix.end2end.index.IndexExpressionIT
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.636 s 
- in org.apache.phoenix.end2end.index.GlobalIndexOptimizationIT
[INFO] Running org.apache.phoenix.end2end.index.IndexMetadataIT
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 237.227 
s - in org.apache.phoenix.end2end.UpgradeIT
[INFO] Running org.apache.phoenix.end2end.index.MutableIndexIT
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 54.576 
s - in org.apache.phoenix.end2end.index.IndexMetadataIT
[INFO] Running org.apache.phoenix.end2end.index.MutableIndexSplitForwardScanIT
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 155.839 
s - in org.apache.phoenix.end2end.index.MutableIndexSplitForwardScanIT
[INFO] Running org.apache.phoenix.end2end.index.MutableIndexSplitReverseScanIT
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 322.108 
s - in org.apache.phoenix.end2end.index.DropColumnIT
[INFO] Running org.apache.phoenix.end2end.index.SaltedIndexIT
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.406 s 
- in org.apache.phoenix.end2end.index.SaltedIndexIT
[INFO] Running org.apache.phoenix.end2end.index.ViewIndexIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.663 s 
- in org.apache.phoenix.end2end.index.ViewIndexIT
[INFO] Running org.apache.phoenix.end2end.index.txn.MutableRollbackIT
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 154.176 
s - in org.apache.phoenix.end2end.index.MutableIndexSplitReverseScanIT
[INFO] Running org.apache.phoenix.end2end.index.txn.RollbackIT
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.747 s 
- in org.apache.phoenix.end2end.index.txn.MutableRollbackIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.169 s 
- in org.apache.phoenix.end2end.salted.SaltedTableIT
[INFO] Tests run: 102, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
1,134.891 s - in org.apache.phoenix.end2end.SortMergeJoinIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 57.937 s 
- in org.apache.phoenix.end2end.index.txn.RollbackIT
[INFO] Running org.apache.phoenix.iterate.PhoenixQueryTimeoutIT
[INFO] Tests run: 67, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 466.912 
s - in org.apache.phoenix.end2end.index.IndexExpressionIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.131 s 
- in org.apache.phoenix.iterate.PhoenixQueryTimeoutIT
[INFO] Running org.apache.phoenix.replication.SystemCatalogWALEntryFilterIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.064 s 
- in org.apache.phoenix.replication.SystemCatalogWALEntryFilterIT
[INFO] Running org.apache.phoenix.rpc.UpdateCacheIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.295 s 
- in org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
[INFO] Running org.apache.phoenix.trace.PhoenixTableMetricsWriterIT
[INFO] Running org.apache.phoenix.iterate.RoundRobinResultIteratorIT
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.409 s 
- in org.apache.phoenix.rpc.UpdateCacheIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.328 s 
- in org.apache.phoenix.trace.PhoenixTableMetricsWriterIT
[INFO] Running org.apache.phoenix.tx.FlappingTransactionIT
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.71 s 
- in org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT
[INFO] Running org.apache.phoenix.tx.ParameterizedTransactionIT
[INFO] Running org.apache.phoenix.trace.PhoenixTracingEndToEndIT
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.618 s 
- in org.apache.phoenix.iterate.RoundRobinResultIteratorIT
[INFO] Running org.apache.phoenix.tx.TransactionIT
[INFO] Tests run: 64, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 496.364 
s - in org.apache.phoenix.end2end.index.MutableIndexIT
[INFO] Running org.apache.phoenix.tx.TxCheckpointIT
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 

Jenkins build is back to normal : Phoenix | Master #1807

[Apache Jenkins Server]

See 

Jenkins build is back to normal : Phoenix-4.x-HBase-1.1 #590

[Apache Jenkins Server]

See 

Apache-Phoenix | 4.x-HBase-1.2 | Build Successful

[Apache Jenkins Server]

4.x-HBase-1.2 branch build status Successful

Source repository https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=shortlog;h=refs/heads/4.x-HBase-1.2

Compiled Artifacts https://builds.apache.org/job/Phoenix-4.x-HBase-1.2/lastSuccessfulBuild/artifact/

Test Report https://builds.apache.org/job/Phoenix-4.x-HBase-1.2/lastCompletedBuild/testReport/

Changes
[jtaylor] PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set

[jtaylor] PHOENIX-4220 Upper bound not being used in partial index rebuilder

[jtaylor] PHOENIX-4189 Introduce a class that wraps the Map of primary key data

[jtaylor] PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set



Build times for last couple of runsLatest build time is the right most | Legend blue: normal, red: test failure, gray: timeout

phoenix git commit: PHOENIX-4212 Addendum to remove unused member variables for DerivedTableIT

[jamestaylor]

Repository: phoenix
Updated Branches:
  refs/heads/master bcdf292da -> 815ce00a3


PHOENIX-4212 Addendum to remove unused member variables for DerivedTableIT


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/815ce00a
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/815ce00a
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/815ce00a

Branch: refs/heads/master
Commit: 815ce00a382e43dd3f9a72b36107a3582f3f730f
Parents: bcdf292
Author: James Taylor 
Authored: Thu Sep 21 11:20:58 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:20:58 2017 -0700

--
 .../src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java| 3 ---
 1 file changed, 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/815ce00a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
index 684d869..a190029 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
@@ -47,7 +47,6 @@ import java.util.List;
 import java.util.Map;
 import java.util.Properties;
 
-import org.apache.commons.collections.map.HashedMap;
 import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.QueryUtil;
 import org.junit.After;
@@ -66,13 +65,11 @@ import com.google.common.collect.Lists;
 public class DerivedTableIT extends ParallelStatsDisabledIT {
 private static final String tenantId = getOrganizationId();
 private static final String dynamicTableName = "_TABLENAME_REPLACEABLE_";
-private static final Map tableNames=new HashedMap();
 @Rule public TestName name = new TestName();
 
 private String[] indexDDL;
 private String[] plans;
 private String tableName;
-private Statement statement;
 
 
 public DerivedTableIT(String[] indexDDL, String[] plans) {

[4/5] phoenix git commit: PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - VariableLengthPKIT (Ethan Wang)

[jamestaylor]

http://git-wip-us.apache.org/repos/asf/phoenix/blob/490cfb68/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
index 753f2c8..e45d01e 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
@@ -43,34 +43,31 @@ import java.util.Properties;
 import org.apache.phoenix.exception.SQLExceptionCode;
 import org.apache.phoenix.schema.ConstraintViolationException;
 import org.apache.phoenix.util.DateUtil;
-import org.apache.phoenix.util.PhoenixRuntime;
 import org.apache.phoenix.util.PropertiesUtil;
 import org.junit.Test;
 
 
-
-public class VariableLengthPKIT extends BaseClientManagedTimeIT {
+public class VariableLengthPKIT extends ParallelStatsDisabledIT {
 private static final String DS1 = "1970-01-01 00:58:00";
 private static final Date D1 = toDate(DS1);
 
 private static Date toDate(String dateString) {
 return DateUtil.parseDate(dateString);
 }
-
-protected static void initGroupByRowKeyColumns(long ts) throws Exception {
-ensureTableCreated(getUrl(),PTSDB_NAME, PTSDB_NAME, null, ts-2, null);
-
+
+protected static void initGroupByRowKeyColumns(String pTSDBtableName) 
throws Exception {
+ensureTableCreated(getUrl(),pTSDBtableName, PTSDB_NAME, null, null, 
null);
 // Insert all rows at ts
-String url = getUrl() + ";" + PhoenixRuntime.CURRENT_SCN_ATTRIB + "=" 
+ ts;
+String url = getUrl();
 Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
 Connection conn = DriverManager.getConnection(url, props);
 PreparedStatement stmt = conn.prepareStatement(
-"upsert into " +
-"PTSDB(" +
-"INST, " +
-"HOST," +
-"\"DATE\")" +
-"VALUES (?, ?, CURRENT_DATE())");
+"upsert into " + pTSDBtableName+
+" (" +
+"INST, " +
+"HOST," +
+"\"DATE\")" +
+"VALUES (?, ?, CURRENT_DATE())");
 stmt.setString(1, "ab");
 stmt.setString(2, "a");
 stmt.execute();
@@ -84,44 +81,70 @@ public class VariableLengthPKIT extends 
BaseClientManagedTimeIT {
 conn.close();
 }
 
-protected static void initTableValues(byte[][] splits, long ts) throws 
Exception {
-ensureTableCreated(getUrl(),PTSDB_NAME, PTSDB_NAME, splits, ts-2, 
null);
+private static void initVarcharKeyTableValues(byte[][] splits, String 
varcharKeyTestTableName) throws Exception {
+String url = getUrl();
+Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+Connection conn = DriverManager.getConnection(url, props);
+
+String ddl = "create table " +varcharKeyTestTableName+
+"   (pk varchar not null primary key)";
+createTestTable(getUrl(), ddl, splits, null);
+PreparedStatement stmt = conn.prepareStatement(
+"upsert into " + varcharKeyTestTableName+
+"(pk) " +
+"VALUES (?)");
+stmt.setString(1, "   def");
+stmt.execute();
+stmt.setString(1, "jkl   ");
+stmt.execute();
+stmt.setString(1, "   ghi   ");
+stmt.execute();
+
+conn.commit();
+conn.close();
+}
 
+private static void initPTSDBTableValues(byte[][] splits, String 
pTSDBtableName) throws Exception {
+ensureTableCreated(getUrl(),pTSDBtableName, PTSDB_NAME, splits, null, 
null);
 // Insert all rows at ts
-String url = getUrl() + ";" + PhoenixRuntime.CURRENT_SCN_ATTRIB + "=" 
+ ts;
+String url = getUrl();
 Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
 Connection conn = DriverManager.getConnection(url, props);
 conn.setAutoCommit(true);
 PreparedStatement stmt = conn.prepareStatement(
-"upsert into " +
-"PTSDB(" +
-"INST, " +
-"HOST," +
-"\"DATE\"," +
-"VAL)" +
-"VALUES (?, ?, ?, ?)");
+"upsert into " + pTSDBtableName +
+" (" +
+"INST, " +
+"HOST," +
+"\"DATE\"," +
+"VAL)" +
+"VALUES (?, ?, ?, ?)");
 stmt.setString(1, "abc");
 stmt.setString(2, "abc-def-ghi");
 stmt.setDate(3, new Date(System.currentTimeMillis()));
 stmt.setBigDecimal(4, 

[2/5] phoenix git commit: PHOENIX-4220 Upper bound not being used in partial index rebuilder

[jamestaylor]

PHOENIX-4220 Upper bound not being used in partial index rebuilder


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/cf6c5e71
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/cf6c5e71
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/cf6c5e71

Branch: refs/heads/4.x-HBase-1.2
Commit: cf6c5e711c911c370c759f4d3b4ec081a81911b0
Parents: 0f08ae6
Author: James Taylor 
Authored: Thu Sep 21 09:48:13 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:16:18 2017 -0700

--
 .../end2end/index/PartialIndexRebuilderIT.java  | 64 +++-
 .../coprocessor/MetaDataRegionObserver.java | 12 ++--
 2 files changed, 66 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/cf6c5e71/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
index 12630f4..dfe5a28 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
@@ -20,6 +20,7 @@ package org.apache.phoenix.end2end.index;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
 import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
 
 import java.io.IOException;
 import java.sql.Connection;
@@ -45,7 +46,6 @@ import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
 import org.apache.phoenix.query.ConnectionQueryServices;
 import org.apache.phoenix.query.QueryServices;
-import org.apache.phoenix.query.QueryServicesOptions;
 import org.apache.phoenix.schema.PIndexState;
 import org.apache.phoenix.schema.PMetaData;
 import org.apache.phoenix.schema.PTable;
@@ -71,12 +71,13 @@ import com.google.common.collect.Maps;
 public class PartialIndexRebuilderIT extends BaseUniqueNamesOwnClusterIT {
 private static final Random RAND = new Random(5);
 private static final int WAIT_AFTER_DISABLED = 5000;
+private static final int REBUILD_INTERVAL = 2000;
 
 @BeforeClass
 public static void doSetup() throws Exception {
 Map serverProps = Maps.newHashMapWithExpectedSize(10);
 serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_ATTRIB, 
Boolean.TRUE.toString());
-
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_INTERVAL_ATTRIB, 
"2000");
+
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_INTERVAL_ATTRIB, 
Long.toString(REBUILD_INTERVAL));
 
serverProps.put(QueryServices.INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD, 
"12"); // give up rebuilding after 2 minutes
 
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_OVERLAP_FORWARD_TIME_ATTRIB,
 Long.toString(WAIT_AFTER_DISABLED));
 setUpTestDriver(new ReadOnlyProps(serverProps.entrySet().iterator()), 
ReadOnlyProps.EMPTY_PROPS);
@@ -641,7 +642,64 @@ public class PartialIndexRebuilderIT extends 
BaseUniqueNamesOwnClusterIT {
 private final static CountDownLatch WAIT_FOR_REBUILD_TO_START = new 
CountDownLatch(1);
 private final static CountDownLatch WAIT_FOR_INDEX_WRITE = new 
CountDownLatch(1);
 
-
+@Test
+public void testUpperBoundSetOnRebuild() throws Throwable {
+String schemaName = generateUniqueName();
+String tableName = generateUniqueName();
+String indexName = generateUniqueName();
+final String fullTableName = SchemaUtil.getTableName(schemaName, 
tableName);
+final String fullIndexName = SchemaUtil.getTableName(schemaName, 
indexName);
+PTableKey key = new PTableKey(null,fullTableName);
+final MyClock clock = new MyClock(1000);
+EnvironmentEdgeManager.injectEdge(clock);
+try (Connection conn = DriverManager.getConnection(getUrl())) {
+PMetaData metaCache = 
conn.unwrap(PhoenixConnection.class).getMetaDataCache();
+conn.createStatement().execute("CREATE TABLE " + fullTableName + 
"(k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 VARCHAR) COLUMN_ENCODED_BYTES = 0, 
STORE_NULLS=true");
+clock.time += 100;
+conn.createStatement().execute("CREATE INDEX " + indexName + " ON 
" + fullTableName + " (v1, v2)");
+clock.time += 100;
+HTableInterface metaTable = 
conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES);
+

[1/5] phoenix git commit: PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set - DerivedTableIT (Ethan Wang)

[jamestaylor]

Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.2 00103008e -> 490cfb689


PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set - 
DerivedTableIT (Ethan Wang)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/0f08ae69
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/0f08ae69
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/0f08ae69

Branch: refs/heads/4.x-HBase-1.2
Commit: 0f08ae69f5f906596638463902c357d1cbefc6b7
Parents: 0010300
Author: James Taylor 
Authored: Mon Sep 18 17:16:55 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:15:38 2017 -0700

--
 .../apache/phoenix/end2end/DerivedTableIT.java  | 954 ++-
 1 file changed, 88 insertions(+), 866 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/0f08ae69/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
index 265ad21..a190029 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
@@ -41,63 +41,114 @@ import java.sql.Connection;
 import java.sql.DriverManager;
 import java.sql.PreparedStatement;
 import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
 import java.util.Properties;
 
 import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.QueryUtil;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Rule;
 import org.junit.Test;
+import org.junit.rules.TestName;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
 
+import com.google.common.collect.Lists;
+
+
+@RunWith(Parameterized.class)
 public class DerivedTableIT extends ParallelStatsDisabledIT {
-private static String[] initTableWithIndex(String tableName) throws 
Exception {
-String tenantId = getOrganizationId();
+private static final String tenantId = getOrganizationId();
+private static final String dynamicTableName = "_TABLENAME_REPLACEABLE_";
+@Rule public TestName name = new TestName();
+
+private String[] indexDDL;
+private String[] plans;
+private String tableName;
+
+
+public DerivedTableIT(String[] indexDDL, String[] plans) {
+this.indexDDL = indexDDL;
+this.plans = plans;
+}
+
+@Before
+public void initTable() throws Exception {
+if(tableName!=null) throw new RuntimeException("Test has not been 
cleaned up.");
+tableName = generateUniqueName();
+
 initATableValues(tableName, tenantId, getDefaultSplits(tenantId), 
null, null, getUrl(), null);
-Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
-Connection conn = DriverManager.getConnection(getUrl(), props);
-String ddl="CREATE INDEX "+tableName+"_DERIVED_IDX ON "+tableName+" 
(a_byte) INCLUDE (A_STRING, B_STRING)";
-conn.createStatement().execute(ddl);
-String[] plans= {"CLIENT PARALLEL 1-WAY FULL SCAN OVER 
"+tableName+"_DERIVED_IDX\n" +
-"SERVER AGGREGATE INTO DISTINCT ROWS BY [\"A_STRING\", 
\"B_STRING\"]\n" +
-"CLIENT MERGE SORT\n" +
-"CLIENT SORTED BY [\"B_STRING\"]\n" +
-"CLIENT SORTED BY [A]\n" +
-"CLIENT AGGREGATE INTO DISTINCT ROWS BY [A]\n" +
-"CLIENT SORTED BY [A DESC]",
-
-"CLIENT PARALLEL 1-WAY FULL SCAN OVER 
"+tableName+"_DERIVED_IDX\n" +
+if (indexDDL != null && indexDDL.length > 0) {
+Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+Connection conn = DriverManager.getConnection(getUrl(), props);
+for (String ddl : indexDDL) {
+ddl=ddl.replace(dynamicTableName,tableName);
+conn.createStatement().execute(ddl);
+}
+}
+String[] newplan=new String[plans.length];
+if(plans !=null && plans.length>0){
+for(int i=0;i< plans.length;i++){
+newplan[i]=plans[i].replace(dynamicTableName,tableName);
+}
+plans = newplan;
+}
+}
+
+@After
+public void cleanUp(){
+tableName=null;
+}
+
+@Parameters(name="DerivedTableIT_{index}") // name is used by failsafe as 
file name in reports
+public static Collection data() {
+List testCases = Lists.newArrayList();
+

[3/5] phoenix git commit: PHOENIX-4189 Introduce a class that wraps the Map of primary key data (addendum)

[jamestaylor]

PHOENIX-4189 Introduce a class that wraps the Map of primary key data (addendum)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/7fc14f91
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/7fc14f91
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/7fc14f91

Branch: refs/heads/4.x-HBase-1.2
Commit: 7fc14f91d12c57354b763c25a955c8f08cf7ea99
Parents: cf6c5e7
Author: James Taylor 
Authored: Mon Sep 18 16:30:19 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:16:34 2017 -0700

--
 .../src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/7fc14f91/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
--
diff --git 
a/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java 
b/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
index b5e9dd9..7773997 100644
--- a/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
+++ b/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
@@ -77,7 +77,7 @@ public class PrimaryKeyData implements Serializable{
 }
 
 public static PrimaryKeyData deserialize(InputStream input) throws 
IOException, ClassNotFoundException {
-try (LookAheadObjectInputStream ois = new 
LookAheadObjectInputStream((InputStream) input)) {
+try (LookAheadObjectInputStream ois = new 
LookAheadObjectInputStream(input)) {
 Object obj = ois.readObject();
 if (obj instanceof PrimaryKeyData) {
 return (PrimaryKeyData) obj;

[5/5] phoenix git commit: PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - VariableLengthPKIT (Ethan Wang)

[jamestaylor]

PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - 
VariableLengthPKIT (Ethan Wang)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/490cfb68
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/490cfb68
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/490cfb68

Branch: refs/heads/4.x-HBase-1.2
Commit: 490cfb68942b18ad6b14742ec449fff8a933a344
Parents: 7fc14f9
Author: James Taylor 
Authored: Thu Sep 21 10:44:02 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:16:46 2017 -0700

--
 .../phoenix/end2end/VariableLengthPKIT.java | 1171 +-
 1 file changed, 570 insertions(+), 601 deletions(-)
--

[2/3] phoenix git commit: PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - VariableLengthPKIT (Ethan Wang)

[jamestaylor]

http://git-wip-us.apache.org/repos/asf/phoenix/blob/c6960642/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
index 753f2c8..e45d01e 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
@@ -43,34 +43,31 @@ import java.util.Properties;
 import org.apache.phoenix.exception.SQLExceptionCode;
 import org.apache.phoenix.schema.ConstraintViolationException;
 import org.apache.phoenix.util.DateUtil;
-import org.apache.phoenix.util.PhoenixRuntime;
 import org.apache.phoenix.util.PropertiesUtil;
 import org.junit.Test;
 
 
-
-public class VariableLengthPKIT extends BaseClientManagedTimeIT {
+public class VariableLengthPKIT extends ParallelStatsDisabledIT {
 private static final String DS1 = "1970-01-01 00:58:00";
 private static final Date D1 = toDate(DS1);
 
 private static Date toDate(String dateString) {
 return DateUtil.parseDate(dateString);
 }
-
-protected static void initGroupByRowKeyColumns(long ts) throws Exception {
-ensureTableCreated(getUrl(),PTSDB_NAME, PTSDB_NAME, null, ts-2, null);
-
+
+protected static void initGroupByRowKeyColumns(String pTSDBtableName) 
throws Exception {
+ensureTableCreated(getUrl(),pTSDBtableName, PTSDB_NAME, null, null, 
null);
 // Insert all rows at ts
-String url = getUrl() + ";" + PhoenixRuntime.CURRENT_SCN_ATTRIB + "=" 
+ ts;
+String url = getUrl();
 Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
 Connection conn = DriverManager.getConnection(url, props);
 PreparedStatement stmt = conn.prepareStatement(
-"upsert into " +
-"PTSDB(" +
-"INST, " +
-"HOST," +
-"\"DATE\")" +
-"VALUES (?, ?, CURRENT_DATE())");
+"upsert into " + pTSDBtableName+
+" (" +
+"INST, " +
+"HOST," +
+"\"DATE\")" +
+"VALUES (?, ?, CURRENT_DATE())");
 stmt.setString(1, "ab");
 stmt.setString(2, "a");
 stmt.execute();
@@ -84,44 +81,70 @@ public class VariableLengthPKIT extends 
BaseClientManagedTimeIT {
 conn.close();
 }
 
-protected static void initTableValues(byte[][] splits, long ts) throws 
Exception {
-ensureTableCreated(getUrl(),PTSDB_NAME, PTSDB_NAME, splits, ts-2, 
null);
+private static void initVarcharKeyTableValues(byte[][] splits, String 
varcharKeyTestTableName) throws Exception {
+String url = getUrl();
+Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+Connection conn = DriverManager.getConnection(url, props);
+
+String ddl = "create table " +varcharKeyTestTableName+
+"   (pk varchar not null primary key)";
+createTestTable(getUrl(), ddl, splits, null);
+PreparedStatement stmt = conn.prepareStatement(
+"upsert into " + varcharKeyTestTableName+
+"(pk) " +
+"VALUES (?)");
+stmt.setString(1, "   def");
+stmt.execute();
+stmt.setString(1, "jkl   ");
+stmt.execute();
+stmt.setString(1, "   ghi   ");
+stmt.execute();
+
+conn.commit();
+conn.close();
+}
 
+private static void initPTSDBTableValues(byte[][] splits, String 
pTSDBtableName) throws Exception {
+ensureTableCreated(getUrl(),pTSDBtableName, PTSDB_NAME, splits, null, 
null);
 // Insert all rows at ts
-String url = getUrl() + ";" + PhoenixRuntime.CURRENT_SCN_ATTRIB + "=" 
+ ts;
+String url = getUrl();
 Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
 Connection conn = DriverManager.getConnection(url, props);
 conn.setAutoCommit(true);
 PreparedStatement stmt = conn.prepareStatement(
-"upsert into " +
-"PTSDB(" +
-"INST, " +
-"HOST," +
-"\"DATE\"," +
-"VAL)" +
-"VALUES (?, ?, ?, ?)");
+"upsert into " + pTSDBtableName +
+" (" +
+"INST, " +
+"HOST," +
+"\"DATE\"," +
+"VAL)" +
+"VALUES (?, ?, ?, ?)");
 stmt.setString(1, "abc");
 stmt.setString(2, "abc-def-ghi");
 stmt.setDate(3, new Date(System.currentTimeMillis()));
 stmt.setBigDecimal(4, 

[1/3] phoenix git commit: PHOENIX-4189 Introduce a class that wraps the Map of primary key data (addendum)

[jamestaylor]

Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.1 5733c0ec1 -> c69606429


PHOENIX-4189 Introduce a class that wraps the Map of primary key data (addendum)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/57694f78
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/57694f78
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/57694f78

Branch: refs/heads/4.x-HBase-1.1
Commit: 57694f78dc086bdf11b34bf45cfa4fe1ece9e6a9
Parents: 5733c0e
Author: James Taylor 
Authored: Mon Sep 18 16:30:19 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:12:55 2017 -0700

--
 .../src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/57694f78/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
--
diff --git 
a/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java 
b/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
index b5e9dd9..7773997 100644
--- a/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
+++ b/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
@@ -77,7 +77,7 @@ public class PrimaryKeyData implements Serializable{
 }
 
 public static PrimaryKeyData deserialize(InputStream input) throws 
IOException, ClassNotFoundException {
-try (LookAheadObjectInputStream ois = new 
LookAheadObjectInputStream((InputStream) input)) {
+try (LookAheadObjectInputStream ois = new 
LookAheadObjectInputStream(input)) {
 Object obj = ois.readObject();
 if (obj instanceof PrimaryKeyData) {
 return (PrimaryKeyData) obj;

[3/3] phoenix git commit: PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - VariableLengthPKIT (Ethan Wang)

[jamestaylor]

PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - 
VariableLengthPKIT (Ethan Wang)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/c6960642
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/c6960642
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/c6960642

Branch: refs/heads/4.x-HBase-1.1
Commit: c69606429d9b48d719ae27910fa9afeba63d157a
Parents: 57694f7
Author: James Taylor 
Authored: Thu Sep 21 10:44:02 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:13:14 2017 -0700

--
 .../phoenix/end2end/VariableLengthPKIT.java | 1171 +-
 1 file changed, 570 insertions(+), 601 deletions(-)
--

[1/2] phoenix git commit: PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set - DerivedTableIT (Ethan Wang)

[jamestaylor]

Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.1 0eed27dd4 -> 5733c0ec1


PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set - 
DerivedTableIT (Ethan Wang)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/83fdf702
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/83fdf702
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/83fdf702

Branch: refs/heads/4.x-HBase-1.1
Commit: 83fdf7025f43d6fe0f654183ead2838e27b4c9b5
Parents: 0eed27d
Author: James Taylor 
Authored: Mon Sep 18 17:16:55 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:07:18 2017 -0700

--
 .../apache/phoenix/end2end/DerivedTableIT.java  | 954 ++-
 1 file changed, 88 insertions(+), 866 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/83fdf702/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
index 265ad21..a190029 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
@@ -41,63 +41,114 @@ import java.sql.Connection;
 import java.sql.DriverManager;
 import java.sql.PreparedStatement;
 import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
 import java.util.Properties;
 
 import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.QueryUtil;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Rule;
 import org.junit.Test;
+import org.junit.rules.TestName;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
 
+import com.google.common.collect.Lists;
+
+
+@RunWith(Parameterized.class)
 public class DerivedTableIT extends ParallelStatsDisabledIT {
-private static String[] initTableWithIndex(String tableName) throws 
Exception {
-String tenantId = getOrganizationId();
+private static final String tenantId = getOrganizationId();
+private static final String dynamicTableName = "_TABLENAME_REPLACEABLE_";
+@Rule public TestName name = new TestName();
+
+private String[] indexDDL;
+private String[] plans;
+private String tableName;
+
+
+public DerivedTableIT(String[] indexDDL, String[] plans) {
+this.indexDDL = indexDDL;
+this.plans = plans;
+}
+
+@Before
+public void initTable() throws Exception {
+if(tableName!=null) throw new RuntimeException("Test has not been 
cleaned up.");
+tableName = generateUniqueName();
+
 initATableValues(tableName, tenantId, getDefaultSplits(tenantId), 
null, null, getUrl(), null);
-Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
-Connection conn = DriverManager.getConnection(getUrl(), props);
-String ddl="CREATE INDEX "+tableName+"_DERIVED_IDX ON "+tableName+" 
(a_byte) INCLUDE (A_STRING, B_STRING)";
-conn.createStatement().execute(ddl);
-String[] plans= {"CLIENT PARALLEL 1-WAY FULL SCAN OVER 
"+tableName+"_DERIVED_IDX\n" +
-"SERVER AGGREGATE INTO DISTINCT ROWS BY [\"A_STRING\", 
\"B_STRING\"]\n" +
-"CLIENT MERGE SORT\n" +
-"CLIENT SORTED BY [\"B_STRING\"]\n" +
-"CLIENT SORTED BY [A]\n" +
-"CLIENT AGGREGATE INTO DISTINCT ROWS BY [A]\n" +
-"CLIENT SORTED BY [A DESC]",
-
-"CLIENT PARALLEL 1-WAY FULL SCAN OVER 
"+tableName+"_DERIVED_IDX\n" +
+if (indexDDL != null && indexDDL.length > 0) {
+Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+Connection conn = DriverManager.getConnection(getUrl(), props);
+for (String ddl : indexDDL) {
+ddl=ddl.replace(dynamicTableName,tableName);
+conn.createStatement().execute(ddl);
+}
+}
+String[] newplan=new String[plans.length];
+if(plans !=null && plans.length>0){
+for(int i=0;i< plans.length;i++){
+newplan[i]=plans[i].replace(dynamicTableName,tableName);
+}
+plans = newplan;
+}
+}
+
+@After
+public void cleanUp(){
+tableName=null;
+}
+
+@Parameters(name="DerivedTableIT_{index}") // name is used by failsafe as 
file name in reports
+public static Collection data() {
+List testCases = Lists.newArrayList();
+

[2/2] phoenix git commit: PHOENIX-4220 Upper bound not being used in partial index rebuilder

[jamestaylor]

PHOENIX-4220 Upper bound not being used in partial index rebuilder


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/5733c0ec
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/5733c0ec
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/5733c0ec

Branch: refs/heads/4.x-HBase-1.1
Commit: 5733c0ec1332fcc59deec5ccc79219fa5503fd44
Parents: 83fdf70
Author: James Taylor 
Authored: Thu Sep 21 09:48:13 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:07:23 2017 -0700

--
 .../end2end/index/PartialIndexRebuilderIT.java  | 64 +++-
 .../coprocessor/MetaDataRegionObserver.java | 12 ++--
 2 files changed, 66 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/5733c0ec/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
index 12630f4..dfe5a28 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
@@ -20,6 +20,7 @@ package org.apache.phoenix.end2end.index;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
 import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
 
 import java.io.IOException;
 import java.sql.Connection;
@@ -45,7 +46,6 @@ import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
 import org.apache.phoenix.query.ConnectionQueryServices;
 import org.apache.phoenix.query.QueryServices;
-import org.apache.phoenix.query.QueryServicesOptions;
 import org.apache.phoenix.schema.PIndexState;
 import org.apache.phoenix.schema.PMetaData;
 import org.apache.phoenix.schema.PTable;
@@ -71,12 +71,13 @@ import com.google.common.collect.Maps;
 public class PartialIndexRebuilderIT extends BaseUniqueNamesOwnClusterIT {
 private static final Random RAND = new Random(5);
 private static final int WAIT_AFTER_DISABLED = 5000;
+private static final int REBUILD_INTERVAL = 2000;
 
 @BeforeClass
 public static void doSetup() throws Exception {
 Map serverProps = Maps.newHashMapWithExpectedSize(10);
 serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_ATTRIB, 
Boolean.TRUE.toString());
-
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_INTERVAL_ATTRIB, 
"2000");
+
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_INTERVAL_ATTRIB, 
Long.toString(REBUILD_INTERVAL));
 
serverProps.put(QueryServices.INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD, 
"12"); // give up rebuilding after 2 minutes
 
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_OVERLAP_FORWARD_TIME_ATTRIB,
 Long.toString(WAIT_AFTER_DISABLED));
 setUpTestDriver(new ReadOnlyProps(serverProps.entrySet().iterator()), 
ReadOnlyProps.EMPTY_PROPS);
@@ -641,7 +642,64 @@ public class PartialIndexRebuilderIT extends 
BaseUniqueNamesOwnClusterIT {
 private final static CountDownLatch WAIT_FOR_REBUILD_TO_START = new 
CountDownLatch(1);
 private final static CountDownLatch WAIT_FOR_INDEX_WRITE = new 
CountDownLatch(1);
 
-
+@Test
+public void testUpperBoundSetOnRebuild() throws Throwable {
+String schemaName = generateUniqueName();
+String tableName = generateUniqueName();
+String indexName = generateUniqueName();
+final String fullTableName = SchemaUtil.getTableName(schemaName, 
tableName);
+final String fullIndexName = SchemaUtil.getTableName(schemaName, 
indexName);
+PTableKey key = new PTableKey(null,fullTableName);
+final MyClock clock = new MyClock(1000);
+EnvironmentEdgeManager.injectEdge(clock);
+try (Connection conn = DriverManager.getConnection(getUrl())) {
+PMetaData metaCache = 
conn.unwrap(PhoenixConnection.class).getMetaDataCache();
+conn.createStatement().execute("CREATE TABLE " + fullTableName + 
"(k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 VARCHAR) COLUMN_ENCODED_BYTES = 0, 
STORE_NULLS=true");
+clock.time += 100;
+conn.createStatement().execute("CREATE INDEX " + indexName + " ON 
" + fullTableName + " (v1, v2)");
+clock.time += 100;
+HTableInterface metaTable = 
conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES);
+

[3/5] phoenix git commit: PHOENIX-4220 Upper bound not being used in partial index rebuilder

[jamestaylor]

PHOENIX-4220 Upper bound not being used in partial index rebuilder


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/2da53a65
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/2da53a65
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/2da53a65

Branch: refs/heads/4.x-HBase-0.98
Commit: 2da53a65c42b5dbe59852be783a4744c27c31d1e
Parents: c984951
Author: James Taylor 
Authored: Thu Sep 21 09:48:13 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:03:47 2017 -0700

--
 .../end2end/index/PartialIndexRebuilderIT.java  | 64 +++-
 .../coprocessor/MetaDataRegionObserver.java | 12 ++--
 2 files changed, 66 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/2da53a65/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
index 12630f4..dfe5a28 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java
@@ -20,6 +20,7 @@ package org.apache.phoenix.end2end.index;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
 import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
 
 import java.io.IOException;
 import java.sql.Connection;
@@ -45,7 +46,6 @@ import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
 import org.apache.phoenix.query.ConnectionQueryServices;
 import org.apache.phoenix.query.QueryServices;
-import org.apache.phoenix.query.QueryServicesOptions;
 import org.apache.phoenix.schema.PIndexState;
 import org.apache.phoenix.schema.PMetaData;
 import org.apache.phoenix.schema.PTable;
@@ -71,12 +71,13 @@ import com.google.common.collect.Maps;
 public class PartialIndexRebuilderIT extends BaseUniqueNamesOwnClusterIT {
 private static final Random RAND = new Random(5);
 private static final int WAIT_AFTER_DISABLED = 5000;
+private static final int REBUILD_INTERVAL = 2000;
 
 @BeforeClass
 public static void doSetup() throws Exception {
 Map serverProps = Maps.newHashMapWithExpectedSize(10);
 serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_ATTRIB, 
Boolean.TRUE.toString());
-
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_INTERVAL_ATTRIB, 
"2000");
+
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_INTERVAL_ATTRIB, 
Long.toString(REBUILD_INTERVAL));
 
serverProps.put(QueryServices.INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD, 
"12"); // give up rebuilding after 2 minutes
 
serverProps.put(QueryServices.INDEX_FAILURE_HANDLING_REBUILD_OVERLAP_FORWARD_TIME_ATTRIB,
 Long.toString(WAIT_AFTER_DISABLED));
 setUpTestDriver(new ReadOnlyProps(serverProps.entrySet().iterator()), 
ReadOnlyProps.EMPTY_PROPS);
@@ -641,7 +642,64 @@ public class PartialIndexRebuilderIT extends 
BaseUniqueNamesOwnClusterIT {
 private final static CountDownLatch WAIT_FOR_REBUILD_TO_START = new 
CountDownLatch(1);
 private final static CountDownLatch WAIT_FOR_INDEX_WRITE = new 
CountDownLatch(1);
 
-
+@Test
+public void testUpperBoundSetOnRebuild() throws Throwable {
+String schemaName = generateUniqueName();
+String tableName = generateUniqueName();
+String indexName = generateUniqueName();
+final String fullTableName = SchemaUtil.getTableName(schemaName, 
tableName);
+final String fullIndexName = SchemaUtil.getTableName(schemaName, 
indexName);
+PTableKey key = new PTableKey(null,fullTableName);
+final MyClock clock = new MyClock(1000);
+EnvironmentEdgeManager.injectEdge(clock);
+try (Connection conn = DriverManager.getConnection(getUrl())) {
+PMetaData metaCache = 
conn.unwrap(PhoenixConnection.class).getMetaDataCache();
+conn.createStatement().execute("CREATE TABLE " + fullTableName + 
"(k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 VARCHAR) COLUMN_ENCODED_BYTES = 0, 
STORE_NULLS=true");
+clock.time += 100;
+conn.createStatement().execute("CREATE INDEX " + indexName + " ON 
" + fullTableName + " (v1, v2)");
+clock.time += 100;
+HTableInterface metaTable = 
conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES);
+

[5/5] phoenix git commit: PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - VariableLengthPKIT (Ethan Wang)

[jamestaylor]

PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - 
VariableLengthPKIT (Ethan Wang)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/f7989151
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/f7989151
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/f7989151

Branch: refs/heads/4.x-HBase-0.98
Commit: f798915151ba3f8fe6bfc49e6119d8fc5e8bd9e5
Parents: 2da53a6
Author: James Taylor 
Authored: Thu Sep 21 10:44:02 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:04:02 2017 -0700

--
 .../phoenix/end2end/VariableLengthPKIT.java | 1171 +-
 1 file changed, 570 insertions(+), 601 deletions(-)
--

[2/5] phoenix git commit: PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set - DerivedTableIT (Ethan Wang)

[jamestaylor]

PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set - 
DerivedTableIT (Ethan Wang)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/c984951d
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/c984951d
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/c984951d

Branch: refs/heads/4.x-HBase-0.98
Commit: c984951d69d45f8578895acc43e42cfd53ae6045
Parents: ed81dce
Author: James Taylor 
Authored: Mon Sep 18 17:16:55 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 11:03:20 2017 -0700

--
 .../apache/phoenix/end2end/DerivedTableIT.java  | 955 ++-
 1 file changed, 87 insertions(+), 868 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/c984951d/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
index 3bbd347..fd769bc 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
@@ -41,63 +41,112 @@ import java.sql.Connection;
 import java.sql.DriverManager;
 import java.sql.PreparedStatement;
 import java.sql.ResultSet;
+import java.util.Collection;
+import java.util.List;
 import java.util.Properties;
 
 import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.QueryUtil;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Rule;
 import org.junit.Test;
+import org.junit.rules.TestName;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
 
+import com.google.common.collect.Lists;
+
+
+@RunWith(Parameterized.class)
 public class DerivedTableIT extends ParallelStatsDisabledIT {
-private static String[] initTableWithIndex(String tableName) throws 
Exception {
-String tenantId = getOrganizationId();
+private static final String tenantId = getOrganizationId();
+private static final String dynamicTableName = "_TABLENAME_REPLACEABLE_";
+@Rule public TestName name = new TestName();
+
+private String[] indexDDL;
+private String[] plans;
+private String tableName;
+
+
+public DerivedTableIT(String[] indexDDL, String[] plans) {
+this.indexDDL = indexDDL;
+this.plans = plans;
+}
+
+@Before
+public void initTable() throws Exception {
+if(tableName!=null) throw new RuntimeException("Test has not been 
cleaned up.");
+tableName = generateUniqueName();
+
 initATableValues(tableName, tenantId, getDefaultSplits(tenantId), 
null, null, getUrl(), null);
-Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
-Connection conn = DriverManager.getConnection(getUrl(), props);
-String ddl="CREATE INDEX "+tableName+"_DERIVED_IDX ON "+tableName+" 
(a_byte) INCLUDE (A_STRING, B_STRING)";
-conn.createStatement().execute(ddl);
-String[] plans= {"CLIENT PARALLEL 1-WAY FULL SCAN OVER 
"+tableName+"_DERIVED_IDX\n" +
-"SERVER AGGREGATE INTO DISTINCT ROWS BY [\"A_STRING\", 
\"B_STRING\"]\n" +
-"CLIENT MERGE SORT\n" +
-"CLIENT SORTED BY [\"B_STRING\"]\n" +
-"CLIENT SORTED BY [A]\n" +
-"CLIENT AGGREGATE INTO DISTINCT ROWS BY [A]\n" +
-"CLIENT SORTED BY [A DESC]",
-
-"CLIENT PARALLEL 1-WAY FULL SCAN OVER 
"+tableName+"_DERIVED_IDX\n" +
+if (indexDDL != null && indexDDL.length > 0) {
+Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+Connection conn = DriverManager.getConnection(getUrl(), props);
+for (String ddl : indexDDL) {
+ddl=ddl.replace(dynamicTableName,tableName);
+conn.createStatement().execute(ddl);
+}
+}
+String[] newplan=new String[plans.length];
+if(plans !=null && plans.length>0){
+for(int i=0;i< plans.length;i++){
+newplan[i]=plans[i].replace(dynamicTableName,tableName);
+}
+plans = newplan;
+}
+}
+
+@After
+public void cleanUp(){
+tableName=null;
+}
+
+@Parameters(name="DerivedTableIT_{index}") // name is used by failsafe as 
file name in reports
+public static Collection data() {
+List testCases = Lists.newArrayList();
+testCases.add(new String[][] {
+{
+"CREATE INDEX "+dynamicTableName+"_DERIVED_IDX ON 
"+dynamicTableName+" 

[4/5] phoenix git commit: PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - VariableLengthPKIT (Ethan Wang)

[jamestaylor]

http://git-wip-us.apache.org/repos/asf/phoenix/blob/f7989151/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
index 753f2c8..e45d01e 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
@@ -43,34 +43,31 @@ import java.util.Properties;
 import org.apache.phoenix.exception.SQLExceptionCode;
 import org.apache.phoenix.schema.ConstraintViolationException;
 import org.apache.phoenix.util.DateUtil;
-import org.apache.phoenix.util.PhoenixRuntime;
 import org.apache.phoenix.util.PropertiesUtil;
 import org.junit.Test;
 
 
-
-public class VariableLengthPKIT extends BaseClientManagedTimeIT {
+public class VariableLengthPKIT extends ParallelStatsDisabledIT {
 private static final String DS1 = "1970-01-01 00:58:00";
 private static final Date D1 = toDate(DS1);
 
 private static Date toDate(String dateString) {
 return DateUtil.parseDate(dateString);
 }
-
-protected static void initGroupByRowKeyColumns(long ts) throws Exception {
-ensureTableCreated(getUrl(),PTSDB_NAME, PTSDB_NAME, null, ts-2, null);
-
+
+protected static void initGroupByRowKeyColumns(String pTSDBtableName) 
throws Exception {
+ensureTableCreated(getUrl(),pTSDBtableName, PTSDB_NAME, null, null, 
null);
 // Insert all rows at ts
-String url = getUrl() + ";" + PhoenixRuntime.CURRENT_SCN_ATTRIB + "=" 
+ ts;
+String url = getUrl();
 Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
 Connection conn = DriverManager.getConnection(url, props);
 PreparedStatement stmt = conn.prepareStatement(
-"upsert into " +
-"PTSDB(" +
-"INST, " +
-"HOST," +
-"\"DATE\")" +
-"VALUES (?, ?, CURRENT_DATE())");
+"upsert into " + pTSDBtableName+
+" (" +
+"INST, " +
+"HOST," +
+"\"DATE\")" +
+"VALUES (?, ?, CURRENT_DATE())");
 stmt.setString(1, "ab");
 stmt.setString(2, "a");
 stmt.execute();
@@ -84,44 +81,70 @@ public class VariableLengthPKIT extends 
BaseClientManagedTimeIT {
 conn.close();
 }
 
-protected static void initTableValues(byte[][] splits, long ts) throws 
Exception {
-ensureTableCreated(getUrl(),PTSDB_NAME, PTSDB_NAME, splits, ts-2, 
null);
+private static void initVarcharKeyTableValues(byte[][] splits, String 
varcharKeyTestTableName) throws Exception {
+String url = getUrl();
+Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+Connection conn = DriverManager.getConnection(url, props);
+
+String ddl = "create table " +varcharKeyTestTableName+
+"   (pk varchar not null primary key)";
+createTestTable(getUrl(), ddl, splits, null);
+PreparedStatement stmt = conn.prepareStatement(
+"upsert into " + varcharKeyTestTableName+
+"(pk) " +
+"VALUES (?)");
+stmt.setString(1, "   def");
+stmt.execute();
+stmt.setString(1, "jkl   ");
+stmt.execute();
+stmt.setString(1, "   ghi   ");
+stmt.execute();
+
+conn.commit();
+conn.close();
+}
 
+private static void initPTSDBTableValues(byte[][] splits, String 
pTSDBtableName) throws Exception {
+ensureTableCreated(getUrl(),pTSDBtableName, PTSDB_NAME, splits, null, 
null);
 // Insert all rows at ts
-String url = getUrl() + ";" + PhoenixRuntime.CURRENT_SCN_ATTRIB + "=" 
+ ts;
+String url = getUrl();
 Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
 Connection conn = DriverManager.getConnection(url, props);
 conn.setAutoCommit(true);
 PreparedStatement stmt = conn.prepareStatement(
-"upsert into " +
-"PTSDB(" +
-"INST, " +
-"HOST," +
-"\"DATE\"," +
-"VAL)" +
-"VALUES (?, ?, ?, ?)");
+"upsert into " + pTSDBtableName +
+" (" +
+"INST, " +
+"HOST," +
+"\"DATE\"," +
+"VAL)" +
+"VALUES (?, ?, ?, ?)");
 stmt.setString(1, "abc");
 stmt.setString(2, "abc-def-ghi");
 stmt.setDate(3, new Date(System.currentTimeMillis()));
 stmt.setBigDecimal(4, 

[1/5] phoenix git commit: PHOENIX-4189 Introduce a class that wraps the Map of primary key data (addendum)

[jamestaylor]

Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-0.98 952260d52 -> f79891515


PHOENIX-4189 Introduce a class that wraps the Map of primary key data (addendum)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/ed81dce1
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/ed81dce1
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/ed81dce1

Branch: refs/heads/4.x-HBase-0.98
Commit: ed81dce1f0c72b6b8ddc1574b8a05448e0b8b44d
Parents: 952260d
Author: James Taylor 
Authored: Mon Sep 18 16:30:19 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 10:56:22 2017 -0700

--
 .../src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/ed81dce1/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
--
diff --git 
a/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java 
b/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
index b5e9dd9..7773997 100644
--- a/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
+++ b/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
@@ -77,7 +77,7 @@ public class PrimaryKeyData implements Serializable{
 }
 
 public static PrimaryKeyData deserialize(InputStream input) throws 
IOException, ClassNotFoundException {
-try (LookAheadObjectInputStream ois = new 
LookAheadObjectInputStream((InputStream) input)) {
+try (LookAheadObjectInputStream ois = new 
LookAheadObjectInputStream(input)) {
 Object obj = ois.readObject();
 if (obj instanceof PrimaryKeyData) {
 return (PrimaryKeyData) obj;

[5/5] phoenix git commit: PHOENIX-4189 Introduce a class that wraps the Map of primary key data (addendum)

[jamestaylor]

PHOENIX-4189 Introduce a class that wraps the Map of primary key data (addendum)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/e47e7847
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/e47e7847
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/e47e7847

Branch: refs/heads/master
Commit: e47e78477802940148b6457021a6362cefb002e6
Parents: 3fb104e
Author: James Taylor 
Authored: Mon Sep 18 16:30:19 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 10:51:19 2017 -0700

--
 .../src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/e47e7847/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
--
diff --git 
a/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java 
b/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
index b5e9dd9..7773997 100644
--- a/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
+++ b/phoenix-hive/src/main/java/org/apache/phoenix/hive/PrimaryKeyData.java
@@ -77,7 +77,7 @@ public class PrimaryKeyData implements Serializable{
 }
 
 public static PrimaryKeyData deserialize(InputStream input) throws 
IOException, ClassNotFoundException {
-try (LookAheadObjectInputStream ois = new 
LookAheadObjectInputStream((InputStream) input)) {
+try (LookAheadObjectInputStream ois = new 
LookAheadObjectInputStream(input)) {
 Object obj = ois.readObject();
 if (obj instanceof PrimaryKeyData) {
 return (PrimaryKeyData) obj;

[3/5] phoenix git commit: PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set - DerivedTableIT (Ethan Wang)

[jamestaylor]

PHOENIX-4212 Disallow DML operations on connections with CURRENT_SCN set - 
DerivedTableIT (Ethan Wang)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/bcb755d1
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/bcb755d1
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/bcb755d1

Branch: refs/heads/master
Commit: bcb755d1de3e6968a12671c46efa34fe2f96a518
Parents: e47e784
Author: James Taylor 
Authored: Mon Sep 18 17:16:55 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 10:51:19 2017 -0700

--
 .../apache/phoenix/end2end/DerivedTableIT.java  | 957 ++-
 1 file changed, 91 insertions(+), 866 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/bcb755d1/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
index 265ad21..684d869 100644
--- a/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
+++ b/phoenix-core/src/it/java/org/apache/phoenix/end2end/DerivedTableIT.java
@@ -41,63 +41,117 @@ import java.sql.Connection;
 import java.sql.DriverManager;
 import java.sql.PreparedStatement;
 import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
 import java.util.Properties;
 
+import org.apache.commons.collections.map.HashedMap;
 import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.QueryUtil;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Rule;
 import org.junit.Test;
+import org.junit.rules.TestName;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
 
+import com.google.common.collect.Lists;
+
+
+@RunWith(Parameterized.class)
 public class DerivedTableIT extends ParallelStatsDisabledIT {
-private static String[] initTableWithIndex(String tableName) throws 
Exception {
-String tenantId = getOrganizationId();
+private static final String tenantId = getOrganizationId();
+private static final String dynamicTableName = "_TABLENAME_REPLACEABLE_";
+private static final Map tableNames=new HashedMap();
+@Rule public TestName name = new TestName();
+
+private String[] indexDDL;
+private String[] plans;
+private String tableName;
+private Statement statement;
+
+
+public DerivedTableIT(String[] indexDDL, String[] plans) {
+this.indexDDL = indexDDL;
+this.plans = plans;
+}
+
+@Before
+public void initTable() throws Exception {
+if(tableName!=null) throw new RuntimeException("Test has not been 
cleaned up.");
+tableName = generateUniqueName();
+
 initATableValues(tableName, tenantId, getDefaultSplits(tenantId), 
null, null, getUrl(), null);
-Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
-Connection conn = DriverManager.getConnection(getUrl(), props);
-String ddl="CREATE INDEX "+tableName+"_DERIVED_IDX ON "+tableName+" 
(a_byte) INCLUDE (A_STRING, B_STRING)";
-conn.createStatement().execute(ddl);
-String[] plans= {"CLIENT PARALLEL 1-WAY FULL SCAN OVER 
"+tableName+"_DERIVED_IDX\n" +
-"SERVER AGGREGATE INTO DISTINCT ROWS BY [\"A_STRING\", 
\"B_STRING\"]\n" +
-"CLIENT MERGE SORT\n" +
-"CLIENT SORTED BY [\"B_STRING\"]\n" +
-"CLIENT SORTED BY [A]\n" +
-"CLIENT AGGREGATE INTO DISTINCT ROWS BY [A]\n" +
-"CLIENT SORTED BY [A DESC]",
-
-"CLIENT PARALLEL 1-WAY FULL SCAN OVER 
"+tableName+"_DERIVED_IDX\n" +
+if (indexDDL != null && indexDDL.length > 0) {
+Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+Connection conn = DriverManager.getConnection(getUrl(), props);
+for (String ddl : indexDDL) {
+ddl=ddl.replace(dynamicTableName,tableName);
+conn.createStatement().execute(ddl);
+}
+}
+String[] newplan=new String[plans.length];
+if(plans !=null && plans.length>0){
+for(int i=0;i< plans.length;i++){
+newplan[i]=plans[i].replace(dynamicTableName,tableName);
+}
+plans = newplan;
+}
+}
+
+@After
+public void cleanUp(){
+tableName=null;
+}
+
+@Parameters(name="DerivedTableIT_{index}") // name is used by failsafe as 
file name in reports
+public static Collection data() {
+   

[1/5] phoenix git commit: PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - VariableLengthPKIT (Ethan Wang)

[jamestaylor]

Repository: phoenix
Updated Branches:
  refs/heads/master 3fb104ee5 -> bcdf292da


http://git-wip-us.apache.org/repos/asf/phoenix/blob/bcdf292d/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
index 753f2c8..e45d01e 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/VariableLengthPKIT.java
@@ -43,34 +43,31 @@ import java.util.Properties;
 import org.apache.phoenix.exception.SQLExceptionCode;
 import org.apache.phoenix.schema.ConstraintViolationException;
 import org.apache.phoenix.util.DateUtil;
-import org.apache.phoenix.util.PhoenixRuntime;
 import org.apache.phoenix.util.PropertiesUtil;
 import org.junit.Test;
 
 
-
-public class VariableLengthPKIT extends BaseClientManagedTimeIT {
+public class VariableLengthPKIT extends ParallelStatsDisabledIT {
 private static final String DS1 = "1970-01-01 00:58:00";
 private static final Date D1 = toDate(DS1);
 
 private static Date toDate(String dateString) {
 return DateUtil.parseDate(dateString);
 }
-
-protected static void initGroupByRowKeyColumns(long ts) throws Exception {
-ensureTableCreated(getUrl(),PTSDB_NAME, PTSDB_NAME, null, ts-2, null);
-
+
+protected static void initGroupByRowKeyColumns(String pTSDBtableName) 
throws Exception {
+ensureTableCreated(getUrl(),pTSDBtableName, PTSDB_NAME, null, null, 
null);
 // Insert all rows at ts
-String url = getUrl() + ";" + PhoenixRuntime.CURRENT_SCN_ATTRIB + "=" 
+ ts;
+String url = getUrl();
 Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
 Connection conn = DriverManager.getConnection(url, props);
 PreparedStatement stmt = conn.prepareStatement(
-"upsert into " +
-"PTSDB(" +
-"INST, " +
-"HOST," +
-"\"DATE\")" +
-"VALUES (?, ?, CURRENT_DATE())");
+"upsert into " + pTSDBtableName+
+" (" +
+"INST, " +
+"HOST," +
+"\"DATE\")" +
+"VALUES (?, ?, CURRENT_DATE())");
 stmt.setString(1, "ab");
 stmt.setString(2, "a");
 stmt.execute();
@@ -84,44 +81,70 @@ public class VariableLengthPKIT extends 
BaseClientManagedTimeIT {
 conn.close();
 }
 
-protected static void initTableValues(byte[][] splits, long ts) throws 
Exception {
-ensureTableCreated(getUrl(),PTSDB_NAME, PTSDB_NAME, splits, ts-2, 
null);
+private static void initVarcharKeyTableValues(byte[][] splits, String 
varcharKeyTestTableName) throws Exception {
+String url = getUrl();
+Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+Connection conn = DriverManager.getConnection(url, props);
+
+String ddl = "create table " +varcharKeyTestTableName+
+"   (pk varchar not null primary key)";
+createTestTable(getUrl(), ddl, splits, null);
+PreparedStatement stmt = conn.prepareStatement(
+"upsert into " + varcharKeyTestTableName+
+"(pk) " +
+"VALUES (?)");
+stmt.setString(1, "   def");
+stmt.execute();
+stmt.setString(1, "jkl   ");
+stmt.execute();
+stmt.setString(1, "   ghi   ");
+stmt.execute();
+
+conn.commit();
+conn.close();
+}
 
+private static void initPTSDBTableValues(byte[][] splits, String 
pTSDBtableName) throws Exception {
+ensureTableCreated(getUrl(),pTSDBtableName, PTSDB_NAME, splits, null, 
null);
 // Insert all rows at ts
-String url = getUrl() + ";" + PhoenixRuntime.CURRENT_SCN_ATTRIB + "=" 
+ ts;
+String url = getUrl();
 Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
 Connection conn = DriverManager.getConnection(url, props);
 conn.setAutoCommit(true);
 PreparedStatement stmt = conn.prepareStatement(
-"upsert into " +
-"PTSDB(" +
-"INST, " +
-"HOST," +
-"\"DATE\"," +
-"VAL)" +
-"VALUES (?, ?, ?, ?)");
+"upsert into " + pTSDBtableName +
+" (" +
+"INST, " +
+"HOST," +
+"\"DATE\"," +
+"VAL)" +
+"VALUES (?, ?, ?, ?)");
 stmt.setString(1, "abc");
 stmt.setString(2, "abc-def-ghi");
 

[2/5] phoenix git commit: PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - VariableLengthPKIT (Ethan Wang)

[jamestaylor]

PHOENIX-4221 Disallow DML operations on connections with CURRENT_SCN set - 
VariableLengthPKIT (Ethan Wang)


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/bcdf292d
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/bcdf292d
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/bcdf292d

Branch: refs/heads/master
Commit: bcdf292dab26b12f210f8758b414b36bc7f4a7c4
Parents: 3ed6023
Author: James Taylor 
Authored: Thu Sep 21 10:44:02 2017 -0700
Committer: James Taylor 
Committed: Thu Sep 21 10:51:19 2017 -0700

--
 .../phoenix/end2end/VariableLengthPKIT.java | 1171 +-
 1 file changed, 570 insertions(+), 601 deletions(-)
--

Build failed in Jenkins: Phoenix Compile Compatibility with HBase #411

[Apache Jenkins Server]

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H25 (couchdbtest ubuntu xenial) in workspace 

[Phoenix_Compile_Compat_wHBase] $ /bin/bash /tmp/jenkins5866101251885790080.sh
core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
scheduling priority (-e) 0
file size   (blocks, -f) unlimited
pending signals (-i) 386417
max locked memory   (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files  (-n) 6
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 8192
cpu time   (seconds, -t) unlimited
max user processes  (-u) 10240
virtual memory  (kbytes, -v) unlimited
file locks  (-x) unlimited
core id : 0
core id : 1
core id : 2
core id : 3
core id : 4
core id : 5
physical id : 0
physical id : 1
MemTotal:   98958120 kB
MemFree:29499480 kB
Filesystem  Size  Used Avail Use% Mounted on
udev 48G 0   48G   0% /dev
tmpfs   9.5G  970M  8.5G  11% /run
/dev/sda1   364G  155G  191G  45% /
tmpfs48G  516K   48G   1% /dev/shm
tmpfs   5.0M 0  5.0M   0% /run/lock
tmpfs48G 0   48G   0% /sys/fs/cgroup
tmpfs   9.5G 0  9.5G   0% /run/user/10025
apache-maven-2.2.1
apache-maven-3.0.4
apache-maven-3.0.5
apache-maven-3.2.1
apache-maven-3.2.5
apache-maven-3.3.3
apache-maven-3.3.9
apache-maven-3.5.0
latest
latest2
latest3


===
Verifying compile level compatibility with HBase 0.98 with Phoenix 
4.x-HBase-0.98
===

Cloning into 'hbase'...
Switched to a new branch '0.98'
Branch 0.98 set up to track remote branch 0.98 from origin.

main:
 [exec] 
~/jenkins-slave/workspace/Phoenix_Compile_Compat_wHBase/hbase/hbase-common 
~/jenkins-slave/workspace/Phoenix_Compile_Compat_wHBase/hbase/hbase-common
 [exec] 
~/jenkins-slave/workspace/Phoenix_Compile_Compat_wHBase/hbase/hbase-common

main:
[mkdir] Created dir: 

 [exec] tar: hadoop-snappy-nativelibs.tar: Cannot open: No such file or 
directory
 [exec] tar: Error is not recoverable: exiting now
 [exec] Result: 2

main:
[mkdir] Created dir: 

 [copy] Copying 20 files to 

[mkdir] Created dir: 

[mkdir] Created dir: 


main:
[mkdir] Created dir: 

 [copy] Copying 17 files to 

[mkdir] Created dir: 


main:
[mkdir] Created dir: 

 [copy] Copying 1 file to 

[mkdir] Created dir: 


HBase pom.xml:

Got HBase version as 0.98.25-SNAPSHOT
Cloning into 'phoenix'...
Switched to a new branch '4.x-HBase-0.98'
Branch 4.x-HBase-0.98 set up to track remote branch 4.x-HBase-0.98 from origin.
ANTLR Parser Generator  Version 3.5.2
Output file 

 does not exist: must build 

PhoenixSQL.g


===
Verifying compile level compatibility with HBase branch-1.3 with Phoenix master