Apache-Phoenix | 4.x-HBase-1.1 | Build Successful

2018-02-27 Thread Apache Jenkins Server 
4.x-HBase-1.1 branch build status Successful

Source repository https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=shortlog;h=refs/heads/4.x-HBase-1.1

Compiled Artifacts https://builds.apache.org/job/Phoenix-4.x-HBase-1.1/lastSuccessfulBuild/artifact/

Test Report https://builds.apache.org/job/Phoenix-4.x-HBase-1.1/lastCompletedBuild/testReport/

Changes
[jtaylor] PHOENIX-4333 Incorrect estimate when stats are updated on a tenant

[jtaylor] PHOENIX-4626 Increase time allowed for partial index rebuild to complete

[jtaylor] PHOENIX-4628 Allow min time between update stats to be configurable



Build times for last couple of runsLatest build time is the right most | Legend blue: normal, red: test failure, gray: timeout

Apache-Phoenix | 4.x-HBase-1.2 | Build Successful

2018-02-27 Thread Apache Jenkins Server 
4.x-HBase-1.2 branch build status Successful

Source repository https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=shortlog;h=refs/heads/4.x-HBase-1.2

Compiled Artifacts https://builds.apache.org/job/Phoenix-4.x-HBase-1.2/lastSuccessfulBuild/artifact/

Test Report https://builds.apache.org/job/Phoenix-4.x-HBase-1.2/lastCompletedBuild/testReport/

Changes
[jtaylor] PHOENIX-4333 Incorrect estimate when stats are updated on a tenant

[jtaylor] PHOENIX-4626 Increase time allowed for partial index rebuild to complete

[jtaylor] PHOENIX-4628 Allow min time between update stats to be configurable



Build times for last couple of runsLatest build time is the right most | Legend blue: normal, red: test failure, gray: timeout

Build failed in Jenkins: Phoenix | Master #1947

2018-02-27 Thread Apache Jenkins Server 
See 


Changes:

[jtaylor] PHOENIX-4333 Incorrect estimate when stats are updated on a tenant

[jtaylor] PHOENIX-4626 Increase time allowed for partial index rebuild to 
complete

[jtaylor] PHOENIX-4628 Allow min time between update stats to be configurable

--
[...truncated 110.49 KB...]
[INFO] Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 360.278 
s - in org.apache.phoenix.end2end.index.LocalMutableTxIndexIT
[INFO] Running org.apache.phoenix.end2end.join.SortMergeJoinNoIndexIT
[INFO] Running org.apache.phoenix.end2end.join.SubqueryIT
[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 187.45 
s - in org.apache.phoenix.end2end.join.HashJoinNoIndexIT
[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 325.283 
s - in org.apache.phoenix.end2end.join.HashJoinGlobalIndexIT
[INFO] Running org.apache.phoenix.end2end.join.SubqueryUsingSortMergeJoinIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableIT
[INFO] Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 197.001 
s - in org.apache.phoenix.end2end.join.SortMergeJoinNoIndexIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.722 s 
- in org.apache.phoenix.end2end.salted.SaltedTableIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.252 s 
- in org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
[INFO] Running org.apache.phoenix.iterate.PhoenixQueryTimeoutIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.719 s 
- in org.apache.phoenix.iterate.PhoenixQueryTimeoutIT
[INFO] Running org.apache.phoenix.iterate.RoundRobinResultIteratorIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT
[INFO] Tests run: 72, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 544.172 
s - in org.apache.phoenix.end2end.index.MutableIndexIT
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.383 s 
- in org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT
[INFO] Running org.apache.phoenix.rpc.UpdateCacheIT
[INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 228.543 
s - in org.apache.phoenix.end2end.join.SubqueryIT
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.089 s 
- in org.apache.phoenix.iterate.RoundRobinResultIteratorIT
[INFO] Running org.apache.phoenix.trace.PhoenixTracingEndToEndIT
[INFO] Running org.apache.phoenix.replication.SystemCatalogWALEntryFilterIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.053 s 
- in org.apache.phoenix.replication.SystemCatalogWALEntryFilterIT
[INFO] Running org.apache.phoenix.tx.FlappingTransactionIT
[INFO] Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 342.646 
s - in org.apache.phoenix.end2end.join.SortMergeJoinGlobalIndexIT
[INFO] Running org.apache.phoenix.trace.PhoenixTableMetricsWriterIT
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.553 s 
- in org.apache.phoenix.rpc.UpdateCacheIT
[INFO] Running org.apache.phoenix.tx.TransactionIT
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.379 s 
- in org.apache.phoenix.tx.FlappingTransactionIT
[INFO] Running org.apache.phoenix.tx.TxCheckpointIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.288 s 
- in org.apache.phoenix.trace.PhoenixTableMetricsWriterIT
[INFO] Running org.apache.phoenix.util.IndexScrutinyIT
[INFO] Running org.apache.phoenix.tx.ParameterizedTransactionIT
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.709 s 
- in org.apache.phoenix.util.IndexScrutinyIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 56.075 s 
- in org.apache.phoenix.tx.TransactionIT
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 165.724 
s - in org.apache.phoenix.end2end.join.SubqueryUsingSortMergeJoinIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 92.93 s 
- in org.apache.phoenix.trace.PhoenixTracingEndToEndIT
[INFO] Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 591.816 
s - in org.apache.phoenix.end2end.join.HashJoinLocalIndexIT
[WARNING] Tests run: 52, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 
241.233 s - in org.apache.phoenix.tx.ParameterizedTransactionIT
[INFO] Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 596.09 
s - in org.apache.phoenix.end2end.join.SortMergeJoinLocalIndexIT
[INFO] Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 294.683 
s - in org.apache.phoenix.tx.TxCheckpointIT
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   ConcurrentMutationsIT.testConcurrentDeletesAndUpsertValues:214 
Expected to find PK in data table: (0,0)
[ERROR]

Build failed in Jenkins: Phoenix-4.x-HBase-1.3 #48

2018-02-27 Thread Apache Jenkins Server 
See 


Changes:

[jtaylor] PHOENIX-4333 Incorrect estimate when stats are updated on a tenant

[jtaylor] PHOENIX-4626 Increase time allowed for partial index rebuild to 
complete

[jtaylor] PHOENIX-4628 Allow min time between update stats to be configurable

--
[...truncated 105.11 KB...]
[INFO] Running org.apache.phoenix.end2end.index.ViewIndexIT
[INFO] Tests run: 37, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 385.253 
s - in org.apache.phoenix.end2end.index.IndexUsageIT
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 113.727 
s - in org.apache.phoenix.end2end.index.txn.MutableRollbackIT
[INFO] Running org.apache.phoenix.end2end.join.HashJoinCacheIT
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.35 s 
- in org.apache.phoenix.end2end.join.HashJoinCacheIT
[INFO] Running org.apache.phoenix.end2end.join.HashJoinGlobalIndexIT
[WARNING] Tests run: 12, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 
116.05 s - in org.apache.phoenix.end2end.index.ViewIndexIT
[INFO] Running org.apache.phoenix.end2end.index.txn.RollbackIT
[INFO] Running org.apache.phoenix.end2end.join.HashJoinLocalIndexIT
[INFO] Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 483.402 
s - in org.apache.phoenix.end2end.index.LocalMutableNonTxIndexIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.142 s 
- in org.apache.phoenix.end2end.index.txn.RollbackIT
[INFO] Running org.apache.phoenix.end2end.join.HashJoinNoIndexIT
[INFO] Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 482.133 
s - in org.apache.phoenix.end2end.index.LocalImmutableNonTxIndexIT
[INFO] Running org.apache.phoenix.end2end.join.HashJoinMoreIT
[INFO] Running org.apache.phoenix.end2end.join.SortMergeJoinGlobalIndexIT
[INFO] Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 513.353 
s - in org.apache.phoenix.end2end.index.LocalImmutableTxIndexIT
[INFO] Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 505.037 
s - in org.apache.phoenix.end2end.index.LocalMutableTxIndexIT
[INFO] Running org.apache.phoenix.end2end.join.SortMergeJoinLocalIndexIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 104.308 
s - in org.apache.phoenix.end2end.join.HashJoinMoreIT
[INFO] Running org.apache.phoenix.end2end.join.SubqueryIT
[INFO] Running org.apache.phoenix.end2end.join.SortMergeJoinNoIndexIT
[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 368.892 
s - in org.apache.phoenix.end2end.join.HashJoinNoIndexIT
[INFO] Running org.apache.phoenix.end2end.join.SubqueryUsingSortMergeJoinIT
[INFO] Tests run: 72, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 849.777 
s - in org.apache.phoenix.end2end.index.MutableIndexIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableIT
[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 643.003 
s - in org.apache.phoenix.end2end.join.HashJoinGlobalIndexIT
[INFO] Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 392.099 
s - in org.apache.phoenix.end2end.join.SortMergeJoinNoIndexIT
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.648 s 
- in org.apache.phoenix.end2end.salted.SaltedTableIT
[INFO] Running org.apache.phoenix.iterate.PhoenixQueryTimeoutIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.571 s 
- in org.apache.phoenix.iterate.PhoenixQueryTimeoutIT
[INFO] Running org.apache.phoenix.iterate.RoundRobinResultIteratorIT
[INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 420.304 
s - in org.apache.phoenix.end2end.join.SubqueryIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT
[INFO] Running org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.359 s 
- in org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
[INFO] Running org.apache.phoenix.rpc.UpdateCacheIT
[INFO] Running org.apache.phoenix.replication.SystemCatalogWALEntryFilterIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.441 s 
- in org.apache.phoenix.replication.SystemCatalogWALEntryFilterIT
[INFO] Running org.apache.phoenix.trace.PhoenixTableMetricsWriterIT
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 56.45 s 
- in org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT
[INFO] Running org.apache.phoenix.trace.PhoenixTracingEndToEndIT
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 80.552 s 
- in org.apache.phoenix.iterate.RoundRobinResultIteratorIT
[INFO] Running org.apache.phoenix.tx.FlappingTransactionIT
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.52 s 
- in org.apache.phoenix.trace.PhoenixTableMetricsWriterIT
[INFO] Running

Jenkins build is back to normal : Phoenix | 4.x-HBase-0.98 #1822

2018-02-27 Thread Apache Jenkins Server 
See

[2/3] phoenix git commit: PHOENIX-4626 Increase time allowed for partial index rebuild to complete

2018-02-27 Thread jamestaylor 
PHOENIX-4626 Increase time allowed for partial index rebuild to complete


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/0e42ecf8
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/0e42ecf8
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/0e42ecf8

Branch: refs/heads/5.x-HBase-2.0
Commit: 0e42ecf8785a79523d15cd67740f33c55dde8cf3
Parents: 8eb31f8
Author: James Taylor 
Authored: Mon Feb 26 15:34:26 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 14:04:59 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/0e42ecf8/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index bbede36..f2cf92d 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -196,7 +196,7 @@ public class QueryServicesOptions {
 public static final long DEFAULT_INDEX_REBUILD_RPC_TIMEOUT = 3 * 60; 
// 30 mins
 public static final long DEFAULT_INDEX_REBUILD_CLIENT_SCANNER_TIMEOUT = 
3 * 60; // 30 mins
 public static final int DEFAULT_INDEX_REBUILD_RPC_RETRIES_COUNTER = 5; // 
5 total tries at rpc level
-public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 3 * 60; // 30 mins
+public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 6 * 60 * 24; // 24 hrs
 
 /**
  * HConstants#HIGH_QOS is the max we will see to a standard table. We go 
higher to differentiate

[1/3] phoenix git commit: PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view

2018-02-27 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/5.x-HBase-2.0 beb3a7676 -> 6dcc24fa4


PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/8eb31f82
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/8eb31f82
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/8eb31f82

Branch: refs/heads/5.x-HBase-2.0
Commit: 8eb31f821f88406b7182661b1e2dc999ea9c574f
Parents: beb3a76
Author: James Taylor 
Authored: Thu Feb 22 10:26:33 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:46:25 2018 -0800

--
 .../end2end/ExplainPlanWithStatsEnabledIT.java  | 302 +++
 .../phoenix/iterate/BaseResultIterators.java| 167 +++---
 2 files changed, 305 insertions(+), 164 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/8eb31f82/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
index fc544cb..201a958 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
@@ -21,6 +21,7 @@ import static 
org.apache.phoenix.query.QueryServicesOptions.DEFAULT_USE_STATS_FO
 import static org.apache.phoenix.util.PhoenixRuntime.TENANT_ID_ATTRIB;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNull;
 import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
@@ -37,10 +38,11 @@ import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixResultSet;
 import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
-import org.apache.phoenix.query.BaseTest;
 import org.apache.phoenix.schema.PTable;
 import org.apache.phoenix.schema.PTableKey;
 import org.apache.phoenix.schema.TableNotFoundException;
+import org.apache.phoenix.schema.types.PInteger;
+import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.EnvironmentEdge;
 import org.apache.phoenix.util.EnvironmentEdgeManager;
 import org.apache.phoenix.util.PhoenixRuntime;
@@ -61,14 +63,14 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 private static String tableB;
 private static String tableWithLargeGPWidth;
 private static String indexOnA;
-private static final long largeGpWidth = 2 * 1000 * 1000l;
+private static final long largeGpWidth = 2 * 1000 * 1000L;
 
 @BeforeClass
 public static void createTables() throws Exception {
 tableA = generateUniqueName();
-initDataAndStats(tableA, 20l);
+initDataAndStats(tableA, 20L);
 tableB = generateUniqueName();
-initDataAndStats(tableB, 20l);
+initDataAndStats(tableB, 20L);
 tableWithLargeGPWidth = generateUniqueName();
 initDataAndStats(tableWithLargeGPWidth, largeGpWidth);
 indexOnA = generateUniqueName();
@@ -117,8 +119,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(200);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 0l, info.estimatedBytes);
-assertEquals((Long) 0l, info.estimatedRows);
+assertEquals((Long) 0L, info.estimatedBytes);
+assertEquals((Long) 0L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -130,8 +132,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(99);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 634l, info.estimatedBytes);
-assertEquals((Long) 10l, info.estimatedRows);
+assertEquals((Long) 634L, info.estimatedBytes);
+assertEquals((Long) 10L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -143,8 +145,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(0);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 691l,

[3/3] phoenix git commit: PHOENIX-4628 Allow min time between update stats to be configurable separately from stats cache TTL

2018-02-27 Thread jamestaylor 
PHOENIX-4628 Allow min time between update stats to be configurable separately 
from stats cache TTL


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/6dcc24fa
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/6dcc24fa
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/6dcc24fa

Branch: refs/heads/5.x-HBase-2.0
Commit: 6dcc24fa4b5d8970c283da8e2ff5f7d1c932c4ce
Parents: 0e42ecf
Author: James Taylor 
Authored: Tue Feb 27 12:24:06 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 14:05:03 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java | 3 +++
 .../src/main/java/org/apache/phoenix/schema/MetaDataClient.java  | 4 +---
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/6dcc24fa/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index f2cf92d..638c36e 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -233,6 +233,8 @@ public class QueryServicesOptions {
 public static final int DEFAULT_STATS_POOL_SIZE = 4;
 // Maximum size (in bytes) that cached table stats should take upm
 public static final long DEFAULT_STATS_MAX_CACHE_SIZE = 256 * 1024 * 1024;
+// Allow stats collection to be initiated by client multiple times 
immediately
+public static final int DEFAULT_MIN_STATS_UPDATE_FREQ_MS = 0;
 
 public static final boolean DEFAULT_USE_REVERSE_SCAN = true;
 
@@ -376,6 +378,7 @@ public class QueryServicesOptions {
 .setIfUnset(DATE_FORMAT_ATTRIB, DEFAULT_DATE_FORMAT)
 .setIfUnset(DATE_FORMAT_TIMEZONE_ATTRIB, 
DEFAULT_DATE_FORMAT_TIMEZONE)
 .setIfUnset(STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_STATS_UPDATE_FREQ_MS)
+.setIfUnset(MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_MIN_STATS_UPDATE_FREQ_MS)
 .setIfUnset(CALL_QUEUE_ROUND_ROBIN_ATTRIB, 
DEFAULT_CALL_QUEUE_ROUND_ROBIN)
 .setIfUnset(MAX_MUTATION_SIZE_ATTRIB, DEFAULT_MAX_MUTATION_SIZE)
 .setIfUnset(ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, 
DEFAULT_FORCE_ROW_KEY_ORDER)

http://git-wip-us.apache.org/repos/asf/phoenix/blob/6dcc24fa/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java 
b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
index bd52070..7dfc2b5 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
@@ -1163,9 +1163,7 @@ public class MetaDataClient {
 private long updateStatisticsInternal(PName physicalName, PTable 
logicalTable, Map statsProps, List cfs, boolean 
checkLastStatsUpdateTime) throws SQLException {
 ReadOnlyProps props = connection.getQueryServices().getProps();
 final long msMinBetweenUpdates = props
-.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB,
-
props.getLong(QueryServices.STATS_UPDATE_FREQ_MS_ATTRIB,
-
QueryServicesOptions.DEFAULT_STATS_UPDATE_FREQ_MS) / 2);
+.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
QueryServicesOptions.DEFAULT_MIN_STATS_UPDATE_FREQ_MS);
 Long scn = connection.getSCN();
 // Always invalidate the cache
 long clientTimeStamp = connection.getSCN() == null ? 
HConstants.LATEST_TIMESTAMP : scn;

[1/3] phoenix git commit: PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view

2018-02-27 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/master 0fca6ca21 -> 1744cd711


PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/eb893fb4
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/eb893fb4
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/eb893fb4

Branch: refs/heads/master
Commit: eb893fb47a71e640f01425f6eecf22f8c1bb95af
Parents: 0fca6ca
Author: James Taylor 
Authored: Thu Feb 22 10:26:33 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:41:28 2018 -0800

--
 .../end2end/ExplainPlanWithStatsEnabledIT.java  | 302 +++
 .../phoenix/iterate/BaseResultIterators.java| 167 +++---
 2 files changed, 305 insertions(+), 164 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/eb893fb4/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
index f369be9..2099f4c 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
@@ -21,6 +21,7 @@ import static 
org.apache.phoenix.query.QueryServicesOptions.DEFAULT_USE_STATS_FO
 import static org.apache.phoenix.util.PhoenixRuntime.TENANT_ID_ATTRIB;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNull;
 import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
@@ -36,10 +37,11 @@ import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixResultSet;
 import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
-import org.apache.phoenix.query.BaseTest;
 import org.apache.phoenix.schema.PTable;
 import org.apache.phoenix.schema.PTableKey;
 import org.apache.phoenix.schema.TableNotFoundException;
+import org.apache.phoenix.schema.types.PInteger;
+import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.EnvironmentEdge;
 import org.apache.phoenix.util.EnvironmentEdgeManager;
 import org.apache.phoenix.util.PhoenixRuntime;
@@ -58,14 +60,14 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 private static String tableB;
 private static String tableWithLargeGPWidth;
 private static String indexOnA;
-private static final long largeGpWidth = 2 * 1000 * 1000l;
+private static final long largeGpWidth = 2 * 1000 * 1000L;
 
 @BeforeClass
 public static void createTables() throws Exception {
 tableA = generateUniqueName();
-initDataAndStats(tableA, 20l);
+initDataAndStats(tableA, 20L);
 tableB = generateUniqueName();
-initDataAndStats(tableB, 20l);
+initDataAndStats(tableB, 20L);
 tableWithLargeGPWidth = generateUniqueName();
 initDataAndStats(tableWithLargeGPWidth, largeGpWidth);
 indexOnA = generateUniqueName();
@@ -114,8 +116,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(200);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 0l, info.estimatedBytes);
-assertEquals((Long) 0l, info.estimatedRows);
+assertEquals((Long) 0L, info.estimatedBytes);
+assertEquals((Long) 0L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -127,8 +129,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(99);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 634l, info.estimatedBytes);
-assertEquals((Long) 10l, info.estimatedRows);
+assertEquals((Long) 634L, info.estimatedBytes);
+assertEquals((Long) 10L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -140,8 +142,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(0);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 691l,

[3/3] phoenix git commit: PHOENIX-4628 Allow min time between update stats to be configurable separately from stats cache TTL

2018-02-27 Thread jamestaylor 
PHOENIX-4628 Allow min time between update stats to be configurable separately 
from stats cache TTL


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/1744cd71
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/1744cd71
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/1744cd71

Branch: refs/heads/master
Commit: 1744cd711c31f36339146e31c0748e4b7bf4da21
Parents: ac8dd33
Author: James Taylor 
Authored: Tue Feb 27 12:24:06 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:41:43 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java | 3 +++
 .../src/main/java/org/apache/phoenix/schema/MetaDataClient.java  | 4 +---
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/1744cd71/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index 6905fc3..8dadbb0 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -235,6 +235,8 @@ public class QueryServicesOptions {
 public static final int DEFAULT_STATS_POOL_SIZE = 4;
 // Maximum size (in bytes) that cached table stats should take upm
 public static final long DEFAULT_STATS_MAX_CACHE_SIZE = 256 * 1024 * 1024;
+// Allow stats collection to be initiated by client multiple times 
immediately
+public static final int DEFAULT_MIN_STATS_UPDATE_FREQ_MS = 0;
 
 public static final boolean DEFAULT_USE_REVERSE_SCAN = true;
 
@@ -381,6 +383,7 @@ public class QueryServicesOptions {
 .setIfUnset(DATE_FORMAT_ATTRIB, DEFAULT_DATE_FORMAT)
 .setIfUnset(DATE_FORMAT_TIMEZONE_ATTRIB, 
DEFAULT_DATE_FORMAT_TIMEZONE)
 .setIfUnset(STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_STATS_UPDATE_FREQ_MS)
+.setIfUnset(MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_MIN_STATS_UPDATE_FREQ_MS)
 .setIfUnset(CALL_QUEUE_ROUND_ROBIN_ATTRIB, 
DEFAULT_CALL_QUEUE_ROUND_ROBIN)
 .setIfUnset(MAX_MUTATION_SIZE_ATTRIB, DEFAULT_MAX_MUTATION_SIZE)
 .setIfUnset(ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, 
DEFAULT_FORCE_ROW_KEY_ORDER)

http://git-wip-us.apache.org/repos/asf/phoenix/blob/1744cd71/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java 
b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
index 8be3dcf..e7f7795 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
@@ -1165,9 +1165,7 @@ public class MetaDataClient {
 private long updateStatisticsInternal(PName physicalName, PTable 
logicalTable, Map statsProps, List cfs, boolean 
checkLastStatsUpdateTime) throws SQLException {
 ReadOnlyProps props = connection.getQueryServices().getProps();
 final long msMinBetweenUpdates = props
-.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB,
-
props.getLong(QueryServices.STATS_UPDATE_FREQ_MS_ATTRIB,
-
QueryServicesOptions.DEFAULT_STATS_UPDATE_FREQ_MS) / 2);
+.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
QueryServicesOptions.DEFAULT_MIN_STATS_UPDATE_FREQ_MS);
 Long scn = connection.getSCN();
 // Always invalidate the cache
 long clientTimeStamp = connection.getSCN() == null ? 
HConstants.LATEST_TIMESTAMP : scn;

[2/3] phoenix git commit: PHOENIX-4626 Increase time allowed for partial index rebuild to complete

2018-02-27 Thread jamestaylor 
PHOENIX-4626 Increase time allowed for partial index rebuild to complete


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/ac8dd331
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/ac8dd331
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/ac8dd331

Branch: refs/heads/master
Commit: ac8dd331a0afa37a15fbab208d8d37ce6a50ebdb
Parents: eb893fb
Author: James Taylor 
Authored: Mon Feb 26 15:34:26 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:41:37 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/ac8dd331/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index d749433..6905fc3 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -197,7 +197,7 @@ public class QueryServicesOptions {
 public static final long DEFAULT_INDEX_REBUILD_RPC_TIMEOUT = 3 * 60; 
// 30 mins
 public static final long DEFAULT_INDEX_REBUILD_CLIENT_SCANNER_TIMEOUT = 
3 * 60; // 30 mins
 public static final int DEFAULT_INDEX_REBUILD_RPC_RETRIES_COUNTER = 5; // 
5 total tries at rpc level
-public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 3 * 60; // 30 mins
+public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 6 * 60 * 24; // 24 hrs
 public static final long DEFAULT_INDEX_PENDING_DISABLE_THRESHOLD = 3; 
// 30 secs
 
 /**

[2/3] phoenix git commit: PHOENIX-4626 Increase time allowed for partial index rebuild to complete

2018-02-27 Thread jamestaylor 
PHOENIX-4626 Increase time allowed for partial index rebuild to complete


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/c3b53d46
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/c3b53d46
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/c3b53d46

Branch: refs/heads/4.x-HBase-1.2
Commit: c3b53d4646bc93b987a85f46448a5813cbbbe648
Parents: 185d74e
Author: James Taylor 
Authored: Mon Feb 26 15:34:26 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:34:37 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/c3b53d46/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index d749433..6905fc3 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -197,7 +197,7 @@ public class QueryServicesOptions {
 public static final long DEFAULT_INDEX_REBUILD_RPC_TIMEOUT = 3 * 60; 
// 30 mins
 public static final long DEFAULT_INDEX_REBUILD_CLIENT_SCANNER_TIMEOUT = 
3 * 60; // 30 mins
 public static final int DEFAULT_INDEX_REBUILD_RPC_RETRIES_COUNTER = 5; // 
5 total tries at rpc level
-public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 3 * 60; // 30 mins
+public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 6 * 60 * 24; // 24 hrs
 public static final long DEFAULT_INDEX_PENDING_DISABLE_THRESHOLD = 3; 
// 30 secs
 
 /**

[1/3] phoenix git commit: PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view

2018-02-27 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.2 9c8bb117f -> 09f78884b


PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/185d74ec
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/185d74ec
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/185d74ec

Branch: refs/heads/4.x-HBase-1.2
Commit: 185d74ec1d0d046270739dadc037a0d6d3d48eb9
Parents: 9c8bb11
Author: James Taylor 
Authored: Thu Feb 22 10:26:33 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:34:21 2018 -0800

--
 .../end2end/ExplainPlanWithStatsEnabledIT.java  | 302 +++
 .../phoenix/iterate/BaseResultIterators.java| 167 +++---
 2 files changed, 305 insertions(+), 164 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/185d74ec/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
index f369be9..2099f4c 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
@@ -21,6 +21,7 @@ import static 
org.apache.phoenix.query.QueryServicesOptions.DEFAULT_USE_STATS_FO
 import static org.apache.phoenix.util.PhoenixRuntime.TENANT_ID_ATTRIB;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNull;
 import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
@@ -36,10 +37,11 @@ import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixResultSet;
 import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
-import org.apache.phoenix.query.BaseTest;
 import org.apache.phoenix.schema.PTable;
 import org.apache.phoenix.schema.PTableKey;
 import org.apache.phoenix.schema.TableNotFoundException;
+import org.apache.phoenix.schema.types.PInteger;
+import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.EnvironmentEdge;
 import org.apache.phoenix.util.EnvironmentEdgeManager;
 import org.apache.phoenix.util.PhoenixRuntime;
@@ -58,14 +60,14 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 private static String tableB;
 private static String tableWithLargeGPWidth;
 private static String indexOnA;
-private static final long largeGpWidth = 2 * 1000 * 1000l;
+private static final long largeGpWidth = 2 * 1000 * 1000L;
 
 @BeforeClass
 public static void createTables() throws Exception {
 tableA = generateUniqueName();
-initDataAndStats(tableA, 20l);
+initDataAndStats(tableA, 20L);
 tableB = generateUniqueName();
-initDataAndStats(tableB, 20l);
+initDataAndStats(tableB, 20L);
 tableWithLargeGPWidth = generateUniqueName();
 initDataAndStats(tableWithLargeGPWidth, largeGpWidth);
 indexOnA = generateUniqueName();
@@ -114,8 +116,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(200);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 0l, info.estimatedBytes);
-assertEquals((Long) 0l, info.estimatedRows);
+assertEquals((Long) 0L, info.estimatedBytes);
+assertEquals((Long) 0L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -127,8 +129,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(99);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 634l, info.estimatedBytes);
-assertEquals((Long) 10l, info.estimatedRows);
+assertEquals((Long) 634L, info.estimatedBytes);
+assertEquals((Long) 10L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -140,8 +142,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(0);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 691l,

[3/3] phoenix git commit: PHOENIX-4628 Allow min time between update stats to be configurable separately from stats cache TTL

2018-02-27 Thread jamestaylor 
PHOENIX-4628 Allow min time between update stats to be configurable separately 
from stats cache TTL


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/09f78884
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/09f78884
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/09f78884

Branch: refs/heads/4.x-HBase-1.2
Commit: 09f78884b971585ef83381c75582ee22899e73da
Parents: c3b53d4
Author: James Taylor 
Authored: Tue Feb 27 12:24:06 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:34:51 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java | 3 +++
 .../src/main/java/org/apache/phoenix/schema/MetaDataClient.java  | 4 +---
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/09f78884/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index 6905fc3..8dadbb0 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -235,6 +235,8 @@ public class QueryServicesOptions {
 public static final int DEFAULT_STATS_POOL_SIZE = 4;
 // Maximum size (in bytes) that cached table stats should take upm
 public static final long DEFAULT_STATS_MAX_CACHE_SIZE = 256 * 1024 * 1024;
+// Allow stats collection to be initiated by client multiple times 
immediately
+public static final int DEFAULT_MIN_STATS_UPDATE_FREQ_MS = 0;
 
 public static final boolean DEFAULT_USE_REVERSE_SCAN = true;
 
@@ -381,6 +383,7 @@ public class QueryServicesOptions {
 .setIfUnset(DATE_FORMAT_ATTRIB, DEFAULT_DATE_FORMAT)
 .setIfUnset(DATE_FORMAT_TIMEZONE_ATTRIB, 
DEFAULT_DATE_FORMAT_TIMEZONE)
 .setIfUnset(STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_STATS_UPDATE_FREQ_MS)
+.setIfUnset(MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_MIN_STATS_UPDATE_FREQ_MS)
 .setIfUnset(CALL_QUEUE_ROUND_ROBIN_ATTRIB, 
DEFAULT_CALL_QUEUE_ROUND_ROBIN)
 .setIfUnset(MAX_MUTATION_SIZE_ATTRIB, DEFAULT_MAX_MUTATION_SIZE)
 .setIfUnset(ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, 
DEFAULT_FORCE_ROW_KEY_ORDER)

http://git-wip-us.apache.org/repos/asf/phoenix/blob/09f78884/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java 
b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
index 8be3dcf..e7f7795 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
@@ -1165,9 +1165,7 @@ public class MetaDataClient {
 private long updateStatisticsInternal(PName physicalName, PTable 
logicalTable, Map statsProps, List cfs, boolean 
checkLastStatsUpdateTime) throws SQLException {
 ReadOnlyProps props = connection.getQueryServices().getProps();
 final long msMinBetweenUpdates = props
-.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB,
-
props.getLong(QueryServices.STATS_UPDATE_FREQ_MS_ATTRIB,
-
QueryServicesOptions.DEFAULT_STATS_UPDATE_FREQ_MS) / 2);
+.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
QueryServicesOptions.DEFAULT_MIN_STATS_UPDATE_FREQ_MS);
 Long scn = connection.getSCN();
 // Always invalidate the cache
 long clientTimeStamp = connection.getSCN() == null ? 
HConstants.LATEST_TIMESTAMP : scn;

[1/3] phoenix git commit: PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view

2018-02-27 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.1 d78023017 -> a891d9fe4


PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/376caa3a
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/376caa3a
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/376caa3a

Branch: refs/heads/4.x-HBase-1.1
Commit: 376caa3a32a7adf6f9c97fa12929d4084cbf0996
Parents: d780230
Author: James Taylor 
Authored: Thu Feb 22 10:26:33 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:24:16 2018 -0800

--
 .../end2end/ExplainPlanWithStatsEnabledIT.java  | 302 +++
 .../phoenix/iterate/BaseResultIterators.java| 167 +++---
 2 files changed, 305 insertions(+), 164 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/376caa3a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
index a342206..969bbfb 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
@@ -21,6 +21,7 @@ import static 
org.apache.phoenix.query.QueryServicesOptions.DEFAULT_USE_STATS_FO
 import static org.apache.phoenix.util.PhoenixRuntime.TENANT_ID_ATTRIB;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNull;
 import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
@@ -36,10 +37,11 @@ import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixResultSet;
 import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
-import org.apache.phoenix.query.BaseTest;
 import org.apache.phoenix.schema.PTable;
 import org.apache.phoenix.schema.PTableKey;
 import org.apache.phoenix.schema.TableNotFoundException;
+import org.apache.phoenix.schema.types.PInteger;
+import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.EnvironmentEdge;
 import org.apache.phoenix.util.EnvironmentEdgeManager;
 import org.apache.phoenix.util.PhoenixRuntime;
@@ -58,14 +60,14 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 private static String tableB;
 private static String tableWithLargeGPWidth;
 private static String indexOnA;
-private static final long largeGpWidth = 2 * 1000 * 1000l;
+private static final long largeGpWidth = 2 * 1000 * 1000L;
 
 @BeforeClass
 public static void createTables() throws Exception {
 tableA = generateUniqueName();
-initDataAndStats(tableA, 20l);
+initDataAndStats(tableA, 20L);
 tableB = generateUniqueName();
-initDataAndStats(tableB, 20l);
+initDataAndStats(tableB, 20L);
 tableWithLargeGPWidth = generateUniqueName();
 initDataAndStats(tableWithLargeGPWidth, largeGpWidth);
 indexOnA = generateUniqueName();
@@ -114,8 +116,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(200);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 0l, info.estimatedBytes);
-assertEquals((Long) 0l, info.estimatedRows);
+assertEquals((Long) 0L, info.estimatedBytes);
+assertEquals((Long) 0L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -127,8 +129,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(99);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 634l, info.estimatedBytes);
-assertEquals((Long) 10l, info.estimatedRows);
+assertEquals((Long) 634L, info.estimatedBytes);
+assertEquals((Long) 10L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -140,8 +142,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(0);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 691l,

[3/3] phoenix git commit: PHOENIX-4628 Allow min time between update stats to be configurable separately from stats cache TTL

2018-02-27 Thread jamestaylor 
PHOENIX-4628 Allow min time between update stats to be configurable separately 
from stats cache TTL


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/a891d9fe
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/a891d9fe
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/a891d9fe

Branch: refs/heads/4.x-HBase-1.1
Commit: a891d9fe43a4fa11a10abbc98ebaac2140226590
Parents: ab7f6c3
Author: James Taylor 
Authored: Tue Feb 27 12:24:06 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:24:54 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java | 3 +++
 .../src/main/java/org/apache/phoenix/schema/MetaDataClient.java  | 4 +---
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/a891d9fe/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index fff05e6..ea81534 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -233,6 +233,8 @@ public class QueryServicesOptions {
 public static final int DEFAULT_STATS_POOL_SIZE = 4;
 // Maximum size (in bytes) that cached table stats should take upm
 public static final long DEFAULT_STATS_MAX_CACHE_SIZE = 256 * 1024 * 1024;
+// Allow stats collection to be initiated by client multiple times 
immediately
+public static final int DEFAULT_MIN_STATS_UPDATE_FREQ_MS = 0;
 
 public static final boolean DEFAULT_USE_REVERSE_SCAN = true;
 
@@ -374,6 +376,7 @@ public class QueryServicesOptions {
 .setIfUnset(DATE_FORMAT_ATTRIB, DEFAULT_DATE_FORMAT)
 .setIfUnset(DATE_FORMAT_TIMEZONE_ATTRIB, 
DEFAULT_DATE_FORMAT_TIMEZONE)
 .setIfUnset(STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_STATS_UPDATE_FREQ_MS)
+.setIfUnset(MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_MIN_STATS_UPDATE_FREQ_MS)
 .setIfUnset(CALL_QUEUE_ROUND_ROBIN_ATTRIB, 
DEFAULT_CALL_QUEUE_ROUND_ROBIN)
 .setIfUnset(MAX_MUTATION_SIZE_ATTRIB, DEFAULT_MAX_MUTATION_SIZE)
 .setIfUnset(ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, 
DEFAULT_FORCE_ROW_KEY_ORDER)

http://git-wip-us.apache.org/repos/asf/phoenix/blob/a891d9fe/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java 
b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
index b3d455e..24eaef8 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
@@ -1159,9 +1159,7 @@ public class MetaDataClient {
 private long updateStatisticsInternal(PName physicalName, PTable 
logicalTable, Map statsProps, List cfs, boolean 
checkLastStatsUpdateTime) throws SQLException {
 ReadOnlyProps props = connection.getQueryServices().getProps();
 final long msMinBetweenUpdates = props
-.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB,
-
props.getLong(QueryServices.STATS_UPDATE_FREQ_MS_ATTRIB,
-
QueryServicesOptions.DEFAULT_STATS_UPDATE_FREQ_MS) / 2);
+.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
QueryServicesOptions.DEFAULT_MIN_STATS_UPDATE_FREQ_MS);
 Long scn = connection.getSCN();
 // Always invalidate the cache
 long clientTimeStamp = connection.getSCN() == null ? 
HConstants.LATEST_TIMESTAMP : scn;

[2/3] phoenix git commit: PHOENIX-4626 Increase time allowed for partial index rebuild to complete

2018-02-27 Thread jamestaylor 
PHOENIX-4626 Increase time allowed for partial index rebuild to complete


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/ab7f6c3d
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/ab7f6c3d
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/ab7f6c3d

Branch: refs/heads/4.x-HBase-1.1
Commit: ab7f6c3d186e3a710b158609f9c92a42a947da5b
Parents: 376caa3
Author: James Taylor 
Authored: Mon Feb 26 15:34:26 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 13:24:39 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/ab7f6c3d/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index dbd4835..fff05e6 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -195,7 +195,7 @@ public class QueryServicesOptions {
 public static final long DEFAULT_INDEX_REBUILD_RPC_TIMEOUT = 3 * 60; 
// 30 mins
 public static final long DEFAULT_INDEX_REBUILD_CLIENT_SCANNER_TIMEOUT = 
3 * 60; // 30 mins
 public static final int DEFAULT_INDEX_REBUILD_RPC_RETRIES_COUNTER = 5; // 
5 total tries at rpc level
-public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 3 * 60; // 30 mins
+public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 6 * 60 * 24; // 24 hrs
 public static final long DEFAULT_INDEX_PENDING_DISABLE_THRESHOLD = 3; 
// 30 secs
 
 /**

[3/3] phoenix git commit: PHOENIX-4628 Allow min time between update stats to be configurable separately from stats cache TTL

2018-02-27 Thread jamestaylor 
PHOENIX-4628 Allow min time between update stats to be configurable separately 
from stats cache TTL


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/e56d92fe
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/e56d92fe
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/e56d92fe

Branch: refs/heads/4.x-HBase-0.98
Commit: e56d92fe95ba4a4dec024b08578c42c45201541f
Parents: 86985b6
Author: James Taylor 
Authored: Tue Feb 27 12:24:06 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 12:58:40 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java | 3 +++
 .../src/main/java/org/apache/phoenix/schema/MetaDataClient.java  | 4 +---
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/e56d92fe/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index 5d39688..b30413c 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -223,6 +223,8 @@ public class QueryServicesOptions {
 public static final int DEFAULT_STATS_POOL_SIZE = 4;
 // Maximum size (in bytes) that cached table stats should take upm
 public static final long DEFAULT_STATS_MAX_CACHE_SIZE = 256 * 1024 * 1024;
+// Allow stats collection to be initiated by client multiple times 
immediately
+public static final int DEFAULT_MIN_STATS_UPDATE_FREQ_MS = 0;
 
 public static final boolean DEFAULT_USE_REVERSE_SCAN = true;
 
@@ -362,6 +364,7 @@ public class QueryServicesOptions {
 .setIfUnset(DATE_FORMAT_ATTRIB, DEFAULT_DATE_FORMAT)
 .setIfUnset(DATE_FORMAT_TIMEZONE_ATTRIB, 
DEFAULT_DATE_FORMAT_TIMEZONE)
 .setIfUnset(STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_STATS_UPDATE_FREQ_MS)
+.setIfUnset(MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_MIN_STATS_UPDATE_FREQ_MS)
 .setIfUnset(CALL_QUEUE_ROUND_ROBIN_ATTRIB, 
DEFAULT_CALL_QUEUE_ROUND_ROBIN)
 .setIfUnset(MAX_MUTATION_SIZE_ATTRIB, DEFAULT_MAX_MUTATION_SIZE)
 .setIfUnset(ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, 
DEFAULT_FORCE_ROW_KEY_ORDER)

http://git-wip-us.apache.org/repos/asf/phoenix/blob/e56d92fe/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java 
b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
index 75b8eff..3b38ede 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
@@ -1159,9 +1159,7 @@ public class MetaDataClient {
 private long updateStatisticsInternal(PName physicalName, PTable 
logicalTable, Map statsProps, List cfs, boolean 
checkLastStatsUpdateTime) throws SQLException {
 ReadOnlyProps props = connection.getQueryServices().getProps();
 final long msMinBetweenUpdates = props
-.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB,
-
props.getLong(QueryServices.STATS_UPDATE_FREQ_MS_ATTRIB,
-
QueryServicesOptions.DEFAULT_STATS_UPDATE_FREQ_MS) / 2);
+.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
QueryServicesOptions.DEFAULT_MIN_STATS_UPDATE_FREQ_MS);
 Long scn = connection.getSCN();
 // Always invalidate the cache
 long clientTimeStamp = connection.getSCN() == null ? 
HConstants.LATEST_TIMESTAMP : scn;

[2/3] phoenix git commit: PHOENIX-4626 Increase time allowed for partial index rebuild to complete

2018-02-27 Thread jamestaylor 
PHOENIX-4626 Increase time allowed for partial index rebuild to complete


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/86985b63
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/86985b63
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/86985b63

Branch: refs/heads/4.x-HBase-0.98
Commit: 86985b638a9de4840ebfc67089ad3d53bb6a1e6b
Parents: 48e4980
Author: James Taylor 
Authored: Mon Feb 26 15:34:26 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 12:58:25 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/86985b63/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index bcc8c01..5d39688 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -185,7 +185,7 @@ public class QueryServicesOptions {
 public static final long DEFAULT_INDEX_REBUILD_RPC_TIMEOUT = 3 * 60; 
// 30 mins
 public static final long DEFAULT_INDEX_REBUILD_CLIENT_SCANNER_TIMEOUT = 
3 * 60; // 30 mins
 public static final int DEFAULT_INDEX_REBUILD_RPC_RETRIES_COUNTER = 5; // 
5 total tries at rpc level
-public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 3 * 60; // 30 mins
+public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 6 * 60 * 24; // 24 hrs
 public static final long DEFAULT_INDEX_PENDING_DISABLE_THRESHOLD = 3; 
// 30 secs
 
 /**

[1/3] phoenix git commit: PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view

2018-02-27 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-0.98 178405d70 -> e56d92fe9


PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/48e4980b
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/48e4980b
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/48e4980b

Branch: refs/heads/4.x-HBase-0.98
Commit: 48e4980b7efd8202fa36270eb7d9827f9cde828e
Parents: 178405d
Author: James Taylor 
Authored: Thu Feb 22 10:26:33 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 12:58:02 2018 -0800

--
 .../end2end/ExplainPlanWithStatsEnabledIT.java  | 302 +++
 .../phoenix/iterate/BaseResultIterators.java| 167 +++---
 2 files changed, 305 insertions(+), 164 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/48e4980b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
index 03cb484..76ed7ba 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
@@ -21,6 +21,7 @@ import static 
org.apache.phoenix.query.QueryServicesOptions.DEFAULT_USE_STATS_FO
 import static org.apache.phoenix.util.PhoenixRuntime.TENANT_ID_ATTRIB;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNull;
 import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
@@ -36,10 +37,11 @@ import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixResultSet;
 import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
-import org.apache.phoenix.query.BaseTest;
 import org.apache.phoenix.schema.PTable;
 import org.apache.phoenix.schema.PTableKey;
 import org.apache.phoenix.schema.TableNotFoundException;
+import org.apache.phoenix.schema.types.PInteger;
+import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.EnvironmentEdge;
 import org.apache.phoenix.util.EnvironmentEdgeManager;
 import org.apache.phoenix.util.PhoenixRuntime;
@@ -58,14 +60,14 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 private static String tableB;
 private static String tableWithLargeGPWidth;
 private static String indexOnA;
-private static final long largeGpWidth = 2 * 1000 * 1000l;
+private static final long largeGpWidth = 2 * 1000 * 1000L;
 
 @BeforeClass
 public static void createTables() throws Exception {
 tableA = generateUniqueName();
-initDataAndStats(tableA, 20l);
+initDataAndStats(tableA, 20L);
 tableB = generateUniqueName();
-initDataAndStats(tableB, 20l);
+initDataAndStats(tableB, 20L);
 tableWithLargeGPWidth = generateUniqueName();
 initDataAndStats(tableWithLargeGPWidth, largeGpWidth);
 indexOnA = generateUniqueName();
@@ -114,8 +116,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(200);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 0l, info.estimatedBytes);
-assertEquals((Long) 0l, info.estimatedRows);
+assertEquals((Long) 0L, info.estimatedBytes);
+assertEquals((Long) 0L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -127,8 +129,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(99);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 634l, info.estimatedBytes);
-assertEquals((Long) 10l, info.estimatedRows);
+assertEquals((Long) 634L, info.estimatedBytes);
+assertEquals((Long) 10L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -140,8 +142,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(0);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 691l,

[3/3] phoenix git commit: PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view

2018-02-27 Thread jamestaylor 
PHOENIX-4333 Incorrect estimate when stats are updated on a tenant specific view


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/db656fba
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/db656fba
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/db656fba

Branch: refs/heads/4.x-HBase-1.3
Commit: db656fbaf6e130fde942f5edd121040e0a5f70f9
Parents: e8d3ed0
Author: James Taylor 
Authored: Thu Feb 22 10:26:33 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 12:46:37 2018 -0800

--
 .../end2end/ExplainPlanWithStatsEnabledIT.java  | 302 +++
 .../phoenix/iterate/BaseResultIterators.java| 167 +++---
 2 files changed, 305 insertions(+), 164 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/db656fba/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
--
diff --git 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
index f369be9..2099f4c 100644
--- 
a/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
+++ 
b/phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
@@ -21,6 +21,7 @@ import static 
org.apache.phoenix.query.QueryServicesOptions.DEFAULT_USE_STATS_FO
 import static org.apache.phoenix.util.PhoenixRuntime.TENANT_ID_ATTRIB;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNull;
 import static org.junit.Assert.assertTrue;
 
 import java.sql.Connection;
@@ -36,10 +37,11 @@ import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.phoenix.jdbc.PhoenixConnection;
 import org.apache.phoenix.jdbc.PhoenixResultSet;
 import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil;
-import org.apache.phoenix.query.BaseTest;
 import org.apache.phoenix.schema.PTable;
 import org.apache.phoenix.schema.PTableKey;
 import org.apache.phoenix.schema.TableNotFoundException;
+import org.apache.phoenix.schema.types.PInteger;
+import org.apache.phoenix.util.ByteUtil;
 import org.apache.phoenix.util.EnvironmentEdge;
 import org.apache.phoenix.util.EnvironmentEdgeManager;
 import org.apache.phoenix.util.PhoenixRuntime;
@@ -58,14 +60,14 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 private static String tableB;
 private static String tableWithLargeGPWidth;
 private static String indexOnA;
-private static final long largeGpWidth = 2 * 1000 * 1000l;
+private static final long largeGpWidth = 2 * 1000 * 1000L;
 
 @BeforeClass
 public static void createTables() throws Exception {
 tableA = generateUniqueName();
-initDataAndStats(tableA, 20l);
+initDataAndStats(tableA, 20L);
 tableB = generateUniqueName();
-initDataAndStats(tableB, 20l);
+initDataAndStats(tableB, 20L);
 tableWithLargeGPWidth = generateUniqueName();
 initDataAndStats(tableWithLargeGPWidth, largeGpWidth);
 indexOnA = generateUniqueName();
@@ -114,8 +116,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(200);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 0l, info.estimatedBytes);
-assertEquals((Long) 0l, info.estimatedRows);
+assertEquals((Long) 0L, info.estimatedBytes);
+assertEquals((Long) 0L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -127,8 +129,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(99);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 634l, info.estimatedBytes);
-assertEquals((Long) 10l, info.estimatedRows);
+assertEquals((Long) 634L, info.estimatedBytes);
+assertEquals((Long) 10L, info.estimatedRows);
 assertTrue(info.estimateInfoTs > 0);
 }
 }
@@ -140,8 +142,8 @@ public class ExplainPlanWithStatsEnabledIT extends 
ParallelStatsEnabledIT {
 binds.add(0);
 try (Connection conn = DriverManager.getConnection(getUrl())) {
 Estimate info = getByteRowEstimates(conn, sql, binds);
-assertEquals((Long) 691l, info.estimatedBytes);
-assertEquals((Long) 10l, info.estimatedRows);
+

[1/3] phoenix git commit: PHOENIX-4628 Allow min time between update stats to be configurable separately from stats cache TTL

2018-02-27 Thread jamestaylor 
Repository: phoenix
Updated Branches:
  refs/heads/4.x-HBase-1.3 e8d3ed00c -> a22c8de6a


PHOENIX-4628 Allow min time between update stats to be configurable separately 
from stats cache TTL


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/a22c8de6
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/a22c8de6
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/a22c8de6

Branch: refs/heads/4.x-HBase-1.3
Commit: a22c8de6a0479745a2c861f1f5c553f219e9466c
Parents: 4110f08
Author: James Taylor 
Authored: Tue Feb 27 12:24:06 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 12:46:37 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java | 3 +++
 .../src/main/java/org/apache/phoenix/schema/MetaDataClient.java  | 4 +---
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/a22c8de6/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index 6905fc3..8dadbb0 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -235,6 +235,8 @@ public class QueryServicesOptions {
 public static final int DEFAULT_STATS_POOL_SIZE = 4;
 // Maximum size (in bytes) that cached table stats should take upm
 public static final long DEFAULT_STATS_MAX_CACHE_SIZE = 256 * 1024 * 1024;
+// Allow stats collection to be initiated by client multiple times 
immediately
+public static final int DEFAULT_MIN_STATS_UPDATE_FREQ_MS = 0;
 
 public static final boolean DEFAULT_USE_REVERSE_SCAN = true;
 
@@ -381,6 +383,7 @@ public class QueryServicesOptions {
 .setIfUnset(DATE_FORMAT_ATTRIB, DEFAULT_DATE_FORMAT)
 .setIfUnset(DATE_FORMAT_TIMEZONE_ATTRIB, 
DEFAULT_DATE_FORMAT_TIMEZONE)
 .setIfUnset(STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_STATS_UPDATE_FREQ_MS)
+.setIfUnset(MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
DEFAULT_MIN_STATS_UPDATE_FREQ_MS)
 .setIfUnset(CALL_QUEUE_ROUND_ROBIN_ATTRIB, 
DEFAULT_CALL_QUEUE_ROUND_ROBIN)
 .setIfUnset(MAX_MUTATION_SIZE_ATTRIB, DEFAULT_MAX_MUTATION_SIZE)
 .setIfUnset(ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, 
DEFAULT_FORCE_ROW_KEY_ORDER)

http://git-wip-us.apache.org/repos/asf/phoenix/blob/a22c8de6/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java 
b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
index 8be3dcf..e7f7795 100644
--- a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
+++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
@@ -1165,9 +1165,7 @@ public class MetaDataClient {
 private long updateStatisticsInternal(PName physicalName, PTable 
logicalTable, Map statsProps, List cfs, boolean 
checkLastStatsUpdateTime) throws SQLException {
 ReadOnlyProps props = connection.getQueryServices().getProps();
 final long msMinBetweenUpdates = props
-.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB,
-
props.getLong(QueryServices.STATS_UPDATE_FREQ_MS_ATTRIB,
-
QueryServicesOptions.DEFAULT_STATS_UPDATE_FREQ_MS) / 2);
+.getLong(QueryServices.MIN_STATS_UPDATE_FREQ_MS_ATTRIB, 
QueryServicesOptions.DEFAULT_MIN_STATS_UPDATE_FREQ_MS);
 Long scn = connection.getSCN();
 // Always invalidate the cache
 long clientTimeStamp = connection.getSCN() == null ? 
HConstants.LATEST_TIMESTAMP : scn;

[2/3] phoenix git commit: PHOENIX-4626 Increase time allowed for partial index rebuild to complete

2018-02-27 Thread jamestaylor 
PHOENIX-4626 Increase time allowed for partial index rebuild to complete


Project: http://git-wip-us.apache.org/repos/asf/phoenix/repo
Commit: http://git-wip-us.apache.org/repos/asf/phoenix/commit/4110f083
Tree: http://git-wip-us.apache.org/repos/asf/phoenix/tree/4110f083
Diff: http://git-wip-us.apache.org/repos/asf/phoenix/diff/4110f083

Branch: refs/heads/4.x-HBase-1.3
Commit: 4110f0830fec85ee9d6337a2cb5603a32f81cce2
Parents: db656fb
Author: James Taylor 
Authored: Mon Feb 26 15:34:26 2018 -0800
Committer: James Taylor 
Committed: Tue Feb 27 12:46:37 2018 -0800

--
 .../main/java/org/apache/phoenix/query/QueryServicesOptions.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/phoenix/blob/4110f083/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
--
diff --git 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
index d749433..6905fc3 100644
--- 
a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
+++ 
b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
@@ -197,7 +197,7 @@ public class QueryServicesOptions {
 public static final long DEFAULT_INDEX_REBUILD_RPC_TIMEOUT = 3 * 60; 
// 30 mins
 public static final long DEFAULT_INDEX_REBUILD_CLIENT_SCANNER_TIMEOUT = 
3 * 60; // 30 mins
 public static final int DEFAULT_INDEX_REBUILD_RPC_RETRIES_COUNTER = 5; // 
5 total tries at rpc level
-public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 3 * 60; // 30 mins
+public static final int DEFAULT_INDEX_REBUILD_DISABLE_TIMESTAMP_THRESHOLD 
= 6 * 60 * 24; // 24 hrs
 public static final long DEFAULT_INDEX_PENDING_DISABLE_THRESHOLD = 3; 
// 30 secs
 
 /**

Apache-Phoenix | origin/4.11-HBase-1.3 | Build Failure

2018-02-27 Thread Apache Jenkins Server 
<<< text/html; charset=UTF-8: Unrecognized >>>