[jira] [Commented] (PHOENIX-3280) Automatic attempt to rebuild all disabled index

2016-09-15 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15492979#comment-15492979
 ] 

Hudson commented on PHOENIX-3280:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1403 (See 
[https://builds.apache.org/job/Phoenix-master/1403/])
PHOENIX-3280 Automatic attempt to rebuild all disabled index (jamestaylor: rev 
2a223adfbeb32f598308da1dc6d3251ee0980d79)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataRegionObserver.java


> Automatic attempt to rebuild all disabled index
> ---
>
> Key: PHOENIX-3280
> URL: https://issues.apache.org/jira/browse/PHOENIX-3280
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3280.patch, PHOENIX-3280_v2.patch
>
>
> Instead of only attempting to rebuild the first disabled index, we should 
> attempt to build each of them. Otherwise, a failure of the first one would 
> block the building of the others. This is follow up work for PHOENIX-3237 and 
> further improvements are identified in PHOENIX-3277.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3280) Automatic attempt to rebuild all disabled index

2016-09-15 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15492968#comment-15492968
 ] 

Hudson commented on PHOENIX-3280:
-

SUCCESS: Integrated in Jenkins build Phoenix-4.8-HBase-1.2 #24 (See 
[https://builds.apache.org/job/Phoenix-4.8-HBase-1.2/24/])
PHOENIX-3280 Automatic attempt to rebuild all disabled index (jamestaylor: rev 
496872c6e85e8e2cfe792035f9635b0817b78135)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataRegionObserver.java


> Automatic attempt to rebuild all disabled index
> ---
>
> Key: PHOENIX-3280
> URL: https://issues.apache.org/jira/browse/PHOENIX-3280
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3280.patch, PHOENIX-3280_v2.patch
>
>
> Instead of only attempting to rebuild the first disabled index, we should 
> attempt to build each of them. Otherwise, a failure of the first one would 
> block the building of the others. This is follow up work for PHOENIX-3237 and 
> further improvements are identified in PHOENIX-3277.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3280) Automatic attempt to rebuild all disabled index

2016-09-15 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15492825#comment-15492825
 ] 

Hadoop QA commented on PHOENIX-3280:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828598/PHOENIX-3280_v2.patch
  against master branch at commit 27697b364511f8557a87d4007bff1a560e4b8490.
  ATTACHMENT ID: 12828598

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
34 warning messages.

{color:red}-1 release audit{color}.  The applied patch generated 1 release 
audit warnings (more than the master's current 0 warnings).

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+if ((dataTable == null || dataTable.length == 0) || 
(indexState == null || indexState.length == 0)) {
+if 
(Bytes.compareTo(PIndexState.DISABLE.getSerializedBytes(), indexState) == 0) {
+indexesToPartiallyRebuild = 
Lists.newArrayListWithExpectedSize(dataPTable.getIndexes().size());
+
QueryServicesOptions.DEFAULT_INDEX_FAILURE_HANDLING_REBUILD_OVERLAP_TIME);
+LOG.info("Starting to build " + dataPTable + " 
indexes " + indexesToPartiallyRebuild
+try (HTableInterface dataHTable = 
conn.getQueryServices().getTable(physicalTableName)) {
+try (ResultScanner dataTableScanner = 
dataHTable.getScanner(dataTableScan)) {
+List mutations = 
Lists.newArrayListWithExpectedSize(batchSize);
+ImmutableBytesWritable indexMetaDataPtr = 
new ImmutableBytesWritable(
+
IndexMaintainer.serializeAdditional(dataPTable, indexMetaDataPtr,

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/577//testReport/
Release audit warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/577//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/577//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/577//console

This message is automatically generated.

> Automatic attempt to rebuild all disabled index
> ---
>
> Key: PHOENIX-3280
> URL: https://issues.apache.org/jira/browse/PHOENIX-3280
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3280.patch, PHOENIX-3280_v2.patch
>
>
> Instead of only attempting to rebuild the first disabled index, we should 
> attempt to build each of them. Otherwise, a failure of the first one would 
> block the building of the others. This is follow up work for PHOENIX-3237 and 
> further improvements are identified in PHOENIX-3277.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3280) Automatic attempt to rebuild all disabled index

2016-09-15 Thread Ankit Singhal (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15492710#comment-15492710
 ] 

Ankit Singhal commented on PHOENIX-3280:


+1 , Thanks [~jamestaylor]

> Automatic attempt to rebuild all disabled index
> ---
>
> Key: PHOENIX-3280
> URL: https://issues.apache.org/jira/browse/PHOENIX-3280
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3280.patch, PHOENIX-3280_v2.patch
>
>
> Instead of only attempting to rebuild the first disabled index, we should 
> attempt to build each of them. Otherwise, a failure of the first one would 
> block the building of the others. This is follow up work for PHOENIX-3237 and 
> further improvements are identified in PHOENIX-3277.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3280) Automatic attempt to rebuild all disabled index

2016-09-15 Thread Ankit Singhal (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15492569#comment-15492569
 ] 

Ankit Singhal commented on PHOENIX-3280:


Thanks [~giacomotaylor] , it looks good but have some minor comments.

Instead, Existing PhoenixRuntime#getTableNoCache() can be used.
{code}
+
+/**
+ * Get the latest table from the server, bypassing the client cache
+ * 
+ * @param conn
+ * @param name
+ * @return
+ * @throws SQLException
+ */
+public static PTable forceGetTable(Connection conn, String name) throws 
SQLException {
{code}

Could be a existing bug, Shouldn't we clear mutations after executing them.
{code}
+if (mutations.size() == batchSize) {
+dataHTable.batch(mutations);
+uuidValue = 
ServerCacheClient.generateId();
 }
{code}

Is the second version more optimized and can be used?
{code}
+List indexesToPartiallyRebuild = 
Lists.newArrayListWithExpectedSize(dataPTable.getIndexes()
+.size());
+List oldIndexesToPartiallyRebuild = 
dataTableToIndexesMap.put(dataPTable,
+indexesToPartiallyRebuild);
+// If there already were indexes, then combine them with 
the new ones
+if (oldIndexesToPartiallyRebuild != null) {
+
indexesToPartiallyRebuild.addAll(oldIndexesToPartiallyRebuild);
+}
 indexesToPartiallyRebuild.add(indexPTable);
{code}
{code}
+
+List oldIndexesToPartiallyRebuild= 
dataTableToIndexesMap.get(dataPTable);
+if (oldIndexesToPartiallyRebuild == null) {
+   oldIndexesToPartiallyRebuild = 
Lists.newArrayListWithExpectedSize(dataPTable.getIndexes()
+.size());
+   
dataTableToIndexesMap.put(dataPTable,indexesToPartiallyRebuild);
+}
 oldIndexesToPartiallyRebuild.add(indexPTable);
 {code}

please remove the comment as it is not valid now.
{code}
 // we need to build indexes of same data table. so 
skip other indexes for this task.
 continue;
{code}

> Automatic attempt to rebuild all disabled index
> ---
>
> Key: PHOENIX-3280
> URL: https://issues.apache.org/jira/browse/PHOENIX-3280
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3280.patch
>
>
> Instead of only attempting to rebuild the first disabled index, we should 
> attempt to build each of them. Otherwise, a failure of the first one would 
> block the building of the others. This is follow up work for PHOENIX-3237 and 
> further improvements are identified in PHOENIX-3277.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3280) Automatic attempt to rebuild all disabled index

2016-09-14 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15492088#comment-15492088
 ] 

Hadoop QA commented on PHOENIX-3280:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12828565/PHOENIX-3280.patch
  against master branch at commit 2c2b552cd7c611c63677b0be075065504d579469.
  ATTACHMENT ID: 12828565

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
35 warning messages.

{color:red}-1 release audit{color}.  The applied patch generated 1 release 
audit warnings (more than the master's current 0 warnings).

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+if ((dataTable == null || dataTable.length == 0) || 
(indexState == null || indexState.length == 0)) {
+if 
(Bytes.compareTo(PIndexState.DISABLE.getSerializedBytes(), indexState) == 0) {
+List indexesToPartiallyRebuild = 
Lists.newArrayListWithExpectedSize(dataPTable.getIndexes()
+List oldIndexesToPartiallyRebuild = 
dataTableToIndexesMap.put(dataPTable,
+
QueryServicesOptions.DEFAULT_INDEX_FAILURE_HANDLING_REBUILD_OVERLAP_TIME);
+LOG.info("Starting to build " + dataPTable + " 
indexes " + indexesToPartiallyRebuild
+try (HTableInterface dataHTable = 
conn.getQueryServices().getTable(physicalTableName)) {
+try (ResultScanner dataTableScanner = 
dataHTable.getScanner(dataTableScan)) {
+List mutations = 
Lists.newArrayListWithExpectedSize(batchSize);
+ImmutableBytesWritable indexMetaDataPtr = 
new ImmutableBytesWritable(

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.rpc.UpdateCacheWithScnIT

Test results: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/576//testReport/
Release audit warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/576//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/576//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/576//console

This message is automatically generated.

> Automatic attempt to rebuild all disabled index
> ---
>
> Key: PHOENIX-3280
> URL: https://issues.apache.org/jira/browse/PHOENIX-3280
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3280.patch
>
>
> Instead of only attempting to rebuild the first disabled index, we should 
> attempt to build each of them. Otherwise, a failure of the first one would 
> block the building of the others. This is follow up work for PHOENIX-3237 and 
> further improvements are identified in PHOENIX-3277.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)