[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186690#comment-16186690 ] Hudson commented on PHOENIX-4238: - FAILURE: Integrated in Jenkins build Phoenix-master #1826 (See [https://builds.apache.org/job/Phoenix-master/1826/]) PHOENIX-4238 MR IndexScrutinyTool break with salted tables and indexes (jtaylor: rev 62defe43212f83c2f8152af162554ea49046bd62) * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/IndexScrutinyToolIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/mapreduce/util/IndexColumnNames.java > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > Fix For: 4.12.0 > > Attachments: PHOENIX-4238.patch, PHOENIX-4238.v1.patch, > PHOENIX-4238.v2.patch, PHOENIX-4238.v3.patch > > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[
https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186559#comment-16186559
]
Hadoop QA commented on PHOENIX-4238:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12889755/PHOENIX-4238.v3.patch
against master branch at commit c4b478dc11898483f11c5880d85b2d03cca427d3.
ATTACHMENT ID: 12889755
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+{ "CREATE TABLE %s (ID INTEGER NOT NULL PRIMARY KEY, NAME
VARCHAR, ZIP INTEGER, EMPLOY_DATE TIMESTAMP, EMPLOYER VARCHAR)", "CREATE LOCAL
INDEX %s ON %s (NAME, EMPLOY_DATE) INCLUDE (ZIP)" },
+{ "CREATE TABLE %s (ID INTEGER NOT NULL PRIMARY KEY, NAME VARCHAR,
ZIP INTEGER, EMPLOY_DATE TIMESTAMP, EMPLOYER VARCHAR) SALT_BUCKETS=2", "CREATE
INDEX %s ON %s (NAME, EMPLOY_DATE) INCLUDE (ZIP)" } });
+"[2, name-2, " + new Timestamp(testTime).toString() + ",
95123]\t[2, name-2, " + new Timestamp(testTime)
+assertEquals("[3, name-3, " + new Timestamp(testTime).toString() + ",
95123]\tTarget row not found",
+2L, 1L, 1L, "[\"ID\" INTEGER, \"NAME\" VARCHAR,
\"EMPLOY_DATE\" TIMESTAMP, \"ZIP\" INTEGER]",
+"[\":ID\" INTEGER, \"0:NAME\" VARCHAR, \"0:EMPLOY_DATE\"
DECIMAL, \"0:ZIP\" INTEGER]", invalidRowsQuery);
+2L, 1L, 2L, "[\"ID\" INTEGER, \"NAME\" VARCHAR,
\"EMPLOY_DATE\" TIMESTAMP, \"ZIP\" INTEGER]",
+"[\":ID\" INTEGER, \"0:NAME\" VARCHAR, \"0:EMPLOY_DATE\"
DECIMAL, \"0:ZIP\" INTEGER]", invalidRowsQuery);
+ResultSet count = conn.createStatement().executeQuery("select count(*)
from " + tableFullName);
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.TxWriteFailureIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.InQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UngroupedIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.FlappingLocalIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PointInTimeQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CaseStatementIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.LocalIndexIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1497//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1497//console
This message is automatically generated.
> MR IndexScrutinyTool break with salted tables and indexes on views
> --
>
> Key: PHOENIX-4238
> URL: https://issues.apache.org/jira/browse/PHOENIX-4238
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Vincent Poon
>Assignee: churro morales
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4238.patch, PHOENIX-4238.v1.patch,
> PHOENIX-4238.v2.patch, PHOENIX-4238.v3.patch
>
>
> The IndexScrutinyTool MR job doesn't work for salted and shared table. We
> should add support for this, similar to PHOENIX-4233
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186417#comment-16186417 ] James Taylor commented on PHOENIX-4238: --- +1. Thanks, [~churromorales]. > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > Attachments: PHOENIX-4238.patch, PHOENIX-4238.v1.patch, > PHOENIX-4238.v2.patch, PHOENIX-4238.v3.patch > > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186313#comment-16186313 ] churro morales commented on PHOENIX-4238: - [~jamestaylor] added local index parameterized test. > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > Attachments: PHOENIX-4238.patch, PHOENIX-4238.v1.patch, > PHOENIX-4238.v2.patch, PHOENIX-4238.v3.patch > > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186270#comment-16186270 ] James Taylor commented on PHOENIX-4238: --- Thanks, [~churromorales]. How about a test for skipping the view index ID? If you create a local index (i.e. just add LOCAL keyword like this: CREATE LOCAL INDEX ...), then you'll be testing that. > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > Attachments: PHOENIX-4238.patch, PHOENIX-4238.v1.patch, > PHOENIX-4238.v2.patch > > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[
https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186223#comment-16186223
]
James Taylor commented on PHOENIX-4238:
---
Thanks, [~churromorales]. There's one more index column you have to skip in
addition to the salt column (which is used for indexes on views and local
indexes). See my patch over on PHOENIX-4233 or this block of code:
{code}
PTable pindex = pconn.getTable(new PTableKey(pconn.getTenantId(),
fullIndexName));
+List indexColumns = pindex.getColumns();
+int indexColumnOffset = 0;
+if (pindex.getBucketNum() != null) {
+indexColumnOffset = 1;
+}
+if (pindex.getViewIndexId() != null) {
+indexColumnOffset++;
+}
+if (indexColumnOffset > 0) {
+indexColumns = indexColumns.subList(indexColumnOffset,
indexColumns.size());
+}
{code}
> MR IndexScrutinyTool break with salted tables and indexes on views
> --
>
> Key: PHOENIX-4238
> URL: https://issues.apache.org/jira/browse/PHOENIX-4238
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Vincent Poon
>Assignee: churro morales
> Attachments: PHOENIX-4238.patch, PHOENIX-4238.v1.patch
>
>
> The IndexScrutinyTool MR job doesn't work for salted and shared table. We
> should add support for this, similar to PHOENIX-4233
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186197#comment-16186197 ] churro morales commented on PHOENIX-4238: - Thank you for the review [~vincentpoon] > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > Attachments: PHOENIX-4238.patch, PHOENIX-4238.v1.patch > > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186189#comment-16186189 ] Vincent Poon commented on PHOENIX-4238: --- lgtm, thanks [~churromorales] > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > Attachments: PHOENIX-4238.patch, PHOENIX-4238.v1.patch > > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186132#comment-16186132 ] churro morales commented on PHOENIX-4238: - [~jamestaylor] Ask and you shall receive, hope this made it in time. > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > Attachments: PHOENIX-4238.patch > > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185308#comment-16185308 ] churro morales commented on PHOENIX-4238: - Yes I can get it for you tomorrow. The code works just have to rewrite some of the tests to support multiple input splits. > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4238) MR IndexScrutinyTool break with salted tables and indexes on views
[ https://issues.apache.org/jira/browse/PHOENIX-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185276#comment-16185276 ] James Taylor commented on PHOENIX-4238: --- [~churromorales] - would it be possible to have a fix for this tomorrow? I'd like to cut an RC and this would be good to complete the index scrutiny tool. > MR IndexScrutinyTool break with salted tables and indexes on views > -- > > Key: PHOENIX-4238 > URL: https://issues.apache.org/jira/browse/PHOENIX-4238 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Vincent Poon >Assignee: churro morales > > The IndexScrutinyTool MR job doesn't work for salted and shared table. We > should add support for this, similar to PHOENIX-4233 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
