[
https://issues.apache.org/jira/browse/PHOENIX-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224920#comment-14224920
]
Maryann Xue edited comment on PHOENIX-1474 at 11/25/14 6:01 PM:
----------------------------------------------------------------
Thank you, [~jchabot72]! I have pinpointed the problem and made a simpler test
case to reproduce this bug:
""CREATE TABLE test_table
(col0 INTEGER NOT NULL,
col1 INTEGER NOT NULL,
col2 INTEGER NOT NULL,
col3 INTEGER
CONSTRAINT pk PRIMARY KEY (col0, col1, col2))
SALT_BUCKETS=4
"SELECT * FROM test_table WHERE (col0, col1) BETWEEN (1,2) AND (3,4) AND col1 =
3"
[~jamestaylor] The NPE occurs in WhereOptimizer.andKeySlots() when visiting the
second and in the above query, which is the AND connecting the RVC conditions
and col1 equal condition. I looked at PHOENIX-1397 and made a similar fix by
adding a if-null test for the NPE point, like:
{code}
for (KeySlots childSlot : childSlots) {
if (childSlot == EMPTY_KEY_SLOTS) {
return EMPTY_KEY_SLOTS;
}
// FIXME: get rid of this min/max range BS now that a key range
can span multiple columns
if (childSlot.getMinMaxRange() != null) { // Only set if in
initial pk position
// TODO: potentially use KeySlot.intersect here. However,
we can't intersect the key ranges in the slot
// with our minMaxRange, since it spans columns and this
would mess up our skip scan.
minMaxRange =
minMaxRange.intersect(childSlot.getMinMaxRange());
for (KeySlot slot : childSlot) {
+ if (slot != null) {
minMaxExtractNodes.addAll(slot.getKeyPart().getExtractNodes());
+ }
}
} else {
for (KeySlot slot : childSlot) {
// We have a nested AND with nothing for this slot, so
continue
if (slot == null) {
continue;
}
if (!intersectSlots(keySlot, slot)) {
return EMPTY_KEY_SLOTS;
}
}
}
}
{code}
Could you please verify for me if this fix is correct?
was (Author: maryannxue):
Thank you, [~jchabot72]! I have pinpointed the problem and made a simpler test
case to replicate this:
""CREATE TABLE test_table
(col0 INTEGER NOT NULL,
col1 INTEGER NOT NULL,
col2 INTEGER NOT NULL,
col3 INTEGER
CONSTRAINT pk PRIMARY KEY (col0, col1, col2))
SALT_BUCKETS=4
"SELECT * FROM test_table WHERE (col0, col1) BETWEEN (1,2) AND (3,4) AND col1 =
3"
[~jamestaylor] The NPE occurs in WhereOptimizer.andKeySlots() when visiting the
second and in the above query, which is the AND connecting the RVC conditions
and col1 equal condition. I looked at PHOENIX-1397 and made a similar fix by
adding a if-null test for the NPE point, like:
{code}
for (KeySlots childSlot : childSlots) {
if (childSlot == EMPTY_KEY_SLOTS) {
return EMPTY_KEY_SLOTS;
}
// FIXME: get rid of this min/max range BS now that a key range
can span multiple columns
if (childSlot.getMinMaxRange() != null) { // Only set if in
initial pk position
// TODO: potentially use KeySlot.intersect here. However,
we can't intersect the key ranges in the slot
// with our minMaxRange, since it spans columns and this
would mess up our skip scan.
minMaxRange =
minMaxRange.intersect(childSlot.getMinMaxRange());
for (KeySlot slot : childSlot) {
+ if (slot != null) {
minMaxExtractNodes.addAll(slot.getKeyPart().getExtractNodes());
+ }
}
} else {
for (KeySlot slot : childSlot) {
// We have a nested AND with nothing for this slot, so
continue
if (slot == null) {
continue;
}
if (!intersectSlots(keySlot, slot)) {
return EMPTY_KEY_SLOTS;
}
}
}
}
{code}
Could you please verify for me if this fix is correct?
> NPE when querying with WHERE clause
> -----------------------------------
>
> Key: PHOENIX-1474
> URL: https://issues.apache.org/jira/browse/PHOENIX-1474
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.2
> Environment: CentOS6, HBase 0.98.5, Phoenix 4.2
> Reporter: Jerry Chabot
> Assignee: Maryann Xue
> Attachments: NPE.PNG
>
>
> My queries worked with 4.1. I assume no code changes are required upgrading
> to 4.2. I used to copy the phoenix-4.1.0-server-hadoop2.jar to the HBase lib
> directory. I did not find instructions on how to install Phoenix 4.2. I
> assumed I had to copy the phoenix-4.2.0-server.jar. But, my queries are
> failing with an NPE.
> Here is a simple example that fails using sqlline.py. It fails as soon as I
> add the WHERE clause.
> 0: jdbc:phoenix:localhost> select * from "asset_metadata" as T1 inner join
> "asset_latest_rev" as T2 on T1.L=T2.L and T1.C=T2.C and T1.A=T2.A AND T1.R =
> T2.R;
> +------------+------------+------------+--------------+------------+-----------+
> | T1.L | T1.C | T1.A | T1.REVERSETS | T1.R | T1.W
> |
> +------------+------------+------------+--------------+------------+-----------+
> | 139359021902594048 | 139359004886302720 | 139359021973897216 |
> 9223370620364 |
> | 139359030865825792 | 139359004886302720 | 139359030958096384 |
> 9223370620364 |
> | 139359035940933632 | 139359004886302720 | 139359035974483968 |
> 9223370620364 |
> | 139359035940933632 | 139359004886302720 | 139359065401720832 |
> 9223370620364 |
> | 139359039808081920 | 139359004886302720 | 139359039824855040 |
> 9223370620364 |
> | 139359039808081920 | 139359004886302720 | 139359066852950016 |
> 9223370620364 |
> | 139359055830323200 | 139359042962194432 | 139359055851290624 |
> 9223370620364 |
> | 139359562284142592 | 139359547713126400 | 139359562313498624 |
> 9223370620364 |
> | 139359562284142592 | 139359547713126400 | 139359570307842048 |
> 9223370620364 |
> | 139359562284142592 | 139359547713126400 | 139359574065938432 |
> 9223370620364 |
> | 139359565945769984 | 139359547713126400 | 139359565975126016 |
> 9223370620364 |
> +------------+------------+------------+--------------+------------+-----------+
> 11 rows selected (0.384 seconds)
> 0: jdbc:phoenix:localhost> select * from "asset_metadata" as T1 inner join
> "asset_latest_rev" as T2 on T1.L=T2.L and T1.C=T2.C and T1.A=T2.A AND T1.R =
> T2.R WHERE T1.C=139359004886302720;
> java.lang.NullPointerException
> at
> org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.andKeySlots(WhereOptimizer.java:643)
> at
> org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.visitLeave(WhereOptimizer.java:825)
> at
> org.apache.phoenix.compile.WhereOptimizer$KeyExpressionVisitor.visitLeave(WhereOptimizer.java:440)
> at
> org.apache.phoenix.expression.AndExpression.accept(AndExpression.java:100)
> at
> org.apache.phoenix.compile.WhereOptimizer.pushKeyExpressionsToScan(WhereOptimizer.java:122)
> at
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:142)
> at
> org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:183)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:220)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:211)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:210)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1040)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3673)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3584)
> at sqlline.SqlLine.dispatch(SqlLine.java:821)
> at sqlline.SqlLine.begin(SqlLine.java:699)
> at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441)
> at sqlline.SqlLine.main(SqlLine.java:424)
> It also works if I changed the WHERE to T2.C=139359004886302720. That is T1
> did not work. These also worked.
> Reverse join
> select * from "asset_latest_rev" T2 inner join "asset_metadata" as T1 on
> T1.L=T2.L and T1.C=T2.C and T1.A=T2.A AND T1.R = T2.R WHERE
> T1.C=139359004886302720;
> Move where clause into join clause
> select * from "asset_latest_rev" T2 inner join "asset_metadata" as T1 on
> T1.L=T2.L and T1.C=T2.C and T1.A=T2.A AND T1.R = T2.R AND
> T1.C=139359004886302720;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
