[jira] [Comment Edited] (PHOENIX-3396) Valid Multi-byte strings whose total byte size is greater than the max char limit cannot be inserted into VARCHAR fields in the PK
[
https://issues.apache.org/jira/browse/PHOENIX-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596812#comment-15596812
]
James Taylor edited comment on PHOENIX-3396 at 10/22/16 1:41 AM:
-
Phoenix interpretes the max as bytes instead of characters. That was done on
purpose, but using characters instead would make more sense. As a workaround,
just increase the max length by a factor of 3 - it won't have any impact on
performance. To do that, you can follow the directions outlined here:
http://search-hadoop.com/m/9UY0h21LtMS1KMm942=Re+Can+I+change+a+String+column+s+size+and+preserve+the+data+
was (Author: jamestaylor):
Phoenix interpretes the max as bytes instead of characters. That was done on
purpose, but using characters instead would make more sense.
> Valid Multi-byte strings whose total byte size is greater than the max char
> limit cannot be inserted into VARCHAR fields in the PK
> ---
>
> Key: PHOENIX-3396
> URL: https://issues.apache.org/jira/browse/PHOENIX-3396
> Project: Phoenix
> Issue Type: Bug
>Reporter: Jan Fernando
>
> We allow users to insert multi-byte characters into VARCHAR columns that are
> part of a table or view's PK. We noticed that Strings that had a valid number
> of characters (i.e. were less than than max char length) were causing upserts
> to fail with with the following exception:
> Caused by: java.sql.SQLException: ERROR 206 (22003): The data exceeds the max
> capacity for the data type. MYTABLE may not exceed 100 bytes
> ('緓嗝加슪䐤㵞靹疸芬꽣汚佃䘯茵䖻埾巆蕤ⱅ澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉ķ窯尬룗㚈Ꝝ퍛爃됰灁ᄠࢥ')
> There appears to be an issue in PTableImpl.newKey() where we check the
> maxLength in chars against the byte length in this check:
> maxLength != null && !type.isArrayType() && byteValue.length > maxLength
> To reproduce you can run the following:
> CREATE TABLE TEXT_FIELD_VALIDATION_PK (TEXT VARCHAR(20), TEXT1 VARCHAR(20)
> CONSTRAINT PK PRIMARY KEY (TEXT));
> UPSERT INTO TEXT_FIELD_VALIDATION_PK VALUES ('澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉碉', 'test');
> The string we insert into the column TEXT is 20 chars, but greater than 20
> bytes.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3392) SortMergeJoinIT#testSubJoin[0] is failing with encodecolumns2 branch
[ https://issues.apache.org/jira/browse/PHOENIX-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596815#comment-15596815 ] Samarth Jain commented on PHOENIX-3392: --- Sorry, didn't update the issue but I did talk to Maryann offline to let her know that I have figured out the issue. > SortMergeJoinIT#testSubJoin[0] is failing with encodecolumns2 branch > > > Key: PHOENIX-3392 > URL: https://issues.apache.org/jira/browse/PHOENIX-3392 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > > The SortMergeJoinIT#testSubJoin[0] is failing with encodecolumns2 branch. To > repro, checkout the encodecolumns2 branch and run the SortMergeJoinIT tests. > Here's the stack trace is here: > https://builds.apache.org/job/Phoenix-encode-columns/16/testReport/org.apache.phoenix.end2end/SortMergeJoinIT/testSubJoin_0_/ > The basic idea of column encoding over mutable tables is that we take control > of the column qualifier name, storing in PColumn the mapping of real column > name to column qualifier name. We use serialized integers as the column > qualifier names so that we can do positional lookups into the List we > get back from HBase APIs. There are a few "reserved" column qualifiers for > things like our empty key value, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3396) Valid Multi-byte strings whose total byte size is greater than the max char limit cannot be inserted into VARCHAR fields in the PK
[
https://issues.apache.org/jira/browse/PHOENIX-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596812#comment-15596812
]
James Taylor commented on PHOENIX-3396:
---
Phoenix interpretes the max as bytes instead of characters. That was done on
purpose, but using characters instead would make more sense.
> Valid Multi-byte strings whose total byte size is greater than the max char
> limit cannot be inserted into VARCHAR fields in the PK
> ---
>
> Key: PHOENIX-3396
> URL: https://issues.apache.org/jira/browse/PHOENIX-3396
> Project: Phoenix
> Issue Type: Bug
>Reporter: Jan Fernando
>
> We allow users to insert multi-byte characters into VARCHAR columns that are
> part of a table or view's PK. We noticed that Strings that had a valid number
> of characters (i.e. were less than than max char length) were causing upserts
> to fail with with the following exception:
> Caused by: java.sql.SQLException: ERROR 206 (22003): The data exceeds the max
> capacity for the data type. MYTABLE may not exceed 100 bytes
> ('緓嗝加슪䐤㵞靹疸芬꽣汚佃䘯茵䖻埾巆蕤ⱅ澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉ķ窯尬룗㚈Ꝝ퍛爃됰灁ᄠࢥ')
> There appears to be an issue in PTableImpl.newKey() where we check the
> maxLength in chars against the byte length in this check:
> maxLength != null && !type.isArrayType() && byteValue.length > maxLength
> To reproduce you can run the following:
> CREATE TABLE TEXT_FIELD_VALIDATION_PK (TEXT VARCHAR(20), TEXT1 VARCHAR(20)
> CONSTRAINT PK PRIMARY KEY (TEXT));
> UPSERT INTO TEXT_FIELD_VALIDATION_PK VALUES ('澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉碉', 'test');
> The string we insert into the column TEXT is 20 chars, but greater than 20
> bytes.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3392) SortMergeJoinIT#testSubJoin[0] is failing with encodecolumns2 branch
[ https://issues.apache.org/jira/browse/PHOENIX-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596804#comment-15596804 ] James Taylor commented on PHOENIX-3392: --- [~maryannxue] - looks like [~samarthjain] figured out the issue. Hopefully you haven't spent time on it already. > SortMergeJoinIT#testSubJoin[0] is failing with encodecolumns2 branch > > > Key: PHOENIX-3392 > URL: https://issues.apache.org/jira/browse/PHOENIX-3392 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > > The SortMergeJoinIT#testSubJoin[0] is failing with encodecolumns2 branch. To > repro, checkout the encodecolumns2 branch and run the SortMergeJoinIT tests. > Here's the stack trace is here: > https://builds.apache.org/job/Phoenix-encode-columns/16/testReport/org.apache.phoenix.end2end/SortMergeJoinIT/testSubJoin_0_/ > The basic idea of column encoding over mutable tables is that we take control > of the column qualifier name, storing in PColumn the mapping of real column > name to column qualifier name. We use serialized integers as the column > qualifier names so that we can do positional lookups into the List we > get back from HBase APIs. There are a few "reserved" column qualifiers for > things like our empty key value, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3396) Valid Multi-byte strings whose total byte size is greater than the max char limit cannot be inserted into VARCHAR fields in the PK
Jan Fernando created PHOENIX-3396:
-
Summary: Valid Multi-byte strings whose total byte size is greater
than the max char limit cannot be inserted into VARCHAR fields in the PK
Key: PHOENIX-3396
URL: https://issues.apache.org/jira/browse/PHOENIX-3396
Project: Phoenix
Issue Type: Bug
Reporter: Jan Fernando
We allow users to insert multi-byte characters into VARCHAR columns that are
part of a table or view's PK. We noticed that Strings that had a valid number
of characters (i.e. were less than than max char length) were causing upserts
to fail with with the following exception:
Caused by: java.sql.SQLException: ERROR 206 (22003): The data exceeds the max
capacity for the data type. MYTABLE may not exceed 100 bytes
('緓嗝加슪䐤㵞靹疸芬꽣汚佃䘯茵䖻埾巆蕤ⱅ澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉ķ窯尬룗㚈Ꝝ퍛爃됰灁ᄠࢥ')
There appears to be an issue in PTableImpl.newKey() where we check the
maxLength in chars against the byte length in this check:
maxLength != null && !type.isArrayType() && byteValue.length > maxLength
To reproduce you can run the following:
CREATE TABLE TEXT_FIELD_VALIDATION_PK (TEXT VARCHAR(20), TEXT1 VARCHAR(20)
CONSTRAINT PK PRIMARY KEY (TEXT));
UPSERT INTO TEXT_FIELD_VALIDATION_PK VALUES ('澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉碉', 'test');
The string we insert into the column TEXT is 20 chars, but greater than 20
bytes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Excessive ExecService RPCs with multi-threaded ingest
Hi folks, I was doing some testing earlier this week and Enis's keen eye caught something rather interesting. When using YCSB to ingest data into a table with a secondary index using 8 threads and batch size of 1000 rows, the number of ExecService coprocessor calls actually exceeded the number of Multi calls to write the data (something like 21k ExecService calls to 18k Multi calls). I dug into this some more and noticed that it's because each thread is creating its own ServerCache to store the serialized IndexMetadata before shipping the data table updates. So, when we have 8 threads all writing mutations for the same data and index table, we have ~8x the ServerCache entries being created than if we had just one thread. Looking at the code, I completely understand why they're local to the thread and not shared on the Connection (very tricky), but I'm curious if anyone had noticed this before or if there are reasons to not try to share these ServerCache(s) across threads. Looking at the data being put into the ServerCache, it appears to be exactly the same for each of the threads sending mutations. I'm thinking that we could do safely by tracking when we are loading (or have loaded) the data into the ServerCache and doing some reference counting to determine when its actually safe to delete the ServerCache. I hope to find/make some time to get a patch up, but thought I'd take a moment to write it up if anyone has opinions/feedback. Thanks! - Josh
[jira] [Commented] (PHOENIX-3342) ORDER BY and LIMIT+OFFSET doesnt work on second column from compound key
[
https://issues.apache.org/jira/browse/PHOENIX-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595924#comment-15595924
]
Hudson commented on PHOENIX-3342:
-
FAILURE: Integrated in Jenkins build Phoenix-master #1447 (See
[https://builds.apache.org/job/Phoenix-master/1447/])
PHOENIX-3342 ORDER BY and LIMIT+OFFSET doesnt work on second column from
(ankitsinghal59: rev 8f5c8cfbd1c1f7894e4cc88e2257d245d1d8c3bf)
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/UnionAllIT.java
* (edit)
phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryWithOffsetIT.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java
* (edit)
phoenix-core/src/test/java/org/apache/phoenix/compile/StatementHintsCompilationTest.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/execute/ScanPlan.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/execute/UnionPlan.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/query/QueryPlanTest.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortTopNResultIterator.java
> ORDER BY and LIMIT+OFFSET doesnt work on second column from compound key
>
>
> Key: PHOENIX-3342
> URL: https://issues.apache.org/jira/browse/PHOENIX-3342
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.1
>Reporter: Alex Batyrshin
>Assignee: Ankit Singhal
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3342-wip.patch, PHOENIX-3342.patch,
> PHOENIX-3342_v1.patch, PHOENIX-3342_v2.patch, PHOENIX-3342_v2_4.8_branch.patch
>
>
> Here is simple test case
> {code}
> CREATE TABLE "test" (
> col1 VARCHAR,
> col2 VARCHAR,
> "foo"."data" VARCHAR,
> CONSTRAINT PK PRIMARY KEY (col1, col2)
> );
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '1', 'd1');
> 1 row affected (0.044 seconds)
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '2', 'd2');
> 1 row affected (0.008 seconds)
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '3', 'd3');
> 1 row affected (0.007 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 1 | d1|
> | 1 | 2 | d2|
> | 1 | 3 | d3|
> +---+---+---+
> 3 rows selected (0.026 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 limit 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 1 | d1|
> +---+---+---+
> 1 row selected (0.026 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 offset 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 2 | d2|
> | 1 | 3 | d3|
> +---+---+---+
> 2 rows selected (0.02 seconds)
> {code}
> And this query doesn't work as expected:
> {code}
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 limit 1 offset
> 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> +---+---+---+
> No rows selected (0.024 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3267) Replace use of SELECT null with CAST(null AS )
[
https://issues.apache.org/jira/browse/PHOENIX-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595841#comment-15595841
]
Julian Hyde commented on PHOENIX-3267:
--
It would be possible to change Calcite to handle this, but it would take effort
(of course) and would be non-standard (so would be only enabled with certain
compliance settings). I suspect that several mainstream databases do allow
'select null' but I'd have to check. I suspect that they have a special type
'nulltype' (better than what Phoenix does, which is to use a null pointer
instead of a type).
> Replace use of SELECT null with CAST(null AS )
>
>
> Key: PHOENIX-3267
> URL: https://issues.apache.org/jira/browse/PHOENIX-3267
> Project: Phoenix
> Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Eric Lomore
>
> It appears that Calcite doesn't allow null as a SELECT expression. If this is
> difficult to change, we can change our tests to cast null to a given data
> type instead, like this: {{CAST(null AS VARCHAR)}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3355) Register Phoenix built-in functions as Calcite functions
[ https://issues.apache.org/jira/browse/PHOENIX-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595822#comment-15595822 ] Maryann Xue commented on PHOENIX-3355: -- Thank you for the patch, [~lomoree]! Yes, you are going in the right direction. A few things, though: 1. I see that you moved "registerBuiltins()" (make it "registerBuiltInFunctions()"?) from PhoenixSchema's constructor to "getFunctions()". Is there any reason why you did that? 2. There is a BUILT_IN_FUNCTION_MAP in ParseNodeFactory. Probably you should use that instead of going through the Expression sub-classes. And that way you don't need to create new instances of BuiltInFunctionInfo any more. And not sure if "createFromBuiltIn()" should live in "PFunction", maybe we can move it out to PhoenixSchema or somewhere else? 3. Once all built-in functions are registered in PhoenixSchema, we can remove those existing function translation in CalciteUtils since they will all go through the UDF translation. > Register Phoenix built-in functions as Calcite functions > > > Key: PHOENIX-3355 > URL: https://issues.apache.org/jira/browse/PHOENIX-3355 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Eric Lomore > Labels: calcite > Attachments: PHOENIX-3355.wip > > > We should register all Phoenix built-in functions that don't exist in Calcite. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3342) ORDER BY and LIMIT+OFFSET doesnt work on second column from compound key
[
https://issues.apache.org/jira/browse/PHOENIX-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595768#comment-15595768
]
Hudson commented on PHOENIX-3342:
-
SUCCESS: Integrated in Jenkins build Phoenix-4.8-HBase-1.2 #39 (See
[https://builds.apache.org/job/Phoenix-4.8-HBase-1.2/39/])
PHOENIX-3342 ORDER BY and LIMIT+OFFSET doesnt work on second column from
(ankitsinghal59: rev b2f0c48c2ff4c425b6b192d85f218e966284152e)
* (edit) phoenix-core/src/main/java/org/apache/phoenix/execute/UnionPlan.java
* (edit)
phoenix-core/src/test/java/org/apache/phoenix/compile/StatementHintsCompilationTest.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/execute/ScanPlan.java
* (edit)
phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryWithOffsetIT.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/query/QueryPlanTest.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/UnionAllIT.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortTopNResultIterator.java
> ORDER BY and LIMIT+OFFSET doesnt work on second column from compound key
>
>
> Key: PHOENIX-3342
> URL: https://issues.apache.org/jira/browse/PHOENIX-3342
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.1
>Reporter: Alex Batyrshin
>Assignee: Ankit Singhal
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3342-wip.patch, PHOENIX-3342.patch,
> PHOENIX-3342_v1.patch, PHOENIX-3342_v2.patch, PHOENIX-3342_v2_4.8_branch.patch
>
>
> Here is simple test case
> {code}
> CREATE TABLE "test" (
> col1 VARCHAR,
> col2 VARCHAR,
> "foo"."data" VARCHAR,
> CONSTRAINT PK PRIMARY KEY (col1, col2)
> );
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '1', 'd1');
> 1 row affected (0.044 seconds)
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '2', 'd2');
> 1 row affected (0.008 seconds)
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '3', 'd3');
> 1 row affected (0.007 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 1 | d1|
> | 1 | 2 | d2|
> | 1 | 3 | d3|
> +---+---+---+
> 3 rows selected (0.026 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 limit 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 1 | d1|
> +---+---+---+
> 1 row selected (0.026 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 offset 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 2 | d2|
> | 1 | 3 | d3|
> +---+---+---+
> 2 rows selected (0.02 seconds)
> {code}
> And this query doesn't work as expected:
> {code}
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 limit 1 offset
> 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> +---+---+---+
> No rows selected (0.024 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3355) Register Phoenix built-in functions as Calcite functions
[ https://issues.apache.org/jira/browse/PHOENIX-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595719#comment-15595719 ] Eric Lomore commented on PHOENIX-3355: -- [~maryannxue] mind taking a look at the patch? Would like to know if I have the right idea or I should be trying another approach. Thanks! > Register Phoenix built-in functions as Calcite functions > > > Key: PHOENIX-3355 > URL: https://issues.apache.org/jira/browse/PHOENIX-3355 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Eric Lomore > Labels: calcite > Attachments: PHOENIX-3355.wip > > > We should register all Phoenix built-in functions that don't exist in Calcite. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3161) Check possibility of moving rebuilding code to coprocessor of data table.
[ https://issues.apache.org/jira/browse/PHOENIX-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595307#comment-15595307 ] Hudson commented on PHOENIX-3161: - SUCCESS: Integrated in Jenkins build Phoenix-master #1446 (See [https://builds.apache.org/job/Phoenix-master/1446/]) PHOENIX-3161 Check possibility of moving rebuilding code to coprocessor (ankitsinghal59: rev a95e8ab1af2b8defef3d8c0ed5a060c9b9881dd9) * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexFailureIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/util/ScanUtil.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/PostDDLCompiler.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/hbase/index/util/IndexManagementUtil.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataRegionObserver.java > Check possibility of moving rebuilding code to coprocessor of data table. > - > > Key: PHOENIX-3161 > URL: https://issues.apache.org/jira/browse/PHOENIX-3161 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Fix For: 4.9.0 > > Attachments: PHOENIX-3161.patch, PHOENIX-3161_v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3342) ORDER BY and LIMIT+OFFSET doesnt work on second column from compound key
[
https://issues.apache.org/jira/browse/PHOENIX-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-3342:
---
Attachment: PHOENIX-3342_v2_4.8_branch.patch
PHOENIX-3342_v2.patch
bq. One note - you brought REBUILD_INDEXES constant in the patch. Looks like a
part of another fix.
Yes, it is a part of PHOENIX-3161 which is also now committed. uploaded v2 with
the same and fix for UnionAllIT test.
Thanks [~sergey.soldatov] for the review, committed v2 in 4.8, 4.x branches and
master.
> ORDER BY and LIMIT+OFFSET doesnt work on second column from compound key
>
>
> Key: PHOENIX-3342
> URL: https://issues.apache.org/jira/browse/PHOENIX-3342
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.1
>Reporter: Alex Batyrshin
> Fix For: 4.9.0, 4.8.2
>
> Attachments: PHOENIX-3342-wip.patch, PHOENIX-3342.patch,
> PHOENIX-3342_v1.patch, PHOENIX-3342_v2.patch, PHOENIX-3342_v2_4.8_branch.patch
>
>
> Here is simple test case
> {code}
> CREATE TABLE "test" (
> col1 VARCHAR,
> col2 VARCHAR,
> "foo"."data" VARCHAR,
> CONSTRAINT PK PRIMARY KEY (col1, col2)
> );
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '1', 'd1');
> 1 row affected (0.044 seconds)
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '2', 'd2');
> 1 row affected (0.008 seconds)
> 0: jdbc:phoenix:localhost> upsert into "test" (COL1, COL2, "data") values
> ('1', '3', 'd3');
> 1 row affected (0.007 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 1 | d1|
> | 1 | 2 | d2|
> | 1 | 3 | d3|
> +---+---+---+
> 3 rows selected (0.026 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 limit 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 1 | d1|
> +---+---+---+
> 1 row selected (0.026 seconds)
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 offset 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> | 1 | 2 | d2|
> | 1 | 3 | d3|
> +---+---+---+
> 2 rows selected (0.02 seconds)
> {code}
> And this query doesn't work as expected:
> {code}
> 0: jdbc:phoenix:localhost> select * from "test" order by col2 limit 1 offset
> 1;
> +---+---+---+
> | COL1 | COL2 | data |
> +---+---+---+
> +---+---+---+
> No rows selected (0.024 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3161) Check possibility of moving rebuilding code to coprocessor of data table.
[ https://issues.apache.org/jira/browse/PHOENIX-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595219#comment-15595219 ] Ankit Singhal commented on PHOENIX-3161: Thanks [~jamestaylor] for the review. committed to 4.x branch and master. > Check possibility of moving rebuilding code to coprocessor of data table. > - > > Key: PHOENIX-3161 > URL: https://issues.apache.org/jira/browse/PHOENIX-3161 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ankit Singhal >Assignee: Ankit Singhal > Fix For: 4.9.0 > > Attachments: PHOENIX-3161.patch, PHOENIX-3161_v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3393) Use Iterables.removeIf instead of Iterator.remove in HBase filters
[ https://issues.apache.org/jira/browse/PHOENIX-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594873#comment-15594873 ] Hudson commented on PHOENIX-3393: - FAILURE: Integrated in Jenkins build Phoenix-master #1444 (See [https://builds.apache.org/job/Phoenix-master/1444/]) PHOENIX-3393 Use Iterables.removeIf instead of Iterator.remove in HBase (ankitsinghal59: rev 5c9fb7b68ebeed20e3876579534df4adf21943a4) * (edit) phoenix-core/src/main/java/org/apache/phoenix/filter/ColumnProjectionFilter.java > Use Iterables.removeIf instead of Iterator.remove in HBase filters > -- > > Key: PHOENIX-3393 > URL: https://issues.apache.org/jira/browse/PHOENIX-3393 > Project: Phoenix > Issue Type: Improvement >Reporter: Robert Yokota >Assignee: Robert Yokota >Priority: Minor > Fix For: 4.9.0 > > Attachments: PHOENIX-3393.master.001.patch > > > This is a performance improvement to use Iterables.removeIf in the > filterRowCells method of ColumnProjectionFilter as described here: > https://rayokota.wordpress.com/2016/10/20/tips-on-writing-custom-hbase-filters/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3393) Use Iterables.removeIf instead of Iterator.remove in HBase filters
[ https://issues.apache.org/jira/browse/PHOENIX-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594811#comment-15594811 ] Ankit Singhal commented on PHOENIX-3393: LGTM as well. Thanks [~rayokota] for the contribution. Checked code for other areas if above optimization can be applied, but didn't find any code with randomAccessList(like Arraylist) and having significant remove operations. Committed your change to 4.x branches and master. > Use Iterables.removeIf instead of Iterator.remove in HBase filters > -- > > Key: PHOENIX-3393 > URL: https://issues.apache.org/jira/browse/PHOENIX-3393 > Project: Phoenix > Issue Type: Improvement >Reporter: Robert Yokota >Assignee: Robert Yokota >Priority: Minor > Fix For: 4.9.0 > > Attachments: PHOENIX-3393.master.001.patch > > > This is a performance improvement to use Iterables.removeIf in the > filterRowCells method of ColumnProjectionFilter as described here: > https://rayokota.wordpress.com/2016/10/20/tips-on-writing-custom-hbase-filters/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3393) Use Iterables.removeIf instead of Iterator.remove in HBase filters
[
https://issues.apache.org/jira/browse/PHOENIX-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594482#comment-15594482
]
Hadoop QA commented on PHOENIX-3393:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12834487/PHOENIX-3393.master.001.patch
against master branch at commit 1e78d3b368b7aa8397224074f691ea2fed340e34.
ATTACHMENT ID: 12834487
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
43 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//console
This message is automatically generated.
> Use Iterables.removeIf instead of Iterator.remove in HBase filters
> --
>
> Key: PHOENIX-3393
> URL: https://issues.apache.org/jira/browse/PHOENIX-3393
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Robert Yokota
>Assignee: Robert Yokota
>Priority: Minor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-3393.master.001.patch
>
>
> This is a performance improvement to use Iterables.removeIf in the
> filterRowCells method of ColumnProjectionFilter as described here:
> https://rayokota.wordpress.com/2016/10/20/tips-on-writing-custom-hbase-filters/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3387) Hive PhoenixStorageHandler fails with join on numeric fields
[
https://issues.apache.org/jira/browse/PHOENIX-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594396#comment-15594396
]
Hadoop QA commented on PHOENIX-3387:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12834615/PHOENIX-3387.patch
against master branch at commit 1e78d3b368b7aa8397224074f691ea2fed340e34.
ATTACHMENT ID: 12834615
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
43 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/630//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/630//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/630//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/630//console
This message is automatically generated.
> Hive PhoenixStorageHandler fails with join on numeric fields
>
>
> Key: PHOENIX-3387
> URL: https://issues.apache.org/jira/browse/PHOENIX-3387
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Labels: HivePhoenix
> Attachments: PHOENIX-3387.patch
>
>
> If condition for join is using numeric field, the MR job fails with class
> cast exception. The reason is that all object inspectors for numeric types
> doesn't implement getPrimitiveWritableObject method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PHOENIX-3395) ResultSet .next() throws commons-io exception
Vivek Paranthaman created PHOENIX-3395: -- Summary: ResultSet .next() throws commons-io exception Key: PHOENIX-3395 URL: https://issues.apache.org/jira/browse/PHOENIX-3395 Project: Phoenix Issue Type: Bug Affects Versions: 4.5.1 Environment: java -version: 1.7 Reporter: Vivek Paranthaman Exception in thread "main" org.apache.phoenix.exception.PhoenixIOException: java.lang.NoSuchMethodError: org.apache.commons.io.output.DeferredFileOutputStream.(ILjava/lang/String;Ljava/lang/String;Ljava/io/File;)V at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108) at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:553) at org.apache.phoenix.iterate.MergeSortResultIterator.getIterators(MergeSortResultIterator.java:48) at org.apache.phoenix.iterate.MergeSortResultIterator.minIterator(MergeSortResultIterator.java:84) at org.apache.phoenix.iterate.MergeSortResultIterator.next(MergeSortResultIterator.java:111) at org.apache.phoenix.iterate.MergeSortTopNResultIterator.next(MergeSortTopNResultIterator.java:85) at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:773) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3387) Hive PhoenixStorageHandler fails with join on numeric fields
[ https://issues.apache.org/jira/browse/PHOENIX-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Soldatov updated PHOENIX-3387: - Attachment: PHOENIX-3387.patch Added missing implementation for getting Writable instances. > Hive PhoenixStorageHandler fails with join on numeric fields > > > Key: PHOENIX-3387 > URL: https://issues.apache.org/jira/browse/PHOENIX-3387 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.8.0 >Reporter: Sergey Soldatov >Assignee: Sergey Soldatov > Labels: HivePhoenix > Attachments: PHOENIX-3387.patch > > > If condition for join is using numeric field, the MR job fails with class > cast exception. The reason is that all object inspectors for numeric types > doesn't implement getPrimitiveWritableObject method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-6) Support ON DUPLICATE KEY construct
[
https://issues.apache.org/jira/browse/PHOENIX-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-6:
---
Attachment: PHOENIX-6_wip3.patch
Working patch. Still needs more testing around indexes and multi-threading
> Support ON DUPLICATE KEY construct
> --
>
> Key: PHOENIX-6
> URL: https://issues.apache.org/jira/browse/PHOENIX-6
> Project: Phoenix
> Issue Type: New Feature
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-6_wip1.patch, PHOENIX-6_wip2.patch,
> PHOENIX-6_wip3.patch
>
>
> To support inserting a new row only if it doesn't already exist, we should
> support the "on duplicate key" construct for UPSERT. With this construct, the
> UPSERT VALUES statement would run atomically and would thus require a read
> before write which would obviously have a negative impact on performance. For
> an example of similar syntax , see MySQL documentation at
> http://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html
> See this discussion for more detail:
> https://groups.google.com/d/msg/phoenix-hbase-user/Bof-TLrbTGg/68bnc8ZcWe0J.
> A related discussion is on PHOENIX-2909.
> Initially we'd support the following:
> # This would prevent the setting of VAL to 0 if the row already exists:
> {code}
> UPSERT INTO T (PK, VAL) VALUES ('a',0)
> ON DUPLICATE KEY IGNORE;
> {code}
> # This would increment the valueS of COUNTER1 and COUNTER2 if the row already
> exists and otherwise initialize them to 0:
> {code}
> UPSERT INTO T (PK, COUNTER1, COUNTER2) VALUES ('a',0,0)
> ON DUPLICATE KEY UPDATE COUNTER1 = COUNTER1 + 1, COUNTER2 = COUNTER2 + 1;
> {code}
> So the general form is:
> {code}
> UPSERT ... VALUES ... [ ON DUPLICATE KEY [IGNORE | UPDATE
> =, ...] ]
> {code}
> The following restrictions will apply:
> - The may not be part of the primary key constraint - only KeyValue
> columns will be allowed.
> To handle the maintenance of immutable indexes, we'll need to push the
> maintenance to the server side.
> This is because the mutations for indexes on immutable tables are calculated
> on the client-side, while this new syntax would potentially modify the value
> on the server-side.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
