[jira] [Commented] (PHOENIX-4700) Fix split policy on system tables other than SYSTEM.CATALOG
[ https://issues.apache.org/jira/browse/PHOENIX-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450167#comment-16450167 ] Thomas D'Silva commented on PHOENIX-4700: - +1 > Fix split policy on system tables other than SYSTEM.CATALOG > --- > > Key: PHOENIX-4700 > URL: https://issues.apache.org/jira/browse/PHOENIX-4700 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor >Priority: Major > Fix For: 4.14.0, 5.0.0 > > Attachments: PHOENIX-4700.patch, PHOENIX-4700_wip1.patch > > > The MetaDataSplitPolicy was changed to cause a table to never split. This is > the right thing to do for SYSTEM.CATALOG, but not for the other system tables > that use it. We need to create a new split policy as it was before for these > other system tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4686) Phoenix stats does not account for server side limit push downs
[ https://issues.apache.org/jira/browse/PHOENIX-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450154#comment-16450154 ] Cody Marcel edited comment on PHOENIX-4686 at 4/24/18 4:26 PM: --- > "Calculates estimates when there's a limit pushed to the server (and no where > clause)" The original issue was a LIMIT in conjunction with a WHERE IN clause. Can we confirm this will fix that issue as well? was (Author: cody.mar...@gmail.com): > "Calculates estimates when there's a limit pushed to the server (and no where > clause)" The original issue was a LIMIT in conjunction with a WHERE IN clause. Can we confirm this will fix that issue as well > Phoenix stats does not account for server side limit push downs > --- > > Key: PHOENIX-4686 > URL: https://issues.apache.org/jira/browse/PHOENIX-4686 > Project: Phoenix > Issue Type: Bug >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Major > Attachments: PHOENIX-4686-wip.master.patch, > PHOENIX-4686.master.patch, PHOENIX-4686_wip1.patch > > > For a query like SELECT * FROM FOO LIMIT 10 the EST_BYTES_READ does not take > into account a limit correctly when there's no WHERE clause (or a WHERE > clause that gets compiled out into start/stop row key on the scan). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
[ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450233#comment-16450233 ] Andrew Purtell commented on PHOENIX-3534: - bq. Do we want to be able to rollback a release that contains this feature? If so we will still have to propagate metadata changes from the base table to all the child views and also ensure SYSTEM.CATALOG does not split. Or is it possible to make an HBase level snapshot of the whole SYSTEM schema, and then if there must be a rollback, we drop all of the upgraded SYSTEM tables and restore from the snapshot? Of course views created inbetween snapshot and rollback will be lost, but I think that is an acceptable tradeoff. Does this then give you the freedom you need to make all of the necessary changes? > Support multi region SYSTEM.CATALOG table > - > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-3534-wip.patch > > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region > based on the server-side row locks being held for operations that impact a > table and all of it's views. For example, adding/removing a column from a > base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), > when a new table is created we can do a lazy cleanup of any rows that may be > left over from a failed DDL call (kudos to [~lhofhansl] for coming up with > this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 > so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on > checking our sequence numbers for each table/view before/after updating. Each > table/view row would be individually locked for their change (metadata for a > view or table cannot span regions due to our split policy), with the sequence > number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
[ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450431#comment-16450431 ] James Taylor commented on PHOENIX-3534: --- IMHO, I don't think the propagation of properties from base table to child views will scale. We'd have to update ~2 million rows, trying to get the locks for all of them. Instead, I think we should disallow changes to a base table that need to be propagated to child views until after the splittable system catalog has been rolled out to both server and client (when we'd basically be saying that we won't be rolling back any longer). > Support multi region SYSTEM.CATALOG table > - > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-3534-wip.patch > > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region > based on the server-side row locks being held for operations that impact a > table and all of it's views. For example, adding/removing a column from a > base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), > when a new table is created we can do a lazy cleanup of any rows that may be > left over from a failed DDL call (kudos to [~lhofhansl] for coming up with > this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 > so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on > checking our sequence numbers for each table/view before/after updating. Each > table/view row would be individually locked for their change (metadata for a > view or table cannot span regions due to our split policy), with the sequence > number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4694) Prevent locking of parent table when dropping view to reduce contention
[ https://issues.apache.org/jira/browse/PHOENIX-4694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450153#comment-16450153 ] Thomas D'Silva commented on PHOENIX-4694: - +1 > Prevent locking of parent table when dropping view to reduce contention > --- > > Key: PHOENIX-4694 > URL: https://issues.apache.org/jira/browse/PHOENIX-4694 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Priority: Major > Fix For: 4.14.0, 5.0.0 > > Attachments: PHOENIX-4694.patch, PHOENIX-4694_v2.patch > > > When there are many views with the same parent table, there's a lot of > contention when adding new views and dropping existing views. The lock is > only necessary when creating/dropping indexes, so should be removed in the > case of views. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
[ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450263#comment-16450263 ] Thomas D'Silva commented on PHOENIX-3534: - As long as we are fine with losing views that were created between the upgrade and if/when we decided to rollback. This would be a lot simpler since we wouldn't have to support the old behavior. > Support multi region SYSTEM.CATALOG table > - > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-3534-wip.patch > > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region > based on the server-side row locks being held for operations that impact a > table and all of it's views. For example, adding/removing a column from a > base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), > when a new table is created we can do a lazy cleanup of any rows that may be > left over from a failed DDL call (kudos to [~lhofhansl] for coming up with > this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 > so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on > checking our sequence numbers for each table/view before/after updating. Each > table/view row would be individually locked for their change (metadata for a > view or table cannot span regions due to our split policy), with the sequence > number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
[ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450225#comment-16450225 ] Thomas D'Silva commented on PHOENIX-3534: - [~jamestaylor] , [~lhofhansl] Do we want to be able to rollback a release that contains this feature? If so we will still have to propagate metadata changes from the base table to all the child views and also ensure SYSTEM.CATALOG does not split. Once we stop propagating metadata changes to child views we won't be able to rollback, since the metadata of the view will only contain columns that were created in the view (and will not contain any parent table columns). So should we continue propagating metadata changes to child views for one more release? > Support multi region SYSTEM.CATALOG table > - > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-3534-wip.patch > > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region > based on the server-side row locks being held for operations that impact a > table and all of it's views. For example, adding/removing a column from a > base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), > when a new table is created we can do a lazy cleanup of any rows that may be > left over from a failed DDL call (kudos to [~lhofhansl] for coming up with > this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 > so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on > checking our sequence numbers for each table/view before/after updating. Each > table/view row would be individually locked for their change (metadata for a > view or table cannot span regions due to our split policy), with the sequence > number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
[ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450284#comment-16450284 ] Andrew Purtell commented on PHOENIX-3534: - [~lhofhansl] WDYT ^^^ > Support multi region SYSTEM.CATALOG table > - > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-3534-wip.patch > > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region > based on the server-side row locks being held for operations that impact a > table and all of it's views. For example, adding/removing a column from a > base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), > when a new table is created we can do a lazy cleanup of any rows that may be > left over from a failed DDL call (kudos to [~lhofhansl] for coming up with > this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 > so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on > checking our sequence numbers for each table/view before/after updating. Each > table/view row would be individually locked for their change (metadata for a > view or table cannot span regions due to our split policy), with the sequence > number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4701) Improve schema of SYSTEM.LOG table
[ https://issues.apache.org/jira/browse/PHOENIX-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451450#comment-16451450 ] James Taylor commented on PHOENIX-4701: --- [~an...@apache.org] - I've attached a wip2 patch that writes using Phoenix APIs so that you can have a composite row key to allow querying, use salting, have the table be column encoded, and potentially add secondary indexes. It needs a bit more work, though, because the column values are set in various places for the same query. I think we should log a single line per query - we can cover the failed/exception case and the success case. I think having so many individual Put RPCs won't scale well and even if it did, having a table with only the query ID in the PK means every query would be a full table scan. Would you have any cycles to take my patch further? Barring that, have we documented this yet? Perhaps we can just call this beta and let users know that the table structure will likely change in the future. Your thoughts would be valuable too, [~elserj]. > Improve schema of SYSTEM.LOG table > -- > > Key: PHOENIX-4701 > URL: https://issues.apache.org/jira/browse/PHOENIX-4701 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor >Priority: Major > Fix For: 4.14.0, 5.0.0 > > Attachments: PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch > > > If possible, the SYSTEM.LOG table would benefit greatly (3-5x perf gain) > from being declared as immutable with a column encoding of 1 byte and a > storage format of SINGLE_CELL_ARRAY_WITH_OFFSETS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
[ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451478#comment-16451478 ] Thomas D'Silva commented on PHOENIX-3534: - I think that approach works best. We would not need to support the old behavior and we would not lose any new views that were created if for some reason we needed to rollback the release. > Support multi region SYSTEM.CATALOG table > - > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-3534-wip.patch > > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region > based on the server-side row locks being held for operations that impact a > table and all of it's views. For example, adding/removing a column from a > base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), > when a new table is created we can do a lazy cleanup of any rows that may be > left over from a failed DDL call (kudos to [~lhofhansl] for coming up with > this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 > so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on > checking our sequence numbers for each table/view before/after updating. Each > table/view row would be individually locked for their change (metadata for a > view or table cannot span regions due to our split policy), with the sequence > number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4701) Improve schema of SYSTEM.LOG table
[
https://issues.apache.org/jira/browse/PHOENIX-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451546#comment-16451546
]
Josh Elser commented on PHOENIX-4701:
-
{quote}If so, we may need to do something similar like what our tracing
framework does where it makes sure writes to SYSTEM.TRACE table do not generate
traces themselves.
{quote}
This is something that Ankit implemented after the initial patch. I believe any
queries against system tables are not logged.
> Improve schema of SYSTEM.LOG table
> --
>
> Key: PHOENIX-4701
> URL: https://issues.apache.org/jira/browse/PHOENIX-4701
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch
>
>
> If possible, the SYSTEM.LOG table would benefit greatly (3-5x perf gain)
> from being declared as immutable with a column encoding of 1 byte and a
> storage format of SINGLE_CELL_ARRAY_WITH_OFFSETS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PHOENIX-4704) Presplit index tables when building asynchronously
Vincent Poon created PHOENIX-4704: - Summary: Presplit index tables when building asynchronously Key: PHOENIX-4704 URL: https://issues.apache.org/jira/browse/PHOENIX-4704 Project: Phoenix Issue Type: Improvement Reporter: Vincent Poon For large data tables with many regions, if we build the index asynchronously using the IndexTool, the index table will initial face a hotspot as all data region mappers attempt to write to the sole new index region. This can potentially lead to the index getting disabled if writes to the index table timeout during this hotspotting. We can add an optional step (or perhaps activate it based on the count of regions in the data table) to the IndexTool to first do a MR job to gather stats on the indexed column values, and then attempt to presplit the index table before we do the actual index build MR job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4701) Improve schema of SYSTEM.LOG table
[ https://issues.apache.org/jira/browse/PHOENIX-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4701: -- Attachment: PHOENIX-4701_wip2.patch > Improve schema of SYSTEM.LOG table > -- > > Key: PHOENIX-4701 > URL: https://issues.apache.org/jira/browse/PHOENIX-4701 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor >Priority: Major > Fix For: 4.14.0, 5.0.0 > > Attachments: PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch > > > If possible, the SYSTEM.LOG table would benefit greatly (3-5x perf gain) > from being declared as immutable with a column encoding of 1 byte and a > storage format of SINGLE_CELL_ARRAY_WITH_OFFSETS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4701) Improve schema of SYSTEM.LOG table
[ https://issues.apache.org/jira/browse/PHOENIX-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451462#comment-16451462 ] Samarth Jain commented on PHOENIX-4701: --- I haven't closely looked at the original commit, [~jamestaylor]. But do you think we can run into some kind of infinite loop by using the Phoenix API for writing to the SYSTEM.LOG table? If so, we may need to do something similar like what our tracing framework does where it makes sure writes to SYSTEM.TRACE table do not generate traces themselves. > Improve schema of SYSTEM.LOG table > -- > > Key: PHOENIX-4701 > URL: https://issues.apache.org/jira/browse/PHOENIX-4701 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor >Priority: Major > Fix For: 4.14.0, 5.0.0 > > Attachments: PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch > > > If possible, the SYSTEM.LOG table would benefit greatly (3-5x perf gain) > from being declared as immutable with a column encoding of 1 byte and a > storage format of SINGLE_CELL_ARRAY_WITH_OFFSETS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4645) PhoenixStorageHandler doesn't handle correctly data/timestamp in push down predicate when engine is tez.
[
https://issues.apache.org/jira/browse/PHOENIX-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449433#comment-16449433
]
Hudson commented on PHOENIX-4645:
-
FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-0.98 #1867 (See
[https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/1867/])
PHOENIX-4645 PhoenixStorageHandler doesn't handle correctly (rajeshbabu: rev
4c31ac3f5f4099d93885e26ebcd91f1991ebf954)
* (edit)
phoenix-hive/src/it/java/org/apache/phoenix/hive/HivePhoenixStoreIT.java
* (edit)
phoenix-hive/src/main/java/org/apache/phoenix/hive/constants/PhoenixStorageHandlerConstants.java
* (edit)
phoenix-hive/src/main/java/org/apache/phoenix/hive/query/PhoenixQueryBuilder.java
> PhoenixStorageHandler doesn't handle correctly data/timestamp in push down
> predicate when engine is tez.
> -
>
> Key: PHOENIX-4645
> URL: https://issues.apache.org/jira/browse/PHOENIX-4645
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Labels: HivePhoenix
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4645-wip.patch, PHOENIX-4645.patch
>
>
> DDLs:
> {noformat}
> CREATE TABLE TEST_PHOENIX
> (
> PART_ID BIGINT NOT NULL,
> COMMIT_TIMESTAMP TIMESTAMP,
> CONSTRAINT pk PRIMARY KEY (PART_ID)
> )
> SALT_BUCKETS=9;
> CREATE EXTERNAL TABLE TEST_HIVE
> (
> PART_ID BIGINT,
> SOURCEDB_COMMIT_TIMESTAMP TIMESTAMP
> )
> STORED BY 'org.apache.phoenix.hive.PhoenixStorageHandler'
> TBLPROPERTIES
> (
> "phoenix.table.name" = "TEST_PHOENIX",
> "phoenix.zookeeper.quorum" = "localhost",
> "phoenix.zookeeper.znode.parent" = "/hbase",
> "phoenix.zookeeper.client.port" = "2181",
> "phoenix.rowkeys" = "PART_ID",
> "phoenix.column.mapping" =
> "part_id:PART_ID,sourcedb_commit_timestamp:COMMIT_TIMESTAMP"
> );
> {noformat}
> Query :
> {noformat}
> hive> select * from TEST_HIVE2 where sourcedb_commit_timestamp between
> '2018-03-01 01:00:00.000' and '2018-03-20 01:00:00.000';
> OK
> Failed with exception java.io.IOException:java.lang.RuntimeException:
> org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type
> mismatch. TIMESTAMP and VARCHAR for "sourcedb_commit_timestamp" >=
> '2018-03-01 01:00:00.000'
> {noformat}
> That happens because we don't use mapped column name when we check whether we
> need to apply to_timestamp/to_date function. For the default mapping, we
> regexp patterns don't take into account that column name is double quoted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4645) PhoenixStorageHandler doesn't handle correctly data/timestamp in push down predicate when engine is tez.
[
https://issues.apache.org/jira/browse/PHOENIX-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449460#comment-16449460
]
Hudson commented on PHOENIX-4645:
-
FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #105 (See
[https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/105/])
PHOENIX-4645 PhoenixStorageHandler doesn't handle correctly (rajeshbabu: rev
e578a869cec2c9136aa6ecfcd60fd46bc4f434f4)
* (edit)
phoenix-hive/src/it/java/org/apache/phoenix/hive/HivePhoenixStoreIT.java
* (edit)
phoenix-hive/src/main/java/org/apache/phoenix/hive/query/PhoenixQueryBuilder.java
* (edit)
phoenix-hive/src/main/java/org/apache/phoenix/hive/constants/PhoenixStorageHandlerConstants.java
> PhoenixStorageHandler doesn't handle correctly data/timestamp in push down
> predicate when engine is tez.
> -
>
> Key: PHOENIX-4645
> URL: https://issues.apache.org/jira/browse/PHOENIX-4645
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Labels: HivePhoenix
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4645-wip.patch, PHOENIX-4645.patch
>
>
> DDLs:
> {noformat}
> CREATE TABLE TEST_PHOENIX
> (
> PART_ID BIGINT NOT NULL,
> COMMIT_TIMESTAMP TIMESTAMP,
> CONSTRAINT pk PRIMARY KEY (PART_ID)
> )
> SALT_BUCKETS=9;
> CREATE EXTERNAL TABLE TEST_HIVE
> (
> PART_ID BIGINT,
> SOURCEDB_COMMIT_TIMESTAMP TIMESTAMP
> )
> STORED BY 'org.apache.phoenix.hive.PhoenixStorageHandler'
> TBLPROPERTIES
> (
> "phoenix.table.name" = "TEST_PHOENIX",
> "phoenix.zookeeper.quorum" = "localhost",
> "phoenix.zookeeper.znode.parent" = "/hbase",
> "phoenix.zookeeper.client.port" = "2181",
> "phoenix.rowkeys" = "PART_ID",
> "phoenix.column.mapping" =
> "part_id:PART_ID,sourcedb_commit_timestamp:COMMIT_TIMESTAMP"
> );
> {noformat}
> Query :
> {noformat}
> hive> select * from TEST_HIVE2 where sourcedb_commit_timestamp between
> '2018-03-01 01:00:00.000' and '2018-03-20 01:00:00.000';
> OK
> Failed with exception java.io.IOException:java.lang.RuntimeException:
> org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type
> mismatch. TIMESTAMP and VARCHAR for "sourcedb_commit_timestamp" >=
> '2018-03-01 01:00:00.000'
> {noformat}
> That happens because we don't use mapped column name when we check whether we
> need to apply to_timestamp/to_date function. For the default mapping, we
> regexp patterns don't take into account that column name is double quoted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
