[
https://issues.apache.org/jira/browse/PHOENIX-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175230#comment-16175230
]
Sergey Soldatov commented on PHOENIX-4222:
------------------------------------------
[~elserj] Well, all this column mapping is really pain in the neck. The main
problem is that we need not only map it from hive to phoenix, but also do the
reverse during the query execution. Keeping all column maps all the time was
looking promising, but I recently run into the case when it was not possible to
map all columns because Hive has limit of 4000 chars for properties. Need to
think how to better handle that.
> PhoenixStorageHandler for Hive is not working
> ---------------------------------------------
>
> Key: PHOENIX-4222
> URL: https://issues.apache.org/jira/browse/PHOENIX-4222
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.11.0
> Reporter: JeongMin Ju
> Assignee: Sergey Soldatov
> Labels: hive
> Attachments: PhoenixQueryBuilder-buildExpressionStringFrom.jpeg,
> PhoenixQueryBuilder-getSelectColumns.jpeg
>
>
> {panel}
> ERROR CliDriver: Failed with exception
> java.io.IOException:java.lang.RuntimeException:
> org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703):
> Undefined column. columnName=PHOENIX.MOVIES.movie_id
> java.io.IOException: java.lang.RuntimeException:
> org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703):
> Undefined column. columnName=PHOENIX.MOVIES.movie_id
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1986)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:240)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:172)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:383)
> at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:775)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:693)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)
> Caused by: java.lang.RuntimeException:
> org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703):
> Undefined column. columnName=PHOENIX.MOVIES.movie_id
> at
> org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:266)
> at
> org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:131)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:363)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:295)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446)
> ... 9 more
> Caused by: org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504
> (42703): Undefined column. columnName=PHOENIX.MOVIES.movie_id
> at
> org.apache.phoenix.schema.PTableImpl.getColumnForColumnName(PTableImpl.java:818)
> at
> org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:478)
> at
> org.apache.phoenix.compile.TupleProjectionCompiler$ColumnRefVisitor.visit(TupleProjectionCompiler.java:208)
> at
> org.apache.phoenix.compile.TupleProjectionCompiler$ColumnRefVisitor.visit(TupleProjectionCompiler.java:1)
> at org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
> at
> org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:109)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:528)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:507)
> at
> org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:202)
> at org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:475)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:1)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1648)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1635)
> at
> org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:260)
> ... 13 more
> {panel}
> {noformat}
> : jdbc:phoenix:juke-cdh2> !columns phoenix.movies
> +------------+--------------+-------------+--------------+------------+------------+--------------+----------------+-----------------+-----------------+-----------+----------+-------------+
> | TABLE_CAT | TABLE_SCHEM | TABLE_NAME | COLUMN_NAME | DATA_TYPE |
> TYPE_NAME | COLUMN_SIZE | BUFFER_LENGTH | DECIMAL_DIGITS | NUM_PREC_RADIX
> | NULLABLE | REMARKS | COLUMN_DEF |
> +------------+--------------+-------------+--------------+------------+------------+--------------+----------------+-----------------+-----------------+-----------+----------+-------------+
> | | PHOENIX | MOVIES | MOVIE_ID | 4 |
> INTEGER | null | null | null | null
> | 0 | | |
> | | PHOENIX | MOVIES | TITLE | 12 |
> VARCHAR | null | null | null | null
> | 1 | | |
> | | PHOENIX | MOVIES | GENRES | 12 |
> VARCHAR | null | null | null | null
> | 1 | | |
> +------------+--------------+-------------+--------------+------------+------------+--------------+----------------+-----------------+-----------------+-----------+----------+-------------+
> {noformat}
> {noformat}
> hive> show create table movies;
> CREATE EXTERNAL TABLE `movies`(
> `movie_id` int COMMENT 'from deserializer',
> `title` string COMMENT 'from deserializer',
> `genres` string COMMENT 'from deserializer')
> ROW FORMAT SERDE
> 'org.apache.phoenix.hive.PhoenixSerDe'
> STORED BY
> 'org.apache.phoenix.hive.PhoenixStorageHandler'
> WITH SERDEPROPERTIES (
> 'serialization.format'='1')
> LOCATION
> 'hdfs://nameservice1/user/hive/warehouse/movies'
> TBLPROPERTIES (
> 'COLUMN_STATS_ACCURATE'='false',
> 'numFiles'='0',
> 'numRows'='-1',
> 'phoenix.table.name'='phoenix.movies',
> 'phoenix.zookeeper.quorum'='xxx',
> 'rawDataSize'='-1',
> 'totalSize'='0',
> 'transient_lastDdlTime'='1503021635')
> {noformat}
> I ran the query "select * from movies" and
> The query created by the phoenix handler is:
> select /*+ NO_CACHE */ "movie_id","title","genres" from phoenix.movies
> But do not enter double quotes like this.
> I looked up where to put double quotes.
> !https://issues.apache.org/jira/secure/attachment/12888228/PhoenixQueryBuilder-getSelectColumns.jpeg!
> !https://issues.apache.org/jira/secure/attachment/12888229/PhoenixQueryBuilder-buildExpressionStringFrom.jpeg!
> Why did you change it to double quotes?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
