[ https://issues.apache.org/jira/browse/HIVE-16667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16019988#comment-16019988 ]
Naveen Gangam commented on HIVE-16667: -------------------------------------- [~rusanu] I have tried a couple of means to change the query in direct sql to get it to read thru the {{text}} columns. They work via native postgresql client {{psql}} but fail via JDO. I have also tried to alter the table definition to force in-line storage of the column {{plain || main}} value but it seems to have no impact on how it is actually persisted. This is going to need a bit more research. Unblocking the release is now a priority. Until we can figure out a proper solution, we can fix it so that extractSqlClob throws an exception if the value is not clob type and that should get it to retry using ORM. I have test this fix locally and seems to be working. Thoughts? > PostgreSQL metastore handling of CLOB types for COLUMNS_V2.TYPE_NAME and > other field is incorrect > ------------------------------------------------------------------------------------------------- > > Key: HIVE-16667 > URL: https://issues.apache.org/jira/browse/HIVE-16667 > Project: Hive > Issue Type: Bug > Reporter: Remus Rusanu > Assignee: Naveen Gangam > Attachments: HiveCLIOutput.txt, PostgresDBOutput.txt > > > The CLOB JDO type introduced with HIVE-12274 does not work correctly with > PostgreSQL. The value is written out-of-band and the LOB handle is written,as > an INT, into the table. SELECTs return the INT value, which should had been > read via the {{lo_get}} PG built-in, and then cast into string. > Furthermore, the behavior is different between fields upgraded from earlier > metastore versions (they retain their string storage) vs. values inserted > after the upgrade (inserted as LOB roots). > Teh code in > {{MetasoreDirectSql.getPartitionsFromPartitionIds/extractSqlClob}} expects > the underlying JDO/Datanucleus to map the column to a {{Clob}} but that does > not happen, the value is a Java String containing the int which is the LOB > root saved by PG. > This manifests at runtime with errors like: > {code} > hive> select * from srcpart; > Failed with exception java.io.IOException:java.lang.IllegalArgumentException: > Error: type expected at the position 0 of '24030:24031' but '24030' is found. > {code} > the 24030:24031 should be 'string:string'. > repro: > {code} > CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT > 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE; > LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO > TABLE srcpart PARTITION (ds="2008-04-09", hr="11"); > select * from srcpart; > {code} > I did not see the issue being hit by non-partitioned/textfile tables, but > that is just the luck of the path taken by the code. Inspection of my PG > metastore shows all the CLOB fields suffering from this issue. -- This message was sent by Atlassian JIRA (v6.3.15#6346)