[jira] [Commented] (HIVE-11201) HCatalog is ignoring user specified avro schema in the table definition
[ https://issues.apache.org/jira/browse/HIVE-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997238#comment-14997238 ] Ashutosh Chauhan commented on HIVE-11201: - +1 > HCatalog is ignoring user specified avro schema in the table definition > > > Key: HIVE-11201 > URL: https://issues.apache.org/jira/browse/HIVE-11201 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 1.2.0, 1.2.1 >Reporter: Bing Li >Assignee: Bing Li >Priority: Critical > Attachments: HIVE-11201.1.patch > > > HCatalog is ignoring user specified avro schema in the table definition , > instead generating its own avro based from hive meta store. > By generating its own schema will result in mismatch names. For exmple Avro > fields name are Case Sensitive. By generating it's own schema will result > in incorrect schema written to the avro file , and result select fail on > read. And also Even if user specified schema does not allow null , when > data is written using Hcatalog , it will write a schema that will allow null. > For example in the table , user specified , all CAPITAL letters in the > schema , and record name as LINEITEM. The schema should be written as it is. > Instead Hcatalog ignores it and generated its own avro schema from the hive > table case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11201) HCatalog is ignoring user specified avro schema in the table definition
[ https://issues.apache.org/jira/browse/HIVE-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736076#comment-14736076 ] Bing Li commented on HIVE-11201: I submitted the review request manually. The link is https://reviews.apache.org/r/34877/ > HCatalog is ignoring user specified avro schema in the table definition > > > Key: HIVE-11201 > URL: https://issues.apache.org/jira/browse/HIVE-11201 > Project: Hive > Issue Type: Bug > Components: HCatalog >Affects Versions: 1.2.0, 1.2.1 >Reporter: Bing Li >Assignee: Bing Li >Priority: Critical > Attachments: HIVE-11201.1.patch > > > HCatalog is ignoring user specified avro schema in the table definition , > instead generating its own avro based from hive meta store. > By generating its own schema will result in mismatch names. For exmple Avro > fields name are Case Sensitive. By generating it's own schema will result > in incorrect schema written to the avro file , and result select fail on > read. And also Even if user specified schema does not allow null , when > data is written using Hcatalog , it will write a schema that will allow null. > For example in the table , user specified , all CAPITAL letters in the > schema , and record name as LINEITEM. The schema should be written as it is. > Instead Hcatalog ignores it and generated its own avro schema from the hive > table case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11201) HCatalog is ignoring user specified avro schema in the table definition
[
https://issues.apache.org/jira/browse/HIVE-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618676#comment-14618676
]
Hive QA commented on HIVE-11201:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12744152/HIVE-11201.1.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9136 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_merge_multi_expressions
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join_merge_multi_expressions
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_louter_join_ppr
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_outer_join_ppr
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4535/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4535/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-4535/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12744152 - PreCommit-HIVE-TRUNK-Build
HCatalog is ignoring user specified avro schema in the table definition
Key: HIVE-11201
URL: https://issues.apache.org/jira/browse/HIVE-11201
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 1.2.0, 1.2.1
Reporter: Bing Li
Assignee: Bing Li
Priority: Critical
Attachments: HIVE-11201.1.patch
HCatalog is ignoring user specified avro schema in the table definition ,
instead generating its own avro based from hive meta store.
By generating its own schema will result in mismatch names. For exmple Avro
fields name are Case Sensitive. By generating it's own schema will result
in incorrect schema written to the avro file , and result select fail on
read. And also Even if user specified schema does not allow null , when
data is written using Hcatalog , it will write a schema that will allow null.
For example in the table , user specified , all CAPITAL letters in the
schema , and record name as LINEITEM. The schema should be written as it is.
Instead Hcatalog ignores it and generated its own avro schema from the hive
table case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
