Sindhu Subhas created AMBARI-23179:
--------------------------------------
Summary: Hive view 2.0 does not parse the TAB delimited CSV files
while using 'Upload Table'
Key: AMBARI-23179
URL: https://issues.apache.org/jira/browse/AMBARI-23179
Project: Ambari
Issue Type: Bug
Components: ambari-views
Affects Versions: 2.6.0
Environment: HDP 2.6.3
Ambari 2.6.0
Reporter: Sindhu Subhas
Problem:
When trying to Upload table from Hive view 2.0 from a TAB delimited file, the
data is not parsed as expected.
Same feature works as expected in Hive view 1.5.0.
Attached output from Hive view 2.0 and output from Hive view 1.5.0.
>From the code, we do see difference in the code for
>src/main/java/org/apache/ambari/view/hive20/resources/uploads/UploadService.java.
Debug from Hive view 2.0:
{code:java}
27 Feb 2018 20:27:28,169 DEBUG
[HiveViewActorSystem-akka.actor.default-dispatcher-4] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] OperationController:53 - } java.util.HashMap
27 Feb 2018 20:27:28,169 DEBUG
[HiveViewActorSystem-akka.actor.default-dispatcher-4] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] HiveActor:41 - Message submitted:
927343c0-5ed6-435c-be60-37448ac1a8b0
27 Feb 2018 20:27:30,925 INFO [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] UploadService:484 - isFirstRowHeader : false,
inputFileType : CSV
27 Feb 2018 20:27:30,934 INFO [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] OpenCSVParser:50 - setting delimiter as T
27 Feb 2018 20:27:30,934 INFO [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] OpenCSVParser:56 - setting Quote char : "
27 Feb 2018 20:27:30,934 INFO [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] OpenCSVParser:62 - setting escapeChar : \
27 Feb 2018 20:27:30,939 INFO [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] Parser:75 - generating preview for :
ParseOptions{options={OPTIONS_CSV_QUOTE=", OPTIONS_CSV_DELIMITER=T,
HEADER=NONE, FILE_TYPE=CSV, OPTIONS_CSV_ESCAPE_CHAR=\}}
27 Feb 2018 20:27:30,939 DEBUG [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] Parser:83 - Illegal number of preview columns supplied
null
27 Feb 2018 20:27:30,943 DEBUG [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] ParseUtils:152 - error while parsing as timestamp string
column1 column2 column3
java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd
hh:mm:ss[.fffffffff]
at java.sql.Timestamp.valueOf(Timestamp.java:204)
.....
27 Feb 2018 20:27:30,948 INFO [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] Parser:149 - datatype detected for column 0 : STRING
27 Feb 2018 20:27:30,949 DEBUG [ambari-client-thread-38] [HIVE 2.0.0
AUTO_HIVE20_INSTANCE] Parser:159 - return headers : [ColumnInfo{name='column1',
type='STRING', precision=null, scale=null, comment='null'}]
{code}
Debug from Hive view 1.5.0:
{code:java}
27 Feb 2018 20:13:15,881 INFO [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] UploadService:499 - isFirstRowHeader : true, inputFileType
: CSV
27 Feb 2018 20:13:15,918 INFO [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] OpenCSVParser:50 - setting delimiter as
27 Feb 2018 20:13:15,919 INFO [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] OpenCSVParser:56 - setting Quote char : "
27 Feb 2018 20:13:15,919 INFO [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] OpenCSVParser:62 - setting escapeChar : \
27 Feb 2018 20:13:15,928 INFO [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] Parser:75 - generating preview for :
ParseOptions{options={OPTIONS_CSV_QUOTE=", OPTIONS_CSV_DELIMITER= ,
HEADER=FIRST_RECORD, FILE_TYPE=CSV, OPTIONS_CSV_ESCAPE_CHAR=\}}
27 Feb 2018 20:13:15,929 DEBUG [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] Parser:83 - Illegal number of preview columns supplied null
27 Feb 2018 20:13:15,936 DEBUG [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] ParseUtils:152 - error while parsing as timestamp string AD
java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd
hh:mm:ss[.fffffffff]
at java.sql.Timestamp.valueOf(Timestamp.java:204)
.........
27 Feb 2018 20:13:15,961 INFO [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] Parser:149 - datatype detected for column 2 : STRING
27 Feb 2018 20:13:15,962 DEBUG [ambari-client-thread-38] [HIVE 1.5.0
AUTO_HIVE_INSTANCE] Parser:159 - return headers : [ColumnDescriptionImpl[name :
column1, type : STRING, position : 0, precision : null, scale : null],
ColumnDescriptionImpl[name : column2, type : STRING, position : 1, precision :
null, scale : null], ColumnDescriptionImpl[name : column3, type : STRING,
position : 2, precision : null, scale : null]]
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
