[
https://issues.apache.org/jira/browse/HIVE-21427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16811331#comment-16811331
]
Prasanth Jayachandran commented on HIVE-21427:
----------------------------------------------
The timestamp change to ConvertAstToSearchArg is incorrect. Orc does not handle
TIMESTAMP and TIMESTAMPLOCALTZ types correctly. ORC-189 is required. The
timestamps in syslog all assume UTC (TIMESTAMP type), but expression filters in
syslog_parser_file_pruning.q gets converted to local TZ (TIMESTAMPLOCALTZ)
giving incorrect results. Also there is no way to force using UTC for the qfile
tests. If we pass "-Duser.timezone=UTC" while running qfile it gives correct
results but then precommit servers cannot run qfile test with
"-Duser.timezone=UTC" option. We probably need a separate CliDriver for qfile
tests that runs tests in UTC but that is out of scope for this ticket. I will
disable syslog_parser_file_pruning.q for now as enabling it will give incorrect
results depending on which TZ ptest server runs. Also reverted changes to
ConvertAstToSearchArg.java file as ORC-189 will take care it.
> Syslog storage handler
> ----------------------
>
> Key: HIVE-21427
> URL: https://issues.apache.org/jira/browse/HIVE-21427
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Affects Versions: 4.0.0
> Reporter: Ashutosh Chauhan
> Assignee: Prasanth Jayachandran
> Priority: Major
> Attachments: HIVE-21427.1.patch, HIVE-21427.2.patch,
> HIVE-21427.3.patch, HIVE-21427.4.patch, HIVE-21427.5.patch
>
>
> It will be useful to read syslog generated log files in Hive. Hive generates
> logs in RFC5424 log4j2 layout and stores it as external table in sys.db. This
> includes a SyslogSerde that can parse RFC5424 formatted logs and maps them to
> logs table schema for query processing by hive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
