[
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332125#comment-16332125
]
Laszlo Bodor edited comment on HIVE-18061 at 1/19/18 11:22 AM:
---------------------------------------------------------------
Unfortunately *q.out_spark files are needed. There are 2 different spark modes,
which are validated with the same set of q.out files: a local (spark), and a
hdfs (miniSparkOnYarn) mode. In case of local mode, the paths are like
"file://", and the patch only handles "hdfs://" paths, so for about 5-6 qouts,
i had to differentiate between them.
QTestUtil is prepared to pick from multiple q.outs according to the current
mode. This functionality had been already implemented, but not used.
sysdb.q.out: let me check and reset that
was (Author: abstractdog):
Unfortunately *q.out_spark files are needed. There are 2 different spark modes,
which are validated with the same set of q.out files: a local (spark), and a
hdfs (miniSparkOnYarn) mode. In case of local mode, the paths are like
"file://", and the patch only handles "hdfs://" paths, so for about 5-6 quots,
i had to differentiate between qout files. QTestUtil is prepared to pick from
multiple q.outs according to the current mode.
sysdb.q.out: let me check and reset that
> q.outs: be more selective with masking hdfs paths
> -------------------------------------------------
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
> Issue Type: Improvement
> Reporter: Zoltan Haindrich
> Assignee: Laszlo Bodor
> Priority: Major
> Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch,
> HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch,
> HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch,
> HIVE-18061.09.patch, HIVE-18061.10.patch, HIVE-18061.11.patch,
> HIVE-18061.12.patch, HIVE-18061.13.patch
>
>
> currently any line which contains a path which looks like an hdfs location is
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception
> message might contain an hdfs location
> noticed in
> HIVE-18012
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
