-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57693/
-----------------------------------------------------------
(Updated April 14, 2017, 6:46 a.m.)
Review request for hive, Zoltan Haindrich, Marta Kuczora, Miklos Csanady,
Naveen Gangam, Vihang Karajgaonkar, Yongzhi Chen, and Barna Zsombor Klara.
Changes
-------
Rebased the patch.
Since after the rebase the TestBeeLineDriver runs more query files, more
replace patterns are needed:
- The output contains the original queries, so we should rewrite
(default.src->src) for the output file as well
- More static regex for hiding the file locations, and serdes to match the
CliDriver output
- More specific regex for hiding the database change in the explain results
Followup jira (HIVE-16449) is created to handle -- SORT_BEFORE_DIFF. Otherwise
the query output files are the same as the CliDriver version output
Bugs: HIVE-16146
https://issues.apache.org/jira/browse/HIVE-16146
Repository: hive-git
Description
-------
The goal was to generate '\0' markers around the raw log items we want to keep
in the golden files.
To archive this I had to do a small functional change and some small refactor:
- Removed the immutability of the format map in BeeLine, so the test could add
the QFile specific OutputFromat as a possible format
- The PostExecutePrinter and the PreExecutePrinter got a common ancestor, which
was due anyway because PostExecutePrinter reused static methods from
PreExecutePrinter. This way I was able to create QFile specific printers which
are generating the desired markers.
- Moved the QFile test to the org.apache.hive.beeline package, so the test
classes can use the package private classes and methods
For one reason or other BeeLine added an extra space character at the end of
the lines for multiline commands - I have removed this space - Will see if this
effects any unit test or not.
With the above mentioned *OutputFromat*, *QFilePreExecutePrinter*,
*QFilePostExecutePrinter* we can mark the lines which are needed in the q.out
file, and during the filtering we can remove the unneeded parts - I prefer to
keep the log level high in the raw files, so in case of a test failure we can
have better understanding of what has happened.
In the test files:
- Updated the beforeExecute, and afterExecute methods to set the new
outputformat and the new hooks
- Removed the query specific filters, since they are non exitstent in the CLI
tests
- Simplified the static filterset - currently only contains the filters which
are really neccessary for the actual tests - might grow to the same size than
in the QTestUtils - but if we do not want to run all of the test we would like
to keep this list as small as possible
- Removed unnecessary configurations from QFileBuilder which will be not needed
in case we want to mimic the CLI results
Diffs (updated)
-----
beeline/src/java/org/apache/hive/beeline/BeeLine.java 27b353c
beeline/src/java/org/apache/hive/beeline/Commands.java d179b37
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 726522b
itests/src/test/resources/testconfiguration.properties df80478
itests/util/src/main/java/org/apache/hive/beeline/qfile/QFile.java 9fae194
itests/util/src/main/java/org/apache/hive/beeline/qfile/QFileBeeLineClient.java
7244bf8
itests/util/src/main/java/org/apache/hive/beeline/qfile/package-info.java
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java a5c0fcd
ql/src/java/org/apache/hadoop/hive/ql/log/LogDivertAppenderForTest.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/session/OperationLog.java c37a633
ql/src/test/results/clientpositive/beeline/drop_with_concurrency.q.out
2bffdf0
ql/src/test/results/clientpositive/beeline/escape_comments.q.out b440d9c
ql/src/test/results/clientpositive/beeline/select_dummy_source.q.out
PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_1.q.out 70a37ca
ql/src/test/results/clientpositive/beeline/smb_mapjoin_10.q.out bdfaefb
ql/src/test/results/clientpositive/beeline/smb_mapjoin_11.q.out 19c07a0
ql/src/test/results/clientpositive/beeline/smb_mapjoin_12.q.out 98bf25e
ql/src/test/results/clientpositive/beeline/smb_mapjoin_13.q.out d303900
ql/src/test/results/clientpositive/beeline/smb_mapjoin_16.q.out b8a06dc
ql/src/test/results/clientpositive/beeline/smb_mapjoin_2.q.out 22a2d6a
ql/src/test/results/clientpositive/beeline/smb_mapjoin_3.q.out 6c9b8e4
ql/src/test/results/clientpositive/beeline/smb_mapjoin_7.q.out b15c951
service/src/java/org/apache/hive/service/cli/operation/OperationManager.java
f62ee4e
service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
7df4563
Diff: https://reviews.apache.org/r/57693/diff/3/
Changes: https://reviews.apache.org/r/57693/diff/2-3/
Testing
-------
Added a new simple query file from CLI driver, and checked that the generated
output is the same
Thanks,
Peter Vary
