[jira] [Commented] (HIVE-14409) Allow regex pattern in the output of qtest
[
https://issues.apache.org/jira/browse/HIVE-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406881#comment-15406881
]
Hive QA commented on HIVE-14409:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12821850/HIVE-14409.00.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10433 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testForcedLocalityMultiplePreemptionsSameHost2
org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.stringifyValidTxns
org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.testTxnRange
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/754/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/754/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-754/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12821850 - PreCommit-HIVE-MASTER-Build
> Allow regex pattern in the output of qtest
> --
>
> Key: HIVE-14409
> URL: https://issues.apache.org/jira/browse/HIVE-14409
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
> Attachments: HIVE-14409.00.patch
>
>
> We cannot guarantee the output from external libraries (ex. HashMap), so we
> need to make the validation infrastructure to allow a sort within a single
> line.
> HIVE-13894, HIVE-13907, and HIVE-14088 are related.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14409) Allow regex pattern in the output of qtest
[ https://issues.apache.org/jira/browse/HIVE-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406426#comment-15406426 ] Sergey Shelukhin commented on HIVE-14409: - Judging by the changes in the file, this doesn't scale very well (aside from not being able to overwrite with maven, which is a huge pain esp. if this is used in many tests). For 2 values, it's ok to "or" the orders... What if there are 4 values? I think for explain it's much easier to just enforce order. Or, given that list format is relatively well-defined, it could have a list of order-agnostic patterns, kinda like it has a list of patterns to ignore. Then it can extract the list for such prefixes/patterns and reorder as needed. We already have similar functionality in SORT_QUERY_RESULTS. > Allow regex pattern in the output of qtest > -- > > Key: HIVE-14409 > URL: https://issues.apache.org/jira/browse/HIVE-14409 > Project: Hive > Issue Type: Improvement > Components: Test >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HIVE-14409.00.patch > > > We cannot guarantee the output from external libraries (ex. HashMap), so we > need to make the validation infrastructure to allow a sort within a single > line. > HIVE-13894, HIVE-13907, and HIVE-14088 are related. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14409) Allow regex pattern in the output of qtest
[ https://issues.apache.org/jira/browse/HIVE-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406147#comment-15406147 ] Peter Vary commented on HIVE-14409: --- Thanks for the patch. These flaky tests are a pain. It will be nice to get rid of them. I think, that with this patch we lose the ability to regenerate the out file with maven. Am I right, or did I miss something? One solution could be putting the patterns into the input file, or even better into a 3rd file, and if that file doesn't exists, then we do not have to run the pattern matching for the whole file when it is not necessary > Allow regex pattern in the output of qtest > -- > > Key: HIVE-14409 > URL: https://issues.apache.org/jira/browse/HIVE-14409 > Project: Hive > Issue Type: Improvement > Components: Test >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HIVE-14409.00.patch > > > We cannot guarantee the output from external libraries (ex. HashMap), so we > need to make the validation infrastructure to allow a sort within a single > line. > HIVE-13894, HIVE-13907, and HIVE-14088 are related. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14409) Allow regex pattern in the output of qtest
[
https://issues.apache.org/jira/browse/HIVE-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406062#comment-15406062
]
Akira Ajisaka commented on HIVE-14409:
--
Thanks [~kgyrtkirk] and [~pvary] for the comments.
bq. i'm afraid that making the test comparisions smarter may even hide bugs
later on...
Agreed. We should use regex pattern only when we really need to do.
bq. i planning to somehow reduce the number of these and put more pressure on
the @Explain based output generator - which i think already can produce stable
outputs independently of jvm
I'm afraid that {{@Explain}} based output affects performance. If this
annotation can only be used in test, it's great.
bq. Probably already aware of this, but there is a partialReservedPlanMask in
QtestUtil
Thank you for the input!
> Allow regex pattern in the output of qtest
> --
>
> Key: HIVE-14409
> URL: https://issues.apache.org/jira/browse/HIVE-14409
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>
> We cannot guarantee the output from external libraries (ex. HashMap), so we
> need to make the validation infrastructure to allow a sort within a single
> line.
> HIVE-13894, HIVE-13907, and HIVE-14088 are related.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14409) Allow regex pattern in the output of qtest
[ https://issues.apache.org/jira/browse/HIVE-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405558#comment-15405558 ] Peter Vary commented on HIVE-14409: --- Probably already aware of this, but there is a partialReservedPlanMask in QtestUtil, which allows pattern matching for the output lines, and regex could be used to match groups with different orders (for example: http://stackoverflow.com/questions/2596868/regex-matching-against-groups-in-different-order-without-repeating-the-group). So if it is not test specific issue, then it could be used to help the problem - and generated more, as [~kgyrtkirk] already pointed out :) > Allow regex pattern in the output of qtest > -- > > Key: HIVE-14409 > URL: https://issues.apache.org/jira/browse/HIVE-14409 > Project: Hive > Issue Type: Improvement > Components: Test >Reporter: Akira Ajisaka > > We cannot guarantee the output from external libraries (ex. HashMap), so we > need to make the validation infrastructure to allow a sort within a single > line. > HIVE-13894, HIVE-13907, and HIVE-14088 are related. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14409) Allow regex pattern in the output of qtest
[
https://issues.apache.org/jira/browse/HIVE-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405494#comment-15405494
]
Zoltan Haindrich commented on HIVE-14409:
-
[~ajisakaa]
i think a micro dsl inside the q-files would be useful - for cases in which the
application output can't be reliably defined
...but I think that hive's output for explain and such should be:
* stable(never change in repeated executions)
* and be as much as independent from the execution environment (like jdk
version) as possible.
i'm afraid that making the test comparisions smarter may even hide bugs later
on...
i think that burning a few more cycles of application time to produce more
stable explain outputs can even help the end user who is changing their
application environment and looking for any regresions after the changes have
been made.
...when you look at just one line in an output you don't really know that the
source of that line was a list or a set; for a set it's perfectly fine to sort
- but for an array it doesn't really align with application internals.
i've worked toward stabilizing explain outputs lately in HIVE-14285,
HIVE-14287, HIVE-14329
i know there are a few more affected...it seems to me that there are many
explain output writing methods here and there ;) ...i planning to somehow
reduce the number of these and put more pressure on the {{@Explain}} based
output generator - which i think already can produce stable outputs
independently of jvm ; but if it's not - theres only one place where it has to
be fixed.
i know this is a "longer" path...what do you think about it?
> Allow regex pattern in the output of qtest
> --
>
> Key: HIVE-14409
> URL: https://issues.apache.org/jira/browse/HIVE-14409
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Akira Ajisaka
>
> We cannot guarantee the output from external libraries (ex. HashMap), so we
> need to make the validation infrastructure to allow a sort within a single
> line.
> HIVE-13894, HIVE-13907, and HIVE-14088 are related.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
