[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410341#comment-15410341
]
Ashutosh Chauhan commented on HIVE-14415:
-
Alright. We can check in this patch as well since it improves atleast one cli
driver (qfile broken temporarily for perflcidriver is ok), but if you are close
to make it work for few others, we can wait as well. Upto you, let me know.
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410335#comment-15410335
]
Zoltan Haindrich commented on HIVE-14415:
-
oh..i'm a bit tired...in this patch it only doesn't work on testperfclidriver ;)
i'm working on the other ones - and i've forgot to pass qfile variable from
maven to the java process.
I was not sure when I started working on this that the ptest executors will
welcome this change or not...the patch proved that it will cause no problems.
Anyway...I think I will drop this ticket and add these changes to the other
one...at least half of these changes have became outdated.
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410321#comment-15410321
]
Ashutosh Chauhan commented on HIVE-14415:
-
Didn't follow you. How is it broken and why ptests failed since they do use it?
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410319#comment-15410319
]
Zoltan Haindrich commented on HIVE-14415:
-
[~ashutoshc] i think this patch has a major issue: i'm afraid {{-Dqfile}} will
be broken with it...i think it would be best to add all these changes too to
HIVE-1
the ptest executor had no problem with this change...so: I think it will have
no problems with a more drastic change either.
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Sub-task
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409916#comment-15409916
]
Zoltan Haindrich commented on HIVE-14415:
-
[~pvary] I'm not actively working on those other things...i've experimented
with cli execution from eclipse about a month ago..and those are some minor
issues I identified..
wow!...i wasn't aware that beeline tests are also disabled...there are about 20
cli tests...testaccumuloclidriver is also disabled - but i was unable to
determine how HIVE-14391 ;)
I think an umbrella jira ticket would be usefuli've created HIVE-14443
i've started working on migrating the other clidrivers one-by-one...I haven't
reached beeline yet...so just submit it as a new ticket; or if you send me a
pastebin link i'll add it to the second patch
cheers! :)
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409678#comment-15409678
]
Ashutosh Chauhan commented on HIVE-14415:
-
+1
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409572#comment-15409572
]
Peter Vary commented on HIVE-14415:
---
I have a patch which uses your one, and contains a TestBeeLineCliDriver -
tested only for one specific query test, but seems ok.
Of course further work is required on it to clean up, and make it more
effective, but at least the former vm based test is ported.
I do not think anyone is checked these tests any more since it was removed from
the automatic run with this jira: HIVE-4162
Do not know how to share, since I am not able to attach a new file to this jira.
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409340#comment-15409340
]
Peter Vary commented on HIVE-14415:
---
I have created a clean copy of hive, and was able to run the ptests from
Intellij without issues.
So most probably the previous errors caused by some problem between the chair
and the keyboard. :)
I do not want to duplicate work, and [~kgyrtkirk] is probably working the
issues with the itests mentioned above, so in the meantime I will take a look
at the beeline test stuff.
[~kgyrtkirk] If you have anything specific where I could help, just tell.
Thanks,
Peter
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408740#comment-15408740
]
Hive QA commented on HIVE-14415:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822174/HIVE-14415.2.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10439 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/778/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/778/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-778/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822174 - PreCommit-HIVE-MASTER-Build
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408651#comment-15408651
]
Zoltan Haindrich commented on HIVE-14415:
-
[~pvary] thank you for the review! i was not aware of HIVE-12033...
after these changes - the execution of these tests in ide won't be readily
available but will be a step closer...i've seen a few things which will have to
be solved prior to getting a working solution...like:
* qtests sometime contain some exact pathreference to datafiles...somewhere
they are using properties...i'm not sure those will work or not (in case of
-Pitests)
* there a few maven repo references here and there - not sure how this can be
avoided
* there is a "poisoning" core-site.xml in the common module...but generally the
ide shows all the resources to the test...which may get confused by the union
of all hive main resources
these are just what i know of...if we team up, we can get a fully working ide
faster ;)
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch, HIVE-14415.2.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14415) Upgrade qtest execution framework to junit4 - TestPerfCliDriver
[
https://issues.apache.org/jira/browse/HIVE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407408#comment-15407408
]
Peter Vary commented on HIVE-14415:
---
[~kgyrtkirk] You were a tad faster, than me on this problem. In this morning I
was about to take one of the jira (HIVE-12033) [~spena] left, which is about
the intellij debugging issue too. Good to see more of us working on the testing
framework. I will try to look at your patch today, or later tomorrow. Although
I am not a committer, but one more eyes always help
> Upgrade qtest execution framework to junit4 - TestPerfCliDriver
> ---
>
> Key: HIVE-14415
> URL: https://issues.apache.org/jira/browse/HIVE-14415
> Project: Hive
> Issue Type: Improvement
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-14415.1.patch
>
>
> I would like to upgrade the current maven+ant+velocimacro+junit4 qtest
> generation framework to use only junit4 - while (trying) to keep
> all the existing features it provides.
> What I can't really do with the current one: execute easily a single qtests
> from an IDE (as a matter of fact I can...but it's way too complicated; after
> this it won't be a cake-walk either...but it will be a step closer ;)
> I think this change will make it more clear how these tests are configured
> and executed.
> I will do this in two phases, currently i will only change
> {{TestPerfCliDriver}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
