[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677586#comment-13677586 ] Hudson commented on HIVE-2670: -- Integrated in Hive-trunk-h0.21 #2131 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2131/]) HIVE-4657 : HCatalog checkstyle violation after HIVE-2670 (Shreepadma Venugopalan via Ashutosh Chauhan) (Revision 1490106) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1490106 Files : * /hive/trunk/hcatalog/src/test/e2e/hcatalog/drivers/TestDriverHiveCmdLine.pm * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tests/hive_cmdline.conf * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tests/hive_nightly.conf A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Assignee: Johnny Zhang Fix For: 0.12.0 Attachments: harness.tar, HIVE-2670_5.patch, HIVE-2670_6.patch, hive_cluster_test_2.patch, hive_cluster_test_3.patch, hive_cluster_test_4.patch, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13675655#comment-13675655 ] Hudson commented on HIVE-2670: -- Integrated in Hive-trunk-h0.21 #2128 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2128/]) HIVE-2670 A cluster test utility for Hive (gates and Johnny Zhang via gates) (Revision 1489376) Result = FAILURE gates : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1489376 Files : * /hive/trunk/hcatalog/src/test/e2e/hcatalog/build.xml * /hive/trunk/hcatalog/src/test/e2e/hcatalog/drivers/TestDriverHiveCmdLine.pm * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tests/hive_cmdline.conf * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tests/hive_nightly.conf * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tools/test/floatpostprocessor.pl A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Assignee: Johnny Zhang Fix For: 0.12.0 Attachments: harness.tar, HIVE-2670_5.patch, HIVE-2670_6.patch, hive_cluster_test_2.patch, hive_cluster_test_3.patch, hive_cluster_test_4.patch, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13676632#comment-13676632 ] Hudson commented on HIVE-2670: -- Integrated in Hive-trunk-hadoop2 #227 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/227/]) HIVE-4657 : HCatalog checkstyle violation after HIVE-2670 (Shreepadma Venugopalan via Ashutosh Chauhan) (Revision 1490106) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1490106 Files : * /hive/trunk/hcatalog/src/test/e2e/hcatalog/drivers/TestDriverHiveCmdLine.pm * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tests/hive_cmdline.conf * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tests/hive_nightly.conf A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Assignee: Johnny Zhang Fix For: 0.12.0 Attachments: harness.tar, HIVE-2670_5.patch, HIVE-2670_6.patch, hive_cluster_test_2.patch, hive_cluster_test_3.patch, hive_cluster_test_4.patch, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13675549#comment-13675549 ] Hudson commented on HIVE-2670: -- Integrated in Hive-trunk-hadoop2 #225 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/225/]) HIVE-2670 A cluster test utility for Hive (gates and Johnny Zhang via gates) (Revision 1489376) Result = ABORTED gates : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1489376 Files : * /hive/trunk/hcatalog/src/test/e2e/hcatalog/build.xml * /hive/trunk/hcatalog/src/test/e2e/hcatalog/drivers/TestDriverHiveCmdLine.pm * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tests/hive_cmdline.conf * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tests/hive_nightly.conf * /hive/trunk/hcatalog/src/test/e2e/hcatalog/tools/test/floatpostprocessor.pl A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Assignee: Johnny Zhang Fix For: 0.12.0 Attachments: harness.tar, HIVE-2670_5.patch, HIVE-2670_6.patch, hive_cluster_test_2.patch, hive_cluster_test_3.patch, hive_cluster_test_4.patch, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672204#comment-13672204 ] Daniel Dai commented on HIVE-2670: -- Checkin_2 fail on my Mac. Change hive_nightly.conf:41, from: 'sortArgs' = ['-t', ' ', '+1', '-2'], to 'sortArgs' = ['-t', '', '-k', '2,2n'], Solve the problem. Otherwise +1. A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Assignee: Johnny Zhang Attachments: harness.tar, HIVE-2670_5.patch, hive_cluster_test_2.patch, hive_cluster_test_3.patch, hive_cluster_test_4.patch, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664390#comment-13664390 ] Johnny Zhang commented on HIVE-2670: Alan, Thanks for the update! A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Assignee: Johnny Zhang Attachments: harness.tar, HIVE-2670_5.patch, hive_cluster_test_2.patch, hive_cluster_test_3.patch, hive_cluster_test_4.patch, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13562937#comment-13562937 ] Johnny Zhang commented on HIVE-2670: sorry for multiple comments, 'hive_cluster_test_4.patch' also Connect the top level build.xml to the build.xml in e2e dir. So, right now the step to run this patch is (1) checkout Hive trunk, apply patch (2) download harness.tar to e2e dir (3) build Hive by 'ant clean package', copy build/dist/lib/*.jar to lib dir (4) run the e2e/scripts/create_test_db.sql in MySQL (5) run 'ant -Dharness.hadoop.home=path_to_hadoop_home -Dharness.hive.home=path_to_hive_you_want_to_test test-e2e-deploy' in Hive top level dir (6) run 'ant -Dharness.hadoop.home=path_to_hadoop_home -Dharness.hive.home=path_to_hive_you_want_to_test test-e2e-test' in Hive top level dir A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Assignee: Johnny Zhang Attachments: harness.tar, hive_cluster_test_2.patch, hive_cluster_test_3.patch, hive_cluster_test_4.patch, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536612#comment-13536612 ] Alan Gates commented on HIVE-2670: -- I think it would actually be ok to comment out those two tests. I don't think they're really the right way to test what I was trying to test, and they cause issues for everyone else. A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Assignee: Johnny Zhang Attachments: harness.tar, hive_cluster_test_2.patch, hive_cluster_test_3.patch, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[
https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13528601#comment-13528601
]
Johnny Zhang commented on HIVE-2670:
[~alangates], regarding two HCat test cases, 'HCat_sudo'. If I start the hive
metastore by
{noformat}
$hivehome/bin/hive --service metastore
{noformat}
while running the tests, it pass. Is this expected? I think so by reading your
comments .To run these two new tests it works better if you set up a Hive
server and... If this is the case, then hive_cluster_test_3.patch is
good. We only need to decide whether test-e2e is a good place to put e2e
tests.
Otherwise need to change the hardness to start metastore before running
HCat_sudo1 and stop it after HCat_sudo2. Do you want it that way? Thanks.
A cluster test utility for Hive
---
Key: HIVE-2670
URL: https://issues.apache.org/jira/browse/HIVE-2670
Project: Hive
Issue Type: New Feature
Components: Testing Infrastructure
Reporter: Alan Gates
Assignee: Johnny Zhang
Attachments: harness.tar, hive_cluster_test_2.patch,
hive_cluster_test_3.patch, hive_cluster_test.patch
Hive has an extensive set of unit tests, but it does not have an
infrastructure for testing in a cluster environment. Pig and HCatalog have
been using a test harness for cluster testing for some time. We have written
Hive drivers and tests to run in this harness.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[
https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13497551#comment-13497551
]
Johnny Zhang commented on HIVE-2670:
{noformat}
Final results , PASSED: 101 FAILED: 7 SKIPPED: 0 ABORTED: 1 FAILED DEPENDENCY: 0
{noformat}
I looked the 7 failures:
(1) the 5 FAILED case seems because hive and mysql has slight different float
number calculation result, for example, hive get -10058.09 while mysql get
-10058.08,
(2) another 2 FAILED case are because of hcatlog related test case (seems need
start hive-metastore process to make it work)
still looking how to fix it
A cluster test utility for Hive
---
Key: HIVE-2670
URL: https://issues.apache.org/jira/browse/HIVE-2670
Project: Hive
Issue Type: New Feature
Components: Testing Infrastructure
Reporter: Alan Gates
Assignee: Johnny Zhang
Attachments: harness.tar, hive_cluster_test_2.patch,
hive_cluster_test.patch
Hive has an extensive set of unit tests, but it does not have an
infrastructure for testing in a cluster environment. Pig and HCatalog have
been using a test harness for cluster testing for some time. We have written
Hive drivers and tests to run in this harness.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[
https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481899#comment-13481899
]
Johnny Zhang commented on HIVE-2670:
I just apply the patch and successfully deploy the data and kick off a fully
run. Will update more when the test is complete.
{noformat}
deploy-base:
[exec]
[exec] LOGGING RESULTS TO
/root/hive/test-e2e/testdist/./out/log/test_harnesss_1350946336
[exec]
[exec] Generating data for studenttab10k
[exec] Loading data into Hive for studenttab10k
[exec] Loading data into MySQL for studenttab10k
[exec] Generating data for votertab10k
[exec] Loading data into Hive for votertab10k
[exec] Loading data into MySQL for votertab10k
[exec] Generating data for studentparttab30k
[exec] Loading data into Hive for studentparttab30k
[exec] Loading data into MySQL for studentparttab30k
[exec] Generating data for studentnull10k
[exec] Loading data into Hive for studentnull10k
[exec] Loading data into MySQL for studentnull10k
[exec] Generating data for all100k
[exec] Loading data into Hive for all100k
[exec] Loading data into MySQL for all100k
[exec] Final results , PASSED: 0 FAILED: 0 SKIPPED: 0 ABORTED: 0 FAILED
DEPENDENCY: 0
BUILD SUCCESSFUL
..
test-base:
[exec]
[exec] LOGGING RESULTS TO
/root/hive/test-e2e/testdist/./out/log/test_harnesss_1350946471
[exec]
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 0 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 1 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 2 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 3 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 4 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 5 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 6 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 7 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 8 FAILED
DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 0 SKIPPED: 0 ABORTED: 9 FAILED
DEPENDENCY: 0
{noformat}
A cluster test utility for Hive
---
Key: HIVE-2670
URL: https://issues.apache.org/jira/browse/HIVE-2670
Project: Hive
Issue Type: New Feature
Components: Testing Infrastructure
Reporter: Alan Gates
Assignee: Zhenxiao Luo
Attachments: harness.tar, hive_cluster_test_2.patch,
hive_cluster_test.patch
Hive has an extensive set of unit tests, but it does not have an
infrastructure for testing in a cluster environment. Pig and HCatalog have
been using a test harness for cluster testing for some time. We have written
Hive drivers and tests to run in this harness.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279174#comment-13279174 ] Zhenxiao Luo commented on HIVE-2670: @Alan 1. Step#4 and Step#5 are the same, is there anything special to setup? 2. Is there a restore script used to restore mysql's initial state. Otherwise, if there is anything wrong running the patch, always have to restore manually 3. Any hints on setting permissions in the mysql script? Always get into problems when loading data using the mysql script. A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Attachments: harness.tar, hive_cluster_test.patch, hive_cluster_test_2.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279190#comment-13279190 ] Zhenxiao Luo commented on HIVE-2670: In testdist/studenttab10k.mysql.sql: load data infile '' into table studenttab10k ... should be updated to: load data local infile '' into table studenttab10k ... mysql errors out with the first syntax, and setting permissions could not fix it. A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Attachments: harness.tar, hive_cluster_test.patch, hive_cluster_test_2.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268595#comment-13268595 ] Namit Jain commented on HIVE-2670: -- https://cwiki.apache.org/confluence/display/Hive/End2EndTests A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Attachments: harness.tar, hive_cluster_test.patch, hive_cluster_test_2.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2670) A cluster test utility for Hive
[ https://issues.apache.org/jira/browse/HIVE-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13174292#comment-13174292 ] Alan Gates commented on HIVE-2670: -- Attached a first patch. This is not ready for inclusion yet, I'm just putting it up here to start getting feedback. The following will need to be resolved before it is checked in: # Currently it just has the base harness code included as a tar file. This really should be externed from the Pig code base, as HCatalog does. # I don't know if this is the right place in SVN or not. I put it all in a test-e2e directory right under trunk. I need feedback on whether this is a good spot or somewhere else would be preferred. # Connect the top level build.xml to this so it is possible to invoke the tests from the top level directory. I was waiting to do this until I had feedback on the proper directory structure. How to use it: After applying the patch you will need to copy the harness.tar file (attached) to test-e2e, since that is not done for you by the patch tool. First you need an existing Hadoop cluster (it can be very small, just a few nodes) and a MySQL database. I ran my tests against Hadoop 0.20.205.0, but this should run against any 0.20.x version of Hadoop. Then: # Run the script test-e2e/scripts/create_test_db.sql against your MySQL database as a user that can create users and databases, and grant to users (root is a good choice) # Run ant package in the top level Hive directory # cd test-e2e # ant -Dharness.hadoop.home=path_to_hadoop_home -Dharness.hive.home=path_to_hive_you_want_to_test deploy # ant -Dharness.hadoop.home=path_to_hadoop_home -Dharness.hive.home=path_to_hive_you_want_to_test deploy Usually path_to_hive_you_want_to_test will be $CWD/../build/dist The basic design of this test harness is each test consists of three phases: run_test, generate_benchmark, and compare_results. In run_test a particular test is run. generate_benchmark runs the same or a similar test against a known source of truth. compare_results then compares the results and declares the test to have succeeded, failed, or aborted. The harness delegates each of these three functions to drivers that are specific to different types of tests. This patch includes two drivers, a Hive driver and a Hive command line driver. The Hive driver uses the MySQL database as a source of truth. Each SQL script is run against Hive and against MySQL and the results compared using the Unix cksum tool. For more information on the test harness, including how to add tests to it, see https://cwiki.apache.org/confluence/display/PIG/HowToTest The Hive driver does not yet support running alternate SQL for benchmarking nor using an old version of Hive for the benchmarks, though those should be added sometime. A cluster test utility for Hive --- Key: HIVE-2670 URL: https://issues.apache.org/jira/browse/HIVE-2670 Project: Hive Issue Type: New Feature Components: Testing Infrastructure Reporter: Alan Gates Attachments: harness.tar, hive_cluster_test.patch Hive has an extensive set of unit tests, but it does not have an infrastructure for testing in a cluster environment. Pig and HCatalog have been using a test harness for cluster testing for some time. We have written Hive drivers and tests to run in this harness. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
