[kudu-CR] [dist-test] build environment for tests in loop mode
Hello Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11905 to look at the new patch set (#2). Change subject: [dist-test] build environment for tests in loop mode .. [dist-test] build environment for tests in loop mode This patch fixes the issue of not deploying corresponding DATA_FILES by dest-test when running a test via the loop sub-command. I used the following command to run the ts_location_assignment-itest in dist-test: KUDU_ALLOW_SLOW_TESTS=1 \ ../../build-support/dist_test.py loop -n 32 \ ./bin/ts_location_assignment-itest --stress_cpu_threads=16 Prior to this patch, all runs failed because of the absence of assign-location.py script at dist-test slaves: http://dist-test.cloudera.org/job?job_id=aserbin.1541627808.8037 With this patch, all runs succeeded: http://dist-test.cloudera.org/job?job_id=aserbin.1541627254.115588 Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d --- M build-support/dist_test.py 1 file changed, 47 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/05/11905/2 -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin
[kudu-CR] [dist-test] build environment for tests in loop mode
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 08 Nov 2018 20:10:01 + Gerrit-HasComments: No
[kudu-CR] [dist-test] build environment for tests in loop mode
Andrew Wong has posted comments on this change. (
http://gerrit.cloudera.org:8080/11905 )
Change subject: [dist-test] build environment for tests in loop mode
..
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py
File build-support/dist_test.py:
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py@534
PS3, Line 534: cular subset of tests provided by the binary. In that case it
> I'll punt on this because from paranoid's point of view that looks fragile:
Ack
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py@537
PS3, Line 537: yield noise. To avoid this, we disable sharding in this case.
: filter_flags = [e for e in options.args
: if e.find("--gtest_filter")
> It's necessary to get just one out of many if it's necessary to avoid shard
Thanks for the comment!
--
To view, visit http://gerrit.cloudera.org:8080/11905
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d
Gerrit-Change-Number: 11905
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 08 Nov 2018 20:12:48 +
Gerrit-HasComments: Yes
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. [dist-test] build environment for tests in loop mode This patch fixes the issue of not deploying corresponding DATA_FILES by dist-test when running a test via the loop sub-command. I used the following command to run the ts_location_assignment-itest in dist-test: KUDU_ALLOW_SLOW_TESTS=1 \ ../../build-support/dist_test.py loop -n 32 \ ./bin/ts_location_assignment-itest --stress_cpu_threads=16 Prior to this patch, all runs failed because of the absence of assign-location.py script at dist-test slaves: http://dist-test.cloudera.org/job?job_id=aserbin.1541627808.8037 With this patch, all runs succeeded: http://dist-test.cloudera.org/job?job_id=aserbin.1541627254.115588 Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Reviewed-on: http://gerrit.cloudera.org:8080/11905 Reviewed-by: Adar Dembo Tested-by: Alexey Serbin --- M build-support/dist_test.py 1 file changed, 54 insertions(+), 20 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Alexey Serbin: Verified -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Removed reviewer Kudu Jenkins with the following votes: * Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteReviewer Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 4: Verified+1 Unrelated flake in org.apache.kudu.spark.kudu.DefaultSourceTest -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Comment-Date: Thu, 08 Nov 2018 20:41:17 + Gerrit-HasComments: No
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/11905 )
Change subject: [dist-test] build environment for tests in loop mode
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/11905/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11905/3//COMMIT_MSG@10
PS3, Line 10: dest
> nit: dist
Done
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py
File build-support/dist_test.py:
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py@525
PS3, Line 525: tests_regex = "^" + os.path.basename(options.cmd) +
"(\.[0-9]+)?$"
> nit: mind describing what this matches?
Done
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py@534
PS3, Line 534: e.find("--gtest_filter") == 0 or e.find("--gtest-filter") == 0
> nit: I think you can do:
I'll punt on this because from paranoid's point of view that looks fragile: I
don't want that to react on --gtest_filter to be elsewhere but in the beginning
of the string. Basically, I don't that to trigger for flags like
'-x=--gtest_filter=*Turbo*' or alike.
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py@537
PS3, Line 537: e = executions[0]
: e.env["GTEST_TOTAL_SHARDS"] = 1
: e.env["GTEST_SHARD_INDEX"] = 0
> What's special about the first execution?
It's necessary to get just one out of many if it's necessary to avoid sharding.
It would be possible to take any, but it's necessary to override shard-related
environment variables.
I'll add corresponding comment.
--
To view, visit http://gerrit.cloudera.org:8080/11905
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d
Gerrit-Change-Number: 11905
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 08 Nov 2018 19:38:05 +
Gerrit-HasComments: Yes
[kudu-CR] [dist-test] build environment for tests in loop mode
Hello Andrew Wong, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11905 to look at the new patch set (#4). Change subject: [dist-test] build environment for tests in loop mode .. [dist-test] build environment for tests in loop mode This patch fixes the issue of not deploying corresponding DATA_FILES by dist-test when running a test via the loop sub-command. I used the following command to run the ts_location_assignment-itest in dist-test: KUDU_ALLOW_SLOW_TESTS=1 \ ../../build-support/dist_test.py loop -n 32 \ ./bin/ts_location_assignment-itest --stress_cpu_threads=16 Prior to this patch, all runs failed because of the absence of assign-location.py script at dist-test slaves: http://dist-test.cloudera.org/job?job_id=aserbin.1541627808.8037 With this patch, all runs succeeded: http://dist-test.cloudera.org/job?job_id=aserbin.1541627254.115588 Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d --- M build-support/dist_test.py 1 file changed, 54 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/05/11905/4 -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong
[kudu-CR] [dist-test] build environment for tests in loop mode
Andrew Wong has posted comments on this change. (
http://gerrit.cloudera.org:8080/11905 )
Change subject: [dist-test] build environment for tests in loop mode
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/11905/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11905/3//COMMIT_MSG@10
PS3, Line 10: dest
nit: dist
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py
File build-support/dist_test.py:
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py@525
PS3, Line 525: tests_regex = "^" + os.path.basename(options.cmd) +
"(\.[0-9]+)?$"
nit: mind describing what this matches?
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py@534
PS3, Line 534: e.find("--gtest_filter") == 0 or e.find("--gtest-filter") == 0
nit: I think you can do:
[e for e in options.args if "--gtest_filter" in e or "--gtest-filter" in e]
http://gerrit.cloudera.org:8080/#/c/11905/3/build-support/dist_test.py@537
PS3, Line 537: e = executions[0]
: e.env["GTEST_TOTAL_SHARDS"] = 1
: e.env["GTEST_SHARD_INDEX"] = 0
What's special about the first execution?
--
To view, visit http://gerrit.cloudera.org:8080/11905
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d
Gerrit-Change-Number: 11905
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Comment-Date: Thu, 08 Nov 2018 18:21:44 +
Gerrit-HasComments: Yes
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 3: Verified+1 Unrelated flake in org.apache.kudu.spark.kudu.DefaultSourceTest -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Comment-Date: Thu, 08 Nov 2018 17:55:18 + Gerrit-HasComments: No
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Removed reviewer Kudu Jenkins with the following votes: * Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteReviewer Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11905/2/build-support/dist_test.py File build-support/dist_test.py: http://gerrit.cloudera.org:8080/#/c/11905/2/build-support/dist_test.py@528 PS2, Line 528: # The presence of the --gtest_filter flag means running some particular test : # scenarios provided by the binary. In that case it doesn't make much sense : # to shard the execution since only the shards containing the matching tests : # scenarios would produce any meaningful results. Also, it's not trivial : # to point which particular scenario is run by particular shard anyway. : # So, simply disable sharding in that case. > Nit: reword? Maybe "The presence of the --gtest_filter flag means the user Done, thanks! -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 08 Nov 2018 16:59:21 + Gerrit-HasComments: Yes
[kudu-CR] [dist-test] build environment for tests in loop mode
Hello Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11905 to look at the new patch set (#3). Change subject: [dist-test] build environment for tests in loop mode .. [dist-test] build environment for tests in loop mode This patch fixes the issue of not deploying corresponding DATA_FILES by dest-test when running a test via the loop sub-command. I used the following command to run the ts_location_assignment-itest in dist-test: KUDU_ALLOW_SLOW_TESTS=1 \ ../../build-support/dist_test.py loop -n 32 \ ./bin/ts_location_assignment-itest --stress_cpu_threads=16 Prior to this patch, all runs failed because of the absence of assign-location.py script at dist-test slaves: http://dist-test.cloudera.org/job?job_id=aserbin.1541627808.8037 With this patch, all runs succeeded: http://dist-test.cloudera.org/job?job_id=aserbin.1541627254.115588 Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d --- M build-support/dist_test.py 1 file changed, 46 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/05/11905/3 -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [dist-test] build environment for tests in loop mode
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11905/2/build-support/dist_test.py File build-support/dist_test.py: http://gerrit.cloudera.org:8080/#/c/11905/2/build-support/dist_test.py@528 PS2, Line 528: # The presence of the --gtest_filter flag means running some particular test : # scenarios provided by the binary. In that case it doesn't make much sense : # to shard the execution since only the shards containing the matching tests : # scenarios would produce any meaningful results. Also, it's not trivial : # to point which particular scenario is run by particular shard anyway. : # So, simply disable sharding in that case. Nit: reword? Maybe "The presence of the --gtest_filter flag means the user is interested in a particular subset of tests provided by the binary. In that case it doesn't make sense to shard the execution since only the shards containing the matching tests would produce interesting results, while the rest would yield noise. To avoid this, we disable sharding in this case." -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 08 Nov 2018 08:32:27 + Gerrit-HasComments: Yes
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py File build-support/dist_test.py: http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py@167 PS1, Line 167: Return an array of TestExecution objects. > Please doc tests_regex and extra_args. Done http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py@170 PS1, Line 170: ctest_argv = [ctest_bin, "-V", "-N", "-LE", "no_dist_test"] > Yep, I'll take a look. http://gerrit.cloudera.org:8080/11907 http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py@517 PS1, Line 517: # Build the list of executions corresponding to the test. It's better to do > Yes, this will automatically shard the test. Isn't it good? In offline discussion Todd suggested to use the presence of the --gtest_filter option to decide whether to shard or not. -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 08 Nov 2018 02:15:56 + Gerrit-HasComments: Yes
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Removed reviewer Kudu Jenkins with the following votes: * Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteReviewer Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 1: Verified+1 Unrelated flake in org.apache.kudu.spark.kudu.DefaultSourceTest -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Comment-Date: Wed, 07 Nov 2018 23:22:14 + Gerrit-HasComments: No
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py File build-support/dist_test.py: http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py@170 PS1, Line 170: ctest_argv = [ctest_bin, "-V", "-N", "-LE", "no_dist_test"] > I ran this command and was surprised to see that we emit KUDU_DATA_FILES in Yep, I'll take a look. http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py@517 PS1, Line 517: # Build the list of executions corresponding to the test. It's better to do > Won't this also shard the test? Seems like the epilog for 'loop' would need Yes, this will automatically shard the test. Isn't it good? I'll take a look at the review item, sure. However, I can report that the 'loop build/latest/bin/foo-test' isn't broken (you can check the commit message for some evidence). Instead, it runs much faster now if the test has shards defined. -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 07 Nov 2018 23:17:09 + Gerrit-HasComments: Yes
[kudu-CR] [dist-test] build environment for tests in loop mode
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11905 ) Change subject: [dist-test] build environment for tests in loop mode .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py File build-support/dist_test.py: http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py@167 PS1, Line 167: Return an array of TestExecution objects. Please doc tests_regex and extra_args. http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py@170 PS1, Line 170: ctest_argv = [ctest_bin, "-V", "-N", "-LE", "no_dist_test"] I ran this command and was surprised to see that we emit KUDU_DATA_FILES into the test environment even when there's no DATA_FILES entries. That appears to be due to the unconditional use of DATA_FILES_LIST in the top-level CMakeLists.txt. Could you fix that? http://gerrit.cloudera.org:8080/#/c/11905/1/build-support/dist_test.py@517 PS1, Line 517: # Build the list of executions corresponding to the test. It's better to do Won't this also shard the test? Seems like the epilog for 'loop' would need to be updated, at a minimum. You should read https://gerrit.cloudera.org/c/9470/2/build-support/dist_test.py#442 before proceeding. I'm worried this change might break the "loop build/latest/bin/foo-test" workflow; does it? -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 07 Nov 2018 22:50:22 + Gerrit-HasComments: Yes
[kudu-CR] [dist-test] build environment for tests in loop mode
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11905 Change subject: [dist-test] build environment for tests in loop mode .. [dist-test] build environment for tests in loop mode This patch fixes the issue of not deploying corresponding DATA_FILES by dest-test when running a test via the loop sub-command. I used the following command to run the ts_location_assignment-itest in dist-test: KUDU_ALLOW_SLOW_TESTS=1 \ ../../build-support/dist_test.py loop -n 32 \ ./bin/ts_location_assignment-itest --stress_cpu_threads=16 Prior to this patch, all runs failed because of the absence of assign-location.py script at dist-test slaves: http://dist-test.cloudera.org/job?job_id=aserbin.1541627808.8037 With this patch, all runs succeeded: http://dist-test.cloudera.org/job?job_id=aserbin.1541627254.115588 Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d --- M build-support/dist_test.py 1 file changed, 16 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/05/11905/1 -- To view, visit http://gerrit.cloudera.org:8080/11905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I487194a396635bbaab457795bb24c0063eebbe5d Gerrit-Change-Number: 11905 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
