[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment This script bootstraps an Impala dev environment on Ubuntu 14.04. It is not hermetic -- it changes some config files for the user and for the OS. It is green on Jenkins, and it runs in about 6.5 hours. The intention is to have this script run in a CI tool for post-commit testing, with the hope that this will make it easier for new developers to get a working development environment. Previously, the new developer workflow lived on wiki pages and tended to bit-rot. Still left to do: migrating the install script into the official Impala repo. Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Reviewed-on: http://gerrit.cloudera.org:8080/4674 Reviewed-by: Jim Apple Tested-by: Internal Jenkins --- A bin/bootstrap_development.sh 1 file changed, 80 insertions(+), 0 deletions(-) Approvals: Jim Apple: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Internal Jenkins has posted comments on this change. Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Jim Apple has posted comments on this change. Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. Patch Set 5: Code-Review+2 Carry +1 into +2 -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Tim Armstrong has posted comments on this change. Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. Patch Set 4: Should be pretty similar once we sort out the LD_LIBRARY_PATH nastiness. -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Jim Apple has posted comments on this change. Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. Patch Set 4: Tim, I added you in case you wanted to talk about how this might work with Ubuntu 16.04 -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Jim Apple has posted comments on this change. Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. Patch Set 4: Code-Review+1 (1 comment) Carry David's +1 http://gerrit.cloudera.org:8080/#/c/4674/3/bin/bootstrap_development.sh File bin/bootstrap_development.sh: PS3, Line 24: # The intended user is a person who wants to start contributing code to Impala. This : # script serves as an executable reference point for how to get started. > I guess I would also add, in some fashion, the following assumptions that t Done -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Hello David Knupp, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/4674 to look at the new patch set (#4). Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment This script bootstraps an Impala dev environment on Ubuntu 14.04. It is not hermetic -- it changes some config files for the user and for the OS. It is green on Jenkins, and it runs in about 6.5 hours. The intention is to have this script run in a CI tool for post-commit testing, with the hope that this will make it easier for new developers to get a working development environment. Previously, the new developer workflow lived on wiki pages and tended to bit-rot. Still left to do: migrating the install script into the official Impala repo. Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e --- A bin/bootstrap_development.sh 1 file changed, 80 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/4674/4 -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
David Knupp has posted comments on this change. Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. Patch Set 3: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/4674/3/bin/bootstrap_development.sh File bin/bootstrap_development.sh: PS3, Line 24: # The intended user is a person who wants to start contributing code to Impala. This : # script serves as an executable reference point for how to get started. I guess I would also add, in some fashion, the following assumptions that this script makes: 1. That test data and metadata will be generated, rather than copied from a snapshot or archive. 2. That a full suite of unit tests and simulated end-to-end tests will be run. -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Jim Apple has posted comments on this change. Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. Patch Set 3: (4 comments) http://gerrit.cloudera.org:8080/#/c/4674/2/bin/bootstrap_development.sh File bin/bootstrap_development.sh: Line 20: # This script bootstraps a development environment from almost nothing; it is known to > So, this first comment may be a nit, but it would be nice to clarify why th Let me know what else you think it needs. Line 31: HOMEDIR="/home/$(whoami)/" > This is really an artificial limitation/oversight of the chef scripts. Woul I think that's a later project, especially given that the chef scripts are not in this repo. PS2, Line 70: orLoca > 99 was used as an arbitrarily large number in the original one-off Jenk 0 does that here, based on https://github.com/pytest-dev/pytest/blob/8639bf75549e50a74cb49d4f48d790d2c8788a87/_pytest/main.py#L560 PS2, Line 72: " > What is the difference between passing in --skiptests and then calling run- None, I think -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Taras Bobrovytsky Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Jim Apple has uploaded a new patch set (#3). Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment This script bootstraps an Impala dev environment on Ubuntu 14.04. It is not hermetic -- it changes some config files for the user and for the OS. It is green on Jenkins, and it runs in about 6.5 hours. The intention is to have this script run in a CI tool for post-commit testing, with the hope that this will make it easier for new developers to get a working development environment. Previously, the new developer workflow lived on wiki pages and tended to bit-rot. Still left to do: migrating the install script into the official Impala repo. Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e --- A bin/bootstrap_development.sh 1 file changed, 77 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/4674/3 -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Taras Bobrovytsky
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
David Knupp has posted comments on this change.
Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
..
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/4674/2/bin/bootstrap_development.sh
File bin/bootstrap_development.sh:
Line 20: # This script bootstraps a development environment from almost
So, this first comment may be a nit, but it would be nice to clarify why this
script came to be added.
We have a various scripts that float around /bin and /testdata/bin that
sometimes seem to have overlapping functions (e.g., there are multiple entry
points for running tests), and it's not always immediately clear why they all
coexist.
This script consolidates some needed steps, such as cloning and running the
chef setup, and building Impala, but in so doing, it also makes implicit
assumptions by hard-coding in the command line args to buildall.sh for bypass
the test warehouse snaphot, and running all tests after the build process
completes.
If nothing else, it might be nice to explicitly state who the intended
users/audience for this script might be, and why these assumptions are being
made.
Line 31: echo "${HOMEDIR} is needed for installing Impala dependencies"
This is really an artificial limitation/oversight of the chef scripts. Would it
make sense to revisit those first, rather than propagating it to another script?
PS2, Line 70: 99
99 was used as an arbitrarily large number in the original one-off Jenkins
job. But maybe there's a way to truly disable the max pytest failures, e.g.,
some tools do this by specifying -1 for values like this?
PS2, Line 72: -skiptests
What is the difference between passing in --skiptests and then calling
run-all-tests.sh separately, versus simply leaving off --skiptests?
--
To view, visit http://gerrit.cloudera.org:8080/4674
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple
Gerrit-Reviewer: David Knupp
Gerrit-Reviewer: Taras Bobrovytsky
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Jim Apple has uploaded a new patch set (#2). Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment This script bootstraps an Impala dev environment on Ubuntu 14.04. It is not hermetic -- it changes some config files for the user and for the OS. It is green on Jenkins, and it runs in about 6.5 hours. The intention is to have this script run in a CI tool for post-commit testing, with the hope that this will make it easier for new developers to get a working development environment. Previously, the new developer workflow lived on wiki pages and tended to bit-rot. Still left to do: migrating the install script into the official Impala repo. Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e --- A bin/bootstrap_development.sh 1 file changed, 74 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/4674/2 -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Taras Bobrovytsky
[Impala-ASF-CR] IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment
Jim Apple has uploaded a new change for review. http://gerrit.cloudera.org:8080/4674 Change subject: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment .. IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment This script bootstraps an Impala dev environment on Ubuntu 14.04. It is not hermetic -- it changes some config files for the user and for the OS. It is green on Jenkins, and it runs in about 6.5 hours. The intention is to have this script run in a CI tool for post-commit testing, with the hope that this will make it easier for new developers to get a working development environment. Previously, the new developer workflow lived on wiki pages and tended to bit-rot. Still left to do: migrating the install script into the official Impala repo. Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e --- A bin/bootstrap_development.sh 1 file changed, 71 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/4674/1 -- To view, visit http://gerrit.cloudera.org:8080/4674 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Jim Apple
