[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged.

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..


IMPALA-5263: test infra: support CA bundles with secure clusters

This patch adds the command line option --ca_cert to the common test
infra CLI options for use alongside --use-ssl. This is useful when
testing against a secured Impala cluster in which the SSL certs are
self-signed. This will allow the SSL request to be validated. Using this
option will also suppress noisy console warnings like:

  InsecureRequestWarning: Unverified HTTPS request is being made. Adding
  certificate verification is strongly advised. See:
  https://urllib3.readthedocs.org/en/latest/security.html

We also go further in this patch and use the warnings module to print
these SSL-related warnings once and only once, instead of all over the
place. In the case of the stress test, this greatly reduces the noise in
the console log.

Testing:
- quick concurrent_select.py calls with and without --ca_cert to observe
  that connections still get made and the test runs smoothly. Some of
  this testing occurred without warning suppression, so that I could be
  sure the InsecureRequestWarnings were not occurring when using
  --ca_cert anymore.
- ensured warnings are printed once, not multiple times

Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Reviewed-on: http://gerrit.cloudera.org:8080/7152
Reviewed-by: David Knupp 
Tested-by: Impala Public Jenkins
---
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/db_connection.py
3 files changed, 56 insertions(+), 11 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  David Knupp: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Wood 

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change.

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..


Patch Set 2: Verified+1

-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Wood 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change.

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..


Patch Set 2:

Build started: http://jenkins.impala.io:8080/job/gerrit-verify-dryrun/726/

-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Wood 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Michael Brown (Code Review)]

Michael Brown has posted comments on this change.

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/7152/2/tests/comparison/cli_options.py
File tests/comparison/cli_options.py:

Line 205: 'once',
> "once"
David, I can't figure out what you're asking me to do.

1. The word "once" is spelled correctly.
2. This file tends to use single quotes.


-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Wood 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[David Knupp (Code Review)]

David Knupp has posted comments on this change.

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..


Patch Set 2: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/7152/2/tests/comparison/cli_options.py
File tests/comparison/cli_options.py:

Line 205: 'once',
"once"

Nice. I've never even heard of this library before.


-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Wood 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Michael Brown (Code Review)]

Michael Brown has posted comments on this change.

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..


Patch Set 2:

> Do you have a Jenkins run that uses this code?

There is currently nothing upstream that exercises this either before or after 
this patch.

-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Wood 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Matthew Mulder (Code Review)]

Matthew Mulder has posted comments on this change.

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..


Patch Set 2:

The code looks good to me. Do you have a Jenkins run that uses this code?

-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Wood 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Michael Brown (Code Review)]

Michael Brown has posted comments on this change.

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/7152/2/tests/comparison/cli_options.py
File tests/comparison/cli_options.py:

Line 204: filterwarnings(
> What's the Python standard for writing multi-word names?  I thought it was 
This is an import of a standard library module, so I don't have a choice.

https://docs.python.org/2/library/warnings.html


-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Wood 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Michael Brown (Code Review)]

Michael Brown has uploaded a new patch set (#2).

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..

IMPALA-5263: test infra: support CA bundles with secure clusters

This patch adds the command line option --ca_cert to the common test
infra CLI options for use alongside --use-ssl. This is useful when
testing against a secured Impala cluster in which the SSL certs are
self-signed. This will allow the SSL request to be validated. Using this
option will also suppress noisy console warnings like:

  InsecureRequestWarning: Unverified HTTPS request is being made. Adding
  certificate verification is strongly advised. See:
  https://urllib3.readthedocs.org/en/latest/security.html

We also go further in this patch and use the warnings module to print
these SSL-related warnings once and only once, instead of all over the
place. In the case of the stress test, this greatly reduces the noise in
the console log.

Testing:
- quick concurrent_select.py calls with and without --ca_cert to observe
  that connections still get made and the test runs smoothly. Some of
  this testing occurred without warning suppression, so that I could be
  sure the InsecureRequestWarnings were not occurring when using
  --ca_cert anymore.
- ensured warnings are printed once, not multiple times

Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
---
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/db_connection.py
3 files changed, 56 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/7152/2
-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown 
Gerrit-Reviewer: Matthew Mulder 
Gerrit-Reviewer: Tim Wood 

[Impala-ASF-CR] IMPALA-5263: test infra: support CA bundles with secure clusters

[Michael Brown (Code Review)]

Michael Brown has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/7152

Change subject: IMPALA-5263: test infra: support CA bundles with secure clusters
..

IMPALA-5263: test infra: support CA bundles with secure clusters

This patch adds commandline option, --ca_cert, to the common test infra
CLI options for use alongside --use-ssl. This is useful when testing
against a secured Impala cluster in which the SSL certs are self-signed.
This will allow the SSL request to be validated. Using this option will
also suppress noisy console warnings like:

  InsecureRequestWarning: Unverified HTTPS request is being made. Adding
  certificate verification is strongly advised. See:
  https://urllib3.readthedocs.org/en/latest/security.html

We also go further in this patch and use the warnings module to print
these SSL-related warnings once and only once, instead of all over the
place. In the case of the stress test, this greatly reduces the noise in
the console log.

Testing:
- quick concurrent_select.py calls with and without --ca_cert to observe
  that connections still get made and the test runs smoothly. Some of
  this testing occurred without warning suppression, so that I could be
  sure the InsecureRequestWarnings were not occurring when using
  --ca_cert anymore.
- ensured warnings are printed once, not multiple times

Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
---
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/db_connection.py
3 files changed, 55 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/7152/1
-- 
To view, visit http://gerrit.cloudera.org:8080/7152
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifb9e466e4b7cde704cdc4cf98159c068c0a400a9
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown