[Impala-ASF-CR] IMPALA-6348: Redact only sensitive fields in runtime profiles

2018-01-06 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8934 )

Change subject: IMPALA-6348: Redact only sensitive fields in runtime profiles
..


Patch Set 4: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/8934
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae3b6726009bf458a7ec73131e5d659b12ab73cf
Gerrit-Change-Number: 8934
Gerrit-PatchSet: 4
Gerrit-Owner: Bharath Vissapragada 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Sat, 06 Jan 2018 22:54:15 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6348: Redact only sensitive fields in runtime profiles

2018-01-06 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8934 )

Change subject: IMPALA-6348: Redact only sensitive fields in runtime profiles
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1683/


--
To view, visit http://gerrit.cloudera.org:8080/8934
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae3b6726009bf458a7ec73131e5d659b12ab73cf
Gerrit-Change-Number: 8934
Gerrit-PatchSet: 4
Gerrit-Owner: Bharath Vissapragada 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Sat, 06 Jan 2018 19:19:10 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6348: Redact only sensitive fields in runtime profiles

2018-01-05 Thread Sailesh Mukil (Code Review) 
Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8934 )

Change subject: IMPALA-6348: Redact only sensitive fields in runtime profiles
..


Patch Set 3: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8934/2/tests/custom_cluster/test_redaction.py
File tests/custom_cluster/test_redaction.py:

http://gerrit.cloudera.org:8080/#/c/8934/2/tests/custom_cluster/test_redaction.py@336
PS2, Line 336: self.assert_query_profile_contains(self.find_last_query_id(), 
user_profile_pattern)
> You mean L315/L317?  We ran a different query in L335, find_last_query_id()
My bad. Ignore.



--
To view, visit http://gerrit.cloudera.org:8080/8934
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae3b6726009bf458a7ec73131e5d659b12ab73cf
Gerrit-Change-Number: 8934
Gerrit-PatchSet: 3
Gerrit-Owner: Bharath Vissapragada 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Fri, 05 Jan 2018 23:55:53 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6348: Redact only sensitive fields in runtime profiles

2018-01-05 Thread Bharath Vissapragada (Code Review) 
Hello Sailesh Mukil, anujphadke,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8934

to look at the new patch set (#3).

Change subject: IMPALA-6348: Redact only sensitive fields in runtime profiles
..

IMPALA-6348: Redact only sensitive fields in runtime profiles

Without this patch, redaction is applied to every field in the
runtime profile. This approach has an undesired side effect when
Kerberos auth + email redaction is in place.

Since the redaction applies to every field, even principals
(from Connected/Delegated User fields) are redacted, as the Kerberos
principal format generally pattern matches with an email redactor
template.

This is particularly problematic for monitoring tools that consume
runtime profiles and use these fields to group the queries by user.

This patch fixes the problem by redacting only the following sensitive
fields.

- Query Statement
- Error logs (since they can contain column references etc.)
- Query Status
- Query Plan

Other fields in the runtime profile are left unredacted.

Change-Id: Iae3b6726009bf458a7ec73131e5d659b12ab73cf
---
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-server.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M tests/custom_cluster/test_redaction.py
6 files changed, 56 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/8934/3
--
To view, visit http://gerrit.cloudera.org:8080/8934
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iae3b6726009bf458a7ec73131e5d659b12ab73cf
Gerrit-Change-Number: 8934
Gerrit-PatchSet: 3
Gerrit-Owner: Bharath Vissapragada 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: anujphadke

[Impala-ASF-CR] IMPALA-6348: Redact only sensitive fields in runtime profiles

2018-01-04 Thread Sailesh Mukil (Code Review) 
Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8934 )

Change subject: IMPALA-6348: Redact only sensitive fields in runtime profiles
..


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/8934/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/8934/2//COMMIT_MSG@29
PS2, Line 29: Other fields in the runtime profile are left unredacted.
This implicitly states that we aren't allowed to log anything sensitive in any 
field other than the above 4 mentioned. Is that commented or documented 
somewhere? If not, it would be good to add that.


http://gerrit.cloudera.org:8080/#/c/8934/2/tests/custom_cluster/test_redaction.py
File tests/custom_cluster/test_redaction.py:

http://gerrit.cloudera.org:8080/#/c/8934/2/tests/custom_cluster/test_redaction.py@336
PS2, Line 336: self.assert_query_profile_contains(self.find_last_query_id(), 
user_profile_pattern)
Should we do this again? It's already done on L313?


http://gerrit.cloudera.org:8080/#/c/8934/2/tests/custom_cluster/test_redaction.py@337
PS2, Line 337: assert_query_profile_contains
Why not use assert_web_ui_redaction() here?



--
To view, visit http://gerrit.cloudera.org:8080/8934
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae3b6726009bf458a7ec73131e5d659b12ab73cf
Gerrit-Change-Number: 8934
Gerrit-PatchSet: 2
Gerrit-Owner: Bharath Vissapragada 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Thu, 04 Jan 2018 21:47:21 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6348: Redact only sensitive fields in runtime profiles

2018-01-03 Thread Bharath Vissapragada (Code Review) 
Bharath Vissapragada has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8934


Change subject: IMPALA-6348: Redact only sensitive fields in runtime profiles
..

IMPALA-6348: Redact only sensitive fields in runtime profiles

Without this patch, redaction is applied to every field in the
runtime profile. This approach has an undesired side effect when
Kerberos auth + email redaction is in place.

Since the redaction applies to every field, even principals
(from Connected/Delegated User fields) are redacted, as the Kerberos
principal format generally pattern matches with an email redactor
template.

This is particularly problematic for monitoring tools that consume
runtime profiles and use these fields to group the queries by user.

This patch fixes the problem by redacting only the following sensitive
fields.

- Query Statement
- Error logs (since they can contain column references etc.)
- Query Status
- Query Plan

Other fields in the runtime profile are left unredacted.

Change-Id: Iae3b6726009bf458a7ec73131e5d659b12ab73cf
---
M be/src/service/client-request-state.cc
M be/src/service/impala-server.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M tests/custom_cluster/test_redaction.py
5 files changed, 47 insertions(+), 12 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/8934/2
--
To view, visit http://gerrit.cloudera.org:8080/8934
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iae3b6726009bf458a7ec73131e5d659b12ab73cf
Gerrit-Change-Number: 8934
Gerrit-PatchSet: 2
Gerrit-Owner: Bharath Vissapragada