[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-20 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..

IMPALA-6225: Part 2: Query profile date-time strings should have ns
precision.

This commit follows 16d8dd58.

This patch adds a test case that inspects the thrift profile of a
completed query, and verifies that the "Start Time" and
"End Time" of the query have nanosecond precision. We chose to
work with the thrift profile directly, rather than parse the debug
web page, as it is the thrift profile which is consumed by
management API clients of Impala.

Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Reviewed-on: http://gerrit.cloudera.org:8080/8784
Reviewed-by: Michael Ho 
Tested-by: Impala Public Jenkins
---
M tests/common/impala_service.py
M tests/query_test/test_observability.py
2 files changed, 78 insertions(+), 1 deletion(-)

Approvals:
  Michael Ho: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 8
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-20 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 7: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 7
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 21 Dec 2017 04:26:32 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-20 Thread Michael Ho (Code Review) 
Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 7: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 7
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 21 Dec 2017 00:49:03 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-20 Thread Zoram Thanga (Code Review) 
Hello Michael Ho, Philip Zeyliger,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8784

to look at the new patch set (#7).

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..

IMPALA-6225: Part 2: Query profile date-time strings should have ns
precision.

This commit follows 16d8dd58.

This patch adds a test case that inspects the thrift profile of a
completed query, and verifies that the "Start Time" and
"End Time" of the query have nanosecond precision. We chose to
work with the thrift profile directly, rather than parse the debug
web page, as it is the thrift profile which is consumed by
management API clients of Impala.

Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
---
M tests/common/impala_service.py
M tests/query_test/test_observability.py
2 files changed, 78 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/8784/7
--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 7
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-20 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 6:

(1 comment)

Thanks for the comments. Please see #7

http://gerrit.cloudera.org:8080/#/c/8784/6/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/6/tests/query_test/test_observability.py@156
PS6, Line 156: tree.validate()
> Just for my own knowledge: where is validate() defined ?
This is defined in shell/gen-py/RuntimeProfile/ttypes.py

If validate() fails it raises a TProtocolException. The validation is very 
minimal, actually. Will move it to get_thrift_profile().



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 6
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 21 Dec 2017 00:08:24 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-20 Thread Philip Zeyliger (Code Review) 
Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8784/6/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/6/tests/query_test/test_observability.py@156
PS6, Line 156: tree.validate()
> Just for my own knowledge: where is validate() defined ?
validate() is defined in the generated code.

https://github.com/apache/thrift/blob/041c3c777db9639b0a9195bc6aa1f935501fd506/compiler/cpp/src/thrift/generate/t_py_generator.cc#L1059
 is the generator

Here's an example of generated code:

  def validate(self):
if self.principalGrants is None:
  raise TProtocol.TProtocolException(message='Required field 
principalGrants is unset!')
return


Note that most validate() implementations are empty; I think they only get 
generated for required fields.

I think you can reasonably move it to get_thrift_profile().



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 6
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Wed, 20 Dec 2017 23:49:18 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-20 Thread Michael Ho (Code Review) 
Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8784/6/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/6/tests/query_test/test_observability.py@156
PS6, Line 156: tree.validate()
Just for my own knowledge: where is validate() defined ?

Also, what would happen if validation fails ? Should this actually belong to 
get_thrift_profile() ?



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 6
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Wed, 20 Dec 2017 23:07:40 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-18 Thread Philip Zeyliger (Code Review) 
Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 6: Code-Review+1

Thanks!


--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 6
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Tue, 19 Dec 2017 05:27:02 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-18 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 6:

Hi PhilZ, are you ok with the latest patch?


--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 6
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Tue, 19 Dec 2017 01:01:37 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-15 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 6:

> (1 comment)
 >
 > Thanks for the comments. Uploading #7.

Sorry ... that's #6


--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 6
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Fri, 15 Dec 2017 23:13:36 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-15 Thread Zoram Thanga (Code Review) 
Hello Michael Ho, Philip Zeyliger,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8784

to look at the new patch set (#6).

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..

IMPALA-6225: Part 2: Query profile date-time strings should have ns
precision.

This commit follows 16d8dd58.

This patch adds a test case that inspects the thrift profile of a
completed query, and verifies that the "Start Time" and
"End Time" of the query have nanosecond precision. We chose to
work with the thrift profile directly, rather than parse the debug
web page, as it is the thrift profile which is consumed by
management API clients of Impala.

Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
---
M tests/common/impala_service.py
M tests/query_test/test_observability.py
2 files changed, 77 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/8784/6
--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 6
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-15 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py
File tests/common/impala_service.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@81
PS4, Line 81: return None
> Ok, that makes sense.
The HTTP response code is 200!



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 5
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Fri, 15 Dec 2017 22:31:55 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-15 Thread Philip Zeyliger (Code Review) 
Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py
File tests/common/impala_service.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@81
PS4, Line 81: return None
> Umm...I should have been clearer on this. It's the Impala server returning
Ok, that makes sense.

So, could we:

a) Assert that the HTTP request returns a non-200 response. (If it returns a 
200 OK, that's a bug, which we can file.)

b) Assert that "Could not obtain runtime profile" is in the output

This way, if Impala ever returns invalid Thrift, we won't accidentally swallow 
it.



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 5
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Fri, 15 Dec 2017 17:30:07 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-13 Thread Philip Zeyliger (Code Review) 
Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py
File tests/common/impala_service.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@81
PS4, Line 81: return None
> The exception is "Incorrect padding". I only see that when I run a specific
This smells like a bug to me. Are we returning something invalid?



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 5
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 14 Dec 2017 00:47:26 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-13 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 5:

> (1 comment)
 >
 > Uploading patch set #6 post rebase.

Umm..that should've said path set #5.


--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 5
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 14 Dec 2017 00:40:42 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-13 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 4:

(1 comment)

Uploading patch set #6 post rebase.

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py
File tests/common/impala_service.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@81
PS4, Line 81: return None
> What's the exception? What's the profile that it failed to read? Is there a
The exception is "Incorrect padding". I only see that when I run a specific 
test case with the -k option. If I run all the tests in 
query_test/test_observability.py, the exception is not thrown. Instead, I get 
the runtime profile that does not yet have the "End Time" value set.



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 4
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 14 Dec 2017 00:39:23 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-13 Thread Zoram Thanga (Code Review) 
Hello Michael Ho, Philip Zeyliger,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8784

to look at the new patch set (#5).

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..

IMPALA-6225: Part 2: Query profile date-time strings should have ns
precision.

This commit follows 16d8dd58.

This patch adds a test case that inspects the thrift profile of a
completed query, and verifies that the "Start Time" and
"End Time" of the query have nanosecond precision. We chose to
work with the thrift profile directly, rather than parse the debug
web page, as it is the thrift profile which is consumed by
management API clients of Impala.

Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
---
M tests/common/impala_service.py
M tests/query_test/test_observability.py
2 files changed, 72 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/8784/5
--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 5
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-13 Thread Philip Zeyliger (Code Review) 
Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 4:

(1 comment)

One last question.

Yes, go ahead and do the rebase with the next patch set.

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py
File tests/common/impala_service.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@81
PS4, Line 81: return None
> I've seen this fail like below:
What's the exception? What's the profile that it failed to read? Is there a bug?

(Basically, if we're seeing an invalid profile via this code path, perhaps a 
tool could see an invalid profile, and we shouldn't be emitting invalid code 
paths.)



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 4
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 14 Dec 2017 00:23:47 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-13 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 4:

(3 comments)

Thanks for the comments. Uploading patch set #5.

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py
File tests/common/impala_service.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@70
PS4, Line 70:   LOG.info("Thrift profile for query %s not yet available: 
%s", query_id, str(e))
> This is really esoteric, but you can get python to print the exception by p
Thanks for the information :)


http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@81
PS4, Line 81: return None
> I don't think this should return None. It's not expected that we can't dese
I've seen this fail like below:

$ ./run-tests.py -k test_query_profile_thrift_timestamps 
query_test/test_observability.py

$ cat $IMPALA_HOME/logs/ee_tests/results/TEST-impala-verify-metrics.xml

-- fetching results from: tests.common.impala_connection.OperationHandle 
object at 0x7fad2b6ac390
-- closing connection to: localhost:21000
MainThread: Exception while deserializing query profile of 
745d01f212903da:243a23c6: Incorrect padding


Sometimes I've seen it raise exception 3 times before finally succeeding in 
retrieving a valid thrift profile. This being an "API" I'd like to return None 
here, and let the caller deal with that.


http://gerrit.cloudera.org:8080/#/c/8784/4/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/query_test/test_observability.py@155
PS4, Line 155: dbg_str = 'Debug thrift profile for query %s' + 
str(query_id) + ' not available in '
> I don't think that %s is interpolated.
Thanks for catching that! Fixed.



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 4
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 14 Dec 2017 00:08:43 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-13 Thread Philip Zeyliger (Code Review) 
Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 4:

(3 comments)

Just a few more comments. Thanks!

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py
File tests/common/impala_service.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@70
PS4, Line 70:   LOG.info("Thrift profile for query %s not yet available: 
%s", query_id, str(e))
This is really esoteric, but you can get python to print the exception by 
passing "exc_info=True" or using LOG.exception.

An example:

>>> import logging
>>> logging.basicConfig(level=logging.INFO)
>>> try:
...   raise Exception("hey")
... except:
...   logging.exception("x")
...   logging.info("y", exc_info=True)
...
ERROR:root:x
Traceback (most recent call last):
  File "", line 2, in 
Exception: hey
INFO:root:y
Traceback (most recent call last):
  File "", line 2, in 
Exception: hey

It doesn't really matter here, but it's a useful think to know. Feel free to 
change this without further review.


http://gerrit.cloudera.org:8080/#/c/8784/4/tests/common/impala_service.py@81
PS4, Line 81: return None
I don't think this should return None. It's not expected that we can't 
deserialize this, right? So, we should fail the test.


http://gerrit.cloudera.org:8080/#/c/8784/4/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/4/tests/query_test/test_observability.py@155
PS4, Line 155: dbg_str = 'Debug thrift profile for query %s' + 
str(query_id) + ' not available in '
I don't think that %s is interpolated.



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 4
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Wed, 13 Dec 2017 21:18:38 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-08 Thread Philip Zeyliger (Code Review) 
Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 3:

(3 comments)

Thanks for the iteration. I think this is almost done. Just a few more comments.

http://gerrit.cloudera.org:8080/#/c/8784/3/tests/common/impala_service.py
File tests/common/impala_service.py:

http://gerrit.cloudera.org:8080/#/c/8784/3/tests/common/impala_service.py@64
PS3, Line 64:   def get_thrift_profile(self, query_id, timeout=10, interval=1):
please add pydoc. This will get used again, I'm sure.


http://gerrit.cloudera.org:8080/#/c/8784/3/tests/common/impala_service.py@73
PS3, Line 73:   LOG.info("Thrift profile for query %s not yet available.")
Is there a more specific exception you can use? Alternately, put the try/except 
around line 68. I worry that zlib.decompress, and base64, and deserialize can 
all throw interesting exceptions that I wouldn't want to swallow.

Also: should this be returning None? or re-throwing? Are you intentionally 
returning an empty thrift thingy?


http://gerrit.cloudera.org:8080/#/c/8784/3/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/3/tests/query_test/test_observability.py@125
PS3, Line 125: service = ImpalaCluster().get_first_impalad().service
I think this maybe should be

service = self.cluster.impalads[0].service

i.e., you shouldn't be creating a new ImpalaCluster(), but rather using the 
onet hat your test has provided you.

"git grep self.cluster.get" is more prominent than "git grep ImpalaCluster\(", 
though the latter certainly identifies several classes of this bug.

09:03:15 [130]pannier Impala [maven ?*] ~/src/Impala
$git grep self.cluster.get
bin/remote_data_load.py:services = dict((s.type, s) for s in 
self.cluster.get_all_services())
tests/common/failure_injector.py:
self.cluster.get_impala_service().set_process_auto_restart_config(value=True)
tests/common/failure_injector.py:# 
self.cluster.get_impala_service().restart()
tests/common/failure_injector.py:num_impalad_procs = 
len(self.cluster.get_impala_service().get_all_impalad_processes())
tests/common/failure_injector.py:   
self.cluster.get_impala_service().get_all_impalad_processes())
tests/common/failure_injector.py:state_store = 
self.cluster.get_impala_service().get_state_store_process()
tests/comparison/cluster.py:endpoint = 
self.cluster.get_hadoop_config("dfs.namenode.http-address",
tests/comparison/cluster.py:  port = 
self.cluster.get_hadoop_config("dfs.https.port", 20102)
tests/comparison/cluster.py:
self.cluster.get_hadoop_config("hive.metastore.warehouse.dir")).path
tests/custom_cluster/test_insert_behaviour.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_insert_behaviour.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_query_concurrency.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_query_expiration.py:impalad = 
self.cluster.get_first_impalad()
tests/custom_cluster/test_query_expiration.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_query_expiration.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_s3a_access.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_scratch_disk.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_scratch_disk.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_scratch_disk.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_scratch_disk.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_scratch_disk.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_seq_file_filtering.py:impalad = 
self.cluster.get_any_impalad()
tests/custom_cluster/test_session_expiration.py:impalad = 
self.cluster.get_any_impalad()
tests/experiments/test_catalog_hms_failures.py:impalad = 
self.cluster.get_any_impalad()
tests/experiments/test_catalog_hms_failures.py:impalad = 
self.cluster.get_any_impalad()
tests/experiments/test_catalog_hms_failures.py:impalad = 
self.cluster.get_any_impalad()
tests/experiments/test_process_failures.py:impalad = 
self.cluster.get_any_impalad()
tests/experiments/test_process_failures.py:impalad = 
self.cluster.get_any_impalad()
tests/experiments/test_process_failures.py:impalad = 
self.cluster.get_any_impalad()
tests/experiments/test_process_failures.py:worker_impalad = 
self.cluster.get_different_impalad(impalad)
tests/experiments/test_process_failures.py:impalad = 
self.cluster.get_any_impalad()
tests/experiments/test_process_failures.py:impalad = 
self.cluster.get_any_impalad()



--
To view, visit

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-07 Thread Zoram Thanga (Code Review) 
Hello Michael Ho, Philip Zeyliger,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8784

to look at the new patch set (#3).

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..

IMPALA-6225: Part 2: Query profile date-time strings should have ns
precision.

This commit follows 16d8dd58.

This patch adds a test case that inspects the thrift profile of a
completed query, and verifies that the "Start Time" and
"End Time" of the query have nanosecond precision. We chose to
work with the thrift profile directly, rather than parse the debug
web page, as it is the thrift profile which is consumed by
management API clients of Impala.

Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
---
M tests/common/impala_service.py
M tests/query_test/test_observability.py
2 files changed, 63 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/8784/3
--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 3
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-07 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 2:

(9 comments)

Thanks for the comments. Please have a look at patch set #3

http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@116
PS2, Line 116:   def __test_query_profile_thrift_timestamps(self):
> I wouldn't have a function named __x as well as a function named x. It's ju
Removed __, and the other function below.


http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@126
PS2, Line 126: service = ImpalaCluster().get_any_impalad().service
> Why is this "any_impalad"? It really has to be the one that you executed th
Is there a way to get this from the handle or something? Would love to use it. 
Changed to get_first_impalad() for now.


http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@132
PS2, Line 132:   tree = service.get_thrift_profile(query_id)
> Where is get_thrift_profile() defined?
This is defined in common/impala_service.py, as part of this patch. I figured 
it could be useful more generally.


http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@134
PS2, Line 134:   start_time = tree.nodes[1].info_strings["Start Time"]
> Add a comment about why it's nodes[1] here.
Done


http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@136
PS2, Line 136:   start_time_sub_sec_str = start_time.split('.')[-1]
> Add a comment here about what start_time looks like, so it's clear what the
Done


http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@142
PS2, Line 142:   assert len(end_time_sub_sec_str) == 9
> Add ", end_time" here, so that we can see the failed string if it ever fail
Done


http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@147
PS2, Line 147: # This could happen due to heavy system load. The test is 
then inconclusive.
> This is a controlled environment--our tests. Let's be assertive and do the
Changed to assert statement


http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@152
PS2, Line 152: print dbg_str
> Prefer logging to printing. We do both, but "import logging; logging.info(.
Done


http://gerrit.cloudera.org:8080/#/c/8784/2/tests/query_test/test_observability.py@155
PS2, Line 155:   def test_query_profile_thrift_timestamps(self):
 : if not self.__test_query_profile_thrift_timestamps():
 :   dbg_str = 'Debug thrift profile not available in ' + 
str(MAX_RETRIES) + ' seconds?'
 :   dbg_str += ' Retrying the test'
 :   assert True, self.__test_query_profile_thrift_timestamps()
> Do we really need to run it twice? Why isn't it passing? I'll bet my hat on
Removed this one. Instead asserting that the actual test above is successful.



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 2
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Fri, 08 Dec 2017 07:03:36 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-07 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py@119
PS1, Line 119: __
> Why __ prefix ?
I meant this as a way to convey that this function is 'private', and is not a 
test case that is exposed. Suggestions for alternatives welcome!


http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py@136
PS1, Line 136: while (retries < 60):
> for i in xrange(MAX_RETRIES):
Done


http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py@139
PS1, Line 139: ["Start Time"
> Is this key always in info_strings[] ? Is there a chance this will result i
I think the keys are always there, unless they are removed from 
ClientRequestState::ClientRequestState() and ClientRequestState::Done(). We'd 
be able to catch the regression then. Since clients expect to see these, I 
think it's good to assume so in the test case as well.


http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py@140
PS1, Line 140: ["End Time"]
> Same question for "End Time".
See the previous reply.



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 1
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 07 Dec 2017 20:07:46 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-07 Thread Zoram Thanga (Code Review) 
Hello Michael Ho, Philip Zeyliger,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8784

to look at the new patch set (#2).

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..

IMPALA-6225: Part 2: Query profile date-time strings should have ns
precision.

This commit follows 16d8dd58.

This patch adds a test case that inspects the thrift profile of a
completed query, and verifies that the "Start Time" and
"End Time" of the query have nanosecond precision. We chose to
work with the thrift profile directly, rather than parse the debug
web page, as it is the thrift profile which is consumed by
management API clients of Impala.

Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
---
M tests/common/impala_service.py
M tests/query_test/test_observability.py
2 files changed, 66 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/8784/2
--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 2
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-06 Thread Michael Ho (Code Review) 
Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py
File tests/query_test/test_observability.py:

http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py@119
PS1, Line 119: __
Why __ prefix ?


http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py@136
PS1, Line 136: while (retries < 60):
for i in xrange(MAX_RETRIES):


http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py@139
PS1, Line 139: ["Start Time"
Is this key always in info_strings[] ? Is there a chance this will result in 
key not found exception ?


http://gerrit.cloudera.org:8080/#/c/8784/1/tests/query_test/test_observability.py@140
PS1, Line 140: ["End Time"]
Same question for "End Time".



--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 1
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Thu, 07 Dec 2017 01:59:29 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-06 Thread Zoram Thanga (Code Review) 
Zoram Thanga has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8784 )

Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..


Patch Set 1:

The earlier attempt parsed the debug webpage; this one gets the thrift profile 
and extracts the time stamps from the profile tree. Thanks PhilZ for sharing 
how to do that.


--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 1
Gerrit-Owner: Zoram Thanga 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Wed, 06 Dec 2017 23:30:25 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6225: Part 2: Query profile date-time strings should have ns precision.

2017-12-06 Thread Zoram Thanga (Code Review) 
Zoram Thanga has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8784


Change subject: IMPALA-6225: Part 2: Query profile date-time strings should 
have ns precision.
..

IMPALA-6225: Part 2: Query profile date-time strings should have ns
precision.

This commit follows 16d8dd58.

This patch adds a test case that inspects the thrift profile of a
completed query, and verifies that the "Start Time" and
"End Time" of the query have nanosecond precision. We chose to
work with the thrift profile directly, rather than parse the debug
web page, as it is the thrift profile which is consumed by
management API clients of Impala.

Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
---
M tests/common/impala_service.py
M tests/query_test/test_observability.py
2 files changed, 70 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/8784/1
--
To view, visit http://gerrit.cloudera.org:8080/8784
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id3421a34cc029ebca551730084c7cbd402d5c109
Gerrit-Change-Number: 8784
Gerrit-PatchSet: 1
Gerrit-Owner: Zoram Thanga