Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/11575 )
Change subject: IMPALA-6964: Track stats about column and page sizes in Parquet reader ...................................................................... Patch Set 16: (1 comment) http://gerrit.cloudera.org:8080/#/c/11575/16/tests/util/counters.py File tests/util/counters.py: http://gerrit.cloudera.org:8080/#/c/11575/16/tests/util/counters.py@19 PS16, Line 19: class SummaryStatsCounter(object): > Is there a way that we use the thrift profile directly instead? By now we a Yeah, thats a good point. We can use TSummaryStatsCounter directly. What about changing this class to counter_utils.py and just defining an is_empty(TSummaryStatsCounter) method? Clients of this class would then create a TSummaryStatsCounter directly and just call counter_utils::is_empty(my_tsummary_stats_counter) "If we want to stick with the text profile, we should have a namedtuple + an is_empty(counter) method where we need it" - not sure I fully understand what you mean? My understanding of the code is that the Beeswax Python client we are using uses the method GetRuntimeProfile (https://github.com/apache/impala/blob/master/common/thrift/ImpalaService.thrift#L408) to return the Runtime Profile, which returns the Runtime Profile as a string (even through there seems to be a Runtime Profile Thrift struct - TRuntimeProfileTree). -- To view, visit http://gerrit.cloudera.org:8080/11575 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I322f9b324b6828df28e5caf79529085c43d7c817 Gerrit-Change-Number: 11575 Gerrit-PatchSet: 16 Gerrit-Owner: Sahil Takiar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Sahil Takiar <[email protected]> Gerrit-Comment-Date: Wed, 09 Jan 2019 20:43:12 +0000 Gerrit-HasComments: Yes
