[
https://issues.apache.org/jira/browse/HIVE-24484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370667#comment-17370667
]
David Mollitor commented on HIVE-24484:
---------------------------------------
Just bumped into another one... the latest Hadoop added some new DEBUG logging
that is *very* chatty. Hive has a feature that allows clients to download the
logging, however, it is currently capped at
{{HIVE_SERVER2_THRIFT_RESULTSET_MAX_FETCH_SIZE}} (default: 10000) right now.
Why is it capped at the fetch size? Because HS will truncate the row count and
return {{hasMoreRows}} 'false' to the client, so the client does not know there
are more rows to fetch. The extra DEBUG log lines explodes the line count and
pushes out the required (by unit tests) log lines past the truncation mark.
https://github.com/apache/hive/blob/f7a21abf5579a8df07117928caff2d72ecae27e3/service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java#L888-L912
It looks like there was some other work related to this via [HIVE-24861].
> Upgrade Hadoop to 3.3.1
> -----------------------
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
> Reporter: David Mollitor
> Assignee: David Mollitor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
