[ https://issues.apache.org/jira/browse/HIVE-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872755#comment-15872755 ]
Vaibhav Gumashta commented on HIVE-14901: ----------------------------------------- [~norrisl] Thanks for the patch. My feedback: # Address the case when thrift serde is not used (basically have an upper bound check in the fetch call). This is to prevent OOMs on server if a user passes an abnormally large value. # For passing the user value to thrift serde, check HiveSessionImpl# configureSession. # Small nit: the name of HIVE_SERVER2_RESULTSET_DEFAULT_FETCH_SIZE/ hive.server2.resultset.default.fetch.size to be consistent with the other params. I think this won't be backward incompat as this hasn't gone into any release yet (but a quick check on the released 2.1 line would be great). # Log the MAX_BUFFERED_ROWS in serde at INFO level. # You might want to think of additional tests if you can to test these changes and if they are needed. > HiveServer2: Use user supplied fetch size to determine #rows serialized in > tasks > -------------------------------------------------------------------------------- > > Key: HIVE-14901 > URL: https://issues.apache.org/jira/browse/HIVE-14901 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC, ODBC > Affects Versions: 2.1.0 > Reporter: Vaibhav Gumashta > Assignee: Norris Lee > Attachments: HIVE-14901.1.patch, HIVE-14901.2.patch, > HIVE-14901.3.patch, HIVE-14901.patch > > > Currently, we use {{hive.server2.thrift.resultset.max.fetch.size}} to decide > the max number of rows that we write in tasks. However, we should ideally use > the user supplied value (which can be extracted from the > ThriftCLIService.FetchResults' request parameter) to decide how many rows to > serialize in a blob in the tasks. We should however use > {{hive.server2.thrift.resultset.max.fetch.size}} to have an upper bound on > it, so that we don't go OOM in tasks and HS2. -- This message was sent by Atlassian JIRA (v6.3.15#6346)