[
https://issues.apache.org/jira/browse/HIVE-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu reassigned HIVE-26213:
----------------------------------
> "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it
> will raise an error
> -----------------------------------------------------------------------------------------------
>
> Key: HIVE-26213
> URL: https://issues.apache.org/jira/browse/HIVE-26213
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.1.2
> Environment: Hive 3.1.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_162
> java.vendor=Oracle Corporation
> Reporter: Jingxuan Fu
> Assignee: Jingxuan Fu
> Priority: Major
>
> In hive-default.xml.template
>
>
>
>
> {code:java}
> <property> <name>hive.limit.pushdown.memory.usage</name> <value>0.1</value>
> <description> Expects value between 0.0f and 1.0f. The fraction of available
> memory to be used for buffering rows in Reducesink operator for limit
> pushdown optimization. </description> </property>
> {code}
>
>
> Based on the description of hive-default.xml.template,
> hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
> hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
> memory of all buffered lines for the limit pushdown optimization, and
> successfully start hiveserver2.
> Then, call the java api to write a program to establish a jdbc connection as
> a client to access hive, using JDBCDemo as an example.
>
> {code:java}
> import demo.utils.JDBCUtils; public class JDBCDemo{ public static void
> main(String[] args) throws Exception { JDBCUtils.init();
> JDBCUtils.createDatabase(); JDBCUtils.showDatabases();
> JDBCUtils.createTable(); JDBCUtils.showTables(); JDBCUtils.descTable();
> JDBCUtils.loadData(); JDBCUtils.selectData(); JDBCUtils.countData();
> JDBCUtils.dropDatabase(); JDBCUtils.dropTable(); JDBCUtils.destory(); } }
> {code}
> After running the client program, both the client and the hiveserver throw
> exceptions.
>
> {code:java}
> 2022-05-09 19:05:36: Starting HiveServer2 SLF4J: Class path contains multiple
> SLF4J bindings. SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory] Hive Session ID =
> 67a6db8d-f957-4d5d-ac18-28403adab7f3 Hive Session ID =
> f9f8772c-5765-4c3e-bcff-ca605c667be7 OK OK OK OK OK OK OK Loading data to
> table default.emp OK FAILED: SemanticException Invalid memory usage value 1.0
> for hive.limit.pushdown.memory.usage{code}
>
>
>
> {code:java}
> liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh SLF4J: Class path contains
> multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory] Running: drop database if
> exists hive_jdbc_test Running: create database hive_jdbc_test Running: show
> databases default hive_jdbc_test Running: drop table if exists emp Running:
> create table emp( empno int, ename string, job string, mgr int, hiredate
> string, sal double, comm double, deptno int ) row format delimited fields
> terminated by '\t' Running: show tables emp Running: desc emp empno int ename
> string job string mgr int hiredate string sal double comm double deptno int
> Running: load data local inpath '/home/liky/hiveJDBCTestData/data.txt'
> overwrite into table emp Running: select * from emp Exception in thread
> "main" org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: SemanticException Invalid memory usage value 1.0 for
> hive.limit.pushdown.memory.usage at
> org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:380) at
> org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:366) at
> org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:354)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:293) at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:509) at
> demo.utils.JDBCUtils.selectData(JDBCUtils.java:98) at
> demo.test.JDBCDemo.main(JDBCDemo.java:19){code}
>
>
> Setting hive.limit.pushdown.memory.usage to 0.0 has no exception.
> So, setting hive.limit.pushdown.memory.usage to 1.0 is not desirable,
> *hive-default.xml.template is not clear enough for the description of the
> boundary of the value, it is better to use the interval to indicate the value
> that is [0.0,1.0).*
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
