You can add these extra configurations in the jdbc connection string.

For the configure keys, please check
https://github.com/apache/incubator-livy/blob/master/thriftserver/server/src/main/scala/org/apache/livy/thriftserver/LivyThriftSessionManager.scala#L563


 原始邮件 
发件人: Wing Yew Poon<wyp...@cloudera.com>
收件人: user<user@livy.incubator.apache.org>
发送时间: 2019年9月27日(周五) 11:59
主题: How to set Spark conf when using beeline to connect to Livy Thrift server


Hi, I am trying out the Thrift sever functionality in Livy 0.6. I use the 
beeline shipped with Hive to connect to the Livy Thrift server. My Spark runs 
on YARN. I am able to run simple queries on my Hive tables. However, I need to 
know how to control configs for the Spark application that the Livy session 
starts. In particular, how do I set configs such as spark.driver.memory and 
spark.executor.memory? In my case, a query I want to run dies because the AM 
exits due to exceeding the physical memory limits (YARN kills the AM). The AM 
has the default 1.5 GB of memory (1 GB + 384 MB, to the nearest 0.5 GB 
increment). I tried beeline -u 
'jdbc:hive2://<livy_server>:<thrift_port>/<db>;?spark.driver.memory=4g' but 
that doesn't seem to have the desired effect. - Wing Yew

Reply via email to