[
https://issues.apache.org/jira/browse/SPARK-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-23089.
---------------------------------
Resolution: Fixed
Fix Version/s: 2.3.0
> "Unable to create operation log session directory" when parent directory not
> present
> ------------------------------------------------------------------------------------
>
> Key: SPARK-23089
> URL: https://issues.apache.org/jira/browse/SPARK-23089
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Environment:
> /usr/hdp/2.6.3.0-235/spark2/jars/spark-hive-thriftserver_2.11-2.2.0.2.6.3.0-235.jar
> $ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.4 (Maipo)
> $ ps aux|grep ^hive.*spark.*thrift
> hive 1468503 0.9 0.5 13319628 1411676 ? Sl Jan15 10:18
> /usr/java/default/bin/java -Dhdp.version=2.6.3.0-235 -cp
> /usr/hdp/current/spark2-thriftserver/conf/:/usr/hdp/current/spark2-thriftserver/jars/*:/usr/hdp/current/hadoop-client/conf/
> -Xmx2048m org.apache.spark.deploy.SparkSubmit --properties-file
> /usr/hdp/current/spark2-thriftserver/conf/spark-thrift-sparkconf.conf --class
> org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --name Thrift
> JDBC/ODBC Server spark-internal
> Reporter: Sean Roberts
> Assignee: Marco Gaido
> Priority: Minor
> Fix For: 2.3.0
>
>
> When creating a session directory, Thrift should create the parent directory
> _(i.e. /tmp/hive/operation_logs)_ if it is not present.
> It's common for operators to clean-up old and empty directories in /tmp, or
> to have tools (systemd-tmpfiles or tmpwatch) that do it automatically.
> This was fixed in HIVE-12262 but not in Spark Thrift as seen by this:
> {code}18/01/15 14:22:49 WARN HiveSessionImpl: Unable to create operation log
> session directory:
> /tmp/hive/operation_logs/683a6318-adc4-42c4-b665-11dad14d7ec7{code}
> Resolved by manually creating /tmp/hive/operation_logs/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
