NITISH SHARMA created SPARK-30967:
-------------------------------------
Summary: Achieve LAST_ACCESS_TIME column update in TBLS table of
hive metastore on table access
Key: SPARK-30967
URL: https://issues.apache.org/jira/browse/SPARK-30967
Project: Spark
Issue Type: Question
Components: Spark Shell
Affects Versions: 2.4.5
Reporter: NITISH SHARMA
I have a requirement where i am looking to update LAST_ACCESS_TIME in TBLS of
Hive metastore whenever any table is accessed through spark. I set this below
property in hive-site.xml and hive honors it and updates the LAST_ACCESS_TIME
everytime it is accessed.
<property>
<name>hive.exec.pre.hooks</name>
<value>org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec</value>
</property>
However, the same thing i want to achieve using pyspark/spark-shell but its not
honoring this property of hive hooks. Is there an alternate approach of
achieving this - 'Update of LAST_ACCESS_TIME in hive metastore on access using
spark'.
I passed the property like this -
spark-sql -e 'set
spark.hadoop.hive.exec.post.hooks=org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec;select
* from db.table;'
as well as i put the same property in /etc/spark/conf/hive-site.xml location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
