I am trying to access a hive table which points to directory with
sub-directories via SparkSQL.
So I need to set "mapred.input.dir.recursive" as true for recursive
exploration.
I noticed that I can explicitly set the configuration using following
command:
val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
sqlContext.setConf("mapreduce.input.fileinputformat.input.dir.recursive","true")
It works as well.
However, I couldn`t find any way to load that configuration from files such
as hive-site.xml or mapred-site.xml.
How to load configuration from file?
Please help me.
Thanks in advance.
Dongkyoung Kim
