Thanks Ted, using HiveTest as my context worked. It still left a metastore directory and Derby log in my current working directory though; I manually added a shutdown hook to delete them and all was well.
On Wed, Apr 8, 2015 at 4:33 PM, Ted Yu <[email protected]> wrote: > Please take a look at > sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala : > > protected def configure(): Unit = { > warehousePath.delete() > metastorePath.delete() > setConf("javax.jdo.option.ConnectionURL", > s"jdbc:derby:;databaseName=$metastorePath;create=true") > setConf("hive.metastore.warehouse.dir", warehousePath.toString) > } > > Cheers > > On Wed, Apr 8, 2015 at 1:07 PM, Daniel Siegmann < > [email protected]> wrote: > >> I am trying to unit test some code which takes an existing HiveContext >> and uses it to execute a CREATE TABLE query (among other things). >> Unfortunately I've run into some hurdles trying to unit test this, and I'm >> wondering if anyone has a good approach. >> >> The metastore DB is automatically created in the local directory, but it >> doesn't seem to be cleaned up afterward. Is there any way to get Spark to >> clean this up when the context is stopped? Or can I point this to some >> other location, such as a temp directory? >> >> Trying to create a table fails because it is using the default warehouse >> directory (/user/hive/warehouse). Is there some way to change this without >> hard-coding a directory in a hive-site.xml; again, I'd prefer to point it >> to a temp directory so it will be automatically removed. I tried a couple >> of things that didn't work: >> >> - hiveContext.sql("SET hive.metastore.warehouse.dir=/tmp/dir/xyz") >> - hiveContext.setConf("hive.metastore.warehouse.dir", "/tmp/dir/xyz") >> >> Any advice from those who have been here before would be appreciated. >> > >
