Thanks Chris will give it a go and report back. Bizarrely if I start the pyspark shell I don't see any issues Kr Marco On 20 Dec 2015 5:02 pm, "Chris Fregly" <ch...@fregly.com> wrote:
> hopping on a plane, but check the hive-site.xml that's in your spark/conf > directory (or should be, anyway). I believe you can change the root path > thru this mechanism. > > if not, this should give you more info google on. > > let me know as this comes up a fair amount. > > > On Dec 19, 2015, at 4:58 PM, Marco Mistroni <mmistr...@gmail.com> wrote: > > > > HI all > > posting again this as i was experiencing this error also under 1.5.1 > > I am running spark 1.5.2 on a Windows 10 laptop (upgraded from Windows 8) > > When i launch spark-shell i am getting this exception, presumably becaus > ei hav eno > > admin right to /tmp directory on my latpop (windows 8-10 seems very > restrictive) > > > > java.lang.RuntimeException: java.lang.RuntimeException: The root scratch > dir: /tmp/hive on HDFS should be writable. Current permissions are: > --------- > > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522) > > at > org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.scala:171) > > at > org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:162) > > at > org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:160) > > at > org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:167) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at > java.lang.reflect.Constructor.newInstance(Constructor.java:422) > > at > org.apache.spark.repl.SparkILoop.createSQLContext(SparkILoop.scala:1028) > > at $iwC$$iwC.<init>(<console>:9) > > at $iwC.<init>(<console>:18) > > at <init>(<console>:20) > > at .<init>(<console>:24) > > at .<clinit>(<console>) > > at .<init>(<console>:7) > > at .<clinit>(<console>) > > at $print(<console>) > > ...... > > Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive > on HDFS should be writable. Current permissions are: --------- > > at > org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:612) > > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:554) > > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:508) > > ... 56 more > > > > <console>:10: error: not found: value sqlContext > > import sqlContext.implicits._ > > ^ > > <console>:10: error: not found: value sqlContext > > import sqlContext.sql > > > > I was wondering how can i configure hive to point to a different > directory....where i have more permissions > > > > kr > > marco > > >
