Hello Meilong! I exported SPARK_HOME inside zeppelin-env.sh configuration file. The documentation says that hive-env.xml should be placed inside zeppelin installation forlder so I did that, but just to be sure I tried what you said (also exporting SPARK_HOME environment variable) but that didn't work either. Thanks, any other suggestion would be really appreciated.
2016-06-27 3:46 GMT-03:00 Meilong Huang <[email protected]>: > Do your export $SPAR_HOME? Is hive-site.xml file in $SPARK_HOME/conf? > > 2016-06-25 2:50 GMT+08:00 Juan Pablo Briganti <[email protected]>: > >> Hello: >> >> I'm configuring zeppelin to work with our cloudera cluster and I'm >> facing some errors. I would like to see if those are known errors or I'm >> doing something wrong: >> >> Zeppelin version: 0.6 >> Cloudera version: 5.7.0 >> Hadoop version: 2.6 >> Spark version: 1.6 >> >> Build maven command: mvn clean package -Pspark-1.6 >> -Dspark.version=1.6.0-cdh5.7.0 -Dhadoop.version=2.6.0-cdh5.7.0 -Phadoop-2.6 >> -Pvendor-repo -DskipTests >> >> I'm running applications using %spark and %pyspark interpreter and seems >> to run ok under yarn-client configuration, using both created and the >> example scripts. But when I try to run any script with %sql I get one of >> the next 2 errors detailed at the end of the email. It's not clear when I >> get each error, I just keep getting the first one until I start getting the >> second one without doing any configuration change, just running again the >> same script. >> I have my hive-site.xml file inside zeppelin/conf folder. >> Log folder provides those exceptions only, no extra information. >> >> >> Any hint you may have about this would help to solve the problem. >> Thanks. >> >> java.lang.NullPointerException >> at >> org.apache.spark.sql.hive.client.ClientWrapper.conf(ClientWrapper.scala:205) >> at >> org.apache.spark.sql.hive.client.ClientWrapper.client(ClientWrapper.scala:261) >> at >> org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$withHiveState$1.apply(ClientWrapper.scala:273) >> at >> org.apache.spark.sql.hive.client.ClientWrapper.liftedTree1$1(ClientWrapper.scala:228) >> at >> org.apache.spark.sql.hive.client.ClientWrapper.retryLocked(ClientWrapper.scala:227) >> at >> org.apache.spark.sql.hive.client.ClientWrapper.withHiveState(ClientWrapper.scala:270) >> at org.apache.spark.sql.hive.HiveQLDialect.parse(HiveContext.scala:65) >> at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211) >> at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211) >> at >> org.apache.spark.sql.execution.SparkSQLParser$$anonfun$org$apache$spark$sql$execution$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:114) >> at >> org.apache.spark.sql.execution.SparkSQLParser$$anonfun$org$apache$spark$sql$execution$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:113) >> at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:136) >> at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135) >> at >> scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242) >> at >> scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242) >> at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222) >> at >> scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254) >> at >> scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254) >> at scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:202) >> at >> scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254) >> at >> scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254) >> at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222) >> at >> scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891) >> at >> scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891) >> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57) >> at scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:890) >> at >> scala.util.parsing.combinator.PackratParsers$$anon$1.apply(PackratParsers.scala:110) >> at >> org.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:34) >> at org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:208) >> at org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:208) >> at >> org.apache.spark.sql.execution.datasources.DDLParser.parse(DDLParser.scala:43) >> at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:231) >> at org.apache.spark.sql.hive.HiveContext.parseSql(HiveContext.scala:333) >> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:817) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.zeppelin.spark.SparkSqlInterpreter.interpret(SparkSqlInterpreter.java:116) >> at >> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93) >> at >> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:344) >> at org.apache.zeppelin.scheduler.Job.run(Job.java:176) >> at >> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> >> >> ******************************************************************************************************** >> >> ******************************************************************************************************** >> java.lang.ClassNotFoundException: >> line2090566716$24.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1 >> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:348) >> at >> org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:68) >> at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1613) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at scala.collection.immutable.$colon$colon.readObject(List.scala:362) >> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at scala.collection.immutable.$colon$colon.readObject(List.scala:362) >> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at scala.collection.immutable.$colon$colon.readObject(List.scala:362) >> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at scala.collection.immutable.$colon$colon.readObject(List.scala:362) >> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at scala.collection.immutable.$colon$colon.readObject(List.scala:362) >> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at scala.collection.immutable.$colon$colon.readObject(List.scala:362) >> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at scala.collection.immutable.$colon$colon.readObject(List.scala:362) >> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at >> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:76) >> at >> org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:115) >> at >> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:64) >> at >> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) >> at org.apache.spark.scheduler.Task.run(Task.scala:89) >> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> >> -- >> *Juan Pablo Briganti* | Data Architect >> *GLOBANT* | AR: +54 11 4109 1700 ext. 19508 | US: +1 877 215 5230 ext. >> 19508 | >> [image: Facebook] <https://www.facebook.com/Globant> [image: Twitter] >> <http://www.twitter.com/globant> [image: Youtube] >> <http://www.youtube.com/Globant> [image: Linkedin] >> <http://www.linkedin.com/company/globant> [image: Pinterest] >> <http://pinterest.com/globant/> [image: Globant] <http://www.globant.com> >> >> The information contained in this e-mail may be confidential. It has been >> sent for the sole use of the intended recipient(s). If the reader of this >> message is not an intended recipient, you are hereby notified that any >> unauthorized review, use, disclosure, dissemination, distribution or >> copying of this communication, or any of its contents, >> is strictly prohibited. If you have received it by mistake please let us >> know by e-mail immediately and delete it from your system. Many thanks. >> >> >> >> La información contenida en este mensaje puede ser confidencial. Ha sido >> enviada para el uso exclusivo del destinatario(s) previsto. Si el lector de >> este mensaje no fuera el destinatario previsto, por el presente queda Ud. >> notificado que cualquier lectura, uso, publicación, diseminación, >> distribución o copiado de esta comunicación o su contenido está >> estrictamente prohibido. En caso de que Ud. hubiera recibido este mensaje >> por error le agradeceremos notificarnos por e-mail inmediatamente y >> eliminarlo de su sistema. Muchas gracias. >> >> > -- *Juan Pablo Briganti* | Data Architect *GLOBANT* | AR: +54 11 4109 1700 ext. 19508 | US: +1 877 215 5230 ext. 19508 | [image: Facebook] <https://www.facebook.com/Globant> [image: Twitter] <http://www.twitter.com/globant> [image: Youtube] <http://www.youtube.com/Globant> [image: Linkedin] <http://www.linkedin.com/company/globant> [image: Pinterest] <http://pinterest.com/globant/> [image: Globant] <http://www.globant.com> -- The information contained in this e-mail may be confidential. It has been sent for the sole use of the intended recipient(s). If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution or copying of this communication, or any of its contents, is strictly prohibited. If you have received it by mistake please let us know by e-mail immediately and delete it from your system. Many thanks. La información contenida en este mensaje puede ser confidencial. Ha sido enviada para el uso exclusivo del destinatario(s) previsto. Si el lector de este mensaje no fuera el destinatario previsto, por el presente queda Ud. notificado que cualquier lectura, uso, publicación, diseminación, distribución o copiado de esta comunicación o su contenido está estrictamente prohibido. En caso de que Ud. hubiera recibido este mensaje por error le agradeceremos notificarnos por e-mail inmediatamente y eliminarlo de su sistema. Muchas gracias.
