I think you need to build Zeppelin from source to run against CDH
On Wed, Sep 14, 2016 at 12:58 PM -0700, "Abhi Basu" <9000r...@gmail.com<mailto:9000r...@gmail.com>> wrote: I feel there is a scala compatibility issue and I will try compiling with the right switches. On Wed, Sep 14, 2016 at 1:54 PM, Abhi Basu <9000r...@gmail.com<mailto:9000r...@gmail.com>> wrote: Yes that fixed some of the problems. I am using Zeppelin 0.6.1 binaries against CDH 5.8 (Spark 1.6.0). Would there be a compatibility issue? Thanks Abhi On Wed, Sep 14, 2016 at 12:55 PM, moon soo Lee <m...@apache.org<mailto:m...@apache.org>> wrote: Could you try to set full path of python command on zeppelin.python property? not the bin directory. On Wed, Sep 14, 2016 at 10:19 AM Abhi Basu <9000r...@gmail.com<mailto:9000r...@gmail.com>> wrote: Tried pyspark command on same machine which uses Anaconda python and sc.version returned value. Zeppelin: zeppelin.python /home/cloudera/anaconda2/bin In zeppelin, nothing is returned. On Wed, Sep 14, 2016 at 11:53 AM, moon soo Lee <m...@apache.org<mailto:m...@apache.org>> wrote: Did you export SPARK_HOME in conf/zeppelin-env.sh? Could you verify the some code works with ${SPARK_HOME}/bin/pyspark, on the same machine that zeppelin runs? Thanks, moon On Wed, Sep 14, 2016 at 8:07 AM Abhi Basu <9000r...@gmail.com<mailto:9000r...@gmail.com>> wrote: Oops sorry. the above code generated this error: RROR [2016-09-14 10:04:27,121] ({qtp2003293121-11} NotebookServer.java[onMessage]:221) - Can't handle message org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TT<http://org.apache.thrift.transport.TT>ransportException at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.cancel(RemoteInterpreter.java:319) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.cancel(LazyOpenInterpreter.java:100) at org.apache.zeppelin.notebook.Paragraph.jobAbort(Paragraph.java:330) at org.apache.zeppelin.scheduler.Job.abort(Job.java:239) at org.apache.zeppelin.socket.NotebookServer.cancelParagraph(NotebookServer.java:995) at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:180) at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:56) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:128) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:122) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:161) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.WebSocketSession.incomingFrame(WebSocketSession.java:309) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:214) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.Parser.notifyFrame(Parser.java:220) at org.eclipse.jetty.websocket.co<http://org.eclipse.jetty.websocket.co>mmon.Parser.parse(Parser.java:258) at org.eclipse.jetty.websocket.common.io<http://common.io>.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:632) at org.eclipse.jetty.websocket.common.io<http://common.io>.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:480) at org.eclipse.jetty.io<http://org.eclipse.jetty.io>.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.thrift.transport.TT<http://org.apache.thrift.transport.TT>ransportException at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) at org.apache.thrift.transport.TT<http://org.apache.thrift.transport.TT>ransport.readAll(TTransport.java:86) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_cancel(RemoteInterpreterService.java:274) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.cancel(RemoteInterpreterService.java:259) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.cancel(RemoteInterpreter.java:316) ... 21 more This is my spark interpreter settings: spark %spark , %spark.pyspark , %spark.r , %spark.sql , %spark.dep Option Interpreter for note Connect to existing process Properties name value args master yarn-client spark.app.name<http://spark.app.name> Zeppelin spark.cores.max spark.executor.memory zeppelin.R.cmd R zeppelin.R.image.width 100% zeppelin.R.knitr true zeppelin.R.render.options out.format = 'html', comment = NA, echo = FALSE, results = 'asis', message = F, warning = F zeppelin.dep.additionalRemoteRepository spark-packages,http://dl.bintray.com/spark-packages/maven,false; zeppelin.dep.localrepo local-repo zeppelin.interpreter.localRepo /usr/local/bin/zeppelin-0.6.1-bin-all/local-repo/2BXF675WU zeppelin.pyspark.python python zeppelin.spark.concurrentSQL false zeppelin.spark.importImplicit true zeppelin.spark.maxResult 1000 zeppelin.spark.printREPLOutput true zeppelin.spark.sql.stacktrace false zeppelin.spark.useHiveContext true On Wed, Sep 14, 2016 at 10:05 AM, Abhi Basu <9000r...@gmail.com<mailto:9000r...@gmail.com>> wrote: %pyspark input_file = "hdfs:////tmp/filenname.gz" raw_rdd = sc.textFile(input_file) -- Abhi Basu -- Abhi Basu -- Abhi Basu -- Abhi Basu
