Re: Having issues with Hive RuntimeException in runing Zeppelin notebook applicaiton

[Md. Rezaul Karim]

on: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@48f2b61e, see the next exception for details.	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)	... 136 moreCaused by: java.sql.SQLException: Database at /home/asif/zeppelin-0.6.2-bin-all/metastore_db has an incompatible format with the current version of the software. The database was created by or upgraded by version 10.11.	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)	at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)	... 133 moreCaused by: ERROR XSLAN: Database at /home/asif/zeppelin-0.6.2-bin-all/metastore_db has an incompatible format with the current version of the software. The database was created by or upgraded by version 10.11.Caused by: org.apache.derby.impl.jdbc.EmbedSQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@48f2b61e, see the next exception for details.at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)... 136 moreCaused by: org.apache.derby.impl.jdbc.EmbedSQLException: Database at /home/asif/zeppelin-0.6.2-bin-all/metastore_db has an incompatible format with the current version of the software. The database was created by or upgraded by version 10.11.at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)... 133 moreCaused by: org.apache.derby.iapi.error.StandardException: Database at /home/asif/zeppelin-0.6.2-bin-all/metastore_db has an incompatible format with the current version of the software. The database was created by or upgraded by version 10.11.
 Thanks and Regards,-Md. Rezaul Karim PhD Researcher, Insight Centre for Data Analytics National University of Ireland Galway 

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

[Md. Rezaul Karim]

+1On Jan 21, 2017 11:17 AM, Mina Lee  wrote:Hi folks,I propose the following RC to be released for the Apache Zeppelin 0.7.0 release.The commit id is bcbe0855175f55eddd10b918303806eb5e5db091 which iscorresponds to the tag v0.7.0-rc1:https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=bcbe0855175f55eddd10b918303806eb5e5db091The release archives (tgz), signature, and checksums are herehttps://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc1/The release candidate consists of the following source distribution archive zeppelin-0.7.0.tgzIn addition, the following supplementary binary distributions are providedfor user convenience at the same locationzeppelin-0.7.0-bin-all.tgzzeppelin-0.7.0-bin-netinst.tgzThe maven artifacts are herehttps://repository.apache.org/content/repositories/orgapachezeppelin-1022You can find the KEYS file here:https://dist.apache.org/repos/dist/release/zeppelin/KEYSRelease notes available athttps://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336544&projectId=12316221Vote will be open for next 72 hours (close at 3:30 am 24/Jan PST).[ ] +1 approve[ ] 0 no opinion[ ] -1 disapprove (and reason why)

Re: [ANNOUNCE] Apache Zeppelin 0.7.0 released

[Md. Rezaul Karim]

Congratulations! Thanks for the great news. I appreciate your hard work for every release.On Feb 6, 2017 7:07 AM, Mina Lee  wrote:The Apache Zeppelin community is pleased to announce the availability ofthe 0.7.0 release.Zeppelin is a collaborative data analytics and visualization tool fordistributed, general-purpose data processing system such as Apache Spark,Apache Flink, etc.The community put significant effort into improving Apache Zeppelin sincethe last release, focusing on multi-user support, pluggable visualization,better interpreter support. More than 100+ contributors provided 700+ patchesfor new features, improvements and bug fixes. More than 480+ issues havebeen resolved.We encourage you to download the latest release fromhttp://zeppelin.apache.org/download.htmlRelease note is available athttp://zeppelin.apache.org/releases/zeppelin-release-0.7.0.htmlWe welcome your help and feedback. For more information on the project andhow to get involved, visit our website at http://zeppelin.apache.org/Thank you all users and contributors who have helped to improve ApacheZeppelin.Regards,The Apache Zeppelin community

Re: NoSuchMethodException in SQL

[Md. Rezaul Karim]

How could I do that? Could you please send me any reference link? On Feb 17, 2017 1:17 AM, Jeff Zhang  wrote:zeppelin will create SparkContext implicitly for users, so it might be too late to set it after interpreter is opened. You can try to set that in interpreter setting.Muhammad Rezaul Karim 于2017年2月16日周四 下午11:52写道：Hi Lee,Thanks for the info that really helped. I set the compression codec in the Spark side -i.e. inside the SPARK_HOME and now the problem resolved. However, I was wondering if it's possible to set the same from the Zeppelin notebook.I tried in the following way: %spark conf.set("spark.io.compression.codec", "lz4") But getting an error. Please suggest. On Thursday, February 16, 2017 7:40 AM, Jongyoul Lee  wrote:   Hi, Can you check if the script passes in spark-shell or not? AFAIK, you have to add compression codec by yourself in Spark side.On Wed, Feb 15, 2017 at 1:10 AM, Muhammad Rezaul Karim  wrote:Hi All,I am receiving the following exception while executing SQL queries:  java.lang. NoSuchMethodException: org.apache.spark.io. LZ4CompressionCodec.( org.apache.spark.SparkConf)    at java.lang.Class. getConstructor0(Class.java: 3082)    at java.lang.Class. getConstructor(Class.java: 1825)    at org.apache.spark.io. CompressionCodec$.createCodec( CompressionCodec.scala:71)    at org.apache.spark.io. CompressionCodec$.createCodec( CompressionCodec.scala:65)    at org.apache.spark.sql. execution.SparkPlan.org$ apache$spark$sql$execution$ SparkPlan$$decodeUnsafeRows( SparkPlan.scala:250)    at org.apache.spark.sql. execution.SparkPlan$$anonfun$ executeCollect$1.apply( SparkPlan.scala:276)    at org.apache.spark.sql. execution.SparkPlan$$anonfun$ executeCollect$1.apply( SparkPlan.scala:275)    at scala.collection. IndexedSeqOptimized$class. foreach(IndexedSeqOptimized. scala:33)    at scala.collection.mutable. ArrayOps$ofRef.foreach( ArrayOps.scala:186)    at org.apache.spark.sql. execution.SparkPlan. executeCollect(SparkPlan. scala:275)    at org.apache.spark.sql. execution.exchange. BroadcastExchangeExec$$ anonfun$relationFuture$1$$ anonfun$apply$1.apply( BroadcastExchangeExec.scala: 78)    at org.apache.spark.sql. execution.exchange. BroadcastExchangeExec$$ anonfun$relationFuture$1$$ anonfun$apply$1.apply( BroadcastExchangeExec.scala: 75)    at org.apache.spark.sql. execution.SQLExecution$. withExecutionId(SQLExecution. scala:94)    at org.apache.spark.sql. execution.exchange. BroadcastExchangeExec$$ anonfun$relationFuture$1. apply(BroadcastExchangeExec. scala:74)    at org.apache.spark.sql. execution.exchange. BroadcastExchangeExec$$ anonfun$relationFuture$1. apply(BroadcastExchangeExec. scala:74)    at scala.concurrent.impl.Future$ PromiseCompletingRunnable. liftedTree1$1(Future.scala:24)    at scala.concurrent.impl.Future$ PromiseCompletingRunnable.run( Future.scala:24)    at java.util.concurrent. ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1142)    at java.util.concurrent. ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:617)    at java.lang.Thread.run(Thread. java:745)My SQL query is: %sql  select * from land where Price >= 1 AND CLUSTER = 2 I am experiencing the above exception in the 1st run always but when I re-execute the same query for the 2nd or 3rd time, I don't get this error. Am I doing something wrong? Someone, please help me out. Kinds regards,Reza-- 이종열, Jongyoul Lee, 李宗烈
http://madeng.net