[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user hayssams commented on the issue: https://github.com/apache/zeppelin/pull/1479 Replaced by PR #2333 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user aspen01 commented on the issue: https://github.com/apache/zeppelin/pull/1479 I made a pull request : https://github.com/apache/zeppelin/pull/1600 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user aspen01 commented on the issue: https://github.com/apache/zeppelin/pull/1479 You know, there is WebHDFS Rest API in hadoop. If we use this, we donât need to care about library dependency and configuration. What do you think using WebHDFS Rest API? HDFS interpreter already use it, so I added HDFSNotebookRepo using this function. Please check this code : https://github.com/aspen01/zeppelin/commit/4b57a2faefdd34f4675e1acd78b25677111a5e57 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1479 @hayssams Do you have any clue that you mentioned? I don't know why two different jars while building Zeppelin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user hayssams commented on the issue: https://github.com/apache/zeppelin/pull/1479 @jongyoul @zjffdu This issue is with hadoop 2.7 only. With hadoop 2.6 making hadoop dependency as provided works well. I get the following exception Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.tracing.SpanReceiverHost.get(Lorg/apache/hadoop/conf/Configuration;Ljava/lang/String;)Lorg/apache/hadoop/tracing/SpanReceiverHost; at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:634) at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:619) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169) at org.apache.zeppelin.util.HdfsSite.mkdirs(HdfsSite.java:98) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1479 Agree with @jongyoul we should make hadoop dependency as provided, @hayssams Could you paste the errors you can see when using provided ? So that someone else can help that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1479 @hayssams AFAIK, those hadoop-* have a lot of dependencies by themselves and it's possible to make a problem like version mismatch even in case that we don't use that feature. What do you think of it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user hayssams commented on the issue: https://github.com/apache/zeppelin/pull/1479 @jongyoul When using provided and bringing in all hadoop dependencies to zengine by adding hadoop lib in HADOOP_CONF_DIR to the classpath, Zeppelin raise an exception at runtime. However by bringing explicitely in scope to the zengine only hadoop-hdfs and hadoop-common, everything works as intended. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1479 @hayssams Why did you removed `provided` scope? If it doesn't, all kind of hadoop dependencies will be included in zengine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user hayssams commented on the issue: https://github.com/apache/zeppelin/pull/1479 @Leemoonsoo Test testClose in org.apache.zeppelin.python.PythonInterpreterTest. Can't understand why it is failing on travis CI but not failing on my laptop. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user hayssams commented on the issue: https://github.com/apache/zeppelin/pull/1479 @Leemoonsoo Just pushed the changes to make hadoop lib provided. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1479: Notebook: HDFS as a backend storage (Read & Write Mode...
Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/1479 Thanks for the contribution. How about let user set not only `HADOOP_CONF_DIR` but also `HADOOP_HOME` in conf/zeppelin-env.sh to use HDFS notebook storage? Then hadoop dependencies in zeppelin-zengine can be provided scope, and that'll reduce binary package size. Also that'll allow user set different hadoop version without rebuild. what do you think? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
