onjee created ZEPPELIN-6030:
-------------------------------
Summary: Bug Report: Apache Zeppelin 0.11.1 Issue with Spark
Interpreter
Key: ZEPPELIN-6030
URL: https://issues.apache.org/jira/browse/ZEPPELIN-6030
Project: Zeppelin
Issue Type: Bug
Components: build
Affects Versions: 0.11.1
Environment: *Environment:*
* Apache Zeppelin version: 0.11.1
* Operating System: Ubuntu 24.04 LTS
* Java version: OpenJDK 8
* Spark version: 2.4.7
* Hadoop: Fully distributed mode
* YARN: Fully distributed mode
* Zeppelin server and Spark running on different servers
* Spark configured to use YARN in cluster mode
Reporter: onjee
Fix For: 0.11.1
*Description:*
While using Apache Zeppelin version 0.11.1, an issue occurs when running the
Spark interpreter. The interpreter fails to initialize properly due to
unexpected metadata files that were likely included during the build process on
a macOS system. These metadata files cause errors in the initialization of the
Spark interpreter, preventing it from functioning correctly.
Maybe there's an error in the build.
*Steps to Reproduce:*
# Set up Apache Zeppelin version 0.11.1.
# Configure the Spark interpreter.
# Attempt to run a Spark paragraph.
*Expected Behavior:* The Spark paragraph should execute without any errors.
*Actual Behavior:* The Spark interpreter fails to initialize, and the following
error messages are logged:
Here, you can see that the JAR file you want to run starts with ._ and it's
actually not a JAR file, it's a metadata file for apple mac.
{code}
Caused by: java.io.IOException: Error accessing
/tmp/hadoop-on5949/nm-local-dir/usercache/on5949/appcache/application_1719905050899_0006/container_1719905050899_0006_01_000001/._spark-scala-2.12-0.11.1.jar
at
scala.reflect.io.FileZipArchive.scala$reflect$io$FileZipArchive$$openZipFile(ZipArchive.scala:190)
at scala.reflect.io.FileZipArchive.root$lzycompute(ZipArchive.scala:238) at
scala.reflect.io.FileZipArchive.root(ZipArchive.scala:235) at
scala.reflect.io.FileZipArchive.allDirs$lzycompute(ZipArchive.scala:272) at
scala.reflect.io.FileZipArchive.allDirs(ZipArchive.scala:272) at
scala.tools.nsc.classpath.ZipArchiveFileLookup.findDirEntry(ZipArchiveFileLookup.scala:76)
at
scala.tools.nsc.classpath.ZipArchiveFileLookup.list(ZipArchiveFileLookup.scala:63)
at
scala.tools.nsc.classpath.ZipArchiveFileLookup.list$(ZipArchiveFileLookup.scala:62)
at
scala.tools.nsc.classpath.ZipAndJarClassPathFactory$ZipArchiveClassPath.list(ZipAndJarFileLookupFactory.scala:58)
at
scala.tools.nsc.classpath.AggregateClassPath.$anonfun$list$3(AggregateClassPath.scala:105)
... 40 more Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method) at
java.util.zip.ZipFile.<init>(ZipFile.java:231) at
java.util.zip.ZipFile.<init>(ZipFile.java:157) at
java.util.zip.ZipFile.<init>(ZipFile.java:171) at
scala.reflect.io.FileZipArchive.scala$reflect$io$FileZipArchive$$openZipFile(ZipArchive.scala:187)
... 49 more
{code}
*Additional Information:*
* The issue is caused by macOS metadata files prefixed with '._' being
included in the Spark interpreter directory. (I don't think this metadata file
was in the previous version, so I think it went in by mistake.)
*References:*
* [Stack Overflow discussion on similar
issue|https://stackoverflow.com/questions/62954023/interpreterexception-in-apache-zeppelin-v-0-11-1-executing-a-paragraph]
* [My Velog post detailing the bug and
solution(Korean)|https://velog.io/@on5949/%ED%99%95%EC%8B%A4%ED%95%98%EC%A7%80-%EC%95%8A%EC%9D%8C-zeppelin-0.11.1-%EB%B2%84%EA%B7%B8-%EB%A6%AC%ED%8F%AC%ED%8A%B8]
* [Where I downloaded Zeppelin |https://zeppelin.apache.org/download.html]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
