[
https://issues.apache.org/jira/browse/SPARK-41071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630761#comment-17630761
]
Yang Jie commented on SPARK-41071:
----------------------------------
cc [~hyukjin.kwon] [~yumwang]
I run
{code:java}
dev/make-distribution.sh --tgz -Phadoop-3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
{code}
on the master branch always fails.
also cc [~pancheng]
> Metaspace OOM when Local run dev/make-distribution.sh
> ------------------------------------------------------
>
> Key: SPARK-41071
> URL: https://issues.apache.org/jira/browse/SPARK-41071
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.4.0
> Reporter: Yang Jie
> Priority: Major
>
> Run
> {code:java}
> dev/make-distribution.sh --tgz -Phadoop-3 -Phadoop-cloud -Pmesos -Pyarn
> -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
> {code}
> {code:java}
> [ERROR] ## Exception when compiling 19 sources to
> /Users/yangjie01/SourceCode/git/spark-mine-12/connector/avro/target/scala-2.12/classes
> java.lang.OutOfMemoryError: Metaspace
> java.lang.ClassLoader.defineClass1(Native Method)
> java.lang.ClassLoader.defineClass(ClassLoader.java:757)
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
> java.net.URLClassLoader.access$100(URLClassLoader.java:74)
> java.net.URLClassLoader$1.run(URLClassLoader.java:369)
> java.net.URLClassLoader$1.run(URLClassLoader.java:363)
> java.security.AccessController.doPrivileged(Native Method)
> java.net.URLClassLoader.findClass(URLClassLoader.java:362)
> java.lang.ClassLoader.loadClass(ClassLoader.java:419)
> scala_maven.ScalaCompilerLoader.loadClass(ScalaCompilerLoader.java:44)
> java.lang.ClassLoader.loadClass(ClassLoader.java:352)
> scala.collection.immutable.Set$Set2.$plus(Set.scala:170)
> scala.collection.immutable.Set$Set2.$plus(Set.scala:164)
> scala.collection.mutable.SetBuilder.$plus$eq(SetBuilder.scala:28)
> scala.collection.mutable.SetBuilder.$plus$eq(SetBuilder.scala:24)
> scala.collection.TraversableLike$grouper$1$.apply(TraversableLike.scala:467)
> scala.collection.TraversableLike$grouper$1$.apply(TraversableLike.scala:455)
> scala.collection.mutable.HashMap$$anon$1.$anonfun$foreach$2(HashMap.scala:153)
> scala.collection.mutable.HashMap$$anon$1$$Lambda$68804/294594059.apply(Unknown
> Source)
> scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:237)
> scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:230)
> scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:44)
> scala.collection.mutable.HashMap$$anon$1.foreach(HashMap.scala:153)
> scala.collection.TraversableLike.groupBy(TraversableLike.scala:524)
> scala.collection.TraversableLike.groupBy$(TraversableLike.scala:454)
> scala.collection.AbstractTraversable.groupBy(Traversable.scala:108)
> scala.tools.nsc.settings.Warnings.$init$(Warnings.scala:91)
> scala.tools.nsc.settings.MutableSettings.<init>(MutableSettings.scala:28)
> scala.tools.nsc.Settings.<init>(Settings.scala:19)
> scala.tools.nsc.Settings.<init>(Settings.scala:20)
> xsbt.CachedCompiler0.<init>(CompilerBridge.scala:79) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
