[
https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157200#comment-15157200
]
Herman van Hovell commented on SPARK-13431:
-------------------------------------------
Test are currently passing and the if you call {{sparkShell}} from sbt you can
work with spark and issue sql commands. So the direct problem is definitely
caused by shading. The thing I am actually not getting is why sbt assemblies
won't fail.
That being said, ANTLR spits out huge files which are very near to but not over
the 64K limit (It won't build otherwise). I am not too confident that splitting
the parser another time is much more than a temporary solution. The current
grammar contains a perticulary nasty rule to determine if a keyword is
non-reserved (this creates a huge state explosion), we should take a look at
that. On the longer run I think that we should do a rewrite of the current
parser with ANTLR4 which should simplify parser rules (and thus the size of the
resulting parser), and make the construction of logical plans less
Rube-Goldbergy/pattern-match-heaven.
> Maven build fails due to: Method code too large! in Catalyst
> ------------------------------------------------------------
>
> Key: SPARK-13431
> URL: https://issues.apache.org/jira/browse/SPARK-13431
> Project: Spark
> Issue Type: Bug
> Components: Build, Optimizer
> Affects Versions: 2.0.0
> Reporter: Stavros Kontopoulos
> Priority: Blocker
>
> Cannot build the project when run the normal build commands:
> eg.
> {code}
> build/mvn -Phadoop-2.6 -Dhadoop.version=2.6.0 clean package
> ./make-distribution.sh --name test --tgz -Phadoop-2.6
> {code}
> Integration builds are also failing:
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.6/229/console
> https://ci.typesafe.com/job/mit-docker-test-zk-ref/12/console
> It looks like this is the commit that introduced the issue:
> https://github.com/apache/spark/commit/7925071280bfa1570435bde3e93492eaf2167d56
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
