Thanks for the pointer. It seems to be really a pathological case, since the file that's in error is part of the splinter file (the smaller one, IndetifiersParser). I'll see if I can work around by splitting it some more.
iulian On Thu, Jan 28, 2016 at 4:43 PM, Ted Yu <yuzhih...@gmail.com> wrote: > After this change: > [SPARK-12681] [SQL] split IdentifiersParser.g into two files > > the biggest file under > sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser is > SparkSqlParser.g > > Maybe split SparkSqlParser.g up as well ? > > On Thu, Jan 28, 2016 at 5:21 AM, Iulian Dragoș <iulian.dra...@typesafe.com > > wrote: > >> Hi, >> >> Has anyone seen this error? >> >> The code of method specialStateTransition(int, IntStream) is exceeding the >> 65535 bytes limit SparkSqlParser_IdentifiersParser.java:39907 >> >> The error is in ANTLR generated files and it’s (according to Stack >> Overflow) due to state explosion in parser (or lexer). That seems >> plausible, given that one file has >50000 lines of code. Some suggest that >> refactoring the grammar would help. >> >> I’m seeing this error only sometimes on the command line (building with >> Sbt), but every time when building with Eclipse (which has its own Java >> compiler, so it’s not surprising that it has a different behavior). Same >> behavior with both Java 1.7 and 1.8. >> >> Any ideas? >> >> iulian >> >> -- >> >> -- >> Iulian Dragos >> >> ------ >> Reactive Apps on the JVM >> www.typesafe.com >> >> > -- -- Iulian Dragos ------ Reactive Apps on the JVM www.typesafe.com
