[
https://issues.apache.org/jira/browse/HIVE-23857?focusedWorklogId=459505&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-459505
]
ASF GitHub Bot logged work on HIVE-23857:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Jul/20 19:50
Start Date: 15/Jul/20 19:50
Worklog Time Spent: 10m
Work Description: t3rmin4t0r commented on a change in pull request #1258:
URL: https://github.com/apache/hive/pull/1258#discussion_r455302352
##########
File path: parser/bin/fixHiveParser.sh
##########
@@ -0,0 +1,44 @@
+#!/bin/bash
+
+# This is a temporary solution for the issue of the "code too large" problem
related to HiveParser.java
+# We got to a point where adding anything to the antlr files lead to an issue
about having a HiveParser.java that can not be compiled due to the compiled
code size limitation in java (maximum 65536 bytes), so to avoid it we temorarly
add this script to remove the huge tokenNames array into a separate file.
+# The real solution would be to switch to antlr 4
+
+tokenFile="target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveParserTokens.java"
+input="target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveParser.java"
+output="target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveParser.java-fixed"
+
+rm $tokenFile > /dev/null 2>&1
+rm $output > /dev/null 2>&1
+
+echo "package org.apache.hadoop.hive.ql.parse;" >> $tokenFile
+echo "" >> $tokenFile
+echo "public class HiveParserTokens {" >> $tokenFile
+
+state="STAY"
+while IFS= read -r line
Review comment:
Looks like AWK reinvented in bash
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 459505)
Time Spent: 0.5h (was: 20m)
> Fix HiveParser "code too large" problem
> ---------------------------------------
>
> Key: HIVE-23857
> URL: https://issues.apache.org/jira/browse/HIVE-23857
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Miklos Gergely
> Assignee: Miklos Gergely
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HivePasrser.g can not be extended anymore as adding any more tokens leads to
> a "code too large" problem, because the compiled code size would exceed 65536
> bytes. The real solution would be to introduce anltr4, in the meantime it can
> be fixed be moving the tokenNames variable into a separate file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
