[
https://issues.apache.org/jira/browse/DERBY-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281306#comment-13281306
]
Dag H. Wanvik commented on DERBY-5125:
--------------------------------------
The problem with the empty "encountered" token lies in the generated code;
SQLParser#jj_add_error_token
In version 4.0, the code looks like this:
private void jj_add_error_token(int kind, int pos) {
if (pos >= 100) return;
if (pos == jj_endpos + 1) {
jj_lasttokens[jj_endpos++] = kind;
} else if (jj_endpos != 0) {
jj_expentry = new int[jj_endpos];
for (int i = 0; i < jj_endpos; i++) {
jj_expentry[i] = jj_lasttokens[i];
}
boolean exists = false;
for (java.util.Enumeration e = jj_expentries.elements();
e.hasMoreElements();) {
int[] oldentry = (int[])(e.nextElement());
if (oldentry.length == jj_expentry.length) {
exists = true;
for (int i = 0; i < jj_expentry.length; i++) {
if (oldentry[i] != jj_expentry[i]) {
exists = false;
break;
}
}
if (exists) break;
}
}
if (!exists) jj_expentries.addElement(jj_expentry); // <------- Missing
in 5.0
if (pos != 0) jj_lasttokens[(jj_endpos = pos) - 1] = kind;
}
In JavaCC 5.0, this has been changed. The representation of jj_expentries ha
sbeen changed from Vector to List, and the logic for initializing it has also
been changed, viz.
private void jj_add_error_token(int kind, int pos) {
if (pos >= 100) return;
if (pos == jj_endpos + 1) {
jj_lasttokens[jj_endpos++] = kind;
} else if (jj_endpos != 0) {
jj_expentry = new int[jj_endpos];
for (int i = 0; i < jj_endpos; i++) {
jj_expentry[i] = jj_lasttokens[i];
}
jj_entries_loop: for (java.util.Iterator it = jj_expentries.iterator();
it.hasNext();) {
int[] oldentry = (int[])(it.next());
if (oldentry.length == jj_expentry.length) {
for (int i = 0; i < jj_expentry.length; i++) {
if (oldentry[i] != jj_expentry[i]) {
continue jj_entries_loop;
}
}
jj_expentries.add(jj_expentry);
break jj_entries_loop;
}
}
if (pos != 0) jj_lasttokens[(jj_endpos = pos) - 1] = kind;
}
}
Now, the line "if (!exists) jj_expentries.addElement(jj_expentry);" from 4.0
has gone missing due to the new logic which removes the use of the exists
variable. Unfortunately this is broken, because the "jj_entries_loop" would
never get executed since jj_expentries is empty initially. So, it remains
uninitialized (empty), hence the problem of the empty encountered token.
We could fix this by building our own JavaCC, or by patching up the generated
code on each build.
I tried to use the new generated version of ParseException, but had to adjust
that code a bit too:
This stanza gave a bit more output than we want:
String retval = "Encountered \"";
Token tok = currentToken.next;
for (int i = 0; i < maxSize; i++) {
if (i != 0) retval += " ";
if (tok.kind == 0) {
retval += tokenImage[0];
break;
}
retval += " " + tokenImage[tok.kind];
retval += " \"";
retval += add_escapes(tok.image);
retval += " \"";
tok = tok.next;
}
which would show:
ij> values CHAR(DEC(67.21,4,2));
ERROR 42X01: Syntax error: Encountered " "dec" "DEC "" at line 1, column 13.
I found reverting that code to:
String retval = "Encountered \"";
Token tok = currentToken.next;
for (int i = 0; i < maxSize; i++) {
if (i != 0) retval += " ";
if (tok.kind == 0) {
retval += tokenImage[0];
break;
}
retval += add_escapes(tok.image);
tok = tok.next;
}
did the trick. I also had to comment out the code to show the possible expected
tokens, as done in our old checked in version of ParseException. With that
change and a fix to SQLParser, the output would look right for the cases I
attempted. I presume the "encountered" bug would apply for ij as well.
> Derby 10.7.1.1 : Build fails with javacc 5 (for Debian packaging)
> -----------------------------------------------------------------
>
> Key: DERBY-5125
> URL: https://issues.apache.org/jira/browse/DERBY-5125
> Project: Derby
> Issue Type: Improvement
> Components: Build tools
> Affects Versions: 10.7.1.1
> Environment: Debian Linux Linux inf-7590.int-evry.fr 2.6.32-5-amd64
> #1 SMP Wed Jan 12 03:40:32 UTC 2011 x86_64 GNU/Linux
> java -version (also tested with sun jvm)
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.7) (6b18-1.8.7-2)
> OpenJDK 64-Bit Server VM (build 16.0-b13, mixed mode)
> ant -version
> Apache Ant version 1.8.0 compiled on March 11 2010
> Reporter: Zied ABID
> Labels: build, libtool
> Fix For: 10.7.1.1, 10.7.1.4, 10.8.2.2, 11.0.0.0
>
> Attachments: d5125-javacc5-and-tests.diff, javacc5.diff
>
>
> I try to and Apache derby into Debian, so I begin to compile derby
> sources [1], derby sources provide each own libs and javacc is one of
> them. In Debian, javacc version is 5.0 [2].
> After some tries, I'm in a this situation : compilation success with the
> official [3] javacc 4.0 and fail with the 5.0 official one.
> I think that is related with javacc options passed in .jj files
> (ex. java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj lines 2-14)
> I don't know much about javacc, so I can't resolve unless modifying
> options blindly and I'm afraid that isn't enough ...
> Best regards,
> Zied
> [1] http://svn.apache.org/repos/asf/db/derby/code/tags/10.7.1.1/
> [2] http://packages.debian.org/squeeze/javacc
> [3] http://java.net/projects/javacc/downloads
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
