[
https://issues.apache.org/jira/browse/AVRO-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
fanguad updated AVRO-3706:
--------------------------
Description:
There is an issue when running the IDL tool if the IDL files have nested import
statements, and the path includes a space (or probably any character that must
be URL encoded). After the first level of import, the files can no longer be
found. Consider the following three files in a {{avro bug}} directory:
{{root.avdl}}
{code:java}
protocol Root {
import idl "level1.avdl";
} {code}
{{level1.avdl}}
{code:java}
protocol Level1 {
import idl "level2.avdl";
} {code}
{{level2.avdl}}
{code:java}
protocol Level2 {
} {code}
executing {{java -jar avro-tools-1.10.2.jar idl "avro bug/root.avdl"}} results
in the expected
{code:java}
{
"protocol" : "Root",
"namespace" : null,
"types" : [ ],
"messages" : { }
} {code}
whereas {{java -jar avro-tools-1.11.1.jar idl "avro bug/root.avdl"}} causes an
exception (the bug is also present in 1.11.0 with a slightly different stack
trace)
{code:java}
Exception in thread "main" org.apache.avro.compiler.idl.ParseException: Error
importing level2.avdl: java.io.FileNotFoundException: level2.avdl, at line 2,
column 26
at org.apache.avro.compiler.idl.Idl.error(Idl.java:88)
at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:537)
at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:535)
at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
at org.apache.avro.tool.IdlTool.run(IdlTool.java:61)
at org.apache.avro.tool.Main.run(Main.java:67)
at org.apache.avro.tool.Main.main(Main.java:56)
{code}
The same issue occurs if the files are located in "avro" but the current folder
has spaces (ie, the absolute path to the avdl files contains spaces).
The bug appears to have been introduced in this commit
[https://github.com/apache/avro/commit/3fe5e306eaa43bdc06cef291321a8c04058a6be9#diff-376865a18691674de38817b4eea7e64c2c3848094a4730030c67f57f54f1028a]
in {{idl.jj}} lines 130 and 169. The second time that line 130 is reached, it
has an absolute path that is already URL-encoded. In 1.10.2, the code {{new
File(input.getPath()).getParentFile()}} returns a String like
{{[file:///home/user/avro%20bug/]}} whereas in 1.11.0 and later {{new
File(input.getPath()).getParentFile().toURI()}} returns a String like
{{[file:///home/user/avro%2520bug]}} - note that URL encoding has happened
twice resulting in %2520 and there is no trailing slash. When resolved on line
169, this results in {{[file:///home/user/level2.avdl]}} instead of the correct
{{{}[file:///home/user/avro%20bug/level2.avdl]{}}}.
This bug can be easily worked around by changing the location of the avdl files
to one without spaces or other characters that must be URL encoded.
was:
There is an issue when running the IDL tool if the IDL files have nested import
statements, and the path includes a space (or probably any character that must
be URL encoded). After the first level of import, the files can no longer be
found. Consider the following three files in a {{avro bug}} directory:
{{root.avdl}}
{code:java}
protocol Root {
import idl "level1.avdl";
} {code}
{{level1.avdl}}
{code:java}
protocol Level1 {
import idl "level2.avdl";
} {code}
{{level2.avdl}}
{code:java}
protocol Level2 {
} {code}
executing {{java -jar avro-tools-1.10.2.jar idl "avro bug/root.avdl"}} results
in the expected
{code:java}
{
"protocol" : "Root",
"namespace" : null,
"types" : [ ],
"messages" : { }
} {code}
whereas {{java -jar avro-tools-1.11.1.jar idl "avro bug/root.avdl"}} causes an
exception (the bug is also present in 1.11.0 with a slightly different stack
trace)
{code:java}
Exception in thread "main" org.apache.avro.compiler.idl.ParseException: Error
importing level2.avdl: java.io.FileNotFoundException: level2.avdl, at line 2,
column 26
at org.apache.avro.compiler.idl.Idl.error(Idl.java:88)
at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:537)
at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:535)
at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
at org.apache.avro.tool.IdlTool.run(IdlTool.java:61)
at org.apache.avro.tool.Main.run(Main.java:67)
at org.apache.avro.tool.Main.main(Main.java:56)
{code}
The same issue occurs if the files are located in "avro" but the current folder
has spaces (ie, the absolute path to the avdl files contains spaces).
The bug appears to have been introduced in this commit
[https://github.com/apache/avro/commit/3fe5e306eaa43bdc06cef291321a8c04058a6be9#diff-376865a18691674de38817b4eea7e64c2c3848094a4730030c67f57f54f1028a]
in {{idl.jj}} lines 130 and 169. The second time that line 130 is reached, it
has an absolute path that is already URL-encoded. In 1.10.2, the code {{new
File(input.getPath()).getParentFile()}} returns a String like
{{[file:///home/user/avro%20bug/]}} whereas in 1.11.0 and later {{new
File(input.getPath()).getParentFile().toURI()}} returns a String like
{{[file:///home/user/avro%2520bug]}} - note that URL encoding has happened
twice resulting in %2520 and there is no trailing slash. When resolved on line
169, this results in {{[file:///home/user/level2.avdl]}} instead of the correct
{{{}[file:///home/user/avro%20bug/level2.avdl]{}}}.
This bug can be easily worked around by changing the location of the avdl files
to one without spaces or other characters that must be URL encoded.
> AVDL nested imports cannot be resolved if path contains spaces
> --------------------------------------------------------------
>
> Key: AVRO-3706
> URL: https://issues.apache.org/jira/browse/AVRO-3706
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.11.0, 1.11.1
> Reporter: fanguad
> Priority: Minor
>
> There is an issue when running the IDL tool if the IDL files have nested
> import statements, and the path includes a space (or probably any character
> that must be URL encoded). After the first level of import, the files can no
> longer be found. Consider the following three files in a {{avro bug}}
> directory:
> {{root.avdl}}
> {code:java}
> protocol Root {
> import idl "level1.avdl";
> } {code}
> {{level1.avdl}}
> {code:java}
> protocol Level1 {
> import idl "level2.avdl";
> } {code}
> {{level2.avdl}}
> {code:java}
> protocol Level2 {
> } {code}
> executing {{java -jar avro-tools-1.10.2.jar idl "avro bug/root.avdl"}}
> results in the expected
>
> {code:java}
> {
> "protocol" : "Root",
> "namespace" : null,
> "types" : [ ],
> "messages" : { }
> } {code}
> whereas {{java -jar avro-tools-1.11.1.jar idl "avro bug/root.avdl"}} causes
> an exception (the bug is also present in 1.11.0 with a slightly different
> stack trace)
>
> {code:java}
> Exception in thread "main" org.apache.avro.compiler.idl.ParseException: Error
> importing level2.avdl: java.io.FileNotFoundException: level2.avdl, at line 2,
> column 26
> at org.apache.avro.compiler.idl.Idl.error(Idl.java:88)
> at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:537)
> at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
> at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
> at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
> at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:535)
> at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
> at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
> at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
> at org.apache.avro.tool.IdlTool.run(IdlTool.java:61)
> at org.apache.avro.tool.Main.run(Main.java:67)
> at org.apache.avro.tool.Main.main(Main.java:56)
> {code}
>
> The same issue occurs if the files are located in "avro" but the current
> folder has spaces (ie, the absolute path to the avdl files contains spaces).
>
> The bug appears to have been introduced in this commit
> [https://github.com/apache/avro/commit/3fe5e306eaa43bdc06cef291321a8c04058a6be9#diff-376865a18691674de38817b4eea7e64c2c3848094a4730030c67f57f54f1028a]
> in {{idl.jj}} lines 130 and 169. The second time that line 130 is reached,
> it has an absolute path that is already URL-encoded. In 1.10.2, the code
> {{new File(input.getPath()).getParentFile()}} returns a String like
> {{[file:///home/user/avro%20bug/]}} whereas in 1.11.0 and later {{new
> File(input.getPath()).getParentFile().toURI()}} returns a String like
> {{[file:///home/user/avro%2520bug]}} - note that URL encoding has happened
> twice resulting in %2520 and there is no trailing slash. When resolved on
> line 169, this results in {{[file:///home/user/level2.avdl]}} instead of the
> correct {{{}[file:///home/user/avro%20bug/level2.avdl]{}}}.
>
> This bug can be easily worked around by changing the location of the avdl
> files to one without spaces or other characters that must be URL encoded.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
