[
https://issues.apache.org/jira/browse/AVRO-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong updated AVRO-3699:
-----------------------------------
Fix Version/s: 1.13.0
(was: 1.12.0)
> avro-tools dependency on hadoop
> -------------------------------
>
> Key: AVRO-3699
> URL: https://issues.apache.org/jira/browse/AVRO-3699
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.11.1
> Reporter: Miguel Costa
> Priority: Major
> Fix For: 1.13.0
>
>
> Hi all, this is not something that hinders the usage of the avro tool but I
> don't understand why this dependency exists.
> Seems to be more for convenience than anything but for example when I ran the
> Avro-tool in java I always get this warnings that are a bit annoying.
>
> {code:java}
> 23/01/05 18:21:24 WARN util.Shell: Did not find winutils.exe: {}
> java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and
> hadoop.home.dir are unset. -see https://wiki.apache.org/hadoop/WindowsProblems
> at org.apache.hadoop.util.Shell.fileNotFoundException(Shell.java:547)
> at org.apache.hadoop.util.Shell.getHadoopHomeDir(Shell.java:568)
> at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:591)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:688)
> at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
> at
> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3741)
> at
> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3736)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3520)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:288)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
> at org.apache.avro.tool.Util.openFromFS(Util.java:88)
> at org.apache.avro.tool.Util.fileOrStdin(Util.java:64)
> at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:87)
> at org.apache.avro.tool.Main.run(Main.java:67)
> at org.apache.avro.tool.Main.main(Main.java:56)
> Caused by: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are
> unset.
> at org.apache.hadoop.util.Shell.checkHadoopHomeInner(Shell.java:467)
> at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:438)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:515)
> ... 13 more
> 23/01/05 18:21:24 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Output:
> {"name":"r","age":{"int":906},"gender":"MALE","address":{"zipcode":631}}
> {"name":"oYBuz","age":{"int":255},"gender":"FEMALE","address":{"zipcode":690}}
> {"name":"xKMgdHyLw","age":null,"gender":"FEMALE","address":{"zipcode":304}}
> {"name":"MPPsYun","age":null,"gender":"MALE","address":{"zipcode":875}}
> {"name":"XTrq","age":{"int":167},"gender":"MALE","address":{"zipcode":973}}
> {"name":"g","age":null,"gender":"FEMALE","address":{"zipcode":351}}
> {"name":"ynx","age":null,"gender":"FEMALE","address":{"zipcode":525}}
> {"name":"xPFZ","age":null,"gender":"FEMALE","address":{"zipcode":921}}
> {"name":"bRHCLEwdglb","age":{"int":541},"gender":"MALE","address":{"zipcode":123}}
> {"name":"UVcDVhxpyCziyBSiRasp","age":null,"gender":"FEMALE","address":{"zipcode":53}}
> {code}
>
>
> It would be great if this could be improved in a future.
> It might just be that there are some hard requirements in other parts of the
> tool that require this, so in that case this issue makes no sense.
> But I did try to use the util.logging or log4j config files to ignore this
> warnings but I was not so lucky.
> something like:
> {code:java}
> java -Djava.util.logging.config.file=utillog.properties -jar
> avro-tools-1.11.1.jar getschema avro/person-10.avro{code}
>
> {code:java}
> java -Dlog4j.configurationFile=log4j.properties -jar avro-tools-1.11.1.jar
> getschema avro/person-10.avro{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
