[
https://issues.apache.org/jira/browse/AVRO-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590807#comment-14590807
]
Ryan Blue commented on AVRO-1682:
---------------------------------
Hi [~jacopo.moronato], thanks for reporting the issue. This actually isn't too
surprising because we just added support for decimals in AVRO-1497. The next
Avro release, 1.7.8, should include the Avro-side support and then libraries
like Pig can start integrating with it. That can't happen quite yet since the
release isn't out, but the Avro side is complete and just waiting for 1.7.8. I
think the next step is to report this to the Pig community so they can add the
integration that is needed in that project. Would you mind opening an issue for
the Pig project? Then I'll link this one to it and close this since it's done!
Thanks!
> Support for bigdecimal type
> ---------------------------
>
> Key: AVRO-1682
> URL: https://issues.apache.org/jira/browse/AVRO-1682
> Project: Avro
> Issue Type: Wish
> Affects Versions: 1.7.6
> Environment: Apache Pig version 0.12.0-cdh5.4.2
> Reporter: Jacopo Moronato
> Priority: Minor
>
> I'm using a pig script to:
> 1) load a semicolon-separate file
> 2) convert some chararray fields to bigdecimal
> 3) store into avro file.
> Point 3 fails raising this exception:
>
> at
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:75)
> at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
> at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
> at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
> at
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:311)
> at org.apache.pig.PigServer.compilePp(PigServer.java:1392)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1317)
> at org.apache.pig.PigServer.execute(PigServer.java:1309)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:387)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:365)
> at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:478)
> at org.apache.pig.PigRunner.run(PigRunner.java:49)
> at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:285)
> at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:228)
> at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:46)
> at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Don't know how to encode type bigdecimal in
> schema
>
> at
> org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities.resourceFieldSchemaToAvroSchema(AvroStorageSchemaConversionUtilities.java:511)
> at
> org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities.resourceSchemaToAvroSchema(AvroStorageSchemaConversionUtilities.java:336)
> at org.apache.pig.builtin.AvroStorage.checkSchema(AvroStorage.java:486)
> at
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:65)
> ... 48 more
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
