[
https://issues.apache.org/jira/browse/PIG-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheolsoo Park updated PIG-3377:
-------------------------------
Description:
The following example demonstrates the issue:
{code}
a = LOAD 'foo' AS (m:map[]);
STORE a INTO 'bar' USING AvroStorage();
{code}
This fails with the following error:
{code}
java.lang.NullPointerException
at
org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities.resourceFieldSchemaToAvroSchema(AvroStorageSchemaConversionUtilities.java:462)
at
org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities.resourceSchemaToAvroSchema(AvroStorageSchemaConversionUtilities.java:335)
at org.apache.pig.builtin.AvroStorage.checkSchema(AvroStorage.java:472)
{code}
Similarly, untyped bag causes the following error:
{code}
Caused by: java.lang.NullPointerException
at org.apache.avro.Schema$ArraySchema.toJson(Schema.java:722)
...
at org.apache.avro.Schema.getElementType(Schema.java:256)
at
org.apache.pig.builtin.AvroStorage.setOutputAvroSchema(AvroStorage.java:491)
{code}
The problem is that AvroStorage cannot derive the output schema from untyped
map/bag/tuple. When type is not defined, it should be assumed as bytearray.
was:
The following example demonstrates the issue:
{code}
a = LOAD 'foo' AS (m:map[]);
STORE a INTO 'bar' USING AvroStorage();
{code}
This fails with the following error:
{code}
java.lang.NullPointerException
at
org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities.resourceFieldSchemaToAvroSchema(AvroStorageSchemaConversionUtilities.java:462)
at
org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities.resourceSchemaToAvroSchema(AvroStorageSchemaConversionUtilities.java:335)
at org.apache.pig.builtin.AvroStorage.checkSchema(AvroStorage.java:472)
{code}
Summary: New AvroStorage throws NPE when storing untyped map/array/bag
(was: New AvroStorage fails with untyped map)
> New AvroStorage throws NPE when storing untyped map/array/bag
> -------------------------------------------------------------
>
> Key: PIG-3377
> URL: https://issues.apache.org/jira/browse/PIG-3377
> Project: Pig
> Issue Type: Bug
> Components: internal-udfs
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.12
>
>
> The following example demonstrates the issue:
> {code}
> a = LOAD 'foo' AS (m:map[]);
> STORE a INTO 'bar' USING AvroStorage();
> {code}
> This fails with the following error:
> {code}
> java.lang.NullPointerException
> at
> org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities.resourceFieldSchemaToAvroSchema(AvroStorageSchemaConversionUtilities.java:462)
> at
> org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities.resourceSchemaToAvroSchema(AvroStorageSchemaConversionUtilities.java:335)
> at org.apache.pig.builtin.AvroStorage.checkSchema(AvroStorage.java:472)
> {code}
> Similarly, untyped bag causes the following error:
> {code}
> Caused by: java.lang.NullPointerException
> at org.apache.avro.Schema$ArraySchema.toJson(Schema.java:722)
> ...
> at org.apache.avro.Schema.getElementType(Schema.java:256)
> at
> org.apache.pig.builtin.AvroStorage.setOutputAvroSchema(AvroStorage.java:491)
> {code}
> The problem is that AvroStorage cannot derive the output schema from untyped
> map/bag/tuple. When type is not defined, it should be assumed as bytearray.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
