subject:"Stack overflow from self referencing Avro schema"

Re: Stack overflow from self referencing Avro schema

2016-03-22 Thread David Kim

Hi Robert!

Thank you! :)

David

On Tue, Mar 22, 2016 at 7:59 AM, Robert Metzger  wrote:

> Hey David,
>
> FLINK-3602 has been merged to master.
>
> On Fri, Mar 11, 2016 at 5:11 PM, David Kim <
> david@braintreepayments.com> wrote:
>
>> Thanks Stephan! :)
>>
>> On Thu, Mar 10, 2016 at 11:06 AM, Stephan Ewen  wrote:
>>
>>> The following issue should track that.
>>> https://issues.apache.org/jira/browse/FLINK-3602
>>>
>>> @Niels: Thanks for looking into this. At this point, I think it may
>>> actually be a Flink issue, since it concerns the interaction of Avro and
>>> Flink's TypeInformation.
>>>
>>> On Thu, Mar 10, 2016 at 6:00 PM, Stephan Ewen  wrote:
>>>
 Hi!

 I think that is a TypeExtractor bug. It may actually be a bug for all
 recursive types.
 Let's check this and come up with a fix...

 Greetings,
 Stephan


 On Thu, Mar 10, 2016 at 4:11 PM, David Kim <
 david@braintreepayments.com> wrote:

> Hello!
>
> Just wanted to check up on this again. Has anyone else seen this
> before or have any suggestions?
>
> Thanks!
> David
>
> On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
> david@braintreepayments.com> wrote:
>
>> Hello all,
>>
>> I'm running into a StackOverflowError using flink 1.0.0. I have an
>> Avro schema that has a self reference. For example:
>>
>> item.avsc
>>
>> {
>>
>>   "namespace": "..."
>>
>>   "type": "record"
>>   "name": "Item",
>>   "fields": [
>> {
>>   "name": "parent"
>>   "type": ["null, "Item"]
>> }
>>   ]
>> }
>>
>>
>> When running my flink job, I'm running into the follow error:
>>
>> Exception in thread "Thread-94" java.lang.StackOverflowError
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>
>>
>> Interestingly if I change the type to an Avro array in the schema,
>> this error is not thrown.
>>
>> Thanks!
>> David
>>
>
>
>
> --
> Note: this information is confidential. It is prohibited to share,
> post online or otherwise publicize without Braintree's prior written
> consent.
>


>>>
>>
>>
>> --
>> Note: this information is confidential. It is prohibited to share, post
>> online or otherwise publicize without Braintree's prior written consent.
>>
>
>


-- 
Note: this information is confidential. It is prohibited to share, post
online or otherwise publicize without Braintree's prior written consent.

Re: Stack overflow from self referencing Avro schema

2016-03-22 Thread Robert Metzger

Hey David,

FLINK-3602 has been merged to master.

On Fri, Mar 11, 2016 at 5:11 PM, David Kim 
wrote:

> Thanks Stephan! :)
>
> On Thu, Mar 10, 2016 at 11:06 AM, Stephan Ewen  wrote:
>
>> The following issue should track that.
>> https://issues.apache.org/jira/browse/FLINK-3602
>>
>> @Niels: Thanks for looking into this. At this point, I think it may
>> actually be a Flink issue, since it concerns the interaction of Avro and
>> Flink's TypeInformation.
>>
>> On Thu, Mar 10, 2016 at 6:00 PM, Stephan Ewen  wrote:
>>
>>> Hi!
>>>
>>> I think that is a TypeExtractor bug. It may actually be a bug for all
>>> recursive types.
>>> Let's check this and come up with a fix...
>>>
>>> Greetings,
>>> Stephan
>>>
>>>
>>> On Thu, Mar 10, 2016 at 4:11 PM, David Kim <
>>> david@braintreepayments.com> wrote:
>>>
 Hello!

 Just wanted to check up on this again. Has anyone else seen this before
 or have any suggestions?

 Thanks!
 David

 On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
 david@braintreepayments.com> wrote:

> Hello all,
>
> I'm running into a StackOverflowError using flink 1.0.0. I have an
> Avro schema that has a self reference. For example:
>
> item.avsc
>
> {
>
>   "namespace": "..."
>
>   "type": "record"
>   "name": "Item",
>   "fields": [
> {
>   "name": "parent"
>   "type": ["null, "Item"]
> }
>   ]
> }
>
>
> When running my flink job, I'm running into the follow error:
>
> Exception in thread "Thread-94" java.lang.StackOverflowError
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>   at 
> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>   at 
> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>   at 
> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>   at 
> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>   at 
> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>   at 
> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>
>
> Interestingly if I change the type to an Avro array in the schema,
> this error is not thrown.
>
> Thanks!
> David
>



 --
 Note: this information is confidential. It is prohibited to share, post
 online or otherwise publicize without Braintree's prior written consent.

>>>
>>>
>>
>
>
> --
> Note: this information is confidential. It is prohibited to share, post
> online or otherwise publicize without Braintree's prior written consent.
>

Re: Stack overflow from self referencing Avro schema

2016-03-11 Thread David Kim

Thanks Stephan! :)

On Thu, Mar 10, 2016 at 11:06 AM, Stephan Ewen  wrote:

> The following issue should track that.
> https://issues.apache.org/jira/browse/FLINK-3602
>
> @Niels: Thanks for looking into this. At this point, I think it may
> actually be a Flink issue, since it concerns the interaction of Avro and
> Flink's TypeInformation.
>
> On Thu, Mar 10, 2016 at 6:00 PM, Stephan Ewen  wrote:
>
>> Hi!
>>
>> I think that is a TypeExtractor bug. It may actually be a bug for all
>> recursive types.
>> Let's check this and come up with a fix...
>>
>> Greetings,
>> Stephan
>>
>>
>> On Thu, Mar 10, 2016 at 4:11 PM, David Kim <
>> david@braintreepayments.com> wrote:
>>
>>> Hello!
>>>
>>> Just wanted to check up on this again. Has anyone else seen this before
>>> or have any suggestions?
>>>
>>> Thanks!
>>> David
>>>
>>> On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
>>> david@braintreepayments.com> wrote:
>>>
 Hello all,

 I'm running into a StackOverflowError using flink 1.0.0. I have an Avro
 schema that has a self reference. For example:

 item.avsc

 {

   "namespace": "..."

   "type": "record"
   "name": "Item",
   "fields": [
 {
   "name": "parent"
   "type": ["null, "Item"]
 }
   ]
 }


 When running my flink job, I'm running into the follow error:

 Exception in thread "Thread-94" java.lang.StackOverflowError
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
at 
 org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
at 
 org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
at 
 org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
at 
 org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
at 
 org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
at 
 org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)


 Interestingly if I change the type to an Avro array in the schema,
 this error is not thrown.

 Thanks!
 David

>>>
>>>
>>>
>>> --
>>> Note: this information is confidential. It is prohibited to share, post
>>> online or otherwise publicize without Braintree's prior written consent.
>>>
>>
>>
>


-- 
Note: this information is confidential. It is prohibited to share, post
online or otherwise publicize without Braintree's prior written consent.

Re: Stack overflow from self referencing Avro schema

2016-03-10 Thread Stephan Ewen

The following issue should track that.
https://issues.apache.org/jira/browse/FLINK-3602

@Niels: Thanks for looking into this. At this point, I think it may
actually be a Flink issue, since it concerns the interaction of Avro and
Flink's TypeInformation.

On Thu, Mar 10, 2016 at 6:00 PM, Stephan Ewen  wrote:

> Hi!
>
> I think that is a TypeExtractor bug. It may actually be a bug for all
> recursive types.
> Let's check this and come up with a fix...
>
> Greetings,
> Stephan
>
>
> On Thu, Mar 10, 2016 at 4:11 PM, David Kim <
> david@braintreepayments.com> wrote:
>
>> Hello!
>>
>> Just wanted to check up on this again. Has anyone else seen this before
>> or have any suggestions?
>>
>> Thanks!
>> David
>>
>> On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
>> david@braintreepayments.com> wrote:
>>
>>> Hello all,
>>>
>>> I'm running into a StackOverflowError using flink 1.0.0. I have an Avro
>>> schema that has a self reference. For example:
>>>
>>> item.avsc
>>>
>>> {
>>>
>>>   "namespace": "..."
>>>
>>>   "type": "record"
>>>   "name": "Item",
>>>   "fields": [
>>> {
>>>   "name": "parent"
>>>   "type": ["null, "Item"]
>>> }
>>>   ]
>>> }
>>>
>>>
>>> When running my flink job, I'm running into the follow error:
>>>
>>> Exception in thread "Thread-94" java.lang.StackOverflowError
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>> at 
>>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>> at 
>>> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>> at 
>>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>> at 
>>> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>> at 
>>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>> at 
>>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>>
>>>
>>> Interestingly if I change the type to an Avro array in the schema, this
>>> error is not thrown.
>>>
>>> Thanks!
>>> David
>>>
>>
>>
>>
>> --
>> Note: this information is confidential. It is prohibited to share, post
>> online or otherwise publicize without Braintree's prior written consent.
>>
>
>

Re: Stack overflow from self referencing Avro schema

2016-03-10 Thread Stephan Ewen

Hi!

I think that is a TypeExtractor bug. It may actually be a bug for all
recursive types.
Let's check this and come up with a fix...

Greetings,
Stephan


On Thu, Mar 10, 2016 at 4:11 PM, David Kim 
wrote:

> Hello!
>
> Just wanted to check up on this again. Has anyone else seen this before or
> have any suggestions?
>
> Thanks!
> David
>
> On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
> david@braintreepayments.com> wrote:
>
>> Hello all,
>>
>> I'm running into a StackOverflowError using flink 1.0.0. I have an Avro
>> schema that has a self reference. For example:
>>
>> item.avsc
>>
>> {
>>
>>   "namespace": "..."
>>
>>   "type": "record"
>>   "name": "Item",
>>   "fields": [
>> {
>>   "name": "parent"
>>   "type": ["null, "Item"]
>> }
>>   ]
>> }
>>
>>
>> When running my flink job, I'm running into the follow error:
>>
>> Exception in thread "Thread-94" java.lang.StackOverflowError
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>
>>
>> Interestingly if I change the type to an Avro array in the schema, this
>> error is not thrown.
>>
>> Thanks!
>> David
>>
>
>
>
> --
> Note: this information is confidential. It is prohibited to share, post
> online or otherwise publicize without Braintree's prior written consent.
>

Re: Stack overflow from self referencing Avro schema

2016-03-10 Thread Niels Basjes

Hi,

Please try to reproduce the problem in a simple (commandline) Java
application (i.e. without Flink and such, just Avro).
If you can reproduce it with Avro 1.8.0 then please file a bug report
(preferable with the simplest reproduction path you can come up with) via.
https://issues.apache.org/jira/browse/AVRO/

Thanks

Niels Basjes

On Thu, Mar 10, 2016 at 4:11 PM, David Kim 
wrote:

> Hello!
>
> Just wanted to check up on this again. Has anyone else seen this before or
> have any suggestions?
>
> Thanks!
> David
>
> On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
> david@braintreepayments.com> wrote:
>
>> Hello all,
>>
>> I'm running into a StackOverflowError using flink 1.0.0. I have an Avro
>> schema that has a self reference. For example:
>>
>> item.avsc
>>
>> {
>>
>>   "namespace": "..."
>>
>>   "type": "record"
>>   "name": "Item",
>>   "fields": [
>> {
>>   "name": "parent"
>>   "type": ["null, "Item"]
>> }
>>   ]
>> }
>>
>>
>> When running my flink job, I'm running into the follow error:
>>
>> Exception in thread "Thread-94" java.lang.StackOverflowError
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>  at 
>> org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>  at 
>> org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>
>>
>> Interestingly if I change the type to an Avro array in the schema, this
>> error is not thrown.
>>
>> Thanks!
>> David
>>
>
>
>
> --
> Note: this information is confidential. It is prohibited to share, post
> online or otherwise publicize without Braintree's prior written consent.
>



-- 
Best regards / Met vriendelijke groeten,

Niels Basjes

Stack overflow from self referencing Avro schema

2016-03-08 Thread David Kim

Hello all,

I'm running into a StackOverflowError using flink 1.0.0. I have an Avro
schema that has a self reference. For example:

item.avsc

{

  "namespace": "..."

  "type": "record"
  "name": "Item",
  "fields": [
{
  "name": "parent"
  "type": ["null, "Item"]
}
  ]
}


When running my flink job, I'm running into the follow error:

Exception in thread "Thread-94" java.lang.StackOverflowError
at 
org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
at 
org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
at 
org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
at 
org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
at 
org.apache.flink.api.java.typeutils.AvroTypeInfo.(AvroTypeInfo.java:48)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
at 
org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
at 
org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)


Interestingly if I change the type to an Avro array in the schema, this
error is not thrown.

Thanks!
David

Re: Stack overflow from self referencing Avro schema

Re: Stack overflow from self referencing Avro schema

Re: Stack overflow from self referencing Avro schema

Re: Stack overflow from self referencing Avro schema

Re: Stack overflow from self referencing Avro schema

Re: Stack overflow from self referencing Avro schema

Stack overflow from self referencing Avro schema

7 matches

Site Navigation

Mail list logo

Footer information