[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[David Carlton (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847773#comment-16847773
 ] 

David Carlton commented on AVRO-1723:
-

[~zolyfarkas] thanks!

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Apache Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
>Priority: Major
> Fix For: 1.9.0
>
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847769#comment-16847769
 ] 

Zoltan Farkas commented on AVRO-1723:
-

[~davidcarltonsumo] should be part of 1.9.0, I see the changes in the 
release-1.9.0 tag.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Apache Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
>Priority: Major
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[David Carlton (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847668#comment-16847668
 ] 

David Carlton commented on AVRO-1723:
-

Is this included in Avro 1.9.0?  It's not showing up in the change list 
https://s.apache.org/avro190, but I'm not sure if that's correct information or 
if it's just not tagged correctly in Jira.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Apache Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
>Priority: Major
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017676#comment-16017676
 ] 

ASF subversion and git services commented on AVRO-1723:
---

Commit 1f4ec52df822f5b5689cb15c8d005c224a271b6d in avro's branch 
refs/heads/master from [~zolyfarkas]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=1f4ec52 ]

Merge pull request #2 from thiru-apache/thiru-AVRO-1723-A

Added a unit-test for forward referencing of named types

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017675#comment-16017675
 ] 

ASF subversion and git services commented on AVRO-1723:
---

Commit 008cd8a209564a958e9065a5f6ff05526a41f4a5 in avro's branch 
refs/heads/master from [~zolyfarkas]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=008cd8a ]

Merge pull request #1 from thiru-apache/thiru-AVRO-1723

Added more tests and fixed a couple of bugs. Also formatted the code

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas]

Thiru, when you get a chance can you look at the last version of the PR? 

thank you

—Z

> On May 14, 2017, at 7:50 PM, Zoltan Farkas (JIRA)  wrote:
> 
> 
>[ 
> https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009886#comment-16009886
>  ] 
> 
> Zoltan Farkas commented on AVRO-1723:
> -
> 
> [~thiru_mg] I have updated the PR to redo the guava code and shaded it just 
> like the main java lib.
> 
> build.sh test passes now, please review, and let me know if this looks good. 
> 
> thank you
> 
>> Add support for forward declarations in avro IDL
>> 
>> 
>>Key: AVRO-1723
>>URL: https://issues.apache.org/jira/browse/AVRO-1723
>>Project: Avro
>> Issue Type: Improvement
>>   Affects Versions: 1.8.0
>>   Reporter: Zoltan Farkas
>>   Assignee: Zoltan Farkas
>>Attachments: AVRO-1723.patch
>> 
>> 
>> Currently Recursive data structures like:
>> record SampleNode {
>>   int count = 0;
>>   array samples = [];
>> }
>> record SamplePair {
>> string name;
>> SampleNode node;
>> }
>> It is not possible to declare in IDL,
>> however it is possible to declare in avsc (with fix from 
>> https://issues.apache.org/jira/browse/AVRO-1667 )
>> It is actually not complicated to implement, here is some detail on a 
>> possible implementation:
>> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
>> This would close a capability gap with google protocol buffers...
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009886#comment-16009886
 ] 

Zoltan Farkas commented on AVRO-1723:
-

[~thiru_mg] I have updated the PR to redo the guava code and shaded it just 
like the main java lib.

build.sh test passes now, please review, and let me know if this looks good. 

thank you

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Thiruvalluvan M. G. (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009655#comment-16009655
 ] 

Thiruvalluvan M. G. commented on AVRO-1723:
---

Hi [~zolyfarkas] I had one problem and couldn't come up with a quick solution. 
{{./build.sh test}} from {{lang/java}} fails because the compiler does not see 
the Guava classes. I thought adding the classes in 
{{lang/java/guava/src/main/java/org/apache/avro/GuavaClasses.java}} would fix 
the problem. It doesn't.

If you have time, can you take a stab at it?

Thank you

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas]

Thiruvalluvan, anything else we need to do before merging this in?

let me know

thank you

—Z

> On May 10, 2017, at 2:45 PM, Zoltan Farkas  
> wrote:
> 
> Anything else left to do?
> 
> let me know
> 
> thank you
> 
> —Z
> 
>> On Apr 30, 2017, at 11:16 PM, Thiruvalluvan M. G. (JIRA)  
>> wrote:
>> 
>> 
>>   [ 
>> https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990573#comment-15990573
>>  ] 
>> 
>> Thiruvalluvan M. G. commented on AVRO-1723:
>> ---
>> 
>> [~zolyfarkas] Great. I just added a test for forward reference: 
>> https://github.com/zolyfarkas/avro/pull/2. Please merge it to your branch 
>> when you can.
>> 
>> I'll pull your patch in two days.
>> 
>>> Add support for forward declarations in avro IDL
>>> 
>>> 
>>>   Key: AVRO-1723
>>>   URL: https://issues.apache.org/jira/browse/AVRO-1723
>>>   Project: Avro
>>>Issue Type: Improvement
>>>  Affects Versions: 1.8.0
>>>  Reporter: Zoltan Farkas
>>>  Assignee: Zoltan Farkas
>>>   Attachments: AVRO-1723.patch
>>> 
>>> 
>>> Currently Recursive data structures like:
>>> record SampleNode {
>>>  int count = 0;
>>>  array samples = [];
>>> }
>>> record SamplePair {
>>> string name;
>>> SampleNode node;
>>> }
>>> It is not possible to declare in IDL,
>>> however it is possible to declare in avsc (with fix from 
>>> https://issues.apache.org/jira/browse/AVRO-1667 )
>>> It is actually not complicated to implement, here is some detail on a 
>>> possible implementation:
>>> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
>>> This would close a capability gap with google protocol buffers...
>> 
>> 
>> 
>> --
>> This message was sent by Atlassian JIRA
>> (v6.3.15#6346)
> 

Re: [jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas]

Anything else left to do?

let me know

thank you

—Z

> On Apr 30, 2017, at 11:16 PM, Thiruvalluvan M. G. (JIRA)  
> wrote:
> 
> 
>[ 
> https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990573#comment-15990573
>  ] 
> 
> Thiruvalluvan M. G. commented on AVRO-1723:
> ---
> 
> [~zolyfarkas] Great. I just added a test for forward reference: 
> https://github.com/zolyfarkas/avro/pull/2. Please merge it to your branch 
> when you can.
> 
> I'll pull your patch in two days.
> 
>> Add support for forward declarations in avro IDL
>> 
>> 
>>Key: AVRO-1723
>>URL: https://issues.apache.org/jira/browse/AVRO-1723
>>Project: Avro
>> Issue Type: Improvement
>>   Affects Versions: 1.8.0
>>   Reporter: Zoltan Farkas
>>   Assignee: Zoltan Farkas
>>Attachments: AVRO-1723.patch
>> 
>> 
>> Currently Recursive data structures like:
>> record SampleNode {
>>   int count = 0;
>>   array samples = [];
>> }
>> record SamplePair {
>> string name;
>> SampleNode node;
>> }
>> It is not possible to declare in IDL,
>> however it is possible to declare in avsc (with fix from 
>> https://issues.apache.org/jira/browse/AVRO-1667 )
>> It is actually not complicated to implement, here is some detail on a 
>> possible implementation:
>> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
>> This would close a capability gap with google protocol buffers...
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997230#comment-15997230
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Let me know if I need to do anything else.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990923#comment-15990923
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Looks good! thank you!

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Thiruvalluvan M. G. (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990573#comment-15990573
 ] 

Thiruvalluvan M. G. commented on AVRO-1723:
---

[~zolyfarkas] Great. I just added a test for forward reference: 
https://github.com/zolyfarkas/avro/pull/2. Please merge it to your branch when 
you can.

I'll pull your patch in two days.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990376#comment-15990376
 ] 

Zoltan Farkas commented on AVRO-1723:
-

[~thiru_mg] Thanks for review.

your cleanup + fixes looks good.

Regarding the order of definition of named schema elements, it will not matter 
anymore.

This implementation will resolve all named elements not only records.

I just tested the following:

{code}
   /* Name Value record */
   record ANameValue {
  /** the name */
  string name;
  /** the value */
  string value;
  /* is the value a json object */
  ValueType type = "PLAIN";
   }
   enum ValueType {JSON, BASE64BIN, PLAIN}
{code}

while this is not supported with AVRO without this change.
It will work just fine from now on. 
Worthwhile adding a unit test.

What we should also do add a jira to move the generic schema walker we created 
for this into the core avro lib and see if there are other places this can be 
re-used as [~rdblue] suggested:

"Also, you might be able to simplify what's happening here by using the visitor 
pattern. I've been thinking that it would be helpful to have an internal 
SchemaVisitor for this kind of operation. That would cut down on the size of 
these methods and separate the recursion from the logic you're trying to 
implement. This may be a good time to add it to the core library."

This is the reason of existence of the CloningVisitor, to add one other use 
case for this generic schema walker... 

thank you!

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Thiruvalluvan M. G. (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990340#comment-15990340
 ] 

Thiruvalluvan M. G. commented on AVRO-1723:
---

[~zolyfarkas] I reviewed your work. It looks great. A few formatting issues and 
conventions. There was one bug related to the use of Dequeue's {{getLast()}}. 
Instead of asking you to make further changes (you have done several rounds 
already), I made a pull request into your git repo. 
https://github.com/zolyfarkas/avro/pull/1 Please see if my modifications are 
good enough. If you like the changes, please merge them to your branch and 
rebase it to the current master of Apache repo. If you don't like my changes or 
want to modify parts of them, please go ahead.

I'll wait for a couple of days for others to comment. If there are no comments 
or objections from the community, I'll merge your pull request.

Thank you for taking this up. This is a very useful improvement to the Schema 
parser.

One more thing. You mention that the order of definition of named schema 
elements in the IDL file doesn't matter any more. I think, while it is true for 
Record types, the other named types such as Enum, Fixed must be defined at the 
first occurrence. This is not a problem because they don't recurse. Probably in 
a future patch we can take this restriction out as well.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983499#comment-15983499
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Can somebody review the pull request?

thank you

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854413#comment-15854413
 ] 

Zoltan Farkas commented on AVRO-1723:
-

 "cloning" a field will not work in certain situations when defaultVal() is 
broken.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234067#comment-15234067
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Ryan, I have updated the pull request based on your comments.
Please take a look and let me know if its current state isa acceptable.

thank you

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199930#comment-15199930
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Ryan, here is a pull request based on the latest HEAD on master:

https://github.com/apache/avro/pull/79

let me know if you have any questions.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199925#comment-15199925
 ] 

ASF GitHub Bot commented on AVRO-1723:
--

GitHub user zolyfarkas opened a pull request:

https://github.com/apache/avro/pull/79

AVRO-1723

Implementation for IDL forward declarations... actually declaration order 
does not matter anymore...

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zolyfarkas/avro trunk-AVRO-1723

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/avro/pull/79.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #79


commit 1c5205ef3f1516fa5f3a323057139bffd5758182
Author: Zoltan Farkas 
Date:   2015-11-18T22:44:50Z

[add] AVRO-1723 and AVRO-1667

commit 1f6fa7b79efde2eb999de34334b686a77b23e46a
Author: Zoltan Farkas 
Date:   2015-11-18T22:44:50Z

[add] AVRO-1723 and AVRO-1667

commit 57bea12e3a0860958b2060cab4f1193391584839
Author: Zoltan Farkas 
Date:   2015-12-09T22:28:36Z

Merge origin/trunk-AVRO-1723 into trunk-AVRO-1723

commit fd6c4228fbecc6eff3e3ecd5ee27d2ceb46339ff
Author: Zoltan Farkas 
Date:   2015-11-18T22:44:50Z

[add] AVRO-1723 and AVRO-1667

commit c299da64ee53de57ea34e589b858d7fe8ada72fd
Author: Zoltan Farkas 
Date:   2016-03-17T14:03:24Z

Merge origin/trunk-AVRO-1723 into trunk-AVRO-1723

Conflicts:
lang/java/avro/src/main/java/org/apache/avro/io/parsing/Symbol.java

commit 3ff1fa8cac828bfaabd082cb82d7ea3a3827f02f
Author: Zoltan Farkas 
Date:   2016-03-17T14:18:02Z

Merge origin/trunk-AVRO-1723 into trunk-AVRO-1723

Conflicts:
lang/java/avro/src/main/java/org/apache/avro/io/parsing/Symbol.java

commit ce16a3fc9e493d50425ecc697e397c486e9c62dd
Author: Zoltan Farkas 
Date:   2016-03-17T14:28:44Z

Merge origin/trunk-AVRO-1723 into trunk-AVRO-1723

Conflicts:
lang/java/avro/src/main/java/org/apache/avro/io/parsing/Symbol.java




> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Ryan Blue (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195529#comment-15195529
 ] 

Ryan Blue commented on AVRO-1723:
-

[~zolyfarkas], now that AVRO-1667 is in, could you rebase this on master? 
Thanks!

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15018137#comment-15018137
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Attached a patch

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
> Attachments: AVRO-1723.patch
>
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012259#comment-15012259
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Doug, finally I have created a feature branch with this based on the latest 
trunk:

https://github.com/zolyfarkas/avro/tree/trunk-AVRO-1723

Please review, the patch also contains:AVRO-1667 without it the unit test to 
test the functionality will break.

Let me know if you have any comments.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971761#comment-14971761
 ] 

Zoltan Farkas commented on AVRO-1723:
-

The creative re-use of the null SCHEMA instead of the UnresolvedSchema is 
problematic!
The main reason is that you cannot have the same type twice in a union, and 
makes it impossible to declare union {null, SomeCircularType}
So this solution is not Workable.

Doug what is your suggestion?

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Doug Cutting (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971803#comment-14971803
 ] 

Doug Cutting commented on AVRO-1723:


Perhaps use a record schema instead?  E.g. something like:

{code}{"type":"record", "name":"org.apache.avro.compiler.idl.UnresolvedSchema",
"resolve":"my.full.Name", "fields":[]}{code}

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971824#comment-14971824
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Thanks, that is pretty much what I ended up doing,
Will test this thoroughly and will prepare a patch.


> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Doug Cutting (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14790846#comment-14790846
 ] 

Doug Cutting commented on AVRO-1723:


Can you please provide a patch or pull request with all and only the changes 
related to this issue so that it can be more easily reviewed?  Thanks.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791276#comment-14791276
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Doug, will gather the changes into a patch should be able to find some time in 
the next week o,

If people are starting taking advantage of this they will hit: AVRO-1667

So it would be very important for AVRO-1667 to be resolved first... can 
somebody look into 1667, it has been sitting there for a while...


> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742065#comment-14742065
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Makes sense, here is the change: 
https://github.com/zolyfarkas/avro/commit/6c51c5dac1436adb0a303af22847fc4aef208924


> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739879#comment-14739879
 ] 

Zoltan Farkas commented on AVRO-1723:
-

Unfortunately UnresolvedSchema I had to declare it in the org.apache.avro 
package since the constructor for Schema is package protected.
to make both private I would need to have them in the same package: 
org.apache.avro.compiler.idl
Which I don't see how I can do without modifying: org.apache.avro.Schema. 
(which I was trying to avoid...)

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Doug Cutting (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739902#comment-14739902
 ] 

Doug Cutting commented on AVRO-1723:


Adding a public subclass of Schema in org.apache.avro that's not usable by 
applications is too confusing.

Rather you might create a schema with a property that identifies it as 
unresolved, e.g., the equivalent of:
{code}{"type":"null", 
"org.apache.avro.compiler.idl.unresolved.name":"my.full.Name"}{code}

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Doug Cutting (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739710#comment-14739710
 ] 

Doug Cutting commented on AVRO-1723:


Any reason UnresolvedSchema and SchemaResolver need to be public classes?  
Can't these rather be private to the IDL compiler?

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739460#comment-14739460
 ] 

Zoltan Farkas commented on AVRO-1723:
-

There is no change to the IDL. It just does not matter anymore the order of 
declaration.

If you look at 
https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
 , I have created:

org.apache.avro.UnresolvedSchema

which is being used instead of throwing an error when building the schema on 
first pass.

There is an extra pass where all UnresolvedSchema's are being resolved. 
(replaced with the actual schema)

it is a fairly simple implementation, let me know if you have any question.

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739467#comment-14739467
 ] 

Zoltan Farkas commented on AVRO-1723:
-

To be a bit more precise, There is a change in the IDL spec: "The declaration 
order does not matter anymore"
you can reference a type that is declared downstream. (just like in java)

> Add support for forward declarations in avro IDL
> 
>
> Key: AVRO-1723
> URL: https://issues.apache.org/jira/browse/AVRO-1723
> Project: Avro
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
>
> Currently Recursive data structures like:
> record SampleNode {
>int count = 0;
>array samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from 
> https://issues.apache.org/jira/browse/AVRO-1667 )
> It is actually not complicated to implement, here is some detail on a 
> possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)