from:"Ramana Suvarapu \(JIRA\)"

[jira] [Updated] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-05-27 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1474:
--

Attachment: AVRO-1474-REUSE-RD.patch
AVRO-1474-MAP.patch
AVRO-1474-ENUM.patch

 C++ resolvind decoder doesn't work when reader schema has more fields than 
 writer schema
 

 Key: AVRO-1474
 URL: https://issues.apache.org/jira/browse/AVRO-1474
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.6
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: AVRO-1474-ENUM.patch, AVRO-1474-MAP.patch, 
 AVRO-1474-REUSE-RD.patch, AVRO-1474-ResolvingDecoder.patch, AVRO-1474.patch, 
 bigrecord, bigrecord_r, reader, writer


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
 Also field ordering is not working. 
 The same issue is reported in AVRO-1360.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-05-27 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009684#comment-14009684
]

Ramana Suvarapu commented on AVRO-1474:
---

Hi Thiru,

I attached 4 patch files (patch over your patch) for the following issues.
1) ResolvingGrammarGenerator::getWriterProduction function doesn’t work with
Schemas with Symbollic nodes. Look at the attached
AVRO-1474-ResolvingDecoder.patch file for the fix.
2) Enum ordering doesn't work. Look at the AVRO-1474-ENUM.patch file for the fix
3) Resolving decoder re-use issue. Look at the AVRO-1474-REUSE-RD.patch for the
fox
4) Bug in AVRO map handling because of shared_ptr. Please look at the
AVRO-1474-MAP.patch file.

Please review these files and make these changes available for the next Avro
Release if it makes sense.

C++ resolvind decoder doesn't work when reader schema has more fields than
writer schema

Key: AVRO-1474
URL: https://issues.apache.org/jira/browse/AVRO-1474
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.6
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1474-ENUM.patch, AVRO-1474-MAP.patch,
AVRO-1474-REUSE-RD.patch, AVRO-1474-ResolvingDecoder.patch, AVRO-1474.patch,
bigrecord, bigrecord_r, reader, writer

When reader schema has more number of fields than writer schema, C++
implementation of resolving decoder is throwing exception throwing exception
Don't know how to handle excess fields for reader.” with out checking
whether fields are optional or fields have default values.
Attached are reader and writer schemas. Record in reader schema has 2
additional fields than writer schema. One field is required field but it has
default value and another one is optional field (union of null and string).
Since one has default value and another is optional both reader and writer
schemas are supposed to be compatible.

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

main()
{
avro::ValidSchema readerSchema = load(reader.json);
avro::ValidSchema writerSchema = load(writer.json);
avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
readerSchema,avro::binaryDecoder());
}

But when I tried to create resolving decoder, I am getting Don't know how to
handle excess fields for reader.” But Java implementation works.
Also field ordering is not working.
The same issue is reported in AVRO-1360.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-04-23 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13979252#comment-13979252
]

Ramana Suvarapu commented on AVRO-1474:
---

Patch is not working in the following scenario.

Hi Thiru,

Attached are writer and reader schema. In writer schema I added new field
“optionalNestedrecord”
to RootRecord which is optional and default value is null. This field doesn’t
exist in Reader schema.

{
name: optionalNestedrecord,
type: [null, Nested],
default: null

}

When I populate this field in writers RootRecord and when I tried to
deserialize using reader schema, It’s throwing following exception.

boost::exception_detail::clone_implboost::exception_detail::error_info_injectorboost::bad_weak_ptr
at memory location 0x00b5d3b0..

This exception is thrown in the following section of void skip(Decoder d) of
Symbol.hh

case Symbol::sSymbolic:
{
ProductionPtr pp(
t.extraboost::weak_ptrProduction ());
parsingStack.pop();
append(pp);
}

Steps to reproduce:
1. Use the attached files to regenerate bigrecord types for both reader
and writer
2. In writer RootRecord, populate optionalNestedrecord and encode it
3. Using ResolvingDecoder and using reader schema, try to deserialize
encoded writer’s record and you will see this exception.

I attached theses schema to AVRO-1474 jira.
Please take a look into this.

Thanks,
Ramana

On Wed, Apr 23, 2014 at 7:41 PM, Srinidhi S sri0...@gmail.com wrote:
Hi Thiru

I found a bug with latest patch. Here is the scenario.

writer record schema has optional field which is defined as union [null,
another record] and here another record is defined as Symbolic to schema
instead of actual definition of full record schema.

If reader schema doesn't have this field and writer populates this field and
try to decode writer's record using reader schema it's throwing boost::bad_ptr
exception. The exception is thrown from symbol class where it's trying to
extract extra info from Production for Symbollic.

I have a test case to reproduce this issue and I will send it in my next email.

Thanks
Venkat

C++ resolvind decoder doesn't work when reader schema has more fields than
writer schema

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-04-23 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1474:
--

Attachment: bigrecord_r
bigrecord

 C++ resolvind decoder doesn't work when reader schema has more fields than 
 writer schema
 

 Key: AVRO-1474
 URL: https://issues.apache.org/jira/browse/AVRO-1474
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.6
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: AVRO-1474.patch, bigrecord, bigrecord_r, reader, writer


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
 Also field ordering is not working. 
 The same issue is reported in AVRO-1360.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-04-21 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13975579#comment-13975579
 ] 

Ramana Suvarapu commented on AVRO-1474:
---

Yes. You can commit this. I created separate JIRA for handling Avro aliases. 

Please include #include stdint.h to GenericDatum.hh. We are getting 
compilation errors when compiling on Windows.

Also please add -DBOOST_IOSTREAMS_DYN_LINK to CMakeLists.txt to following 
section. This is to fix Windows compilation errors.

if (WIN32 AND NOT CYGWIN AND NOT MSYS)
add_definitions (/EHa)
add_definitions (
-DBOOST_REGEX_DYN_LINK
-DBOOST_FILESYSTEM_DYN_LINK
-DBOOST_SYSTEM_DYN_LINK
-DBOOST_IOSTREAMS_DYN_LINK
-DBOOST_PROGRAM_OPTIONS_DYN_LINK
-DBOOST_ALL_NO_LIB)
endif()





 C++ resolvind decoder doesn't work when reader schema has more fields than 
 writer schema
 

 Key: AVRO-1474
 URL: https://issues.apache.org/jira/browse/AVRO-1474
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.6
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: AVRO-1474.patch, reader, writer


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
 Also field ordering is not working. 
 The same issue is reported in AVRO-1360.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (AVRO-1496) Avro aliases support for C++

2014-04-17 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1496:
-

 Summary: Avro aliases support for C++
 Key: AVRO-1496
 URL: https://issues.apache.org/jira/browse/AVRO-1496
 Project: Avro
  Issue Type: Bug
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.


Currently Avro C++ doesn't have avro aliases support and other languages like 
Java and C# supporte this feature. Please implement this feature in C++ and 
enhance the resolving decoder to handle aliases



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (AVRO-1496) Avro aliases support for C++

2014-04-17 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1496:
--

Component/s: c++

 Avro aliases support for C++
 

 Key: AVRO-1496
 URL: https://issues.apache.org/jira/browse/AVRO-1496
 Project: Avro
  Issue Type: Bug
  Components: c++
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.

 Currently Avro C++ doesn't have avro aliases support and other languages like 
 Java and C# supporte this feature. Please implement this feature in C++ and 
 enhance the resolving decoder to handle aliases



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-03-03 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1474:
-

 Summary: C++ resolvind decoder doesn't work when reader schema has 
more fields than writer schema
 Key: AVRO-1474
 URL: https://issues.apache.org/jira/browse/AVRO-1474
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.6
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.


When reader schema has more number of fields than writer schema, C++ 
implementation of resolving decoder is throwing exception throwing exception 
Don't know how to handle excess fields for reader.” with out checking whether 
fields are optional or fields have default values.

Attached are reader and writer schemas. Record in reader schema has 2 
additional fields than writer schema. One field is required field but it has 
default value and another one is optional field (union of null and string). 
Since one has default value and another is optional both reader and writer 
schemas are supposed to be compatible. 
 
{name: defaultField, type: string, default: DEFAULT, 
declared:true}, 
{name: optionalField, type: [string, null],declared:true},
 
main()
{
  avro::ValidSchema readerSchema = load(reader.json);
  avro::ValidSchema writerSchema = load(writer.json);
  avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
readerSchema,avro::binaryDecoder());
}
 
But when I tried to create resolving decoder, I am getting Don't know how to 
handle excess fields for reader.” But Java implementation works.  

Also field ordering is not working. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-03-03 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1474:
--

Attachment: writer
reader

 C++ resolvind decoder doesn't work when reader schema has more fields than 
 writer schema
 

 Key: AVRO-1474
 URL: https://issues.apache.org/jira/browse/AVRO-1474
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.6
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: reader, writer


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
 Also field ordering is not working. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-03-03 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1474:
--

Description: 
When reader schema has more number of fields than writer schema, C++ 
implementation of resolving decoder is throwing exception throwing exception 
Don't know how to handle excess fields for reader.” with out checking whether 
fields are optional or fields have default values.

Attached are reader and writer schemas. Record in reader schema has 2 
additional fields than writer schema. One field is required field but it has 
default value and another one is optional field (union of null and string). 
Since one has default value and another is optional both reader and writer 
schemas are supposed to be compatible. 
 
{name: defaultField, type: string, default: DEFAULT, 
declared:true}, 
{name: optionalField, type: [string, null],declared:true},
 
main()
{
  avro::ValidSchema readerSchema = load(reader.json);
  avro::ValidSchema writerSchema = load(writer.json);
  avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
readerSchema,avro::binaryDecoder());
}
 
But when I tried to create resolving decoder, I am getting Don't know how to 
handle excess fields for reader.” But Java implementation works.  

Also field ordering is not working. 

The same issue is reported in AVRO-1360.

  was:
When reader schema has more number of fields than writer schema, C++ 
implementation of resolving decoder is throwing exception throwing exception 
Don't know how to handle excess fields for reader.” with out checking whether 
fields are optional or fields have default values.

Attached are reader and writer schemas. Record in reader schema has 2 
additional fields than writer schema. One field is required field but it has 
default value and another one is optional field (union of null and string). 
Since one has default value and another is optional both reader and writer 
schemas are supposed to be compatible. 
 
{name: defaultField, type: string, default: DEFAULT, 
declared:true}, 
{name: optionalField, type: [string, null],declared:true},
 
main()
{
  avro::ValidSchema readerSchema = load(reader.json);
  avro::ValidSchema writerSchema = load(writer.json);
  avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
readerSchema,avro::binaryDecoder());
}
 
But when I tried to create resolving decoder, I am getting Don't know how to 
handle excess fields for reader.” But Java implementation works.  

Also field ordering is not working. 


 C++ resolvind decoder doesn't work when reader schema has more fields than 
 writer schema
 

 Key: AVRO-1474
 URL: https://issues.apache.org/jira/browse/AVRO-1474
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.6
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: reader, writer


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
 Also field ordering is not working. 
 The same issue is reported in AVRO-1360.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2014-03-03 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1360:
--

Status: Open  (was: Patch Available)

This patch can't be applied to latest trunk. Lot of changes were introduced 
since creation of this patch. Also this JIRA can be closed as new JIRA is 
created to track this issue

AVRO-1474 C++ resolvind decoder doesn't work when reader schema has more fields 
than writer schema

 C++ Resolving decoder is not working when reader schema has more fields than 
 writer schema
 --

 Key: AVRO-1360
 URL: https://issues.apache.org/jira/browse/AVRO-1360
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360-4.patch, 
 AVRO-1360-5.patch, AVRO-1360.patch, AVRO-RD.patch, callstack.txt, model.avsc, 
 testreader, testreader-1, testreader.hh, testwriter, testwriter-1, 
 testwriter.hh


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
  
 Can you please let us know if there are any other limitations with c++ 
 implementation of ResolvingDecoder? We are planning to use it in our project 
 and we want to make sure it works as per avro specification.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema

2014-03-03 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918483#comment-13918483
 ] 

Ramana Suvarapu commented on AVRO-1474:
---

First few patches of AVRo-1360 needs to be reapplied to fix this issue.

 C++ resolvind decoder doesn't work when reader schema has more fields than 
 writer schema
 

 Key: AVRO-1474
 URL: https://issues.apache.org/jira/browse/AVRO-1474
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.6
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: reader, writer


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
 Also field ordering is not working. 
 The same issue is reported in AVRO-1360.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (AVRO-1444) Unable to reuse JSonDecoder and JSONDecoder crashes

2014-01-22 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1444:
--

Assignee: Thiruvalluvan M. G.

 Unable to reuse JSonDecoder and JSONDecoder crashes
 ---

 Key: AVRO-1444
 URL: https://issues.apache.org/jira/browse/AVRO-1444
 Project: Avro
  Issue Type: Bug
  Components: c++
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Fix For: 1.7.6

 Attachments: AVRO-1444.patch, JsonEncodingTest.cpp


 I am trying to re-use the JSONDecoder to decode the JSONText file which 
 contains JSONEncoded Objects. After first item is processed successfully, on 
 the second item, when I tried to re-use the decoder, decoder crashes.
 The reason is when jsonDecoder is initialized with new stream, internal stack 
 of jsonDecoder is getting reset and StreamReader of jsonDecoder is also not 
 getting reset.
 Attached is the program to re-produce this issue and also I attach patch to 
 fix this issue. But patch works but needs to be reviewed 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (AVRO-1444) Unable to reuse JSonDecoder and JSONDecoder crashes

2014-01-22 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879001#comment-13879001
 ] 

Ramana Suvarapu commented on AVRO-1444:
---

Hi Thiru,
 
One more observation about StreamReader of JSonDecoder. After the JSONText is 
decoded successfully, the next ptr of StreamReader not getting moved to end of 
the JSONText. Ideally after the end of decoding, next ptr should be at end ptr 
of the data stream. But it’s not happening.
 
Thanks,
Ramana

 Unable to reuse JSonDecoder and JSONDecoder crashes
 ---

 Key: AVRO-1444
 URL: https://issues.apache.org/jira/browse/AVRO-1444
 Project: Avro
  Issue Type: Bug
  Components: c++
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Fix For: 1.7.6

 Attachments: AVRO-1444.patch, JsonEncodingTest.cpp


 I am trying to re-use the JSONDecoder to decode the JSONText file which 
 contains JSONEncoded Objects. After first item is processed successfully, on 
 the second item, when I tried to re-use the decoder, decoder crashes.
 The reason is when jsonDecoder is initialized with new stream, internal stack 
 of jsonDecoder is getting reset and StreamReader of jsonDecoder is also not 
 getting reset.
 Attached is the program to re-produce this issue and also I attach patch to 
 fix this issue. But patch works but needs to be reviewed 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (AVRO-1444) Unable to reuse JSonDecoder and JSONDecoder crashes

2014-01-21 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1444:
-

 Summary: Unable to reuse JSonDecoder and JSONDecoder crashes
 Key: AVRO-1444
 URL: https://issues.apache.org/jira/browse/AVRO-1444
 Project: Avro
  Issue Type: Bug
  Components: c++
Reporter: Ramana Suvarapu


I am trying to re-use the JSONDecoder to decode the JSONText file which 
contains JSONEncoded Objects. After first item is processed successfully, on 
the second item, when I tried to re-use the decoder, decoder crashes.

The reason is when jsonDecoder is initialized with new stream, internal stack 
of jsonDecoder is getting reset and StreamReader of jsonDecoder is also not 
getting reset.

Attached is the program to re-produce this issue and also I attach patch to fix 
this issue. But patch works but needs to be reviewed 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (AVRO-1444) Unable to reuse JSonDecoder and JSONDecoder crashes

2014-01-21 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1444:
--

Fix Version/s: 1.7.6

 Unable to reuse JSonDecoder and JSONDecoder crashes
 ---

 Key: AVRO-1444
 URL: https://issues.apache.org/jira/browse/AVRO-1444
 Project: Avro
  Issue Type: Bug
  Components: c++
Reporter: Ramana Suvarapu
 Fix For: 1.7.6


 I am trying to re-use the JSONDecoder to decode the JSONText file which 
 contains JSONEncoded Objects. After first item is processed successfully, on 
 the second item, when I tried to re-use the decoder, decoder crashes.
 The reason is when jsonDecoder is initialized with new stream, internal stack 
 of jsonDecoder is getting reset and StreamReader of jsonDecoder is also not 
 getting reset.
 Attached is the program to re-produce this issue and also I attach patch to 
 fix this issue. But patch works but needs to be reviewed 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (AVRO-1444) Unable to reuse JSonDecoder and JSONDecoder crashes

2014-01-21 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1444:
--

Attachment: JsonEncodingTest.cpp

 Unable to reuse JSonDecoder and JSONDecoder crashes
 ---

 Key: AVRO-1444
 URL: https://issues.apache.org/jira/browse/AVRO-1444
 Project: Avro
  Issue Type: Bug
  Components: c++
Reporter: Ramana Suvarapu
 Fix For: 1.7.6

 Attachments: JsonEncodingTest.cpp


 I am trying to re-use the JSONDecoder to decode the JSONText file which 
 contains JSONEncoded Objects. After first item is processed successfully, on 
 the second item, when I tried to re-use the decoder, decoder crashes.
 The reason is when jsonDecoder is initialized with new stream, internal stack 
 of jsonDecoder is getting reset and StreamReader of jsonDecoder is also not 
 getting reset.
 Attached is the program to re-produce this issue and also I attach patch to 
 fix this issue. But patch works but needs to be reviewed 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (AVRO-1444) Unable to reuse JSonDecoder and JSONDecoder crashes

2014-01-21 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1444:
--

Attachment: AVRO-1444.patch

 Unable to reuse JSonDecoder and JSONDecoder crashes
 ---

 Key: AVRO-1444
 URL: https://issues.apache.org/jira/browse/AVRO-1444
 Project: Avro
  Issue Type: Bug
  Components: c++
Reporter: Ramana Suvarapu
 Fix For: 1.7.6

 Attachments: AVRO-1444.patch, JsonEncodingTest.cpp


 I am trying to re-use the JSONDecoder to decode the JSONText file which 
 contains JSONEncoded Objects. After first item is processed successfully, on 
 the second item, when I tried to re-use the decoder, decoder crashes.
 The reason is when jsonDecoder is initialized with new stream, internal stack 
 of jsonDecoder is getting reset and StreamReader of jsonDecoder is also not 
 getting reset.
 Attached is the program to re-produce this issue and also I attach patch to 
 fix this issue. But patch works but needs to be reviewed 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (AVRO-1424) ValidatingDecoder hangs on large schema

2014-01-07 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13864276#comment-13864276
 ] 

Ramana Suvarapu commented on AVRO-1424:
---

Hi Thiru, the patch worked and you can check-in the changes to trunk.

 ValidatingDecoder hangs on large schema
 ---

 Key: AVRO-1424
 URL: https://issues.apache.org/jira/browse/AVRO-1424
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.5
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Fix For: 1.7.6

 Attachments: AVRO-1424.patch, model.avsc


 Try to create Validation decoder using attached schema. It hangs and causes 
 huge memory allocation. The problem is because of boost::any which is making 
 excessive copies of objects when creating Symbols.
 And also fixup method which is being called in validation decoder creation 
 stack is also taking long very very long time.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (AVRO-1424) ValidatingDecoder hangs on large schema

2014-01-07 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13864280#comment-13864280
 ] 

Ramana Suvarapu commented on AVRO-1424:
---

Now ValidationDecoder is faster than the old one.

 ValidatingDecoder hangs on large schema
 ---

 Key: AVRO-1424
 URL: https://issues.apache.org/jira/browse/AVRO-1424
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.5
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Fix For: 1.7.6

 Attachments: AVRO-1424.patch, model.avsc


 Try to create Validation decoder using attached schema. It hangs and causes 
 huge memory allocation. The problem is because of boost::any which is making 
 excessive copies of objects when creating Symbols.
 And also fixup method which is being called in validation decoder creation 
 stack is also taking long very very long time.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (AVRO-1424) ValidatingDecoder hangs on large schema

2014-01-03 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13861924#comment-13861924
 ] 

Ramana Suvarapu commented on AVRO-1424:
---

Unable to apply the patch as patch was not generated from Avro-1.7.5 version of 
source code.

 ValidatingDecoder hangs on large schema
 ---

 Key: AVRO-1424
 URL: https://issues.apache.org/jira/browse/AVRO-1424
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.5
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: AVRO-1424.patch, model.avsc


 Try to create Validation decoder using attached schema. It hangs and causes 
 huge memory allocation. The problem is because of boost::any which is making 
 excessive copies of objects when creating Symbols.
 And also fixup method which is being called in validation decoder creation 
 stack is also taking long very very long time.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (AVRO-1424) ValidatingDecoder hangs on large schema

2014-01-02 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1424:
-

 Summary: ValidatingDecoder hangs on large schema
 Key: AVRO-1424
 URL: https://issues.apache.org/jira/browse/AVRO-1424
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.5
Reporter: Ramana Suvarapu


Try to create Validation decoder using attached schema. It hangs and causes 
huge memory allocation. The problem is because of boost::any which is making 
excessive copies of objects when creating Symbols.

And also fixup method which is being called in validation decoder creation 
stack is also taking long very very long time.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (AVRO-1424) ValidatingDecoder hangs on large schema

2014-01-02 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1424:
--

Attachment: model.avsc

 ValidatingDecoder hangs on large schema
 ---

 Key: AVRO-1424
 URL: https://issues.apache.org/jira/browse/AVRO-1424
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.5
Reporter: Ramana Suvarapu
 Attachments: model.avsc


 Try to create Validation decoder using attached schema. It hangs and causes 
 huge memory allocation. The problem is because of boost::any which is making 
 excessive copies of objects when creating Symbols.
 And also fixup method which is being called in validation decoder creation 
 stack is also taking long very very long time.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-12-16 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850066#comment-13850066
 ] 

Ramana Suvarapu commented on AVRO-1360:
---

Hi Thiru,

Finally I am able rep huge memory leak issue with large test schema. Attached 
is test schema (model.avsc). Try to create ValidatingDecoder and you can see 
that program hangs and memory grows forever.

Thanks,
Ramana

testValidatingDecoder()
{
ValidSchema s;
ifstream ifs(c:\\temp\\model.avsc);
compileJsonSchema(ifs, s);
   DecoderPtr d = validatingDecoder(s, binaryDecoder());
}


}

 C++ Resolving decoder is not working when reader schema has more fields than 
 writer schema
 --

 Key: AVRO-1360
 URL: https://issues.apache.org/jira/browse/AVRO-1360
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360-4.patch, 
 AVRO-1360-5.patch, AVRO-1360.patch, AVRO-RD.patch, callstack.txt, testreader, 
 testreader-1, testreader.hh, testwriter, testwriter-1, testwriter.hh


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
  
 Can you please let us know if there are any other limitations with c++ 
 implementation of ResolvingDecoder? We are planning to use it in our project 
 and we want to make sure it works as per avro specification.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-12-16 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1360:
--

Attachment: model.avsc

 C++ Resolving decoder is not working when reader schema has more fields than 
 writer schema
 --

 Key: AVRO-1360
 URL: https://issues.apache.org/jira/browse/AVRO-1360
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360-4.patch, 
 AVRO-1360-5.patch, AVRO-1360.patch, AVRO-RD.patch, callstack.txt, model.avsc, 
 testreader, testreader-1, testreader.hh, testwriter, testwriter-1, 
 testwriter.hh


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
  
 Can you please let us know if there are any other limitations with c++ 
 implementation of ResolvingDecoder? We are planning to use it in our project 
 and we want to make sure it works as per avro specification.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Created] (AVRO-1415) C++ binary encoder and decoder doesn't handle uninitialzed enums

2013-12-16 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1415:
-

 Summary: C++ binary encoder and decoder doesn't handle 
uninitialzed enums
 Key: AVRO-1415
 URL: https://issues.apache.org/jira/browse/AVRO-1415
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu


When enums are not properly initialized and when they get encoded / decoded, 
C++ enum encoding and decoding traits don't check for uninitialed enums and it 
encodes the wrong values. When Java or C# tries to decode them, they throw out 
of boundary exceptions.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Updated] (AVRO-1415) C++ binary encoder and decoder doesn't handle uninitialzed enums

2013-12-16 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1415:
--

Attachment: AVRO-1415.patch

 C++ binary encoder and decoder doesn't handle uninitialzed enums
 

 Key: AVRO-1415
 URL: https://issues.apache.org/jira/browse/AVRO-1415
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
 Attachments: AVRO-1415.patch


 When enums are not properly initialized and when they get encoded / decoded, 
 C++ enum encoding and decoding traits don't check for uninitialed enums and 
 it encodes the wrong values. When Java or C# tries to decode them, they throw 
 out of boundary exceptions.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Commented] (AVRO-1415) C++ binary encoder and decoder doesn't handle uninitialzed enums

2013-12-16 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13850106#comment-13850106
 ] 

Ramana Suvarapu commented on AVRO-1415:
---

Attached a patch to fix this problem

 C++ binary encoder and decoder doesn't handle uninitialzed enums
 

 Key: AVRO-1415
 URL: https://issues.apache.org/jira/browse/AVRO-1415
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
 Attachments: AVRO-1415.patch


 When enums are not properly initialized and when they get encoded / decoded, 
 C++ enum encoding and decoding traits don't check for uninitialed enums and 
 it encodes the wrong values. When Java or C# tries to decode them, they throw 
 out of boundary exceptions.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

[jira] [Created] (AVRO-1383) Unabe to build on Visual Studio 2003 when generated file is huge

2013-10-08 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1383:
-

 Summary: Unabe to build on Visual Studio 2003 when generated file 
is huge
 Key: AVRO-1383
 URL: https://issues.apache.org/jira/browse/AVRO-1383
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
 Environment: Windows VS 2003
Reporter: Ramana Suvarapu


Hi,

This is related to AVRO-1370. Currently C++ code generation produces single 
file and with lot of inline functions.  If the schema file is huge, it's 
generating huge header file. When this header file  is used to to build the 
project, we are getting object file format limit exceeded : more than 65,279 
sections. To fix this problem we had to use /bigobj flag to the project  and 
this fixed the problem.

Unfortunately /bigobj is only supported from VS 2005. Prior versions of VS 2005 
don't have this flag. 

Is it possible to split the generated file into multiple parts by class name 
and it's avro traits code.




--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-10-02 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13784242#comment-13784242
]

Ramana Suvarapu commented on AVRO-1360:
---

Hi Thiru,

I did some investigation to find out the root cause for the slowness of
resolving decoder and why it’s taking huge memory when dealing with larger
schemas. When I tried ResolvingDecoder using our schema,
ValidatingGrammer::generate step is taking forever and after 2 to 3 minutes
it’s taking the entire memory of the system and method never returns. Here is
the task manager image during the ValidatingGrammer::generate step.

I debugged the code and from the call stack I noticed that huge amount of time
and memory is being spent during boost::any and std::vector operations,
constructors, destructors . Currently Production, RootInfo and RepeaterInfo are
type-define as

typedef std::vectorSymbol Production;
typedef boost::tuplesize_t, bool, Production, Production RepeaterInfo;
typedef boost::tupleProduction, Production RootInfo;
Since each of these collection stores Objects, lot of temporary objects are
being created and destructed. Please find the attached call-stack for more
information.

I modified Production, RepeaterInfo and Rootinfo collections to store
shared_ptrs instead of Objects and modified the decoder code accordingly and
when I ran the program, ValidatingGrammer::generate performed fast and return
the result within few milliseconds (less than 10 ms).

typedef std::vectorboost::shared_ptrSymbol Production;
typedef boost::tuplesize_t, bool, boost::shared_ptrProduction,
boost::shared_ptrProduction RepeaterInfo;
typedef boost::tupleboost::shared_ptrProduction,
boost::shared_ptrProduction RootInfo;

I created a patch for my changes and attached in this email. Please take a look
and let me know your thoughts on the changes.

After these changes, ResolvingGrammarGenerator::generate() is getting executed
faster.

Another biggest bottle neck is fixup method() and which is taking lot of time
when production vector has lot of nested symbols.
What is the significance of recursive method fixup()? This method is also
taking long time to execute. Can you please let me how this method works? Is
there anyway it can be improved?

C++ Resolving decoder is not working when reader schema has more fields than
writer schema
--

Key: AVRO-1360
URL: https://issues.apache.org/jira/browse/AVRO-1360
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360.patch,
testreader, testreader-1, testreader.hh, testwriter, testwriter-1,
testwriter.hh

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

But when I tried to create resolving decoder, I am getting Don't know how to
handle excess fields for reader.” But Java implementation works.

Can you please let us know if there are any other limitations with c++
implementation of ResolvingDecoder? We are planning to use it in our project
and we want to make sure it works as per avro specification.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-09-04 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ramana Suvarapu updated AVRO-1360:
--

Attachment: (was: writer.json)

C++ Resolving decoder is not working when reader schema has more fields than
writer schema
--

Key: AVRO-1360
URL: https://issues.apache.org/jira/browse/AVRO-1360
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1360-2.patch, AVRO-1360.patch, testreader,
testreader.hh, testwriter, testwriter.hh

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

But when I tried to create resolving decoder, I am getting Don't know how to
handle excess fields for reader.” But Java implementation works.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-09-04 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ramana Suvarapu updated AVRO-1360:
--

Attachment: (was: reader.json)

C++ Resolving decoder is not working when reader schema has more fields than
writer schema
--

Key: AVRO-1360
URL: https://issues.apache.org/jira/browse/AVRO-1360
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1360-2.patch, AVRO-1360.patch, testreader,
testreader.hh, testwriter, testwriter.hh

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

But when I tried to create resolving decoder, I am getting Don't know how to
handle excess fields for reader.” But Java implementation works.

[jira] [Commented] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-09-04 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13758284#comment-13758284
]

Ramana Suvarapu commented on AVRO-1360:
---

Please use attached testreader-1, testwriter-2 versions to test the resolving
decoder functionaliy. reader schema as 2 addional fields with default types. 1)
string type with default value and 2) union type [string, null] with default
value null.

C++ Resolving decoder is not working when reader schema has more fields than
writer schema
--

Key: AVRO-1360
URL: https://issues.apache.org/jira/browse/AVRO-1360
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1360-2.patch, AVRO-1360.patch, testreader,
testreader.hh, testwriter, testwriter.hh

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

But when I tried to create resolving decoder, I am getting Don't know how to
handle excess fields for reader.” But Java implementation works.

[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-09-04 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ramana Suvarapu updated AVRO-1360:
--

Attachment: testreader-1

C++ Resolving decoder is not working when reader schema has more fields than
writer schema
--

Key: AVRO-1360
URL: https://issues.apache.org/jira/browse/AVRO-1360
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1360-2.patch, AVRO-1360.patch, testreader,
testreader-1, testreader.hh, testwriter, testwriter.hh

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

But when I tried to create resolving decoder, I am getting Don't know how to
handle excess fields for reader.” But Java implementation works.

[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-09-04 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ramana Suvarapu updated AVRO-1360:
--

Attachment: testwriter-1

C++ Resolving decoder is not working when reader schema has more fields than
writer schema
--

Key: AVRO-1360
URL: https://issues.apache.org/jira/browse/AVRO-1360
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1360-2.patch, AVRO-1360.patch, testreader,
testreader-1, testreader.hh, testwriter, testwriter-1, testwriter.hh

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

But when I tried to create resolving decoder, I am getting Don't know how to
handle excess fields for reader.” But Java implementation works.

[jira] [Commented] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-09-03 Thread Ramana Suvarapu (JIRA)

[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13757442#comment-13757442
]

Ramana Suvarapu commented on AVRO-1360:
---

Patch seems to be working. But I still need to test couple of scenarios and I
will let you know.

C++ Resolving decoder is not working when reader schema has more fields than
writer schema
--

Key: AVRO-1360
URL: https://issues.apache.org/jira/browse/AVRO-1360
Project: Avro
Issue Type: Bug
Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1360-2.patch, AVRO-1360.patch, reader.json,
testreader, testreader.hh, testwriter, testwriter.hh, writer.json

{name: defaultField, type: string, default: DEFAULT,
declared:true},
{name: optionalField, type: [string, null],declared:true},

But when I tried to create resolving decoder, I am getting Don't know how to
handle excess fields for reader.” But Java implementation works.

[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-08-30 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1360:
--

Attachment: testwriter
testreader
testwriter.hh
testreader.hh

 C++ Resolving decoder is not working when reader schema has more fields than 
 writer schema
 --

 Key: AVRO-1360
 URL: https://issues.apache.org/jira/browse/AVRO-1360
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
 Attachments: AVRO-1360.patch, reader.json, testreader, testreader.hh, 
 testwriter, testwriter.hh, writer.json


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
  
 Can you please let us know if there are any other limitations with c++ 
 implementation of ResolvingDecoder? We are planning to use it in our project 
 and we want to make sure it works as per avro specification.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (AVRO-1370) C++ code generation should generate individual header files for each type

2013-08-28 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1370:
-

 Summary: C++ code generation should generate individual header 
files for each type
 Key: AVRO-1370
 URL: https://issues.apache.org/jira/browse/AVRO-1370
 Project: Avro
  Issue Type: Improvement
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu


Currently C++ code generation generates single header file which inludes all 
the type definitions and encoding and decoding of the types. When generated 
header file is huge, we are experiencing issues when debugging in VS2010.
The issues are 
1) Unable to set the breake point at desired location.
2) When compilation errors occur, doube click on an error taking the control to 
wrong location of the file.

If split the file into mulitple parts, then it works. Is it possible to make 
the changes to codegeneration to do generate mulitple header files for type?


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (AVRO-1363) C# UnionSchema fails to parse the the unions with same type names with different namespaces

2013-08-20 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1363:
--

Attachment: reader.avpr

 C# UnionSchema fails to parse the the unions with same type names with 
 different namespaces
 ---

 Key: AVRO-1363
 URL: https://issues.apache.org/jira/browse/AVRO-1363
 Project: Avro
  Issue Type: Bug
  Components: csharp
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
 Attachments: reader.avpr


 C# Union schema is unable to handle same type names with different 
 namespaces. It's throwing SchemaParseException(Duplicate type in union: 
 ...) exception.
 In the below code, key of uniqueSchemas should be FullName of the type which 
 is similar to Java implementation. 
 internal static UnionSchema NewInstance(JArray jarr, PropertyMap props, 
 SchemaNames names, string encspace)
 {
 ListSchema schemas = new ListSchema();
 IDictionarystring, string uniqueSchemas = new 
 Dictionarystring, string();
 foreach (JToken jvalue in jarr)
 {
 Schema unionType = Schema.ParseJson(jvalue, names, encspace);
 if (null == unionType)
 throw new SchemaParseException(Invalid JSON in union + 
 jvalue.ToString());
 string name = unionType.Name;
 if (uniqueSchemas.ContainsKey(name))
 throw new SchemaParseException(Duplicate type in union: 
  + name);
 uniqueSchemas.Add(name, name);
 schemas.Add(unionType);
 }
 return new UnionSchema(schemas, props);
 }
 See the attachement to recreate the issue

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (AVRO-1363) C# UnionSchema fails to parse the the unions with same type names with different namespaces

2013-08-20 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1363:
-

 Summary: C# UnionSchema fails to parse the the unions with same 
type names with different namespaces
 Key: AVRO-1363
 URL: https://issues.apache.org/jira/browse/AVRO-1363
 Project: Avro
  Issue Type: Bug
  Components: csharp
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
 Attachments: reader.avpr

C# Union schema is unable to handle same type names with different namespaces. 
It's throwing SchemaParseException(Duplicate type in union: ...) exception.

In the below code, key of uniqueSchemas should be FullName of the type which is 
similar to Java implementation. 

internal static UnionSchema NewInstance(JArray jarr, PropertyMap props, 
SchemaNames names, string encspace)
{
ListSchema schemas = new ListSchema();
IDictionarystring, string uniqueSchemas = new Dictionarystring, 
string();

foreach (JToken jvalue in jarr)
{
Schema unionType = Schema.ParseJson(jvalue, names, encspace);
if (null == unionType)
throw new SchemaParseException(Invalid JSON in union + 
jvalue.ToString());

string name = unionType.Name;
if (uniqueSchemas.ContainsKey(name))
throw new SchemaParseException(Duplicate type in union:  
+ name);

uniqueSchemas.Add(name, name);
schemas.Add(unionType);
}

return new UnionSchema(schemas, props);
}

See the attachement to recreate the issue

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-08-12 Thread Ramana Suvarapu (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramana Suvarapu updated AVRO-1360:
--

Attachment: writer.json
reader.json

 C++ Resolving decoder is not working when reader schema has more fields than 
 writer schema
 --

 Key: AVRO-1360
 URL: https://issues.apache.org/jira/browse/AVRO-1360
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
 Attachments: reader.json, writer.json


 When reader schema has more number of fields than writer schema, C++ 
 implementation of resolving decoder is throwing exception throwing exception 
 Don't know how to handle excess fields for reader.” with out checking 
 whether fields are optional or fields have default values.
 Attached are reader and writer schemas. Record in reader schema has 2 
 additional fields than writer schema. One field is required field but it has 
 default value and another one is optional field (union of null and string). 
 Since one has default value and another is optional both reader and writer 
 schemas are supposed to be compatible. 
  
 {name: defaultField, type: string, default: DEFAULT, 
 declared:true}, 
 {name: optionalField, type: [string, null],declared:true},
  
 main()
 {
   avro::ValidSchema readerSchema = load(reader.json);
   avro::ValidSchema writerSchema = load(writer.json);
   avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
 readerSchema,avro::binaryDecoder());
 }
  
 But when I tried to create resolving decoder, I am getting Don't know how to 
 handle excess fields for reader.” But Java implementation works.  
  
 Can you please let us know if there are any other limitations with c++ 
 implementation of ResolvingDecoder? We are planning to use it in our project 
 and we want to make sure it works as per avro specification.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema

2013-08-12 Thread Ramana Suvarapu (JIRA)

Ramana Suvarapu created AVRO-1360:
-

 Summary: C++ Resolving decoder is not working when reader schema 
has more fields than writer schema
 Key: AVRO-1360
 URL: https://issues.apache.org/jira/browse/AVRO-1360
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.7.4
Reporter: Ramana Suvarapu
 Attachments: reader.json, writer.json

When reader schema has more number of fields than writer schema, C++ 
implementation of resolving decoder is throwing exception throwing exception 
Don't know how to handle excess fields for reader.” with out checking whether 
fields are optional or fields have default values.

Attached are reader and writer schemas. Record in reader schema has 2 
additional fields than writer schema. One field is required field but it has 
default value and another one is optional field (union of null and string). 
Since one has default value and another is optional both reader and writer 
schemas are supposed to be compatible. 
 
{name: defaultField, type: string, default: DEFAULT, 
declared:true}, 
{name: optionalField, type: [string, null],declared:true},
 
main()
{
  avro::ValidSchema readerSchema = load(reader.json);
  avro::ValidSchema writerSchema = load(writer.json);
  avro::DecoderPtr d = avro::resolvingDecoder(writerSchema, 
readerSchema,avro::binaryDecoder());
}
 
But when I tried to create resolving decoder, I am getting Don't know how to 
handle excess fields for reader.” But Java implementation works.  
 
Can you please let us know if there are any other limitations with c++ 
implementation of ResolvingDecoder? We are planning to use it in our project 
and we want to make sure it works as per avro specification.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (AVRO-855) Use boost:variants for union types in C++ code generation

2011-07-06 Thread Ramana Suvarapu (JIRA)

Use boost:variants for union types in C++ code generation
-

 Key: AVRO-855
 URL: https://issues.apache.org/jira/browse/AVRO-855
 Project: Avro
  Issue Type: Improvement
  Components: c++
Affects Versions: 1.5.1
Reporter: Ramana Suvarapu
 Fix For: 1.5.2


Hi,

Current C++ code avrocodegencpp generates union_n (where n = 0 to n numbered) 
structure with the member boost::any for union schemas.

Is it possible to use boost::variants instead of union_n structures? The 
advantage of boost::variant is we can define our boost::visitors and apply them 
to retrieve the actual type in the union.

Let me know?
-Ramana

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (AVRO-840) C++ generate nullable types for optional fields int the schema

2011-06-16 Thread Ramana Suvarapu (JIRA)

C++ generate nullable types for optional fields int the schema 
---

 Key: AVRO-840
 URL: https://issues.apache.org/jira/browse/AVRO-840
 Project: Avro
  Issue Type: Improvement
  Components: c++
Reporter: Ramana Suvarapu
Priority: Critical


To represent optional fields, we use unions in our schema. See the example 
below.
{
   type : record,
   name : Contact,
   fields : [ 
{name : FirstName,type : [string ]},
{name : MiddleName,type : [null, string ]},
{name : LastName,  type : [string ]},
{name : PhoneNum,type : [null, string ]},
{name : Id,type : [null, long ]}
   ] 
}
In this schema PhoneNum, MiddleName and Id fields are declared as unions as 
they are optional fields.  For this schema, Avrogencpp generates Contact 
structure and 3 separate union structures for each of the optional fields in 
the schema.

struct  Contact
{
String FirstName;
String LastName;
Union_0 MiddleName;
Union_1 PhoneNum;
Union_3 Id;
}
Instead is it possible to create a new template based NullableType to represent 
optional fields. Basically if the schema has union with 2 fields and first 
field is null, it should generate Nullable type.

For the above scheme, it should generate something like this.
struct  Contact
{
   String FirstName;
   String LastName;
   Nullablestring MiddleName;
   Nullablestring PhoneNum;
   Nullable long  Id; 
}
This will reduce the number of generated unions in generated code and improve 
the readability and usability of the code.

Let me know if it's feasible to implement this.
Thanks,
Ramana


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (AVRO-825) C++ code generation issues with records when member of the record is of type itself

2011-05-31 Thread Ramana Suvarapu (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13041563#comment-13041563
 ] 

Ramana Suvarapu commented on AVRO-825:
--

Tested the latest patch AVRO0825-v5.patch and all the code generation and 
compilation issues have been resolved.

 C++ code generation issues with records when member of the record is of type 
 itself
 ---

 Key: AVRO-825
 URL: https://issues.apache.org/jira/browse/AVRO-825
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.5.1
Reporter: Ramana Suvarapu
Assignee: Thiruvalluvan M. G.
Priority: Blocker
 Fix For: 1.5.2

 Attachments: AVRO-825-v2.patch, AVRO-825-v5.patch, AVRO-825.patch


 I am trying to generate C++ code for the following schema using new 
 avrogencpp. This schema is one the of the example given in Avro specification
 {
   type: record, 
   name: LongList,
   aliases: [LinkedLongs],  // old name for this
   fields : [
 {name: value, type: long}, // each element has a long
 {name: next, type: [LongList, null]} // optional next element
   ]
 }
 It generated the following .hh file and when I tried to compile I am getting 
 #ifndef __TEST_LINKEDLIST_HH_1817083040__H_
 #define __TEST_LINKEDLIST_HH_1817083040__H_
 #include boost/any.hpp
 #include Specific.hh
 #include Encoder.hh
 #include Decoder.hh
 struct _imaginary_avr_Union__0__ {
 private:
 size_t idx_;
 boost::any value_;
 public:
 size_t idx() const { return idx_; }
 LongList get_LongList() const {
 if (idx_ != 0) {
 throw avro::Exception(Invalid type for union);
 }
 return boost::any_castLongList (value_);
 }
 void set_LongList(const LongList v) {
 idx_ = 0;
 value_ = v;
 }
 void set_null() {
 idx_ = 1;
 value_ = boost::any();
 }
 _imaginary_avr_Union__0__() : idx_(0) {
 value_ = LongList();
 }
 };
 struct LongList {
 int64_t value;
 _imaginary_avr_Union__0__ next;
 };
 namespace avro {
 template struct codec_traits_imaginary_avr_Union__0__ {
 static void encode(Encoder e, _imaginary_avr_Union__0__ v) {
 e.encodeUnionIndex(v.idx());
 switch (v.idx()) {
 case 0:
 avro::encode(e, v.get_LongList());
 break;
 case 1:
 e.encodeNull();
 break;
 }
 }
 static void decode(Decoder d, _imaginary_avr_Union__0__ v) {
 size_t n = d.decodeUnionIndex();
 if (n = 2) { throw avro::Exception(Union index too big); }
 switch (n) {
 case 0:
 {
 LongList vv;
 avro::decode(d, vv);
 v.set_LongList(vv);
 }
 break;
 case 1:
 d.decodeNull();
 v.set_null();
 break;
 }
 }
 };
 template struct codec_traitsLongList {
 static void encode(Encoder e, const LongList v) {
 avro::encode(e, v.value);
 avro::encode(e, v.next);
 }
 static void decode(Decoder d, LongList v) {
 avro::decode(d, v.value);
 avro::decode(d, v.next);
 }
 };
 }
 #endif
 When I tried to compile I am getting following errors due to incomplete type.
 Errors
 In file included from 
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/AvrogencppTests.cc:24:
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:34: error: 
 `LongList' does not name a type
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:40: error: expected 
 `,' or `...' before '' token
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:40: error: ISO C++ 
 forbids declaration of `LongList' with no type
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh: In member function 
 `void _imaginary_avr_Union__0__::set_LongList(int)':
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:42: error: `v' was 
 not declared in this scope
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh: In constructor 
 `_imaginary_avr_Union__0__::_imaginary_avr_Union__0__()':
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:49: error: 
 `LongList' was not declared in this scope
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh: In static member 
 function `static void 
 avro::codec_traits_imaginary_avr_Union__0__::encode(avro::Encoder, 
 _imaginary_avr_Union__0__)':
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:64: error: 'struct 
 _imaginary_avr_Union__0__' has no member named 'get_LongList'
 /home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh: In static member 
 function `static void 
 avro::codec_traits_imaginary_avr_Union__0__::decode(avro::Decoder,

[jira] [Created] (AVRO-825) C++ code generation issues with records when member of the record is of type itself

2011-05-19 Thread Ramana Suvarapu (JIRA)

C++ code generation issues with records when member of the record is of type 
itself
---

 Key: AVRO-825
 URL: https://issues.apache.org/jira/browse/AVRO-825
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.5.1
Reporter: Ramana Suvarapu
Priority: Blocker


I am trying to generate C++ code for the following schema using new avrogencpp. 
This schema is one the of the example given in Avro specification

{
  type: record, 
  name: LongList,
  aliases: [LinkedLongs],  // old name for this
  fields : [
{name: value, type: long}, // each element has a long
{name: next, type: [LongList, null]} // optional next element
  ]
}

It generated the following .hh file and when I tried to compile I am getting 

#ifndef __TEST_LINKEDLIST_HH_1817083040__H_
#define __TEST_LINKEDLIST_HH_1817083040__H_


#include boost/any.hpp
#include Specific.hh
#include Encoder.hh
#include Decoder.hh
struct _imaginary_avr_Union__0__ {
private:
size_t idx_;
boost::any value_;
public:
size_t idx() const { return idx_; }
LongList get_LongList() const {
if (idx_ != 0) {
throw avro::Exception(Invalid type for union);
}
return boost::any_castLongList (value_);
}
void set_LongList(const LongList v) {
idx_ = 0;
value_ = v;
}
void set_null() {
idx_ = 1;
value_ = boost::any();
}
_imaginary_avr_Union__0__() : idx_(0) {
value_ = LongList();
}
};

struct LongList {
int64_t value;
_imaginary_avr_Union__0__ next;
};

namespace avro {
template struct codec_traits_imaginary_avr_Union__0__ {
static void encode(Encoder e, _imaginary_avr_Union__0__ v) {
e.encodeUnionIndex(v.idx());
switch (v.idx()) {
case 0:
avro::encode(e, v.get_LongList());
break;
case 1:
e.encodeNull();
break;
}
}
static void decode(Decoder d, _imaginary_avr_Union__0__ v) {
size_t n = d.decodeUnionIndex();
if (n = 2) { throw avro::Exception(Union index too big); }
switch (n) {
case 0:
{
LongList vv;
avro::decode(d, vv);
v.set_LongList(vv);
}
break;
case 1:
d.decodeNull();
v.set_null();
break;
}
}
};

template struct codec_traitsLongList {
static void encode(Encoder e, const LongList v) {
avro::encode(e, v.value);
avro::encode(e, v.next);
}
static void decode(Decoder d, LongList v) {
avro::decode(d, v.value);
avro::decode(d, v.next);
}
};

}
#endif

When I tried to compile I am getting following errors due to incomplete type.


Errors

In file included from 
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/AvrogencppTests.cc:24:
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:34: error: `LongList' 
does not name a type
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:40: error: expected 
`,' or `...' before '' token
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:40: error: ISO C++ 
forbids declaration of `LongList' with no type
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh: In member function 
`void _imaginary_avr_Union__0__::set_LongList(int)':
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:42: error: `v' was 
not declared in this scope
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh: In constructor 
`_imaginary_avr_Union__0__::_imaginary_avr_Union__0__()':
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:49: error: `LongList' 
was not declared in this scope
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh: In static member 
function `static void 
avro::codec_traits_imaginary_avr_Union__0__::encode(avro::Encoder, 
_imaginary_avr_Union__0__)':
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:64: error: 'struct 
_imaginary_avr_Union__0__' has no member named 'get_LongList'
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh: In static member 
function `static void 
avro::codec_traits_imaginary_avr_Union__0__::decode(avro::Decoder, 
_imaginary_avr_Union__0__)':
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:79: error: no 
matching function for call to 
`_imaginary_avr_Union__0__::set_LongList(LongList)'
/home/rsuvarap/avro-src-1.5.1/lang/c++/test/linkedlist.hh:40: note: candidates 
are: void _imaginary_avr_Union__0__::set_LongList(int)
make[2]: *** [CMakeFiles/AvrogencppTests.dir/test/AvrogencppTests.cc.o] Error 1
make[1]: *** [CMakeFiles/AvrogencppTests.dir/all] Error 2
make: *** [all] Error 2


I think the issue is get_LongList of the union should return pointer to 
LongList instead of value. It's a same issue with c++

46 matches

Mail list logo