[jira] [Updated] (AVRO-1116) C++ code crashes on Data files with no data
[ https://issues.apache.org/jira/browse/AVRO-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1116: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed revision 1357185. > C++ code crashes on Data files with no data > --- > > Key: AVRO-1116 > URL: https://issues.apache.org/jira/browse/AVRO-1116 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.6.0, 1.6.1, 1.6.2, > 1.6.3, 1.7.0 >Reporter: Thiruvalluvan M. G. >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.1 > > Attachments: AVRO-1116.patch > > > It is because a single variable objectCount_ is used to represent the number > of objects in the current block. There is no way to distinguish between the > current block having zero objects and having read all the records in the > current (non-empty) block. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1109) CSharp specific fails on multidimensional arrays.
[
https://issues.apache.org/jira/browse/AVRO-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412749#comment-13412749
]
Thiruvalluvan M. G. commented on AVRO-1109:
---
Verified on Visual C# Express on Windows 7 that the problem exists before the
patch and goes away after the patch.
> CSharp specific fails on multidimensional arrays.
> -
>
> Key: AVRO-1109
> URL: https://issues.apache.org/jira/browse/AVRO-1109
> Project: Avro
> Issue Type: Bug
> Components: csharp
>Affects Versions: 1.5.1, 1.6.3, 1.7.0
> Environment: Windows
>Reporter: Mark Farnan
> Labels: c-sharp,, read, specific, test
> Fix For: 1.7.1
>
> Attachments: AVRO-1109.patch
>
>
> Using Multidimensional Arrays throws errors during READ for Specific.
> This type of construct SHOULD work, but dosn't
> { ""name"" : ""myArray3"", ""type"" : { ""type"" : ""array"", ""items"" : {
> ""type"" : ""array"", ""items"" : [ ""double"", ""string"", ""null"" ] } } }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1109) CSharp specific fails on multidimensional arrays.
[
https://issues.apache.org/jira/browse/AVRO-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1109:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> CSharp specific fails on multidimensional arrays.
> -
>
> Key: AVRO-1109
> URL: https://issues.apache.org/jira/browse/AVRO-1109
> Project: Avro
> Issue Type: Bug
> Components: csharp
>Affects Versions: 1.5.1, 1.6.3, 1.7.0
> Environment: Windows
>Reporter: Mark Farnan
> Labels: c-sharp,, read, specific, test
> Fix For: 1.7.1
>
> Attachments: AVRO-1109.patch
>
>
> Using Multidimensional Arrays throws errors during READ for Specific.
> This type of construct SHOULD work, but dosn't
> { ""name"" : ""myArray3"", ""type"" : { ""type"" : ""array"", ""items"" : {
> ""type"" : ""array"", ""items"" : [ ""double"", ""string"", ""null"" ] } } }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1109) CSharp specific fails on multidimensional arrays.
[
https://issues.apache.org/jira/browse/AVRO-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413032#comment-13413032
]
Thiruvalluvan M. G. commented on AVRO-1109:
---
Committed revision 1360842.
After verifying on Ubuntu 12.04 and fixed a couple of typos.
Thank you Mark Farnan.
> CSharp specific fails on multidimensional arrays.
> -
>
> Key: AVRO-1109
> URL: https://issues.apache.org/jira/browse/AVRO-1109
> Project: Avro
> Issue Type: Bug
> Components: csharp
>Affects Versions: 1.5.1, 1.6.3, 1.7.0
> Environment: Windows
>Reporter: Mark Farnan
> Labels: c-sharp,, read, specific, test
> Fix For: 1.7.1
>
> Attachments: AVRO-1109.patch
>
>
> Using Multidimensional Arrays throws errors during READ for Specific.
> This type of construct SHOULD work, but dosn't
> { ""name"" : ""myArray3"", ""type"" : { ""type"" : ""array"", ""items"" : {
> ""type"" : ""array"", ""items"" : [ ""double"", ""string"", ""null"" ] } } }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-939) Java: optimize BinaryData#compareBytes() to use sun.misc.Unsafe when available
[ https://issues.apache.org/jira/browse/AVRO-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420861#comment-13420861 ] Thiruvalluvan M. G. commented on AVRO-939: -- We seem to violate principles of good programming by using something that is not a part of published API and by accessing private stuff through reflection. The designers of Unsafe would have thought enough and decided to keep it out of the published API and declaring the function private. This is setting a bad precedent. The code will not compile where com.sun.Unsafe is not available. There is no guarantee that it will be present in any Java distribution. Neither is there a guarantee that this will be present in Sun's (Oracle's) distribution in later releases. Traditionally, Java has done not so great job in defining module boundaries. (Package-private is of some use, but not really a great facility). Hadoop community, for example, has taken pains to annotate classes and interfaces to indicate their accessibility outside the module. Java language is trying to fix it in Java 8 using Jigsaw. Once that is in, probably this patch won't work. I'm not sure if marginal performance improvement in a micro-benchmark is a good enough reason. Do we have evidence that this patch would significantly improve performance in a real application? Probably, this debate has happened and settled in the Hadoop community or they considered it a Guava's internal matter. Since we decided to use "Unsafe" directly, we should have an opinion. I'm raising it here because even if we accept this patch, I think, we should acknowledge that we are doing it with full knowledge of its consequences. > Java: optimize BinaryData#compareBytes() to use sun.misc.Unsafe when available > -- > > Key: AVRO-939 > URL: https://issues.apache.org/jira/browse/AVRO-939 > Project: Avro > Issue Type: New Feature > Components: java >Affects Versions: 1.7.1 >Reporter: Doug Cutting > Attachments: AVRO-939-1.patch, AVRO-939-2.patch, AVRO-939-3.patch, > AVRO-939-4.patch, AVRO-939.patch > > > Google's Guava libraries include an optimized implementation of lexicographic > byte comparison based on sun.misc.Unsafe that's ~4x faster than the normal > Java implementation. > http://hiroshiyamauchi.blogspot.com/2010/08/fast-unsigned-byte-lexicographical.html > http://www.google.com/codesearch#UKMs0lhE9bg/trunk/src/com/google/common/primitives/UnsignedBytes.java&l=276 > We might similarly optimize BinaryData#compareBytes(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1131) Generated build makefiles for MSYS/MinGW use Visual Studio compiler flags
[ https://issues.apache.org/jira/browse/AVRO-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1131: -- Assignee: Laurent Moss Status: Patch Available (was: Open) > Generated build makefiles for MSYS/MinGW use Visual Studio compiler flags > - > > Key: AVRO-1131 > URL: https://issues.apache.org/jira/browse/AVRO-1131 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 > Environment: Windows XP Professional 32-bit SP3, MSYS, MinGW GCC 4.5.1 >Reporter: Laurent Moss >Assignee: Laurent Moss >Priority: Minor > Labels: build > Attachments: AVRO-1131.diff > > > I generated MSYS Makefiles for Avro C++ by executing the following command in > the avro/lang/c++ directory: > cmake -G "MSYS Makefiles" > However, when trying to "make" Avro C++ with the generated makefiles and with > MinGW GCC, the build fails with the following error: > g++.exe: C:/msys/EHa: No such file or directory > make[2]: *** [CMakeFiles/avrocpp_s.dir/impl/Compiler.cc.obj] Error 1 > make[1]: *** [CMakeFiles/avrocpp_s.dir/all] Error 2 > make: *** [all] Error 2 > It turns out that the Visual Studio compiler flag "/Eha" was added to the > generated makefiles, which MSYS then interpreted as a file path. MinGW GCC > would not understand or need that flag anyway. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1131) Generated build makefiles for MSYS/MinGW use Visual Studio compiler flags
[ https://issues.apache.org/jira/browse/AVRO-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1131: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed revision 1364872. > Generated build makefiles for MSYS/MinGW use Visual Studio compiler flags > - > > Key: AVRO-1131 > URL: https://issues.apache.org/jira/browse/AVRO-1131 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 > Environment: Windows XP Professional 32-bit SP3, MSYS, MinGW GCC 4.5.1 >Reporter: Laurent Moss >Assignee: Laurent Moss >Priority: Minor > Labels: build > Attachments: AVRO-1131.diff > > > I generated MSYS Makefiles for Avro C++ by executing the following command in > the avro/lang/c++ directory: > cmake -G "MSYS Makefiles" > However, when trying to "make" Avro C++ with the generated makefiles and with > MinGW GCC, the build fails with the following error: > g++.exe: C:/msys/EHa: No such file or directory > make[2]: *** [CMakeFiles/avrocpp_s.dir/impl/Compiler.cc.obj] Error 1 > make[1]: *** [CMakeFiles/avrocpp_s.dir/all] Error 2 > make: *** [all] Error 2 > It turns out that the Visual Studio compiler flag "/Eha" was added to the > generated makefiles, which MSYS then interpreted as a file path. MinGW GCC > would not understand or need that flag anyway. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1133) Build failing with Visual Studio C++ 2008 due to missing stdint.h
[ https://issues.apache.org/jira/browse/AVRO-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421132#comment-13421132 ] Thiruvalluvan M. G. commented on AVRO-1133: --- We have to make a tradeoff here. The boost solution is better because we already depend on boost and no new dependency gets added. But all references to int8_t etc should now become boost::int8_t, which doesn't look nice. On the other hand, the changes are minimal with msinttypes, but we need to copy the code from google code. My preference is for the latter. Is it possible for you submit a patch after verifying that it works with VS 2008? > Build failing with Visual Studio C++ 2008 due to missing stdint.h > - > > Key: AVRO-1133 > URL: https://issues.apache.org/jira/browse/AVRO-1133 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 > Environment: Windows XP Professional 32-bit SP3, Microsoft Visual > Studio 2008 SP1 >Reporter: Laurent Moss > > Several Avro C++ API files refer to stdint.h. However, this file is not > available on Microsoft Visual Studio 2008 (and previous versions). This > results in several build errors such as: > C:\workspace\avro-cpp\api\Validator.hh(24) : fatal error C1083: Cannot open > include file: 'stdint.h': No such file or directory > This is similar to an issue previously faced by the Avro C API: > https://issues.apache.org/jira/browse/AVRO-551 > This was issue was fixed in the Avro C API by integrating open-source ISO C9x > compliant stdint.h and inttypes.h files for Microsoft Visual Studio: > https://code.google.com/p/msinttypes/ > An alternative for the Avro C++ API would be to replace references to > stdint.h by references to Boost's cstdint.hpp > http://www.boost.org/doc/libs/1_50_0/boost/cstdint.hpp -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1132) Build failing on MSYS/MinGW due to missing struct iovec
[ https://issues.apache.org/jira/browse/AVRO-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421244#comment-13421244 ] Thiruvalluvan M. G. commented on AVRO-1132: --- I don't have MinGW or MSYS. But this patch appears inappropriate as it comments out portions of the code for all Win32 platforms (Visual Studio, Cygwin and MinGW). Probably, nobody will be affected because the the offending code is very old and is not used in the newer API since Avro 1.5 or so. But who knows, someone may still be using it. I think we should officially discontinue supporting the buffer based API (prevalent before 1.5). Since that would mean incompatibility it can only be done in 1.8 release. Since it works with gcc on Linux and Cygwin, there must be something peculiar to MinGW/MSYS. Is it possible to figure that out? Alternatively, we can comment out the offending portion just for MinGW, if you prefer. Not perfect, but better than breaking other Win32 platforms. Thanks > Build failing on MSYS/MinGW due to missing struct iovec > --- > > Key: AVRO-1132 > URL: https://issues.apache.org/jira/browse/AVRO-1132 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 > Environment: Windows XP Professional 32-bit SP3, MSYS, MinGW GCC 4.5.1 >Reporter: Laurent Moss >Assignee: Laurent Moss > Labels: build > Fix For: 1.7.2 > > Attachments: AVRO-1132.diff > > > Avro C++ fails to build on MSYS with MinGW GCC due to references to > undeclared struct iovec: > In file included from C:/workspace/avro-cpp/api/buffer/BufferReader.hh:22:0, > from C:/workspace/avro-cpp/api/Reader.hh:30, > from C:/workspace/avro-cpp/api/ResolverSchema.hh:28, > from c:/workspace/avro-cpp/impl/ResolverSchema.cc:20: > C:/workspace/avro-cpp/api/buffer/Buffer.hh: In function 'void > avro::toIovec(BufferType&, std::vector&)': > C:/workspace/avro-cpp/api/buffer/Buffer.hh:517:15: error: invalid use of > incomplete type 'struct avro::iovec' > C:/workspace/avro-cpp/api/buffer/Buffer.hh:511:57: error: forward declaration > of 'struct avro::iovec' > C:/workspace/avro-cpp/api/buffer/Buffer.hh:518:15: error: invalid use of > incomplete type 'struct avro::iovec' > C:/workspace/avro-cpp/api/buffer/Buffer.hh:511:57: error: forward declaration > of 'struct avro::iovec' > make[2]: *** [CMakeFiles/avrocpp_s.dir/impl/ResolverSchema.cc.obj] Error 1 > make[1]: *** [CMakeFiles/avrocpp_s.dir/all] Error 2 > make: *** [all] Error 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (AVRO-1135) Avro C++ fails to build on Mac
Thiruvalluvan M. G. created AVRO-1135: - Summary: Avro C++ fails to build on Mac Key: AVRO-1135 URL: https://issues.apache.org/jira/browse/AVRO-1135 Project: Avro Issue Type: Bug Components: c++ Reporter: Thiruvalluvan M. G. Assignee: Thiruvalluvan M. G. The file unittest.cc fails to compile because the compiler gets confused about the overloaded function readFixed in Parser class. There are two overloaded functions with the name readFixed, both templates taking size_t as template arguments. When the template argument is passed explicitly, the Mac compiler is not able to resolve the overload. The forthcoming patch addresses that issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1135) Avro C++ fails to build on Mac
[ https://issues.apache.org/jira/browse/AVRO-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1135: -- Fix Version/s: 1.7.2 Affects Version/s: 1.7.1 Status: Patch Available (was: Open) > Avro C++ fails to build on Mac > -- > > Key: AVRO-1135 > URL: https://issues.apache.org/jira/browse/AVRO-1135 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Thiruvalluvan M. G. >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.2 > > Attachments: AVRO-1135.patch > > > The file unittest.cc fails to compile because the compiler gets confused > about the overloaded function readFixed in Parser class. There are two > overloaded functions with the name readFixed, both templates taking size_t as > template arguments. When the template argument is passed explicitly, the Mac > compiler is not able to resolve the overload. The forthcoming patch addresses > that issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1135) Avro C++ fails to build on Mac
[ https://issues.apache.org/jira/browse/AVRO-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1135: -- Attachment: AVRO-1135.patch > Avro C++ fails to build on Mac > -- > > Key: AVRO-1135 > URL: https://issues.apache.org/jira/browse/AVRO-1135 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Thiruvalluvan M. G. >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.2 > > Attachments: AVRO-1135.patch > > > The file unittest.cc fails to compile because the compiler gets confused > about the overloaded function readFixed in Parser class. There are two > overloaded functions with the name readFixed, both templates taking size_t as > template arguments. When the template argument is passed explicitly, the Mac > compiler is not able to resolve the overload. The forthcoming patch addresses > that issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1135) Avro C++ fails to build on Mac
[ https://issues.apache.org/jira/browse/AVRO-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428794#comment-13428794 ] Thiruvalluvan M. G. commented on AVRO-1135: --- I need to check if this patch breaks anything in other platforms, which have been working so far - Windows, Cygwin and Linux. > Avro C++ fails to build on Mac > -- > > Key: AVRO-1135 > URL: https://issues.apache.org/jira/browse/AVRO-1135 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Thiruvalluvan M. G. >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.2 > > Attachments: AVRO-1135.patch > > > The file unittest.cc fails to compile because the compiler gets confused > about the overloaded function readFixed in Parser class. There are two > overloaded functions with the name readFixed, both templates taking size_t as > template arguments. When the template argument is passed explicitly, the Mac > compiler is not able to resolve the overload. The forthcoming patch addresses > that issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1135) Avro C++ fails to build on Mac
[ https://issues.apache.org/jira/browse/AVRO-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437702#comment-13437702 ] Thiruvalluvan M. G. commented on AVRO-1135: --- Verified that this patch works on Windows and Cygwin > Avro C++ fails to build on Mac > -- > > Key: AVRO-1135 > URL: https://issues.apache.org/jira/browse/AVRO-1135 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Thiruvalluvan M. G. >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.2 > > Attachments: AVRO-1135.patch > > > The file unittest.cc fails to compile because the compiler gets confused > about the overloaded function readFixed in Parser class. There are two > overloaded functions with the name readFixed, both templates taking size_t as > template arguments. When the template argument is passed explicitly, the Mac > compiler is not able to resolve the overload. The forthcoming patch addresses > that issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1140) Buffer.hh includes Config.hh without "../"
[ https://issues.apache.org/jira/browse/AVRO-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437706#comment-13437706 ] Thiruvalluvan M. G. commented on AVRO-1140: --- Verified that the fix works on Windows (Visual Studio Express 2010) and Cygwin > Buffer.hh includes Config.hh without "../" > -- > > Key: AVRO-1140 > URL: https://issues.apache.org/jira/browse/AVRO-1140 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Jan van der Lugt > Attachments: AVRO-1140.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > Buffer.hh includes the file Config.hh, which is located in the parent > directory. Config.hh is included like via #include "Config.hh", but this > should be #include "../Config.hh". Both work when compiling the AVRO > distribution, but if you install AVRO using make install, g++ complains that > it can't find Config.hh, which is included via DataFile.hh in my case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1135) Avro C++ fails to build on Mac
[ https://issues.apache.org/jira/browse/AVRO-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1135: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed revision 1374935. > Avro C++ fails to build on Mac > -- > > Key: AVRO-1135 > URL: https://issues.apache.org/jira/browse/AVRO-1135 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Thiruvalluvan M. G. >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.2 > > Attachments: AVRO-1135.patch > > > The file unittest.cc fails to compile because the compiler gets confused > about the overloaded function readFixed in Parser class. There are two > overloaded functions with the name readFixed, both templates taking size_t as > template arguments. When the template argument is passed explicitly, the Mac > compiler is not able to resolve the overload. The forthcoming patch addresses > that issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1140) Buffer.hh includes Config.hh without "../"
[ https://issues.apache.org/jira/browse/AVRO-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1140: -- Resolution: Fixed Status: Resolved (was: Patch Available) Verified that the patch works on Mac. Committed revision 1374938. Thank you Jan van der Lugt. > Buffer.hh includes Config.hh without "../" > -- > > Key: AVRO-1140 > URL: https://issues.apache.org/jira/browse/AVRO-1140 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Jan van der Lugt > Attachments: AVRO-1140.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > Buffer.hh includes the file Config.hh, which is located in the parent > directory. Config.hh is included like via #include "Config.hh", but this > should be #include "../Config.hh". Both work when compiling the AVRO > distribution, but if you install AVRO using make install, g++ complains that > it can't find Config.hh, which is included via DataFile.hh in my case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1141) Avro data files are created without O_TRUNC
[ https://issues.apache.org/jira/browse/AVRO-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1141: -- Attachment: AVRO-1141.patch +1 Tested on Cygwin. Added a unit-test to catch the problem. > Avro data files are created without O_TRUNC > --- > > Key: AVRO-1141 > URL: https://issues.apache.org/jira/browse/AVRO-1141 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Martin Nagy > Attachments: AVRO-1141.patch, cpp_filestream_o_trunc.patch > > > Avro data files are created without O_TRUNC. If the file existed before and > the new file is smaller than the old one, the resulting data file will be > corrupted (containing remnants of the old file). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (AVRO-1141) Avro data files are created without O_TRUNC
[ https://issues.apache.org/jira/browse/AVRO-1141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. resolved AVRO-1141. --- Resolution: Fixed Committed revision 1377567. After verifying that the fix works with Mac and Visual Studio. Than you Martin Nagy. > Avro data files are created without O_TRUNC > --- > > Key: AVRO-1141 > URL: https://issues.apache.org/jira/browse/AVRO-1141 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Martin Nagy > Attachments: AVRO-1141.patch, cpp_filestream_o_trunc.patch > > > Avro data files are created without O_TRUNC. If the file existed before and > the new file is smaller than the old one, the resulting data file will be > corrupted (containing remnants of the old file). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1143) avrogencpp generates $Undefined$ for some union types
[ https://issues.apache.org/jira/browse/AVRO-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1143: -- Assignee: Thiruvalluvan M. G. Status: Patch Available (was: Open) > avrogencpp generates $Undefined$ for some union types > - > > Key: AVRO-1143 > URL: https://issues.apache.org/jira/browse/AVRO-1143 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Jan van der Lugt >Assignee: Thiruvalluvan M. G. > Attachments: AVRO-1143.patch, gm_avro_graph.avpr > > > When converting the attached file to a C++ header, a vector of type > $Undefined$ is generated. I believe this is because there is no check for > AVRO_UNION in CodeGen::cppTypeOf(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1143) avrogencpp generates $Undefined$ for some union types
[ https://issues.apache.org/jira/browse/AVRO-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1143: -- Attachment: AVRO-1143.patch Good catch. Attached patch has a modified test that catches the problem and the fix for the problem. I'm sorry I couldn't resolve the issue earlier. > avrogencpp generates $Undefined$ for some union types > - > > Key: AVRO-1143 > URL: https://issues.apache.org/jira/browse/AVRO-1143 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Jan van der Lugt > Attachments: AVRO-1143.patch, gm_avro_graph.avpr > > > When converting the attached file to a C++ header, a vector of type > $Undefined$ is generated. I believe this is because there is no check for > AVRO_UNION in CodeGen::cppTypeOf(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1143) avrogencpp generates $Undefined$ for some union types
[ https://issues.apache.org/jira/browse/AVRO-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1143: -- Resolution: Fixed Fix Version/s: 1.7.2 Status: Resolved (was: Patch Available) Committed revision 1382397. Thank you Jan van der Lugt for verifying the fix. > avrogencpp generates $Undefined$ for some union types > - > > Key: AVRO-1143 > URL: https://issues.apache.org/jira/browse/AVRO-1143 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 >Reporter: Jan van der Lugt >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.2 > > Attachments: AVRO-1143.patch, gm_avro_graph.avpr > > > When converting the attached file to a C++ header, a vector of type > $Undefined$ is generated. I believe this is because there is no check for > AVRO_UNION in CodeGen::cppTypeOf(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1172) Avro C++ Json Decoder: Double cannot be decoded
[
https://issues.apache.org/jira/browse/AVRO-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1172:
--
Attachment: AVRO-1172.patch
Good catch.
This patch fixes the problem.
In the sample code, instead of flushing the stream, if the encoder is flushes,
the terminating close-braces would get written.
> Avro C++ Json Decoder: Double cannot be decoded
> ---
>
> Key: AVRO-1172
> URL: https://issues.apache.org/jira/browse/AVRO-1172
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.1
> Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit
> machine.
>Reporter: Sam Overend
> Labels: patch
> Attachments: AVRO-1172.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Short version: Looks like the C++ version of AVRO-1099.
> Long version: When a non-decimal double is read from a json file, the parser
> treats it as an long, not a double, and therefore throws an exception.
> Two possible solutions: (1) The decoder should be able to convert longs to
> doubles or acknowledge that a long is a type of double.
> (2) The encoder should always output a double with a decimal point.
> Example code is included below. Output is:
> (1.01, 2.13)
> terminate called after throwing an instance of 'avro::Exception'
> what(): Incorrect token in the stream. Expected: Double, found Integer
> After running complex.json is: {"re":1,"im":2.13
> #include
> #include
> using namespace std;
> #include "cpx.hh"
> #include "avro/Compiler.hh"
> #include "avro/Encoder.hh"
> #include "avro/Decoder.hh"
> avro::ValidSchema load(const char* filename)
> {
> std::ifstream ifs(filename);
> avro::ValidSchema result;
> avro::compileJsonSchema(ifs, result);
> return result;
> }
> void OutTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.01;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void OutTest2()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.0;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void InTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr in =
> avro::fileInputStream("complex.json",1);
> avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
> d->init(*in);
> c::cpx c2;
> avro::decode(*d, c2);
> std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
> }
> int main()
> {
> OutTest();
> InTest();
> OutTest2();
> InTest();
> return 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1172) Avro C++ Json Decoder: Double cannot be decoded
[
https://issues.apache.org/jira/browse/AVRO-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1172:
--
Status: Patch Available (was: Open)
> Avro C++ Json Decoder: Double cannot be decoded
> ---
>
> Key: AVRO-1172
> URL: https://issues.apache.org/jira/browse/AVRO-1172
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.1
> Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit
> machine.
>Reporter: Sam Overend
> Labels: patch
> Attachments: AVRO-1172.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Short version: Looks like the C++ version of AVRO-1099.
> Long version: When a non-decimal double is read from a json file, the parser
> treats it as an long, not a double, and therefore throws an exception.
> Two possible solutions: (1) The decoder should be able to convert longs to
> doubles or acknowledge that a long is a type of double.
> (2) The encoder should always output a double with a decimal point.
> Example code is included below. Output is:
> (1.01, 2.13)
> terminate called after throwing an instance of 'avro::Exception'
> what(): Incorrect token in the stream. Expected: Double, found Integer
> After running complex.json is: {"re":1,"im":2.13
> #include
> #include
> using namespace std;
> #include "cpx.hh"
> #include "avro/Compiler.hh"
> #include "avro/Encoder.hh"
> #include "avro/Decoder.hh"
> avro::ValidSchema load(const char* filename)
> {
> std::ifstream ifs(filename);
> avro::ValidSchema result;
> avro::compileJsonSchema(ifs, result);
> return result;
> }
> void OutTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.01;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void OutTest2()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.0;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void InTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr in =
> avro::fileInputStream("complex.json",1);
> avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
> d->init(*in);
> c::cpx c2;
> avro::decode(*d, c2);
> std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
> }
> int main()
> {
> OutTest();
> InTest();
> OutTest2();
> InTest();
> return 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1173) C++ API for dynamic reading/writing based on schema
[ https://issues.apache.org/jira/browse/AVRO-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464049#comment-13464049 ] Thiruvalluvan M. G. commented on AVRO-1173: --- C++ Generic is meant for reading and writing of dynamic content. One can read from and write into streams or data file using generic object. Did you look into Generic.hh? > C++ API for dynamic reading/writing based on schema > --- > > Key: AVRO-1173 > URL: https://issues.apache.org/jira/browse/AVRO-1173 > Project: Avro > Issue Type: Wish > Components: c++ >Reporter: Stefan Langer > > When I started looking at Avro I hoped it would offer some API to read values > by name/id (or at least get name/id of datum while iterating over all > entries). > When looking at examples for C: > http://avro.apache.org/docs/1.6.3/api/c/index.html#_examples > ... or some Java examples > There are getters/setters which have name-arguments, and there are > Record-objects constructed from schema which help reading/writing data. > While testing the C++ API, I couldn't find a way to do so with it! > I'm still not sure if I'm missing some part of the API or if it is just not > yet part of the C++ Interface. > About C API: I could not use it, because it is C99 focused, so it can't be > compiled on our VS2008 ... For the C++ API it's just some tiny tweaks to get > it running. > About Generator: I'm not interested in generating code (if I would be there > are enough alternatives to Avro ...) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1178) Typos in LL(1) CFG document
[ https://issues.apache.org/jira/browse/AVRO-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477582#comment-13477582 ] Thiruvalluvan M. G. commented on AVRO-1178: --- +1 Looks great! Thanks for improving this. I'll commit this soon, unless someone objects. > Typos in LL(1) CFG document > --- > > Key: AVRO-1178 > URL: https://issues.apache.org/jira/browse/AVRO-1178 > Project: Avro > Issue Type: Bug >Affects Versions: 1.7.2 >Reporter: Martin Kleppmann >Priority: Trivial > Attachments: AVRO-1178.patch > > > Numerous typos in > lang/java/avro/src/main/java/org/apache/avro/io/parsing/doc-files/parsing.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1189) cpp build requires cmake 2.8.4
[ https://issues.apache.org/jira/browse/AVRO-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490450#comment-13490450 ] Thiruvalluvan M. G. commented on AVRO-1189: --- This patch works well with single configuration systems like Unix Makefile. But with multi configuration systems like Mac and Windows, ctest -c fails (it doesn't generate correct executable name). Ctest is a handy tool to run all tests at once on these platforms. I think it's better that we bump up the requirement to 2.8.4, which is anyway 18 months old. An alternative is to remove working directory requirements in the tests, which requires little bit of work. > cpp build requires cmake 2.8.4 > -- > > Key: AVRO-1189 > URL: https://issues.apache.org/jira/browse/AVRO-1189 > Project: Avro > Issue Type: Bug > Components: build, c++ >Affects Versions: 1.7.2 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Attachments: avro-1189.txt > > > The cmake_minimum_required setting for the C++ build is 2.6, but when I try > to compile using cmake 2.8.2 I get an error because add_test doesn't > understand the WORKING_DIRECTORY property. It turns out this was added in > cmake 2.8.4. The CMakeLists.txt should either be updated to use some kind of > workaround, or bump the minimum requirement to 2.8.4 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1182) DataFileReader missing seek, sync methods
[
https://issues.apache.org/jira/browse/AVRO-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568444#comment-13568444
]
Thiruvalluvan M. G. commented on AVRO-1182:
---
Daniel,
It appears that you have attached the entire C++ source tree as the patch. Is
it possible to submit just the differences?
You can generate patches using something like this at top level:
{{svn diff > AVRO-1182.patch}}
or
{{git diff > AVRO-1182.patch}}
depending on if you work with svn or git
You can apply the patch using:
{{patch -p0 < AVRO-1182.patch}}
> DataFileReader missing seek, sync methods
> -
>
> Key: AVRO-1182
> URL: https://issues.apache.org/jira/browse/AVRO-1182
> Project: Avro
> Issue Type: Improvement
> Components: c++
>Affects Versions: 1.7.3
>Reporter: Daniel Russel
> Attachments: add_seek
>
>
> The DataFileReader is missing the seek and sync methods that are found in the
> java version making it hard to navigate a file except in a linear fashion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1182) DataFileReader missing seek, sync methods
[
https://issues.apache.org/jira/browse/AVRO-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568470#comment-13568470
]
Thiruvalluvan M. G. commented on AVRO-1182:
---
Thanks Daniel.
The patch looks great. A few minor issues:
* It appears that your editor inserts hard-tabs. We use 4 spaces instead of
tabs so that the code looks the same in all editors and in patches.
* A few styling issues. E.g space after '{' or before '}' in single-line
functions or space before and after binary operators. (If it is hard to fix,
don't bother. I'll have it fixed before we check in).
* Since you are using int64_t for sizeBytes() and blockOffsetBytes() in may be
prudent to use the same (instead of size_t) for seekBlockBytes() as well,
especially since it refers to the offset from the beginning of the file.
* The documentation for remainingBytes() in stream.hh is somewhat ambiguous.
Since the zero-copy streams don't have a file pointer, it has a range of bytes
in the exposed buffer, you want to specify if remainingBytes() is the number of
bytes remaining from the beginning or end of the exposed buffer. I prefer it to
refer from the end of the exposed buffer.
* sync_match can be made a bit faster if we replace indexes with pointers.
> DataFileReader missing seek, sync methods
> -
>
> Key: AVRO-1182
> URL: https://issues.apache.org/jira/browse/AVRO-1182
> Project: Avro
> Issue Type: Improvement
> Components: c++
>Affects Versions: 1.7.3
>Reporter: Daniel Russel
> Attachments: add_seek
>
>
> The DataFileReader is missing the seek and sync methods that are found in the
> java version making it hard to navigate a file except in a linear fashion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1290) AvroCpp json text backend produces "inf" for double values of infinity but won't consume them
[
https://issues.apache.org/jira/browse/AVRO-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13680577#comment-13680577
]
Thiruvalluvan M. G. commented on AVRO-1290:
---
JSON specification does not say how to encode +infinity, -infinity or NaN.
Consequently other language bindings of Avro do not honor infinity etc.
Interestingly, Jackson, a popular Java JSON library, encodes +infinity,
-infinity and Nan as strings "Infinity", "-Infinity" and "NaN" respectively. It
is also able to decode them properly back to appropriate data. Ruby's string
conversion of these are exactly the same as Jackson convention. However, Ruby's
to_json method that comes with the language runtime refuses to encode
+infinity, -infinity or NaN. Apparently PHP does something very different:
http://stackoverflow.com/questions/13581843/php-how-to-encode-infinity-or-nan-numbers-to-json
We can do either of the following:
1. Throw an exception if we encounter +infinity, -infinity or NaN while
encoding into JSON. (Ruby behavior)
2. Stick to Jackson/Ruby convention. This is better than inventing a new
convention of our own.
My personal preference is for (1), because that conforms to JSON standard. But
if others think (2) is better, we can go with it as well.
If we go with option (2), the format for serialization would be as follows.
Let's say we have an object with a single double field "d" in it. Examples of
encoded JSON are:
{ "d": 1.45 }
{ "d": "Infinity" }
{ "d": "-Infinity" }
{ "d": "NaN" }
Please note that the double is encoded a string if it is one of the special
values.
> AvroCpp json text backend produces "inf" for double values of infinity but
> won't consume them
> -
>
> Key: AVRO-1290
> URL: https://issues.apache.org/jira/browse/AVRO-1290
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.3
>Reporter: Daniel Russel
> Attachments: patch
>
>
> If you use the json encoder and pass it a double with value e.g.
> std::numeric_limits::infinity() it happily writes the literal "inf".
> However, the decoder chokes on that literal.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1290) AvroCpp json text backend produces "inf" for double values of infinity but won't consume them
[ https://issues.apache.org/jira/browse/AVRO-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1290: -- Attachment: AVRO-1290.patch If we choose to go with Jackson style encoding of +/- infinity and NaN, this patch implements it. > AvroCpp json text backend produces "inf" for double values of infinity but > won't consume them > - > > Key: AVRO-1290 > URL: https://issues.apache.org/jira/browse/AVRO-1290 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.3 >Reporter: Daniel Russel > Attachments: AVRO-1290.patch, patch > > > If you use the json encoder and pass it a double with value e.g. > std::numeric_limits::infinity() it happily writes the literal "inf". > However, the decoder chokes on that literal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1181) compileJsonSchemaFromString(std::string) declared in Compiler.hh but not defined
[
https://issues.apache.org/jira/browse/AVRO-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13680589#comment-13680589
]
Thiruvalluvan M. G. commented on AVRO-1181:
---
+1.
I'll commit this tomorrow unless there are any objections.
> compileJsonSchemaFromString(std::string) declared in Compiler.hh but not
> defined
>
>
> Key: AVRO-1181
> URL: https://issues.apache.org/jira/browse/AVRO-1181
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.2, 1.7.3, 1.7.4
> Environment: linux, mac os
>Reporter: Daniel Russel
> Attachments: AVRO-1181.patch
>
>
> compileJsonSchemaFromString(std::string) is declared in the header, but not
> defined. Need to add
> +AVRO_DECL ValidSchema compileJsonSchemaFromString(const std::string& input)
> +{
> + return compileJsonSchemaFromString(input.c_str());
> +}
> +
> to the Compiler.cc or remove the prototype.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1182) DataFileReader missing seek, sync methods
[ https://issues.apache.org/jira/browse/AVRO-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13680593#comment-13680593 ] Thiruvalluvan M. G. commented on AVRO-1182: --- I'm sorry I took this long to respond on this. The patch would work in most situations. But, as you point out being not tested, if partial sync record has been read into current buffer, it will fail. When such a situation happens, it will be very hard to debug. Is it possible to address that problem? I understand it is not straightforward because of no-copy input streams. > DataFileReader missing seek, sync methods > - > > Key: AVRO-1182 > URL: https://issues.apache.org/jira/browse/AVRO-1182 > Project: Avro > Issue Type: Improvement > Components: c++ >Affects Versions: 1.7.3 >Reporter: Daniel Russel > Attachments: add_seek > > > The DataFileReader is missing the seek and sync methods that are found in the > java version making it hard to navigate a file except in a linear fashion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1172) Avro C++ Json Decoder: Double cannot be decoded
[
https://issues.apache.org/jira/browse/AVRO-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13680598#comment-13680598
]
Thiruvalluvan M. G. commented on AVRO-1172:
---
I just submitted a patch to AVRO-1290 which sometimes treats incoming strings
as double. This situation can be handled the same way. I'll have this fixed
once 1290 is resolved, which will be in the next 2 days or so.
> Avro C++ Json Decoder: Double cannot be decoded
> ---
>
> Key: AVRO-1172
> URL: https://issues.apache.org/jira/browse/AVRO-1172
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.1
> Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit
> machine.
>Reporter: Sam Overend
> Labels: patch
> Attachments: AVRO-1172.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Short version: Looks like the C++ version of AVRO-1099.
> Long version: When a non-decimal double is read from a json file, the parser
> treats it as an long, not a double, and therefore throws an exception.
> Two possible solutions: (1) The decoder should be able to convert longs to
> doubles or acknowledge that a long is a type of double.
> (2) The encoder should always output a double with a decimal point.
> Example code is included below. Output is:
> (1.01, 2.13)
> terminate called after throwing an instance of 'avro::Exception'
> what(): Incorrect token in the stream. Expected: Double, found Integer
> After running complex.json is: {"re":1,"im":2.13
> #include
> #include
> using namespace std;
> #include "cpx.hh"
> #include "avro/Compiler.hh"
> #include "avro/Encoder.hh"
> #include "avro/Decoder.hh"
> avro::ValidSchema load(const char* filename)
> {
> std::ifstream ifs(filename);
> avro::ValidSchema result;
> avro::compileJsonSchema(ifs, result);
> return result;
> }
> void OutTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.01;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void OutTest2()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.0;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void InTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr in =
> avro::fileInputStream("complex.json",1);
> avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
> d->init(*in);
> c::cpx c2;
> avro::decode(*d, c2);
> std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
> }
> int main()
> {
> OutTest();
> InTest();
> OutTest2();
> InTest();
> return 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1181) compileJsonSchemaFromString(std::string) declared in Compiler.hh but not defined
[
https://issues.apache.org/jira/browse/AVRO-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1181:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 1492812.
Thank you Daniel.
> compileJsonSchemaFromString(std::string) declared in Compiler.hh but not
> defined
>
>
> Key: AVRO-1181
> URL: https://issues.apache.org/jira/browse/AVRO-1181
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.2, 1.7.3, 1.7.4
> Environment: linux, mac os
>Reporter: Daniel Russel
> Attachments: AVRO-1181.patch
>
>
> compileJsonSchemaFromString(std::string) is declared in the header, but not
> defined. Need to add
> +AVRO_DECL ValidSchema compileJsonSchemaFromString(const std::string& input)
> +{
> + return compileJsonSchemaFromString(input.c_str());
> +}
> +
> to the Compiler.cc or remove the prototype.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (AVRO-1181) compileJsonSchemaFromString(std::string) declared in Compiler.hh but not defined
[
https://issues.apache.org/jira/browse/AVRO-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. reassigned AVRO-1181:
-
Assignee: Daniel Russel
> compileJsonSchemaFromString(std::string) declared in Compiler.hh but not
> defined
>
>
> Key: AVRO-1181
> URL: https://issues.apache.org/jira/browse/AVRO-1181
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.2, 1.7.3, 1.7.4
> Environment: linux, mac os
>Reporter: Daniel Russel
>Assignee: Daniel Russel
> Attachments: AVRO-1181.patch
>
>
> compileJsonSchemaFromString(std::string) is declared in the header, but not
> defined. Need to add
> +AVRO_DECL ValidSchema compileJsonSchemaFromString(const std::string& input)
> +{
> + return compileJsonSchemaFromString(input.c_str());
> +}
> +
> to the Compiler.cc or remove the prototype.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1290) Handling NaN and positive and negative infinities in C++ Json
[ https://issues.apache.org/jira/browse/AVRO-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1290: -- Summary: Handling NaN and positive and negative infinities in C++ Json (was: AvroCpp json text backend produces "inf" for double values of infinity but won't consume them) > Handling NaN and positive and negative infinities in C++ Json > - > > Key: AVRO-1290 > URL: https://issues.apache.org/jira/browse/AVRO-1290 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.3 >Reporter: Daniel Russel > Attachments: AVRO-1290.patch, patch > > > If you use the json encoder and pass it a double with value e.g. > std::numeric_limits::infinity() it happily writes the literal "inf". > However, the decoder chokes on that literal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1290) Handling NaN and positive and negative infinities in C++ Json
[ https://issues.apache.org/jira/browse/AVRO-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1290: -- Resolution: Fixed Assignee: Daniel Russel Status: Resolved (was: Patch Available) Committed revision 1492821. Thank you Daniel. > Handling NaN and positive and negative infinities in C++ Json > - > > Key: AVRO-1290 > URL: https://issues.apache.org/jira/browse/AVRO-1290 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.3 >Reporter: Daniel Russel >Assignee: Daniel Russel > Attachments: AVRO-1290.patch, patch > > > If you use the json encoder and pass it a double with value e.g. > std::numeric_limits::infinity() it happily writes the literal "inf". > However, the decoder chokes on that literal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (AVRO-1080) JsonIO.cc should allow \u escape sequence in string
[
https://issues.apache.org/jira/browse/AVRO-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. reassigned AVRO-1080:
-
Assignee: Keh-Li Sheng
> JsonIO.cc should allow \u escape sequence in string
> ---
>
> Key: AVRO-1080
> URL: https://issues.apache.org/jira/browse/AVRO-1080
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.6.3
> Environment: C++
>Reporter: Keh-Li Sheng
>Assignee: Keh-Li Sheng
> Fix For: 1.7.0
>
> Attachments: AVRO-1080.patch
>
>
> If an avro string contains a unicode escape sequence that begins with "\u"
> instead of "\U" an exception is thrown by the parser. The problematic code is
> at JsonIO.cc line 269.
> {code}
> JsonParser::Token JsonParser::tryString()
> {
> sv.clear();
> for ( ; ;) {
> char ch = in_.read();
> if (ch == '"') {
> return tkString;
> } else if (ch == '\\') {
> ch = in_.read();
> switch (ch) {
> case '"':
> case '\\':
> case '/':
> sv.push_back(ch);
> continue;
> case 'b':
> sv.push_back('\b');
> continue;
> case 'f':
> sv.push_back('\f');
> continue;
> case 'n':
> sv.push_back('\n');
> continue;
> case 'r':
> sv.push_back('\r');
> continue;
> case 't':
> sv.push_back('\t');
> continue;
> case 'U':
> {
> unsigned int n = 0;
> char e[4];
> in_.readBytes(reinterpret_cast(e), 4);
> for (int i = 0; i < 4; i++) {
> n *= 16;
> char c = e[i];
> if (isdigit(c)) {
> n += c - '0';
> } else if (c >= 'a' && c <= 'f') {
> n += c - 'a' + 10;
> } else if (c >= 'A' && c <= 'F') {
> n += c - 'A' + 10;
> } else {
> unexpected(c);
> }
> }
> sv.push_back(n);
> }
> break;
> default:
> unexpected(ch);
> }
> } else {
> sv.push_back(ch);
> }
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1172) Avro C++ Json Decoder: Double cannot be decoded
[
https://issues.apache.org/jira/browse/AVRO-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1172:
--
Attachment: AVRO-1172-2.patch
A slightly better version which exploits recent patch for AVRO-1290 along with
unit-tests.
> Avro C++ Json Decoder: Double cannot be decoded
> ---
>
> Key: AVRO-1172
> URL: https://issues.apache.org/jira/browse/AVRO-1172
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.1
> Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit
> machine.
>Reporter: Sam Overend
> Labels: patch
> Attachments: AVRO-1172-2.patch, AVRO-1172.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Short version: Looks like the C++ version of AVRO-1099.
> Long version: When a non-decimal double is read from a json file, the parser
> treats it as an long, not a double, and therefore throws an exception.
> Two possible solutions: (1) The decoder should be able to convert longs to
> doubles or acknowledge that a long is a type of double.
> (2) The encoder should always output a double with a decimal point.
> Example code is included below. Output is:
> (1.01, 2.13)
> terminate called after throwing an instance of 'avro::Exception'
> what(): Incorrect token in the stream. Expected: Double, found Integer
> After running complex.json is: {"re":1,"im":2.13
> #include
> #include
> using namespace std;
> #include "cpx.hh"
> #include "avro/Compiler.hh"
> #include "avro/Encoder.hh"
> #include "avro/Decoder.hh"
> avro::ValidSchema load(const char* filename)
> {
> std::ifstream ifs(filename);
> avro::ValidSchema result;
> avro::compileJsonSchema(ifs, result);
> return result;
> }
> void OutTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.01;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void OutTest2()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.0;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void InTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr in =
> avro::fileInputStream("complex.json",1);
> avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
> d->init(*in);
> c::cpx c2;
> avro::decode(*d, c2);
> std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
> }
> int main()
> {
> OutTest();
> InTest();
> OutTest2();
> InTest();
> return 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1172) Avro C++ Json Decoder: Double cannot be decoded
[
https://issues.apache.org/jira/browse/AVRO-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13689411#comment-13689411
]
Thiruvalluvan M. G. commented on AVRO-1172:
---
I'll commit this tomorrow if there are no objections.
> Avro C++ Json Decoder: Double cannot be decoded
> ---
>
> Key: AVRO-1172
> URL: https://issues.apache.org/jira/browse/AVRO-1172
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.1
> Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit
> machine.
>Reporter: Sam Overend
> Labels: patch
> Attachments: AVRO-1172-2.patch, AVRO-1172.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Short version: Looks like the C++ version of AVRO-1099.
> Long version: When a non-decimal double is read from a json file, the parser
> treats it as an long, not a double, and therefore throws an exception.
> Two possible solutions: (1) The decoder should be able to convert longs to
> doubles or acknowledge that a long is a type of double.
> (2) The encoder should always output a double with a decimal point.
> Example code is included below. Output is:
> (1.01, 2.13)
> terminate called after throwing an instance of 'avro::Exception'
> what(): Incorrect token in the stream. Expected: Double, found Integer
> After running complex.json is: {"re":1,"im":2.13
> #include
> #include
> using namespace std;
> #include "cpx.hh"
> #include "avro/Compiler.hh"
> #include "avro/Encoder.hh"
> #include "avro/Decoder.hh"
> avro::ValidSchema load(const char* filename)
> {
> std::ifstream ifs(filename);
> avro::ValidSchema result;
> avro::compileJsonSchema(ifs, result);
> return result;
> }
> void OutTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.01;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void OutTest2()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.0;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void InTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr in =
> avro::fileInputStream("complex.json",1);
> avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
> d->init(*in);
> c::cpx c2;
> avro::decode(*d, c2);
> std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
> }
> int main()
> {
> OutTest();
> InTest();
> OutTest2();
> InTest();
> return 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1346) C++: schema parser cannot parse verbose primitive types
[
https://issues.apache.org/jira/browse/AVRO-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1346:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 1495109. Thank you kye Wanderman-Milne.
> C++: schema parser cannot parse verbose primitive types
> ---
>
> Key: AVRO-1346
> URL: https://issues.apache.org/jira/browse/AVRO-1346
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Skye Wanderman-Milne
>Assignee: Skye Wanderman-Milne
> Attachments: AVRO-1346.2.patch, AVRO-1346.3.patch, AVRO-1346.patch
>
>
> The Avro C++ library's schema parser currently throws an "Unknown additional
> Json fields" exception if a primitive type is not represented as a string
> literal. As per
> http://avro.apache.org/docs/current/spec.html#schema_primitive, primitive
> types can be defined as e.g. "\{type: string\}" or "string". Extra attributes
> are allowed too, e.g. "\{"avro.java.string":"String","type":"string"\}" (from
> the spec: "Attributes not defined in this document are permitted as metadata,
> but must not affect the format of serialized data.").
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1348) Improve Utf8 to String conversion
[
https://issues.apache.org/jira/browse/AVRO-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13690041#comment-13690041
]
Thiruvalluvan M. G. commented on AVRO-1348:
---
The patch seems fine. But it leads to subtle bugs:
- The patch caches the string output in {{toString()}}. Since UTF8 exposes the
underlying byte array through {{getBytes()}}, any change made to the contents
of the array after first invocation of toString() will not be reflected in the
future output of toString(). I don't think there is any simple way to intercept
changes to byte array. One way is to do this - (a) don't cache if someone has
ever called {{getBytes}} in the past (b) invalidate cache if {{getBytes()}} is
called later (c) if Utf8 is constructed using {{Utf8(byte[] bytes)}} do not
cache. Hopefully, in the most common cases, byte array is not exposed and hence
cache would still work. If all these appear too complicated, we can just drop
caching.
- Thread-safety. CharsetDecoder is not thread-safe. If two threads invoke
{{toString()}} simultaneously, the behavior is undefined. Thread-safety need to
be brought in. I'm not sure how expensive is {{Charset.newDocoder()}}. Since we
need to serialize access to {{decode()}}, we can have a single static
CharsetDecoder and get some additional performance.
Apart from these, there are some minor coding-style violations.
> Improve Utf8 to String conversion
> -
>
> Key: AVRO-1348
> URL: https://issues.apache.org/jira/browse/AVRO-1348
> Project: Avro
> Issue Type: Bug
>Reporter: Mark Wagner
>Assignee: Mohammad Kamrul Islam
> Attachments: AVRO1348v1.patch
>
>
> AVRO-1241 found that the existing method of creating Strings from Utf8 byte
> arrays could be made faster. The same method is being used in the
> Utf8.toString(), and could likely be sped up by doing the same thing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1172) Avro C++ Json Decoder: Double cannot be decoded
[
https://issues.apache.org/jira/browse/AVRO-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1172:
--
Resolution: Fixed
Assignee: Sam Overend
Status: Resolved (was: Patch Available)
Committed revision 1495955.
Thank you Sam Overend.
> Avro C++ Json Decoder: Double cannot be decoded
> ---
>
> Key: AVRO-1172
> URL: https://issues.apache.org/jira/browse/AVRO-1172
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.1
> Environment: Built under msys and gcc-4.6.1 on a Windows7/64 bit
> machine.
>Reporter: Sam Overend
>Assignee: Sam Overend
> Labels: patch
> Attachments: AVRO-1172-2.patch, AVRO-1172.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Short version: Looks like the C++ version of AVRO-1099.
> Long version: When a non-decimal double is read from a json file, the parser
> treats it as an long, not a double, and therefore throws an exception.
> Two possible solutions: (1) The decoder should be able to convert longs to
> doubles or acknowledge that a long is a type of double.
> (2) The encoder should always output a double with a decimal point.
> Example code is included below. Output is:
> (1.01, 2.13)
> terminate called after throwing an instance of 'avro::Exception'
> what(): Incorrect token in the stream. Expected: Double, found Integer
> After running complex.json is: {"re":1,"im":2.13
> #include
> #include
> using namespace std;
> #include "cpx.hh"
> #include "avro/Compiler.hh"
> #include "avro/Encoder.hh"
> #include "avro/Decoder.hh"
> avro::ValidSchema load(const char* filename)
> {
> std::ifstream ifs(filename);
> avro::ValidSchema result;
> avro::compileJsonSchema(ifs, result);
> return result;
> }
> void OutTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.01;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void OutTest2()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr out =
> avro::fileOutputStream("complex.json",1);
> avro::EncoderPtr e = avro::jsonEncoder(cpxSchema);
> e->init(*out);
> c::cpx c1;
> c1.re = 1.0;
> c1.im = 2.13;
> avro::encode(*e, c1);
> out->flush();
> }
> void InTest()
> {
> avro::ValidSchema cpxSchema = load("cpx_schema.json");
> std::auto_ptr in =
> avro::fileInputStream("complex.json",1);
> avro::DecoderPtr d = avro::jsonDecoder(cpxSchema);
> d->init(*in);
> c::cpx c2;
> avro::decode(*d, c2);
> std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
> }
> int main()
> {
> OutTest();
> InTest();
> OutTest2();
> InTest();
> return 0;
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-523) records with the same name as a member generate bad c++ code
[
https://issues.apache.org/jira/browse/AVRO-523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-523:
-
Description:
records with the same name as a member generate bad c++ code:
{code}
{
"type" : "array",
"name" : "optionals",
"items" : [
{ "name" : "l", "type" : "record", "fields" : [ { "name" : "l", "type":
"long"} ] },
{ "name" : "r", "type" : "record", "fields" : [ { "name" : "r", "type":
"long"} ] }
]
}
{code}
produces c++ code such that when it is compiled it produces:
union2.h:42: error: field 'int64_t avrouser::l::l' with same name as class
was:
records with the same name as a member generate bad c++ code:
{
"type" : "array",
"name" : "optionals",
"items" : [
{ "name" : "l", "type" : "record", "fields" : [ { "name" : "l", "type":
"long"} ] },
{ "name" : "r", "type" : "record", "fields" : [ { "name" : "r", "type":
"long"} ] }
]
}
produces c++ code such that when it is compiled it produces:
union2.h:42: error: field 'int64_t avrouser::l::l' with same name as class
> records with the same name as a member generate bad c++ code
>
>
> Key: AVRO-523
> URL: https://issues.apache.org/jira/browse/AVRO-523
> Project: Avro
> Issue Type: Bug
> Components: c++
>Reporter: John Plevyak
>
> records with the same name as a member generate bad c++ code:
> {code}
> {
> "type" : "array",
> "name" : "optionals",
> "items" : [
>{ "name" : "l", "type" : "record", "fields" : [ { "name" : "l",
> "type": "long"} ] },
>{ "name" : "r", "type" : "record", "fields" : [ { "name" : "r",
> "type": "long"} ] }
> ]
> }
> {code}
> produces c++ code such that when it is compiled it produces:
> union2.h:42: error: field 'int64_t avrouser::l::l' with same name as class
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. reassigned AVRO-1360:
-
Assignee: Thiruvalluvan M. G.
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: reader.json, writer.json
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1360:
--
Attachment: AVRO-1360.patch
This patch enables AVRO C++ to handle schema resolution.
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360.patch, reader.json, writer.json
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1360:
--
Status: Patch Available (was: Open)
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360.patch, reader.json, writer.json
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751124#comment-13751124
]
Thiruvalluvan M. G. commented on AVRO-1360:
---
I'll commit this soon unless there are objections or suggestions for
improvement.
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360.patch, reader.json, writer.json
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13755000#comment-13755000
]
Thiruvalluvan M. G. commented on AVRO-1360:
---
Ramana,
I suppose you added {{#include }} in {{GenericDatum.hh}}. Am I right?
Thanks
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360.patch, reader.json, writer.json
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1360:
--
Attachment: AVRO-1360-2.patch
Ramana,
Great catch. The trouble was not in the resolver, but in the code generator.
While resolving the user of {{ResolvingDecoder}} must adjust itself for field
ordering. The generated code was not doing that.
The new patch fixes it. Unfortunately, the generated code now becomes even
larger than then the previous one.
Thank you for testing it.
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360-2.patch, AVRO-1360.patch, reader.json,
> testreader, testreader.hh, testwriter, testwriter.hh, writer.json
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13762317#comment-13762317
]
Thiruvalluvan M. G. commented on AVRO-1360:
---
Ramana,
Please have a look at the latest. I think the problem is fixed.
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360.patch,
> testreader, testreader-1, testreader.hh, testwriter, testwriter-1,
> testwriter.hh
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1360:
--
Attachment: AVRO-1360-3.patch
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360.patch,
> testreader, testreader-1, testreader.hh, testwriter, testwriter-1,
> testwriter.hh
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1360:
--
Attachment: AVRO-1360-4.patch
Ramana,
I think the current patch fixes the problems you have been encountering. It
caches most of its work. So a lot of repeated resolutions within a single
schema should be fast. Please let us know whether you still have slowness
issues. Thanks
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360-4.patch,
> AVRO-1360.patch, AVRO-RD.patch, callstack.txt, testreader, testreader-1,
> testreader.hh, testwriter, testwriter-1, testwriter.hh
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1360:
--
Attachment: AVRO-1360-5.patch
This patch fixes a typo and a related bug
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360-4.patch,
> AVRO-1360-5.patch, AVRO-1360.patch, AVRO-RD.patch, callstack.txt, testreader,
> testreader-1, testreader.hh, testwriter, testwriter-1, testwriter.hh
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (AVRO-1406) Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and setters with field name argument
[
https://issues.apache.org/jira/browse/AVRO-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13845040#comment-13845040
]
Thiruvalluvan M. G. commented on AVRO-1406:
---
I think Doug's latest idea of adding a new method to GenericRecord is the
cleanest solution. That actually means four new methods. Three accessors:
* {{const GenericDatum& field(const std::string& name) const;}}
* {{GenericDatum& field(const std::string& name);}}
* {{void setField(const std::string& name, const GenericDatum& v);}}
All of them will use another method:
* {{size_t fieldIndex(const std::string& name);}}
and then use {{fieldAt()}} or {{setFieldAt()}}. I think it makes sense to let
{{fieldIndex()}} also public so that if a client wants to cache field index, it
can. {{fieldIndex()}} should throw an exception if there is no field with the
given name.
> Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and
> setters with field name argument
>
>
> Key: AVRO-1406
> URL: https://issues.apache.org/jira/browse/AVRO-1406
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.5
>Reporter: Iaroslav Zeigerman
> Labels: c++
> Fix For: 1.7.6
>
> Attachments: AVRO-1406.patch, AVRO-1406.patch
>
>
> In Java implementation there is GenericData.Record which can use field names
> to set and get data. There is nothing similar in C++ implementation.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (AVRO-1406) Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and setters with field name argument
[ https://issues.apache.org/jira/browse/AVRO-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13847022#comment-13847022 ] Thiruvalluvan M. G. commented on AVRO-1406: --- The patch looks good to me. +1 for Doug's two comments. > Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and > setters with field name argument > > > Key: AVRO-1406 > URL: https://issues.apache.org/jira/browse/AVRO-1406 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Iaroslav Zeigerman > Labels: c++ > Fix For: 1.7.6 > > Attachments: AVRO-1406.patch, AVRO-1406.patch, AVRO-1406_2.patch > > > In Java implementation there is GenericData.Record which can use field names > to set and get data. There is nothing similar in C++ implementation. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (AVRO-1406) Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and setters with field name argument
[ https://issues.apache.org/jira/browse/AVRO-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13847433#comment-13847433 ] Thiruvalluvan M. G. commented on AVRO-1406: --- +1. Looks great. Thank you Iaroslav Zeigerman. I can commit it after testing on my machine tomorrow if there are no objections from others. Doug, if you want to commit it, please go ahead. > Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and > setters with field name argument > > > Key: AVRO-1406 > URL: https://issues.apache.org/jira/browse/AVRO-1406 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Iaroslav Zeigerman > Labels: c++ > Fix For: 1.7.6 > > Attachments: AVRO-1406.patch, AVRO-1406.patch, AVRO-1406_2.patch, > AVRO-1406_3.patch > > > In Java implementation there is GenericData.Record which can use field names > to set and get data. There is nothing similar in C++ implementation. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (AVRO-1406) Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and setters with field name argument
[ https://issues.apache.org/jira/browse/AVRO-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1406: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed revision 1551029. Thank you Iaroslav Zeigerman > Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and > setters with field name argument > > > Key: AVRO-1406 > URL: https://issues.apache.org/jira/browse/AVRO-1406 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Iaroslav Zeigerman > Labels: c++ > Fix For: 1.7.6 > > Attachments: AVRO-1406.patch, AVRO-1406.patch, AVRO-1406_2.patch, > AVRO-1406_3.patch > > > In Java implementation there is GenericData.Record which can use field names > to set and get data. There is nothing similar in C++ implementation. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Assigned] (AVRO-1406) Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and setters with field name argument
[ https://issues.apache.org/jira/browse/AVRO-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. reassigned AVRO-1406: - Assignee: Iaroslav Zeigerman > Avro C++ GenericRecord (GenericDatum, etc.) doesn't support getters and > setters with field name argument > > > Key: AVRO-1406 > URL: https://issues.apache.org/jira/browse/AVRO-1406 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Iaroslav Zeigerman >Assignee: Iaroslav Zeigerman > Labels: c++ > Fix For: 1.7.6 > > Attachments: AVRO-1406.patch, AVRO-1406.patch, AVRO-1406_2.patch, > AVRO-1406_3.patch > > > In Java implementation there is GenericData.Record which can use field names > to set and get data. There is nothing similar in C++ implementation. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Assigned] (AVRO-1424) ValidatingDecoder hangs on large schema
[ https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. reassigned AVRO-1424: - Assignee: Thiruvalluvan M. G. > ValidatingDecoder hangs on large schema > --- > > Key: AVRO-1424 > URL: https://issues.apache.org/jira/browse/AVRO-1424 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Ramana Suvarapu >Assignee: Thiruvalluvan M. G. > Attachments: model.avsc > > > Try to create Validation decoder using attached schema. It hangs and causes > huge memory allocation. The problem is because of boost::any which is making > excessive copies of objects when creating Symbols. > And also fixup method which is being called in validation decoder creation > stack is also taking long very very long time. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (AVRO-1424) ValidatingDecoder hangs on large schema
[ https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1424: -- Attachment: AVRO-1424.patch > ValidatingDecoder hangs on large schema > --- > > Key: AVRO-1424 > URL: https://issues.apache.org/jira/browse/AVRO-1424 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Ramana Suvarapu >Assignee: Thiruvalluvan M. G. > Attachments: AVRO-1424.patch, model.avsc > > > Try to create Validation decoder using attached schema. It hangs and causes > huge memory allocation. The problem is because of boost::any which is making > excessive copies of objects when creating Symbols. > And also fixup method which is being called in validation decoder creation > stack is also taking long very very long time. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (AVRO-1424) ValidatingDecoder hangs on large schema
[ https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13860469#comment-13860469 ] Thiruvalluvan M. G. commented on AVRO-1424: --- The attached patch addresses the problem. > ValidatingDecoder hangs on large schema > --- > > Key: AVRO-1424 > URL: https://issues.apache.org/jira/browse/AVRO-1424 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Ramana Suvarapu >Assignee: Thiruvalluvan M. G. > Attachments: AVRO-1424.patch, model.avsc > > > Try to create Validation decoder using attached schema. It hangs and causes > huge memory allocation. The problem is because of boost::any which is making > excessive copies of objects when creating Symbols. > And also fixup method which is being called in validation decoder creation > stack is also taking long very very long time. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (AVRO-1424) ValidatingDecoder hangs on large schema
[ https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1424: -- Status: Patch Available (was: Open) > ValidatingDecoder hangs on large schema > --- > > Key: AVRO-1424 > URL: https://issues.apache.org/jira/browse/AVRO-1424 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Ramana Suvarapu >Assignee: Thiruvalluvan M. G. > Attachments: model.avsc > > > Try to create Validation decoder using attached schema. It hangs and causes > huge memory allocation. The problem is because of boost::any which is making > excessive copies of objects when creating Symbols. > And also fixup method which is being called in validation decoder creation > stack is also taking long very very long time. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (AVRO-1424) ValidatingDecoder hangs on large schema
[ https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13862115#comment-13862115 ] Thiruvalluvan M. G. commented on AVRO-1424: --- The patch is against the latest trunk. I tested again against a fresh checkout and it applies cleanly. > ValidatingDecoder hangs on large schema > --- > > Key: AVRO-1424 > URL: https://issues.apache.org/jira/browse/AVRO-1424 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Ramana Suvarapu >Assignee: Thiruvalluvan M. G. > Attachments: AVRO-1424.patch, model.avsc > > > Try to create Validation decoder using attached schema. It hangs and causes > huge memory allocation. The problem is because of boost::any which is making > excessive copies of objects when creating Symbols. > And also fixup method which is being called in validation decoder creation > stack is also taking long very very long time. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (AVRO-1424) ValidatingDecoder hangs on large schema
[ https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13863858#comment-13863858 ] Thiruvalluvan M. G. commented on AVRO-1424: --- Yes Doug. I'm waiting for Ramana to confirm that this works for his real use cases. Of course, I've tested it with the sample schema he gave I just want to make sure that his other use cases are taken care, too. > ValidatingDecoder hangs on large schema > --- > > Key: AVRO-1424 > URL: https://issues.apache.org/jira/browse/AVRO-1424 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Ramana Suvarapu >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.6 > > Attachments: AVRO-1424.patch, model.avsc > > > Try to create Validation decoder using attached schema. It hangs and causes > huge memory allocation. The problem is because of boost::any which is making > excessive copies of objects when creating Symbols. > And also fixup method which is being called in validation decoder creation > stack is also taking long very very long time. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (AVRO-1424) ValidatingDecoder hangs on large schema
[ https://issues.apache.org/jira/browse/AVRO-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1424: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed revision 1556285. Thank you Ramana. > ValidatingDecoder hangs on large schema > --- > > Key: AVRO-1424 > URL: https://issues.apache.org/jira/browse/AVRO-1424 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.5 >Reporter: Ramana Suvarapu >Assignee: Thiruvalluvan M. G. > Fix For: 1.7.6 > > Attachments: AVRO-1424.patch, model.avsc > > > Try to create Validation decoder using attached schema. It hangs and causes > huge memory allocation. The problem is because of boost::any which is making > excessive copies of objects when creating Symbols. > And also fixup method which is being called in validation decoder creation > stack is also taking long very very long time. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (AVRO-1415) C++ binary encoder and decoder doesn't handle uninitialzed enums
[ https://issues.apache.org/jira/browse/AVRO-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1415: -- Attachment: AVRO-1415-2.patch Also added default (i.e. no-args) constructor for generated classes for records so that the enums are initialized properly even when the programmer fails to set the value explicitly. > C++ binary encoder and decoder doesn't handle uninitialzed enums > > > Key: AVRO-1415 > URL: https://issues.apache.org/jira/browse/AVRO-1415 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.4 >Reporter: Ramana Suvarapu > Attachments: AVRO-1415-2.patch, AVRO-1415.patch > > > When enums are not properly initialized and when they get encoded / decoded, > C++ enum encoding and decoding traits don't check for uninitialed enums and > it encodes the wrong values. When Java or C# tries to decode them, they throw > out of boundary exceptions. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (AVRO-1415) C++ binary encoder and decoder doesn't handle uninitialzed enums
[
https://issues.apache.org/jira/browse/AVRO-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13866287#comment-13866287
]
Thiruvalluvan M. G. commented on AVRO-1415:
---
+1 except for a missing {{include}} directive in the generated code. Attached
is the modified patch which inserted the directive. Also, generated the patch
from the top level folder, instead of from {{lang/c\+\+/impl}}.
I'll commit this tomorrow, if there are no objections.
> C++ binary encoder and decoder doesn't handle uninitialzed enums
>
>
> Key: AVRO-1415
> URL: https://issues.apache.org/jira/browse/AVRO-1415
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
> Attachments: AVRO-1415-2.patch, AVRO-1415.patch
>
>
> When enums are not properly initialized and when they get encoded / decoded,
> C++ enum encoding and decoding traits don't check for uninitialed enums and
> it encodes the wrong values. When Java or C# tries to decode them, they throw
> out of boundary exceptions.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Resolved] (AVRO-1415) C++ binary encoder and decoder doesn't handle uninitialzed enums
[ https://issues.apache.org/jira/browse/AVRO-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. resolved AVRO-1415. --- Resolution: Fixed Assignee: Ramana Suvarapu Committed revision 1557041. Thank you Ramana > C++ binary encoder and decoder doesn't handle uninitialzed enums > > > Key: AVRO-1415 > URL: https://issues.apache.org/jira/browse/AVRO-1415 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.4 >Reporter: Ramana Suvarapu >Assignee: Ramana Suvarapu > Attachments: AVRO-1415-2.patch, AVRO-1415.patch > > > When enums are not properly initialized and when they get encoded / decoded, > C++ enum encoding and decoding traits don't check for uninitialed enums and > it encodes the wrong values. When Java or C# tries to decode them, they throw > out of boundary exceptions. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (AVRO-1360) C++ Resolving decoder is not working when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. resolved AVRO-1360.
---
Resolution: Duplicate
The patches for this JIRA have become stale because the codebase have moved.
AVRO-1474 tracks this issue anew.
> C++ Resolving decoder is not working when reader schema has more fields than
> writer schema
> --
>
> Key: AVRO-1360
> URL: https://issues.apache.org/jira/browse/AVRO-1360
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.4
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1360-2.patch, AVRO-1360-3.patch, AVRO-1360-4.patch,
> AVRO-1360-5.patch, AVRO-1360.patch, AVRO-RD.patch, callstack.txt, model.avsc,
> testreader, testreader-1, testreader.hh, testwriter, testwriter-1,
> testwriter.hh
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
>
> Can you please let us know if there are any other limitations with c++
> implementation of ResolvingDecoder? We are planning to use it in our project
> and we want to make sure it works as per avro specification.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1474:
--
Attachment: AVRO-1474.patch
This patch introduces support for default fields in reader schema.
> C++ resolvind decoder doesn't work when reader schema has more fields than
> writer schema
>
>
> Key: AVRO-1474
> URL: https://issues.apache.org/jira/browse/AVRO-1474
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.6
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1474.patch, reader, writer
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
> Also field ordering is not working.
> The same issue is reported in AVRO-1360.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1474:
--
Status: Patch Available (was: Open)
> C++ resolvind decoder doesn't work when reader schema has more fields than
> writer schema
>
>
> Key: AVRO-1474
> URL: https://issues.apache.org/jira/browse/AVRO-1474
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.6
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1474.patch, reader, writer
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
> Also field ordering is not working.
> The same issue is reported in AVRO-1360.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974170#comment-13974170
]
Thiruvalluvan M. G. commented on AVRO-1474:
---
Ramana,
Can we commit this?
> C++ resolvind decoder doesn't work when reader schema has more fields than
> writer schema
>
>
> Key: AVRO-1474
> URL: https://issues.apache.org/jira/browse/AVRO-1474
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.6
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1474.patch, reader, writer
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
> Also field ordering is not working.
> The same issue is reported in AVRO-1360.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (AVRO-1352) Schema for fixed types corrupted when writing out in JSON format
[
https://issues.apache.org/jira/browse/AVRO-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045414#comment-14045414
]
Thiruvalluvan M. G. commented on AVRO-1352:
---
+1
Looks good to me. I'll have it committed this weekend, after testing it on
Windows and Mac.
> Schema for fixed types corrupted when writing out in JSON format
>
>
> Key: AVRO-1352
> URL: https://issues.apache.org/jira/browse/AVRO-1352
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.4
>Reporter: Steve Roehrs
>Priority: Minor
> Labels: patch,
> Fix For: 1.7.7
>
> Attachments: AVRO-1352-Test.patch, AVRO-1352.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A change made in NodeImpl.cc under AVRO-1026 has introduced a bug that
> corrupts JSON schemas when writing them to disk or dumping to a stream. This
> manifested when using DataFileWriter to generate a data file, and the schema
> stored in the header was corrupted from the original schema provided.
>
> e.g
>
> This:
> {noformat}
> {
> “type”: “fixed”,
> “name”: “Unsigned16”,
> “size”: 2
> }
> {noformat}
>
> Becomes this:
> {noformat}
> {
> “type”: “fixed”,
> “size”: 2,
> “name” : “Unsigned16”,
> }
> {noformat}
> Note the extraneous comma after the ‘name’ attribute.
> The bug exists for Avro 'fixed' types, and has not been observed for other
> types. A test case will be developed.
> The bug can be fixed by simply re-ordering the code in NodeFixed::printJson.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (AVRO-1474) C++ resolvind decoder doesn't work when reader schema has more fields than writer schema
[
https://issues.apache.org/jira/browse/AVRO-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1474:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 1606545. Thank you Ramana.
> C++ resolvind decoder doesn't work when reader schema has more fields than
> writer schema
>
>
> Key: AVRO-1474
> URL: https://issues.apache.org/jira/browse/AVRO-1474
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.6
>Reporter: Ramana Suvarapu
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1474-ENUM.patch, AVRO-1474-MAP.patch,
> AVRO-1474-REUSE-RD.patch, AVRO-1474-ResolvingDecoder.patch, AVRO-1474.patch,
> bigrecord, bigrecord_r, reader, writer
>
>
> When reader schema has more number of fields than writer schema, C++
> implementation of resolving decoder is throwing exception "throwing exception
> "Don't know how to handle excess fields for reader.” with out checking
> whether fields are optional or fields have default values.
> Attached are reader and writer schemas. Record in reader schema has 2
> additional fields than writer schema. One field is required field but it has
> default value and another one is optional field (union of null and string).
> Since one has default value and another is optional both reader and writer
> schemas are supposed to be compatible.
>
> {"name": "defaultField", "type": "string", "default": "DEFAULT",
> "declared":"true"},
> {"name": "optionalField", "type": ["string", "null"],"declared":"true"},
>
> main()
> {
> avro::ValidSchema readerSchema = load("reader.json");
> avro::ValidSchema writerSchema = load("writer.json");
> avro::DecoderPtr d = avro::resolvingDecoder(writerSchema,
> readerSchema,avro::binaryDecoder());
> }
>
> But when I tried to create resolving decoder, I am getting "Don't know how to
> handle excess fields for reader.” But Java implementation works.
> Also field ordering is not working.
> The same issue is reported in AVRO-1360.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (AVRO-1352) Schema for fixed types corrupted when writing out in JSON format
[
https://issues.apache.org/jira/browse/AVRO-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1352:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 1606560.
> Schema for fixed types corrupted when writing out in JSON format
>
>
> Key: AVRO-1352
> URL: https://issues.apache.org/jira/browse/AVRO-1352
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.4
>Reporter: Steve Roehrs
>Priority: Minor
> Labels: patch,
> Fix For: 1.7.7
>
> Attachments: AVRO-1352-Test.patch, AVRO-1352.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A change made in NodeImpl.cc under AVRO-1026 has introduced a bug that
> corrupts JSON schemas when writing them to disk or dumping to a stream. This
> manifested when using DataFileWriter to generate a data file, and the schema
> stored in the header was corrupted from the original schema provided.
>
> e.g
>
> This:
> {noformat}
> {
> “type”: “fixed”,
> “name”: “Unsigned16”,
> “size”: 2
> }
> {noformat}
>
> Becomes this:
> {noformat}
> {
> “type”: “fixed”,
> “size”: 2,
> “name” : “Unsigned16”,
> }
> {noformat}
> Note the extraneous comma after the ‘name’ attribute.
> The bug exists for Avro 'fixed' types, and has not been observed for other
> types. A test case will be developed.
> The bug can be fixed by simply re-ordering the code in NodeFixed::printJson.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (AVRO-1540) C++ doesn't build in Ubuntu
Thiruvalluvan M. G. created AVRO-1540:
-
Summary: C++ doesn't build in Ubuntu
Key: AVRO-1540
URL: https://issues.apache.org/jira/browse/AVRO-1540
Project: Avro
Issue Type: Bug
Reporter: Thiruvalluvan M. G.
Assignee: Thiruvalluvan M. G.
Fix For: 1.7.7
It is because {{uint8_t}} type is not available in GenericDatum.hh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (AVRO-1540) C++ doesn't build in Ubuntu
[
https://issues.apache.org/jira/browse/AVRO-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1540:
--
Status: Patch Available (was: Open)
> C++ doesn't build in Ubuntu
> ---
>
> Key: AVRO-1540
> URL: https://issues.apache.org/jira/browse/AVRO-1540
> Project: Avro
> Issue Type: Bug
>Reporter: Thiruvalluvan M. G.
>Assignee: Thiruvalluvan M. G.
> Fix For: 1.7.7
>
> Attachments: AVRO-1540.patch
>
>
> It is because {{uint8_t}} type is not available in GenericDatum.hh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (AVRO-1540) C++ doesn't build in Ubuntu
[
https://issues.apache.org/jira/browse/AVRO-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1540:
--
Attachment: AVRO-1540.patch
> C++ doesn't build in Ubuntu
> ---
>
> Key: AVRO-1540
> URL: https://issues.apache.org/jira/browse/AVRO-1540
> Project: Avro
> Issue Type: Bug
>Reporter: Thiruvalluvan M. G.
>Assignee: Thiruvalluvan M. G.
> Fix For: 1.7.7
>
> Attachments: AVRO-1540.patch
>
>
> It is because {{uint8_t}} type is not available in GenericDatum.hh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (AVRO-1540) C++ doesn't build in Ubuntu
[
https://issues.apache.org/jira/browse/AVRO-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058571#comment-14058571
]
Thiruvalluvan M. G. commented on AVRO-1540:
---
Tested the patch on Ubuntu 13.04 and Macos 10.9.4.
> C++ doesn't build in Ubuntu
> ---
>
> Key: AVRO-1540
> URL: https://issues.apache.org/jira/browse/AVRO-1540
> Project: Avro
> Issue Type: Bug
>Reporter: Thiruvalluvan M. G.
>Assignee: Thiruvalluvan M. G.
> Fix For: 1.7.7
>
> Attachments: AVRO-1540.patch
>
>
> It is because {{uint8_t}} type is not available in GenericDatum.hh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (AVRO-1540) C++ doesn't build in Ubuntu
[
https://issues.apache.org/jira/browse/AVRO-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1540:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 1610798.
Thank you Doug.
> C++ doesn't build in Ubuntu
> ---
>
> Key: AVRO-1540
> URL: https://issues.apache.org/jira/browse/AVRO-1540
> Project: Avro
> Issue Type: Bug
> Components: c++
>Reporter: Thiruvalluvan M. G.
>Assignee: Thiruvalluvan M. G.
>Priority: Blocker
> Fix For: 1.7.7
>
> Attachments: AVRO-1540.patch
>
>
> It is because {{uint8_t}} type is not available in GenericDatum.hh
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (AVRO-1448) Python3 setup.py is broken
[ https://issues.apache.org/jira/browse/AVRO-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063073#comment-14063073 ] Thiruvalluvan M. G. commented on AVRO-1448: --- The change does not affect functionality and thus is fine. +1. But the technique of comparing two JSONs is fragile because JSON doesn't have unique representation. It is especially true with respect to the order of fields in a map. White space is another example, as in this JIRA. This needs to be addressed at some time. > Python3 setup.py is broken > -- > > Key: AVRO-1448 > URL: https://issues.apache.org/jira/browse/AVRO-1448 > Project: Avro > Issue Type: Bug > Components: python >Affects Versions: 1.7.6 >Reporter: Christophe Taton >Assignee: Christophe Taton > Fix For: 1.7.7 > > Attachments: AVRO-1448-fix.patch, AVRO-1448.20140127-232000-0800.diff > > > source distribution on pypi cannot be installed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (AVRO-1448) Python3 setup.py is broken
[ https://issues.apache.org/jira/browse/AVRO-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063087#comment-14063087 ] Thiruvalluvan M. G. commented on AVRO-1448: --- +1 Great! Thanks. > Python3 setup.py is broken > -- > > Key: AVRO-1448 > URL: https://issues.apache.org/jira/browse/AVRO-1448 > Project: Avro > Issue Type: Bug > Components: python >Affects Versions: 1.7.6 >Reporter: Christophe Taton >Assignee: Christophe Taton > Fix For: 1.7.7 > > Attachments: AVRO-1448-fix.patch, > AVRO-1448.20140127-232000-0800.diff, AVRO-1448.diff > > > source distribution on pypi cannot be installed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (AVRO-1593) C++ json encoder assumes "C" locale and generates invalid UTF-8 sequence
[
https://issues.apache.org/jira/browse/AVRO-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14254446#comment-14254446
]
Thiruvalluvan M. G. commented on AVRO-1593:
---
Looks simple enough. But constructing a locale object for every string is very
expensive. On my machine, a quick microbenchmark shows that locale construction
takes 200 times more times than doing iscntrl() on a 20 character string.
{noformat}
cat: l: No such file or directory
$ cat l.cpp
#include
#include
int main()
{
for (size_t i = 0; i < 10; ++i) {
std::locale cl("C");
for (size_t j = 0; j < 20; ++j) {
std::iscntrl(j % 127 + 1);
}
}
}
$ make l
c++ l.cpp -o l
$ time ./l
real0m6.272s
user0m6.244s
sys 0m0.015s
{noformat}
{noformat}
$ cat l.cpp
#include
#include
int main()
{
std::locale cl("C");
for (size_t i = 0; i < 10; ++i) {
for (size_t j = 0; j < 20; ++j) {
std::iscntrl(j % 127 + 1);
}
}
}
$ make l
c++ l.cpp -o l
$ time ./l
real0m0.033s
user0m0.027s
sys 0m0.003s
{noformat}
The current implementation will have considerable impact on performance. But if
we move the local as a (non-static or even better static) member of the class,
will have insignificant impact on performance.
> C++ json encoder assumes "C" locale and generates invalid UTF-8 sequence
> -
>
> Key: AVRO-1593
> URL: https://issues.apache.org/jira/browse/AVRO-1593
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.7
> Environment: windows-1252 encoding
>Reporter: Hatem Helal
>Priority: Critical
> Fix For: 1.7.8
>
>
> encoding a multibyte UTF-8 code point such as:
> "\xEF\xBD\x81"
> Incorrectly becomes:
> "\xEF\xBD\U0081"
> When encoded in the service running in the windows-1252 locale. This isn¹t a
> valid UTF-8 sequence so we end up with Mojibake when reading back the JSON
> encoded string.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (AVRO-1593) C++ json encoder assumes "C" locale and generates invalid UTF-8 sequence
[
https://issues.apache.org/jira/browse/AVRO-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14254446#comment-14254446
]
Thiruvalluvan M. G. edited comment on AVRO-1593 at 12/20/14 1:53 AM:
-
Looks simple enough. But constructing a locale object for every string is very
expensive. On my machine, a quick microbenchmark shows that locale construction
takes 200 times more times than doing iscntrl() on a 20 character string.
{noformat}
$ cat l.cpp
#include
#include
int main()
{
for (size_t i = 0; i < 10; ++i) {
std::locale cl("C");
for (size_t j = 0; j < 20; ++j) {
std::iscntrl(j % 127 + 1);
}
}
}
$ make l
c++ l.cpp -o l
$ time ./l
real0m6.272s
user0m6.244s
sys 0m0.015s
{noformat}
{noformat}
$ cat l.cpp
#include
#include
int main()
{
std::locale cl("C");
for (size_t i = 0; i < 10; ++i) {
for (size_t j = 0; j < 20; ++j) {
std::iscntrl(j % 127 + 1);
}
}
}
$ make l
c++ l.cpp -o l
$ time ./l
real0m0.033s
user0m0.027s
sys 0m0.003s
{noformat}
The current implementation will have considerable impact on performance. But if
we move the local as a (non-static or even better static) member of the class,
will have insignificant impact on performance.
was (Author: thiru_mg):
Looks simple enough. But constructing a locale object for every string is very
expensive. On my machine, a quick microbenchmark shows that locale construction
takes 200 times more times than doing iscntrl() on a 20 character string.
{noformat}
cat: l: No such file or directory
$ cat l.cpp
#include
#include
int main()
{
for (size_t i = 0; i < 10; ++i) {
std::locale cl("C");
for (size_t j = 0; j < 20; ++j) {
std::iscntrl(j % 127 + 1);
}
}
}
$ make l
c++ l.cpp -o l
$ time ./l
real0m6.272s
user0m6.244s
sys 0m0.015s
{noformat}
{noformat}
$ cat l.cpp
#include
#include
int main()
{
std::locale cl("C");
for (size_t i = 0; i < 10; ++i) {
for (size_t j = 0; j < 20; ++j) {
std::iscntrl(j % 127 + 1);
}
}
}
$ make l
c++ l.cpp -o l
$ time ./l
real0m0.033s
user0m0.027s
sys 0m0.003s
{noformat}
The current implementation will have considerable impact on performance. But if
we move the local as a (non-static or even better static) member of the class,
will have insignificant impact on performance.
> C++ json encoder assumes "C" locale and generates invalid UTF-8 sequence
> -
>
> Key: AVRO-1593
> URL: https://issues.apache.org/jira/browse/AVRO-1593
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.7
> Environment: windows-1252 encoding
>Reporter: Hatem Helal
>Priority: Critical
> Fix For: 1.7.8
>
>
> encoding a multibyte UTF-8 code point such as:
> "\xEF\xBD\x81"
> Incorrectly becomes:
> "\xEF\xBD\U0081"
> When encoded in the service running in the windows-1252 locale. This isn¹t a
> valid UTF-8 sequence so we end up with Mojibake when reading back the JSON
> encoded string.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1719) Avro fails to build against Boost 1.59.0
[ https://issues.apache.org/jira/browse/AVRO-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725448#comment-14725448 ] Thiruvalluvan M. G. commented on AVRO-1719: --- +1 Looks good to me. > Avro fails to build against Boost 1.59.0 > > > Key: AVRO-1719 > URL: https://issues.apache.org/jira/browse/AVRO-1719 > Project: Avro > Issue Type: Bug > Components: build >Affects Versions: 1.7.7 >Reporter: Tim Smith > > Avro fails to build on OS X with Boost 1.59.0, dying on errors about > undeclared BOOST_ identifiers. > Build logs are here: > https://gist.github.com/anonymous/03736608223d42f45ab1#file-02-make-L180 > Homebrew is tracking packages which fail to build against Boost 1.59.0 at > https://github.com/Homebrew/homebrew/pull/42960. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AVRO-1754) C++ ValiditingDecoder handles null incorrectly
[
https://issues.apache.org/jira/browse/AVRO-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027509#comment-15027509
]
Thiruvalluvan M. G. commented on AVRO-1754:
---
+1
Looks good to me.
I'll commit it tomorrow if there are no objections.
> C++ ValiditingDecoder handles null incorrectly
> --
>
> Key: AVRO-1754
> URL: https://issues.apache.org/jira/browse/AVRO-1754
> Project: Avro
> Issue Type: Bug
> Components: c++
>Reporter: John McClean
> Attachments: AVRO-1754.patch
>
>
> When decoding null, the ValidatingDecoder does not call the underlying
> 'decodeNull' method. The result is that the next field decoded causes an
> exception.
> For example, if this json
> {quote}
> \{"a":null,"b":"bar"}
> {quote}
> is decoded with this schema
> {quote}
> \{
> "name": "foo",
> "type": "record",
> "fields": [
> \{ "name": "a", "type": ["null", "string"] },
> \{ "name": "b", "type": "string" }
> ]
> }
> {quote}
> it throws an exception with the message "Invalid operation. Expected: String
> got Null". This happens when 'b' is being decoded. I'll attach a patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1636) C++ JsonDecoder expects json object to be ordered
[
https://issues.apache.org/jira/browse/AVRO-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027733#comment-15027733
]
Thiruvalluvan M. G. commented on AVRO-1636:
---
Avro C++ streams in JSON objects. That is it reads the next token only when the
client is ready to consume it. The consequence is that the order of fields in
the JSON object should exactly match than in the schema. An alternative
approach could have been to read the entire JSON object into memory and then
interpret it. The flip side of this approach is that if the object is large, it
will take up a lot of memory.
We made a trade off in favor of conserving memory. Avro decoder will be able to
read JSON streams written using Avro encoder, but if the order of fields is
altered afterwards, it may fail.
The limitation is common at least to C++ and Java implementations. As mentioned
in the [Avro Specification|https://avro.apache.org/docs/1.7.7/spec.html], the
JSON encoding exists as an aid to developers to help debug issues and not meant
to be used in production.
> C++ JsonDecoder expects json object to be ordered
> -
>
> Key: AVRO-1636
> URL: https://issues.apache.org/jira/browse/AVRO-1636
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.7
>Reporter: Mann Du
>
> I am using Shafquat Rahman's original post for this problem reported in Avro
> user mailing list in last May for the description - ( Thiru provided a fix
> for the exact problem for Java in Oct. 2011 with Avro-895.)
> I have been experimenting with avro in C++ (version 1.7.5) and ran into an
> issue with the json decoder which expects ordered json objects. The problem I
> am seeing appears similar to this post I found for an older avro java library:
> http://search-hadoop.com/m/7WG37aVaBd/v=plain
> I have a simple record:
> {
> "name" : "SimpleRecord",
> "type" : "record",
> "fields" :[
> { "name" : "A", "type" : "int"},
> { "name" : "B", "type" : "int"}
> ]
> }
> I generate the C++ header using avrogencpp. The generated code has
> codec_traits specialization for SimpleRecord that fixes the order for the
> JsonEncoder and JsonDecoder.
> ...snip...
> namespace avro {
> template<> struct codec_traits {
> static void encode(Encoder& e, const SimpleRecord& v) {
> avro::encode(e, v.A);
> avro::encode(e, v.B);
> }
> static void decode(Decoder& d, SimpleRecord& v) {
> avro::decode(d, v.A);
> avro::decode(d, v.B);
> }
> };
> ...snip...
> The JsonDecoder successfully decodes json objects of the form{"A" : 1, "B" :
> 2} into SimpleRecord. But if I try to decode {"B" : 2, "A" : 1} it throws
> 'avro::Exception' with "Incorrect field" from impl/parsing/JsonCodec.cc:182
> in the following method:
> JsonDecoderHandler(JsonParser& p) : in_(p) { }
> size_t handle(const Symbol& s) {
> switch (s.kind()) {
> case Symbol::sRecordStart:
> expectToken(in_, JsonParser::tkObjectStart);
> break;
> case Symbol::sRecordEnd:
> expectToken(in_, JsonParser::tkObjectEnd);
> break;
> case Symbol::sField:
> expectToken(in_, JsonParser::tkString);
> if (s.extra() != in_.stringValue()) {
> throw Exception("Incorrect field");
> }
> break;
> default:
> break;
> }
> return 0;
> }
> The stack shows that avro::decode(d, v.A) is the call the eventually causes
> the exception.
> According to the json spec the fields in a json object are unordered. ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AVRO-1750) GenericDatum API behavior breaking change
[
https://issues.apache.org/jira/browse/AVRO-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1750:
--
Assignee: John McClean
> GenericDatum API behavior breaking change
> -
>
> Key: AVRO-1750
> URL: https://issues.apache.org/jira/browse/AVRO-1750
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.7
>Reporter: Braden McDaniel
>Assignee: John McClean
> Fix For: 1.8.0
>
>
> It appears that a change was introduced to the {{avro::GenericDatum}}
> implementation between 1.7.6 and 1.7.7 that causes unions to be handled
> differently.
> The 1.7.6 implementation does this:
> {noformat}
> inline Type AVRO_DECL GenericDatum::type() const {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->type() : type_;
> }
> template
> const T& GenericDatum::value() const {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->value() :
> *boost::any_cast(&value_);
> }
> template
> T& GenericDatum::value() {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->value() :
> *boost::any_cast(&value_);
> }
> {noformat}
> …whereas the 1.7.7 implementation does this:
> {noformat}
> /**
> * The avro data type this datum holds.
> */
> Type type() const {
> return type_;
> }
> /**
> * Returns the value held by this datum.
> * T The type for the value. This must correspond to the
> * avro type returned by type().
> */
> template const T& value() const {
> return *boost::any_cast(&value_);
> }
> /**
> * Returns the reference to the value held by this datum, which
> * can be used to change the contents. Please note that only
> * value can be changed, the data type of the value held cannot
> * be changed.
> *
> * T The type for the value. This must correspond to the
> * avro type returned by type().
> */
> template T& value() {
> return *boost::any_cast(&value_);
> }
> {noformat}
> The result of this is that, if the underlying value is an {{AVRO_UNION}},
> calls to {{GenericDatum::type}} and {{GenericDatum::value<>}} that previously
> resolved to the union member type no longer do so (and user code relying on
> that behavior has been broken).
> This change apparently was made as part of the changes for AVRO-1474;
> however, looking at the comments in that issue, it's not clear to me why it
> was required for that fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (AVRO-1750) GenericDatum API behavior breaking change
[
https://issues.apache.org/jira/browse/AVRO-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. resolved AVRO-1750.
---
Resolution: Fixed
Fix Version/s: 1.8.0
> GenericDatum API behavior breaking change
> -
>
> Key: AVRO-1750
> URL: https://issues.apache.org/jira/browse/AVRO-1750
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.7
>Reporter: Braden McDaniel
>Assignee: John McClean
> Fix For: 1.8.0
>
>
> It appears that a change was introduced to the {{avro::GenericDatum}}
> implementation between 1.7.6 and 1.7.7 that causes unions to be handled
> differently.
> The 1.7.6 implementation does this:
> {noformat}
> inline Type AVRO_DECL GenericDatum::type() const {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->type() : type_;
> }
> template
> const T& GenericDatum::value() const {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->value() :
> *boost::any_cast(&value_);
> }
> template
> T& GenericDatum::value() {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->value() :
> *boost::any_cast(&value_);
> }
> {noformat}
> …whereas the 1.7.7 implementation does this:
> {noformat}
> /**
> * The avro data type this datum holds.
> */
> Type type() const {
> return type_;
> }
> /**
> * Returns the value held by this datum.
> * T The type for the value. This must correspond to the
> * avro type returned by type().
> */
> template const T& value() const {
> return *boost::any_cast(&value_);
> }
> /**
> * Returns the reference to the value held by this datum, which
> * can be used to change the contents. Please note that only
> * value can be changed, the data type of the value held cannot
> * be changed.
> *
> * T The type for the value. This must correspond to the
> * avro type returned by type().
> */
> template T& value() {
> return *boost::any_cast(&value_);
> }
> {noformat}
> The result of this is that, if the underlying value is an {{AVRO_UNION}},
> calls to {{GenericDatum::type}} and {{GenericDatum::value<>}} that previously
> resolved to the union member type no longer do so (and user code relying on
> that behavior has been broken).
> This change apparently was made as part of the changes for AVRO-1474;
> however, looking at the comments in that issue, it's not clear to me why it
> was required for that fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AVRO-1754) C++ ValiditingDecoder handles null incorrectly
[
https://issues.apache.org/jira/browse/AVRO-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1754:
--
Assignee: John McClean
> C++ ValiditingDecoder handles null incorrectly
> --
>
> Key: AVRO-1754
> URL: https://issues.apache.org/jira/browse/AVRO-1754
> Project: Avro
> Issue Type: Bug
> Components: c++
>Reporter: John McClean
>Assignee: John McClean
> Fix For: 1.8.0
>
> Attachments: AVRO-1754.patch
>
>
> When decoding null, the ValidatingDecoder does not call the underlying
> 'decodeNull' method. The result is that the next field decoded causes an
> exception.
> For example, if this json
> {quote}
> \{"a":null,"b":"bar"}
> {quote}
> is decoded with this schema
> {quote}
> \{
> "name": "foo",
> "type": "record",
> "fields": [
> \{ "name": "a", "type": ["null", "string"] },
> \{ "name": "b", "type": "string" }
> ]
> }
> {quote}
> it throws an exception with the message "Invalid operation. Expected: String
> got Null". This happens when 'b' is being decoded. I'll attach a patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AVRO-1754) C++ ValiditingDecoder handles null incorrectly
[
https://issues.apache.org/jira/browse/AVRO-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1754:
--
Resolution: Fixed
Fix Version/s: 1.8.0
Status: Resolved (was: Patch Available)
> C++ ValiditingDecoder handles null incorrectly
> --
>
> Key: AVRO-1754
> URL: https://issues.apache.org/jira/browse/AVRO-1754
> Project: Avro
> Issue Type: Bug
> Components: c++
>Reporter: John McClean
>Assignee: John McClean
> Fix For: 1.8.0
>
> Attachments: AVRO-1754.patch
>
>
> When decoding null, the ValidatingDecoder does not call the underlying
> 'decodeNull' method. The result is that the next field decoded causes an
> exception.
> For example, if this json
> {quote}
> \{"a":null,"b":"bar"}
> {quote}
> is decoded with this schema
> {quote}
> \{
> "name": "foo",
> "type": "record",
> "fields": [
> \{ "name": "a", "type": ["null", "string"] },
> \{ "name": "b", "type": "string" }
> ]
> }
> {quote}
> it throws an exception with the message "Invalid operation. Expected: String
> got Null". This happens when 'b' is being decoded. I'll attach a patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (AVRO-1750) GenericDatum API behavior breaking change
[
https://issues.apache.org/jira/browse/AVRO-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. reopened AVRO-1750:
---
Assignee: (was: John McClean)
> GenericDatum API behavior breaking change
> -
>
> Key: AVRO-1750
> URL: https://issues.apache.org/jira/browse/AVRO-1750
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.7
>Reporter: Braden McDaniel
> Fix For: 1.8.0
>
>
> It appears that a change was introduced to the {{avro::GenericDatum}}
> implementation between 1.7.6 and 1.7.7 that causes unions to be handled
> differently.
> The 1.7.6 implementation does this:
> {noformat}
> inline Type AVRO_DECL GenericDatum::type() const {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->type() : type_;
> }
> template
> const T& GenericDatum::value() const {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->value() :
> *boost::any_cast(&value_);
> }
> template
> T& GenericDatum::value() {
> return (type_ == AVRO_UNION) ?
> boost::any_cast(&value_)->value() :
> *boost::any_cast(&value_);
> }
> {noformat}
> …whereas the 1.7.7 implementation does this:
> {noformat}
> /**
> * The avro data type this datum holds.
> */
> Type type() const {
> return type_;
> }
> /**
> * Returns the value held by this datum.
> * T The type for the value. This must correspond to the
> * avro type returned by type().
> */
> template const T& value() const {
> return *boost::any_cast(&value_);
> }
> /**
> * Returns the reference to the value held by this datum, which
> * can be used to change the contents. Please note that only
> * value can be changed, the data type of the value held cannot
> * be changed.
> *
> * T The type for the value. This must correspond to the
> * avro type returned by type().
> */
> template T& value() {
> return *boost::any_cast(&value_);
> }
> {noformat}
> The result of this is that, if the underlying value is an {{AVRO_UNION}},
> calls to {{GenericDatum::type}} and {{GenericDatum::value<>}} that previously
> resolved to the union member type no longer do so (and user code relying on
> that behavior has been broken).
> This change apparently was made as part of the changes for AVRO-1474;
> however, looking at the comments in that issue, it's not clear to me why it
> was required for that fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1784) Should have a C++ json encoder that pretty prints.
[ https://issues.apache.org/jira/browse/AVRO-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176972#comment-15176972 ] Thiruvalluvan M. G. commented on AVRO-1784: --- I briefly looked at this and it appears good. I'll work on this this weekend. I'm sorry about not able to devote time in the recent months. > Should have a C++ json encoder that pretty prints. > -- > > Key: AVRO-1784 > URL: https://issues.apache.org/jira/browse/AVRO-1784 > Project: Avro > Issue Type: Improvement > Components: c++ >Reporter: John McClean > Attachments: AVRO-1784.patch > > > It would nice to allow json ouput to be pretty printed. It's obviously > possible to post-process, but that frequently re-orders which results in the > problem described in AVRO-1636. In addition it obviously requires another > pass through the data. > I'll attach a patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AVRO-1784) Should have a C++ json encoder that pretty prints.
[ https://issues.apache.org/jira/browse/AVRO-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1784: -- Assignee: John McClean > Should have a C++ json encoder that pretty prints. > -- > > Key: AVRO-1784 > URL: https://issues.apache.org/jira/browse/AVRO-1784 > Project: Avro > Issue Type: Improvement > Components: c++ >Reporter: John McClean >Assignee: John McClean > Attachments: AVRO-1784.patch > > > It would nice to allow json ouput to be pretty printed. It's obviously > possible to post-process, but that frequently re-orders which results in the > problem described in AVRO-1636. In addition it obviously requires another > pass through the data. > I'll attach a patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AVRO-1784) Should have a C++ json encoder that pretty prints.
[
https://issues.apache.org/jira/browse/AVRO-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15236617#comment-15236617
]
Thiruvalluvan M. G. commented on AVRO-1784:
---
I'm sorry for the delay.
I made two small modifications:
* Changed {{FORMATTER}} to {{F}}, just to keep the convention of single-letter
names for template parameters.
* Made {{JsonNullFormatter}} the default value for {{F}}. This will make any
old code work normally. It is unlikely that anybody used the code outside this
library because it is not part of the api. But still, I made it backward
compatible.
I could not commit because of apache had some permission issue. I'll commit it
as soon as the problem is resolved.
> Should have a C++ json encoder that pretty prints.
> --
>
> Key: AVRO-1784
> URL: https://issues.apache.org/jira/browse/AVRO-1784
> Project: Avro
> Issue Type: Improvement
> Components: c++
>Reporter: John McClean
>Assignee: John McClean
> Attachments: AVRO-1784.patch
>
>
> It would nice to allow json ouput to be pretty printed. It's obviously
> possible to post-process, but that frequently re-orders which results in the
> problem described in AVRO-1636. In addition it obviously requires another
> pass through the data.
> I'll attach a patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AVRO-1784) Should have a C++ json encoder that pretty prints.
[ https://issues.apache.org/jira/browse/AVRO-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-1784: -- Attachment: AVRO-1784-A.patch > Should have a C++ json encoder that pretty prints. > -- > > Key: AVRO-1784 > URL: https://issues.apache.org/jira/browse/AVRO-1784 > Project: Avro > Issue Type: Improvement > Components: c++ >Reporter: John McClean >Assignee: John McClean > Attachments: AVRO-1784-A.patch, AVRO-1784.patch > > > It would nice to allow json ouput to be pretty printed. It's obviously > possible to post-process, but that frequently re-orders which results in the > problem described in AVRO-1636. In addition it obviously requires another > pass through the data. > I'll attach a patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AVRO-1701) warning: comparison between 'const enum testgen_r::ExampleEnum' and 'const enum testgen::ExampleEnum'
[
https://issues.apache.org/jira/browse/AVRO-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15236651#comment-15236651
]
Thiruvalluvan M. G. commented on AVRO-1701:
---
Normally, there is no need to compare two different schemas. Here we are
testing schema evolution where the writer and reader schemas are compatible in
Avro sense, but are different in C++ viewpoint. The code is the test code which
wants to ensure that the enum value read corresponds to to enum value written.
But instead of making implicit conversion from enum to {{int}} and compare, it
is preferable to make explicit comparison. I've attached a modified patch.
> warning: comparison between 'const enum testgen_r::ExampleEnum' and 'const
> enum testgen::ExampleEnum'
> -
>
> Key: AVRO-1701
> URL: https://issues.apache.org/jira/browse/AVRO-1701
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.7
>Reporter: peter liu
>Assignee: peter liu
>Priority: Minor
> Attachments: AVRO-1701-A.patch, AVRO-1701.1.patch
>
>
> saw below warning while compiling with g++ 4.2 and g++ 4.8 on mac os and linux
> {quote}
> [ 85%] Building CXX object
> CMakeFiles/AvrogencppTests.dir/test/AvrogencppTests.cc.o
> In file included from
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/impl/framework.ipp:29:0,
> from
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/included/unit_test.hpp:20,
> from
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/included/unit_test_framework.hpp:2,
> from
> /Users/liuyanbo/git/avro/lang/c++/test/AvrogencppTests.cc:33:
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp: In
> instantiation of 'boost::test_tools::predicate_result
> boost::test_tools::tt_detail::equal_impl(const Left&, const Right&) [with
> Left = testgen_r::ExampleEnum; Right = testgen::ExampleEnum]':
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp:560:40:
> required from 'boost::test_tools::predicate_result
> boost::test_tools::tt_detail::equal_impl_frwd::call_impl(const Left&, const
> Right&, mpl_::false_) const [with Left = testgen_r::ExampleEnum; Right =
> testgen::ExampleEnum; mpl_::false_ = mpl_::bool_]'
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp:575:56:
> required from 'boost::test_tools::predicate_result
> boost::test_tools::tt_detail::equal_impl_frwd::operator()(const Left&, const
> Right&) const [with Left = testgen_r::ExampleEnum; Right =
> testgen::ExampleEnum]'
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp:523:1:
> required from 'bool boost::test_tools::tt_detail::check_frwd(Pred, const
> boost::unit_test::lazy_ostream&, boost::test_tools::const_string,
> std::size_t, boost::test_tools::tt_detail::tool_level,
> boost::test_tools::tt_detail::check_type, const Arg0&, const char*, const
> Arg1&, const char*) [with Pred =
> boost::test_tools::tt_detail::equal_impl_frwd; Arg0 = testgen_r::ExampleEnum;
> Arg1 = testgen::ExampleEnum; boost::test_tools::const_string =
> boost::unit_test::basic_cstring; std::size_t = long unsigned int]'
> /Users/liuyanbo/git/avro/lang/c++/test/AvrogencppTests.cc:124:5: required
> from 'void checkRecord(const T1&, const T2&) [with T1 =
> testgen_r::RootRecord; T2 = testgen::RootRecord]'
> /Users/liuyanbo/git/avro/lang/c++/test/AvrogencppTests.cc:180:23: required
> from here
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp:536:17:
> warning: comparison between 'const enum testgen_r::ExampleEnum' and 'const
> enum testgen::ExampleEnum' [-Wenum-compare]
> return left == right;
> ^
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AVRO-1701) warning: comparison between 'const enum testgen_r::ExampleEnum' and 'const enum testgen::ExampleEnum'
[
https://issues.apache.org/jira/browse/AVRO-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1701:
--
Attachment: AVRO-1701-A.patch
> warning: comparison between 'const enum testgen_r::ExampleEnum' and 'const
> enum testgen::ExampleEnum'
> -
>
> Key: AVRO-1701
> URL: https://issues.apache.org/jira/browse/AVRO-1701
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.7.7
>Reporter: peter liu
>Assignee: peter liu
>Priority: Minor
> Attachments: AVRO-1701-A.patch, AVRO-1701.1.patch
>
>
> saw below warning while compiling with g++ 4.2 and g++ 4.8 on mac os and linux
> {quote}
> [ 85%] Building CXX object
> CMakeFiles/AvrogencppTests.dir/test/AvrogencppTests.cc.o
> In file included from
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/impl/framework.ipp:29:0,
> from
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/included/unit_test.hpp:20,
> from
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/included/unit_test_framework.hpp:2,
> from
> /Users/liuyanbo/git/avro/lang/c++/test/AvrogencppTests.cc:33:
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp: In
> instantiation of 'boost::test_tools::predicate_result
> boost::test_tools::tt_detail::equal_impl(const Left&, const Right&) [with
> Left = testgen_r::ExampleEnum; Right = testgen::ExampleEnum]':
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp:560:40:
> required from 'boost::test_tools::predicate_result
> boost::test_tools::tt_detail::equal_impl_frwd::call_impl(const Left&, const
> Right&, mpl_::false_) const [with Left = testgen_r::ExampleEnum; Right =
> testgen::ExampleEnum; mpl_::false_ = mpl_::bool_]'
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp:575:56:
> required from 'boost::test_tools::predicate_result
> boost::test_tools::tt_detail::equal_impl_frwd::operator()(const Left&, const
> Right&) const [with Left = testgen_r::ExampleEnum; Right =
> testgen::ExampleEnum]'
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp:523:1:
> required from 'bool boost::test_tools::tt_detail::check_frwd(Pred, const
> boost::unit_test::lazy_ostream&, boost::test_tools::const_string,
> std::size_t, boost::test_tools::tt_detail::tool_level,
> boost::test_tools::tt_detail::check_type, const Arg0&, const char*, const
> Arg1&, const char*) [with Pred =
> boost::test_tools::tt_detail::equal_impl_frwd; Arg0 = testgen_r::ExampleEnum;
> Arg1 = testgen::ExampleEnum; boost::test_tools::const_string =
> boost::unit_test::basic_cstring; std::size_t = long unsigned int]'
> /Users/liuyanbo/git/avro/lang/c++/test/AvrogencppTests.cc:124:5: required
> from 'void checkRecord(const T1&, const T2&) [with T1 =
> testgen_r::RootRecord; T2 = testgen::RootRecord]'
> /Users/liuyanbo/git/avro/lang/c++/test/AvrogencppTests.cc:180:23: required
> from here
> /Users/liuyanbo/Downloads/boost_1_56_0/boost/test/test_tools.hpp:536:17:
> warning: comparison between 'const enum testgen_r::ExampleEnum' and 'const
> enum testgen::ExampleEnum' [-Wenum-compare]
> return left == right;
> ^
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
