[jira] [Created] (AVRO-2239) Fix interop tests
Fokko Driesprong created AVRO-2239: -- Summary: Fix interop tests Key: AVRO-2239 URL: https://issues.apache.org/jira/browse/AVRO-2239 Project: Avro Issue Type: Improvement Reporter: Fokko Driesprong Currently the interop tests are failing: /avro/lang/ruby/lib/avro/schema_validator.rb:127:in `validate_map': undefined method `keys' for nil:NilClass (NoMethodError) from /avro/lang/ruby/lib/avro/schema_validator.rb:103:in `validate_recursive' from /avro/lang/ruby/lib/avro/schema_validator.rb:156:in `block in validate_possible_types' from /avro/lang/ruby/lib/avro/schema_validator.rb:154:in `map' from /avro/lang/ruby/lib/avro/schema_validator.rb:154:in `validate_possible_types' from /avro/lang/ruby/lib/avro/schema_validator.rb:141:in `validate_union' from /avro/lang/ruby/lib/avro/schema_validator.rb:105:in `validate_recursive' from /avro/lang/ruby/lib/avro/schema_validator.rb:110:in `block in validate_recursive' from /avro/lang/ruby/lib/avro/schema_validator.rb:108:in `each' from /avro/lang/ruby/lib/avro/schema_validator.rb:108:in `validate_recursive' from /avro/lang/ruby/lib/avro/schema_validator.rb:67:in `validate!' from /avro/lang/ruby/lib/avro/schema.rb:97:in `validate' from /avro/lang/ruby/lib/avro/io.rb:503:in `write_data' from /avro/lang/ruby/lib/avro/io.rb:499:in `write' from /avro/lang/ruby/lib/avro/ipc.rb:136:in `write_handshake_request' from /avro/lang/ruby/lib/avro/ipc.rb:105:in `request' from lang/ruby/test/tool.rb:69:in `send_message' from lang/ruby/test/tool.rb:137:in `main' from lang/ruby/test/tool.rb:143:in `' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2241) Fix Jsr310/Joda tests when comparing datetimes
Fokko Driesprong created AVRO-2241:
--
Summary: Fix Jsr310/Joda tests when comparing datetimes
Key: AVRO-2241
URL: https://issues.apache.org/jira/browse/AVRO-2241
Project: Avro
Issue Type: Improvement
Components: java
Affects Versions: 1.8.2
Reporter: Fokko Driesprong
Assignee: Fokko Driesprong
Fix For: 1.9.0
Whe comparing Instant, DateTime, LocalTime with its Joda equivalents the tests
fail since the java.time.* adds support for microseconds, and Joda only
miliseconds. For example:
Failed tests:
TestSpecificLogicalTypes.testAbilityToReadJodaRecordWrittenAsJsr310Record:183
Expected: is "11:45:07.170331"
but: was "11:45:07.170"
TestSpecificLogicalTypes.testRecordWithJsr310LogicalTypes:115 Should match
written record expected:<{"b": true, "i32": 34, "i64": 35, "f32": 3.14, "f64":
3019.34, "s": null, "d": 2018-10-11, "t": 11:45:06.513252, "ts":
2018-10-11T11:45:06.513358Z, "dec": 123.45}> but was:<{"b": true, "i32": 34,
"i64": 35, "f32": 3.14, "f64": 3019.34, "s": null, "d": 2018-10-11, "t":
11:45:06.513, "ts": 2018-10-11T11:45:06.513Z, "dec": 123.45}>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AVRO-2246) Enable Travis-CI for Apache Avro
Fokko Driesprong created AVRO-2246: -- Summary: Enable Travis-CI for Apache Avro Key: AVRO-2246 URL: https://issues.apache.org/jira/browse/AVRO-2246 Project: Avro Issue Type: Task Reporter: Fokko Driesprong We would like to enable Travis CI for Apache Avro to run tests on each commit and PR's. Thanks! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2246) Enable Travis-CI for Apache Avro
[ https://issues.apache.org/jira/browse/AVRO-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2246. Resolution: Won't Fix > Enable Travis-CI for Apache Avro > > > Key: AVRO-2246 > URL: https://issues.apache.org/jira/browse/AVRO-2246 > Project: Avro > Issue Type: Task >Reporter: Fokko Driesprong >Priority: Major > > We would like to enable Travis CI for Apache Avro to run tests on each commit > and PR's. Thanks! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2249) Fix the C++ tests
Fokko Driesprong created AVRO-2249:
--
Summary: Fix the C++ tests
Key: AVRO-2249
URL: https://issues.apache.org/jira/browse/AVRO-2249
Project: Avro
Issue Type: Task
Components: c++
Affects Versions: 1.8.2
Reporter: Fokko Driesprong
Fix For: 1.9.0
Currently the C++ tests are failing:
```
Scanning dependencies of target avrocpp_s
[ 1%] Building CXX object CMakeFiles/avrocpp_s.dir/impl/Compiler.cc.o
/testptch/unknown/lang/c++/impl/Compiler.cc:174:15: warning: ‘std::string
avro::nameof(const NodePtr&)’ defined but not used [-Wunused-function]
static string nameof(const NodePtr& n)
^
[ 2%] Building CXX object CMakeFiles/avrocpp_s.dir/impl/Node.cc.o
[ 4%] Building CXX object CMakeFiles/avrocpp_s.dir/impl/NodeImpl.cc.o
/testptch/unknown/lang/c++/impl/NodeImpl.cc: In function ‘std::string
avro::{anonymous}::escape(const string&)’:
/testptch/unknown/lang/c++/impl/NodeImpl.cc:31:8: warning: ‘auto’ changes
meaning in C++11; please remove it [-Wc++0x-compat]
for (auto c : unescaped) {
^
/testptch/unknown/lang/c++/impl/NodeImpl.cc:31:13: error: ‘c’ does not name
a type
for (auto c : unescaped) {
^
/testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected ‘;’
before ‘return’
return s;
^
/testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected
primary-expression before ‘return’
/testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected ‘;’
before ‘return’
/testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected
primary-expression before ‘return’
/testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected ‘)’
before ‘return’
In file included from /usr/include/boost/assert.hpp:51:0,
from /usr/include/boost/format/parsing.hpp:21,
from /usr/include/boost/format.hpp:50,
from /testptch/unknown/lang/c++/api/Exception.hh:24,
from /testptch/unknown/lang/c++/api/Node.hh:28,
from /testptch/unknown/lang/c++/api/GenericDatum.hh:29,
from /testptch/unknown/lang/c++/api/NodeImpl.hh:23,
from /testptch/unknown/lang/c++/impl/NodeImpl.cc:21:
/testptch/unknown/lang/c++/impl/NodeImpl.cc: In member function ‘virtual void
avro::NodeRecord::printJson(std::ostream&, int) const’:
/testptch/unknown/lang/c++/impl/NodeImpl.cc:247:59: warning: comparison between
signed and unsigned integer expressions [-Wsign-compare]
assert(defaultValues.empty() || (defaultValues.size() == fields));
^
/testptch/unknown/lang/c++/impl/NodeImpl.cc: In member function ‘virtual void
avro::NodePrimitive::printDefaultToJson(const avro::GenericDatum&,
std::ostream&, int) const’:
/testptch/unknown/lang/c++/impl/NodeImpl.cc:292:13: error: ‘to_string’ is
not a member of ‘std’
os << std::to_string(g.value());
^
/testptch/unknown/lang/c++/impl/NodeImpl.cc:295:13: error: ‘to_string’ is
not a member of ‘std’
os << std::to_string(g.value());
^
/testptch/unknown/lang/c++/impl/NodeImpl.cc: In function ‘std::string
avro::{anonymous}::escape(const string&)’:
/testptch/unknown/lang/c++/impl/NodeImpl.cc:68:1: warning: control reaches end
of non-void function [-Wreturn-type]
}
^
CMakeFiles/avrocpp_s.dir/build.make:100: recipe for target
'CMakeFiles/avrocpp_s.dir/impl/NodeImpl.cc.o' failed
make[2]: *** [CMakeFiles/avrocpp_s.dir/impl/NodeImpl.cc.o] Error 1
make[1]: *** [CMakeFiles/avrocpp_s.dir/all] Error 2
CMakeFiles/Makefile2:425: recipe for target 'CMakeFiles/avrocpp_s.dir/all'
failed
make: *** [all] Error 2
Makefile:147: recipe for target 'all' failed
```
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-1695) Add LogicalType support to the Ruby library
[ https://issues.apache.org/jira/browse/AVRO-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-1695: --- Fix Version/s: 1.9.0 > Add LogicalType support to the Ruby library > --- > > Key: AVRO-1695 > URL: https://issues.apache.org/jira/browse/AVRO-1695 > Project: Avro > Issue Type: New Feature > Components: ruby >Reporter: Daniel Schierbeck >Priority: Major > Fix For: 1.9.0 > > > It would be nice if the Ruby library had feature parity. I would be willing > to contribute work if someone could review and merge the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-1695) Add LogicalType support to the Ruby library
[ https://issues.apache.org/jira/browse/AVRO-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-1695: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Add LogicalType support to the Ruby library > --- > > Key: AVRO-1695 > URL: https://issues.apache.org/jira/browse/AVRO-1695 > Project: Apache Avro > Issue Type: New Feature > Components: ruby >Reporter: Daniel Schierbeck >Assignee: Martin Jubelgas >Priority: Major > Fix For: 1.9.0 > > > It would be nice if the Ruby library had feature parity. I would be willing > to contribute work if someone could review and merge the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-2234) Use native MappingProxyType instead of hand-written ImmutableDict
[ https://issues.apache.org/jira/browse/AVRO-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong reassigned AVRO-2234: -- Assignee: Michael A. Smith > Use native MappingProxyType instead of hand-written ImmutableDict > - > > Key: AVRO-2234 > URL: https://issues.apache.org/jira/browse/AVRO-2234 > Project: Apache Avro > Issue Type: Improvement > Components: python >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > Fix For: 1.9.0 > > > Per the rejection notice in > [PEP-416,|https://www.python.org/dev/peps/pep-0416/] there is no native > ImmutableDict in Python. However, there is a MappingProxyType that makes a > proxy to the mapping of a dictionary, and is fully native. For most purposes, > including avro's, it's a fully suitable replacement for a real ImmutableDict > implementation (except for a minor detail about JSON encoding that is easier > to solve and support than maintaining a complete implementation of > ImmutableDict.) > I propose that we drop ImmutableDict and replace it with MappingProxyType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2240) egg_info fails for avro-python3 using python2
[
https://issues.apache.org/jira/browse/AVRO-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AVRO-2240.
Resolution: Fixed
> egg_info fails for avro-python3 using python2
> -
>
> Key: AVRO-2240
> URL: https://issues.apache.org/jira/browse/AVRO-2240
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.8.2
>Reporter: Luke Zulauf
>Assignee: Michael A. Smith
>Priority: Major
> Fix For: 1.9.0
>
>
> When managing dependencies, the ability to call egg_info on any package is
> especially useful (even if the package is not valid for installation with a
> specific python version).
> For example, pipenv can't build a lockfile in python2 if avro-python3 is
> listed as a dependency (for any python version)
>
> The opposite works (using python3 to check the egg_info of avro (the python2
> version).
>
> {code:java}
> $ python3 avro/setup.py egg_info
> (succeeds)
> $ python2 avro/setup.py egg_info
> (succeeds)
> $ python3 avro-python3/setup.py egg_info
> (succeeds)
> $ python2 avro-python3/setup.py egg_info
> (fails){code}
> It would be great if you didn't raise an assertion error for all usages of
> running setup.py, especially when running egg_info.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (AVRO-2240) egg_info fails for avro-python3 using python2
[
https://issues.apache.org/jira/browse/AVRO-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong reassigned AVRO-2240:
--
Assignee: Michael A. Smith
> egg_info fails for avro-python3 using python2
> -
>
> Key: AVRO-2240
> URL: https://issues.apache.org/jira/browse/AVRO-2240
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.8.2
>Reporter: Luke Zulauf
>Assignee: Michael A. Smith
>Priority: Major
> Fix For: 1.9.0
>
>
> When managing dependencies, the ability to call egg_info on any package is
> especially useful (even if the package is not valid for installation with a
> specific python version).
> For example, pipenv can't build a lockfile in python2 if avro-python3 is
> listed as a dependency (for any python version)
>
> The opposite works (using python3 to check the egg_info of avro (the python2
> version).
>
> {code:java}
> $ python3 avro/setup.py egg_info
> (succeeds)
> $ python2 avro/setup.py egg_info
> (succeeds)
> $ python3 avro-python3/setup.py egg_info
> (succeeds)
> $ python2 avro-python3/setup.py egg_info
> (fails){code}
> It would be great if you didn't raise an assertion error for all usages of
> running setup.py, especially when running egg_info.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-1808) It would be useful to be able to change lock mechanics for handshake at Requestor
[ https://issues.apache.org/jira/browse/AVRO-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-1808: --- Status: Open (was: Patch Available) > It would be useful to be able to change lock mechanics for handshake at > Requestor > -- > > Key: AVRO-1808 > URL: https://issues.apache.org/jira/browse/AVRO-1808 > Project: Apache Avro > Issue Type: Improvement > Components: java >Reporter: Konstantin Usachev >Assignee: Konstantin Usachev >Priority: Minor > > For now there is no option to change lock mechanics for handshake at > org.apache.avro.ipc.Requestor - it uses ReentrantLock. We can't use this type > of lock in our system, so we need some way to override it, like the way you > have already done for org.apache.avro.ipc.Transceiver.channelLock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2240) egg_info fails for avro-python3 using python2
[
https://issues.apache.org/jira/browse/AVRO-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong updated AVRO-2240:
---
Fix Version/s: 1.9.0
> egg_info fails for avro-python3 using python2
> -
>
> Key: AVRO-2240
> URL: https://issues.apache.org/jira/browse/AVRO-2240
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.8.2
>Reporter: Luke Zulauf
>Assignee: Michael A. Smith
>Priority: Major
> Fix For: 1.9.0
>
>
> When managing dependencies, the ability to call egg_info on any package is
> especially useful (even if the package is not valid for installation with a
> specific python version).
> For example, pipenv can't build a lockfile in python2 if avro-python3 is
> listed as a dependency (for any python version)
>
> The opposite works (using python3 to check the egg_info of avro (the python2
> version).
>
> {code:java}
> $ python3 avro/setup.py egg_info
> (succeeds)
> $ python2 avro/setup.py egg_info
> (succeeds)
> $ python3 avro-python3/setup.py egg_info
> (succeeds)
> $ python2 avro-python3/setup.py egg_info
> (fails){code}
> It would be great if you didn't raise an assertion error for all usages of
> running setup.py, especially when running egg_info.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (AVRO-1808) It would be useful to be able to change lock mechanics for handshake at Requestor
[ https://issues.apache.org/jira/browse/AVRO-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-1808. Resolution: Won't Fix > It would be useful to be able to change lock mechanics for handshake at > Requestor > -- > > Key: AVRO-1808 > URL: https://issues.apache.org/jira/browse/AVRO-1808 > Project: Apache Avro > Issue Type: Improvement > Components: java >Reporter: Konstantin Usachev >Assignee: Konstantin Usachev >Priority: Minor > > For now there is no option to change lock mechanics for handshake at > org.apache.avro.ipc.Requestor - it uses ReentrantLock. We can't use this type > of lock in our system, so we need some way to override it, like the way you > have already done for org.apache.avro.ipc.Transceiver.channelLock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-1695) Add LogicalType support to the Ruby library
[ https://issues.apache.org/jira/browse/AVRO-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong reassigned AVRO-1695: -- Assignee: Martin Jubelgas > Add LogicalType support to the Ruby library > --- > > Key: AVRO-1695 > URL: https://issues.apache.org/jira/browse/AVRO-1695 > Project: Apache Avro > Issue Type: New Feature > Components: ruby >Reporter: Daniel Schierbeck >Assignee: Martin Jubelgas >Priority: Major > Fix For: 1.9.0 > > > It would be nice if the Ruby library had feature parity. I would be willing > to contribute work if someone could review and merge the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2183) Nameless schema should raise AttributeError when attempting to access name
[
https://issues.apache.org/jira/browse/AVRO-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AVRO-2183.
Resolution: Fixed
Assignee: Michael A. Smith
Hadoop Flags: Reviewed
Fix Version/s: 1.9.0
> Nameless schema should raise AttributeError when attempting to access name
> --
>
> Key: AVRO-2183
> URL: https://issues.apache.org/jira/browse/AVRO-2183
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Reporter: Michael A. Smith
>Assignee: Michael A. Smith
>Priority: Major
> Fix For: 1.9.0
>
>
> In the "py" implementation, this works as expected:
> {noformat}
> >>> from avro.schema import parse
> >>> s = parse('{"type": "array", "items": "int"}')
> >>> s.name
> Traceback (most recent call last):
> File "", line 1, in
> AttributeError: 'ArraySchema' object has no attribute 'name'{noformat}
> but in the py3 implementation, a {{NameError}} is raised instead:
> {noformat}
> >>> from avro.schema import Parse
> >>> s=Parse('{"type":"array","items":"int"}')
> >>> s.name
> Traceback (most recent call last):
> File "", line 1, in
> File "/home/michaels/dev/avro/lang/py3/avro/schema.py", line 224, in name
> return self._props['name']
> KeyError: 'name'{noformat}
> This behavior breaks several python idioms, including the ability to get a
> default value using {{getattr}}:
> {noformat}
> >>> getattr(s, "name", "default")
> Traceback (most recent call last):
> File "", line 1, in
> File "/home/michaels/dev/avro/lang/py3/avro/schema.py", line 224, in name
> return self._props['name']
> KeyError: 'name'{noformat}
> I will open a PR with tests and a fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2261) Bump RAT plugin to 0.13
[ https://issues.apache.org/jira/browse/AVRO-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2261: --- Affects Version/s: 1.8.2 > Bump RAT plugin to 0.13 > --- > > Key: AVRO-2261 > URL: https://issues.apache.org/jira/browse/AVRO-2261 > Project: Apache Avro > Issue Type: Task >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2261) Bump RAT plugin to 0.13
Fokko Driesprong created AVRO-2261: -- Summary: Bump RAT plugin to 0.13 Key: AVRO-2261 URL: https://issues.apache.org/jira/browse/AVRO-2261 Project: Apache Avro Issue Type: Task Reporter: Fokko Driesprong -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2262) Java compression codec improvements
[ https://issues.apache.org/jira/browse/AVRO-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2262. Resolution: Fixed > Java compression codec improvements > --- > > Key: AVRO-2262 > URL: https://issues.apache.org/jira/browse/AVRO-2262 > Project: Apache Avro > Issue Type: Task > Components: java >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Jacob Tolar >Priority: Major > Fix For: 1.9.0 > > > * Update a few things to use try-with-resources > * Updated CodecFactory to reference constants for codec names > * Fixed a small bug in Snappy and BZip2: compression/decompression were > incorrect if the input ByteBuffer was a a slice(). I don't see anywhere that > this would actually happen currently, but some codecs were written to account > for this correctly; now they're all correct. Updated everything to compute > the correct offset into the underlying array. (I can add a test for this in > TestAllCodecs once #351 is merged). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2262) Java compression codec improvements
Fokko Driesprong created AVRO-2262: -- Summary: Java compression codec improvements Key: AVRO-2262 URL: https://issues.apache.org/jira/browse/AVRO-2262 Project: Apache Avro Issue Type: Task Components: java Affects Versions: 1.8.2 Reporter: Fokko Driesprong Assignee: Jacob Tolar Fix For: 1.9.0 * Update a few things to use try-with-resources * Updated CodecFactory to reference constants for codec names * Fixed a small bug in Snappy and BZip2: compression/decompression were incorrect if the input ByteBuffer was a a slice(). I don't see anywhere that this would actually happen currently, but some codecs were written to account for this correctly; now they're all correct. Updated everything to compute the correct offset into the underlying array. (I can add a test for this in TestAllCodecs once #351 is merged). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2265) Remove Guava as a dependency
[ https://issues.apache.org/jira/browse/AVRO-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2265. Resolution: Fixed > Remove Guava as a dependency > > > Key: AVRO-2265 > URL: https://issues.apache.org/jira/browse/AVRO-2265 > Project: Apache Avro > Issue Type: Task > Components: java >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > > Get rid of Guava in the tests. Guava is a dependency that can easily clash > with different versions. Since it is used very little, it makes sense to get > rid of it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2264) Bump Apache commons-lang3 to 3.5.1
Fokko Driesprong created AVRO-2264: -- Summary: Bump Apache commons-lang3 to 3.5.1 Key: AVRO-2264 URL: https://issues.apache.org/jira/browse/AVRO-2264 Project: Apache Avro Issue Type: Task Components: java Affects Versions: 1.8.2 Reporter: Fokko Driesprong Assignee: Fokko Driesprong -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2264) Bump Apache commons-lang3 to 3.8.1
[ https://issues.apache.org/jira/browse/AVRO-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2264: --- Summary: Bump Apache commons-lang3 to 3.8.1 (was: Bump Apache commons-lang3 to 3.5.1) > Bump Apache commons-lang3 to 3.8.1 > -- > > Key: AVRO-2264 > URL: https://issues.apache.org/jira/browse/AVRO-2264 > Project: Apache Avro > Issue Type: Task > Components: java >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2263) Bump Apache commons-cli to 1.4
Fokko Driesprong created AVRO-2263: -- Summary: Bump Apache commons-cli to 1.4 Key: AVRO-2263 URL: https://issues.apache.org/jira/browse/AVRO-2263 Project: Apache Avro Issue Type: Task Components: tools Affects Versions: 1.8.2 Reporter: Fokko Driesprong Assignee: Fokko Driesprong -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2263) Bump Apache commons-cli to 1.4
[ https://issues.apache.org/jira/browse/AVRO-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2263. Resolution: Fixed Fix Version/s: 1.9.0 > Bump Apache commons-cli to 1.4 > -- > > Key: AVRO-2263 > URL: https://issues.apache.org/jira/browse/AVRO-2263 > Project: Apache Avro > Issue Type: Task > Components: tools >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2261) Bump RAT plugin to 0.13
[ https://issues.apache.org/jira/browse/AVRO-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2261. Resolution: Fixed Assignee: Fokko Driesprong Fix Version/s: 1.9.0 > Bump RAT plugin to 0.13 > --- > > Key: AVRO-2261 > URL: https://issues.apache.org/jira/browse/AVRO-2261 > Project: Apache Avro > Issue Type: Task >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-1705) Set up CI job to test all languages using Docker
[ https://issues.apache.org/jira/browse/AVRO-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-1705. Resolution: Fixed Assignee: Fokko Driesprong Fix Version/s: 1.9.0 > Set up CI job to test all languages using Docker > > > Key: AVRO-1705 > URL: https://issues.apache.org/jira/browse/AVRO-1705 > Project: Apache Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.7 >Reporter: Tom White >Assignee: Fokko Driesprong >Priority: Critical > Labels: starter > Fix For: 1.9.0 > > > The ASF Jenkins instance now supports Docker (BUILDS-25), so we could run all > the tests (for all languages that Avro supports) using the Avro Dockerfile. > We might also do a nightly build of the whole distribution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-1563) when server use Netty server , clients can't use Http protocal to connect
[ https://issues.apache.org/jira/browse/AVRO-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-1563. Resolution: Cannot Reproduce > when server use Netty server , clients can't use Http protocal to connect > - > > Key: AVRO-1563 > URL: https://issues.apache.org/jira/browse/AVRO-1563 > Project: Apache Avro > Issue Type: New Feature >Affects Versions: 1.7.7 >Reporter: wu zhi hui >Priority: Major > > when server use Netty server , clients can't use Http protocal to connect > if server use Http server ,the performance is bad! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-597) Avro should be usable from JDK 1.5 programs
[ https://issues.apache.org/jira/browse/AVRO-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-597. --- Resolution: Won't Fix Fix Version/s: 1.3.3 Won't fix anything for JDK 1.5 anymore. > Avro should be usable from JDK 1.5 programs > --- > > Key: AVRO-597 > URL: https://issues.apache.org/jira/browse/AVRO-597 > Project: Apache Avro > Issue Type: Improvement > Components: build, java >Affects Versions: 1.3.3 >Reporter: Jon Travis >Priority: Major > Fix For: 1.3.3 > > Attachments: avro-597-jdk15.patch > > > Our application needs to run on a 1.5 JDK, but Avro classes are built for > 1.6. > The change to get this to work are pretty minimal. > - Removed @Override on interfaces, since 1.5 does not support this > - Replace usage of Arrays.copyOf with our own version of it -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-1579) Support CI for patches submitted through JIRA
[ https://issues.apache.org/jira/browse/AVRO-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-1579. Resolution: Fixed Assignee: Fokko Driesprong Fix Version/s: 1.9.0 This has been fixed using Travis > Support CI for patches submitted through JIRA > - > > Key: AVRO-1579 > URL: https://issues.apache.org/jira/browse/AVRO-1579 > Project: Apache Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.7 >Reporter: Stevo Slavic >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > > Other Apache projects (e.g. Kafka) have already JIRA - CI integration > enabled, so that patches submitted through JIRA get automatically built and > build report published back to JIRA as comment. Please consider supporting > this for Avro project as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2239) Fix interop tests
[ https://issues.apache.org/jira/browse/AVRO-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685235#comment-16685235 ] Fokko Driesprong commented on AVRO-2239: [~tjwp] Please do, I had to disable the tests for now: https://github.com/apache/avro/blob/master/share/test/interop/bin/test_rpc_interop.sh#L37-L38 > Fix interop tests > - > > Key: AVRO-2239 > URL: https://issues.apache.org/jira/browse/AVRO-2239 > Project: Apache Avro > Issue Type: Improvement > Components: interop >Reporter: Fokko Driesprong >Assignee: Tim Perkins >Priority: Major > > Currently the interop tests are failing: > /avro/lang/ruby/lib/avro/schema_validator.rb:127:in `validate_map': undefined > method `keys' for nil:NilClass (NoMethodError) > from /avro/lang/ruby/lib/avro/schema_validator.rb:103:in > `validate_recursive' > from /avro/lang/ruby/lib/avro/schema_validator.rb:156:in `block in > validate_possible_types' > from /avro/lang/ruby/lib/avro/schema_validator.rb:154:in `map' > from /avro/lang/ruby/lib/avro/schema_validator.rb:154:in > `validate_possible_types' > from /avro/lang/ruby/lib/avro/schema_validator.rb:141:in > `validate_union' > from /avro/lang/ruby/lib/avro/schema_validator.rb:105:in > `validate_recursive' > from /avro/lang/ruby/lib/avro/schema_validator.rb:110:in `block in > validate_recursive' > from /avro/lang/ruby/lib/avro/schema_validator.rb:108:in `each' > from /avro/lang/ruby/lib/avro/schema_validator.rb:108:in > `validate_recursive' > from /avro/lang/ruby/lib/avro/schema_validator.rb:67:in `validate!' > from /avro/lang/ruby/lib/avro/schema.rb:97:in `validate' > from /avro/lang/ruby/lib/avro/io.rb:503:in `write_data' > from /avro/lang/ruby/lib/avro/io.rb:499:in `write' > from /avro/lang/ruby/lib/avro/ipc.rb:136:in `write_handshake_request' > from /avro/lang/ruby/lib/avro/ipc.rb:105:in `request' > from lang/ruby/test/tool.rb:69:in `send_message' > from lang/ruby/test/tool.rb:137:in `main' > from lang/ruby/test/tool.rb:143:in `' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-1887) set up precommit checks via Apache Yetus
[ https://issues.apache.org/jira/browse/AVRO-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-1887: --- Resolution: Fixed Assignee: Fokko Driesprong (was: Suraj Acharya) Hadoop Flags: Reviewed Fix Version/s: 1.9.0 Status: Resolved (was: Patch Available) > set up precommit checks via Apache Yetus > > > Key: AVRO-1887 > URL: https://issues.apache.org/jira/browse/AVRO-1887 > Project: Avro > Issue Type: Task > Components: build, community >Reporter: Sean Busbey >Assignee: Fokko Driesprong >Priority: Blocker > Fix For: 1.9.0 > > Attachments: AVRO-1887-0.patch, AVRO-1887-1.patch, AVRO-1887-2.patch > > > The lack of automated checks for things like doing a full build is negatively > impacting my ability to do timely reviews. We should leverage the work done > by other communities and get precommit checks going with Apache Yetus ([ref > Apache Yetus > Precommit|http://yetus.apache.org/documentation/0.3.0/precommit-basic/]) > Yetus has built in support for using Docker as a containerized build > environment ([ref > docs|https://yetus.apache.org/documentation/0.3.0/precommit-advanced/#Docker_Support]), > but I don't know if we can use the one we have directly or if we'll need to > do some refactoring. > Implementing this will ultimately require a committer (so that they can make > changes to builds.apache.org), but a non-cmmitter can get the bulk of the > work done by managing to do local yetus checks with the docker feature active. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2234) Use native MappingProxyType instead of hand-written ImmutableDict
[ https://issues.apache.org/jira/browse/AVRO-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2234. Resolution: Fixed Fix Version/s: 1.9.0 > Use native MappingProxyType instead of hand-written ImmutableDict > - > > Key: AVRO-2234 > URL: https://issues.apache.org/jira/browse/AVRO-2234 > Project: Avro > Issue Type: Improvement > Components: python >Reporter: Michael A. Smith >Priority: Major > Fix For: 1.9.0 > > > Per the rejection notice in > [PEP-416,|https://www.python.org/dev/peps/pep-0416/] there is no native > ImmutableDict in Python. However, there is a MappingProxyType that makes a > proxy to the mapping of a dictionary, and is fully native. For most purposes, > including avro's, it's a fully suitable replacement for a real ImmutableDict > implementation (except for a minor detail about JSON encoding that is easier > to solve and support than maintaining a complete implementation of > ImmutableDict.) > I propose that we drop ImmutableDict and replace it with MappingProxyType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2228) Bump Apache Velocity to 2.0
[ https://issues.apache.org/jira/browse/AVRO-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2228. Resolution: Fixed Assignee: Fokko Driesprong Fix Version/s: 1.9.0 > Bump Apache Velocity to 2.0 > --- > > Key: AVRO-2228 > URL: https://issues.apache.org/jira/browse/AVRO-2228 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > > Apache Velocity 1.7 contains CVE's: CVE-2015-6420, CVE-2017-15708 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2265) Remove Guava as a dependency
[ https://issues.apache.org/jira/browse/AVRO-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2265: --- Fix Version/s: 1.9.0 > Remove Guava as a dependency > > > Key: AVRO-2265 > URL: https://issues.apache.org/jira/browse/AVRO-2265 > Project: Apache Avro > Issue Type: Task > Components: java >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > > Get rid of Guava in the tests. Guava is a dependency that can easily clash > with different versions. Since it is used very little, it makes sense to get > rid of it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2265) Remove Guava as a dependency
Fokko Driesprong created AVRO-2265: -- Summary: Remove Guava as a dependency Key: AVRO-2265 URL: https://issues.apache.org/jira/browse/AVRO-2265 Project: Apache Avro Issue Type: Task Components: java Affects Versions: 1.8.2 Reporter: Fokko Driesprong Assignee: Fokko Driesprong Get rid of Guava in the tests. Guava is a dependency that can easily clash with different versions. Since it is used very little, it makes sense to get rid of it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AVRO-2265) Remove Guava as a dependency
[ https://issues.apache.org/jira/browse/AVRO-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AVRO-2265 started by Fokko Driesprong. -- > Remove Guava as a dependency > > > Key: AVRO-2265 > URL: https://issues.apache.org/jira/browse/AVRO-2265 > Project: Apache Avro > Issue Type: Task > Components: java >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > > Get rid of Guava in the tests. Guava is a dependency that can easily clash > with different versions. Since it is used very little, it makes sense to get > rid of it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2217) Vulnerabilities in avro bundled packages
[ https://issues.apache.org/jira/browse/AVRO-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2217. Resolution: Fixed Assignee: Fokko Driesprong Fix Version/s: 1.9.0 Jackson and Apache is already updated. We will get rid of Guava in AVRO-2217. > Vulnerabilities in avro bundled packages > > > Key: AVRO-2217 > URL: https://issues.apache.org/jira/browse/AVRO-2217 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.2 >Reporter: Prasanth Pallamreddy >Assignee: Fokko Driesprong >Priority: Critical > Fix For: 1.9.0 > > > The following vulnerabilities exist in the packages bundled by Avro. These > packages need to be upgraded to the latest versions. Although a few of these > vulnerabilities were raised a couple of years ago in AVRO-1126 and an attempt > to address the backwards compatibility issue in AVRO-1605 there does not > appear to be a resolution. If there is no resolution on these issues, we may > be forced to fork based on [this PR|https://github.com/apache/avro/pull/87]. > > org.codehaus.jackson:jackson-mapper-asl:1.9.13 which is known to have these > critical / high vulns: > [https://nvd.nist.gov/vuln/detail/CVE-2018-7489] > [https://nvd.nist.gov/vuln/detail/CVE-2017-15095] > [https://nvd.nist.gov/vuln/detail/CVE-2017-7525] > [https://nvd.nist.gov/vuln/detail/CVE-2017-17485] > [https://nvd.nist.gov/vuln/detail/CVE-2018-5968] > org.codehaus.jackson:jackson-core-asl:1.9.13 which has this high > vulnerability: > - [https://nvd.nist.gov/vuln/detail/CVE-2016-7051] > org.apache.commons:commons-compress:1.8.1 has a DOS vulnerability: > - [https://nvd.nist.gov/vuln/detail/CVE-2018-11771] > com.google.guava:guava:11.0.2 > -[https://nvd.nist.gov/vuln/detail/CVE-2018-10237] > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2203) avro module in python generates different bytes while writing file to local storage and s3
[ https://issues.apache.org/jira/browse/AVRO-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2203. Resolution: Cannot Reproduce Can't open the link provided. Please resubmit the issue when it still persists with Avro 1.8.2 > avro module in python generates different bytes while writing file to local > storage and s3 > --- > > Key: AVRO-2203 > URL: https://issues.apache.org/jira/browse/AVRO-2203 > Project: Apache Avro > Issue Type: Bug > Components: python >Affects Versions: 1.8.0 > Environment: S3. UNIX, HDFS, python >Reporter: Vinuthna >Priority: Blocker > > Hi, > I am trying to convert a csv file to avro format and store it on S3 storage > using python. During this process, I see that there is data loss in the file > written to s3 storage. This is confirmed by converting the avro file on local > storage and avro file on s3 storage to json format by comparing the content > and total number of lines present in each file. > A deep investigation into this issue shows that avro data generated while > writing to local storage is not exactly same as the avro data generated while > writing to s3 storage. > I suspect issue is in getting a writer object using DatumWriter. > writer = avro.datafile.DataFileWriter(, avro.io.DatumWriter(), > schema) > Exact code is present in git hub link below- > https://github.com/mpenkov/smart_open/blob/209/integration-tests/test_209.py > Could you please help solve this issue? > > Thanks > Vinuthna > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-1218) Avro 1.7.3 fails to build
[ https://issues.apache.org/jira/browse/AVRO-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-1218. Resolution: Cannot Reproduce I'm able to build Avro on MacOSX. Please reopen a new issue if it still persists with Avro 1.8.x > Avro 1.7.3 fails to build > -- > > Key: AVRO-1218 > URL: https://issues.apache.org/jira/browse/AVRO-1218 > Project: Apache Avro > Issue Type: Bug > Components: build >Affects Versions: 1.7.3 > Environment: OS X 10.8.2 >Reporter: Russell Jurney >Priority: Blocker > Labels: avro, build, for, pig, piggybank, wont > Attachments: build.log > > > I am trying to build Avro 1.7.3 from source as a workaround for issues in > PIG-3015. It does not build :( > Errors attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2259) Enable C++ tests for build using docker
[
https://issues.apache.org/jira/browse/AVRO-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AVRO-2259.
Resolution: Fixed
Fix Version/s: 1.9.0
> Enable C++ tests for build using docker
> ---
>
> Key: AVRO-2259
> URL: https://issues.apache.org/jira/browse/AVRO-2259
> Project: Apache Avro
> Issue Type: Improvement
> Components: c++
>Affects Versions: 1.8.2
> Environment: From project root, the following sequence of commands
> fails:
> {code:java}
> $ ./build.sh docker
> user@f407ae86ead3:~/avro$ cd lang/c++
> thiru@f407ae86ead3:~/avro/lang/c++$ rm -rf build
> thiru@f407ae86ead3:~/avro/lang/c++$ ./build.sh test{code}
>Reporter: Thiruvalluvan M. G.
>Assignee: Thiruvalluvan M. G.
>Priority: Major
> Fix For: 1.9.0
>
>
> Presently, C++ tests are disabled. Re-enable the tests so that developers can
> use docker for building and testing C++ binding.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (AVRO-1578) Add a link to CI
[ https://issues.apache.org/jira/browse/AVRO-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-1578. Resolution: Fixed Assignee: Fokko Driesprong Fix Version/s: 1.9.0 Travis CI has been set up, and integrated with Github. > Add a link to CI > > > Key: AVRO-1578 > URL: https://issues.apache.org/jira/browse/AVRO-1578 > Project: Apache Avro > Issue Type: Improvement > Components: doc >Affects Versions: 1.7.7 >Reporter: Stevo Slavic >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > > Please consider adding a link to CI job(s) on project site and/or wiki. > I found one Avro CI job here: https://builds.apache.org/view/All/job/AvroJava/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2233) Add CI to the repository
[ https://issues.apache.org/jira/browse/AVRO-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2233. Resolution: Duplicate > Add CI to the repository > > > Key: AVRO-2233 > URL: https://issues.apache.org/jira/browse/AVRO-2233 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Priority: Major > > Since we now have a Docker testsuite that is able to test all languages, it > would be nice to run this testsuite at every pull request, so we can be sure > that the codebase is still good, and we don't have to do it on our own > machines every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1705) Set up Jenkins job to test all languages using Docker
[ https://issues.apache.org/jira/browse/AVRO-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639117#comment-16639117 ] Fokko Driesprong commented on AVRO-1705: I would like to move to Travis (I'm open for other CI services), to run CI at every commit and PR, so we can make sure that everything is still working as it should. And as a committer we don't have to run the whole testsuite on our own machine. > Set up Jenkins job to test all languages using Docker > - > > Key: AVRO-1705 > URL: https://issues.apache.org/jira/browse/AVRO-1705 > Project: Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.7 >Reporter: Tom White >Priority: Critical > Labels: starter > > The ASF Jenkins instance now supports Docker (BUILDS-25), so we could run all > the tests (for all languages that Avro supports) using the Avro Dockerfile. > We might also do a nightly build of the whole distribution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2238) Update Docker image from java to openjdk
Fokko Driesprong created AVRO-2238: -- Summary: Update Docker image from java to openjdk Key: AVRO-2238 URL: https://issues.apache.org/jira/browse/AVRO-2238 Project: Avro Issue Type: Improvement Reporter: Fokko Driesprong Currently the docker image to run the tests is still using java which is deprecated: https://hub.docker.com/_/java/ Therefore we should move to openjdk (https://hub.docker.com/_/openjdk/). Starting with version 8, and also adding 10 and 11 to it to make sure that Avro is compatible with future version of Java. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1705) Set up CI job to test all languages using Docker
[ https://issues.apache.org/jira/browse/AVRO-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640588#comment-16640588 ] Fokko Driesprong commented on AVRO-1705: We use Travis for Apache Airflow, but I'm definitely not married to Travis. I have to look into Yetus. Thanks for providing the link to the PR. I've noticed that they are using something similar for Beam: https://github.com/apache/beam/pull/6598 > Set up CI job to test all languages using Docker > > > Key: AVRO-1705 > URL: https://issues.apache.org/jira/browse/AVRO-1705 > Project: Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.7 >Reporter: Tom White >Priority: Critical > Labels: starter > > The ASF Jenkins instance now supports Docker (BUILDS-25), so we could run all > the tests (for all languages that Avro supports) using the Avro Dockerfile. > We might also do a nightly build of the whole distribution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-1705) Set up CI job to test all languages using Docker
[ https://issues.apache.org/jira/browse/AVRO-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-1705: --- Summary: Set up CI job to test all languages using Docker (was: Set up Jenkins job to test all languages using Docker) > Set up CI job to test all languages using Docker > > > Key: AVRO-1705 > URL: https://issues.apache.org/jira/browse/AVRO-1705 > Project: Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.7 >Reporter: Tom White >Priority: Critical > Labels: starter > > The ASF Jenkins instance now supports Docker (BUILDS-25), so we could run all > the tests (for all languages that Avro supports) using the Avro Dockerfile. > We might also do a nightly build of the whole distribution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2228) Bump Apache Velocity to 2.0
Fokko Driesprong created AVRO-2228: -- Summary: Bump Apache Velocity to 2.0 Key: AVRO-2228 URL: https://issues.apache.org/jira/browse/AVRO-2228 Project: Avro Issue Type: Improvement Affects Versions: 1.8.2 Reporter: Fokko Driesprong Apache Velocity 1.7 contains CVE's: CVE-2015-6420, CVE-2017-15708 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2228) Bump Apache Velocity to 2.0
[ https://issues.apache.org/jira/browse/AVRO-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626140#comment-16626140 ] Fokko Driesprong commented on AVRO-2228: PR has been made: https://github.com/apache/avro/pull/332 > Bump Apache Velocity to 2.0 > --- > > Key: AVRO-2228 > URL: https://issues.apache.org/jira/browse/AVRO-2228 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Priority: Major > > Apache Velocity 1.7 contains CVE's: CVE-2015-6420, CVE-2017-15708 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2229) Ability to test and release using a Docker image
Fokko Driesprong created AVRO-2229: -- Summary: Ability to test and release using a Docker image Key: AVRO-2229 URL: https://issues.apache.org/jira/browse/AVRO-2229 Project: Avro Issue Type: Improvement Affects Versions: 1.8.2 Reporter: Fokko Driesprong -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2233) Add CI to the repository
Fokko Driesprong created AVRO-2233: -- Summary: Add CI to the repository Key: AVRO-2233 URL: https://issues.apache.org/jira/browse/AVRO-2233 Project: Avro Issue Type: Improvement Affects Versions: 1.8.2 Reporter: Fokko Driesprong Since we now have a Docker testsuite that is able to test all languages, it would be nice to run this testsuite at every pull request, so we can be sure that the codebase is still good, and we don't have to do it on our own machines every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2229) Ability to test using a Docker image
[ https://issues.apache.org/jira/browse/AVRO-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2229: --- Fix Version/s: 1.9.0 > Ability to test using a Docker image > > > Key: AVRO-2229 > URL: https://issues.apache.org/jira/browse/AVRO-2229 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2229) Ability to test using a Docker image
[ https://issues.apache.org/jira/browse/AVRO-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2229: --- Summary: Ability to test using a Docker image (was: Ability to test and release using a Docker image) > Ability to test using a Docker image > > > Key: AVRO-2229 > URL: https://issues.apache.org/jira/browse/AVRO-2229 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2266) Avoid m2e plugin warning
[
https://issues.apache.org/jira/browse/AVRO-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong updated AVRO-2266:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Avoid m2e plugin warning
>
>
> Key: AVRO-2266
> URL: https://issues.apache.org/jira/browse/AVRO-2266
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.8.2
>Reporter: Thiruvalluvan M. G.
>Assignee: Thiruvalluvan M. G.
>Priority: Minor
> Fix For: 1.9.0, 1.8.3
>
>
> When you build Java binding with maven you get
> {code:java}
> [WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing,
> no dependency information available
> [WARNING] Failed to retrieve plugin descriptor for
> org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin
> org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not
> be resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in
> https://repo.maven.apache.org/maven2 was cached in the local repository,
> resolution will not be reattempted until the update interval of central has
> elapsed or updates are forced{code}
> Even though the warning is not harmful, one has to constantly remind oneself
> that it is indeed harmless. It will be nice if we can get rid of the warning.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2239) Fix interop tests
[ https://issues.apache.org/jira/browse/AVRO-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2239: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.9.0 Status: Resolved (was: Patch Available) > Fix interop tests > - > > Key: AVRO-2239 > URL: https://issues.apache.org/jira/browse/AVRO-2239 > Project: Apache Avro > Issue Type: Improvement > Components: interop >Reporter: Fokko Driesprong >Assignee: Tim Perkins >Priority: Major > Fix For: 1.9.0 > > > Currently the interop tests are failing: > /avro/lang/ruby/lib/avro/schema_validator.rb:127:in `validate_map': undefined > method `keys' for nil:NilClass (NoMethodError) > from /avro/lang/ruby/lib/avro/schema_validator.rb:103:in > `validate_recursive' > from /avro/lang/ruby/lib/avro/schema_validator.rb:156:in `block in > validate_possible_types' > from /avro/lang/ruby/lib/avro/schema_validator.rb:154:in `map' > from /avro/lang/ruby/lib/avro/schema_validator.rb:154:in > `validate_possible_types' > from /avro/lang/ruby/lib/avro/schema_validator.rb:141:in > `validate_union' > from /avro/lang/ruby/lib/avro/schema_validator.rb:105:in > `validate_recursive' > from /avro/lang/ruby/lib/avro/schema_validator.rb:110:in `block in > validate_recursive' > from /avro/lang/ruby/lib/avro/schema_validator.rb:108:in `each' > from /avro/lang/ruby/lib/avro/schema_validator.rb:108:in > `validate_recursive' > from /avro/lang/ruby/lib/avro/schema_validator.rb:67:in `validate!' > from /avro/lang/ruby/lib/avro/schema.rb:97:in `validate' > from /avro/lang/ruby/lib/avro/io.rb:503:in `write_data' > from /avro/lang/ruby/lib/avro/io.rb:499:in `write' > from /avro/lang/ruby/lib/avro/ipc.rb:136:in `write_handshake_request' > from /avro/lang/ruby/lib/avro/ipc.rb:105:in `request' > from lang/ruby/test/tool.rb:69:in `send_message' > from lang/ruby/test/tool.rb:137:in `main' > from lang/ruby/test/tool.rb:143:in `' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2256) Error checking compatibility of record reader schema with union writer schema
[ https://issues.apache.org/jira/browse/AVRO-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2256. Resolution: Fixed > Error checking compatibility of record reader schema with union writer schema > - > > Key: AVRO-2256 > URL: https://issues.apache.org/jira/browse/AVRO-2256 > Project: Apache Avro > Issue Type: Bug > Components: ruby >Affects Versions: 1.9.0 >Reporter: Tim Perkins >Assignee: Tim Perkins >Priority: Critical > Fix For: 1.9.0 > > > This raises a missing method error because the comparison is invalid. > Fix: https://github.com/apache/avro/pull/368 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2290) TestSpecificLogicalTypes.testRecordWithJsr310LogicalTypes breaks on Java 11
[
https://issues.apache.org/jira/browse/AVRO-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731813#comment-16731813
]
Fokko Driesprong commented on AVRO-2290:
It shouldn't be too hard to run the CI on Java11 as well. Let me see if I can
get this working somewhere the upcoming days.
> TestSpecificLogicalTypes.testRecordWithJsr310LogicalTypes breaks on Java 11
> ---
>
> Key: AVRO-2290
> URL: https://issues.apache.org/jira/browse/AVRO-2290
> Project: Apache Avro
> Issue Type: Sub-task
> Components: java
>Reporter: Ismaël Mejía
>Priority: Minor
>
> Java 11 has nano-second precision so the test breaks, the test should now
> detect this and adapt accordingly.
> {code}
> [ERROR]
> testRecordWithJsr310LogicalTypes(org.apache.avro.specific.TestSpecificLo
> gicalTypes) Time elapsed: 0.24 s <<< FAILURE!
> java.lang.AssertionError: Should match written record expected:<{"b": true,
> "i32
> ": 34, "i64": 35, "f32": 3.14, "f64": 3019.34, "s": null, "d": 2018-12-20,
> "t":
> 15:02:53.535103, "ts": 2018-12-20T14:02:53.535127Z, "dec": 123.45}> but
> was:<{"b
> ": true, "i32": 34, "i64": 35, "f32": 3.14, "f64": 3019.34, "s": null, "d":
> 2018
> -12-20, "t": 15:02:53.535, "ts": 2018-12-20T14:02:53.535Z, "dec": 123.45}>
> at
> org.apache.avro.specific.TestSpecificLogicalTypes.testRecordWithJsr31
> 0LogicalTypes(TestSpecificLogicalTypes.java:132)
> [ERROR]
> testAbilityToReadJodaRecordWrittenAsJsr310Record(org.apache.avro.specifi
> c.TestSpecificLogicalTypes) Time elapsed: 0.005 s <<< FAILURE!
> java.lang.AssertionError:
> Expected: is "15:02:53.639158"
> but: was "15:02:53.639"
> at
> org.apache.avro.specific.TestSpecificLogicalTypes.testAbilityToReadJo
> daRecordWrittenAsJsr310Record(TestSpecificLogicalTypes.java:204)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (AVRO-2303) Test C# against .NET Core instead of Mono
[ https://issues.apache.org/jira/browse/AVRO-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong reassigned AVRO-2303: -- Assignee: Brian Lachniet > Test C# against .NET Core instead of Mono > - > > Key: AVRO-2303 > URL: https://issues.apache.org/jira/browse/AVRO-2303 > Project: Apache Avro > Issue Type: Improvement > Components: build, csharp >Reporter: Brian Lachniet >Assignee: Brian Lachniet >Priority: Major > Fix For: 1.9.0 > > > In AVRO-2112, we upgraded to the latest Mono v5+ to test the C# project in > the Travis CI builds. We should test against the .NET Core SDK instead for > the following reasons: > # .NET Core/Standard should become the default that we build and test > against because that is quickly becoming the default across the community. > # Installing the latest Mono significantly increased Travis CI build times, > as can be seen from the [build > history|https://travis-ci.org/apache/avro/builds]. Before build #317 (the one > that merges the C# updates), builds were generally under 40m. Following > builds regularly push the [50m Travis CI build > limit|https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts]. > By switching to .NET Core SDK instead, build times drop back down to the > normal before the C# updates (see [this > example|https://travis-ci.org/blachniet/avro/builds/481154272]). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2304) Constrain mysqlclient to <1.4 to maintain Python2 compatibility
Fokko Driesprong created AVRO-2304: -- Summary: Constrain mysqlclient to <1.4 to maintain Python2 compatibility Key: AVRO-2304 URL: https://issues.apache.org/jira/browse/AVRO-2304 Project: Apache Avro Issue Type: Improvement Reporter: Fokko Driesprong Assignee: Fokko Driesprong -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2285) Remove the run_duration option
Fokko Driesprong created AVRO-2285: -- Summary: Remove the run_duration option Key: AVRO-2285 URL: https://issues.apache.org/jira/browse/AVRO-2285 Project: Apache Avro Issue Type: Task Reporter: Fokko Driesprong We should not use the `run_duration` option anymore. This used to be for restarting the scheduler from time to time, but right now the scheduler is getting more stable and therefore using this setting is considered bad and might cause an inconsistent state. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2273) Release 1.8.3
[ https://issues.apache.org/jira/browse/AVRO-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706286#comment-16706286 ] Fokko Driesprong commented on AVRO-2273: I agree with you on this one Ismaël. > Release 1.8.3 > - > > Key: AVRO-2273 > URL: https://issues.apache.org/jira/browse/AVRO-2273 > Project: Apache Avro > Issue Type: Task >Reporter: Thiruvalluvan M. G. >Priority: Major > Fix For: 1.8.3 > > > This ticket is for releasing Avro 1.8.3 and discussing any topics related to > it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2192) Remove paranamer dependency from Avro
[ https://issues.apache.org/jira/browse/AVRO-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2192. Resolution: Fixed > Remove paranamer dependency from Avro > - > > Key: AVRO-2192 > URL: https://issues.apache.org/jira/browse/AVRO-2192 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.8.2 >Reporter: RAJKUMAR NATARAJAN >Assignee: Ismaël Mejía >Priority: Minor > Fix For: 1.9.0 > > > Hi, > > Avro seems to be depend upon the com.thoughtworks.paranamer. Avro 1.7.7 > depends upon 2.7 and 1.8. depends on 2.8 paranamer versions. > > Please remove this dependency in the future versions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2250) Release 1.9.0
[ https://issues.apache.org/jira/browse/AVRO-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698177#comment-16698177 ] Fokko Driesprong commented on AVRO-2250: Thanks, Evan. Please keep in mind that Guava was only used in the tests, so there is no real security risk. Please refer to the Pull Request for more details: https://github.com/apache/avro/pull/373. Hopefully this won't block you from going to production :-) > Release 1.9.0 > - > > Key: AVRO-2250 > URL: https://issues.apache.org/jira/browse/AVRO-2250 > Project: Apache Avro > Issue Type: Task >Reporter: Nandor Kollar >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2267) Duplicate code RandomData.java and its dependency problem
[
https://issues.apache.org/jira/browse/AVRO-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AVRO-2267.
Resolution: Fixed
Fix Version/s: 1.9.0
> Duplicate code RandomData.java and its dependency problem
> -
>
> Key: AVRO-2267
> URL: https://issues.apache.org/jira/browse/AVRO-2267
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: Thiruvalluvan M. G.
>Assignee: Thiruvalluvan M. G.
>Priority: Major
> Fix For: 1.9.0
>
>
> There are two issues with {{RandomData}} class:
> * There are almost identical copies of the same code in two modules:
> {{avro}} and {{avro-ipc}}. We should use a single source file.
> * Both the copies belong to {{test}} subfolders in their respective module.
> But {{avro-tools}} module uses this class in {{main}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2039) Ruby encoding performance improvement
[ https://issues.apache.org/jira/browse/AVRO-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2039: --- Resolution: Fixed Fix Version/s: 1.9.0 Status: Resolved (was: Patch Available) > Ruby encoding performance improvement > - > > Key: AVRO-2039 > URL: https://issues.apache.org/jira/browse/AVRO-2039 > Project: Apache Avro > Issue Type: Improvement > Components: ruby >Reporter: Tim Perkins >Assignee: Tim Perkins >Priority: Critical > Fix For: 1.9.0 > > > For a use case with a few levels of nesting and unions in several places > within the schema we saw a 5x improvement in encoding performance with these > changes to encoding using Ruby. > 1. Avoid the exhaustive validation of schemas in a union > 2. Avoid the repeated validation of nested schemas > https://github.com/apache/avro/pull/230 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2241) Fix Jsr310/Joda tests when comparing datetimes
[
https://issues.apache.org/jira/browse/AVRO-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AVRO-2241.
Resolution: Fixed
> Fix Jsr310/Joda tests when comparing datetimes
> --
>
> Key: AVRO-2241
> URL: https://issues.apache.org/jira/browse/AVRO-2241
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.8.2
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.9.0
>
>
> Whe comparing Instant, DateTime, LocalTime with its Joda equivalents the
> tests fail since the java.time.* adds support for microseconds, and Joda only
> miliseconds. For example:
> Failed tests:
>
> TestSpecificLogicalTypes.testAbilityToReadJodaRecordWrittenAsJsr310Record:183
> Expected: is "11:45:07.170331"
> but: was "11:45:07.170"
> TestSpecificLogicalTypes.testRecordWithJsr310LogicalTypes:115 Should match
> written record expected:<{"b": true, "i32": 34, "i64": 35, "f32": 3.14,
> "f64": 3019.34, "s": null, "d": 2018-10-11, "t": 11:45:06.513252, "ts":
> 2018-10-11T11:45:06.513358Z, "dec": 123.45}> but was:<{"b": true, "i32": 34,
> "i64": 35, "f32": 3.14, "f64": 3019.34, "s": null, "d": 2018-10-11, "t":
> 11:45:06.513, "ts": 2018-10-11T11:45:06.513Z, "dec": 123.45}>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2244) Problems with TestSpecificLogicalTypes.testAbilityToReadJsr310RecordWrittenAsJodaRecord:148
[
https://issues.apache.org/jira/browse/AVRO-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693045#comment-16693045
]
Fokko Driesprong commented on AVRO-2244:
I fixed this over here:
https://github.com/apache/avro/commit/25032e10a86851bdabfa29b99885ccfc03c6667e#diff-a2bcc0c4e92dfb4332f75a9d56cdb4daR73
> Problems with
> TestSpecificLogicalTypes.testAbilityToReadJsr310RecordWrittenAsJodaRecord:148
> ---
>
> Key: AVRO-2244
> URL: https://issues.apache.org/jira/browse/AVRO-2244
> Project: Apache Avro
> Issue Type: Bug
> Components: logical types
>Reporter: Raymie Stata
>Assignee: Thiruvalluvan M. G.
>Priority: Major
>
> I've seen an intermittent test failure that looks like this:
> {{Failed tests:}}
> {{
> TestSpecificLogicalTypes.testAbilityToReadJsr310RecordWrittenAsJodaRecord:148}}
> {{Expected: is "20:35:18.720"}}
> {{ but: was "20:35:18.72"}}
> When I see this failure, it's always the case that the trailing digit is
> zero. I suspect that it's a bug where the trailing zero is not printed.
> Since the test cases use the current time, then most of the time the trailing
> digit isn't zero and the bug isn't tickled. But once-in-a-while the current
> time has a trailing zero, which tickles the bug.
> If this diagnosis is correct, then in addition to fixing the bug, it might be
> a good idea to add tests with hard-wired, static times that cover corner
> cases like this one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (AVRO-2244) Problems with TestSpecificLogicalTypes.testAbilityToReadJsr310RecordWrittenAsJodaRecord:148
[
https://issues.apache.org/jira/browse/AVRO-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AVRO-2244.
Resolution: Fixed
Fix Version/s: 1.9.0
> Problems with
> TestSpecificLogicalTypes.testAbilityToReadJsr310RecordWrittenAsJodaRecord:148
> ---
>
> Key: AVRO-2244
> URL: https://issues.apache.org/jira/browse/AVRO-2244
> Project: Apache Avro
> Issue Type: Bug
> Components: logical types
>Reporter: Raymie Stata
>Assignee: Thiruvalluvan M. G.
>Priority: Major
> Fix For: 1.9.0
>
>
> I've seen an intermittent test failure that looks like this:
> {{Failed tests:}}
> {{
> TestSpecificLogicalTypes.testAbilityToReadJsr310RecordWrittenAsJodaRecord:148}}
> {{Expected: is "20:35:18.720"}}
> {{ but: was "20:35:18.72"}}
> When I see this failure, it's always the case that the trailing digit is
> zero. I suspect that it's a bug where the trailing zero is not printed.
> Since the test cases use the current time, then most of the time the trailing
> digit isn't zero and the bug isn't tickled. But once-in-a-while the current
> time has a trailing zero, which tickles the bug.
> If this diagnosis is correct, then in addition to fixing the bug, it might be
> a good idea to add tests with hard-wired, static times that cover corner
> cases like this one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2297) Ruby build fails in docker container
[ https://issues.apache.org/jira/browse/AVRO-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736384#comment-16736384 ] Fokko Driesprong commented on AVRO-2297: Yes, thanks [~tjwp] > Ruby build fails in docker container > > > Key: AVRO-2297 > URL: https://issues.apache.org/jira/browse/AVRO-2297 > Project: Apache Avro > Issue Type: Improvement > Components: ruby >Reporter: Thiruvalluvan M. G. >Assignee: Tim Perkins >Priority: Major > > It appears that the latest release of Ruby bundler on Jan 4, 2019 breaks our > build. We are installing the bundler without version constraints and hence it > installs the latest version of bundler, which in turn requires ruby 2.3.0. > Can someone familiar with our Ruby binding take care of this. Until it is > fixed, we'll disable ruby build in docker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2297) Ruby build fails in docker container
[ https://issues.apache.org/jira/browse/AVRO-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2297. Resolution: Fixed Fix Version/s: 1.9.0 > Ruby build fails in docker container > > > Key: AVRO-2297 > URL: https://issues.apache.org/jira/browse/AVRO-2297 > Project: Apache Avro > Issue Type: Improvement > Components: ruby >Reporter: Thiruvalluvan M. G. >Assignee: Tim Perkins >Priority: Major > Fix For: 1.9.0 > > > It appears that the latest release of Ruby bundler on Jan 4, 2019 breaks our > build. We are installing the bundler without version constraints and hence it > installs the latest version of bundler, which in turn requires ruby 2.3.0. > Can someone familiar with our Ruby binding take care of this. Until it is > fixed, we'll disable ruby build in docker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2112) c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies
[ https://issues.apache.org/jira/browse/AVRO-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2112: --- Resolution: Fixed Fix Version/s: 1.9.0 Status: Resolved (was: Patch Available) > c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies > > > Key: AVRO-2112 > URL: https://issues.apache.org/jira/browse/AVRO-2112 > Project: Apache Avro > Issue Type: Improvement > Components: csharp > Environment: - Visual Studio For Mac > - Visual Studio 2017 > - mono 5.4.1.7 MacOSX > - dotnet 2.0 (MacOSX and Windows 10) >Reporter: Miljenko Cvjetko >Assignee: Miljenko Cvjetko >Priority: Minor > Fix For: 1.9.0 > > Original Estimate: 0h > Remaining Estimate: 0h > > Suugestion is to add .NET Standard/Core support. > In order to support modern/new .NET (both standard netfx and dotnet core) it > is necessary to convert projects to support .NET Standard Libraries. > - conversion to .NET Standard [DONE] > - added dotnet core sample (Avro.perf) [DONE] > - added netfx (standrd .NET) sample Avro.perf.netfx [DONE] > - unit testing updated to use NUnit 3 > Url for the github forked repo with branch will be added. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-2238) Update Docker image from java to openjdk
[ https://issues.apache.org/jira/browse/AVRO-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong reassigned AVRO-2238: -- Assignee: Fokko Driesprong > Update Docker image from java to openjdk > > > Key: AVRO-2238 > URL: https://issues.apache.org/jira/browse/AVRO-2238 > Project: Apache Avro > Issue Type: Improvement > Components: docker >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > > Currently the docker image to run the tests is still using java which is > deprecated: https://hub.docker.com/_/java/ > Therefore we should move to openjdk (https://hub.docker.com/_/openjdk/). > Starting with version 8, and also adding 10 and 11 to it to make sure that > Avro is compatible with future version of Java. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2250) Release 1.9.0
[ https://issues.apache.org/jira/browse/AVRO-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807589#comment-16807589 ] Fokko Driesprong commented on AVRO-2250: [~mariano.achaval] Please subscribe to the dev list, we will cut 1.9 soon. > Release 1.9.0 > - > > Key: AVRO-2250 > URL: https://issues.apache.org/jira/browse/AVRO-2250 > Project: Apache Avro > Issue Type: Task > Components: release >Reporter: Nandor Kollar >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-2200) Option to fail when extra fields are in the payload
[ https://issues.apache.org/jira/browse/AVRO-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong reassigned AVRO-2200: -- Assignee: Fokko Driesprong > Option to fail when extra fields are in the payload > --- > > Key: AVRO-2200 > URL: https://issues.apache.org/jira/browse/AVRO-2200 > Project: Apache Avro > Issue Type: Improvement > Components: ruby >Reporter: Daniel Orner >Assignee: Fokko Driesprong >Priority: Minor > > Currently, when passing a datum into the schema validator, the datum will > pass even if the datum has more fields than the schema does. This can be > dangerous because the client code is assuming that the schema has these > values, but they are actually being silently swallowed when they are encoded. > I'd like to open a PR where we can provide an option to make the validator > fail on any extra fields. See https://github.com/salsify/avro-patches/pull/15 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2200) Option to fail when extra fields are in the payload
[ https://issues.apache.org/jira/browse/AVRO-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2200. Resolution: Fixed Fix Version/s: 1.9.0 > Option to fail when extra fields are in the payload > --- > > Key: AVRO-2200 > URL: https://issues.apache.org/jira/browse/AVRO-2200 > Project: Apache Avro > Issue Type: Improvement > Components: ruby >Reporter: Daniel Orner >Assignee: Tim Perkins >Priority: Minor > Fix For: 1.9.0 > > > Currently, when passing a datum into the schema validator, the datum will > pass even if the datum has more fields than the schema does. This can be > dangerous because the client code is assuming that the schema has these > values, but they are actually being silently swallowed when they are encoded. > I'd like to open a PR where we can provide an option to make the validator > fail on any extra fields. See https://github.com/salsify/avro-patches/pull/15 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-2200) Option to fail when extra fields are in the payload
[ https://issues.apache.org/jira/browse/AVRO-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong reassigned AVRO-2200: -- Assignee: Tim Perkins (was: Fokko Driesprong) > Option to fail when extra fields are in the payload > --- > > Key: AVRO-2200 > URL: https://issues.apache.org/jira/browse/AVRO-2200 > Project: Apache Avro > Issue Type: Improvement > Components: ruby >Reporter: Daniel Orner >Assignee: Tim Perkins >Priority: Minor > > Currently, when passing a datum into the schema validator, the datum will > pass even if the datum has more fields than the schema does. This can be > dangerous because the client code is assuming that the schema has these > values, but they are actually being silently swallowed when they are encoded. > I'd like to open a PR where we can provide an option to make the validator > fail on any extra fields. See https://github.com/salsify/avro-patches/pull/15 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-1810) GenericDatumWriter broken with Enum
[
https://issues.apache.org/jira/browse/AVRO-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong updated AVRO-1810:
---
Fix Version/s: (was: 1.8.4)
> GenericDatumWriter broken with Enum
> ---
>
> Key: AVRO-1810
> URL: https://issues.apache.org/jira/browse/AVRO-1810
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.0
>Reporter: Ryon Day
>Assignee: Fokko Driesprong
>Priority: Blocker
> Fix For: 1.9.0
>
>
> {panel:title=Description|titleBGColor=#3FA|bgColor=#DDD}
> Using the GenericDatumWriter with either Generic OR SpecificRecord will break
> if an Enum is present.
> {panel}
> {panel:title=Steps To Reproduce|titleBGColor=#8DB|bgColor=#DDD}
> I have been tracking Avro decoding oddities for a while.
> The tests for this issue can be found
> [here|https://github.com/ryonday/avroDecodingHelp/blob/master/src/test/java/com/ryonday/test/Avro180EnumFail.java]
> {panel}
> {panel:title=Notes|titleBGColor=#3AF|bgColor=#DDD}
> Due to the debacle that is the Avro "UTF8" object, we have been avoiding it
> by using the following scheme:
> * Write incoming records to a byte array using the GenericDatumWriter
> * Read back the byte array to our compiled Java domain objects using a
> SpecificDatumWriter
> This worked great with Avro 1.7.7, and this is a binary-incompatable breaking
> change with 1.8.0.
> This would appear to be caused by an addition in the
> {{GenericDatumWriter:163-164}}:
> {code}
> if (!data.isEnum(datum))
> throw new AvroTypeException("Not an enum: "+datum);
> {code}
> {panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2299) Get Plain Schema
[
https://issues.apache.org/jira/browse/AVRO-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong updated AVRO-2299:
---
Fix Version/s: (was: 1.8.4)
(was: 1.8.3)
(was: 1.8.2)
(was: 1.9.0)
1.10.0
> Get Plain Schema
>
>
> Key: AVRO-2299
> URL: https://issues.apache.org/jira/browse/AVRO-2299
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.8.2
>Reporter: Rumeshkrishnan
>Priority: Minor
> Labels: features
> Fix For: 1.10.0
>
>
> {panel:title=Avro Schema Reserved Keys:}
> "doc", "fields", "items", "name", "namespace",
> "size", "symbols", "values", "type", "aliases", "default"
> {panel}
> AVRO also supports user defined properties for both Schema and Field.
> Is there way to get the schema with reserved property (key, value)?
> Input Schema:
> {code:java}
> {
> "name": "testSchema",
> "namespace": "com.avro",
> "type": "record",
> "fields": [
> {
> "name": "email",
> "type": "string",
> "doc": "email id",
> "user_field_prop": "x"
> }
> ],
> "user_schema_prop": "xx"
> }{code}
> Expected Plain Schema:
> {code:java}
> {
> "name": "testSchema",
> "namespace": "com.avro",
> "type": "record",
> "fields": [
> {
> "name": "email",
> "type": "string",
> "doc": "email id"
> }
> ]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-1132) Build failing on MSYS/MinGW due to missing struct iovec
[ https://issues.apache.org/jira/browse/AVRO-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-1132: --- Fix Version/s: (was: 1.8.4) 1.9.0 > Build failing on MSYS/MinGW due to missing struct iovec > --- > > Key: AVRO-1132 > URL: https://issues.apache.org/jira/browse/AVRO-1132 > Project: Apache Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.7.1 > Environment: Windows XP Professional 32-bit SP3, MSYS, MinGW GCC 4.5.1 >Reporter: Laurent Moss >Assignee: Laurent Moss >Priority: Major > Labels: build > Fix For: 1.9.0 > > Attachments: AVRO-1132.diff > > > Avro C++ fails to build on MSYS with MinGW GCC due to references to > undeclared struct iovec: > In file included from C:/workspace/avro-cpp/api/buffer/BufferReader.hh:22:0, > from C:/workspace/avro-cpp/api/Reader.hh:30, > from C:/workspace/avro-cpp/api/ResolverSchema.hh:28, > from c:/workspace/avro-cpp/impl/ResolverSchema.cc:20: > C:/workspace/avro-cpp/api/buffer/Buffer.hh: In function 'void > avro::toIovec(BufferType&, std::vector&)': > C:/workspace/avro-cpp/api/buffer/Buffer.hh:517:15: error: invalid use of > incomplete type 'struct avro::iovec' > C:/workspace/avro-cpp/api/buffer/Buffer.hh:511:57: error: forward declaration > of 'struct avro::iovec' > C:/workspace/avro-cpp/api/buffer/Buffer.hh:518:15: error: invalid use of > incomplete type 'struct avro::iovec' > C:/workspace/avro-cpp/api/buffer/Buffer.hh:511:57: error: forward declaration > of 'struct avro::iovec' > make[2]: *** [CMakeFiles/avrocpp_s.dir/impl/ResolverSchema.cc.obj] Error 1 > make[1]: *** [CMakeFiles/avrocpp_s.dir/all] Error 2 > make: *** [all] Error 2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2249) Fix the C++ tests
[
https://issues.apache.org/jira/browse/AVRO-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong updated AVRO-2249:
---
Fix Version/s: (was: 1.8.4)
1.9.0
> Fix the C++ tests
> -
>
> Key: AVRO-2249
> URL: https://issues.apache.org/jira/browse/AVRO-2249
> Project: Apache Avro
> Issue Type: Task
> Components: c++
>Affects Versions: 1.8.2
>Reporter: Fokko Driesprong
>Assignee: Thiruvalluvan M. G.
>Priority: Major
> Fix For: 1.9.0
>
>
> Currently the C++ tests are failing:
> ```
> Scanning dependencies of target avrocpp_s
> [ 1%] Building CXX object CMakeFiles/avrocpp_s.dir/impl/Compiler.cc.o
> /testptch/unknown/lang/c++/impl/Compiler.cc:174:15: warning: ‘std::string
> avro::nameof(const NodePtr&)’ defined but not used [-Wunused-function]
> static string nameof(const NodePtr& n)
>^
> [ 2%] Building CXX object CMakeFiles/avrocpp_s.dir/impl/Node.cc.o
> [ 4%] Building CXX object CMakeFiles/avrocpp_s.dir/impl/NodeImpl.cc.o
> /testptch/unknown/lang/c++/impl/NodeImpl.cc: In function ‘std::string
> avro::{anonymous}::escape(const string&)’:
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:31:8: warning: ‘auto’ changes
> meaning in C++11; please remove it [-Wc++0x-compat]
>for (auto c : unescaped) {
> ^
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:31:13: error: ‘c’ does not
> name a type
>for (auto c : unescaped) {
> ^
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected ‘;’
> before ‘return’
>return s;
>^
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected
> primary-expression before ‘return’
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected ‘;’
> before ‘return’
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected
> primary-expression before ‘return’
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:67:3: error: expected ‘)’
> before ‘return’
> In file included from /usr/include/boost/assert.hpp:51:0,
> from /usr/include/boost/format/parsing.hpp:21,
> from /usr/include/boost/format.hpp:50,
> from /testptch/unknown/lang/c++/api/Exception.hh:24,
> from /testptch/unknown/lang/c++/api/Node.hh:28,
> from /testptch/unknown/lang/c++/api/GenericDatum.hh:29,
> from /testptch/unknown/lang/c++/api/NodeImpl.hh:23,
> from /testptch/unknown/lang/c++/impl/NodeImpl.cc:21:
> /testptch/unknown/lang/c++/impl/NodeImpl.cc: In member function ‘virtual
> void avro::NodeRecord::printJson(std::ostream&, int) const’:
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:247:59: warning: comparison
> between signed and unsigned integer expressions [-Wsign-compare]
> assert(defaultValues.empty() || (defaultValues.size() == fields));
>^
> /testptch/unknown/lang/c++/impl/NodeImpl.cc: In member function ‘virtual
> void avro::NodePrimitive::printDefaultToJson(const avro::GenericDatum&,
> std::ostream&, int) const’:
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:292:13: error: ‘to_string’ is
> not a member of ‘std’
>os << std::to_string(g.value());
> ^
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:295:13: error: ‘to_string’ is
> not a member of ‘std’
>os << std::to_string(g.value());
> ^
> /testptch/unknown/lang/c++/impl/NodeImpl.cc: In function ‘std::string
> avro::{anonymous}::escape(const string&)’:
> /testptch/unknown/lang/c++/impl/NodeImpl.cc:68:1: warning: control reaches
> end of non-void function [-Wreturn-type]
> }
> ^
> CMakeFiles/avrocpp_s.dir/build.make:100: recipe for target
> 'CMakeFiles/avrocpp_s.dir/impl/NodeImpl.cc.o' failed
> make[2]: *** [CMakeFiles/avrocpp_s.dir/impl/NodeImpl.cc.o] Error 1
> make[1]: *** [CMakeFiles/avrocpp_s.dir/all] Error 2
> CMakeFiles/Makefile2:425: recipe for target 'CMakeFiles/avrocpp_s.dir/all'
> failed
> make: *** [all] Error 2
> Makefile:147: recipe for target 'all' failed
> ```
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (AVRO-2335) Remove Joda Time Library
[
https://issues.apache.org/jira/browse/AVRO-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong reassigned AVRO-2335:
--
Assignee: Fokko Driesprong
> Remove Joda Time Library
>
>
> Key: AVRO-2335
> URL: https://issues.apache.org/jira/browse/AVRO-2335
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: David Mollitor
>Assignee: Fokko Driesprong
>Priority: Major
>
> Since [AVRO-2079] was completed, Avro supports Java's standard {{java.time}}
> libraries. Please remove the Joda Time implementations and the Joda Time
> dependency from Avro. Avro 2.0 perhaps?
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. *Users are now asked to migrate to java.time (JSR-310).*
> https://www.joda.org/joda-time/
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2360) Java 8 timestamp-millis / timestamp-micros type inconsistency
[ https://issues.apache.org/jira/browse/AVRO-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804921#comment-16804921 ] Fokko Driesprong commented on AVRO-2360: Thanks for giving 1.9-SNAPSHOT a try. We need to get this fixed before we can ship it. > Java 8 timestamp-millis / timestamp-micros type inconsistency > - > > Key: AVRO-2360 > URL: https://issues.apache.org/jira/browse/AVRO-2360 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.9.0 > Environment: JDK 11.0.2 on Linux >Reporter: Raman Gupta >Assignee: Daniel Kulp >Priority: Blocker > Fix For: 1.9.0 > > > I am trying the new JSR 310 date/time types with a snapshot of 1.9.0. > I see what seems to be an inconsistency. If I generate my code with a logical > type of "timestamp-millis", then the code is generated with `Instant`, as > expected. However, on JDK11 on Linux, if I do `Instant.now()` the Instant > value created contains microseconds. When setting this Instant on an instance > of the generated Avro SpecificRecord, I am unable to round-trip the data: > Before serialization to bytes: > System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665308Z > After serializing to bytes, and then back into a specific record the > microseconds are now truncated: > System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665Z > I believe the compiler should generate a class that truncates the > microseconds at `setter` time for "timestamp-millis", so that the value > before serialization, and after deserialization, are the same. This can be > done with a call to the method `truncatedTo(ChronoUnit.MILLIS)`. > Another, possibly related, oddity is that the "timestamp-micros" type > generates a class with `long` as the type of the field. Since Instant can > support both milli and micro-second precision, I don't see the reason for > this behavior. > Followup on AVRO-2079. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2199) Validate that field defaults have the correct type
[ https://issues.apache.org/jira/browse/AVRO-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2199: --- Resolution: Fixed Assignee: Tim Perkins Status: Resolved (was: Patch Available) > Validate that field defaults have the correct type > --- > > Key: AVRO-2199 > URL: https://issues.apache.org/jira/browse/AVRO-2199 > Project: Apache Avro > Issue Type: Improvement > Components: ruby >Affects Versions: 1.8.2 >Reporter: Daniel Orner >Assignee: Tim Perkins >Priority: Minor > Fix For: 1.9.0 > > Attachments: AVRO-2199.patch > > > Currently, on the master branch, when a schema is parsed, it is possible to > define a field with a type and a default of a totally different type. E.g. if > the field has type "string", the default can be set to "null". > I'd like to open a PR which will fix this by running the default through the > SchemaValidator whenever a new Field is created. See > [https://github.com/salsify/avro-patches/pull/16] > cc: [~tjwp] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2335) Remove Joda Time Library
[
https://issues.apache.org/jira/browse/AVRO-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong updated AVRO-2335:
---
Fix Version/s: 1.10.0
> Remove Joda Time Library
>
>
> Key: AVRO-2335
> URL: https://issues.apache.org/jira/browse/AVRO-2335
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: David Mollitor
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.10.0
>
>
> Since [AVRO-2079] was completed, Avro supports Java's standard {{java.time}}
> libraries. Please remove the Joda Time implementations and the Joda Time
> dependency from Avro. Avro 2.0 perhaps?
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. *Users are now asked to migrate to java.time (JSR-310).*
> https://www.joda.org/joda-time/
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (AVRO-2335) Remove Joda Time Library
[
https://issues.apache.org/jira/browse/AVRO-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on AVRO-2335 started by Fokko Driesprong.
--
> Remove Joda Time Library
>
>
> Key: AVRO-2335
> URL: https://issues.apache.org/jira/browse/AVRO-2335
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: David Mollitor
>Assignee: Fokko Driesprong
>Priority: Major
>
> Since [AVRO-2079] was completed, Avro supports Java's standard {{java.time}}
> libraries. Please remove the Joda Time implementations and the Joda Time
> dependency from Avro. Avro 2.0 perhaps?
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. *Users are now asked to migrate to java.time (JSR-310).*
> https://www.joda.org/joda-time/
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (AVRO-2326) Bump netty-codec-http2 from 4.1.30.Final to 4.1.33.Final
[ https://issues.apache.org/jira/browse/AVRO-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2326. Resolution: Fixed > Bump netty-codec-http2 from 4.1.30.Final to 4.1.33.Final > > > Key: AVRO-2326 > URL: https://issues.apache.org/jira/browse/AVRO-2326 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2325) Bump commons-codec from 1.11 to 1.12
[ https://issues.apache.org/jira/browse/AVRO-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2325. Resolution: Fixed > Bump commons-codec from 1.11 to 1.12 > > > Key: AVRO-2325 > URL: https://issues.apache.org/jira/browse/AVRO-2325 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2212) GenericData.deepCopyRaw not working with java.lang.BigDecimal
[
https://issues.apache.org/jira/browse/AVRO-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AVRO-2212.
Resolution: Fixed
Fix Version/s: 1.8.2
> GenericData.deepCopyRaw not working with java.lang.BigDecimal
> -
>
> Key: AVRO-2212
> URL: https://issues.apache.org/jira/browse/AVRO-2212
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
>Reporter: Sean Policarpio
>Priority: Critical
> Fix For: 1.8.2
>
> Attachments: SomethingWithDec.java
>
>
> With an AVDL of:
> {code:java}
> @namespace("com.foo")
> record SomethingWithDec {
> decimal(8,2) dec;
> }
> {code}
> and the Scala/Java code of:
> {code}
> object Foo extends App {
> import com.foo.SomethingWithDec
> val x = SomethingWithDec.newBuilder().setDec(new
> java.math.BigDecimal("1234"))
> val y = SomethingWithDec.newBuilder(x).build()
> }
> {code}
> I get the exception:
> {noformat}
> Exception in thread "main" java.lang.ClassCastException: java.math.BigDecimal
> cannot be cast to java.nio.ByteBuffer
> at
> org.apache.avro.generic.GenericData.deepCopyRaw(GenericData.java:1089)
> at org.apache.avro.generic.GenericData.deepCopy(GenericData.java:1065)
> at com.foo.SomethingWithDec$Builder.(SomethingWithDec.java:167)
> at com.foo.SomethingWithDec$Builder.(SomethingWithDec.java:150)
> at com.foo.SomethingWithDec.newBuilder(SomethingWithDec.java:135)
> {noformat}
> I have attached the generated Java class for {{SomethingWithDec.}}
> This seems related to AVRO-1895, but I believe that was meant to have been
> fixed in 1.8.2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (AVRO-1810) GenericDatumWriter broken with Enum
[
https://issues.apache.org/jira/browse/AVRO-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AVRO-1810.
Resolution: Fixed
Assignee: Fokko Driesprong
> GenericDatumWriter broken with Enum
> ---
>
> Key: AVRO-1810
> URL: https://issues.apache.org/jira/browse/AVRO-1810
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.0
>Reporter: Ryon Day
>Assignee: Fokko Driesprong
>Priority: Blocker
> Fix For: 1.9.0, 1.8.4
>
>
> {panel:title=Description|titleBGColor=#3FA|bgColor=#DDD}
> Using the GenericDatumWriter with either Generic OR SpecificRecord will break
> if an Enum is present.
> {panel}
> {panel:title=Steps To Reproduce|titleBGColor=#8DB|bgColor=#DDD}
> I have been tracking Avro decoding oddities for a while.
> The tests for this issue can be found
> [here|https://github.com/ryonday/avroDecodingHelp/blob/master/src/test/java/com/ryonday/test/Avro180EnumFail.java]
> {panel}
> {panel:title=Notes|titleBGColor=#3AF|bgColor=#DDD}
> Due to the debacle that is the Avro "UTF8" object, we have been avoiding it
> by using the following scheme:
> * Write incoming records to a byte array using the GenericDatumWriter
> * Read back the byte array to our compiled Java domain objects using a
> SpecificDatumWriter
> This worked great with Avro 1.7.7, and this is a binary-incompatable breaking
> change with 1.8.0.
> This would appear to be caused by an addition in the
> {{GenericDatumWriter:163-164}}:
> {code}
> if (!data.isEnum(datum))
> throw new AvroTypeException("Not an enum: "+datum);
> {code}
> {panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (AVRO-2331) Remove commons-codec dependency
[ https://issues.apache.org/jira/browse/AVRO-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2331. Resolution: Fixed > Remove commons-codec dependency > --- > > Key: AVRO-2331 > URL: https://issues.apache.org/jira/browse/AVRO-2331 > Project: Apache Avro > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 1.9.0 > > > Use [JDK > Base64|https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html] > instead of Apache Codec. > Will allow for removal of Apache Commons Codec dependency. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2316) Update ant to version 1.10.5
[ https://issues.apache.org/jira/browse/AVRO-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2316: --- Fix Version/s: 1.9.0 > Update ant to version 1.10.5 > > > Key: AVRO-2316 > URL: https://issues.apache.org/jira/browse/AVRO-2316 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.9.0 >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2318) Bump Hadoop to 3.x
Fokko Driesprong created AVRO-2318: -- Summary: Bump Hadoop to 3.x Key: AVRO-2318 URL: https://issues.apache.org/jira/browse/AVRO-2318 Project: Apache Avro Issue Type: Improvement Components: java Affects Versions: 1.8.2 Reporter: Fokko Driesprong Assignee: Fokko Driesprong Fix For: 1.9.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2319) Bump grpc.version from 1.17.1 to 1.18.0
Fokko Driesprong created AVRO-2319: -- Summary: Bump grpc.version from 1.17.1 to 1.18.0 Key: AVRO-2319 URL: https://issues.apache.org/jira/browse/AVRO-2319 Project: Apache Avro Issue Type: Improvement Components: java Affects Versions: 1.8.2 Reporter: Fokko Driesprong Assignee: Fokko Driesprong Fix For: 1.9.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2317) Update hamcrest-library to version 2.1 and remove hamcrest-all
[ https://issues.apache.org/jira/browse/AVRO-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2317. Resolution: Fixed > Update hamcrest-library to version 2.1 and remove hamcrest-all > -- > > Key: AVRO-2317 > URL: https://issues.apache.org/jira/browse/AVRO-2317 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.9.0 >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2316) Update ant to version 1.10.5
[ https://issues.apache.org/jira/browse/AVRO-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2316. Resolution: Fixed > Update ant to version 1.10.5 > > > Key: AVRO-2316 > URL: https://issues.apache.org/jira/browse/AVRO-2316 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.9.0 >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2315) Update zstd-jni to version 1.3.8-3
[ https://issues.apache.org/jira/browse/AVRO-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2315: --- Fix Version/s: 1.9.0 > Update zstd-jni to version 1.3.8-3 > -- > > Key: AVRO-2315 > URL: https://issues.apache.org/jira/browse/AVRO-2315 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.9.0 >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-2315) Update zstd-jni to version 1.3.8-3
[ https://issues.apache.org/jira/browse/AVRO-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AVRO-2315. Resolution: Fixed > Update zstd-jni to version 1.3.8-3 > -- > > Key: AVRO-2315 > URL: https://issues.apache.org/jira/browse/AVRO-2315 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.9.0 >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2317) Update hamcrest-library to version 2.1 and remove hamcrest-all
[ https://issues.apache.org/jira/browse/AVRO-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-2317: --- Fix Version/s: 1.9.0 > Update hamcrest-library to version 2.1 and remove hamcrest-all > -- > > Key: AVRO-2317 > URL: https://issues.apache.org/jira/browse/AVRO-2317 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.9.0 >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-1856) Update ConcatTool to support appending to an input file instead of creating a new output file
[ https://issues.apache.org/jira/browse/AVRO-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-1856: --- Fix Version/s: (was: 1.9.0) 1.10.0 > Update ConcatTool to support appending to an input file instead of creating a > new output file > - > > Key: AVRO-1856 > URL: https://issues.apache.org/jira/browse/AVRO-1856 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.7, 1.8.1 >Reporter: Mike Hurley >Assignee: Mike Hurley >Priority: Major > Fix For: 1.10.0 > > Attachments: AVRO-1856-ConcatTool-AppendToInput.diff > > > It would be nice to have ConcatTool be able to append other input files into > one of the input files instead of creating a new output file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-1284) Python: validation should be a method of Schema objects
[
https://issues.apache.org/jira/browse/AVRO-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong updated AVRO-1284:
---
Fix Version/s: (was: 1.9.0)
1.10.0
> Python: validation should be a method of Schema objects
> ---
>
> Key: AVRO-1284
> URL: https://issues.apache.org/jira/browse/AVRO-1284
> Project: Apache Avro
> Issue Type: Improvement
> Components: python
>Reporter: Jeremy Kahn
>Assignee: Jeremy Kahn
>Priority: Minor
> Labels: patch
> Fix For: 1.10.0
>
> Attachments: validation-as-method-backwards-compatible.patch,
> validation-as-method.patch
>
>
> In Python, validation of a datum by the schema was done in
> {{avro.io.validate}} function.
> The {{avro.io.validate}} function is a complex, recursively-called switch
> statement.
> Instead of calling a two-argument {{avro.io.validate}} with a Schema object
> and a datum, it is easier to understand and extend if they are one-argument
> methods on the schema.
> I (Jeremy) have written a patch that implements {{validate}} methods on
> Schema objects. This patch will form the prerequisite for AVRO-1265 (see
> "easier to extend" above).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-1265) Python: schema objects should support builder() default-filling behavior
[ https://issues.apache.org/jira/browse/AVRO-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AVRO-1265: --- Fix Version/s: (was: 1.9.0) 1.10.0 > Python: schema objects should support builder() default-filling behavior > > > Key: AVRO-1265 > URL: https://issues.apache.org/jira/browse/AVRO-1265 > Project: Apache Avro > Issue Type: Improvement > Components: python >Reporter: Jeremy Kahn >Assignee: Jeremy Kahn >Priority: Minor > Labels: features > Fix For: 1.10.0 > > Attachments: avro-1265a-build-defaults.patch, avro-1265b-tests.patch > > > There seems to be no way to easily use the avro libraries in Python (where I > feel most qualified to comment) to encode generics with "missing default > values" and have them transmitted in well-formed avro binary. > If you fill in the "missing" default values, the Python libraries will > transmit correctly. > I'd be happy to add methods to the avro.RecordSchema objects (in the Python > libraries) that "fill defaults" on missing member fields of a record, > recursively (which probably means method extension of other schema classes as > well). > For backwards compatibility (and probably to avoid unnecessary data > traversal), clients probably want to explicitly ask the schema to fill in > defaults before transmission in the cases where you'd like to set only the > non-default values. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
