[jira] [Commented] (AVRO-2164) Make Decimal a first class type.

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675887#comment-16675887
 ] 

Zoltan Farkas commented on AVRO-2164:
-

I am in favor of making decimal a first class type. 

One extra benefit is that we would be able to properly serialize the decimals 
in the json format. (Number instead of binary)

Regarding evolution, we should allow for increases of precision and scale. 

why can't this be part of avro 2.0?

Side note:

In my own decimal logical type implementation, I serialize the scale along with 
the unscaled value.  So if I have a decimal ( 16, 8) and I serialize a decimal 
value with a scale <= 8, the value will be serialized with its original scale, 
for everything with a larger scale, I either error or round depending if a 
rounding mode is specified via a type attribute... 




> Make Decimal a first class type.
> 
>
> Key: AVRO-2164
> URL: https://issues.apache.org/jira/browse/AVRO-2164
> Project: Avro
>  Issue Type: Improvement
>  Components: logical types
>Affects Versions: 1.8.2
>Reporter: Andy Coates
>Priority: Major
>
> I'd be interested to hear the communities thoughts on making decimal a first 
> class type. 
> The current logical type encodes a decimal into a _bytes_ or _fixed_. This 
> encoding does not include any information about the scale, i.e. this encoding 
> is lossy. 
> There are open issues around the compatibility / evolvability of schemas 
> containing decimal logical types, (e.g. AVRO-2078 & AVRO-1721), that mean 
> reading data that was previously written with a different scale will result 
> in data corruption.
> If these issues were fixed, with suitable compatibility checks put in place, 
> this would then make it impossible to evolve an Avro schema where the scale 
> needs to be changed. This inability to evolve the scale is very restrictive, 
> and can result in high overhead for organizations that _need_ to change the 
> scale, i.e. they may potentially need to copy their entire data set, 
> deserializing with the old scale and re-serializing with the new.
> If _decimal_ were promoted to a first class type, this would allow the scale 
> to be captured in the serialized form, allow for schema evolution support.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (AVRO-2253) Documentation of java-class -related annotations

[Samuel Martin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Samuel Martin updated AVRO-2253:

Description: 
There is a lack of consistency in the documentation about the use of 
@java-class (plus, its relation with avro.java.string), @java-element-class and 
@java-key-class annotations
For instance, see the references to 
`[@java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]`,
 and the [mentions to the same 
annotation|https://sematext.com/opensee/?tag=avro=Avro==%22java-element-class%22]
 and to `ELEMENT_PROP` as well (in the different classes where constants with 
that name appear).
In some places, documentation talks about 
[@java-element|https://avro.apache.org/docs/1.6.3/api/java/org/apache/avro/reflect/package-summary.html]
 instead of @java-element-class
Even the test case that is supposed to be provided as an [example of use of 
@java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]
 indeed employs @java-class :
`map<@java-class("java.math.BigDecimal") string> mapWithBigDecimalElements`
is equivalent to
`@java-element-classmap("java.math.BigDecimal")  
mapWithBigDecimalElements`
but it is not clear from the documentation


  was:
There is a lack of consistency in the documentation about the use of 
@java-class (plus, its relation with avro.java.string), @java-element-class and 
@java-key-class annotations
For instance, see the references to 
`[@java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]`,
 and the [mentions to the same 
annotation|https://sematext.com/opensee/?tag=avro=Avro==%22java-element-class%22]
 and to `ELEMENT_PROP` as well (in the different classes where constants with 
that name appear).
In some places, documentation talks about 
[@java-element|https://avro.apache.org/docs/1.6.3/api/java/org/apache/avro/reflect/package-summary.html]
 instead of @java-element-class
Even the test case that is supposed to be provided as an [example of use of 
@java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]
 indeed employs @java-class 


> Documentation of java-class -related annotations
> 
>
> Key: AVRO-2253
> URL: https://issues.apache.org/jira/browse/AVRO-2253
> Project: Avro
>  Issue Type: Bug
>  Components: java, spec
>Reporter: Samuel Martin
>Priority: Minor
>
> There is a lack of consistency in the documentation about the use of 
> @java-class (plus, its relation with avro.java.string), @java-element-class 
> and @java-key-class annotations
> For instance, see the references to 
> `[@java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]`,
>  and the [mentions to the same 
> annotation|https://sematext.com/opensee/?tag=avro=Avro==%22java-element-class%22]
>  and to `ELEMENT_PROP` as well (in the different classes where constants with 
> that name appear).
> In some places, documentation talks about 
> [@java-element|https://avro.apache.org/docs/1.6.3/api/java/org/apache/avro/reflect/package-summary.html]
>  instead of @java-element-class
> Even the test case that is supposed to be provided as an [example of use of 
> @java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]
>  indeed employs @java-class :
> `map<@java-class("java.math.BigDecimal") string> mapWithBigDecimalElements`
> is equivalent to
> `@java-element-classmap("java.math.BigDecimal")  
> mapWithBigDecimalElements`
> but it is not clear from the documentation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (AVRO-2253) Documentation of java-class -related annotations

[Samuel Martin (JIRA)]

Samuel Martin created AVRO-2253:
---

 Summary: Documentation of java-class -related annotations
 Key: AVRO-2253
 URL: https://issues.apache.org/jira/browse/AVRO-2253
 Project: Avro
  Issue Type: Bug
  Components: java, spec
Reporter: Samuel Martin


There is a lack of consistency in the documentation about the use of 
@java-class (plus, its relation with avro.java.string), @java-element-class and 
@java-key-class annotations
For instance, see the references to 
`[@java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]`,
 and the [mentions to the same 
annotation|https://sematext.com/opensee/?tag=avro=Avro==%22java-element-class%22]
 and to `ELEMENT_PROP` as well (in the different classes where constants with 
that name appear).
In some places, documentation talks about 
[@java-element|https://avro.apache.org/docs/1.6.3/api/java/org/apache/avro/reflect/package-summary.html]
 instead of @java-element-class
Even the test case that is supposed to be provided as an [example of use of 
@java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]
 indeed employs @java-class 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (AVRO-2253) Documentation of java-class -related annotations

[Samuel Martin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Samuel Martin updated AVRO-2253:

Priority: Minor  (was: Major)

> Documentation of java-class -related annotations
> 
>
> Key: AVRO-2253
> URL: https://issues.apache.org/jira/browse/AVRO-2253
> Project: Avro
>  Issue Type: Bug
>  Components: java, spec
>Reporter: Samuel Martin
>Priority: Minor
>
> There is a lack of consistency in the documentation about the use of 
> @java-class (plus, its relation with avro.java.string), @java-element-class 
> and @java-key-class annotations
> For instance, see the references to 
> `[@java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]`,
>  and the [mentions to the same 
> annotation|https://sematext.com/opensee/?tag=avro=Avro==%22java-element-class%22]
>  and to `ELEMENT_PROP` as well (in the different classes where constants with 
> that name appear).
> In some places, documentation talks about 
> [@java-element|https://avro.apache.org/docs/1.6.3/api/java/org/apache/avro/reflect/package-summary.html]
>  instead of @java-element-class
> Even the test case that is supposed to be provided as an [example of use of 
> @java-element-class|https://github.com/apache/avro/search?q=java-element-class_q=java-element-class]
>  indeed employs @java-class 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Python 3 support

[Driesprong, Fokko]

Hi Michael,

Thanks for pointing out the Python versions. I'm fine with the suggestion
of supporting 3.4 up to 3.7. I would also consider supporting from 3.5
onwards. Right now I've done some tests against 3.4 and that seems to run
fine. I'm fine with adding the classifiers.

Cheers, Fokko

Op wo 31 okt. 2018 om 15:39 schreef Michael A. Smith :

> In comments to https://github.com/apache/avro/pull/347 it was suggested
> that we hold a vote to discuss which versions of Python 3 we consider
> officially supported. I assume we follow the Apache Voting Process for code
> modifications
>  >:
>
> Scott Belden used the pypi API to get the following data:
>
> [{"python_version":"3.5","download_count":"30258"},
>  {"python_version":"3.6","download_count":"26228"},
>  {"python_version":"3.4","download_count":"9140"},
>  {"python_version":"3.7","download_count":"2502"},
>  {"python_version":null,"download_count":"793"},
>  {"python_version":"2.7","download_count":"173"},
>  {"python_version":"2.6","download_count":"3"}]
>
> It seems clear that there's still a fair 3.4 user base out there, although
> it's shrinking. So...
>
> - For the package at https://github.com/apache/avro/tree/master/lang/py3
> - For Avro versions < 2.0, at least
> - As long as PYPI is counting python 3.4 downloads at > 1000/month
>
> I move that we consider Python 3.4 and up to the latest release of Python 3
> (currently 3.7) officially supported. If I don't hear any objections by
> Saturday, I'll set the trove classifiers to specify those versions in
> https://github.com/apache/avro/pull/347.
>
> -Michael A. Smith (kojiro/kojiromike)
>
> PS: please kindly let me know if I'm not following some proper procedure.
>

[jira] [Commented] (AVRO-1887) set up precommit checks via Apache Yetus

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675297#comment-16675297
 ] 

ASF GitHub Bot commented on AVRO-1887:
--

Fokko commented on issue #353: AVRO-1887 Integrate Yetus and fix tests
URL: https://github.com/apache/avro/pull/353#issuecomment-435911385
 
 
   @kojiromike My suggestion would be to merge this first, and then you can 
freely rework the shell scripts. You're much better at it than me :-)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> set up precommit checks via Apache Yetus
> 
>
> Key: AVRO-1887
> URL: https://issues.apache.org/jira/browse/AVRO-1887
> Project: Avro
>  Issue Type: Task
>  Components: build, community
>Reporter: Sean Busbey
>Assignee: Suraj Acharya
>Priority: Blocker
> Attachments: AVRO-1887-0.patch, AVRO-1887-1.patch, AVRO-1887-2.patch
>
>
> The lack of automated checks for things like doing a full build is negatively 
> impacting my ability to do timely reviews. We should leverage the work done 
> by other communities and get precommit checks going with Apache Yetus ([ref 
> Apache Yetus 
> Precommit|http://yetus.apache.org/documentation/0.3.0/precommit-basic/])
> Yetus has built in support for using Docker as a containerized build 
> environment ([ref 
> docs|https://yetus.apache.org/documentation/0.3.0/precommit-advanced/#Docker_Support]),
>  but I don't know if we can use the one we have directly or if we'll need to 
> do some refactoring.
> Implementing this will ultimately require a committer (so that they can make 
> changes to builds.apache.org), but a non-cmmitter can get the bulk of the 
> work done by managing to do local yetus checks with the docker feature active.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1887) set up precommit checks via Apache Yetus

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675292#comment-16675292
 ] 

ASF GitHub Bot commented on AVRO-1887:
--

Fokko commented on issue #353: AVRO-1887 Integrate Yetus and fix tests
URL: https://github.com/apache/avro/pull/353#issuecomment-435909844
 
 
   @kojiromike @cutting PTAL.
   
   I know there are a crazy amount of changes. Instead of having fixed paths, I 
updated all the tests to use TemporaryDirectories using junit. Also, I've 
rewritten a lot of the file operations using `try-with-resource`. A lot of the 
streams weren't closed at all.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> set up precommit checks via Apache Yetus
> 
>
> Key: AVRO-1887
> URL: https://issues.apache.org/jira/browse/AVRO-1887
> Project: Avro
>  Issue Type: Task
>  Components: build, community
>Reporter: Sean Busbey
>Assignee: Suraj Acharya
>Priority: Blocker
> Attachments: AVRO-1887-0.patch, AVRO-1887-1.patch, AVRO-1887-2.patch
>
>
> The lack of automated checks for things like doing a full build is negatively 
> impacting my ability to do timely reviews. We should leverage the work done 
> by other communities and get precommit checks going with Apache Yetus ([ref 
> Apache Yetus 
> Precommit|http://yetus.apache.org/documentation/0.3.0/precommit-basic/])
> Yetus has built in support for using Docker as a containerized build 
> environment ([ref 
> docs|https://yetus.apache.org/documentation/0.3.0/precommit-advanced/#Docker_Support]),
>  but I don't know if we can use the one we have directly or if we'll need to 
> do some refactoring.
> Implementing this will ultimately require a committer (so that they can make 
> changes to builds.apache.org), but a non-cmmitter can get the bulk of the 
> work done by managing to do local yetus checks with the docker feature active.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-2252) I'd like to improve Avro .NET (C#) library (many points)

[Anton Ryzhov (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674889#comment-16674889
 ] 

Anton Ryzhov commented on AVRO-2252:


Thanks for the answer. The changes you did look promising. 
I also have had much experience using StyleCop.Analyzers and 
Microsoft.CodeAnalysis.FxCopAnalyzers in many projects.
Is there any points I could help you working with?

> I'd like to improve Avro .NET (C#) library (many points)
> 
>
> Key: AVRO-2252
> URL: https://issues.apache.org/jira/browse/AVRO-2252
> Project: Avro
>  Issue Type: Wish
>  Components: csharp
>Reporter: Anton Ryzhov
>Priority: Major
>
> Hello all,
> The company where I'm working as a .NET developer is actively using Avro 
> format.
> I'd like to improve Avro .NET (C#) library:
> 1) clean-up the code:
>  - remove trailing spaces, unused namespace usings, etc.
>  - remove unused dependency of log4net library
>  - replace dependency of json library from direct reference to Nuget package
> 2) format the code to unify code style everywhere in the library
>  - possibly using the Microsoft recommended code style for C# 
> [https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/inside-a-program/coding-conventions]
> 3) use the latest C# 7.0 language features to make the code more compact and 
> readable
> 4) make .NET 4.5 and .NET standard 2.0 versions of the library, keeping the 
> existing compatibility with the .NET 3.5
>  - add asynchronous API to the .NET 4.5 and .NET standard 2.0 versions (async 
> methods along with the synchronous ones).
> What do you think?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)