[jira] [Created] (AVRO-3010) Support for Meta Annotations when using Schema Annotations on Java Objects
Brian Cullen created AVRO-3010:
--
Summary: Support for Meta Annotations when using Schema
Annotations on Java Objects
Key: AVRO-3010
URL: https://issues.apache.org/jira/browse/AVRO-3010
Project: Apache Avro
Issue Type: Improvement
Components: java
Reporter: Brian Cullen
I'd like to propose (and offer to implement) support for meta annotations when
using the schema annotations from the org.apache.avro.reflect package to
generate schemas from Java POJOs.
As a simple example from my use case consider a object that could hold a number
of optional UUIDs. Currently I would probably annotate it as follows.
{code:java}
@AvroMeta(key = "logicalType", value = "UUID")
@Nullable
private String someUUID;
{code}
Whereas if the use of meta annotation were allowed you could define an
annotation along these lines to use instead.
{code:java}
@AvroMeta(key = "logicalType", value = "UUID")
@Nullable
public @interface OptionalUUID {
}
{code}
An additional benefit is where custom annotations are already being used on the
class for validation or persistence purposes these annotation could be updated
without having to change the annotated code at all.
I have experimented with this idea in the code base and it does appear to work,
although there are some details that would need to be confirmed. The changes
required involve changing how annotation are accessed in the ReflectData class
and updating the target annotation of the existing annotations to include
ElementType.ANNOTATION_TYPE, so that they can be applied to other annotations.
On the surface this change does not appear to be a large one and should not
affect existing functionality. It would however give developers the ability to
cut down on repeated code if they are using annotation a lot.
Do you think this fits in with the rest of the system or are there issues that
I'm not aware of?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-3006) Update documentation on pypi
[ https://issues.apache.org/jira/browse/AVRO-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael A. Smith updated AVRO-3006: --- Affects Version/s: (was: 1.10.1) > Update documentation on pypi > > > Key: AVRO-3006 > URL: https://issues.apache.org/jira/browse/AVRO-3006 > Project: Apache Avro > Issue Type: Improvement > Components: python >Affects Versions: 1.11.0 >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > > The avro python packages have outdated or missing documentation at > https://pypi.org/project/avro-python3/ (which should state that this package > is deprecated in favor of…) > https://pypi.org/project/avro/ (which should have a document describing the > package and should correctly state which version of Python it supports) -- This message was sent by Atlassian Jira (v8.3.4#803005)
Will we have a 1.10.2?
The commit log for branch-1.10 says "Preparing for 1.10.2 development", but I can't indicate in Jira that AVRO-3006 will be in that release. Is that release still going to happen?
[jira] [Updated] (AVRO-3006) Update documentation on pypi
[ https://issues.apache.org/jira/browse/AVRO-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael A. Smith updated AVRO-3006: --- Affects Version/s: 1.10.1 1.11.0 > Update documentation on pypi > > > Key: AVRO-3006 > URL: https://issues.apache.org/jira/browse/AVRO-3006 > Project: Apache Avro > Issue Type: Improvement > Components: python >Affects Versions: 1.11.0, 1.10.1 >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > > The avro python packages have outdated or missing documentation at > https://pypi.org/project/avro-python3/ (which should state that this package > is deprecated in favor of…) > https://pypi.org/project/avro/ (which should have a document describing the > package and should correctly state which version of Python it supports) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AVRO-3006) Update documentation on pypi
[ https://issues.apache.org/jira/browse/AVRO-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252565#comment-17252565 ] ASF subversion and git services commented on AVRO-3006: --- Commit ca6d72dd618b7b52aedd32654bfaa6c35d950ccf in avro's branch refs/heads/master from Michael A. Smith [ https://gitbox.apache.org/repos/asf?p=avro.git;h=ca6d72d ] AVRO-3006: Documentation for Pypi Packages (#1041) * AVRO-3006: Update Pypi Docs for py3 * AVRO-3006: Make py3 README a Markdown File * AVRO-3006: Add README for lang/py * AVRO-3006: Add Development Status Trove Classifiers > Update documentation on pypi > > > Key: AVRO-3006 > URL: https://issues.apache.org/jira/browse/AVRO-3006 > Project: Apache Avro > Issue Type: Improvement > Components: python >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > > The avro python packages have outdated or missing documentation at > https://pypi.org/project/avro-python3/ (which should state that this package > is deprecated in favor of…) > https://pypi.org/project/avro/ (which should have a document describing the > package and should correctly state which version of Python it supports) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AVRO-3006) Update documentation on pypi
[ https://issues.apache.org/jira/browse/AVRO-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252567#comment-17252567 ] ASF subversion and git services commented on AVRO-3006: --- Commit ca6d72dd618b7b52aedd32654bfaa6c35d950ccf in avro's branch refs/heads/master from Michael A. Smith [ https://gitbox.apache.org/repos/asf?p=avro.git;h=ca6d72d ] AVRO-3006: Documentation for Pypi Packages (#1041) * AVRO-3006: Update Pypi Docs for py3 * AVRO-3006: Make py3 README a Markdown File * AVRO-3006: Add README for lang/py * AVRO-3006: Add Development Status Trove Classifiers > Update documentation on pypi > > > Key: AVRO-3006 > URL: https://issues.apache.org/jira/browse/AVRO-3006 > Project: Apache Avro > Issue Type: Improvement > Components: python >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > > The avro python packages have outdated or missing documentation at > https://pypi.org/project/avro-python3/ (which should state that this package > is deprecated in favor of…) > https://pypi.org/project/avro/ (which should have a document describing the > package and should correctly state which version of Python it supports) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AVRO-3006) Update documentation on pypi
[ https://issues.apache.org/jira/browse/AVRO-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252568#comment-17252568 ] ASF subversion and git services commented on AVRO-3006: --- Commit ca6d72dd618b7b52aedd32654bfaa6c35d950ccf in avro's branch refs/heads/master from Michael A. Smith [ https://gitbox.apache.org/repos/asf?p=avro.git;h=ca6d72d ] AVRO-3006: Documentation for Pypi Packages (#1041) * AVRO-3006: Update Pypi Docs for py3 * AVRO-3006: Make py3 README a Markdown File * AVRO-3006: Add README for lang/py * AVRO-3006: Add Development Status Trove Classifiers > Update documentation on pypi > > > Key: AVRO-3006 > URL: https://issues.apache.org/jira/browse/AVRO-3006 > Project: Apache Avro > Issue Type: Improvement > Components: python >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > > The avro python packages have outdated or missing documentation at > https://pypi.org/project/avro-python3/ (which should state that this package > is deprecated in favor of…) > https://pypi.org/project/avro/ (which should have a document describing the > package and should correctly state which version of Python it supports) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AVRO-3006) Update documentation on pypi
[ https://issues.apache.org/jira/browse/AVRO-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252566#comment-17252566 ] ASF subversion and git services commented on AVRO-3006: --- Commit ca6d72dd618b7b52aedd32654bfaa6c35d950ccf in avro's branch refs/heads/master from Michael A. Smith [ https://gitbox.apache.org/repos/asf?p=avro.git;h=ca6d72d ] AVRO-3006: Documentation for Pypi Packages (#1041) * AVRO-3006: Update Pypi Docs for py3 * AVRO-3006: Make py3 README a Markdown File * AVRO-3006: Add README for lang/py * AVRO-3006: Add Development Status Trove Classifiers > Update documentation on pypi > > > Key: AVRO-3006 > URL: https://issues.apache.org/jira/browse/AVRO-3006 > Project: Apache Avro > Issue Type: Improvement > Components: python >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > > The avro python packages have outdated or missing documentation at > https://pypi.org/project/avro-python3/ (which should state that this package > is deprecated in favor of…) > https://pypi.org/project/avro/ (which should have a document describing the > package and should correctly state which version of Python it supports) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AVRO-3006) Update documentation on pypi
[ https://issues.apache.org/jira/browse/AVRO-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252564#comment-17252564 ] ASF subversion and git services commented on AVRO-3006: --- Commit ca6d72dd618b7b52aedd32654bfaa6c35d950ccf in avro's branch refs/heads/master from Michael A. Smith [ https://gitbox.apache.org/repos/asf?p=avro.git;h=ca6d72d ] AVRO-3006: Documentation for Pypi Packages (#1041) * AVRO-3006: Update Pypi Docs for py3 * AVRO-3006: Make py3 README a Markdown File * AVRO-3006: Add README for lang/py * AVRO-3006: Add Development Status Trove Classifiers > Update documentation on pypi > > > Key: AVRO-3006 > URL: https://issues.apache.org/jira/browse/AVRO-3006 > Project: Apache Avro > Issue Type: Improvement > Components: python >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > > The avro python packages have outdated or missing documentation at > https://pypi.org/project/avro-python3/ (which should state that this package > is deprecated in favor of…) > https://pypi.org/project/avro/ (which should have a document describing the > package and should correctly state which version of Python it supports) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [avro] kojiromike merged pull request #1041: AVRO-3006: Documentation for Pypi Packages
kojiromike merged pull request #1041: URL: https://github.com/apache/avro/pull/1041 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (AVRO-3008) Threading.is_alive Spelling for Py3.9
[ https://issues.apache.org/jira/browse/AVRO-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252557#comment-17252557 ] ASF subversion and git services commented on AVRO-3008: --- Commit 0fbc98b7625577a9b51c4c52467d62cf08cbc16f in avro's branch refs/heads/master from Michael A. Smith [ https://gitbox.apache.org/repos/asf?p=avro.git;h=0fbc98b ] AVRO-3008: Threading.is_alive Spelling for Py3.9 (#1042) * `Threading.isAlive` is a synonym for `Threading.is_alive`, which has been supported since Python 2.7. * `Threading.isAlive` has been deprecated since 3.1. * `Threading.isAlive` was removed entirely in Python 3.9. > Threading.is_alive Spelling for Py3.9 > - > > Key: AVRO-3008 > URL: https://issues.apache.org/jira/browse/AVRO-3008 > Project: Apache Avro > Issue Type: Task > Components: python >Reporter: Michael A. Smith >Assignee: Michael A. Smith >Priority: Major > > Python dropped support for the old 'isAlive' spelling in 3.9. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [avro] kojiromike merged pull request #1042: AVRO-3008: Threading.is_alive Spelling for Py3.9
kojiromike merged pull request #1042: URL: https://github.com/apache/avro/pull/1042 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: GitHub Actions
On Sun, Dec 20, 2020 at 6:13 AM Driesprong, Fokko wrote: > > Hi Michael, > > Thanks for bringing this up. I think it would be a great idea. I don't have > anything against Travis, but I like GA a lot. For example, their container > support is much better, and the syntax is cleaner. It also integrates > extremely well with Github itself. This can be nice if we want to have some > flow someday. > > When it comes to Apache Yetus, I must admit, I've implemented Yetus at the > time, but I'm not super familiar with the tool. I think the current > implementation doesn't get the value out of it that it promises to do. > Also, one of the reasons that the implementation is far from optimal > because it doesn't fit the project that well. I would suggest to remove it. > > One thing that concerns me a bit is the scattering of the commands in the > GA yml files and the build.sh. I would suggest moving everything into one > place. In the case of Github Actions, you can also run it easily locally: > https://github.com/nektos/act That sounds great. Is this something we can do iteratively, or did you have in mind doing it all in the one PR? > > Cheers, Fokko > > > Op zo 20 dec. 2020 om 06:05 schreef Michael A. Smith : > > > I created a PR to implement our tests in GitHub actions. I'd like to > > know if other folks are interested in me pursuing this further and > > replacing the Travis/Yetus build system. > > > > Some data: > > - In its current configuration, a Travis build that doesn't fail takes > > around 70 minutes. > > - Travis usually fails, often for reasons unrelated to a particular PR. > > - Understanding why it fails requires spelunking through thousands of > > lines of log files. > > - Casual contributors are disinclined to set up Travis for their > > forks, and can end up triggering multiple travis builds in an Apache > > PR to track down a bug. > > - The single Docker megafile tightly couples every language toolchain, > > so testing multiple language versions is difficult. > > > > All of these problems can be fixed within the Travis/Yetus build > > system (except maybe the "casual contributors" thing), I'm sure. But I > > have looked into it before and haven't been able to figure it out. > > > > Here's what I've done with GitHub actions: > > - Jobs are isolated by lang/* and only trigger when a change touches > > that language. Even if a problem is causing, say, Ruby tests to fail > > in master, PHP contributions can still make it through. > > - The tests are run in parallel, both across languages and within, > > across multiple language versions and interop and unit tests. > > - The slowest jobs (the Java tests) take 15 minutes. The worst case > > test run (aside from an outage) will probably be under 20 minutes, if > > we are heavily queued. > > - This PR tests java 8 and 11, js using node 10, 11 and 12, php 7.3, > > 7.4 and 8, python 3.6-3.9 and pypy3.6 and 3.7. Adding and removing > > language implementations is trivial. > > - If we merge this PR, anyone who forks the repo will get these > > actions in their fork. > > > > One thing I haven't yet implemented is an action for > > share/test/interop/bin/test_rpc_interop.sh. I think I can do that, > > too, but I want to know if this can go anywhere before I work on it > > more. > > > > WDYT? > > > > - Michael > >
Re: GitHub Actions
Can anyone help me with the perl tests? They errored out with String found where operator expected at ./Makefile.PL line 32, near "readme_from 'lib/Avro.pm'" include /home/runner/work/avro/avro/lang/perl/inc/Module/Install.pm (Do you need to predeclare readme_from?) syntax error at ./Makefile.PL line 32, near "readme_from 'lib/Avro.pm'" Execution of ./Makefile.PL aborted due to compilation errors. but strangely, this was counted as a successful run: https://github.com/kojiromike/avro/runs/1583496286?check_suite_focus=true I tried again, adding inc::Module::Install::ReadmeFromPod to the modules installed by cpan, but got Couldn't find module or a distribution inc::Module::Install::ReadmeFromPod What is the right way to set up and install the tests for perl in a GitHub action? On Sun, Dec 20, 2020 at 6:13 AM Driesprong, Fokko wrote: > > Hi Michael, > > Thanks for bringing this up. I think it would be a great idea. I don't have > anything against Travis, but I like GA a lot. For example, their container > support is much better, and the syntax is cleaner. It also integrates > extremely well with Github itself. This can be nice if we want to have some > flow someday. > > When it comes to Apache Yetus, I must admit, I've implemented Yetus at the > time, but I'm not super familiar with the tool. I think the current > implementation doesn't get the value out of it that it promises to do. > Also, one of the reasons that the implementation is far from optimal > because it doesn't fit the project that well. I would suggest to remove it. > > One thing that concerns me a bit is the scattering of the commands in the > GA yml files and the build.sh. I would suggest moving everything into one > place. In the case of Github Actions, you can also run it easily locally: > https://github.com/nektos/act > > Cheers, Fokko > > > Op zo 20 dec. 2020 om 06:05 schreef Michael A. Smith : > > > I created a PR to implement our tests in GitHub actions. I'd like to > > know if other folks are interested in me pursuing this further and > > replacing the Travis/Yetus build system. > > > > Some data: > > - In its current configuration, a Travis build that doesn't fail takes > > around 70 minutes. > > - Travis usually fails, often for reasons unrelated to a particular PR. > > - Understanding why it fails requires spelunking through thousands of > > lines of log files. > > - Casual contributors are disinclined to set up Travis for their > > forks, and can end up triggering multiple travis builds in an Apache > > PR to track down a bug. > > - The single Docker megafile tightly couples every language toolchain, > > so testing multiple language versions is difficult. > > > > All of these problems can be fixed within the Travis/Yetus build > > system (except maybe the "casual contributors" thing), I'm sure. But I > > have looked into it before and haven't been able to figure it out. > > > > Here's what I've done with GitHub actions: > > - Jobs are isolated by lang/* and only trigger when a change touches > > that language. Even if a problem is causing, say, Ruby tests to fail > > in master, PHP contributions can still make it through. > > - The tests are run in parallel, both across languages and within, > > across multiple language versions and interop and unit tests. > > - The slowest jobs (the Java tests) take 15 minutes. The worst case > > test run (aside from an outage) will probably be under 20 minutes, if > > we are heavily queued. > > - This PR tests java 8 and 11, js using node 10, 11 and 12, php 7.3, > > 7.4 and 8, python 3.6-3.9 and pypy3.6 and 3.7. Adding and removing > > language implementations is trivial. > > - If we merge this PR, anyone who forks the repo will get these > > actions in their fork. > > > > One thing I haven't yet implemented is an action for > > share/test/interop/bin/test_rpc_interop.sh. I think I can do that, > > too, but I want to know if this can go anywhere before I work on it > > more. > > > > WDYT? > > > > - Michael > >
Re: GitHub Actions
Hi Michael, Thanks for bringing this up. I think it would be a great idea. I don't have anything against Travis, but I like GA a lot. For example, their container support is much better, and the syntax is cleaner. It also integrates extremely well with Github itself. This can be nice if we want to have some flow someday. When it comes to Apache Yetus, I must admit, I've implemented Yetus at the time, but I'm not super familiar with the tool. I think the current implementation doesn't get the value out of it that it promises to do. Also, one of the reasons that the implementation is far from optimal because it doesn't fit the project that well. I would suggest to remove it. One thing that concerns me a bit is the scattering of the commands in the GA yml files and the build.sh. I would suggest moving everything into one place. In the case of Github Actions, you can also run it easily locally: https://github.com/nektos/act Cheers, Fokko Op zo 20 dec. 2020 om 06:05 schreef Michael A. Smith : > I created a PR to implement our tests in GitHub actions. I'd like to > know if other folks are interested in me pursuing this further and > replacing the Travis/Yetus build system. > > Some data: > - In its current configuration, a Travis build that doesn't fail takes > around 70 minutes. > - Travis usually fails, often for reasons unrelated to a particular PR. > - Understanding why it fails requires spelunking through thousands of > lines of log files. > - Casual contributors are disinclined to set up Travis for their > forks, and can end up triggering multiple travis builds in an Apache > PR to track down a bug. > - The single Docker megafile tightly couples every language toolchain, > so testing multiple language versions is difficult. > > All of these problems can be fixed within the Travis/Yetus build > system (except maybe the "casual contributors" thing), I'm sure. But I > have looked into it before and haven't been able to figure it out. > > Here's what I've done with GitHub actions: > - Jobs are isolated by lang/* and only trigger when a change touches > that language. Even if a problem is causing, say, Ruby tests to fail > in master, PHP contributions can still make it through. > - The tests are run in parallel, both across languages and within, > across multiple language versions and interop and unit tests. > - The slowest jobs (the Java tests) take 15 minutes. The worst case > test run (aside from an outage) will probably be under 20 minutes, if > we are heavily queued. > - This PR tests java 8 and 11, js using node 10, 11 and 12, php 7.3, > 7.4 and 8, python 3.6-3.9 and pypy3.6 and 3.7. Adding and removing > language implementations is trivial. > - If we merge this PR, anyone who forks the repo will get these > actions in their fork. > > One thing I haven't yet implemented is an action for > share/test/interop/bin/test_rpc_interop.sh. I think I can do that, > too, but I want to know if this can go anywhere before I work on it > more. > > WDYT? > > - Michael >
