[jira] [Updated] (AVRO-2091) Eliminate org.apache.avro.specific.use_custom_coder feature flag
[ https://issues.apache.org/jira/browse/AVRO-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymie Stata updated AVRO-2091: --- Description: After the implementation of "custom coders" (AVRO-2090) is complete and seen more production usage, this feature flag should be eliminated. (More specifically, the initial release of AVRO-2090 should set USE_CUSTOM_CODERS to false by default, to get some initial production testing. The release after that should set this flag to true by default, but allow folks to fall back on the old way in case there are corner cases that aren't working. The release after that should remove this feature flag altogether, under the assumption that it works just fine and there's no need to maintain two ways of doing things.) (was: After the implementation of "custom coders" (AVRO-2090) is complete and seen more production usage, this feature flag should be eliminated.) > Eliminate org.apache.avro.specific.use_custom_coder feature flag > > > Key: AVRO-2091 > URL: https://issues.apache.org/jira/browse/AVRO-2091 > Project: Avro > Issue Type: Improvement > Components: java >Reporter: Raymie Stata >Priority: Minor > > After the implementation of "custom coders" (AVRO-2090) is complete and seen > more production usage, this feature flag should be eliminated. (More > specifically, the initial release of AVRO-2090 should set USE_CUSTOM_CODERS > to false by default, to get some initial production testing. The release > after that should set this flag to true by default, but allow folks to fall > back on the old way in case there are corner cases that aren't working. The > release after that should remove this feature flag altogether, under the > assumption that it works just fine and there's no need to maintain two ways > of doing things.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2229) Ability to test using a Docker image
[ https://issues.apache.org/jira/browse/AVRO-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633420#comment-16633420 ] ASF GitHub Bot commented on AVRO-2229: -- busbey commented on issue #336: [AVRO-2229] Test using a Docker image URL: https://github.com/apache/avro/pull/336#issuecomment-425732775 If I'm going to provide a review you'll have to be extremely patient. I have very limited volunteer time these days and right now it mostly gets used up in email related to the project. Given the importance of this changeset, I'd expect my turn around time to be 1-2 weeks. That's not to say anyone should block on me; I'm just setting expectations. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Ability to test using a Docker image > > > Key: AVRO-2229 > URL: https://issues.apache.org/jira/browse/AVRO-2229 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2229) Ability to test using a Docker image
[ https://issues.apache.org/jira/browse/AVRO-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633407#comment-16633407 ] ASF GitHub Bot commented on AVRO-2229: -- Fokko commented on issue #336: [AVRO-2229] Test using a Docker image URL: https://github.com/apache/avro/pull/336#issuecomment-425729391 @busbey @nandorKollar @cutting PTAL. Any further suggestions, or it this good to go? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Ability to test using a Docker image > > > Key: AVRO-2229 > URL: https://issues.apache.org/jira/browse/AVRO-2229 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Fokko Driesprong >Priority: Major > Fix For: 1.9.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Python 3 type hints
I've begun trying to add type hints to avro/lang/py3. This is uncovering a lot of variance problems in the current implementation. I need some advice and guidance. For example, class Schema has a constructor like def __init__( self, type: str, other_props: typing.Optional[typing.Dict[str, typing.Any]]=None ) -> None: but most of its subclasses don't have type in their constructors at all, much less require it. It's not possible to make the type checker happy without fixing all these problems, but fixing the problems is a big departure from merely adding the type hints. It would require careful planning as it will certainly change some of the signatures of these methods. (You could think of it as compatibility changes, or maybe just recognizing inconsistencies that exist today.) Also, opening individual bugs for everything I find might be discouraging, and they'll overlap a lot. I'd like to continue working on the type hints and let mypy complain about the things it finds, first, and then once we have the type hints somewhat in place and merged in, including running them in build.sh, then I'd like to pursue fixing the problems. What do you think? Is that a good approach, or should I try a different tactic?
[jira] [Commented] (AVRO-2196) Build fails on a clean machine
[
https://issues.apache.org/jira/browse/AVRO-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633309#comment-16633309
]
ASF GitHub Bot commented on AVRO-2196:
--
nielsbasjes closed pull request #318: AVRO-2196: Fix build problem about
missing avro-mapred tests jar
URL: https://github.com/apache/avro/pull/318
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/lang/java/mapred/pom.xml b/lang/java/mapred/pom.xml
index e6dd2c547..e836ba66a 100644
--- a/lang/java/mapred/pom.xml
+++ b/lang/java/mapred/pom.xml
@@ -90,6 +90,17 @@
+
+org.apache.maven.plugins
+maven-jar-plugin
+
+
+
+ test-jar
+
+
+
+
org.apache.maven.plugins
maven-enforcer-plugin
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Build fails on a clean machine
> --
>
> Key: AVRO-2196
> URL: https://issues.apache.org/jira/browse/AVRO-2196
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.0
>Reporter: Niels Basjes
>Assignee: Niels Basjes
>Priority: Major
> Fix For: 1.9.0
>
>
> On a clean machine (i.e. empty ~/.m2/reporsitory) when I simply run mvn clean
> package the build fails with
> {code:java}
> {code:java}
> [INFO]
>
> [INFO] Building Apache Avro Tools 1.9.0-SNAPSHOT
> [INFO]
>
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Avro Toplevel .. SUCCESS [1.009s]
> [INFO] Apache Avro Java .. SUCCESS [0.930s]
> [INFO] Apache Avro Guava Dependencies SUCCESS [1.377s]
> [INFO] Apache Avro ... SUCCESS [1:02.460s]
> [INFO] Apache Avro Compiler .. SUCCESS [10.705s]
> [INFO] Apache Avro Maven Plugin .. SUCCESS [5.190s]
> [INFO] Apache Avro IPC ... SUCCESS [47.988s]
> [INFO] Trevni Java ... SUCCESS [0.089s]
> [INFO] Trevni Java Core .. SUCCESS [4.526s]
> [INFO] Apache Avro Mapred API SUCCESS [1:29.866s]
> [INFO] Trevni Java Avro .. SUCCESS [14.612s]
> [INFO] Trevni Specification .. SUCCESS [0.229s]
> [INFO] Apache Avro Tools . FAILURE [0.042s]
> [INFO] Apache Avro Protobuf Compatibility SKIPPED
> [INFO] Apache Avro Thrift Compatibility .. SKIPPED
> [INFO] Apache Avro Maven Archetypes .. SKIPPED
> [INFO] Apache Avro Maven Service Archetype ... SKIPPED
> [INFO] Apache Avro gRPC .. SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 3:59.636s
> [INFO] Finished at: Fri Jun 29 09:14:04 UTC 2018
> [INFO] Final Memory: 80M/1309M
> [INFO]
>
> [ERROR] Failed to execute goal on project avro-tools: Could not resolve
> dependencies for project org.apache.avro:avro-tools:jar:1.9.0-SNAPSHOT: Could
> not find artifact org.apache.avro:avro-mapred:jar:tests:1.9.0-SNAPSHOT ->
> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :avro-tools
> {code}
>
> Apparently the
> {code:java}
> org.apache.avro:avro-mapred:jar:tests:1.9.0-SNAPSHOT{code}
> is not created (but
[jira] [Resolved] (AVRO-2196) Build fails on a clean machine
[
https://issues.apache.org/jira/browse/AVRO-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niels Basjes resolved AVRO-2196.
Resolution: Fixed
> Build fails on a clean machine
> --
>
> Key: AVRO-2196
> URL: https://issues.apache.org/jira/browse/AVRO-2196
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.0
>Reporter: Niels Basjes
>Assignee: Niels Basjes
>Priority: Major
> Fix For: 1.9.0
>
>
> On a clean machine (i.e. empty ~/.m2/reporsitory) when I simply run mvn clean
> package the build fails with
> {code:java}
> {code:java}
> [INFO]
>
> [INFO] Building Apache Avro Tools 1.9.0-SNAPSHOT
> [INFO]
>
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Avro Toplevel .. SUCCESS [1.009s]
> [INFO] Apache Avro Java .. SUCCESS [0.930s]
> [INFO] Apache Avro Guava Dependencies SUCCESS [1.377s]
> [INFO] Apache Avro ... SUCCESS [1:02.460s]
> [INFO] Apache Avro Compiler .. SUCCESS [10.705s]
> [INFO] Apache Avro Maven Plugin .. SUCCESS [5.190s]
> [INFO] Apache Avro IPC ... SUCCESS [47.988s]
> [INFO] Trevni Java ... SUCCESS [0.089s]
> [INFO] Trevni Java Core .. SUCCESS [4.526s]
> [INFO] Apache Avro Mapred API SUCCESS [1:29.866s]
> [INFO] Trevni Java Avro .. SUCCESS [14.612s]
> [INFO] Trevni Specification .. SUCCESS [0.229s]
> [INFO] Apache Avro Tools . FAILURE [0.042s]
> [INFO] Apache Avro Protobuf Compatibility SKIPPED
> [INFO] Apache Avro Thrift Compatibility .. SKIPPED
> [INFO] Apache Avro Maven Archetypes .. SKIPPED
> [INFO] Apache Avro Maven Service Archetype ... SKIPPED
> [INFO] Apache Avro gRPC .. SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 3:59.636s
> [INFO] Finished at: Fri Jun 29 09:14:04 UTC 2018
> [INFO] Final Memory: 80M/1309M
> [INFO]
>
> [ERROR] Failed to execute goal on project avro-tools: Could not resolve
> dependencies for project org.apache.avro:avro-tools:jar:1.9.0-SNAPSHOT: Could
> not find artifact org.apache.avro:avro-mapred:jar:tests:1.9.0-SNAPSHOT ->
> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :avro-tools
> {code}
>
> Apparently the
> {code:java}
> org.apache.avro:avro-mapred:jar:tests:1.9.0-SNAPSHOT{code}
> is not created (but seems needed).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2196) Build fails on a clean machine
[
https://issues.apache.org/jira/browse/AVRO-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633310#comment-16633310
]
ASF subversion and git services commented on AVRO-2196:
---
Commit e1c30a236a6b8244fc6e78e759c886617e9a05fb in avro's branch
refs/heads/master from [~nielsbasjes]
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=e1c30a2 ]
AVRO-2196: Fix build problem about missing avro-mapred tests jar
> Build fails on a clean machine
> --
>
> Key: AVRO-2196
> URL: https://issues.apache.org/jira/browse/AVRO-2196
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.0
>Reporter: Niels Basjes
>Assignee: Niels Basjes
>Priority: Major
> Fix For: 1.9.0
>
>
> On a clean machine (i.e. empty ~/.m2/reporsitory) when I simply run mvn clean
> package the build fails with
> {code:java}
> {code:java}
> [INFO]
>
> [INFO] Building Apache Avro Tools 1.9.0-SNAPSHOT
> [INFO]
>
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Avro Toplevel .. SUCCESS [1.009s]
> [INFO] Apache Avro Java .. SUCCESS [0.930s]
> [INFO] Apache Avro Guava Dependencies SUCCESS [1.377s]
> [INFO] Apache Avro ... SUCCESS [1:02.460s]
> [INFO] Apache Avro Compiler .. SUCCESS [10.705s]
> [INFO] Apache Avro Maven Plugin .. SUCCESS [5.190s]
> [INFO] Apache Avro IPC ... SUCCESS [47.988s]
> [INFO] Trevni Java ... SUCCESS [0.089s]
> [INFO] Trevni Java Core .. SUCCESS [4.526s]
> [INFO] Apache Avro Mapred API SUCCESS [1:29.866s]
> [INFO] Trevni Java Avro .. SUCCESS [14.612s]
> [INFO] Trevni Specification .. SUCCESS [0.229s]
> [INFO] Apache Avro Tools . FAILURE [0.042s]
> [INFO] Apache Avro Protobuf Compatibility SKIPPED
> [INFO] Apache Avro Thrift Compatibility .. SKIPPED
> [INFO] Apache Avro Maven Archetypes .. SKIPPED
> [INFO] Apache Avro Maven Service Archetype ... SKIPPED
> [INFO] Apache Avro gRPC .. SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 3:59.636s
> [INFO] Finished at: Fri Jun 29 09:14:04 UTC 2018
> [INFO] Final Memory: 80M/1309M
> [INFO]
>
> [ERROR] Failed to execute goal on project avro-tools: Could not resolve
> dependencies for project org.apache.avro:avro-tools:jar:1.9.0-SNAPSHOT: Could
> not find artifact org.apache.avro:avro-mapred:jar:tests:1.9.0-SNAPSHOT ->
> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :avro-tools
> {code}
>
> Apparently the
> {code:java}
> org.apache.avro:avro-mapred:jar:tests:1.9.0-SNAPSHOT{code}
> is not created (but seems needed).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2229) Ability to test using a Docker image
[
https://issues.apache.org/jira/browse/AVRO-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633260#comment-16633260
]
ASF GitHub Bot commented on AVRO-2229:
--
Fokko commented on a change in pull request #336: [AVRO-2229] Test using a
Docker image
URL: https://github.com/apache/avro/pull/336#discussion_r221448847
##
File path: share/docker/run-tests.sh
##
@@ -0,0 +1,38 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+headline(){
+ echo -e
"\e[1;34m#"
+ echo -e "# $1 \e[1;37m"
+ echo -e
"\e[1;34m#\e[0m"
+}
+
+set -e
+
+for lang in $(ls /avro/lang/)
Review comment:
Thanks, just did
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Ability to test using a Docker image
>
>
> Key: AVRO-2229
> URL: https://issues.apache.org/jira/browse/AVRO-2229
> Project: Avro
> Issue Type: Improvement
>Affects Versions: 1.8.2
>Reporter: Fokko Driesprong
>Priority: Major
> Fix For: 1.9.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
