Hello everyone! The bad news: I'm cancelling the RC1 for a potential issue with one of the SDKs, currently under investigation. This will also give us some time to check out the build issues that Martin raised!
The good news: I've gotten some pretty thorough validation of the Python and Java SDKs (the ones we use internally at our work). With our in-house code, we exercise a lot of different schemas with generic and specific records. I've also compiled Parquet with 1.11.0-rc1 and all tests pass. Some notes that should probably included in the release notes as migration issues: * I did run into a Java problem with some custom templates that override the "classic" record.vm provided with Avro. The `SpecificCompiler.deprecatedFields()` method has been removed, which means that any projects that override these templates probably will need to be refreshed with the changes from Avro 1.11.0. Does anyone think this might be a major issue? * I ran into a source compatibility with the Python SDK issue due to exceptions changing packages (avro.schema.SchemaParseException to avro.errors.SchemaParseException). I'd consider this minor, and I've only encountered it in unit tests. We're looking at what we need to create an RC2, I'll send out an update shortly. All my best, Ryan On Mon, Oct 11, 2021 at 2:38 PM Ryan Skraba <[email protected]> wrote: > > Thanks so much for the feedback Martin, this sounds vaguely -1 ish :D > I'm trying to take a closer look. Being able to build Avro from > source is kind of important, but can also be kind of tricky (which is > why I end up relying on the docker build). I'm going through these to > see if I can reproduce and if they're blockers for the release. I'm > pretty confident in the Java and python builds, but less so for C, > PHP, and C# since I use them less personally! > > The first one (python), for example, is probably a tox setup issue > (and yet tox isn't listed as a requirement in BUILD.md). > > I propose extending the vote to clarify the build state for the > artifacts! Does that sound reasonable to the community? Does anybody > have any specific requests for timing? > > [BUILD.md]: https://github.com/apache/avro/blob/release-1.11.0-rc1/BUILD.md > > On Fri, Oct 8, 2021 at 2:13 PM Martin Grigorov <[email protected]> wrote: > > > > Hi, > > > > I haven't done this before, so I may miss something in the test steps. > > > > On my dev machine (Ubuntu 21.04 x86_64) the following tests fail: > > > > 1) cd lang/py && ./build.sh test > > > > Traceback (most recent call last): > > File "/usr/lib/python3.9/runpy.py", line 188, in _run_module_as_main > > mod_name, mod_spec, code = _get_module_details(mod_name, _Error) > > File "/usr/lib/python3.9/runpy.py", line 147, in _get_module_details > > return _get_module_details(pkg_main_name, error) > > File "/usr/lib/python3.9/runpy.py", line 111, in _get_module_details > > __import__(pkg_name) > > File "/home/martin/.local/lib/python3.9/site-packages/tox/__init__.py", > > line 32, in <module> > > from .session import cmdline # isort:skip > > File > > "/home/martin/.local/lib/python3.9/site-packages/tox/session/__init__.py", > > line 20, in <module> > > from tox import reporter > > File "/home/martin/.local/lib/python3.9/site-packages/tox/reporter.py", > > line 127, in <module> > > _INSTANCE = Reporter() > > File "/home/martin/.local/lib/python3.9/site-packages/tox/reporter.py", > > line 32, in __init__ > > self._reset(**kwargs) > > File "/home/martin/.local/lib/python3.9/site-packages/tox/reporter.py", > > line 38, in _reset > > self.tw = py.io.TerminalWriter() > > AttributeError: module 'py' has no attribute 'io' > > > > This passes on my Ubuntu 20.04.3 ARM64 machine, so I guess it is because of > > the Python3 version. > > > > 2) cd lang/c && ./build.sh test > > > > The following tests FAILED: > > 3 - memcheck_test_data_structures (Failed) > > 5 - memcheck_test_avro_schema (Failed) > > 7 - memcheck_test_avro_schema_names (Failed) > > 9 - memcheck_test_avro_values (Failed) > > 11 - memcheck_test_avro_766 (Failed) > > 13 - memcheck_test_avro_968 (Failed) > > 15 - memcheck_test_avro_984 (Failed) > > 17 - memcheck_test_avro_1034 (Failed) > > 19 - memcheck_test_avro_1084 (Failed) > > 21 - memcheck_test_avro_1087 (Failed) > > 23 - memcheck_test_avro_1165 (Failed) > > 25 - memcheck_test_avro_1167 (Failed) > > 27 - memcheck_test_avro_1237 (Failed) > > 29 - memcheck_test_avro_1238 (Failed) > > 31 - memcheck_test_avro_1279 (Failed) > > 33 - memcheck_test_avro_1405 (Failed) > > 35 - memcheck_test_avro_1572 (Failed) > > 38 - memcheck_test_refcount (Failed) > > 40 - memcheck_test_avro_1379 (Failed) > > 42 - memcheck_test_avro_1691 (Failed) > > 44 - memcheck_test_avro_1906 (Failed) > > 46 - memcheck_test_avro_1904 (Failed) > > Errors while running CTest > > make: *** [Makefile:104: test] Error 8 > > make: Leaving directory '/tmp/avro-1.11.0-rc1/avro-src-1.11.0/build/c' > > > > The tests pass on Linux ARM64. I am not sure what is the reason to fail > > locally. > > > > 3) cd lang/php && ./build.test > > > > ./build.sh test > > Composer is operating significantly slower than normal because you do not > > have the PHP curl extension enabled. > > No lock file found. Updating dependencies instead of installing from lock > > file. Use composer update over composer install if you do not have a lock > > file. > > Loading composer repositories with package information > > Updating dependencies > > Your requirements could not be resolved to an installable set of packages. > > > > Problem 1 > > - phpunit/phpunit[9.1.0, ..., 9.5.10] require ext-dom * -> it is > > missing from your system. Install or enable PHP's dom extension. > > - Root composer.json requires phpunit/phpunit ^9.1 -> satisfiable by > > phpunit/phpunit[9.1.0, ..., 9.5.10]. > > > > To enable extensions, verify that they are enabled in your .ini files: > > - /etc/php/7.4/cli/php.ini > > - /etc/php/7.4/cli/conf.d/10-opcache.ini > > - /etc/php/7.4/cli/conf.d/10-pdo.ini > > - /etc/php/7.4/cli/conf.d/20-calendar.ini > > - /etc/php/7.4/cli/conf.d/20-ctype.ini > > - /etc/php/7.4/cli/conf.d/20-exif.ini > > - /etc/php/7.4/cli/conf.d/20-ffi.ini > > - /etc/php/7.4/cli/conf.d/20-fileinfo.ini > > - /etc/php/7.4/cli/conf.d/20-ftp.ini > > - /etc/php/7.4/cli/conf.d/20-gettext.ini > > - /etc/php/7.4/cli/conf.d/20-iconv.ini > > - /etc/php/7.4/cli/conf.d/20-json.ini > > - /etc/php/7.4/cli/conf.d/20-mbstring.ini > > - /etc/php/7.4/cli/conf.d/20-phar.ini > > - /etc/php/7.4/cli/conf.d/20-posix.ini > > - /etc/php/7.4/cli/conf.d/20-readline.ini > > - /etc/php/7.4/cli/conf.d/20-shmop.ini > > - /etc/php/7.4/cli/conf.d/20-sockets.ini > > - /etc/php/7.4/cli/conf.d/20-sysvmsg.ini > > - /etc/php/7.4/cli/conf.d/20-sysvsem.ini > > - /etc/php/7.4/cli/conf.d/20-sysvshm.ini > > - /etc/php/7.4/cli/conf.d/20-tokenizer.ini > > You can also run `php --ini` inside terminal to see which files are used by > > PHP in CLI mode. > > > > Fails the same way on Linux ARM64. > > > > 4) cd lang/perl && ./build.sh test > > include /tmp/avro-src-1.11.0/lang/perl/inc/Module/Install.pm > > include inc/Module/Install/Metadata.pm > > include inc/Module/Install/Base.pm > > include inc/Module/Install/ReadmeFromPod.pm > > readme_from lib/Avro.pm to txt > > Unknown function is found at ./Makefile.PL line 51. > > Execution of ./Makefile.PL aborted due to runtime errors. > > > > If you're a contributor to a project, you may need to install > > some Module::Install extensions from CPAN (or other repository). > > If you're a user of a module, please contact the author. > > > > I've installed all dependencies from .github/workflows/test-lang-perl.yml > > before that > > > > 5) cd lang/csharp && ./build.sh test > > > > Testhost process exited with error: It was not possible to find any > > compatible framework version > > The framework 'Microsoft.NETCore.App', version '2.1.0' was not found. > > - The following frameworks were found: > > 5.0.1 at [/home/ubuntu/devel/dotnet/shared/Microsoft.NETCore.App] > > You can resolve the problem by installing the specified framework and/or > > SDK. > > The specified framework can be found at: > > - > > https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=2.1.0&arch=arm64&rid=ubuntu.20.04-arm64 > > . Please check the diagnostic logs for more information. > > > > Test Run Aborted. > > > > I won't be able to test it. I don't want to pollute my setup with old > > versions of .NET since I don't need it for anything else. > > > > The other modules' tests pass! > > > > Regards, > > Martin > > > > > > On Wed, Oct 6, 2021 at 7:52 PM Ryan Skraba <[email protected]> wrote: > > > > > Hi everyone, > > > > > > I'd like to propose the following RC1 to be released as the official > > > Apache > > > Avro 1.11.0 release. > > > > > > The commit id is 622343fdc705ccbc5a918f598d60a0df802119d8 > > > * This corresponds to the tag: release-1.11.0-rc1 > > > * https://github.com/apache/avro/releases/tag/release-1.11.0-rc1 > > > > > > The release tarball, signature, and checksums are here (revision 50295.) > > > * https://dist.apache.org/repos/dist/dev/avro/avro-1.11.0-rc1/ > > > > > > You can find the KEYS file here: > > > * https://dist.apache.org/repos/dist/dev/avro/KEYS > > > > > > Binary artifacts for Java are staged in Nexus here: > > > * > > > https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.11.0/ > > > > > > This release includes ~120 Jira issues: > > > * > > > https://issues.apache.org/jira/issues/?jql=project%3DAVRO%20AND%20fixVersion%3D1.11.0 > > > > > > I'll send a follow-up email with the notable highlights for each > > > language and SDK, but > > > I wanted to give everyone with their favourite fixed JIRAs and > > > features a head start! > > > > > > If I understand correctly, the rust source is present and in an > > > experimental state in > > > this release, but has not been versioned as 1.11.0 and the binary > > > artifacts will > > > not be distributed yet. > > > > > > Please download, verify, and test. This vote will remain open for at least > > > 72 hours. Given sufficient votes, I would like to close after the weekend > > > on > > > 12h UTC Monday, October 11th, 2021 > > > > > > [ ] +1 Release this as Apache Avro 1.11.0 > > > [ ] +0 > > > [ ] -1 Do not release this because... > > > > > > Best regards, > > > Ryan Skraba > > >
