[GitHub] [avro] tradercentric opened a new pull request, #2519: AVRO-3856: [C#] Fixing Newtonsoft usage in Schema.cs to support up to 64 level depth in Avro schema
tradercentric opened a new pull request, #2519: URL: https://github.com/apache/avro/pull/2519 ## What is the purpose of the change I am an end-user/developer to use Confluent Kafka to consume Avro message seems to be over 32 depth level. The producer/vendor cannot reduce the depth level of the Avro message.Confluent support has assisted me to open an issue with Apache Avro project. The issue is logged in https://issues.apache.org/jira/browse/AVRO-3856.I am in need of code fix so I can continue. The current code in Schema.cs parsing Avro schema using JObject.Parse(json), JArray.Parse(json) do not have option to override max depth level. On https://github.com/JamesNK/Newtonsoft.Json/pull/2904, Newtonsoft Author advised to use JObject.Load(Jsonreader), so we can set the MaxDepth in JsonReader once instantiated. In doing so, I also discovered JsonReader is over-counting the depths in Avro schema. A 32 level depth Avro schema can be counted as 92 depth levels in JsonReader.Load (calling the Push method). Here is my observation of over-counting in Jsonreader: Avro Schema Depth | JsonReader Depth Count 4 | 11 16 | 44 32 | 92 64 | 188 So to compensate, I had hardcoded JsonReader MaxDepth to 192 to support parsing of Avro Schema with 64 level depth. ## Verifying this change This change added tests and can be verified as follows: I added 3 test cases in SchemaTests.cs: Parse16DepthLevelSchemaTest Parse32DepthLevelSchemaTest Parse64DepthLevelSchemaTest To verify the fixes. ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [avro] dependabot[bot] opened a new pull request, #2518: Bump io.netty:netty-bom from 4.1.97.Final to 4.1.98.Final in /lang/java
dependabot[bot] opened a new pull request, #2518: URL: https://github.com/apache/avro/pull/2518 Bumps [io.netty:netty-bom](https://github.com/netty/netty) from 4.1.97.Final to 4.1.98.Final. Commits https://github.com/netty/netty/commit/656dadf053b4426bc3a820d9a05f60ce33159ad1;>656dadf [maven-release-plugin] prepare release netty-4.1.98.Final https://github.com/netty/netty/commit/651d7783a7b24fe6da16736d1b1972f8410f;>651d778 Upgrade to maven 3.9.4 (https://redirect.github.com/netty/netty/issues/13624;>#13624) https://github.com/netty/netty/commit/4e17ab6528d7b18236826b75cf55b0cf61b1a129;>4e17ab6 Filter out unresolved addresses when parsing resolv.conf (https://redirect.github.com/netty/netty/issues/13614;>#13614) https://github.com/netty/netty/commit/7317f566fddab0910b0405a1fb7f83483c0c4374;>7317f56 Cleanup docker images (https://redirect.github.com/netty/netty/issues/13620;>#13620) https://github.com/netty/netty/commit/709137e8821e84e98646de7b5b5ac1fd84451041;>709137e Revert HttpHeaderValidationUtil should reject chars past the 1 byte range (... https://github.com/netty/netty/commit/6cb6d38b34fa351ecd5268fd65f799b3eb0b5c8c;>6cb6d38 Clarify default TCP fallback behaviour (https://redirect.github.com/netty/netty/issues/13621;>#13621) https://github.com/netty/netty/commit/d453175611c7866db51f75cb98de135d9b15ea5b;>d453175 Revert Run action that frees up diskspace during workflows (https://redirect.github.com/netty/netty/issues/13572;>#13572) (https://redirect.github.com/netty/netty/issues/13617;>#13617) https://github.com/netty/netty/commit/f1fa227ddf675f055766d04900cce6804fd7f710;>f1fa227 Lazy tableSizeChangeOutput (https://redirect.github.com/netty/netty/issues/13589;>#13589) https://github.com/netty/netty/commit/27c7ce9dc4e602337ff31d53a60704676ad7c93d;>27c7ce9 Make CipherSuiteConverter also cache failures (https://redirect.github.com/netty/netty/issues/13591;>#13591) https://github.com/netty/netty/commit/ea801cf19773a0eeb1d9d342c4a17e23762fdf2c;>ea801cf Prevent classloader leak via JNI (https://redirect.github.com/netty/netty/issues/13584;>#13584) Additional commits viewable in https://github.com/netty/netty/compare/netty-4.1.97.Final...netty-4.1.98.Final;>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [avro] sarutak opened a new pull request, #2517: AVRO-3870: [Rust][Build] Speed up CI for Rust
sarutak opened a new pull request, #2517: URL: https://github.com/apache/avro/pull/2517 AVRO-3870 ## What is the purpose of the change This PR aims to make the CI for Rust faster. In the current master, there are something wrong about actions/cache in `test-lang-rust-ci.yml`. First, a directory `target` is tend to be cached but the path is wrong. The correct path is `lang/rust/target`, not `~/target`. Second, as of Rust 1.70.0, Cargo changes the way to download dependencies. https://blog.rust-lang.org/2023/06/01/Rust-1.70.0.html#sparse-by-default-for-cratesio So, it's better not to share the cache for `~/.cargo` ## Verifying this change Done by CI. ## Documentation - Does this pull request introduce a new feature? (no) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [avro] dependabot[bot] opened a new pull request, #2516: Bump org.xerial.snappy:snappy-java from 1.1.10.3 to 1.1.10.4 in /lang/java
dependabot[bot] opened a new pull request, #2516: URL: https://github.com/apache/avro/pull/2516 Bumps [org.xerial.snappy:snappy-java](https://github.com/xerial/snappy-java) from 1.1.10.3 to 1.1.10.4. Release notes Sourced from https://github.com/xerial/snappy-java/releases;>org.xerial.snappy:snappy-java's releases. v1.1.10.4 What's Changed Security Fix Fixed SnappyInputStream so as not to allocate too large memory when decompressing data with an extremely large chunk size by https://github.com/tunnelshade;>@tunnelshade (https://github.com/xerial/snappy-java/commit/9f8c3cf74223ed0a8a834134be9c917b9f10ceb5;>code change) This does not affect users only using Snappy.compress/uncompress methods Features feature: Upgrade the internal snappy version to 1.1.10 (1.1.8 was wrongly used before) by https://github.com/xerial;>@xerial in https://redirect.github.com/xerial/snappy-java/pull/508;>xerial/snappy-java#508 Support JDK21 (no internal change) Dependency Updates Update scalafmt-core to 3.7.11 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/485;>xerial/snappy-java#485 Update sbt to 1.9.3 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/483;>xerial/snappy-java#483 Update scalafmt-core to 3.7.12 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/487;>xerial/snappy-java#487 Bump actions/checkout from 3 to 4 by https://github.com/dependabot;>@dependabot in https://redirect.github.com/xerial/snappy-java/pull/502;>xerial/snappy-java#502 Update sbt to 1.9.4 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/496;>xerial/snappy-java#496 Update scalafmt-core to 3.7.14 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/501;>xerial/snappy-java#501 Update sbt to 1.9.6 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/505;>xerial/snappy-java#505 Update native libraries by https://github.com/github-actions;>@github-actions in https://redirect.github.com/xerial/snappy-java/pull/503;>xerial/snappy-java#503 Internal Updates Update airframe-log to 23.7.4 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/486;>xerial/snappy-java#486 Update airframe-log to 23.8.0 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/488;>xerial/snappy-java#488 Update sbt-scalafmt to 2.5.2 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/500;>xerial/snappy-java#500 Update airframe-log to 23.8.6 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/497;>xerial/snappy-java#497 Update sbt-scalafmt to 2.5.1 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/499;>xerial/snappy-java#499 Update airframe-log to 23.9.1 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/504;>xerial/snappy-java#504 Update airframe-log to 23.9.2 by https://github.com/xerial-bot;>@xerial-bot in https://redirect.github.com/xerial/snappy-java/pull/509;>xerial/snappy-java#509 Other Changes Update NOTICE by https://github.com/imsudiproy;>@imsudiproy in https://redirect.github.com/xerial/snappy-java/pull/492;>xerial/snappy-java#492 Full Changelog: https://github.com/xerial/snappy-java/compare/v1.1.10.3...v1.1.10.4;>https://github.com/xerial/snappy-java/compare/v1.1.10.3...v1.1.10.4 Commits https://github.com/xerial/snappy-java/commit/9f8c3cf74223ed0a8a834134be9c917b9f10ceb5;>9f8c3cf Merge pull request from GHSA-55g7-9cwv-5qfv https://github.com/xerial/snappy-java/commit/49d700175f18ed5f8c5d371b7c2f80c75979bd68;>49d7001 Update airframe-log to 23.9.2 (https://redirect.github.com/xerial/snappy-java/issues/509;>#509) https://github.com/xerial/snappy-java/commit/1f07c3182c2dc89d4226e9a6d8945b8458870a0a;>1f07c31 Update native libraries for f2e97f27be0dc6c691369040ba8a673bface484c (https://redirect.github.com/xerial/snappy-java/issues/503;>#503) https://github.com/xerial/snappy-java/commit/13f8db197c4c44f0b6a02240c04205e8362b8e62;>13f8db1 Update sbt to 1.9.6 (https://redirect.github.com/xerial/snappy-java/issues/505;>#505) https://github.com/xerial/snappy-java/commit/f2e97f27be0dc6c691369040ba8a673bface484c;>f2e97f2 feature: Upgrade the internal snappy version to 1.1.10 (1.1.8 was wrongly use... https://github.com/xerial/snappy-java/commit/98b22256fe4ed00ccaadd2dac98b1622563cc50b;>98b2225 Update airframe-log to 23.9.1
[jira] [Created] (AVRO-3870) Speed up CI for Rust
Kousuke Saruta created AVRO-3870: Summary: Speed up CI for Rust Key: AVRO-3870 URL: https://issues.apache.org/jira/browse/AVRO-3870 Project: Apache Avro Issue Type: Improvement Components: build, rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta In the current master, there are something wrong about actions/cache in `test-lang-rust-ci.yml`. First, a directory "target" is tend to be cached but the path is wrong. the correct path is "lang/rust/target", not "~/target". Second, as of Rust 1.70.0, Cargo changes the way to download dependencies. https://blog.rust-lang.org/2023/06/01/Rust-1.70.0.html#sparse-by-default-for-cratesio So, it's better not to share the cache for "~/.cargo" -- This message was sent by Atlassian Jira (v8.20.10#820010)
