[
https://issues.apache.org/jira/browse/BEAM-8715?focusedWorklogId=482578&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-482578
]
ASF GitHub Bot logged work on BEAM-8715:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Sep/20 20:03
Start Date: 12/Sep/20 20:03
Worklog Time Spent: 10m
Work Description: zeidoo commented on pull request #12748:
URL: https://github.com/apache/beam/pull/12748#issuecomment-690946531
Per module breakdown of the high level changes.
!!! = someone should probably confirm this is alright.
!!! beam.sdks.java.extensions.avro: (New module)
- moved from core.coders: AvroCoder, AvroGenericCoder
- moved from core.io: AvroIO, AvroSchemaIOProvider, AvroSink, AvroSource,
ConstantAvroDestination, DynamicAvroDestinations, GenericRecordWriteConverter,
SerializableAvroCodecFactory
- moved from core.schemas: AvroByteBuddyUtils, AvroUtils, AvroRecordSchema
- moved from core.test: user.avsc, test.avsc, AvroCoderTest,
AvroCoderTestPojo, AvroIOTest, AvroSourceTest,
SerializableAvroCodecFactoryTest, AvroGenerators, AvroUtilsTest,
AvroSchemaCoderTest, AvroSchemaTest
beam:
- settings.gradle -> include ref to avro module
beam.examples.java:
- gradle dep on Avro module, needs Avro coder
- only imports changed
beam.examples.kotlin:
- only imports (1 instance) changed
beam.runners.core-construction-java:
- gradle dep on Avro module, needs AvroGenericCoder and AvroCoder (for tests
only)
- !!! AvroGenericCoderRegistrar uses the ClassLoader to check if the
AvroGenericCoder class can be loaded, otherwise DOES NOT try to load that class
beam.runners.google-cloud-dataflow-java:
- gradle dep on Avro module, needs AvroCoder and AvroReader
- only imports changed
beam.runners.spark:
- NO gradle dep on Avro module, depends on
beam.runners.google-cloud-dataflow-java which depends on Avro
- only imports changed (in 1 test file, uses AvroIO)
beam.sdks.java.core:
- !!! DefaultCoderProvider is now public (previously private)
- !!! BlockBasedReader is now public (previously was protected)
- !!! BlockBasedReader.isAtSplitPoint() is now public (previously was
protected)
- !!! CounterMarkCoder introduced, previously was serialized with AvroCoder
- !!! DefaultFilenamePolicy.constructName() is now public (previously
private)
- !!! FileIO.getEmptyMatchTreatment() and FileIO.getWatchInterval are now
public (previously private)
- !!! new MyValueACoder for CoderRegistryTest.MyValueA, previously using
AvroCoder
- !!! DefaultCoderTest Avro parts of these tests moved into the new Avro
module
- !!! TextIOWriteTest added a UserWriteTypeCoder for UserWriteType,
previously using AvroCoder
- !!! TextIOWriteTest.testDynamicDefaultFilenamePolicy, UserWriteTypeCoder
now registered on the pipeline, previously this wasn't needed
- !!! SchemasTest Avro parts of these tests moved into the new Avro module
- !!! LatestTest, replaced AvroCoder usages with StringUtf8Coder
beam.sdks.java.extensions.sketching:
- gradle dep on Avro module, needs AvroCoder for tests
beam.sdk.extensions.sql:
- gradle dep on Avro module, needs AvroIO and AvroSchemaIOProvider
beam.sdks.java.io.file-based-io:
- NO gradle dep on Avro module, indirect dep needed for AvroCoder and AvroIO
beam.sdks.java.io.google-cloud-platform:
- gradle dep on Avro module, needs AvroSource, AvroCoder and AvroUtils
- copy from core.test: user.avsc
beam.sdks.java.io.hadoop-format:
- gradle dep on Avro module, needs AvroCoder for tests only
beam.sdks.java.io.kafka:
- gradle dep on Avro module, and added applyAvroNature(), needs AvroCoder
- moved from core.test: user.avsc
beam.sdks.java.io.parquet:
- gradle dep on Avro module, needs AvroUtils and AvroCoder
beam.sdks.java.io.snowflake:
- gradle dep on Avro module, and added applyAvroNature(), needs AvroCoder
for tests only
- copy from core.test: user.avsc
beam.sdks.java.testing.expansion-service:
- gradle dep on Avro module, needs AvroCoder for tests only
beam.sdks.java.testing.nexmark:
- gradle dep on Avro module, needs AvroIO and AvroCoder
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 482578)
Time Spent: 4h 50m (was: 4h 40m)
> Beam Dependency Update Request: org.apache.avro:avro
> ----------------------------------------------------
>
> Key: BEAM-8715
> URL: https://issues.apache.org/jira/browse/BEAM-8715
> Project: Beam
> Issue Type: Sub-task
> Components: dependencies
> Reporter: Beam JIRA Bot
> Priority: P3
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> ------------------------- 2019-11-15 19:43:34.352607
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2019-11-19 21:10:39.325444
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2019-12-02 12:17:09.195761
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2019-12-09 12:16:24.532415
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2019-12-23 12:16:46.807801
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2019-12-30 14:11:57.626749
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-01-06 12:15:35.813790
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-01-13 12:14:55.385116
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-01-20 12:12:23.837063
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-01-27 12:13:20.278513
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-02-03 12:14:35.025849
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-02-10 12:12:18.571083
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.1
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-02-17 12:12:52.735152
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.2
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-02-24 12:12:23.741494
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.2
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-03-24 13:54:40.005377
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.2
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-03-30 12:12:21.888626
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.2
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-04-06 12:12:17.487906
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.2
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-04-13 12:10:27.219668
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.9.2
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-06-29 12:15:42.018789
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-07-06 12:14:34.172461
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-07-08 10:40:45.786925
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-07-13 12:16:44.119892
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-07-20 12:17:55.763569
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-07-27 12:21:35.098986
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-08-03 12:17:36.195129
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-08-10 12:16:03.914678
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-08-17 12:16:15.068188
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-08-24 12:18:53.260605
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-08-28 09:22:04.206034
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-08-28 15:59:07.193183
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-08-28 16:30:32.065005
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-08-31 12:20:01.249503
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
> ------------------------- 2020-09-07 12:19:22.657424
> -------------------------
> Please consider upgrading the dependency org.apache.avro:avro.
> The current version is 1.8.2. The latest version is 1.10.0
> cc:
> Please refer to [Beam Dependency Guide
> |https://beam.apache.org/contribute/dependencies/]for more information.
> Do Not Modify The Description Above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
