[
https://issues.apache.org/jira/browse/BEAM-13416?focusedWorklogId=733023&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-733023
]
ASF GitHub Bot logged work on BEAM-13416:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Feb/22 12:04
Start Date: 25/Feb/22 12:04
Worklog Time Spent: 10m
Work Description: mosche opened a new pull request #16947:
URL: https://github.com/apache/beam/pull/16947
@echauchot @aromanenko-dev Not really a PR (yet), I was just looking into
understand usage of schemas vs coders better.
The (most) interesting part here is probably `SdkPojoSchema` that works for
all the generated AWS models in the SDK v2 (though, in some cases recursive
types are used which obviously impose a challenge).
I wasn't able to find a lot of good technical documentation (design
decisions) for schemas in Beam, though definitely see the value over low level
coders! It seems to make sense to bother users as little as possible with
coders and [not expose these on any public
API](https://docs.google.com/document/d/1V2FkGGunVgvLwi1dKHr-7mtDuwYjTuvuESV_oPzVnfQ/edit?resourcekey=0-KvfQq-5iCcMlu3f3MFJ-GQ#heading=h.xl97lw3dyot7)
(M11). Currently coders are all over the place for AWS IOs, with this it seems
feasible to drop them (or deprecate).
Though, I was wondering a couple of things:
- Looks like I cannot register a provider for an interface (`SdkPojo`), that
means I have to "announce" it globally using an auto service.
- Is there a way a Sink/ WriteIO can set the coder for it's input like that
can be done for a Source on the output? Looks like it's only possible to
register such a coder globally, regardless if that sink is used or not.
- I noticed that a RowCoder cannot be automatically inferred for a row
despite every row carrying its schema. That makes sense given runners may use
the schema ahead of time for query planning. But it made me wonder if there's
any relationship between the schema of the row coder and the runtime schema
enforced...
If you have any good resources on the topic, I'd be happy to hear ...
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] [**Choose
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more
tips on [how to make review process
smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit
[https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more
information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 733023)
Remaining Estimate: 0h
Time Spent: 10m
> SqsIO.write input / construction of Sqs request
> -----------------------------------------------
>
> Key: BEAM-13416
> URL: https://issues.apache.org/jira/browse/BEAM-13416
> Project: Beam
> Issue Type: Improvement
> Components: io-java-aws
> Reporter: Moritz Mack
> Priority: P2
> Labels: aws, aws-sdk-v2, sqs
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SqsIO.write should support arbitrary types T and be implemented in terms of a
> mapper function that converts T into an AWS request.
> The current implementation of write as
> {{PTransform<PCollection<SendMessageRequest>, PDone>}} is a rather poor
> choice. It may requires an additional unnecessary serialization &
> deserialisation round. But even worse, the current
> {{SendMessageRequestCoder}} does not support any advanced configuration
> beyond queueUrl and messageBody and will silently drop it without even
> warning the user.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
