[
https://issues.apache.org/jira/browse/BEAM-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Logan HAUSPIE updated BEAM-5676:
--------------------------------
Description:
When we use PubSub to push or pull messages, we currently have no choice of
serialization implementation because PubsubIO use internaly the
*_PubsubJsonClient.FACTORY_* _of type_ _*PubsubJsonClientFactory*_ _to
serialize deserialize messages_.
It should be nice to be able to set a different factory (e.g.
*_PubsubGrpcClientFactory_*) to decrease the size of messages (and then
decrease the price of using Pubsub).
I guess It could be possible to write something like:
{{PubsubIO.Write<PubsubMessage> write =}}
{{ PubsubIO.writeMessages()}}
{{ .to("projects/project/topics/topic")}}
{{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
{{ .withTimestampAttribute("timestamp")}}{{;}}
or
{{PubsubIO.Read<PubsubMessage> read = }}
{{ PubsubIO.readMessages()}}
{{ .fromSubscription("projects/project/subscriptions/name")}}
{{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
{{ .withTimestampAttribute("timestamp");}}
I feel confident to do it if needed.
was:
When we use PubSub to push or pull messages, we currently have no choice of
serialization implementation because PubsubIO use internaly the
*_PubsubJsonClient.FACTORY_* _of type_ *__* *_PubsubJsonClientFactory_* _to
serialize deserialize messages_.
It should be nice to be able to set a different factory (e.g.
*_PubsubGrpcClientFactory_*) to decrease the size of messages (and then
decrease the price of using Pubsub).
I guess It could be possible to write something like:
{{PubsubIO.Write<PubsubMessage> write =}}
{{ PubsubIO.writeMessages()}}
{{ .to("projects/project/topics/topic")}}
{{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
{{ .withTimestampAttribute("timestamp")}}{{;}}
or
{{PubsubIO.Read<PubsubMessage> read = }}
{{ PubsubIO.readMessages()}}
{{ .fromSubscription("projects/project/subscriptions/name")}}
{{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
{{ .withTimestampAttribute("timestamp");}}
I feel confident to do it if needed.
> Allow to change the PubsubClientFactory when using PubsubIO
> -----------------------------------------------------------
>
> Key: BEAM-5676
> URL: https://issues.apache.org/jira/browse/BEAM-5676
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Affects Versions: 2.7.0
> Reporter: Logan HAUSPIE
> Assignee: Chamikara Jayalath
> Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When we use PubSub to push or pull messages, we currently have no choice of
> serialization implementation because PubsubIO use internaly the
> *_PubsubJsonClient.FACTORY_* _of type_ _*PubsubJsonClientFactory*_ _to
> serialize deserialize messages_.
> It should be nice to be able to set a different factory (e.g.
> *_PubsubGrpcClientFactory_*) to decrease the size of messages (and then
> decrease the price of using Pubsub).
>
> I guess It could be possible to write something like:
> {{PubsubIO.Write<PubsubMessage> write =}}
> {{ PubsubIO.writeMessages()}}
> {{ .to("projects/project/topics/topic")}}
> {{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
> {{ .withTimestampAttribute("timestamp")}}{{;}}
> or
> {{PubsubIO.Read<PubsubMessage> read = }}
> {{ PubsubIO.readMessages()}}
> {{ .fromSubscription("projects/project/subscriptions/name")}}
> {{ *.withFactory(PubsubGrpcClient.FACTORY)*}}
> {{ .withTimestampAttribute("timestamp");}}
>
> I feel confident to do it if needed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
