[
https://issues.apache.org/jira/browse/BEAM-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-4735:
-------------------------------
Description:
BEAM-4020 introduces HBaseIO reads based on SDF. So far the read() method still
uses the Source based API because SDF does not support Dynamic Work Rebalancing
(DWR) but the Source API of HBase does, so changing it means losing some
functionality.
Since DWR is only supported by Dataflow once Dataflow supports SDF + DWR we
can move the main read() function to use the SDF API and remove the Source
based implementation. The rest of the runners already support Bounded based
Reads (like HBase based on SDF does) via a default translation without DWR.
was:
BEAM-4020 introduces HBaseIO reads based on SDF. So far the read() method still
uses the Source based API for two reasons:
1. Most distributed runners don't supports Bounded SDF today.
2. SDF does not support Dynamic Work Rebalancing but the Source API of HBase
already supports it so changing it means losing some functionality.
Once there is improvements in both (1) and (2) we should consider moving the
main read() function to use the SDF API and remove the Source based
implementation.
> Make HBaseIO.read() based on SDF by default
> -------------------------------------------
>
> Key: BEAM-4735
> URL: https://issues.apache.org/jira/browse/BEAM-4735
> Project: Beam
> Issue Type: Improvement
> Components: io-java-hbase
> Reporter: Ismaël Mejía
> Priority: Minor
>
> BEAM-4020 introduces HBaseIO reads based on SDF. So far the read() method
> still uses the Source based API because SDF does not support Dynamic Work
> Rebalancing (DWR) but the Source API of HBase does, so changing it means
> losing some functionality.
> Since DWR is only supported by Dataflow once Dataflow supports SDF + DWR we
> can move the main read() function to use the SDF API and remove the Source
> based implementation. The rest of the runners already support Bounded based
> Reads (like HBase based on SDF does) via a default translation without DWR.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
