[jira] [Commented] (BEAM-4735) Make HBaseIO.read() based on SDF by default
[ https://issues.apache.org/jira/browse/BEAM-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037345#comment-17037345 ] Ismaël Mejía commented on BEAM-4735: Since Dataflow is the only runner that implements Dynamic Work Rebalancing (DWR) once Dataflow supports DWR we can switch HBaseIO to it. > Make HBaseIO.read() based on SDF by default > --- > > Key: BEAM-4735 > URL: https://issues.apache.org/jira/browse/BEAM-4735 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Priority: Minor > > BEAM-4020 introduces HBaseIO reads based on SDF. So far the read() method > still uses the Source based API because SDF does not support Dynamic Work > Rebalancing (DWR) but the Source API of HBase does, so changing it means > losing some functionality. > Since DWR is only supported by Dataflow once Dataflow supports SDF + DWR we > can move the main read() function to use the SDF API and remove the Source > based implementation. The rest of the runners already support Bounded based > Reads (like HBase based on SDF does) via a default translation without DWR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-4735) Make HBaseIO.read() based on SDF by default
[ https://issues.apache.org/jira/browse/BEAM-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037344#comment-17037344 ] Ismaël Mejía commented on BEAM-4735: Thanks, is BEAM-4287 still a pending thing? > Make HBaseIO.read() based on SDF by default > --- > > Key: BEAM-4735 > URL: https://issues.apache.org/jira/browse/BEAM-4735 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Priority: Minor > > BEAM-4020 introduces HBaseIO reads based on SDF. So far the read() method > still uses the Source based API because SDF does not support Dynamic Work > Rebalancing (DWR) but the Source API of HBase does, so changing it means > losing some functionality. > Since DWR is only supported by Dataflow once Dataflow supports SDF + DWR we > can move the main read() function to use the SDF API and remove the Source > based implementation. The rest of the runners already support Bounded based > Reads (like HBase based on SDF does) via a default translation without DWR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-4735) Make HBaseIO.read() based on SDF by default
[ https://issues.apache.org/jira/browse/BEAM-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037335#comment-17037335 ] Luke Cwik commented on BEAM-4735: - https://issues.apache.org/jira/browse/BEAM-4737 > Make HBaseIO.read() based on SDF by default > --- > > Key: BEAM-4735 > URL: https://issues.apache.org/jira/browse/BEAM-4735 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Priority: Minor > > BEAM-4020 introduces HBaseIO reads based on SDF. So far the read() method > still uses the Source based API because SDF does not support Dynamic Work > Rebalancing (DWR) but the Source API of HBase does, so changing it means > losing some functionality. > Since DWR is only supported by Dataflow once Dataflow supports SDF + DWR we > can move the main read() function to use the SDF API and remove the Source > based implementation. The rest of the runners already support Bounded based > Reads (like HBase based on SDF does) via a default translation without DWR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-4735) Make HBaseIO.read() based on SDF by default
[ https://issues.apache.org/jira/browse/BEAM-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17033429#comment-17033429 ] Ismaël Mejía commented on BEAM-4735: [~lcwik] is there any Jira tracking Dynamic Work Rebalancing on Google Dataflow for SplittableDoFn? > Make HBaseIO.read() based on SDF by default > --- > > Key: BEAM-4735 > URL: https://issues.apache.org/jira/browse/BEAM-4735 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Priority: Minor > > BEAM-4020 introduces HBaseIO reads based on SDF. So far the read() method > still uses the Source based API because SDF does not support Dynamic Work > Rebalancing (DWR) but the Source API of HBase does, so changing it means > losing some functionality. > Since DWR is only supported by Dataflow once Dataflow supports SDF + DWR we > can move the main read() function to use the SDF API and remove the Source > based implementation. The rest of the runners already support Bounded based > Reads (like HBase based on SDF does) via a default translation without DWR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
