[
https://issues.apache.org/jira/browse/BEAM-4543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16516984#comment-16516984
]
Valentyn Tymofieiev commented on BEAM-4543:
-------------------------------------------
I looked a little bit into what it will take to migrate from googledatastore
(Old Client) to google-cloud-datastore (New Client), in particular how we use
googledatastore in DatastoreIO and how both clients differ.
# Old Client is using oauth2client for authentication [1] while New Client is
using google-auth[2].
# DatastoreIO uses run_query() and commit() methods of Datastore client. There
is some difference in how we trigger these methods using Old and New clients.
The difference appears to be syntactic:
Old Client takes a RunQueryRequest[3], while New Client takes fields that
constitute the request[4].
# DatastoreIO uses a few functions from helper.py module that belongs to the
old client[5], which I did not see in the new client. Example: [6]. These
functions help manimulate Datastore protos. I think we can adopt relevant
helper functions from googledatastore/helper.py into Beam codebase.
[1]
https://github.com/GoogleCloudPlatform/google-cloud-datastore/blob/b760d0e8767d60a3d169b6f9de7cdb81a966a308/python/googledatastore/connection.py#L45
[2]
https://github.com/GoogleCloudPlatform/google-cloud-python/blob/ba619be365bfc17c3de11b295940d6479e7cf9f6/datastore/google/cloud/datastore_v1/gapic/datastore_client.py#L69
[3]
https://github.com/GoogleCloudPlatform/google-cloud-datastore/blob/b760d0e8767d60a3d169b6f9de7cdb81a966a308/python/googledatastore/connection.py#L98
[4]
https://github.com/GoogleCloudPlatform/google-cloud-python/blob/ba619be365bfc17c3de11b295940d6479e7cf9f6/datastore/google/cloud/datastore_v1/gapic/datastore_client.py#L286
[5]
https://github.com/GoogleCloudPlatform/google-cloud-datastore/blob/master/python/googledatastore/helper.py
[6]
https://github.com/apache/beam/blob/e9cfd3e17bebab4e19fb4480b89d0cbed57130e3/sdks/python/apache_beam/io/gcp/datastore/v1/datastoreio.py#L265
> Remove dependency on googledatastore in favor of google-cloud-datastore.
> ------------------------------------------------------------------------
>
> Key: BEAM-4543
> URL: https://issues.apache.org/jira/browse/BEAM-4543
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Valentyn Tymofieiev
> Priority: Minor
>
> apache-beam[gcp] package depends [1] on googledatastore package [2]. We
> should replace this dependency with google-cloud-datastore [3] which is
> officially supported, has better release cadence and also has Python 3
> support.
> [1]
> https://github.com/apache/beam/blob/fad655462f8fadfdfaab0b7a09cab538f076f94e/sdks/python/setup.py#L126
> [2] [https://pypi.org/project/googledatastore/]
> [3] [https://pypi.org/project/google-cloud-datastore/]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
