[
https://issues.apache.org/jira/browse/BEAM-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289655#comment-16289655
]
Solomon Duskis commented on BEAM-3342:
--------------------------------------
I started with a simple pipeline that writes to Cloud Bigtable via the
google.cloud bigtable package, which works locally with google.cloud installed,
but doesn't work when I use a dataflow runner. Here's what I get:
==
message: "Not processing workitem 2633526545277283048 since a deferred
exception was found: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 706, in run
self._load_main_session(self.local_staging_directory)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 446, in _load_main_session
pickler.load_session(session_file)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line
247, in load_session
return dill.load_session(file_path)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 363, in
load_session
module = unpickler.load()
File "/usr/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1133, in load_reduce
value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 767, in
_import_module
return getattr(__import__(module, None, None, [obj]), obj)
AttributeError: 'module' object has no attribute 'bigtable'
==
Can I use the standard google.cloud bigtable client? If so, how, and why don't
BigQuery and Storage use the google.cloud clients?
> Create a Cloud Bigtable Python connector
> ----------------------------------------
>
> Key: BEAM-3342
> URL: https://issues.apache.org/jira/browse/BEAM-3342
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Solomon Duskis
> Assignee: Ahmet Altay
>
> I would like to create a Cloud Bigtable python connector.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
