I'm trying to read data from a simple query in BigQuery and cannot get it to work.
I'm following the steps from this URL: https://beam.apache.org/documentation/io/built-in/google-bigquery/ The process is trying to query a very small table with one column and five records. I have this code from the URL: from apache_beam import pipeline import apache_beam as beam customer_id = ( pipeline | 'QueryTable' >> beam.io.ReadFromBigQuery( query='SELECT customer_id FROM [cs-clientu-ad00007609-sbx5615:cuwi_acq_int.jv_test_data]') # Each row is a dictionary where the keys are the BigQuery columns | beam.Map(lambda elem: elem['customer_id'])) Below is the error output: C:\Users\juan.vega\AppData\Local\Continuum\anaconda3\python.exe C:/Users/juan.vega/PycharmProjects/us_1715_BQ_Stg_to_Intg/us_1715_main.py Traceback (most recent call last): File "C:/Users/juan.vega/PycharmProjects/us_1715_BQ_Stg_to_Intg/us_1715_main.py", line 18, in <module> | beam.Map(lambda elem: elem['customer_id'])) File "C:\Users\juan.vega\AppData\Local\Continuum\anaconda3\lib\site-packages\apache_beam\transforms\ptransform.py", line 1092, in __ror__ return self.transform.__ror__(pvalueish, self.label) File "C:\Users\juan.vega\AppData\Local\Continuum\anaconda3\lib\site-packages\apache_beam\transforms\ptransform.py", line 609, in __ror__ for (ix, v) in enumerate(pvalues) File "C:\Users\juan.vega\AppData\Local\Continuum\anaconda3\lib\site-packages\apache_beam\transforms\ptransform.py", line 610, in <dictcomp> if not isinstance(v, pvalue.PValue) and v is not None File "C:\Users\juan.vega\AppData\Local\Continuum\anaconda3\lib\site-packages\apache_beam\transforms\core.py", line 3179, in __init__ self.values = tuple(values) TypeError: 'module' object is not iterable Process finished with exit code 1 I've tried many things and I really need help. Do you have sample code to simply query some data from BigQuery using Dataflow? Thanks. Classification: Schwab Internal
