Hi,

I have the following BigQuery table:

Name: DailyVwap
Field name Type
TIMESTAMP       DATE    
SYMBOL_NAME     STRING
DAILY_VWAP      STRING  
inserted_time   TIMESTAMP

I want to read it into the following proto:
message DailyVwap {
    google.protobuf.Timestamp TIMESTAMP = 1;
    string SYMBOL_NAME = 2;
    float DAILY_VWAP = 3;
    google.protobuf.Timestamp inserted_time = 4;
}

This is the call that I make in my pipeline:
ReadFromBigQuery(query='SELECT * FROM `my_project.my_dataset.DailyVwap`', 
                    use_standard_sql=True, 
                    project='my_project',
                    coder=beam.coders.ProtoCoder(DailyVwap().__class__),
                    gcs_location=temp_location)

But the result is always a dictionary in the form:
{'TIMESTAMP': datetime.date(2021, 9, 17), 'SYMBOL_NAME': 'AACIU', 'DAILY_VWAP': 
'null', 'inserted_time': datetime.datetime(2021, 9, 27, 16, 45, 33, 779000, 
tzinfo=datetime.timezone.utc)}

With or without the coder in the call. No error message or warning in the logs.

Any help our pointer appreciated!

Thanks
     Mark

Reply via email to