antxxxx commented on issue #4003: [AIRFLOW-3163] add operator to enable setting 
table description in BigQuery table
URL: 
https://github.com/apache/incubator-airflow/pull/4003#issuecomment-427662813
 
 
   There is not a bigquery api exposed to set the description when you want to 
run a query and save the results to a table - which is the main reason I wanted 
this.   
   My flow is load data into staging tables -> aggregate data using sql and 
save the results to new table -> set table description on new table with 
metadata such as timestamp when aggregate was performed, source systems etc.    
  
   
   The other use case I want it for is to include metadata such as latest 
extract date or load date in history tables. This obviously changes each time 
new data is added to the table so can not be set at table creation time, and 
again there does not seem an api exposed to set description when adding data to 
an existing table.   
   
   I could change the existing operators to include a description parameters, 
and then call the hook to set the description after it has run, but it seemed 
simpler and more useful to add a new operator to set the description

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

Reply via email to