@kaxil, I need your advice and review.. I have issues with names or if it's okay - I leave it as it is.
Main idea of operators - get possible to check BQ environment before pipeline work start, for example, if dataset not exist - I want to create it or alert support team with detailed issue. To get such information I need CheckDatasetOperator. And also get possible to check list of datasets, if I have a big env-prerequisites (several datasets using in one DAG). after logic discussion if all ok - I will add tests and info in docs. I have some doubts about keeping logic in operator (BigQueryDatasetsListCheckExistOperator) https://github.com/apache/incubator-airflow/pull/3894/files#diff-6424e11be84d0621ef24935634e3db55R683 If you could, provide some recomendations ok it or not and what are more good place to move it. [ Full content available at: https://github.com/apache/incubator-airflow/pull/3894 ] This message was relayed via gitbox.apache.org for [email protected]
