The GitHub Actions job "Tests" on airflow.git has failed. Run started by GitHub user potiuk (triggered by potiuk).
Head commit for run: be7cb1e837b875f44fcf7903329755245dd02dc3 / Benji Lampel <[email protected]> Common SQLCheckOperators Various Functionality Update (#25164) * Add batching to SQL Check Operators Commit adds a WHERE clause to the sql statement that allows for arbitrary batching in a given table. * Fix bug with multiple table checks When multiple table checks are given to the SQLTableCheckOperator and at least one is not a fully aggregate statement, a GROUP BY clause was previously needed. This commit updates the operator to use the get_pandas_df() method instead of _get_first() to return a pandas dataframe object that contains the check names and check results from the new style of query. The new style of query uses UNION ALL to run each test as its own SELECT statement, bypassing the need to do a GROUP BY. * Update test failure logic Changed name of method from _get_failed_tests to _get_failed_checks to better match naming, and updated logic of the method to include an optional column param. The query in the column check operator is removed from the failed test exception message, as it was only ever showing the last query, instead of the relevant one(s). This is replaced by the column, which will be more useful in debugging. * Add table alias to SQLTableCheckOperator query Without a table alias, the query does not run on Postgres and other databases. The alias is arbitrary and used only for proper query execution. * Fix formatting error in operator * Add batching to SQL Check Operators Commit adds a WHERE clause to the sql statement that allows for arbitrary batching in a given table. * Fix bug with multiple table checks When multiple table checks are given to the SQLTableCheckOperator and at least one is not a fully aggregate statement, a GROUP BY clause was previously needed. This commit updates the operator to use the get_pandas_df() method instead of _get_first() to return a pandas dataframe object that contains the check names and check results from the new style of query. The new style of query uses UNION ALL to run each test as its own SELECT statement, bypassing the need to do a GROUP BY. * Update test failure logic Changed name of method from _get_failed_tests to _get_failed_checks to better match naming, and updated logic of the method to include an optional column param. The query in the column check operator is removed from the failed test exception message, as it was only ever showing the last query, instead of the relevant one(s). This is replaced by the column, which will be more useful in debugging. * Add table alias to SQLTableCheckOperator query Without a table alias, the query does not run on Postgres and other databases. The alias is arbitrary and used only for proper query execution. * Fix formatting error in operator * Move alias to proper query build statement The table alias should be in the self.sql query build statement as that is where the table it needs to alias is defined. * Add batching to SQL Check Operators Commit adds a WHERE clause to the sql statement that allows for arbitrary batching in a given table. * Fix bug with multiple table checks When multiple table checks are given to the SQLTableCheckOperator and at least one is not a fully aggregate statement, a GROUP BY clause was previously needed. This commit updates the operator to use the get_pandas_df() method instead of _get_first() to return a pandas dataframe object that contains the check names and check results from the new style of query. The new style of query uses UNION ALL to run each test as its own SELECT statement, bypassing the need to do a GROUP BY. * Update test failure logic Changed name of method from _get_failed_tests to _get_failed_checks to better match naming, and updated logic of the method to include an optional column param. The query in the column check operator is removed from the failed test exception message, as it was only ever showing the last query, instead of the relevant one(s). This is replaced by the column, which will be more useful in debugging. * Add table alias to SQLTableCheckOperator query Without a table alias, the query does not run on Postgres and other databases. The alias is arbitrary and used only for proper query execution. * Fix formatting error in operator * Bug fixes and updates to test and operator Fixed bug in test where the dataframe column names did not match the operator's expected dataframe column names. Added more info to the SQLColumnCheckOperator's batch arg. Fixed the location of table aliasing in SQLTableCheckOperator. * Remove merge conflict lines * Rename parameter batch to partition_clause Gives a clearer name to the parameter and adds templating to the SQLTableCheckOperator. * Fix typo in docstring * Reformat operator file Report URL: https://github.com/apache/airflow/actions/runs/2719084822 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
