[
https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773002#comment-16773002
]
jack commented on AIRFLOW-3804:
-------------------------------
Performed some additional tests to see if the problem is in the hook or in the
operator. tried the following code:
{code:java}
import airflow
from airflow.hooks.mysql_hook import MySqlHook
mysql = MySqlHook(mysql_conn_id='mycon')
conn = mysql.get_conn()
cursor = conn.cursor()
sql = "SELECT * FROM table where modifiedTS>2000-01-01 00:00:00 and
modifiedTS<= 2019-02-04 13:55:21"
cursor.execute(sql){code}
This raises error as expected.
{code:java}
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in
defaulterrorhandler
raise errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL
syntax; check the manual that corresponds to your MySQL server version for the
right syntax to use near 'table where modifiedTS>2000-01-01 00:00:00 and
modifiedTS<= 2019-02-04 13:55:21' at line 1"){code}
This means that the MySqlToGoogleCloudStorageOperator ignore the error
produced by the hook.
I imagine that the problem is in *_query_mysql* function of
MySqlToGoogleCloudStorageOperator :
[https://github.com/apache/airflow/blob/master/airflow/contrib/operators/mysql_to_gcs.py#L122]
{code:java}
def _query_mysql(self):
"""
Queries mysql and returns a cursor to the results.
"""
mysql = MySqlHook(mysql_conn_id=self.mysql_conn_id)
conn = mysql.get_conn()
cursor = conn.cursor()
cursor.execute(self.sql)
return cursor{code}
It doesn't check if error was reported from the hook It simply perform return
of the result.
> MySqlToGoogleCloudStorageOperator success when it should fail
> -------------------------------------------------------------
>
> Key: AIRFLOW-3804
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3804
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: jack
> Priority: Major
>
> Testing the following query on MySqlToGoogleCloudStorageOperator.
>
> {code:java}
> SELECT * FROM table where modifiedTS>2000-01-01 00:00:00 and modifiedTS<=
> 2019-02-04 13:55:21{code}
>
> The operator runs smoothly and report success so airflow continue to execute
> the down stream of the operator.
> However this query is invalid.
> Running it on MySQL will give:
>
> {code:java}
> Error Code: 1064. You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near
> '00:00:00 and modifiedTS<= 2019-02-04 13:55:21' at line 11{code}
>
> The operator should have *FAILD* when running this query it has syntax error.
> There is probably a problem with how this operator treats the result of this
> query and confuses it with valid result of no rows returned.
>
> Not sure if it's related but I'm running the query with SQL file using :
> filename option of the operator.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
