potiuk commented on issue #38681: URL: https://github.com/apache/airflow/issues/38681#issuecomment-2034383160
Yes. I think I also would like to the bottom of this. I believe (after a quick look) snowlfake connector dropped pyarrow because it's well, optional, and it's very likely that what you are doing and the query you are running is triggering this error becasue of some implicit conversion of timestamp. I saw Snowflake suggesting FROM_TIMESTAMP and other ways of getting timestamp that does not require `pyarrow` to do some kind of conversion. So I think 1st you should take a look at the queries you use and see if by modifying your queries you could get rid of the pyarrow requirement - followin what snowflake did. I'd wait for your report on attempting to do so if latest version does not work you already. And even if there are cases where pyarrow might be useful, then at MOST this would call for an optional feature of the provider, not requirement. For example we could have `apache-airflow-providers-snowflake[pyarrow]` to install additionally pyarrow. This would be easy to do - we have mechanisms for that - and then you will have to install it this way in the future if you need pyarrow. But I think the questions above should be answered first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
