Github user sureshthalamati commented on the issue:
https://github.com/apache/spark/pull/15196
@srowen Thanks for the clarification. My understanding is because read
path takes query, "select * from $table where 1!=0 is preferred when it works
. Hope is that source will optimize the query , and does not execute the user
specified query completely.
getTableExistsQuery() is primarily used to check the table in the data
frame write path, there is no user specified query in this case. "select 1
from $table limit 1" performance might be ok. If we find select * from $table
where 1 !=0 is better, then this query can be changed; otherwise leaving it as
is may be the safest bet.
I did not see CassandraDialect in the change set. Are you planning to add
one ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
