dosubot[bot] commented on issue #36292: URL: https://github.com/apache/superset/issues/36292#issuecomment-3581693016
<!-- Greeting --> Hi @shalom-t! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the superset team. <!-- Answer --> This issue happens because Superset handles "Custom SQL" filters (expressionType: SQL) differently from "Simple" filters. Custom SQL filters get placed in the `extras.where` field and are sanitized using the engine's dialect (via sqlglot), but this process relies on the user to provide correct quoting for Snowflake. If the identifier (like LOAN_ID) isn't double-quoted, Snowflake may reject the query or return no results, especially in aggregation mode. In contrast, "Simple" filters are structured and Superset automatically quotes identifiers correctly for Snowflake, so they work as expected. There isn't explicit logic in Superset to fix or re-quote custom SQL filters for Snowflake; the only enforcement is through the `sanitize_clause` function, which formats the clause but doesn't guarantee correct quoting if the input is not already quoted ["superset/utils/core.py"](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/core.py#L861-L1241), ["superset/sql/parse.py"](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/sql/parse.py#L29-L1446). **Workaround:** Whenever possible, use "Simple" filters for columns like LOAN_ID to ensure proper quoting and compatibility with Snowflake. If you must use a custom SQL filter, manually double-quote identifiers (e.g., `"LOAN_ID" IN ('YAPN927')`) in your SQL expression. However, even with manual quoting, there may be edge cases where the query builder logic or aggregation context causes issues, since custom SQL filters are injected as raw SQL. If you need more flexibility, consider creating a virtual dataset in SQL Lab with your custom filter logic, then build your chart on top of that dataset. Let me know if you need more details on the filter processing pipeline or want to discuss possible code-level changes! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/35655164-b51b-4b10-a1c6-e9b291997983?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/35655164-b51b-4b10-a1c6-e9b291997983?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/35655164-b51b-4b10-a1c6-e9b291997983?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/35655164-b51b-4b10-a1c6-e9b291997983?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/35655164-b51b-4b10-a1c6-e9b291997983?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/35655164-b51b-4b10-a1c6-e9b291997983?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/35655164-b51b-4b10-a1c6-e9b291997983?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=This%20issue%20happens%20because%20Superset%20handles%20%22Custom%20SQL%22%20filters%20%28expressionType%3A%20SQL%29%20differently%20from%20%22Simple%22%20filters.%20Custom%20SQL%20filters%20get%20placed%20in%20the%20%60extras.where%60%20field%20and%20are%20sanitized%20using%20the%20engine%27s%20dialect%20%28via%20sqlglot%29%2C%20but%20this%20process%20relies%20on%20the%20user%20to%20provide%20correct%20quoting%20for%20Snowflake.%20If%20the%20identifier%20%28like%20LOAN_ID%29%20isn%27t%20double-quoted%2C%20Snowflake%20may%20reject%20the%20query%20or%20return%20no%20results%2C%20especially%20in%20aggregation%20mode.%20In%20contrast%2C%20%22Simple%22%20filters%20are%20structured%20and%20Superset%20automatically%20quotes%20identifiers%20correctly%20fo r%20Snowflake%2C%20so%20they%20work%20as%20expected.%0A%0AThere%20isn%27t%20explicit%20logic%20in%20Superset%20to%20fix%20or%20re-quote%20custom%20SQL%20filters%20for%20Snowflake%3B%20the%20only%20enforcement%20is%20through%20the%20%60sanitize_clause%60%20function%2C%20which%20formats%20the%20clause%20but%20doesn%27t%20guarantee%20correct%20quoting%20if%20the%20input%20is%20not%20already%20quoted%20%5B%22superset/utils/core.py%22%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/utils/core.py%23L861-L1241%29%2C%20%5B%22superset/sql/parse.py%22%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/sql/parse.py%23L29-L1446%29.%0A%0A%2A%2AWorkaround%3A%2A%2A%20%20%0AWhenever%20possible%2C%20use%20%22Simple%22%20filters%20for%20columns%20like%20LOAN_ID%20to%20ensure%20proper%20quoting%20and%20compatibility%20with%20Snowflake.%20If%20you%20must%20use%20a%20custom%20SQL%20filter%2C%20manually%20doubl e-quote%20identifiers%20%28e.g.%2C%20%60%22LOAN_ID%22%20IN%20%28%27YAPN927%27%29%60%29%20in%20your%20SQL%20expression.%20However%2C%20even%20with%20manual%20quoting%2C%20there%20may%20be%20edge%20cases%20where%20the%20query%20builder%20logic%20or%20aggregation%20context%20causes%20issues%2C%20since%20custom%20SQL%20filters%20are%20injected%20as%20raw%20SQL.%0A%0AIf%20you%20need%20more%20flexibility%2C%20consider%20creating%20a%20virtual%20dataset%20in%20SQL%20Lab%20with%20your%20custom%20filter%20logic%2C%20then%20build%20your%20chart%20on%20top%20of%20that%20dataset.%0A%0ALet%20me%20know%20if%20you%20need%20more%20details%20on%20the%20filter%20processing%20pipeline%20or%20want%20to%20discuss%20possible%20code-level%20changes%21) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped% 20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/36292) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
