Jonathan Dayton created SPARK-57233:
---------------------------------------
Summary: SparkSession.sql() type hint is incomplete
Key: SPARK-57233
URL: https://issues.apache.org/jira/browse/SPARK-57233
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 3.5.8
Reporter: Jonathan Dayton
Spark 3.5 added support for positional parameters in {{SparkSession.sql()}}'s
{{args}} parameter. The type hint for positional parameters is a
non-subscripted container ({{List}} instead of {{List[ElementType]}}). As a
result, the type of this function is partially unknown, which causes an error
in Pylance (strict mode) anytime this function is used.
{code:python}
from pyspark.sql import SparkSession
def repro(spark: SparkSession):
df = spark.sql("SELECT * FROM my_table") # Error: Type of "sql" is
partially unknown
df.show()
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
