Ruslan Dautkhanov created SPARK-21488:
-----------------------------------------
Summary: Make saveAsTable() return dataframe of created table
Key: SPARK-21488
URL: https://issues.apache.org/jira/browse/SPARK-21488
Project: Spark
Issue Type: Improvement
Components: PySpark, Spark Core, SQL
Affects Versions: 2.2.0
Reporter: Ruslan Dautkhanov
It would be great to make saveAsTable() return dataframe of created table,
so you could pipe result further as for example
{code}
(sqlc.sql('''
SELECT ...
FROM
''')
.write.format("parquet").mode("overwrite")
.saveAsTable('test.parquet_table')
.createOrReplaceTempView('mv_table')
)
{code}
... Above code returns now expectedly:
{noformat}
AttributeError: 'NoneType' object has no attribute 'createOrReplaceTempView'
{noformat}
If this is implemented, we can skip a step like
{code}
sqlc.sql('SELECT * FROM test.parquet_table').createOrReplaceTempView('mv_table')
{code}
We have this pattern very frequently.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
