kapjoshi-cisco commented on issue #8160:
URL: https://github.com/apache/hudi/issues/8160#issuecomment-1465295990
Additionally im unable to drop columns using spark sql in hudi.
```bash
deleteQuery = f"ALTER TABLE {hudidb}.{hudi_table} DROP COLUMN
RegisteredDevice"
print('sql query ', deleteQuery)
spark.sql(deleteQuery)
spark_df = spark.sql(f"SELECT * FROM {hudidb}.{hudi_table}")
print("********************")
print(spark_df.show())
spark_df.printSchema()
print("********************")
....
....
# follows by hudi merge called
hudi_merge(spark_df, combined_conf)
this throws below mentioned error
```bash
2023-03-12 20:41:45,885 ERROR [main] glue.ProcessLauncher
(Logging.scala:logError(77)): Error from Python:Traceback (most recent call
last):
File "/tmp/second-delete-upsert.py", line 176, in <module>
main()
File "/tmp/second-delete-upsert.py", line 160, in main
start_merging()
File "/tmp/second-delete-upsert.py", line 143, in start_merging
spark.sql(deleteQuery)
File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/session.py",
line 1034, in sql
return DataFrame(self._jsparkSession.sql(sqlQuery), self)
File
"/opt/amazon/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", line
1321, in __call__
return_value = get_return_value(
File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line
196, in deco
raise converted from None
pyspark.sql.utils.AnalysisException: DROP COLUMN is only supported with v2
tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
