liangz1 commented on a change in pull request #27565: [SPARK-30791] Dataframe
add sameSemantics and sementicHash method
URL: https://github.com/apache/spark/pull/27565#discussion_r379226694
##########
File path: python/pyspark/sql/dataframe.py
##########
@@ -2153,6 +2153,22 @@ def transform(self, func):
"should have been DataFrame." %
type(result)
return result
+ @since(3.0)
+ def sameSemantics(self, other):
+ """
+ Return true when the query plan of the given :class:`DataFrame` will
return the same
+ results as this :class:`DataFrame`.
+ """
+ return self._jdf.sameSemantics(other)
Review comment:
It becomes the same error as the second one:
```
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/Users/liang.zhang/work/repos/apache/spark/python/pyspark/sql/dataframe.py",
line 2162, in sameSemantics
return self._jdf.sameSemantics(other._jdf)
File
"/Users/liang.zhang/mypy3/lib/python3.7/site-packages/py4j/java_gateway.py",
line 1257, in __call__
answer, self.gateway_client, self.target_id, self.name)
File
"/Users/liang.zhang/work/repos/apache/spark/python/pyspark/sql/utils.py", line
98, in deco
return f(*a, **kw)
File
"/Users/liang.zhang/mypy3/lib/python3.7/site-packages/py4j/protocol.py", line
332, in get_return_value
format(target_id, ".", name, value))
py4j.protocol.Py4JError: An error occurred while calling o42.sameSemantics.
Trace:
py4j.Py4JException: Method sameSemantics([class
org.apache.spark.sql.Dataset]) does not exist
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
at py4j.Gateway.invoke(Gateway.java:274)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
