cloud-fan commented on code in PR #44689:
URL: https://github.com/apache/spark/pull/44689#discussion_r1452009435
##########
python/pyspark/sql/connect/dataframe.py:
##########
@@ -1719,14 +1724,31 @@ def __getitem__(self, item: Union[Column, List, Tuple])
-> "DataFrame":
def __getitem__(self, item: Union[int, str, Column, List, Tuple]) ->
Union[Column, "DataFrame"]:
if isinstance(item, str):
- # validate the column name
- if not hasattr(self._session, "is_mock_session"):
- self.select(item).isLocal()
-
- return _to_col_with_plan_id(
- col=item,
- plan_id=self._plan._plan_id,
- )
+ if item == "*":
+ return Column(
+ UnresolvedStar(
+ unparsed_target=None,
+ plan_id=self._plan._plan_id,
+ )
+ )
+ else:
+ # TODO: revisit vanilla Spark's Dataset.col
Review Comment:
We can probably skip it in spark connect. It's really a weird feature and
non-standard.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
