BryanCutler commented on issue #20280: [SPARK-22232][PYTHON][SQL] Fixed Row pickling to include __from_dict__ flag URL: https://github.com/apache/spark/pull/20280#issuecomment-546150912 Yeah, this has been in my queue to look at again, but haven't been able to yet. Besides dropping support for python versions, is it possible to remove the need for sorting fields by requiring an OrderedDict be used for versions before 3.6 or something similar? On Thu, Oct 24, 2019, 3:16 AM Hyukjin Kwon <[email protected]> wrote: > *@HyukjinKwon* commented on this pull request. > ------------------------------ > > In python/pyspark/sql/types.py > <https://github.com/apache/spark/pull/20280#discussion_r338491481>: > > > @@ -1445,15 +1447,15 @@ def __new__(self, *args, **kwargs): > raise ValueError("Can not use both args " > "and kwargs to create Row") > if kwargs: > - # create row objects > + # create row object from named arguments, order not guaranteed so will be sorted > names = sorted(kwargs.keys()) > > I think Spark 2.5 idea looks almost failed (and I either don't personally > support that idea). This thing is rather legitimate to make it happen > between minor releases. Actually, 3.0 is getting close. We could just let > 3.0 release first and remove that sorting behaviour too optionally. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/20280?email_source=notifications&email_token=ABCTA5J4RET3FGOHSNSRW6LQQFYYHA5CNFSM4EMBIFR2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJCEDVA#discussion_r338491481>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABCTA5MQ3Q4TE35RBHRPFP3QQFYYHANCNFSM4EMBIFRQ> > . >
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
