Github user opme commented on the issue:
https://github.com/apache/spark/pull/14995
@witgo I have a Pyspark application that was failing in 3 different places
but is able to run without errors now. I'm glad for this patch as I am not
sure how I would have explained to my professors why the big data application I
chose to do my analysis has 32 bit limitations. This is my final project for a
Georgia Tech Big data class and I will write about the these limitations of
Spark in my paper. My app is called the Surgeon Scorecard and it computes
surgical complication rate for surgeons on the Medicare synthetic cms dataset
which is about 1.6 billion records. https://github.com/opme/SurgeonScorecard.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
