[GitHub] spark issue #15043: [SPARK-17491] Close serialization stream to fix wrong an...

Sat, 10 Sep 2016 03:01:42 -0700 

Github user lins05 commented on the issue:

    https://github.com/apache/spark/pull/15043
  
    Did a simple test and it does fix the bug. One interesting thing is while 
records.count() returns a smaller number than the actual count, the spark UI 
still shows the correct records number, in my test case it's 2999808 v.s. 
300000.
    
    ![screen shot 2016-09-10 at 5 57 38 
pm](https://cloud.githubusercontent.com/assets/717363/18409696/70a407e0-7780-11e6-9f22-7b55c24b0595.png)
    
    ![screen shot 2016-09-10 at 5 58 26 
pm](https://cloud.githubusercontent.com/assets/717363/18409697/75f5254e-7780-11e6-98fd-5cae496f7c22.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to