[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948511#comment-14948511
]
Sean Owen commented on SPARK-10914:
---
I don't think having it on one machine necessarily matters. You
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948492#comment-14948492
]
Ben Moran commented on SPARK-10914:
---
I just tried moving the master to the worker box, so it's entirely
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948486#comment-14948486
]
Sean Owen commented on SPARK-10914:
---
Still kind of guessing here... but what if the problem is that the
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948460#comment-14948460
]
Ben Moran commented on SPARK-10914:
---
On latest master for me .count() also always seems to return 5 for
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948513#comment-14948513
]
Ben Moran commented on SPARK-10914:
---
I think you've got it - if I also turn off UseCompressedOops for
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948438#comment-14948438
]
Sean Owen commented on SPARK-10914:
---
I ran the latest master in standalone mode, with
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948494#comment-14948494
]
Ben Moran commented on SPARK-10914:
---
Either using the large heap, or -XX:-UseCompressedOops triggers
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948464#comment-14948464
]
Ben Moran commented on SPARK-10914:
---
I also don't see it if I run spark-shell without setting --master.
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947711#comment-14947711
]
Reynold Xin commented on SPARK-10914:
-
I don't think size estimator would impact the result.
If I
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946623#comment-14946623
]
Ben Moran commented on SPARK-10914:
---
I added a properties file to set
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947307#comment-14947307
]
Sean Owen commented on SPARK-10914:
---
It fails with a small heap, right -- if you set
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945443#comment-14945443
]
Ben Moran commented on SPARK-10914:
---
Ah, I did it the wrong way around! With a *small* heap and the
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945638#comment-14945638
]
Sean Owen commented on SPARK-10914:
---
Hm, it could be a valid lead after all. The size estimator code is
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945436#comment-14945436
]
Sean Owen commented on SPARK-10914:
---
Sure, pass -XX:-UseCompressedOops. So if it fails with a small
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945358#comment-14945358
]
Ben Moran commented on SPARK-10914:
---
Thanks for looking into it. I have narrowed it down a lot now.
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945441#comment-14945441
]
Ben Moran commented on SPARK-10914:
---
I just ran with
--executor-memory 100g --conf
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945417#comment-14945417
]
Sean Owen commented on SPARK-10914:
---
One long-shot question -- can you figure out whether your JVM is
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945426#comment-14945426
]
Ben Moran commented on SPARK-10914:
---
java -XX:+PrintFlagsFinal says:
bool UseCompressedOops
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945418#comment-14945418
]
Sean Owen commented on SPARK-10914:
---
One long-shot question -- can you figure out whether your JVM is
[
https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943855#comment-14943855
]
Wenchen Fan commented on SPARK-10914:
-
hi [~benm], I can't reproduce this bug on spark
20 matches
Mail list logo
