[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633719#comment-15633719
]
Nic Eggert commented on SPARK-15822:
Yes, I'm caching right before the action that causes the crash.
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633509#comment-15633509
]
Herman van Hovell commented on SPARK-15822:
---
Could you share your plan? Are you using caching?
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633487#comment-15633487
]
Nic Eggert commented on SPARK-15822:
I'm seeing the segmentation violation at
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633480#comment-15633480
]
Herman van Hovell commented on SPARK-15822:
---
What are you seeing? The InMemory cache issue or
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633231#comment-15633231
]
Nic Eggert commented on SPARK-15822:
I think I'm seeing this bug in 1.6.2 when using the G1 garbage
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15465353#comment-15465353
]
Ruben Hernando commented on SPARK-15822:
Created
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15465342#comment-15465342
]
Ruben Hernando commented on SPARK-15822:
Thank you!
I'll do it
> segmentation violation in
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15465338#comment-15465338
]
Herman van Hovell commented on SPARK-15822:
---
Could you open a new ticket for this one? It is
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15465335#comment-15465335
]
Herman van Hovell commented on SPARK-15822:
---
Thanks! Do you also have this problem when you
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15465316#comment-15465316
]
Ruben Hernando commented on SPARK-15822:
{quote}
== Parsed Logical Plan ==
'Project [*]
+-
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464962#comment-15464962
]
Herman van Hovell commented on SPARK-15822:
---
Could you share the query plan?
> segmentation
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464860#comment-15464860
]
Ruben Hernando commented on SPARK-15822:
I'm doing joins between SQL views. Data source is JDBC
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464861#comment-15464861
]
Ruben Hernando commented on SPARK-15822:
I'm doing joins between SQL views. Data source is JDBC
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464808#comment-15464808
]
Herman van Hovell commented on SPARK-15822:
---
[~rhernando] It should not be related to this one.
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464809#comment-15464809
]
Herman van Hovell commented on SPARK-15822:
---
[~rhernando] It should not be related to this one.
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464793#comment-15464793
]
Ruben Hernando commented on SPARK-15822:
Encountered a similar error (using 2.0.0):
{quote}#
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335099#comment-15335099
]
Apache Spark commented on SPARK-15822:
--
User 'hvanhovell' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334003#comment-15334003
]
Apache Spark commented on SPARK-15822:
--
User 'robbinspg' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333594#comment-15333594
]
Pete Robbins commented on SPARK-15822:
--
Tracking where the memory is being allocated it is
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333586#comment-15333586
]
Pete Robbins commented on SPARK-15822:
--
OK so I think I know what is happening!
In the following
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333462#comment-15333462
]
Pete Robbins commented on SPARK-15822:
--
Tracing through off heap memory allocation this looks like
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331580#comment-15331580
]
Pete Robbins commented on SPARK-15822:
--
I can also recreate this issue on Oracle JDK 1.8:
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331375#comment-15331375
]
Pete Robbins commented on SPARK-15822:
--
and the plan:
{noformat}
== Parsed Logical Plan ==
'Project
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331370#comment-15331370
]
Pete Robbins commented on SPARK-15822:
--
Chatting with [~hvanhovell] here is the current state. I can
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331371#comment-15331371
]
Pete Robbins commented on SPARK-15822:
--
The generated code is:
{code}
Top Arrival Carrier
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329474#comment-15329474
]
Pete Robbins commented on SPARK-15822:
--
modified app to remove .cache()'s and still get a segv on
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328200#comment-15328200
]
Herman van Hovell commented on SPARK-15822:
---
[~robbinspg] Could you try this without caching?
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327992#comment-15327992
]
Pete Robbins commented on SPARK-15822:
--
So this does seem to cause the NPE or SEGV intermittently,
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327703#comment-15327703
]
Herman van Hovell commented on SPARK-15822:
---
[~robbinspg] You can dump the plan to the console
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325761#comment-15325761
]
Pete Robbins commented on SPARK-15822:
--
has failed on latest Branch 2.0 and master. Currently using
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325759#comment-15325759
]
Pete Robbins commented on SPARK-15822:
--
The stack trace is taken earlier when I detect that the
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325567#comment-15325567
]
Herman van Hovell commented on SPARK-15822:
---
[~robbinspg] I have tried to reproduce the problem
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325465#comment-15325465
]
Adam Roberts commented on SPARK-15822:
--
I added a link above to the dataset, it's 658mb when
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325417#comment-15325417
]
Davies Liu commented on SPARK-15822:
The latest stacktrace is different than previous one, it seems
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325159#comment-15325159
]
Pete Robbins commented on SPARK-15822:
--
I am forcing a system dump when I detect a corrupt
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325100#comment-15325100
]
Herman van Hovell commented on SPARK-15822:
---
Using spark session:
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325093#comment-15325093
]
Pete Robbins commented on SPARK-15822:
--
How do I disable whole-stage codegen?
> segmentation
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324907#comment-15324907
]
Davies Liu commented on SPARK-15822:
SortMergeJoin assume that the keys do not have null in them (we
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324875#comment-15324875
]
Davies Liu commented on SPARK-15822:
Could you try to disable whole-stage codegen to see whether this
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324506#comment-15324506
]
Pete Robbins commented on SPARK-15822:
--
generated SMJ code from the stack:
{code}
public Object
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324475#comment-15324475
]
Adam Roberts commented on SPARK-15822:
--
Herman, here's the application, note my HashedRelation
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324455#comment-15324455
]
Herman van Hovell commented on SPARK-15822:
---
[~aroberts][~robbinspg] Can you share a
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324352#comment-15324352
]
Adam Roberts commented on SPARK-15822:
--
The comment for baseOffset which has the corrupt value
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324341#comment-15324341
]
Adam Roberts commented on SPARK-15822:
--
Note that this segv happens regardless of
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324331#comment-15324331
]
Pete Robbins commented on SPARK-15822:
--
I'm still looking into this tracing back through the code
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323781#comment-15323781
]
Apache Spark commented on SPARK-15822:
--
User 'hvanhovell' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320380#comment-15320380
]
Pete Robbins commented on SPARK-15822:
--
I'm investigating this and will attach the app and config
47 matches
Mail list logo
