[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461024#comment-16461024
]
Matt Cheah commented on SPARK-24135:
I think we should not count these towards job failures, and that
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460047#comment-16460047
]
Matt Cheah commented on SPARK-24135:
_> But I'm not sure how much this buys us because very likely
[
https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459869#comment-16459869
]
Matt Cheah commented on SPARK-24137:
Benchmark results on this subject are here:
[
https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459860#comment-16459860
]
Matt Cheah commented on SPARK-24137:
[~foxish] [~liyinan926] for SA.
We actually did this on our
Matt Cheah created SPARK-24137:
--
Summary: [K8s] Mount temporary directories in emptydir volumes
Key: SPARK-24137
URL: https://issues.apache.org/jira/browse/SPARK-24137
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459754#comment-16459754
]
Matt Cheah commented on SPARK-24135:
[~foxish] [~eje] [~liyinan926] wanted to get feedback on this -
Matt Cheah created SPARK-24135:
--
Summary: [K8s] Executors that fail to start up because of
init-container errors are not retried and limit the executor pool size
Key: SPARK-24135
URL:
[
https://issues.apache.org/jira/browse/SPARK-24028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah resolved SPARK-24028.
Resolution: Cannot Reproduce
Closing this for now - we're continuing to investigate this
[
https://issues.apache.org/jira/browse/SPARK-24028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444898#comment-16444898
]
Matt Cheah commented on SPARK-24028:
[~liyinan926] just noticed your comment edit - agreed that the
[
https://issues.apache.org/jira/browse/SPARK-24028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444894#comment-16444894
]
Matt Cheah commented on SPARK-24028:
I don't think the 2.3.0 release creates any volume mounts except
[
https://issues.apache.org/jira/browse/SPARK-24028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444887#comment-16444887
]
Matt Cheah commented on SPARK-24028:
[~liyinan926] what specific point version of Kubernetes are you
Matt Cheah created SPARK-24028:
--
Summary: [K8s] Creating secrets and config maps before creating
the driver pod has unpredictable behavior
Key: SPARK-24028
URL: https://issues.apache.org/jira/browse/SPARK-24028
[
https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah resolved SPARK-23825.
Resolution: Fixed
Fix Version/s: 2.4.0
> [K8s] Spark pods should request memory +
[
https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414738#comment-16414738
]
Matt Cheah commented on SPARK-22839:
Design was proposed and agreed upon in
[
https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah reopened SPARK-22839:
Actually we haven't done the refactor in and of itself yet,
[
https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah resolved SPARK-22839.
Resolution: Fixed
Fix Version/s: 2.4.0
Done in https://github.com/apache/spark/pull/20669
[
https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289875#comment-16289875
]
Matt Cheah commented on SPARK-22778:
And then notice we don't even have a {{resources}} directory on
[
https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289871#comment-16289871
]
Matt Cheah commented on SPARK-22778:
I see the problem. We're missing the {{META-INF.services}} file
[
https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-22778:
---
Priority: Critical (was: Major)
> Kubernetes scheduler at master failing to run applications
[
https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289836#comment-16289836
]
Matt Cheah edited comment on SPARK-22778 at 12/13/17 8:28 PM:
--
The
[
https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289836#comment-16289836
]
Matt Cheah commented on SPARK-22778:
The `canCreate` method for `KubernetesClusterManager` should
[
https://issues.apache.org/jira/browse/SPARK-22778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289815#comment-16289815
]
Matt Cheah commented on SPARK-22778:
Think that URI should be 'k8s://https://xx.yy.zz.ww' - notice
[
https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-18278:
---
Attachment: (was: SPARK-18278 - Spark on Kubernetes Design Proposal.pdf)
> SPIP: Support native
[
https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879205#comment-15879205
]
Matt Cheah commented on SPARK-18278:
[~hkothari] I created SPARK-19700 to track the pluggable
Matt Cheah created SPARK-19700:
--
Summary: Design an API for pluggable scheduler implementations
Key: SPARK-19700
URL: https://issues.apache.org/jira/browse/SPARK-19700
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803050#comment-15803050
]
Matt Cheah commented on SPARK-18278:
I refactored the scheduler code as a thought experiment on what
[
https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15745944#comment-15745944
]
Matt Cheah commented on SPARK-18278:
[~rxin] - thanks for thinking about this!
The concerns around
[
https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716701#comment-15716701
]
Matt Cheah commented on SPARK-18278:
There has also been some discussion about this on the document
[
https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-18278:
---
Attachment: SPARK-18278 - Spark on Kubernetes Design Proposal.pdf
I attached a proposal outlining a
[
https://issues.apache.org/jira/browse/SPARK-13912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196366#comment-15196366
]
Matt Cheah commented on SPARK-13912:
Yup workarounds for me have been putting all spark.hadoop.*
[
https://issues.apache.org/jira/browse/SPARK-13912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195797#comment-15195797
]
Matt Cheah edited comment on SPARK-13912 at 3/15/16 5:54 PM:
-
it's not
[
https://issues.apache.org/jira/browse/SPARK-13912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195797#comment-15195797
]
Matt Cheah commented on SPARK-13912:
it's not exactly, if I'm reading the PR for SPARK-13403 right.
Matt Cheah created SPARK-13912:
--
Summary: spark.hadoop.* configurations are not applied for Parquet
Data Frame Readers
Key: SPARK-13912
URL: https://issues.apache.org/jira/browse/SPARK-13912
Project:
[
https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-13335:
---
Priority: Minor (was: Major)
> Optimize Data Frames collect_list and collect_set with declarative
[
https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148199#comment-15148199
]
Matt Cheah commented on SPARK-13335:
I have a prototypical patch for this and can submit a PR
[
https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-13335:
---
Summary: Optimize Data Frames collect_list and collect_set with declarative
aggregates (was:
[
https://issues.apache.org/jira/browse/SPARK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-13335:
---
Component/s: SQL
> Optimize collect_list and collect_set with declarative aggregates
>
Matt Cheah created SPARK-13335:
--
Summary: Optimize collect_list and collect_set with declarative
aggregates
Key: SPARK-13335
URL: https://issues.apache.org/jira/browse/SPARK-13335
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145752#comment-15145752
]
Matt Cheah commented on SPARK-12154:
Sorry this had to be pushed back - but I'll work on it in the
[
https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126818#comment-15126818
]
Matt Cheah commented on SPARK-12154:
I can look at this next week!
> Upgrade to Jersey 2
>
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042516#comment-15042516
]
Matt Cheah commented on SPARK-11081:
Upgrading to Jersey 2 definitely sounds more reasonable. Perhaps
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah resolved SPARK-11081.
Resolution: Not A Problem
Ok, I filed SPARK-12154 to track it.
> Make spark-core pull in Jersey
Matt Cheah created SPARK-12154:
--
Summary: Upgrade to Jersey 2
Key: SPARK-12154
URL: https://issues.apache.org/jira/browse/SPARK-12154
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037040#comment-15037040
]
Matt Cheah commented on SPARK-11081:
Quick question as I develop this as I'm fairly new to Maven -
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-11081:
---
Description:
As seen from this thread
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-11081:
---
Summary: Make spark-core pull in Jersey and javax.ws.rs dependencies
separately for easier
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-11081:
---
Description:
As seen from this thread
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-11081:
---
Description:
As seen from this thread
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-11081:
---
Description:
As seen from this thread
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-11081:
---
Description:
As seen from this thread
[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-11081:
---
Description:
As seen from this thread
[
https://issues.apache.org/jira/browse/SPARK-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994356#comment-14994356
]
Matt Cheah commented on SPARK-11516:
Ah event logging may be what I'm missing. Then again, if the
Matt Cheah created SPARK-11516:
--
Summary: Spark application cannot be found from JSON API even
though it exists
Key: SPARK-11516
URL: https://issues.apache.org/jira/browse/SPARK-11516
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961550#comment-14961550
]
Matt Cheah commented on SPARK-10877:
Is it this? https://github.com/apache/spark/pull/8987
>
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961507#comment-14961507
]
Matt Cheah commented on SPARK-10877:
[~davies] can you link the PR that resolved this?
> Assertions
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961457#comment-14961457
]
Matt Cheah commented on SPARK-10877:
The exception is occurring executor side and not in any code
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944002#comment-14944002
]
Matt Cheah commented on SPARK-10877:
Can you turn off assertions when you spawn the shell? Assertions
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944002#comment-14944002
]
Matt Cheah edited comment on SPARK-10877 at 10/5/15 8:48 PM:
-
Can you turn on
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944010#comment-14944010
]
Matt Cheah commented on SPARK-10877:
Is it possible that this error is JVM or platform dependent?
>
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944150#comment-14944150
]
Matt Cheah commented on SPARK-10877:
Does spark-submit enable assertions? I'm not sure how SBT passes
Matt Cheah created SPARK-10926:
--
Summary: Refactor ContextCleaner to allow weak reference cleaning
to be done outside of the driver
Key: SPARK-10926
URL: https://issues.apache.org/jira/browse/SPARK-10926
Matt Cheah created SPARK-10877:
--
Summary: Assertions fail straightforward DataFrame job
Key: SPARK-10877
URL: https://issues.apache.org/jira/browse/SPARK-10877
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-10877:
---
Summary: Assertions fail straightforward DataFrame job due to word
alignment (was: Assertions fail
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-10877:
---
Attachment: SparkFilterByKeyTest.scala
I've attached the Scala script that manifests the problem on
[
https://issues.apache.org/jira/browse/SPARK-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936259#comment-14936259
]
Matt Cheah commented on SPARK-10877:
Also the error doesn't occur if I turn code-generation off
[
https://issues.apache.org/jira/browse/SPARK-10568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901382#comment-14901382
]
Matt Cheah commented on SPARK-10568:
Actually I think this was handled by
[
https://issues.apache.org/jira/browse/SPARK-10568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901164#comment-14901164
]
Matt Cheah commented on SPARK-10568:
Yup!
> Error thrown in stopping one component in
[
https://issues.apache.org/jira/browse/SPARK-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-10570:
---
Issue Type: New Feature (was: Improvement)
> Add Spark version endpoint to standalone JSON API
>
Matt Cheah created SPARK-10568:
--
Summary: Error thrown in stopping one component in
SparkContext.stop() doesn't allow other components to be stopped
Key: SPARK-10568
URL:
[
https://issues.apache.org/jira/browse/SPARK-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-10570:
---
Component/s: Web UI
> Add Spark version endpoint to standalone JSON API
>
[
https://issues.apache.org/jira/browse/SPARK-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-10570:
---
Labels: api (was: )
> Add Spark version endpoint to standalone JSON API
>
[
https://issues.apache.org/jira/browse/SPARK-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741502#comment-14741502
]
Matt Cheah commented on SPARK-10570:
The master web UI has it I think, but only in the HTML.
> Add
[
https://issues.apache.org/jira/browse/SPARK-10568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741526#comment-14741526
]
Matt Cheah commented on SPARK-10568:
Also I think the YARN issue is something more suited for
[
https://issues.apache.org/jira/browse/SPARK-10568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-10568:
---
Description:
When I shut down a Java process that is running a SparkContext, it invokes a
shutdown
Matt Cheah created SPARK-10570:
--
Summary: Add Spark version endpoint to standalone JSON API
Key: SPARK-10570
URL: https://issues.apache.org/jira/browse/SPARK-10570
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-10568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741525#comment-14741525
]
Matt Cheah commented on SPARK-10568:
I'm more concerned about the general case which is that failing
[
https://issues.apache.org/jira/browse/SPARK-10568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741525#comment-14741525
]
Matt Cheah edited comment on SPARK-10568 at 9/11/15 8:56 PM:
-
I'm more
Matt Cheah created SPARK-10458:
--
Summary: Would like to know if a given Spark Context is stopped or
currently stopping
Key: SPARK-10458
URL: https://issues.apache.org/jira/browse/SPARK-10458
Project:
Matt Cheah created SPARK-10407:
--
Summary: Possible Stack-overflow using InheritableThreadLocal
nested-properties for SparkContext.localProperties
Key: SPARK-10407
URL:
[
https://issues.apache.org/jira/browse/SPARK-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-10407:
---
Description:
In my long-running web server that eventually uses a SparkContext, I eventually
came
Matt Cheah created SPARK-10374:
--
Summary: Spark-core 1.5.0-RC2 can create version conflicts with
apps depending on protobuf-2.4
Key: SPARK-10374
URL: https://issues.apache.org/jira/browse/SPARK-10374
[
https://issues.apache.org/jira/browse/SPARK-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723768#comment-14723768
]
Matt Cheah commented on SPARK-10374:
I intend to create a smaller standalone program that reproduces
[
https://issues.apache.org/jira/browse/SPARK-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723826#comment-14723826
]
Matt Cheah commented on SPARK-10374:
I'll try switching the Akka version pulled in by Spark and see
[
https://issues.apache.org/jira/browse/SPARK-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724568#comment-14724568
]
Matt Cheah commented on SPARK-10374:
Ok it works when we switch the Akka version back for our CDH4
Matt Cheah created SPARK-10250:
--
Summary: Scala PairRDDFuncitons.groupByKey() should be
fault-tolerant of single large groups
Key: SPARK-10250
URL: https://issues.apache.org/jira/browse/SPARK-10250
[
https://issues.apache.org/jira/browse/SPARK-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631820#comment-14631820
]
Matt Cheah commented on SPARK-5269:
---
Sweet - working with someone else on this actually,
[
https://issues.apache.org/jira/browse/SPARK-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630783#comment-14630783
]
Matt Cheah commented on SPARK-5269:
---
I'd be interested in working on this with
[
https://issues.apache.org/jira/browse/SPARK-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617202#comment-14617202
]
Matt Cheah commented on SPARK-7917:
---
Just wanted to clarify: Worker shutdown, or
[
https://issues.apache.org/jira/browse/SPARK-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617115#comment-14617115
]
Matt Cheah commented on SPARK-7917:
---
[~sowen] was there a patch specifically written in
[
https://issues.apache.org/jira/browse/SPARK-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617166#comment-14617166
]
Matt Cheah commented on SPARK-7917:
---
Definitely not 7503 - the PR there only did things
[
https://issues.apache.org/jira/browse/SPARK-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617166#comment-14617166
]
Matt Cheah edited comment on SPARK-7917 at 7/7/15 6:45 PM:
---
[
https://issues.apache.org/jira/browse/SPARK-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612676#comment-14612676
]
Matt Cheah commented on SPARK-5581:
---
I'd be interested in taking something like this on
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606561#comment-14606561
]
Matt Cheah commented on SPARK-8597:
---
I'm also concerned about the possibility that using
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603816#comment-14603816
]
Matt Cheah commented on SPARK-8597:
---
Cool, a coworker and I think we have something
[
https://issues.apache.org/jira/browse/SPARK-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603913#comment-14603913
]
Matt Cheah commented on SPARK-8167:
---
One thought is to have, whenever a task fails from
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601687#comment-14601687
]
Matt Cheah commented on SPARK-8597:
---
I did some more digging. The memory space is taken
[
https://issues.apache.org/jira/browse/SPARK-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601678#comment-14601678
]
Matt Cheah commented on SPARK-8167:
---
[~joshrosen] any thoughts on this?
Tasks that
[
https://issues.apache.org/jira/browse/SPARK-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600012#comment-14600012
]
Matt Cheah commented on SPARK-8167:
---
What's curious here as I'm trying to design this is
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah updated SPARK-8597:
--
Summary: DataFrame partitionBy memory pressure scales extremely poorly
(was: DataFrame partitionBy
Matt Cheah created SPARK-8597:
-
Summary: DataFrame partitionBy scales extremely poorly
Key: SPARK-8597
URL: https://issues.apache.org/jira/browse/SPARK-8597
Project: Spark
Issue Type: Bug
101 - 200 of 270 matches
Mail list logo
