[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Description:
Problem: in Datasets API, it is a very common pattern to do something like
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Description:
Problem: in Datasets API, it is a very common pattern to do something like
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Description:
Problem: in Datasets API, it is a very common pattern to do something like
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Description:
Problem: in Datasets API, it is a very common pattern to do something like
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Issue Type: New Feature (was: Improvement)
> reduceGroupValues function for
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157128#comment-17157128
]
Martin Loncaric edited comment on SPARK-31356 at 7/19/20, 1:07 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157128#comment-17157128
]
Martin Loncaric edited comment on SPARK-31356 at 7/19/20, 1:07 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Description:
Problem: in Datasets API, it is a very common pattern to do something like
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Summary: reduceGroupValues function for KeyValueGroupedDataset (was:
reduceGroupValues
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Summary: reduceGroupValues function for Dataset (was: Splitting Aggregate
node into
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157128#comment-17157128
]
Martin Loncaric commented on SPARK-31356:
-
Actually, there seem to be 3 separate performance
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Summary: Splitting Aggregate node into separate Aggregate and Serialize for
Optimizer
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Description:
Problem: in Datasets API, it is a very common pattern to do something like
[
https://issues.apache.org/jira/browse/SPARK-31356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-31356:
Description:
Problem: in Datasets API, it is a very common pattern to do something like
Martin Loncaric created SPARK-31356:
---
Summary: KeyValueGroupedDataset method to reduce and take values
only
Key: SPARK-31356
URL: https://issues.apache.org/jira/browse/SPARK-31356
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853245#comment-16853245
]
Martin Loncaric commented on SPARK-27098:
-
After upgrading to Hadoop 2.9 and using
[
https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848516#comment-16848516
]
Martin Loncaric commented on SPARK-26192:
-
[~dongjoon] I made a PR:
[
https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804294#comment-16804294
]
Martin Loncaric edited comment on SPARK-27098 at 3/28/19 8:53 PM:
--
An
[
https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804294#comment-16804294
]
Martin Loncaric commented on SPARK-27098:
-
An INFO of this format also appears for each missing
[
https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793082#comment-16793082
]
Martin Loncaric edited comment on SPARK-27098 at 3/14/19 9:28 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793082#comment-16793082
]
Martin Loncaric commented on SPARK-27098:
-
[~ste...@apache.org] Does this make more sense to
[
https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793080#comment-16793080
]
Martin Loncaric commented on SPARK-27098:
-
I've gotten the debug logs for (1.), but can't make
[
https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27098:
Attachment: sanitized_stdout_1.txt
> Flaky missing file parts when writing to Ceph
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792240#comment-16792240
]
Martin Loncaric commented on SPARK-26555:
-
This is an existing issue with scala:
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792086#comment-16792086
]
Martin Loncaric edited comment on SPARK-26555 at 3/13/19 8:26 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792086#comment-16792086
]
Martin Loncaric commented on SPARK-26555:
-
Update: I have been able to replicate this without
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788773#comment-16788773
]
Martin Loncaric edited comment on SPARK-26555 at 3/9/19 7:04 PM:
-
You
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788773#comment-16788773
]
Martin Loncaric commented on SPARK-26555:
-
You can literally try any dataset and replicate this
Martin Loncaric created SPARK-27098:
---
Summary: Flaky missing file parts when writing to Ceph without
error
Key: SPARK-27098
URL: https://issues.apache.org/jira/browse/SPARK-27098
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783944#comment-16783944
]
Martin Loncaric commented on SPARK-27015:
-
Created a PR:
[
https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783810#comment-16783810
]
Martin Loncaric commented on SPARK-26192:
-
[~dongjoon] Thanks, I will pay more attention to
[
https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783810#comment-16783810
]
Martin Loncaric edited comment on SPARK-26192 at 3/4/19 9:51 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-27014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783811#comment-16783811
]
Martin Loncaric commented on SPARK-27014:
-
Sure, will keep that in mind.
> Support removal of
[
https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783810#comment-16783810
]
Martin Loncaric edited comment on SPARK-26192 at 3/4/19 9:49 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Affects Version/s: (was: 2.5.0)
(was: 3.0.0)
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Fix Version/s: 3.0.0
2.5.0
> spark-submit does not properly escape
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Affects Version/s: (was: 2.3.3)
(was: 2.4.0)
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782821#comment-16782821
]
Martin Loncaric commented on SPARK-26555:
-
Yes - when I take away any randomness and use the
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782821#comment-16782821
]
Martin Loncaric edited comment on SPARK-26555 at 3/3/19 6:56 PM:
-
Yes -
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782258#comment-16782258
]
Martin Loncaric commented on SPARK-26555:
-
I can also replicate with different schemas
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782239#comment-16782239
]
Martin Loncaric edited comment on SPARK-26555 at 3/2/19 1:25 AM:
-
I was
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782239#comment-16782239
]
Martin Loncaric commented on SPARK-26555:
-
I was able to replicate with both all rows as
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782185#comment-16782185
]
Martin Loncaric commented on SPARK-26555:
-
Will try it out and report back
> Thread safety
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Issue Type: Bug (was: New Feature)
> spark-submit does not properly escape arguments
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27014:
Fix Version/s: 3.0.0
2.5.0
> Support removal of jars and Spark
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
[
https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27015:
Description:
Arguments sent to the dispatcher must be escaped; for instance,
Martin Loncaric created SPARK-27015:
---
Summary: spark-submit does not properly escape arguments sent to
Mesos dispatcher
Key: SPARK-27015
URL: https://issues.apache.org/jira/browse/SPARK-27015
[
https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26192:
Fix Version/s: 3.0.0
2.4.1
2.3.4
>
[
https://issues.apache.org/jira/browse/SPARK-27014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-27014:
Affects Version/s: (was: 2.4.0)
2.5.0
> Support removal of
Martin Loncaric created SPARK-27014:
---
Summary: Support removal of jars and Spark binaries from Mesos
driver and executor sandboxes
Key: SPARK-27014
URL: https://issues.apache.org/jira/browse/SPARK-27014
[
https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26192:
Description:
There is at least one option accessed in MesosClusterScheduler that should
[
https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26192:
Description:
There are at least two options accessed in MesosClusterScheduler that should
[
https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26192:
Description:
There are at least two options accessed in MesosClusterScheduler that should
Martin Loncaric created SPARK-26555:
---
Summary: Thread safety issue causes createDataset to fail with
misleading errors
Key: SPARK-26555
URL: https://issues.apache.org/jira/browse/SPARK-26555
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26555:
Description:
This can be replicated (~2% of the time) with
{code:scala}
import
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26555:
Description:
This can be replicated (~2% of the time) with
{code:scala}
import
[
https://issues.apache.org/jira/browse/SPARK-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26555:
Description:
This can be replicated (~2% of the time) with
{code:scala}
import
Martin Loncaric created SPARK-26192:
---
Summary: MesosClusterScheduler reads options from dispatcher conf
instead of submission conf
Key: SPARK-26192
URL: https://issues.apache.org/jira/browse/SPARK-26192
[
https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26082:
Description:
Currently in
[
https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26082:
Description:
Currently in
[
https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Loncaric updated SPARK-26082:
Description:
Currently in
Martin Loncaric created SPARK-26082:
---
Summary: Misnaming of spark.mesos.fetch(er)Cache.enable in
MesosClusterScheduler
Key: SPARK-26082
URL: https://issues.apache.org/jira/browse/SPARK-26082
73 matches
Mail list logo
