[
https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861991#comment-16861991
]
Gerard Maas commented on SPARK-28025:
-
I reproduced the issue in a spark-shell session:
[
https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-28025:
Summary: HDFSBackedStateStoreProvider should not leak .crc files (was:
[
https://issues.apache.org/jira/browse/SPARK-28025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861983#comment-16861983
]
Gerard Maas commented on SPARK-28025:
-
Same problem. A different part of the code.
>
Gerard Maas created SPARK-28025:
---
Summary: HDFSBackedStateStoreProvider leaks .crc files
Key: SPARK-28025
URL: https://issues.apache.org/jira/browse/SPARK-28025
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-24264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492046#comment-16492046
]
Gerard Maas commented on SPARK-24264:
-
This scenario demonstrates the claims in this ticket:
Given
[
https://issues.apache.org/jira/browse/SPARK-24264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24264:
Issue Type: Bug (was: Improvement)
> [Structured Streaming] Remove 'mergeSchema' option from
Gerard Maas created SPARK-24264:
---
Summary: [Structured Streaming] Remove 'mergeSchema' option from
Parquet source configuration
Key: SPARK-24264
URL: https://issues.apache.org/jira/browse/SPARK-24264
[
https://issues.apache.org/jira/browse/SPARK-24202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24202:
Description:
The current implementation of the implicits in SparkSession passes the current
[
https://issues.apache.org/jira/browse/SPARK-24202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24202:
Summary: Separate SQLContext dependency from SparkSession.implicits (was:
Separate SQLContext
[
https://issues.apache.org/jira/browse/SPARK-24202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24202:
Description:
The current implementation of the implicits in SparkSession passes the current
[
https://issues.apache.org/jira/browse/SPARK-24202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24202:
Description:
The current implementation of the implicits in SparkSession passes the current
Gerard Maas created SPARK-24202:
---
Summary: Separate SQLContext dependencies from
SparkSession.implicits
Key: SPARK-24202
URL: https://issues.apache.org/jira/browse/SPARK-24202
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447349#comment-16447349
]
Gerard Maas commented on SPARK-24046:
-
In graphical form, this is what is happening when rampUpTime
[
https://issues.apache.org/jira/browse/SPARK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24046:
Attachment: image-2018-04-22-22-06-49-202.png
> Rate Source doesn't gradually increase rate when
[
https://issues.apache.org/jira/browse/SPARK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24046:
Attachment: image-2018-04-22-22-03-03-945.png
> Rate Source doesn't gradually increase rate when
[
https://issues.apache.org/jira/browse/SPARK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447347#comment-16447347
]
Gerard Maas commented on SPARK-24046:
-
The problem seems to come from this integer division:
[
https://issues.apache.org/jira/browse/SPARK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24046:
Summary: Rate Source doesn't gradually increase rate when
rampUpTime>=RowsPerSecond (was: Rate
[
https://issues.apache.org/jira/browse/SPARK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-24046:
Summary: Rate Source does gradually increase rate when
rampUpTime>=RowsPerSecond (was: rampUpTime
Gerard Maas created SPARK-24046:
---
Summary: rampUpTime in Rate Source does not work for
rampUpTime>=RowsPerSecond
Key: SPARK-24046
URL: https://issues.apache.org/jira/browse/SPARK-24046
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123485#comment-16123485
]
Gerard Maas commented on SPARK-21710:
-
PR: https://github.com/apache/spark/pull/18923
> ConsoleSink
[
https://issues.apache.org/jira/browse/SPARK-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-21710:
Description:
ConsoleSink does a full collect of the streaming dataset in order to show few
lines
Gerard Maas created SPARK-21710:
---
Summary: ConsoleSink causes OOM crashes with large inputs.
Key: SPARK-21710
URL: https://issues.apache.org/jira/browse/SPARK-21710
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114906#comment-15114906
]
Gerard Maas commented on SPARK-3958:
[~pwendell] [~joshrosen] We just hit this bug in one of our
Gerard Maas created SPARK-8009:
--
Summary: [Mesos] Allow provisioning of executor logging
configuration
Key: SPARK-8009
URL: https://issues.apache.org/jira/browse/SPARK-8009
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276934#comment-14276934
]
Gerard Maas commented on SPARK-5095:
Great. I left you some comments on the docs. I'll
[
https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271508#comment-14271508
]
Gerard Maas commented on SPARK-5095:
If I understand correctly, setting
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268440#comment-14268440
]
Gerard Maas edited comment on SPARK-4940 at 1/8/15 9:21 AM:
Hi
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269038#comment-14269038
]
Gerard Maas edited comment on SPARK-4940 at 1/8/15 9:28 AM:
I
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269038#comment-14269038
]
Gerard Maas commented on SPARK-4940:
I forgot to mention that in the previous example,
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gerard Maas updated SPARK-4940:
---
Attachment: mesos-config-difference-3nodes-vs-2nodes.png
Difference of job performance due to
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268440#comment-14268440
]
Gerard Maas commented on SPARK-4940:
Hi Tim,
We are indeed using Coarse Grain mode.
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268440#comment-14268440
]
Gerard Maas edited comment on SPARK-4940 at 1/7/15 10:54 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268440#comment-14268440
]
Gerard Maas edited comment on SPARK-4940 at 1/7/15 10:53 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268440#comment-14268440
]
Gerard Maas edited comment on SPARK-4940 at 1/7/15 10:52 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264579#comment-14264579
]
Gerard Maas commented on SPARK-4940:
From the perspective of evenly allocating Spark
[
https://issues.apache.org/jira/browse/SPARK-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225945#comment-14225945
]
Gerard Maas commented on SPARK-4537:
I was waiting for internal clearance to put a PR
Gerard Maas created SPARK-4537:
--
Summary: Add 'processing delay' and 'totalDelay' to the metrics
reported by the Spark Streaming subsystem
Key: SPARK-4537
URL: https://issues.apache.org/jira/browse/SPARK-4537
Gerard Maas created SPARK-2620:
--
Summary: case class cannot be used as key for reduce
Key: SPARK-2620
URL: https://issues.apache.org/jira/browse/SPARK-2620
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-2620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070443#comment-14070443
]
Gerard Maas commented on SPARK-2620:
[~sowen] No, doesn't look like it is.
case
Gerard Maas created SPARK-1985:
--
Summary: SPARK_HOME shouldn't be required when spark.executor.uri
is provided
Key: SPARK-1985
URL: https://issues.apache.org/jira/browse/SPARK-1985
Project: Spark
40 matches
Mail list logo
