Russell Spitzer created SPARK-16614:
---
Summary: DirectJoin with DataSource for SparkSQL
Key: SPARK-16614
URL: https://issues.apache.org/jira/browse/SPARK-16614
Project: Spark
Issue Type:
Russell Spitzer created SPARK-16616:
---
Summary: Allow Catalyst to take Advantage of Hash Partitioned
DataSources
Key: SPARK-16616
URL: https://issues.apache.org/jira/browse/SPARK-16616
Project:
[
https://issues.apache.org/jira/browse/SPARK-16614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-16614:
Description:
Join behaviors against some datasources can be improved by skipping a full
[
https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423539#comment-15423539
]
Russell Spitzer commented on SPARK-16725:
-
In our case it's exposing a library which exposes the
[
https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423498#comment-15423498
]
Russell Spitzer commented on SPARK-16725:
-
I think *But it works* is a bit of an overstatement.
[
https://issues.apache.org/jira/browse/SPARK-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423555#comment-15423555
]
Russell Spitzer commented on SPARK-16725:
-
I'm well aware as we've been dealing with this since
[
https://issues.apache.org/jira/browse/SPARK-16614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458784#comment-15458784
]
Russell Spitzer commented on SPARK-16614:
-
Yes. This would be similar to how Presto works by
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-17673:
Labels: correctness (was: )
> Reused Exchange Aggregations Produce Incorrect Results
>
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524641#comment-15524641
]
Russell Spitzer commented on SPARK-17673:
-
I only ran this on 2.0.0 and 2.0.1
> Reused Exchange
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524722#comment-15524722
]
Russell Spitzer commented on SPARK-17673:
-
Ugh I made a typo in my Parquet Example I don't see it
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524734#comment-15524734
]
Russell Spitzer edited comment on SPARK-17673 at 9/27/16 1:38 AM:
--
Well
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524734#comment-15524734
]
Russell Spitzer commented on SPARK-17673:
-
Well in this case they are equal correct?
> Reused
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524734#comment-15524734
]
Russell Spitzer edited comment on SPARK-17673 at 9/27/16 1:39 AM:
--
Well
[
https://issues.apache.org/jira/browse/SPARK-10501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15559271#comment-15559271
]
Russell Spitzer commented on SPARK-10501:
-
It's not that we need it as a unique identifier. It's
Russell Spitzer created SPARK-17673:
---
Summary: Reused Exchange Aggregations Produce Incorrect Results
Key: SPARK-17673
URL: https://issues.apache.org/jira/browse/SPARK-17673
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525101#comment-15525101
]
Russell Spitzer edited comment on SPARK-17673 at 9/27/16 5:17 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525101#comment-15525101
]
Russell Spitzer commented on SPARK-17673:
-
{code}== Parsed Logical Plan ==
Union
:- Aggregate
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525096#comment-15525096
]
Russell Spitzer commented on SPARK-17673:
-
Ah yeah there would definitely be different pruning in
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524848#comment-15524848
]
Russell Spitzer commented on SPARK-17673:
-
I couldn't get this to happen without C*, hopefully
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524999#comment-15524999
]
Russell Spitzer commented on SPARK-17673:
-
We shouldn't be ... The only thing we cache are
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525007#comment-15525007
]
Russell Spitzer edited comment on SPARK-17673 at 9/27/16 4:12 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525007#comment-15525007
]
Russell Spitzer commented on SPARK-17673:
-
Looking at this plan
```
Union
:-
[
https://issues.apache.org/jira/browse/SPARK-17673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531265#comment-15531265
]
Russell Spitzer commented on SPARK-17673:
-
Looks good on my end
{code}
scala>
[
https://issues.apache.org/jira/browse/SPARK-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561305#comment-15561305
]
Russell Spitzer commented on SPARK-17845:
-
FWIW, I Find (2) much more readable
> Improve window
[
https://issues.apache.org/jira/browse/SPARK-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15561305#comment-15561305
]
Russell Spitzer edited comment on SPARK-17845 at 10/10/16 5:14 AM:
---
Russell Spitzer created SPARK-18851:
---
Summary: DataSet limit.distinct Results in NPE in Codegen
Key: SPARK-18851
URL: https://issues.apache.org/jira/browse/SPARK-18851
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-18851:
Labels: regresion (was: )
> DataSet Limit into Aggregate Results in NPE in Codegen
>
[
https://issues.apache.org/jira/browse/SPARK-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer resolved SPARK-18851.
-
Resolution: Duplicate
> DataSet Limit into Aggregate Results in NPE in Codegen
>
[
https://issues.apache.org/jira/browse/SPARK-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-18851:
Summary: DataSet Limit into Aggregate Results in NPE in Codegen (was:
DataSet
[
https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130355#comment-16130355
]
Russell Spitzer commented on SPARK-15689:
-
Thanks [~cloud_fan] for posting the design doc it was
[
https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052321#comment-16052321
]
Russell Spitzer commented on SPARK-15689:
-
I've been trying to work with making Catalyst
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-22316:
Summary: Cannot Select ReducedAggregator Column (was: Cannot Select
ReducedAggreagtor
Russell Spitzer created SPARK-22316:
---
Summary: Cannot Select ReducedAggreagtor Column
Key: SPARK-22316
URL: https://issues.apache.org/jira/browse/SPARK-22316
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211528#comment-16211528
]
Russell Spitzer edited comment on SPARK-22316 at 10/19/17 6:46 PM:
---
I
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211528#comment-16211528
]
Russell Spitzer commented on SPARK-22316:
-
I can imagine many reasons i might want to access a
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211528#comment-16211528
]
Russell Spitzer edited comment on SPARK-22316 at 10/19/17 6:49 PM:
---
I
[
https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234640#comment-16234640
]
Russell Spitzer commented on SPARK-15689:
-
Something I just noticed, it may be helpful to also
[
https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234640#comment-16234640
]
Russell Spitzer edited comment on SPARK-15689 at 11/1/17 7:58 PM:
--
[
https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234899#comment-16234899
]
Russell Spitzer edited comment on SPARK-15689 at 11/1/17 10:52 PM:
---
I
[
https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234899#comment-16234899
]
Russell Spitzer commented on SPARK-15689:
-
I think knowing whether or not the count was occurring
[
https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234820#comment-16234820
]
Russell Spitzer commented on SPARK-15689:
-
It does not, we can tell that a count (or similar
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217666#comment-16217666
]
Russell Spitzer commented on SPARK-22316:
-
[~hvanhovell] This was the ticket I told you about :)
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211846#comment-16211846
]
Russell Spitzer commented on SPARK-22316:
-
You are right, i meant to have createDataset up there.
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211849#comment-16211849
]
Russell Spitzer commented on SPARK-22316:
-
Nope I'm allowed to have columns with parens in this
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211849#comment-16211849
]
Russell Spitzer edited comment on SPARK-22316 at 10/19/17 10:21 PM:
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-22316:
Description:
Given a dataset which has been run through reduceGroups like this
{code}
case
[
https://issues.apache.org/jira/browse/SPARK-22316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-22316:
Description:
Given a dataset which has been run through reduceGroups like this
{code}
case
[
https://issues.apache.org/jira/browse/SPARK-22976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-22976:
Description:
Spark Standalone worker cleanup finds directories to remove with a listFiles
[
https://issues.apache.org/jira/browse/SPARK-22976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-22976:
Summary: Worker cleanup can remove running driver directories (was:
Workerer cleanup can
Russell Spitzer created SPARK-22976:
---
Summary: Workerer cleanup can remove running driver directories
Key: SPARK-22976
URL: https://issues.apache.org/jira/browse/SPARK-22976
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-22976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317367#comment-16317367
]
Russell Spitzer commented on SPARK-22976:
-
Made a PR against 2.0 but it's valid against all
Russell Spitzer created SPARK-25003:
---
Summary: Pyspark Does not use Spark Sql Extensions
Key: SPARK-25003
URL: https://issues.apache.org/jira/browse/SPARK-25003
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-25003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568415#comment-16568415
]
Russell Spitzer commented on SPARK-25003:
-
[~holden.karau] , Wrote up a PR for each branch
[
https://issues.apache.org/jira/browse/SPARK-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560034#comment-16560034
]
Russell Spitzer commented on SPARK-21216:
-
For anyone else searching, this also fixes custom
Russell Spitzer created SPARK-25560:
---
Summary: Allow Function Injection in SparkSessionExtensions
Key: SPARK-25560
URL: https://issues.apache.org/jira/browse/SPARK-25560
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-26518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736091#comment-16736091
]
Russell Spitzer commented on SPARK-26518:
-
Yeah I basically came to the same conclusion, there
Russell Spitzer created SPARK-26518:
---
Summary: UI Application Info Race Condition Can Throw NoSuchElement
Key: SPARK-26518
URL: https://issues.apache.org/jira/browse/SPARK-26518
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-25003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807121#comment-16807121
]
Russell Spitzer edited comment on SPARK-25003 at 4/1/19 7:48 PM:
-
There
[
https://issues.apache.org/jira/browse/SPARK-25003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807121#comment-16807121
]
Russell Spitzer commented on SPARK-25003:
-
There was no interest in putting in OSS 2.4, but I
[
https://issues.apache.org/jira/browse/SPARK-32977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-32977:
Description:
The JavaDoc says that the default save mode is dependent on DataSource
Russell Spitzer created SPARK-32977:
---
Summary: [SQL] JavaDoc on Default Save mode Incorrect
Key: SPARK-32977
URL: https://issues.apache.org/jira/browse/SPARK-32977
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-32977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17200930#comment-17200930
]
Russell Spitzer commented on SPARK-32977:
-
[~brkyvz] We talked about this a while back, just
[
https://issues.apache.org/jira/browse/SPARK-33041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-33041:
Affects Version/s: 3.0.1
> Better error messages when PySpark Java Gateway Crashes
>
[
https://issues.apache.org/jira/browse/SPARK-33041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205059#comment-17205059
]
Russell Spitzer commented on SPARK-33041:
-
To elaborate, this could be the case for any failure
[
https://issues.apache.org/jira/browse/SPARK-33041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-33041:
Summary: Better error messages when PySpark Java Gateway Crashes (was:
Better error
Russell Spitzer created SPARK-33041:
---
Summary: Better error messages when PySpark Java Gateway Fails to
Start or Crashes
Key: SPARK-33041
URL: https://issues.apache.org/jira/browse/SPARK-33041
[
https://issues.apache.org/jira/browse/SPARK-37061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-37061:
Description:
Currently CustomMetrics uses `getCanonicalName` to get the metric type name
Russell Spitzer created SPARK-37061:
---
Summary: Custom V2 Metrics uses wrong classname in for lookup
Key: SPARK-37061
URL: https://issues.apache.org/jira/browse/SPARK-37061
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-37061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russell Spitzer updated SPARK-37061:
Summary: Custom V2 Metrics uses wrong classname for lookup (was: Custom V2
Metrics uses
69 matches
Mail list logo
