[
https://issues.apache.org/jira/browse/SPARK-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463391#comment-16463391
]
Hyukjin Kwon commented on SPARK-23291:
--
[~felixcheung] WDYT?
> SparkR : substr : In SparkR
ABHISHEK KUMAR GUPTA created SPARK-24176:
Summary: The hdfs file path with wildcard can not be identified
when loading data
Key: SPARK-24176
URL: https://issues.apache.org/jira/browse/SPARK-24176
[
https://issues.apache.org/jira/browse/SPARK-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463369#comment-16463369
]
Wenchen Fan commented on SPARK-23291:
-
Yea I think we should backport it.
> SparkR : substr : In
[
https://issues.apache.org/jira/browse/SPARK-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463359#comment-16463359
]
Michael Ransley commented on SPARK-22918:
-
I think this relates to
[
https://issues.apache.org/jira/browse/SPARK-24174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Sokolov updated SPARK-24174:
Affects Version/s: (was: 2.3.0)
2.1.0
> Expose Hadoop config as
[
https://issues.apache.org/jira/browse/SPARK-24170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463336#comment-16463336
]
ABHISHEK KUMAR GUPTA commented on SPARK-24170:
--
I think this behavior need to re look and
[
https://issues.apache.org/jira/browse/SPARK-24175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-24175:
Description:
The current Spark 2.4 migration guide is not well phrased. We should
1. State the
Wenchen Fan created SPARK-24175:
---
Summary: improve the Spark 2.4 migration guide
Key: SPARK-24175
URL: https://issues.apache.org/jira/browse/SPARK-24175
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-24175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-24175:
Description: The current Spark 2.4 migration guide is not well phrased.
> improve the Spark 2.4
[
https://issues.apache.org/jira/browse/SPARK-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463280#comment-16463280
]
Hyukjin Kwon commented on SPARK-23291:
--
[~cloud_fan], you feel like we should backport this? I was
[
https://issues.apache.org/jira/browse/SPARK-24167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-24167.
-
Resolution: Fixed
Fix Version/s: 2.4.0
> ParquetFilters should not access SQLConf at
[
https://issues.apache.org/jira/browse/SPARK-24167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-24167:
Affects Version/s: (was: 2.3.0)
2.4.0
> ParquetFilters should not
[
https://issues.apache.org/jira/browse/SPARK-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-24147.
--
Resolution: Duplicate
> .count() reports wrong size of dataframe when filtering dataframe on
Nikolay Sokolov created SPARK-24174:
---
Summary: Expose Hadoop config as part of /environment API
Key: SPARK-24174
URL: https://issues.apache.org/jira/browse/SPARK-24174
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-24168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-24168.
-
Resolution: Fixed
Fix Version/s: 2.4.0
2.3.1
> WindowExec should not access
[
https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-23489:
Fix Version/s: 2.2.2
> Flaky Test: HiveExternalCatalogVersionsSuite
>
[
https://issues.apache.org/jira/browse/SPARK-24163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463219#comment-16463219
]
Xiao Li commented on SPARK-24163:
-
This is just nice to have.
> Support "ANY" or sub-query for Pivot
[
https://issues.apache.org/jira/browse/SPARK-24164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li reassigned SPARK-24164:
---
Assignee: Maryann Xue
> Support column list as the pivot column in Pivot
>
[
https://issues.apache.org/jira/browse/SPARK-24162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li reassigned SPARK-24162:
---
Assignee: Maryann Xue
> Support aliased literal values for Pivot "IN" clause
>
[
https://issues.apache.org/jira/browse/SPARK-24035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-24035.
-
Resolution: Fixed
Fix Version/s: 2.4.0
> SQL syntax for Pivot
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-23703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463209#comment-16463209
]
Jungtaek Lim commented on SPARK-23703:
--
Agreed. Is it worth to discuss in dev. mailing list? Or we
[
https://issues.apache.org/jira/browse/SPARK-23703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463190#comment-16463190
]
Jose Torres commented on SPARK-23703:
-
No, I don't know of any actual use cases for this. I think
[
https://issues.apache.org/jira/browse/SPARK-23703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463189#comment-16463189
]
Jungtaek Lim commented on SPARK-23703:
--
Actually I haven't hear about multiple watermarks on same
[
https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463003#comment-16463003
]
thua...@fsoft.com.vn edited comment on SPARK-13446 at 5/3/18 11:10 PM:
[
https://issues.apache.org/jira/browse/SPARK-21824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463054#comment-16463054
]
Sunitha Kambhampati commented on SPARK-21824:
-
I'll look into this.
> DROP TABLE should
[
https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463003#comment-16463003
]
thua...@fsoft.com.vn commented on SPARK-13446:
--
I am running into the same problem in
[
https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462971#comment-16462971
]
Apache Spark commented on SPARK-23489:
--
User 'dongjoon-hyun' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24119:
Assignee: (was: Apache Spark)
> Add interpreted execution to SortPrefix expression
>
[
https://issues.apache.org/jira/browse/SPARK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462939#comment-16462939
]
Apache Spark commented on SPARK-24119:
--
User 'bersprockets' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24119:
Assignee: Apache Spark
> Add interpreted execution to SortPrefix expression
>
[
https://issues.apache.org/jira/browse/SPARK-24173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-24173:
--
Description:
*BRANCH-2.2*
-
[
https://issues.apache.org/jira/browse/SPARK-12764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462827#comment-16462827
]
Martin Tapp commented on SPARK-12764:
-
Using Spark 2.1.1 and can't load a table with a JSONB column
[
https://issues.apache.org/jira/browse/SPARK-10943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462797#comment-16462797
]
Stephen Boesch commented on SPARK-10943:
Given the comment by Daniel Davis can this issue be
Dongjoon Hyun created SPARK-24173:
-
Summary: Flaky Test: VersionsSuite
Key: SPARK-24173
URL: https://issues.apache.org/jira/browse/SPARK-24173
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-23622:
--
Summary: Flaky Test: HiveClientSuites (was: HiveClientSuites fails with
[
https://issues.apache.org/jira/browse/SPARK-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-23622:
--
Description:
-
[
https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Imran Rashid resolved SPARK-23433.
--
Resolution: Fixed
Fix Version/s: 2.4.0
2.3.1
2.2.2
[
https://issues.apache.org/jira/browse/SPARK-23433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Imran Rashid reassigned SPARK-23433:
Assignee: Imran Rashid
> java.lang.IllegalStateException: more than one active taskSet
[
https://issues.apache.org/jira/browse/SPARK-24154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462630#comment-16462630
]
Sergey Zhemzhitsky commented on SPARK-24154:
> If users have to support mixin traits, they
[
https://issues.apache.org/jira/browse/SPARK-24169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-24169.
-
Resolution: Fixed
Fix Version/s: 2.3.1
2.4.0
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-24172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24172:
Assignee: Wenchen Fan (was: Apache Spark)
> we should not apply operator pushdown to
[
https://issues.apache.org/jira/browse/SPARK-24172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462587#comment-16462587
]
Apache Spark commented on SPARK-24172:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24172:
Assignee: Apache Spark (was: Wenchen Fan)
> we should not apply operator pushdown to
[
https://issues.apache.org/jira/browse/SPARK-23703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462584#comment-16462584
]
Jose Torres commented on SPARK-23703:
-
I'm no longer entirely convinced that this (and the parent
Wenchen Fan created SPARK-24172:
---
Summary: we should not apply operator pushdown to data source v2
many times
Key: SPARK-24172
URL: https://issues.apache.org/jira/browse/SPARK-24172
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Herman van Hovell updated SPARK-24133:
--
Fix Version/s: 2.3.1
> Reading Parquet files containing large strings can fail with
>
[
https://issues.apache.org/jira/browse/SPARK-24170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462545#comment-16462545
]
Marco Gaido commented on SPARK-24170:
-
This is true for every datasource. This is the expected
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462540#comment-16462540
]
Imran Rashid commented on SPARK-24135:
--
Honestly I don't understand the failure mode described here
[
https://issues.apache.org/jira/browse/SPARK-23697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23697:
Assignee: (was: Apache Spark)
> Accumulators of Spark 1.x no longer work with Spark
[
https://issues.apache.org/jira/browse/SPARK-23697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23697:
Assignee: Apache Spark
> Accumulators of Spark 1.x no longer work with Spark 2.x
>
[
https://issues.apache.org/jira/browse/SPARK-23697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462465#comment-16462465
]
Apache Spark commented on SPARK-23697:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462413#comment-16462413
]
Wenchen Fan commented on SPARK-24154:
-
I think this is a trade-off: In accumulator v1, we have
[
https://issues.apache.org/jira/browse/SPARK-24090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462363#comment-16462363
]
Stavros Kontopoulos commented on SPARK-24090:
-
Any plans for adding more items on this list?
[
https://issues.apache.org/jira/browse/SPARK-24171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462360#comment-16462360
]
Apache Spark commented on SPARK-24171:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24171:
Assignee: Apache Spark
> Update comments for non-deterministic functions
>
[
https://issues.apache.org/jira/browse/SPARK-24171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24171:
Assignee: (was: Apache Spark)
> Update comments for non-deterministic functions
>
[
https://issues.apache.org/jira/browse/SPARK-24166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-24166.
-
Resolution: Fixed
Fix Version/s: 2.3.1
2.4.0
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-23703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462339#comment-16462339
]
Jungtaek Lim commented on SPARK-23703:
--
[~joseph.torres]
Could you provide simple code or query
Maxim Gekk created SPARK-24171:
--
Summary: Update comments for non-deterministic functions
Key: SPARK-24171
URL: https://issues.apache.org/jira/browse/SPARK-24171
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-23715:
Assignee: Wenchen Fan
> from_utc_timestamp returns incorrect results for some UTC
[
https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-23715.
--
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21169
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-24152.
--
Resolution: Fixed
> SparkR CRAN feasibility check server problem
>
[
https://issues.apache.org/jira/browse/SPARK-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462287#comment-16462287
]
Apache Spark commented on SPARK-24133:
--
User 'ala' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462280#comment-16462280
]
Liang-Chi Hsieh commented on SPARK-24152:
-
Can be resolved now as I saw Jenkins test passed.
>
[
https://issues.apache.org/jira/browse/SPARK-10943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462244#comment-16462244
]
Daniel Davis commented on SPARK-10943:
--
According to parquet data types
[
https://issues.apache.org/jira/browse/SPARK-21429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462225#comment-16462225
]
Jungtaek Lim commented on SPARK-21429:
--
I agree that shortcut would help, but a bit afraid that such
[
https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462149#comment-16462149
]
Jungtaek Lim commented on SPARK-10816:
--
I'm still curious about out-of-box support on session
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462145#comment-16462145
]
Anirudh Ramanathan commented on SPARK-24135:
cc/ [~mridulm80] [~irashid] for thoughts on
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462139#comment-16462139
]
Anirudh Ramanathan edited comment on SPARK-24135 at 5/3/18 9:01 AM:
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462139#comment-16462139
]
Anirudh Ramanathan edited comment on SPARK-24135 at 5/3/18 8:58 AM:
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462139#comment-16462139
]
Anirudh Ramanathan commented on SPARK-24135:
It is increasingly common for people to write
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462106#comment-16462106
]
Matt Cheah edited comment on SPARK-24135 at 5/3/18 8:35 AM:
Not necessarily -
[
https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462106#comment-16462106
]
Matt Cheah commented on SPARK-24135:
Not necessarily - if the pods fail to start up, we should retry
[
https://issues.apache.org/jira/browse/SPARK-24030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-24030.
--
Resolution: Cannot Reproduce
> SparkSQL percentile_approx function is too slow for over
[
https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-23489:
---
Assignee: Dongjoon Hyun
> Flaky Test: HiveExternalCatalogVersionsSuite
>
[
https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-23489.
-
Resolution: Fixed
Fix Version/s: 2.3.1
2.4.0
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-24116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462040#comment-16462040
]
Rui Li commented on SPARK-24116:
To reproduce:
{code}
create table test_text(x int);
insert overwrite
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462030#comment-16462030
]
Liang-Chi Hsieh commented on SPARK-24152:
-
I think it is fixed now. It works in local. But better
ABHISHEK KUMAR GUPTA created SPARK-24170:
Summary: [Spark SQL] json file format is not dropped after
dropping table
Key: SPARK-24170
URL: https://issues.apache.org/jira/browse/SPARK-24170
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461983#comment-16461983
]
Felix Cheung edited comment on SPARK-24152 at 5/3/18 6:29 AM:
--
ok good.
in
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Felix Cheung updated SPARK-24152:
-
Summary: SparkR CRAN feasibility check server problem (was: Flaky Test:
SparkR)
> SparkR CRAN
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461993#comment-16461993
]
Felix Cheung commented on SPARK-24152:
--
(I updated the bug title - it's not really flaky..)
>
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461983#comment-16461983
]
Felix Cheung edited comment on SPARK-24152 at 5/3/18 6:26 AM:
--
ok good.
in
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461983#comment-16461983
]
Felix Cheung edited comment on SPARK-24152 at 5/3/18 6:26 AM:
--
ok good.
in
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461983#comment-16461983
]
Felix Cheung commented on SPARK-24152:
--
ok good.
in the event this reoccurs persistently,
option
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461980#comment-16461980
]
Hyukjin Kwon commented on SPARK-24152:
--
Seems fixed just now. I found one build passed -
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461974#comment-16461974
]
Felix Cheung commented on SPARK-24152:
--
Is this still a problem?
> Flaky Test: SparkR
>
[
https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461971#comment-16461971
]
Hyukjin Kwon commented on SPARK-24152:
--
FYI [~smilegator] and [~cloud_fan]
> Flaky Test: SparkR
>
88 matches
Mail list logo
