Dongqing Liu created SPARK-26404:
-
Summary: set spark.pyspark.python or PYSPARK_PYTHON doesn't work
in k8s client-cluster mode.
Key: SPARK-26404
URL: https://issues.apache.org/jira/browse/SPARK-26404
[
https://issues.apache.org/jira/browse/SPARK-26393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-26393.
--
Resolution: Won't Fix
> Different behaviors of date_add when calling it inside expr
>
[
https://issues.apache.org/jira/browse/SPARK-26393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724765#comment-16724765
]
Hyukjin Kwon commented on SPARK-26393:
--
PySaprk API is matched to Scala side primarily, and looks
[
https://issues.apache.org/jira/browse/SPARK-21514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724763#comment-16724763
]
Noritaka Sekiyama commented on SPARK-21514:
---
I'm working on fixing this. Will update once I
[
https://issues.apache.org/jira/browse/SPARK-26381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724750#comment-16724750
]
Hyukjin Kwon commented on SPARK-26381:
--
[~ryan.clancy], please provide the codes to reproduce.
>
[
https://issues.apache.org/jira/browse/SPARK-26391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-26391.
--
Resolution: Invalid
Questions should go to mailing list. You could have a better answer from
[
https://issues.apache.org/jira/browse/SPARK-26388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724751#comment-16724751
]
Hyukjin Kwon commented on SPARK-26388:
--
Can you add expected input / output in the description?
[
https://issues.apache.org/jira/browse/SPARK-26381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-26381.
--
Resolution: Incomplete
> Pickle Serialization Error Causing Crash
>
[
https://issues.apache.org/jira/browse/SPARK-26377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-26377.
--
Resolution: Invalid
I'm going to leave this resolved. Let's iterate in Spark mailing lists
[
https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-26366.
-
Resolution: Fixed
Fix Version/s: 3.0.0
2.4.1
> Except with transform
[
https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724744#comment-16724744
]
ASF GitHub Bot commented on SPARK-26366:
asfgit closed pull request #23315: [SPARK-26366][SQL]
[
https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li reassigned SPARK-26366:
---
Assignee: Marco Gaido
> Except with transform regression
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-26288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
weixiuli updated SPARK-26288:
-
Component/s: Spark Core
> add initRegisteredExecutorsDB in ExternalShuffleService
>
Huon Wilson created SPARK-26403:
---
Summary: DataFrame pivot using array column fails with
"Unsupported literal type class"
Key: SPARK-26403
URL: https://issues.apache.org/jira/browse/SPARK-26403
[
https://issues.apache.org/jira/browse/SPARK-26397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuanjian Li updated SPARK-26397:
Description:
As the comment in
[
https://issues.apache.org/jira/browse/SPARK-26397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuanjian Li updated SPARK-26397:
Environment: (was: As the comment in
[
https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edwina Lu updated SPARK-26399:
--
Description:
Add the peak values for the metrics to the stages REST API. Also add a new
[
https://issues.apache.org/jira/browse/SPARK-26402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-26402:
Summary: Canonicalization on GetStructField (was: GetStructField with
different optional names are
[
https://issues.apache.org/jira/browse/SPARK-26402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-26402:
Description: GetStructField with different optional names should be
semantically equal. We will use this
DB Tsai created SPARK-26402:
---
Summary: GetStructField with different optional names are
semantically equal
Key: SPARK-26402
URL: https://issues.apache.org/jira/browse/SPARK-26402
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-25857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25857:
Assignee: Apache Spark
> Document delegation token code in Spark
>
[
https://issues.apache.org/jira/browse/SPARK-25857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25857:
Assignee: (was: Apache Spark)
> Document delegation token code in Spark
>
[
https://issues.apache.org/jira/browse/SPARK-25857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724570#comment-16724570
]
ASF GitHub Bot commented on SPARK-25857:
vanzin opened a new pull request #23348:
Stanis Shkel created SPARK-26401:
Summary: [k8s] Init container drops necessary config options for
pulling jars from azure storage
Key: SPARK-26401
URL: https://issues.apache.org/jira/browse/SPARK-26401
Stanis Shkel created SPARK-26400:
Summary: [k8s] Init container silently swallows errors when
fetching jars from remote url
Key: SPARK-26400
URL: https://issues.apache.org/jira/browse/SPARK-26400
[
https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin resolved SPARK-25815.
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 22911
[
https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724486#comment-16724486
]
ASF GitHub Bot commented on SPARK-25815:
asfgit closed pull request #22911: [SPARK-25815][k8s]
[
https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin reassigned SPARK-25815:
--
Assignee: Marcelo Vanzin
> Kerberos Support in Kubernetes resource manager (Client
Edwina Lu created SPARK-26399:
-
Summary: Add new stage-level REST APIs and parameters
Key: SPARK-26399
URL: https://issues.apache.org/jira/browse/SPARK-26399
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-23431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edwina Lu updated SPARK-23431:
--
Description:
Collect and show the new executor memory metrics for each stage, to provide
more
[
https://issues.apache.org/jira/browse/SPARK-23431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724472#comment-16724472
]
Edwina Lu commented on SPARK-23431:
---
Splitting off the new REST APIs into a new subtask.
> Expose the
[
https://issues.apache.org/jira/browse/SPARK-25869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin reassigned SPARK-25869:
--
Assignee: (was: Marcelo Vanzin)
> Spark on YARN: the original diagnostics is
[
https://issues.apache.org/jira/browse/SPARK-25869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724453#comment-16724453
]
ASF GitHub Bot commented on SPARK-25869:
vanzin closed pull request #22876: [SPARK-25869] [YARN]
[
https://issues.apache.org/jira/browse/SPARK-25869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin reassigned SPARK-25869:
--
Assignee: Marcelo Vanzin
> Spark on YARN: the original diagnostics is missing when
[
https://issues.apache.org/jira/browse/SPARK-26398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724436#comment-16724436
]
Rong Ou commented on SPARK-26398:
-
https://github.com/apache/spark/pull/23347
> Support building GPU
Rong Ou created SPARK-26398:
---
Summary: Support building GPU docker images
Key: SPARK-26398
URL: https://issues.apache.org/jira/browse/SPARK-26398
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-25953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724361#comment-16724361
]
shane knapp commented on SPARK-25953:
-
no, not yet. i just got back from vacation and am planning
[
https://issues.apache.org/jira/browse/SPARK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724325#comment-16724325
]
ASF GitHub Bot commented on SPARK-26394:
srowen closed pull request #23346: [SPARK-26394][core]
[
https://issues.apache.org/jira/browse/SPARK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-26394.
---
Resolution: Fixed
Fix Version/s: 2.4.1
3.0.0
2.3.3
[
https://issues.apache.org/jira/browse/SPARK-26382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724319#comment-16724319
]
ASF GitHub Bot commented on SPARK-26382:
asfgit closed pull request #23334: [SPARK-26382][CORE]
Yuanjian Li created SPARK-26397:
---
Summary: Driver-side only metrics support
Key: SPARK-26397
URL: https://issues.apache.org/jira/browse/SPARK-26397
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen reassigned SPARK-26394:
-
Assignee: Jackey Lee
> Annotation error for Utils.timeStringAsMs
>
[
https://issues.apache.org/jira/browse/SPARK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-26394:
--
Priority: Trivial (was: Minor)
Component/s: Documentation
This doesn't need a JIRA if the
[
https://issues.apache.org/jira/browse/SPARK-26382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-26382.
---
Resolution: Fixed
Fix Version/s: 3.0.0
2.4.1
This is resolved via
[
https://issues.apache.org/jira/browse/SPARK-26316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724258#comment-16724258
]
ASF GitHub Bot commented on SPARK-26316:
dongjoon-hyun closed pull request #23319:
[
https://issues.apache.org/jira/browse/SPARK-26316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-26316:
--
Fix Version/s: 2.3.3
> Because of the perf degradation in TPC-DS, we currently partial revert
[
https://issues.apache.org/jira/browse/SPARK-26154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724238#comment-16724238
]
sandeep katta commented on SPARK-26154:
---
CC [~c...@koeninger.org] [~tdas] . this looks like
[
https://issues.apache.org/jira/browse/SPARK-11430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724203#comment-16724203
]
sathiyarajan commented on SPARK-11430:
--
[~ramk256] [~srowen] : is the except issue got fixed in the
[
https://issues.apache.org/jira/browse/SPARK-26384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724201#comment-16724201
]
ASF GitHub Bot commented on SPARK-26384:
asfgit closed pull request #23345: [SPARK-26384][SQL]
[
https://issues.apache.org/jira/browse/SPARK-26384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-26384:
Assignee: Maxim Gekk
> CSV schema inferring does not respect
[
https://issues.apache.org/jira/browse/SPARK-26384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-26384.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 23345
[
https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724154#comment-16724154
]
ASF GitHub Bot commented on SPARK-17383:
srowen closed pull request #14940:
[
https://issues.apache.org/jira/browse/SPARK-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-17383.
---
Resolution: Won't Fix
> improvement LabelPropagation of graphx lib
>
Kaspar Tint created SPARK-26396:
---
Summary: Kafka consumer cache overflow since 2.4.x
Key: SPARK-26396
URL: https://issues.apache.org/jira/browse/SPARK-26396
Project: Spark
Issue Type: Bug
Konstantinos Andrikopoulos created SPARK-26395:
--
Summary: Spark Thrift server memory leak
Key: SPARK-26395
URL: https://issues.apache.org/jira/browse/SPARK-26395
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-26336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724098#comment-16724098
]
Marco Gaido commented on SPARK-26336:
-
[~csevilla] the point is always the same, ie. the presence of
[
https://issues.apache.org/jira/browse/SPARK-26336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724081#comment-16724081
]
Carlos commented on SPARK-26336:
[~mgaido] I think I choose a bad objects to example that.
data1 = {
[
https://issues.apache.org/jira/browse/SPARK-24680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724047#comment-16724047
]
ASF GitHub Bot commented on SPARK-24680:
srowen closed pull request #21663:
[
https://issues.apache.org/jira/browse/SPARK-25953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724048#comment-16724048
]
Sean Owen commented on SPARK-25953:
---
Was this resolved? I think we can just install the latest JDK 11.
[
https://issues.apache.org/jira/browse/SPARK-26336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724052#comment-16724052
]
Marco Gaido commented on SPARK-26336:
-
That's correct because NULLs do not match. The usual
[
https://issues.apache.org/jira/browse/SPARK-17939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-17939:
--
Priority: Major (was: Critical)
Issue Type: Improvement (was: Bug)
> Spark-SQL Nullability:
[
https://issues.apache.org/jira/browse/SPARK-20712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-20712.
---
Resolution: Cannot Reproduce
> [SPARK 2.1 REGRESSION][SQL] Spark can't read Hive table when column
[
https://issues.apache.org/jira/browse/SPARK-26381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-26381:
--
Priority: Major (was: Critical)
What is failing to serialize here?
> Pickle Serialization Error
[
https://issues.apache.org/jira/browse/SPARK-26377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-26377:
--
Priority: Major (was: Critical)
> java.lang.IllegalStateException: No current assignment for
[
https://issues.apache.org/jira/browse/SPARK-24680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen reassigned SPARK-24680:
-
Assignee: StanZhai
> spark.executorEnv.JAVA_HOME does not take effect in Standalone mode
>
[
https://issues.apache.org/jira/browse/SPARK-24680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-24680.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 21663
[
https://issues.apache.org/jira/browse/SPARK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-26394:
Assignee: Apache Spark
> Annotation error for Utils.timeStringAsMs
>
[
https://issues.apache.org/jira/browse/SPARK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-26394:
Assignee: (was: Apache Spark)
> Annotation error for Utils.timeStringAsMs
>
[
https://issues.apache.org/jira/browse/SPARK-26394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724040#comment-16724040
]
ASF GitHub Bot commented on SPARK-26394:
stczwd opened a new pull request #23346:
[
https://issues.apache.org/jira/browse/SPARK-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724034#comment-16724034
]
ASF GitHub Bot commented on SPARK-26081:
asfgit closed pull request #23341:
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724011#comment-16724011
]
Jackey Lee commented on SPARK-24630:
[~jackylk] Sorry for the late reply.
I haven't considered
Jackey Lee created SPARK-26394:
--
Summary: Annotation error for Utils.timeStringAsMs
Key: SPARK-26394
URL: https://issues.apache.org/jira/browse/SPARK-26394
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723960#comment-16723960
]
ASF GitHub Bot commented on SPARK-19228:
HyukjinKwon closed pull request #21363:
[
https://issues.apache.org/jira/browse/SPARK-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-19228:
-
Labels: (was: easyfix)
> inferSchema function processed csv date column as string and
[
https://issues.apache.org/jira/browse/SPARK-26393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Kamal` updated SPARK-26393:
-
Description:
When Calling date_add from pyspark.sql.functions directly without using expr,
Ahmed Kamal` created SPARK-26393:
Summary: Different behaviors of date_add when calling it inside
expr
Key: SPARK-26393
URL: https://issues.apache.org/jira/browse/SPARK-26393
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-26377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723908#comment-16723908
]
pavan commented on SPARK-26377:
---
Hi Hyun,
I dint tried 2.4.0. I am raising this as a bug because
[
https://issues.apache.org/jira/browse/SPARK-26377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pavan updated SPARK-26377:
--
Description:
Hi,
I am using sparkkafkaDirectStream with subscriberPattern with initial
offsets for
[
https://issues.apache.org/jira/browse/SPARK-26384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-26384:
Assignee: Apache Spark
> CSV schema inferring does not respect
[
https://issues.apache.org/jira/browse/SPARK-26384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-26384:
Assignee: (was: Apache Spark)
> CSV schema inferring does not respect
[
https://issues.apache.org/jira/browse/SPARK-26384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723886#comment-16723886
]
ASF GitHub Bot commented on SPARK-26384:
MaxGekk opened a new pull request #23345:
[
https://issues.apache.org/jira/browse/SPARK-26392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-26392:
Assignee: (was: Apache Spark)
> Cancel pending allocate requests by taking locality
[
https://issues.apache.org/jira/browse/SPARK-26392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-26392:
Assignee: Apache Spark
> Cancel pending allocate requests by taking locality preference
[
https://issues.apache.org/jira/browse/SPARK-26392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723855#comment-16723855
]
ASF GitHub Bot commented on SPARK-26392:
Ngone51 opened a new pull request #23344:
wuyi created SPARK-26392:
Summary: Cancel pending allocate requests by taking locality
preference into account
Key: SPARK-26392
URL: https://issues.apache.org/jira/browse/SPARK-26392
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-26385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T M updated SPARK-26385:
Description:
Hello,
I have Spark Structured Streaming job which is runnning on YARN(Hadoop 2.6.0,
Spark
Rishabh created SPARK-26391:
---
Summary: Spark Streaming Kafka with Offset Gaps
Key: SPARK-26391
URL: https://issues.apache.org/jira/browse/SPARK-26391
Project: Spark
Issue Type: Question
87 matches
Mail list logo
