[
https://issues.apache.org/jira/browse/SPARK-34076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-34076:
-
Affects Version/s: (was: 3.0.1)
3.1.0
> SQLContext.dropTempTable fails if cac
Chao Sun created SPARK-34076:
Summary: SQLContext.dropTempTable fails if cache is non-empty
Key: SPARK-34076
URL: https://issues.apache.org/jira/browse/SPARK-34076
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262826#comment-17262826
]
Chao Sun commented on SPARK-33507:
--
[~dongjoon], [~hyukjin.kwon]: can either of you rem
[
https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33507:
-
Description:
This is an umbrella JIRA to track fixes & improvements for caching behavior in
Spark datas
[
https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33507:
-
Description:
This is an umbrella JIRA to track fixes & improvements for caching behavior in
Spark datas
[
https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-34052:
-
Parent: SPARK-33507
Issue Type: Sub-task (was: Bug)
> A cached view should become invalid after
[
https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-34052:
-
Parent: (was: SPARK-33392)
Issue Type: Bug (was: Sub-task)
> A cached view should become in
[
https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-34052:
-
Summary: A cached view should become invalid after a table is dropped
(was: A view doesn't become inval
[
https://issues.apache.org/jira/browse/SPARK-33729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33729:
-
Parent: SPARK-33507
Issue Type: Sub-task (was: Bug)
> When refreshing cache, Spark should not u
[
https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-34052:
-
Affects Version/s: 3.1.0
> A view doesn't become invalid after a table is dropped or replaced in V2
> --
[
https://issues.apache.org/jira/browse/SPARK-34039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-34039:
-
Affects Version/s: 3.0.1
> [DSv2] ReplaceTable should invalidate cache
> ---
Chao Sun created SPARK-34052:
Summary: A view doesn't become invalid after a table is dropped or
replaced in V2
Key: SPARK-34052
URL: https://issues.apache.org/jira/browse/SPARK-34052
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-34039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-34039:
-
Parent: SPARK-33507
Issue Type: Sub-task (was: Bug)
> [DSv2] ReplaceTable should invalidate cac
[
https://issues.apache.org/jira/browse/SPARK-34039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261468#comment-17261468
]
Chao Sun commented on SPARK-34039:
--
Thanks for verifying this [~dongjoon]!
> [DSv2] R
[
https://issues.apache.org/jira/browse/SPARK-34039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261036#comment-17261036
]
Chao Sun commented on SPARK-34039:
--
I'm not sure if this is a bug since:
1. the command
[
https://issues.apache.org/jira/browse/SPARK-34039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260280#comment-17260280
]
Chao Sun commented on SPARK-34039:
--
I'll submit a patch for this soon.
> [DSv2] Replac
Chao Sun created SPARK-34039:
Summary: [DSv2] ReplaceTable should invalidate cache
Key: SPARK-34039
URL: https://issues.apache.org/jira/browse/SPARK-34039
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-33937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17256192#comment-17256192
]
Chao Sun commented on SPARK-33937:
--
This looks like a duplicate of SPARK-32480.
> Mov
Chao Sun created SPARK-33880:
Summary: Replace Hadoop profile names by using hadoop-2 and
hadoop-3
Key: SPARK-33880
URL: https://issues.apache.org/jira/browse/SPARK-33880
Project: Spark
Issue Ty
Chao Sun created SPARK-33729:
Summary: When refreshing cache, Spark should not use cached plan
when recaching data
Key: SPARK-33729
URL: https://issues.apache.org/jira/browse/SPARK-33729
Project: Spark
Chao Sun created SPARK-33653:
Summary: DSv2: REFRESH TABLE should recache the table itself
Key: SPARK-33653
URL: https://issues.apache.org/jira/browse/SPARK-33653
Project: Spark
Issue Type: Sub-t
Chao Sun created SPARK-33652:
Summary: DSv2: DeleteFrom should refresh cache
Key: SPARK-33652
URL: https://issues.apache.org/jira/browse/SPARK-33652
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-33652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33652:
-
Parent: SPARK-33507
Issue Type: Sub-task (was: Improvement)
> DSv2: DeleteFrom should refresh c
Chao Sun created SPARK-33567:
Summary: DSv2: Use callback instead of passing Spark session and
v2 relation for refreshing cache
Key: SPARK-33567
URL: https://issues.apache.org/jira/browse/SPARK-33567
Proj
[
https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236521#comment-17236521
]
Chao Sun commented on SPARK-33507:
--
[~dongjoon] Yes this is for Spark 3.1 _mostly_ (som
[
https://issues.apache.org/jira/browse/SPARK-33507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33507:
-
Description:
This is an umbrella JIRA to track fixes & improvements for caching behavior in
Spark datas
[
https://issues.apache.org/jira/browse/SPARK-33305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33305:
-
Parent: (was: SPARK-33392)
Issue Type: Bug (was: Sub-task)
> DSv2: DROP TABLE command shoul
[
https://issues.apache.org/jira/browse/SPARK-33290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33290:
-
Parent: SPARK-33507
Issue Type: Sub-task (was: Bug)
> REFRESH TABLE should invalidate cache eve
[
https://issues.apache.org/jira/browse/SPARK-33305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33305:
-
Parent: SPARK-33507
Issue Type: Sub-task (was: Bug)
> DSv2: DROP TABLE command should also inva
[
https://issues.apache.org/jira/browse/SPARK-33435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33435:
-
Parent: SPARK-33507
Issue Type: Sub-task (was: Bug)
> DSv2: REFRESH TABLE should invalidate cac
[
https://issues.apache.org/jira/browse/SPARK-33492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33492:
-
Parent: SPARK-33507
Issue Type: Sub-task (was: Bug)
> DSv2: Append/Overwrite/ReplaceTable shoul
Chao Sun created SPARK-33507:
Summary: Improve and fix cache behavior in v1 and v2
Key: SPARK-33507
URL: https://issues.apache.org/jira/browse/SPARK-33507
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-33493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33493:
-
Component/s: SQL
> Scala 2.13 build with SBT failure
> -
>
>
Chao Sun created SPARK-33493:
Summary: Scala 2.13 build with SBT failure
Key: SPARK-33493
URL: https://issues.apache.org/jira/browse/SPARK-33493
Project: Spark
Issue Type: Bug
Component
Chao Sun created SPARK-33492:
Summary: DSv2: Append/Overwrite/ReplaceTable should invalidate
cache
Key: SPARK-33492
URL: https://issues.apache.org/jira/browse/SPARK-33492
Project: Spark
Issue Ty
[
https://issues.apache.org/jira/browse/SPARK-33435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33435:
-
Labels: DSv2 (was: )
> DSv2: REFRESH TABLE should invalidate caches
> -
[
https://issues.apache.org/jira/browse/SPARK-33435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33435:
-
Environment: (was: R)
> DSv2: REFRESH TABLE should invalidate caches
> -
Chao Sun created SPARK-33437:
Summary: Support CACHE TABLE command in DSv2
Key: SPARK-33437
URL: https://issues.apache.org/jira/browse/SPARK-33437
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-33437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33437:
-
Issue Type: New Feature (was: Improvement)
> Support CACHE TABLE command in DSv2
>
Chao Sun created SPARK-33435:
Summary: DSv2: REFRESH TABLE should invalidate caches
Key: SPARK-33435
URL: https://issues.apache.org/jira/browse/SPARK-33435
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17228892#comment-17228892
]
Chao Sun commented on SPARK-33213:
--
Thanks. I think there are still a few improvements
Chao Sun created SPARK-33376:
Summary: Remove the option of "sharesHadoopClasses" in Hive
IsolatedClientLoader
Key: SPARK-33376
URL: https://issues.apache.org/jira/browse/SPARK-33376
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-33311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33311:
-
Description:
Currently, the semantics for {{REFRESH TABLE t}} is not well defined for view
(let's say {
[
https://issues.apache.org/jira/browse/SPARK-33311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33311:
-
Description:
Currently, the semantics for {{REFRESH TABLE t}} is not well defined for view
(let's say {
Chao Sun created SPARK-33311:
Summary: Improve semantics for REFRESH TABLE
Key: SPARK-33311
URL: https://issues.apache.org/jira/browse/SPARK-33311
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-33290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33290:
-
Affects Version/s: 2.4.7
> REFRESH TABLE should invalidate cache even though the table itself may not be
Chao Sun created SPARK-33305:
Summary: DSv2: DROP TABLE command should also invalidate cache
Key: SPARK-33305
URL: https://issues.apache.org/jira/browse/SPARK-33305
Project: Spark
Issue Type: Bug
Chao Sun created SPARK-33293:
Summary: Refactor WriteToDataSourceV2Exec and reduce code
duplication
Key: SPARK-33293
URL: https://issues.apache.org/jira/browse/SPARK-33293
Project: Spark
Issue T
Chao Sun created SPARK-33290:
Summary: REFRESH TABLE should invalidate cache even though the
table itself may not be cached
Key: SPARK-33290
URL: https://issues.apache.org/jira/browse/SPARK-33290
Project:
[
https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-29250:
-
Summary: Upgrade to Hadoop 3.3.0 (was: Upgrade to Hadoop 3.2.2)
> Upgrade to Hadoop 3.3.0
> ---
[
https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33212:
-
Description:
Hadoop 3.x+ offers shaded client jars: hadoop-client-api and
hadoop-client-runtime, which
[
https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33212:
-
Description:
Hadoop 3.x+ offers shaded client jars: hadoop-client-api and
hadoop-client-runtime, which
[
https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-33212:
-
Description:
Hadoop 3.x+ offers shaded client jars: hadoop-client-api and
hadoop-client-runtime, which
Chao Sun created SPARK-33213:
Summary: Upgrade Apache Arrow to 2.0.0
Key: SPARK-33213
URL: https://issues.apache.org/jira/browse/SPARK-33213
Project: Spark
Issue Type: Dependency upgrade
[
https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-29250:
-
Summary: Upgrade to Hadoop 3.2.2 (was: Upgrade to Hadoop 3.2.1 and move to
shaded client)
> Upgrade to
Chao Sun created SPARK-33212:
Summary: Move to shaded clients for Hadoop 3.x profile
Key: SPARK-33212
URL: https://issues.apache.org/jira/browse/SPARK-33212
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-29250:
-
Summary: Upgrade to Hadoop 3.2.1 and move to shaded client (was: Upgrade
to Hadoop 3.2.1 Upgrade to Had
Chao Sun created SPARK-33135:
Summary: Use listLocatedStatus from FileSystem implementations
Key: SPARK-33135
URL: https://issues.apache.org/jira/browse/SPARK-33135
Project: Spark
Issue Type: Imp
[
https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212804#comment-17212804
]
Chao Sun commented on SPARK-27733:
--
[~iemejia] I can help with Hive releases if you can
[
https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201094#comment-17201094
]
Chao Sun commented on SPARK-27733:
--
[~sha...@uber.com] sure, I can join in the next syn
[
https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32858:
-
Summary: UnwrapCastInBinaryComparison: support other numeric types (was:
UnwrapCastInBinaryComparison:
Chao Sun created SPARK-32880:
Summary: Improve and refactor parallel listing in HadoopFSUtils
Key: SPARK-32880
URL: https://issues.apache.org/jira/browse/SPARK-32880
Project: Spark
Issue Type: Im
[
https://issues.apache.org/jira/browse/SPARK-24994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-24994:
-
Summary: Support cast pushdown for integral types (was: Support pushdown
casts for )
> Support cast pu
[
https://issues.apache.org/jira/browse/SPARK-24994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-24994:
-
Summary: Support pushdown casts for (was: When the data type of the field
is converted to other types,
Chao Sun created SPARK-32858:
Summary: UnwrapCastInBinaryComparison: support more types other
than Integral
Key: SPARK-32858
URL: https://issues.apache.org/jira/browse/SPARK-32858
Project: Spark
Chao Sun created SPARK-32802:
Summary: Avoid using SpecificInternalRow in
RunLengthEncoding#Encoder
Key: SPARK-32802
URL: https://issues.apache.org/jira/browse/SPARK-32802
Project: Spark
Issue T
[
https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17187849#comment-17187849
]
Chao Sun commented on SPARK-27733:
--
bq. BTW, is Hive community will to introduce the ve
[
https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32721:
-
Description:
The following if clause:
{code:sql}
if(p, null, false)
{code}
can be simplified to:
{code:
[
https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32721:
-
Description:
The following if clause:
{code:sql}
if(p, null, false)
{code}
can be simplified to:
{code:
[
https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32721:
-
Description:
The following if clause:
{code:sql}
if(p, null, false)
{code}
can be simplified to:
{code:
Chao Sun created SPARK-32721:
Summary: Simplify if clauses with null and boolean
Key: SPARK-32721
URL: https://issues.apache.org/jira/browse/SPARK-32721
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32721:
-
Description:
The following if clause:
{code:sql}
if(p, null, false)
{code}
can be simplified to:
{code:
[
https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185293#comment-17185293
]
Chao Sun commented on SPARK-27733:
--
bq. Apache Spark 3.0 will ship both Hive 1.2.1 buil
[
https://issues.apache.org/jira/browse/SPARK-32703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32703:
-
Description: Parquet vectorized reader still uses the old API for
{{filterRowGroups}} and only filters o
[
https://issues.apache.org/jira/browse/SPARK-32703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32703:
-
Summary: Enable dictionary filtering for Parquet vectorized reader (was:
Re-enable dictionary filtering
Chao Sun created SPARK-32703:
Summary: Re-enable dictionary filtering for Parquet
Key: SPARK-32703
URL: https://issues.apache.org/jira/browse/SPARK-32703
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-32694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17184614#comment-17184614
]
Chao Sun commented on SPARK-32694:
--
Thanks [~rakson] for the pointer! didn't know there
[
https://issues.apache.org/jira/browse/SPARK-32694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17183592#comment-17183592
]
Chao Sun commented on SPARK-32694:
--
Seems there could be more cases like this, such as:
[
https://issues.apache.org/jira/browse/SPARK-32694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32694:
-
Description:
Currently we don't support pushing down cast to data source (see
[link|http://apache-spark
[
https://issues.apache.org/jira/browse/SPARK-32694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-32694:
-
Description:
Currently we don't support pushing down cast to data source (see
[link|http://apache-spark
Chao Sun created SPARK-32694:
Summary: Pushdown cast to data sources
Key: SPARK-32694
URL: https://issues.apache.org/jira/browse/SPARK-32694
Project: Spark
Issue Type: Improvement
Compo
Chao Sun created SPARK-32674:
Summary: Add suggestion for parallel directory listing in tuning
doc
Key: SPARK-32674
URL: https://issues.apache.org/jira/browse/SPARK-32674
Project: Spark
Issue Ty
401 - 482 of 482 matches
Mail list logo
