Mark Jarvin created SPARK-47029:
-----------------------------------
Summary: ALTER COLUMN DROP DEFAULT test fails with JSON data
sources
Key: SPARK-47029
URL: https://issues.apache.org/jira/browse/SPARK-47029
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 3.5.0, 4.0.0
Reporter: Mark Jarvin
Enabling the JSON data source causes a test case to fail:
{code:java}
[info] - SPARK-39557 INSERT INTO statements with tables with map defaults ***
FAILED *** (1 second, 498 milliseconds)
[info] Results do not match for query:
[info] Timezone:
sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
[info] Timezone Env:
[info]
[info] == Parsed Logical Plan ==
[info] 'UnresolvedRelation [t], [], false
[info]
[info] == Analyzed Logical Plan ==
[info] i: int, s:
struct<x:array<struct<a:int,b:int>>,y:array<map<string,boolean>>>, t:
array<map<string,boolean>>
[info] SubqueryAlias spark_catalog.default.t
[info] +- Relation spark_catalog.default.t[i#13929,s#13930,t#13931] json
[info]
[info] == Optimized Logical Plan ==
[info] Relation spark_catalog.default.t[i#13929,s#13930,t#13931] json
[info]
[info] == Physical Plan ==
[info] FileScan json spark_catalog.default.t[i#13929,s#13930,t#13931]
Batched: false, DataFilters: [], Format: JSON, Location: InMemoryFileIndex(1
paths)[file:/home/mark.jarvin/photon/spark/sql/core/spark-warehouse/org.apach...,
PartitionFilters: [], PushedFilters: [], ReadSchema:
struct<i:int,s:struct<x:array<struct<a:int,b:int>>,y:array<map<string,boolean>>>,t:array<map<stri...
[info]
[info] == Results ==
[info]
[info] == Results ==
[info] !== Correct Answer - 4 ==
== Spark Answer - 4 ==
[info] !struct<>
struct<i:int,s:struct<x:array<struct<a:int,b:int>>,y:array<map<string,boolean>>>,t:array<map<string,boolean>>>
[info] ![1,[List([1,2]),List(Map(def -> false, jkl -> true))],List(Map(xyz ->
true))] [1,[ArraySeq([1,2]),ArraySeq(Map(def -> false, jkl ->
true))],ArraySeq(Map(xyz -> true))]
[info] ![2,null,List(Map(xyz -> true))]
[2,[ArraySeq([1,2]),ArraySeq(Map(def -> false, jkl ->
true))],ArraySeq(Map(xyz -> true))]
[info] ![3,[List([3,4]),List(Map(mno -> false, pqr -> true))],List(Map(xyz ->
true))] [3,[ArraySeq([3,4]),ArraySeq(Map(mno -> false, pqr ->
true))],ArraySeq(Map(xyz -> true))]
[info] ![4,[List([3,4]),List(Map(mno -> false, pqr -> true))],List(Map(xyz ->
true))] [4,[ArraySeq([3,4]),ArraySeq(Map(mno -> false, pqr ->
true))],ArraySeq(Map(xyz -> true))] (QueryTest.scala:267){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
