Github user jainaks commented on the issue:

    https://github.com/apache/spark/pull/21320
  
    Hi @mallman ,
    I found another major issue after having this fix.
    Schema:
    a: struct (nullable = true)
     |    |-- b: struct (nullable = true)
     |    |    |-- c1: string (nullable = true)
     |    |    |-- c2: string (nullable = true)
     |    |    |-- c3: string (nullable = true)
     |    |    |-- c4: string (nullable = true)
     |    |    |-- c5: boolean (nullable = true)
    id: struct (nullable = true)
     |    |-- i1: struct (nullable = true)
     |    |    |-- i2: string (nullable = true)
    timestamp: bigint
    **Query:**
    select      a.b.c3 as c3, 
                first(a.b.c3) over (partition by id.i1.i2 order by timestamp 
rows between current row and unbounded following) as first_c3
    from        temp;
    The column "first_c3" gets the value of column "c2".
    It works well, if i just turn the parquetSchemaPrunning flag to false.
    It may sound odd in the first look and so does for me, but this is what i 
am getting.
    PS: I am running all my tests using #16578 pr.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to