eejbyfeldt commented on pull request #32783: URL: https://github.com/apache/spark/pull/32783#issuecomment-858394340
> `StructConverter` converts from Catalyst struct to a `Row`. Will it be a behavior change? Although it is Catalyst expression. From my understanding this is exactly the bug being fixed. So there will be behavior change. But the behavior is changed such that the interpreted path and the code gen path has the same behavior. The thinking being that the old behavior was undesired and incorrect. My understanding of the failure the test cases (if added without the patch): ``` [info] - encode/decode for map with case class as value: Map(1 -> IntAndString(1,a)) (interpreted path) *** FAILED *** (64 milliseconds) [info] Encoded/Decoded data does not match input data [info] [info] in: Map(1 -> IntAndString(1,a)) [info] out: Map(1 -> [1,a]) ``` Is that the value of the `Map` was converted to an `[1,a]` which I believe is an `InternalRow` instead of the expected `IntAndString`. This sounds like the change you are mentioning? The reason I believe my change is the correct is that using the `key/valueLambdaFunction` is since this is how it is done inside `MapObjects`: https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala#L820-L826 Please correct me if I am wrong about anything or misunderstood the question as I am new to the code base. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
