guykhazma commented on pull request #28826:
URL: https://github.com/apache/spark/pull/28826#issuecomment-718496266
@viirya @maropu any comments?
This is an automated message from the Apache Git Service.
To respond to the
guykhazma commented on pull request #28826:
URL: https://github.com/apache/spark/pull/28826#issuecomment-678771791
@viirya @maropu can you please take a look and see if this can get in.
thanks
This is an automated message
guykhazma commented on pull request #28826:
URL: https://github.com/apache/spark/pull/28826#issuecomment-656699933
@viirya @maropu any comments?
This is an automated message from the Apache Git Service.
To respond to the
guykhazma commented on pull request #28826:
URL: https://github.com/apache/spark/pull/28826#issuecomment-647571832
@viirya `derivedFromAtt` is set to `false` when the expression is requesting
a nested field .
The metadata for a nested column is not preserved (also in Spark 2.4) so I
am
guykhazma commented on pull request #28826:
URL: https://github.com/apache/spark/pull/28826#issuecomment-647140487
@HyukjinKwon @maropu @dongjoon-hyun @viirya added a test and also another
fix that was needed - the function `sortLeftFieldsByRight` also ignored the
`metadata` field.
guykhazma commented on pull request #28826:
URL: https://github.com/apache/spark/pull/28826#issuecomment-647112895
@dongjoon-hyun did you make sure to run it with datasource v2?
I will add tests for both and a better explanation later today.
guykhazma commented on pull request #28826:
URL: https://github.com/apache/spark/pull/28826#issuecomment-644146555
@maropu which test do you suggest to add?
This is private function which is not tested anywhere also not all file
formats are able to save the metadata (for example csv) so
