li36909 edited a comment on pull request #2790: URL: https://github.com/apache/hudi/pull/2790#issuecomment-818547029
there are some UT fail cause by: https://github.com/apache/parquet-mr/pull/747/files at this pr, parquet set requiredSchema first then do filter, and when we run count() at spark morRelation, the requiredSchma is empty, then the filter result is empty. I upgrade parquet version for spark also, and sprk has not problem, and I hand check the reson is that: spark add filter attributes to requiredSchema like this: val requiredExpressions: Seq[NamedExpression] = filterAttributes.toSeq ++ projects -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
