dchristle commented on pull request #32826: URL: https://github.com/apache/spark/pull/32826#issuecomment-858146238
> Thank you for your efforts. BTW, @dchristle . Please note that your ORC PR is not about ZSTD-JNI. It's native ZSTD library only. I commented on your ORC PR about the difference. > > > > For the following, I saw Kafka failures. > > > They appear to pass their respective CIs. > > <img width="852" alt="Screen Shot 2021-06-08 at 11 16 21 PM" src="https://user-images.githubusercontent.com/9700541/121303167-9afc7380-c8af-11eb-9e2e-7500a3467325.png";> > > > > No worry~ For Apache Avro, they have a dependency bot. I guess they will catch up soon. Let's wait and see their activity. > > > I have less familiarity with Avro's build chains/codebase, so I did not attempt to test it yet. > > > > In addition, all libraries should be synced inside Apache Spark because Apache Spark is using everything. Thank you for your message. The Kafka PR failures seem to not be related to the Zstd change -- the tests appear to be bugged/flaky, as many other recent PRs are also failing. I looked over the Zstd release notes but did not catch any obvious big changes that could trigger an incompatibility. However, my plan is to wait till the Kafka PR can pass the CI, and shepherd the change through. Regarding Spark: Is it necessary to have all dependencies upgraded to Zstd 1.5.x before merging if the Spark CI/dependency tests appear to pass? For instance, the move to 1.5.0-1 is a scheduled for Kafka 3.0 (unless there is a back-port), but I imagine that release will be some time from now. Regarding this PR: Do we have a good understanding of why the benchmark tests fail? I cannot tell if it is actually related to this code change. Thank you for your guidance with this process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/9700541/121303167-9afc7380-c8af-11eb-9e2e-7500a3467325.png"](https://user-images.githubusercontent.com/9700541/121303167-9afc7380-c8af-11eb-9e2e-7500a3467325.png"){kind=link}