steveloughran commented on issue #1948: HADOOP-16855. wildfly classpath/loading issues URL: https://github.com/apache/hadoop/pull/1948#issuecomment-613339093 > Sure, wildfly is another dependency, but it has no transitive dependencies, so not sure what the harm is with including it. It doesn't do any weird shading of other dependencies either. This also forces 1 .It's another dependency that everyone downstream has to get right 2. It's a new one which nobody was expecting and if they don't get their packaging done it will be missing: S3A stops working. 3. if it is on your CP and there's a native OpenSSL implementation which wildfly was incompatible with, you get to see a stack trace 4. and if someone somehow gets a 1.0.4.Final version on the CP (and remember, it's been hidden in third-partly JARs like the azure-datalake JAR), then instantiating S3 or ABFS clients will crash on any Linux distro with 1.1.1 in. issues 3 & 4 is critical for me, even on the "this is harmless" execution path, wildfly on the CP could break things. Yes, that's probably a bug in wildfly, and it's fixed now. but that doesn't mean that a new openSSL release will cause it to come back. * there are lots of deployments where wildfly is not only not needed, if it causes things to break it is in fact dangerous( bq. the main concern I have is with the behavior change to the openssl option.) An ASF Hadoop release with wildfly support hasn't shipped. There is no behaviour change. Yes, there may be changes with releases by other entities, but as past changes have long determined, that cannot be a reason to hole back changes in the hadoop codebase. bq. I can see the hesitation in adding another dependency. on the other hand, the wildfly jar is just a runtime dependency, not compile time. If you need it on the classpath downstream the difference between compile and runtime is moot. If it is not there, your code doesn't work. If it is there and incompatible with your next distribution, your application breaks. This is not a transparent change. Put differently: If I had noticed the previous patch mandated widfly as a dependency, it would have been held back for an update to remove that requirement. > the main concern I have is with the behavior change to the openssl option. The S3A Binding already handled lots of failures including class not found. I'm just expanding it a bit, with the goal of resilience.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
