Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1751#issuecomment-51303742
@tdas the problem is that the dependency is already there. Spark core uses
Zookeeper classes directly in `SparkCuratorUtil`. If you remove Curator, the
code no longer compiles. Now, granted, if you removed Curator, you'd be
removing this class too. So it's kind of OK to let the transitive dependency
happen in practice in Spark Core.
However, it is not pulling in the version declared in `zookeeper.version`
(necessarily). In fact, that property is not used at all.
It exists I think for the benefit of vendors who are building the whole
thing for a system that uses a particular zookeeper version, as evidenced by
its presence in the MapR build. I think the intent is to control the version
Curator depends on. So I think there is an intent for Core to depend directly
on ZK for this reason?
I wouldn't agree that letting the transitive dependency happen to cover
this is a good idea for the Kafka test though. There, it uses Zookeeper
independently of Curator. It's a test dependency too so doesn't affect the
non-test artifacts.
It would be more correct/robust to simply express the dependency on
zookeeper in this case. I'll open a PR that shows what that looks like.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
