Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/940#issuecomment-50379548
@dlwh Thanks for the quick reply! The `commons-math3` problem is not which
version to use but how to match the version hadoop depends on. We can switch to
3.1.1 to match hadoop 2.3/2.4 for now. But if hadoop upgrades to 3.2 or 3.3 in
the future, we still have version conflicts unless we maintain a version
mapping in our build or shade commons-math3. I tried to downgrade
`commons-math3` to 3.1.1 in breeze and found two missing in 3.1.1: `Erf` and
`LevyDistribution`.
mllib uses `LBFGS` from breeze, which depends on `FirstOrderMinimizer`. But
it doesn't touch `RandBasis`. I'll check whether it is safe to exclude
`commons-math3` in mllib. If it is, we should probably do that.
@marmbrus Thanks! Then it should be safe to exclude `quasiquotes`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
