We removed commons-math3 from dependencies to avoid version conflict
with hadoop-common. hadoop-common-2.3+ depends on commons-math3-3.1.1,
while breeze depends on commons-math3-3.3. 3.3 is not backward
compatible with 3.1.1. So we removed it because the breeze functions
we use do not touch commons-math3 code. As Sean suggested, please
include breeze in the dependency set of your project. Do not rely on
transitive dependencies. -Xiangrui

On Fri, Sep 26, 2014 at 9:08 AM, Jaonary Rabarisoa <[email protected]> wrote:
> I solve the problem by including the commons-math3 package in my sbt
> dependencies as Sean suggested. Thanks.
>
> On Fri, Sep 26, 2014 at 6:05 PM, Ted Yu <[email protected]> wrote:
>>
>> You can use scope of runtime.
>>
>> See
>> http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
>>
>> Cheers
>>
>> On Fri, Sep 26, 2014 at 8:57 AM, Jaonary Rabarisoa <[email protected]>
>> wrote:
>>>
>>> Thank Ted. Can you tell me how to adjust the scope ?
>>>
>>> On Fri, Sep 26, 2014 at 5:47 PM, Ted Yu <[email protected]> wrote:
>>>>
>>>> spark-core's dependency on commons-math3 is @ test scope (core/pom.xml):
>>>>     <dependency>
>>>>       <groupId>org.apache.commons</groupId>
>>>>       <artifactId>commons-math3</artifactId>
>>>>       <version>3.3</version>
>>>>       <scope>test</scope>
>>>>     </dependency>
>>>>
>>>> Adjusting the scope should solve the problem below.
>>>>
>>>> On Fri, Sep 26, 2014 at 8:42 AM, Jaonary Rabarisoa <[email protected]>
>>>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I'm using some functions from Breeze in a spark job but I get the
>>>>> following build error :
>>>>>
>>>>> Error:scalac: bad symbolic reference. A signature in RandBasis.class
>>>>> refers to term math3
>>>>> in package org.apache.commons which is not available.
>>>>> It may be completely missing from the current classpath, or the version
>>>>> on
>>>>> the classpath might be incompatible with the version used when
>>>>> compiling RandBasis.class.
>>>>>
>>>>> In my case, I just declare a new Gaussian distribution
>>>>>
>>>>> val g = new Gaussian(0d,1d)
>>>>>
>>>>> I'm using spark 1.1
>>>>>
>>>>>
>>>>> Any ideas to fix this ?
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>>
>>>>> Jao
>>>>
>>>>
>>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to