These two are giving me griefs:
scala> import org.apache.spark.internal.Logging
<console>:26: error: object internal is not a member of package
org.apache.spark
import org.apache.spark.internal.Logging
scala> import org.apache.spark.streaming.kafka.KafkaCluster.LeaderOffset
<console>:29: error: object KafkaCluster in package kafka cannot be
accessed in package org.apache.spark.streaming.kafka
import org.apache.spark.streaming.kafka.KafkaCluster.LeaderOffset
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 21 April 2016 at 18:29, Mich Talebzadeh <[email protected]>
wrote:
>
> Thanks
>
> jar tvf spark-core_2.10-1.5.1-tests.jar | grep SparkFunSuite
> 1787 Wed Sep 23 23:34:26 BST 2015
> org/apache/spark/SparkFunSuite$$anonfun$withFixture$1.class
> 1780 Wed Sep 23 23:34:26 BST 2015
> org/apache/spark/SparkFunSuite$$anonfun$withFixture$2.class
> 3982 Wed Sep 23 23:34:26 BST 2015 org/apache/spark/SparkFunSuite.class
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn *
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> On 21 April 2016 at 18:21, Ted Yu <[email protected]> wrote:
>
>> Please replace version number for the release you are using :
>>
>> spark-core_2.10-1.5.1-tests.jar
>>
>> On Thu, Apr 21, 2016 at 10:18 AM, Mich Talebzadeh <
>> [email protected]> wrote:
>>
>>> I don't seem to be able to locate
>>> spark-core_2.11-2.0.0-SNAPSHOT-tests.jar file :(
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn *
>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>>
>>> On 21 April 2016 at 17:45, Ted Yu <[email protected]> wrote:
>>>
>>>> It is in core-XX-tests jar:
>>>>
>>>> $ jar tvf ./core/target/spark-core_2.11-2.0.0-SNAPSHOT-tests.jar | grep
>>>> SparkFunSuite
>>>> 1830 Thu Apr 21 08:19:14 PDT 2016
>>>> org/apache/spark/SparkFunSuite$$anonfun$withFixture$1.class
>>>> 1823 Thu Apr 21 08:19:14 PDT 2016
>>>> org/apache/spark/SparkFunSuite$$anonfun$withFixture$2.class
>>>> 6232 Thu Apr 21 08:19:14 PDT 2016 org/apache/spark/SparkFunSuite.class
>>>>
>>>> On Thu, Apr 21, 2016 at 9:39 AM, Mich Talebzadeh <
>>>> [email protected]> wrote:
>>>>
>>>>> like war of attrition :)
>>>>>
>>>>> now I get with sbt
>>>>>
>>>>> object SparkFunSuite is not a member of package org.apache.spark
>>>>>
>>>>>
>>>>> Dr Mich Talebzadeh
>>>>>
>>>>>
>>>>>
>>>>> LinkedIn *
>>>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>>>
>>>>>
>>>>>
>>>>> http://talebzadehmich.wordpress.com
>>>>>
>>>>>
>>>>>
>>>>> On 21 April 2016 at 17:22, Ted Yu <[email protected]> wrote:
>>>>>
>>>>>> Have you tried the following ?
>>>>>>
>>>>>> libraryDependencies += "org.scalatest" %% "scalatest" % "2.2.6"
>>>>>>
>>>>>> On Thu, Apr 21, 2016 at 9:19 AM, Mich Talebzadeh <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> Unfortunately this sbt dependency is not working
>>>>>>>
>>>>>>> libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.1"
>>>>>>> % "provided"
>>>>>>> libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.5.1"
>>>>>>> % "provided"
>>>>>>> libraryDependencies += "org.apache.spark" %% "spark-hive" % "1.5.1"
>>>>>>> % "provided"
>>>>>>> libraryDependencies += "junit" % "junit" % "4.12"
>>>>>>> libraryDependencies += "org.scala-sbt" % "test-interface" % "1.0"
>>>>>>> libraryDependencies += "org.apache.spark" %% "spark-streaming" %
>>>>>>> "1.6.1" % "provided"
>>>>>>> libraryDependencies += "org.apache.spark" %% "spark-streaming-kafka"
>>>>>>> % "1.6.1"
>>>>>>> libraryDependencies += "org.scalactic" %% "scalactic" % "2.2.6"
>>>>>>> *libraryDependencies += "org.scalatest" %% "scalatest" % "2.2.6" %
>>>>>>> "test"*
>>>>>>>
>>>>>>> Getting error
>>>>>>>
>>>>>>> [info] Compiling 1 Scala source to
>>>>>>> /data6/hduser/scala/CEP_assembly/target/scala-2.10/classes...
>>>>>>> [error]
>>>>>>> /data6/hduser/scala/CEP_assembly/src/main/scala/myPackage/CEP_assemly.scala:32:
>>>>>>> object scalatest is not a member of package org
>>>>>>> [error] import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll}
>>>>>>> [error] ^
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Dr Mich Talebzadeh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> LinkedIn *
>>>>>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>>>>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> http://talebzadehmich.wordpress.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 21 April 2016 at 16:49, Mich Talebzadeh <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> Thanks Ted. It was a typo in my alias and it is sorted now
>>>>>>>>
>>>>>>>> slong='rlwrap spark-shell --master spark://50.140.197.217:7077
>>>>>>>> --jars
>>>>>>>> /home/hduser/jars/ojdbc6.jar,/home/hduser/jars/jconn4.jar,/home/hduser/jars/junit-4.12.jar,/home/hduser/jars/spark-streaming-kafka-assembly_2.10-1.6.1.jar,/home/hduser/jars/scalatest_2.11-2.2.6.jar'
>>>>>>>>
>>>>>>>>
>>>>>>>> scala> import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll}
>>>>>>>> import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll}
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Dr Mich Talebzadeh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> LinkedIn *
>>>>>>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>>>>>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> http://talebzadehmich.wordpress.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 21 April 2016 at 16:44, Ted Yu <[email protected]> wrote:
>>>>>>>>
>>>>>>>>> I tried on refreshed copy of master branch:
>>>>>>>>>
>>>>>>>>> $ bin/spark-shell --jars
>>>>>>>>> /home/hbase/.m2/repository/org/scalatest/scalatest_2.11/2.2.6/scalatest_2.11-2.2.6.jar
>>>>>>>>> ...
>>>>>>>>> scala> import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll}
>>>>>>>>> import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll}
>>>>>>>>>
>>>>>>>>> BTW I noticed an extra leading comma after '--jars' in your email.
>>>>>>>>> Not sure if that matters.
>>>>>>>>>
>>>>>>>>> On Thu, Apr 21, 2016 at 8:39 AM, Ted Yu <[email protected]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Mich:
>>>>>>>>>>
>>>>>>>>>> $ jar tvf
>>>>>>>>>> /home/hbase/.m2/repository/org/scalatest/scalatest_2.11/2.2.6/scalatest_2.11-2.2.6.jar
>>>>>>>>>> | grep BeforeAndAfter
>>>>>>>>>> 4257 Sat Dec 26 14:35:48 PST 2015
>>>>>>>>>> org/scalatest/BeforeAndAfter$class.class
>>>>>>>>>> 2602 Sat Dec 26 14:35:48 PST 2015
>>>>>>>>>> org/scalatest/BeforeAndAfter.class
>>>>>>>>>> 1998 Sat Dec 26 14:35:48 PST 2015
>>>>>>>>>> org/scalatest/BeforeAndAfterAll$$anonfun$run$1.class
>>>>>>>>>>
>>>>>>>>>> FYI
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 21, 2016 at 8:35 AM, Mich Talebzadeh <
>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>>> to Mario, Alonso, Luciano, user
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Following example in
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/agsachin/spark/blob/CEP/external/kafka/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala#L532
>>>>>>>>>>>
>>>>>>>>>>> Does anyone know which jar file this belongs to?
>>>>>>>>>>>
>>>>>>>>>>> I use *scalatest_2.11-2.2.6.jar *in my spark-shell
>>>>>>>>>>>
>>>>>>>>>>> spark-shell --master spark://50.140.197.217:7077 --jars
>>>>>>>>>>> ,/home/hduser/jars/junit-4.12.jar,/home/hduser/jars/spark-streaming-kafka-assembly_2.10-1.6.1.jar,
>>>>>>>>>>> /*home/hduser/jars/scalatest_2.11-2.2.6.jar'*
>>>>>>>>>>>
>>>>>>>>>>> scala> import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll}
>>>>>>>>>>> <console>:28: error: object scalatest is not a member of package
>>>>>>>>>>> org
>>>>>>>>>>> import org.scalatest.{BeforeAndAfter, BeforeAndAfterAll}
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>> Dr Mich Talebzadeh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> LinkedIn *
>>>>>>>>>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>>>>>>>>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://talebzadehmich.wordpress.com
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
