Prachicsa:
If the number of EC tokens is high, please consider using a set instead of
array for better lookup performance.
BTW use short, descriptive subject for future emails.
> On Sep 9, 2015, at 3:13 AM, Akhil Das <ak...@sigmoidanalytics.com> wrote:
>
> Try this:
>
> val tocks = Array("EC-17A5206955089011B","EC-17A5206955089011A")
>
> val rddAll = sc.parallelize(List("This contains EC-17A5206955089011B","This
> doesnt"))
>
> rddAll.filter(line => {
> var found = false
> for(item <- tocks){
> if(line.contains(item)) found = true
> }
> found
> }).collect()
>
>
> Output:
> res8: Array[String] = Array(This contains EC-17A5206955089011B)
>
> Thanks
> Best Regards
>
>> On Wed, Sep 9, 2015 at 3:25 PM, prachicsa <prachi...@gmail.com> wrote:
>>
>>
>> I am very new to Spark.
>>
>> I have a very basic question. I have an array of values:
>>
>> listofECtokens: Array[String] = Array(EC-17A5206955089011B,
>> EC-17A5206955089011A)
>>
>> I want to filter an RDD for all of these token values. I tried the following
>> way:
>>
>> val ECtokens = for (token <- listofECtokens) rddAll.filter(line =>
>> line.contains(token))
>>
>> Output:
>>
>> ECtokens: Unit = ()
>>
>> I got an empty Unit even when there are records with these tokens. What am I
>> doing wrong?
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/I-am-very-new-to-Spark-I-have-a-very-basic-question-I-have-an-array-of-values-listofECtokens-Array-S-tp24617.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
>
