Re: How to integrate Spark with OpenCV?

[Jishnu Prathap]

Hi Purbanir


Integrating Spark with OpenCV was pretty straightforward .Only 2 things to keep 
in mind ,OpenCV  should be installed in each worker and Sytem.loadlibrary() 
should be written in the program such that it is invoked for each worker once.


Thanks & Regards

 Jishnu Prathap

 jishnuprathap...@gmail.com



From: purbanir [via Apache Spark User List] 

Sent: Wednesday, May 18, 2016 9:43:40 PM
To: Jishnu Menath Prathap (CTO Office)
Subject: Re: How to integrate Spark with OpenCV?


** This mail has been sent from an external source **

Hi

I will need to implement the same thing in few months. It seems that nobody has 
done this before. Nevertheless there are some information we can use:
http://personals.ac.upc.edu/rtous/howto_spark_opencv.xhtml
or this
http://samos-it.com/posts/computer-vision-opencv-sift-surf-kmeans-on-spark.html
Let me know how are you doing with this problem.

Cheers



If you reply to this email, your message will be added to the discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/How-to-integrate-Spark-with-OpenCV-tp21133p26977.html
To unsubscribe from How to integrate Spark with OpenCV?, click 
here<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=21133&code=amlzaG51LnByYXRoYXBAd2lwcm8uY29tfDIxMTMzfC0xNzUwOTc3MjE3>.
NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
The information contained in this electronic message and any attachments to 
this message are intended for the exclusive use of the addressee(s) and may 
contain proprietary, confidential or privileged information. If you are not the 
intended recipient, you should not disseminate, distribute or copy this e-mail. 
Please notify the sender immediately and destroy all copies of this message and 
any attachments. WARNING: Computer viruses can be transmitted via email. The 
recipient should check this email and any attachments for the presence of 
viruses. The company accepts no liability for any damage caused by any virus 
transmitted by this email. www.wipro.com




--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/How-to-integrate-Spark-with-OpenCV-tp21133p27000.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: How to integrate Spark with OpenCV?

[Alonso Isidoro Roman]

thanks Jorn, sorry for the special character your name needs, i dont know
how to use it. I was thinking the same. Do you know somebody that tries to
use this approach?

Alonso Isidoro Roman.

Mis citas preferidas (de hoy) :
"Si depurar es el proceso de quitar los errores de software, entonces
programar debe ser el proceso de introducirlos..."
 -  Edsger Dijkstra

My favorite quotes (today):
"If debugging is the process of removing software bugs, then programming
must be the process of putting ..."
  - Edsger Dijkstra

"If you pay peanuts you get monkeys"


2015-01-14 12:50 GMT+01:00 Jörn Franke :

> Basically, you have to think about how to split the data (for pictures
> this can be for instance 8x8 matrices) and use spark to distribute it to
> different workers which themselves call opencv with the data. Afterwards
> you need to combine all results again. It really depends on your image /
> video domain how you do this, so I can give you only general instructions.
> Le 14 janv. 2015 12:44, "Alonso Isidoro Roman"  a
> écrit :
>
> I was thinking about if it could be possible to use apache spark, and
>> opencv in order to recognize stellar objects from the data set provided by
>> Nasa,Esa, and the others space agencies. I am asking about advice or
>> feelings about it, nor example codes or something else, but if there were
>> any, please provide.
>>
>> Thanks again and forgive the possible off topic.
>>
>> Alonso
>>
>> Alonso Isidoro Roman.
>>
>> Mis citas preferidas (de hoy) :
>> "Si depurar es el proceso de quitar los errores de software, entonces
>> programar debe ser el proceso de introducirlos..."
>>  -  Edsger Dijkstra
>>
>> My favorite quotes (today):
>> "If debugging is the process of removing software bugs, then programming
>> must be the process of putting ..."
>>   - Edsger Dijkstra
>>
>> "If you pay peanuts you get monkeys"
>>
>>
>> 2015-01-14 10:49 GMT+01:00 Akhil Das :
>>
>>> Here's an example for detecting the face
>>> https://github.com/openreserach/bin2seq/blob/master/src/test/java/com/openresearchinc/hadoop/test/SparkTest.java#L190
>>> you might find it useful.
>>>
>>> Thanks
>>> Best Regards
>>>
>>> On Wed, Jan 14, 2015 at 3:06 PM,  wrote:
>>>
>>>>  Hi Akhil
>>>>
>>>> Thanks for the response
>>>>
>>>> Our use case is  Object detection in  multiple videos. It's kind of
>>>> searching an image if present in the video by matching the image with all
>>>> the frames of the video. I am able to do it in normal java code using
>>>> OpenCV lib now but I don't think it is scalable to an extend we could
>>>> implement it for thousands of large sized videos. So I thought we could
>>>> leverage distributed computing and performance of spark *If possible*
>>>> *.*
>>>>
>>>> I could see Jaonary Rabarisoa
>>>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=340>
>>>> has tried to use OpenCV with spark
>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Getting-started-using-spark-for-computer-vision-and-video-analytics-td1551.html.
>>>> But I don't have any code reference on how to do it with OpenCV.
>>>>
>>>> In case any Image+Video processing library works better with Spark plz
>>>> let me know. Any help would be really appreciated.
>>>>
>>>> .
>>>>
>>>> Thanks & Regards
>>>>
>>>> Jishnu Menath Prathap
>>>>
>>>> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com]
>>>> *Sent:* Wednesday, January 14, 2015 12:35 PM
>>>> *To:* Jishnu Menath Prathap (WT01 - BAS)
>>>> *Cc:* user@spark.apache.org
>>>> *Subject:* Re: How to integrate Spark with OpenCV?
>>>>
>>>>
>>>>
>>>> I ddn't played with OpenCV yet, but i was just wondering about your
>>>> use-case. What exactly are you trying to do?
>>>>
>>>>
>>>>   Thanks
>>>>
>>>> Best Regards
>>>>
>>>>
>>>>
>>>> Jishnu Prathap  wrote:
>>>>
>>>>
>>>>
>>>> Hi, Can somone suggest any Video+image processing library which works
>>>> well with spark. Currently i am trying to integrate OpenCV with Spark. I am
>>>> relatively new to both spark and OpenCV It would really help me if someone
>>>> could share some sample code how to use Mat ,IplImage and spark rdd 's
>>>> together .Any help would be really appreciated. Thanks in Advance!!
>>>>  --
>>>>
>>>> View this message in context: How to integrate Spark with OpenCV?
>>>> <http://apache-spark-user-list.1001560.n3.nabble.com/How-to-integrate-Spark-with-OpenCV-tp21133.html>
>>>> Sent from the Apache Spark User List mailing list archive
>>>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>>>>
>>>>
>>>>
>>>
>>>
>>

Re: How to integrate Spark with OpenCV?

[Jörn Franke]

Basically, you have to think about how to split the data (for pictures this
can be for instance 8x8 matrices) and use spark to distribute it to
different workers which themselves call opencv with the data. Afterwards
you need to combine all results again. It really depends on your image /
video domain how you do this, so I can give you only general instructions.
Le 14 janv. 2015 12:44, "Alonso Isidoro Roman"  a écrit
:

> I was thinking about if it could be possible to use apache spark, and
> opencv in order to recognize stellar objects from the data set provided by
> Nasa,Esa, and the others space agencies. I am asking about advice or
> feelings about it, nor example codes or something else, but if there were
> any, please provide.
>
> Thanks again and forgive the possible off topic.
>
> Alonso
>
> Alonso Isidoro Roman.
>
> Mis citas preferidas (de hoy) :
> "Si depurar es el proceso de quitar los errores de software, entonces
> programar debe ser el proceso de introducirlos..."
>  -  Edsger Dijkstra
>
> My favorite quotes (today):
> "If debugging is the process of removing software bugs, then programming
> must be the process of putting ..."
>   - Edsger Dijkstra
>
> "If you pay peanuts you get monkeys"
>
>
> 2015-01-14 10:49 GMT+01:00 Akhil Das :
>
>> Here's an example for detecting the face
>> https://github.com/openreserach/bin2seq/blob/master/src/test/java/com/openresearchinc/hadoop/test/SparkTest.java#L190
>> you might find it useful.
>>
>> Thanks
>> Best Regards
>>
>> On Wed, Jan 14, 2015 at 3:06 PM,  wrote:
>>
>>>  Hi Akhil
>>>
>>> Thanks for the response
>>>
>>> Our use case is  Object detection in  multiple videos. It’s kind of
>>> searching an image if present in the video by matching the image with all
>>> the frames of the video. I am able to do it in normal java code using
>>> OpenCV lib now but I don’t think it is scalable to an extend we could
>>> implement it for thousands of large sized videos. So I thought we could
>>> leverage distributed computing and performance of spark *If possible**.*
>>>
>>> I could see Jaonary Rabarisoa
>>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=340>
>>> has tried to use OpenCV with spark
>>> http://apache-spark-user-list.1001560.n3.nabble.com/Getting-started-using-spark-for-computer-vision-and-video-analytics-td1551.html.
>>> But I don’t have any code reference on how to do it with OpenCV.
>>>
>>> In case any Image+Video processing library works better with Spark plz
>>> let me know. Any help would be really appreciated.
>>>
>>> .
>>>
>>> Thanks & Regards
>>>
>>> Jishnu Menath Prathap
>>>
>>> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com]
>>> *Sent:* Wednesday, January 14, 2015 12:35 PM
>>> *To:* Jishnu Menath Prathap (WT01 - BAS)
>>> *Cc:* user@spark.apache.org
>>> *Subject:* Re: How to integrate Spark with OpenCV?
>>>
>>>
>>>
>>> I ddn't played with OpenCV yet, but i was just wondering about your
>>> use-case. What exactly are you trying to do?
>>>
>>>
>>>   Thanks
>>>
>>> Best Regards
>>>
>>>
>>>
>>> Jishnu Prathap  wrote:
>>>
>>>
>>>
>>> Hi, Can somone suggest any Video+image processing library which works
>>> well with spark. Currently i am trying to integrate OpenCV with Spark. I am
>>> relatively new to both spark and OpenCV It would really help me if someone
>>> could share some sample code how to use Mat ,IplImage and spark rdd 's
>>> together .Any help would be really appreciated. Thanks in Advance!!
>>>  --
>>>
>>> View this message in context: How to integrate Spark with OpenCV?
>>> <http://apache-spark-user-list.1001560.n3.nabble.com/How-to-integrate-Spark-with-OpenCV-tp21133.html>
>>> Sent from the Apache Spark User List mailing list archive
>>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>>>
>>>
>>>
>>
>>
>

Re: How to integrate Spark with OpenCV?

[Alonso Isidoro Roman]

I was thinking about if it could be possible to use apache spark, and
opencv in order to recognize stellar objects from the data set provided by
Nasa,Esa, and the others space agencies. I am asking about advice or
feelings about it, nor example codes or something else, but if there were
any, please provide.

Thanks again and forgive the possible off topic.

Alonso

Alonso Isidoro Roman.

Mis citas preferidas (de hoy) :
"Si depurar es el proceso de quitar los errores de software, entonces
programar debe ser el proceso de introducirlos..."
 -  Edsger Dijkstra

My favorite quotes (today):
"If debugging is the process of removing software bugs, then programming
must be the process of putting ..."
  - Edsger Dijkstra

"If you pay peanuts you get monkeys"


2015-01-14 10:49 GMT+01:00 Akhil Das :

> Here's an example for detecting the face
> https://github.com/openreserach/bin2seq/blob/master/src/test/java/com/openresearchinc/hadoop/test/SparkTest.java#L190
> you might find it useful.
>
> Thanks
> Best Regards
>
> On Wed, Jan 14, 2015 at 3:06 PM,  wrote:
>
>>  Hi Akhil
>>
>> Thanks for the response
>>
>> Our use case is  Object detection in  multiple videos. It's kind of
>> searching an image if present in the video by matching the image with all
>> the frames of the video. I am able to do it in normal java code using
>> OpenCV lib now but I don't think it is scalable to an extend we could
>> implement it for thousands of large sized videos. So I thought we could
>> leverage distributed computing and performance of spark *If possible**.*
>>
>> I could see Jaonary Rabarisoa
>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=340>
>> has tried to use OpenCV with spark
>> http://apache-spark-user-list.1001560.n3.nabble.com/Getting-started-using-spark-for-computer-vision-and-video-analytics-td1551.html.
>> But I don't have any code reference on how to do it with OpenCV.
>>
>> In case any Image+Video processing library works better with Spark plz
>> let me know. Any help would be really appreciated.
>>
>> .
>>
>> Thanks & Regards
>>
>> Jishnu Menath Prathap
>>
>> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com]
>> *Sent:* Wednesday, January 14, 2015 12:35 PM
>> *To:* Jishnu Menath Prathap (WT01 - BAS)
>> *Cc:* user@spark.apache.org
>> *Subject:* Re: How to integrate Spark with OpenCV?
>>
>>
>>
>> I ddn't played with OpenCV yet, but i was just wondering about your
>> use-case. What exactly are you trying to do?
>>
>>
>>   Thanks
>>
>> Best Regards
>>
>>
>>
>> Jishnu Prathap  wrote:
>>
>>
>>
>> Hi, Can somone suggest any Video+image processing library which works
>> well with spark. Currently i am trying to integrate OpenCV with Spark. I am
>> relatively new to both spark and OpenCV It would really help me if someone
>> could share some sample code how to use Mat ,IplImage and spark rdd 's
>> together .Any help would be really appreciated. Thanks in Advance!!
>>  --
>>
>> View this message in context: How to integrate Spark with OpenCV?
>> <http://apache-spark-user-list.1001560.n3.nabble.com/How-to-integrate-Spark-with-OpenCV-tp21133.html>
>> Sent from the Apache Spark User List mailing list archive
>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>>
>>
>>
>
>

Re: How to integrate Spark with OpenCV?

[Akhil Das]

Here's an example for detecting the face
https://github.com/openreserach/bin2seq/blob/master/src/test/java/com/openresearchinc/hadoop/test/SparkTest.java#L190
you might find it useful.

Thanks
Best Regards

On Wed, Jan 14, 2015 at 3:06 PM,  wrote:

>  Hi Akhil
>
> Thanks for the response
>
> Our use case is  Object detection in  multiple videos. It’s kind of
> searching an image if present in the video by matching the image with all
> the frames of the video. I am able to do it in normal java code using
> OpenCV lib now but I don’t think it is scalable to an extend we could
> implement it for thousands of large sized videos. So I thought we could
> leverage distributed computing and performance of spark *If possible**.*
>
> I could see Jaonary Rabarisoa
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=340>
> has tried to use OpenCV with spark
> http://apache-spark-user-list.1001560.n3.nabble.com/Getting-started-using-spark-for-computer-vision-and-video-analytics-td1551.html.
> But I don’t have any code reference on how to do it with OpenCV.
>
> In case any Image+Video processing library works better with Spark plz let
> me know. Any help would be really appreciated.
>
> .
>
> Thanks & Regards
>
> Jishnu Menath Prathap
>
> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com]
> *Sent:* Wednesday, January 14, 2015 12:35 PM
> *To:* Jishnu Menath Prathap (WT01 - BAS)
> *Cc:* user@spark.apache.org
> *Subject:* Re: How to integrate Spark with OpenCV?
>
>
>
> I ddn't played with OpenCV yet, but i was just wondering about your
> use-case. What exactly are you trying to do?
>
>
>   Thanks
>
> Best Regards
>
>
>
> Jishnu Prathap  wrote:
>
>
>
> Hi, Can somone suggest any Video+image processing library which works well
> with spark. Currently i am trying to integrate OpenCV with Spark. I am
> relatively new to both spark and OpenCV It would really help me if someone
> could share some sample code how to use Mat ,IplImage and spark rdd 's
> together .Any help would be really appreciated. Thanks in Advance!!
>  --
>
> View this message in context: How to integrate Spark with OpenCV?
> <http://apache-spark-user-list.1001560.n3.nabble.com/How-to-integrate-Spark-with-OpenCV-tp21133.html>
> Sent from the Apache Spark User List mailing list archive
> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>
>
>

RE: How to integrate Spark with OpenCV?

[jishnu.prathap]

Hi Akhil
Thanks for the response
Our use case is  Object detection in  multiple videos. It’s kind of searching 
an image if present in the video by matching the image with all the frames of 
the video. I am able to do it in normal java code using OpenCV lib now but I 
don’t think it is scalable to an extend we could implement it for thousands of 
large sized videos. So I thought we could leverage distributed computing and 
performance of spark If possible.
I could see Jaonary 
Rabarisoa<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=340>
 has tried to use OpenCV with spark 
http://apache-spark-user-list.1001560.n3.nabble.com/Getting-started-using-spark-for-computer-vision-and-video-analytics-td1551.html.
 But I don’t have any code reference on how to do it with OpenCV.
In case any Image+Video processing library works better with Spark plz let me 
know. Any help would be really appreciated.
.
Thanks & Regards
Jishnu Menath Prathap
From: Akhil Das [mailto:ak...@sigmoidanalytics.com]
Sent: Wednesday, January 14, 2015 12:35 PM
To: Jishnu Menath Prathap (WT01 - BAS)
Cc: user@spark.apache.org
Subject: Re: How to integrate Spark with OpenCV?

I ddn't played with OpenCV yet, but i was just wondering about your use-case. 
What exactly are you trying to do?

Thanks
Best Regards

Jishnu Prathap mailto:jishnu.prat...@wipro.com>> 
wrote:

Hi, Can somone suggest any Video+image processing library which works well with 
spark. Currently i am trying to integrate OpenCV with Spark. I am relatively 
new to both spark and OpenCV It would really help me if someone could share 
some sample code how to use Mat ,IplImage and spark rdd 's together .Any help 
would be really appreciated. Thanks in Advance!!

View this message in context: How to integrate Spark with 
OpenCV?<http://apache-spark-user-list.1001560.n3.nabble.com/How-to-integrate-Spark-with-OpenCV-tp21133.html>
Sent from the Apache Spark User List mailing list 
archive<http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.

Re: How to integrate Spark with OpenCV?

[Akhil Das]

I ddn't played with OpenCV yet, but i was just wondering about your
use-case. What exactly are you trying to do?

Thanks
Best Regards

On Wed, Jan 14, 2015 at 12:02 PM, Jishnu Prathap 
wrote:

> Hi, Can somone suggest any Video+image processing library which works well
> with spark. Currently i am trying to integrate OpenCV with Spark. I am
> relatively new to both spark and OpenCV It would really help me if someone
> could share some sample code how to use Mat ,IplImage and spark rdd 's
> together .Any help would be really appreciated. Thanks in Advance!!
> --
> View this message in context: How to integrate Spark with OpenCV?
> 
> Sent from the Apache Spark User List mailing list archive
>  at Nabble.com.
>