[hpx-users] HPX smart executors data generation

[Gabriel Laberge]

Hi,
In have a question about the HPX smart executors talked about in this article
https://arxiv.org/pdf/1711.01519.pdf . I'm not sure to understand the  
way the way the data is generated. As an example, let's say I want to  
optimize a matrix multiplication task. The executor will use a  
logistic regression to optimize the task but has the logistic  
regression already been trained with previous data from a large data  
set? Or does the executor generate data from the task I give it and  
then it trains a logistic regression using that data which would mean  
that the regression is perfectly optimized for the given task but take  
some time to train.
Thank you very much,


___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

Re: [hpx-users] GSoC 2018, on "applying machine learning technques ..." project

[Patrick Diehl]

Hi Ray,

welcome to the community. A good starting point for this project would
be to read this publication [0]:

Zahra Khatami, Lukas Troska, Hartmut Kaiser, J. Ramanujan and Adrian
Serio, “HPX Smart Executors”, In Proceedings of ESPM2’17: Third
International Workshop on Extreme Scale Programming Models and
Middleware (ESPM2’17), 2017. doi: 10.1145/3152041.3152084,

>> Then is the goal to analyze data? or also implement the algorithms?

The goal is to provide a new execution policy [1], which utilizes
machine learning techniques to optimize the computational time.

So the first step would be to analyze the data.

In my opinion there are two different ways, one can integrate machine
learning.

1) You could collect the data and train a model to obtain parameters for
your function f(x,y,z) -> chunk size. In this case you could use any
existing machine learning library to get these parameters for your function.

Here, you will implement this function in the execution policy to
estimate e.g. the "best" chunk size for given x,y,z.

2) You could train your model at run time to obtain e.g. the "best"
chunk size. In this case you have to implement your chosen machine
learning algorithm for the integration in hpx.

Best,

Patrick


[0] http://stellar.cct.lsu.edu/pubs/khatami_espm2_2017.pdf
[1]
https://stellar-group.github.io/hpx/docs/html/hpx/manual/parallel/executor_parameters.html

On 15/02/18 01:53 PM, 김규래 wrote:
> HI, my name is Ray Kim.
> 
> I'm a junior EE student in Sogang Univ. Korea.
> 
> I have a little experience in C++ and HPC applications and machine learning.
> 
> Here is a link to my github profile for some of my personal projects. 
> 
> https://github.com/Red-Portal 
> 
>  
> 
> I'm interested in the project "Applying Machine Learning Techniques on
> HPX Parallel Algorithms",
> 
> however I would like to have more details about it.
> 
> The description talks about implementing the algorithms and analyzing
> the performance of these algorithms.
> 
> Then is the goal to analyze data? or also implement the algorithms?
> 
>  
> 
> Great admirations for everyone working on HPX.
> 
> It would be an honor if I could work with you all.
> 
> msca8h at naver dot com
> 
> msca8h at sogang dot ac dot kr
> 
> Ray Kim
> 
> 
> 
> ___
> hpx-users mailing list
> hpx-users@stellar.cct.lsu.edu
> https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
> 
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

Re: [hpx-users] GSoC : "HPX Backend for OpenCV"

[Biddiscombe, John A.]

Just an update on the project from my ‘wish list’

Task #1 : have a lookm at opencv and how the threading backends are implemented
Task #2: Get a good understanding of the HPX threading framework. Make sure you 
appreciate the difference between kernel level threads and lightweight HPX 
threads. Reading from a webcam or other device might make low level calls that 
need to be done on a kernel thread rather than an hpx worker/task thread.
Task #3: Implement an HPX backend to replace OpenMP/TBB in openCV and test out 
standard opencv algorithms
Task #4: make sure that the threading implementation doesn’t violate the terms 
of #2 and that hpc threads are not blocking in wait staes whilst polling for 
camera/image data
Task #5: understand that HPX supports thread pools and we can create a custom 
thread for any opencv camera poll work so that the problems I just alluded to 
do not happen
Task #6: Together a simple Qt demo with a GUI that displays images in a nice 
GUI window and overlays some image processing data etc. I have an application I 
wrote for monitoring wildlife in my garden that can be adapted for this - it 
tracks movement by doing pixel comparisons after applying  bunch of filters to 
the webcam images - when movement is detected, it starts recording so I can see 
what happened whilst I was asleep etc (infra-red cameras for night vision).
Task #7: Understand that Qt GUI threads can’t always be run on hpx task threads 
and/or tht Qt guis are not generally thread safe and we may need a special 
thread for GUI updates (or careful use of Qt synchronization), so a thread pool 
dedicated for that might/would be a good idea.

Task #0 - learn as much as you can about all of the above before gsoc starts 
and write a decent proposal “in your own words” that describes how you plan to 
do the above (and insert extra tasks because I wrote  this fast and skipped 
lots of details).

Feel free to ask questions on IRC or here - I’ll update the task list above if 
I think of more - and note that #1-#5 are the main essentials and #6-#7 are 
bonus work for anyone who makes good progress on the first tasks.

JB




From: Marcin Copik [mailto:mco...@gmail.com]
Sent: 15 February 2018 18:02
To: Ashish Jha ; hpx-users@stellar.cct.lsu.edu
Cc: Patrick Diehl ; Biddiscombe, John A. 

Subject: Re: GSoC : "HPX Backend for OpenCV"

Dear Ashish,

cc'ing this message to the mailing list to let the other mentors and the 
community to chime in and propose their suggestions.

The project idea gives a link to the existing implementation of parallel task 
processing in OpenCV. It's a good starting point since understanding how OpenCV 
currently handles the parallelization is necessary to propose an integration 
scheme for HPX. You may also want to take a look at the guide to writing a 
successful proposal[1].

Best regards,
Marcin Copik

[1] - https://github.com/STEllAR-GROUP/hpx/wiki/Hints-for-Successful-Proposals
śr., 14 lut 2018 o 16:38 użytkownik Ashish Jha 
> napisał:
[https://api.openbracket.co/track/f8842b410f36ecfb3fc588d5bf81c7c9.gif]
Respected Sir,
I am Ashish Jha, 3rd-year computer science undergrad at NIT Rourkela.

I have worked on OpenMP project and has recently been introduced to OpenCV.  I 
will love to work on the project HPX Backend for OpenCV.

Can you please guide me where to start?
I am having exam until 27th February so I can devote up to 1 or 2 hour to the 
project and after that, I can devote full time on the project

​Attached is my CV.​

-
​​
-
Thanking you,
Ashish Kumar Jha
3rd year CSE Undergrad,
National Institute of Technology, Rourkela.
Contact: +91-9430682550
Facebook  | 
LinkedIn
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

[hpx-users] GSoC 2018, on "applying machine learning technques ..." project

[김규래]

HI, my name is Ray Kim.
I'm a junior EE student in Sogang Univ. Korea.
I have a little experience in C++ and HPC applications and machine learning.
Here is a link to my github profile for some of my personal projects. 
https://github.com/Red-Portal 
 
I'm interested in the project "Applying Machine Learning Techniques on HPX 
Parallel Algorithms",
however I would like to have more details about it.
The description talks about implementing the algorithms and analyzing the 
performance of these algorithms.
Then is the goal to analyze data? or also implement the algorithms?
 
Great admirations for everyone working on HPX.
It would be an honor if I could work with you all.
msca8h at naver dot com
msca8h at sogang dot ac dot kr
Ray Kim
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

[hpx-users] Fwd: GSoC '18 - Concurrent Data Structures

[Shikhar S]

-- Forwarded message --
From: Shikhar S 
Date: Thu, Feb 15, 2018 at 10:21 PM
Subject: Re: [hpx-users] GSoC '18 - Concurrent Data Structures
To: Patrick Diehl 


Yes I have. I have also discussed the project on freenode channel as
CaptainRubik.What I want to know is what are the data structures that the
mentors have in mind for this project apart from those in issue #2235?


On Feb 15, 2018 8:34 PM, "Patrick Diehl"  wrote:

> Hi Shikar,
>
> welcome to the community. Have you seen the hints for writing a
> successful proposals [0]?
>
> Best,
>
> Patrick
>
> [0] https://github.com/STEllAR-GROUP/hpx/wiki/Hints-for-Successf
> ul-Proposals
>
> On 15/02/18 12:14 AM, Shikhar S wrote:
> > Hi,
> > I am an engineering student doing my Bachelors from Bits Pilani,
> > currently working as an intern for Microsoft Research and working on
> > LevelDB (a concurrent API for persistent storage of string key-value
> > pairs). I want to work on Concurrent Data Structures project. I went
> > through issue #2235 and the code for concurrent vector in the Concurrent
> > Data Structure branch. I have built hpx and ran tests successfully. What
> > other data structures are expected to be implemented in hpx? Am I right
> > in assuming that we will also need lockfree implementations for vector
> > and unordered set/map ?
> >
> >
> > Regards
> > Shikhar Bharadwaj
> >
> >
> > ___
> > hpx-users mailing list
> > hpx-users@stellar.cct.lsu.edu
> > https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
> >
>
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

Re: [hpx-users] GSoC : "HPX Backend for OpenCV"

[Marcin Copik]

Dear Ashish,

cc'ing this message to the mailing list to let the other mentors and the
community to chime in and propose their suggestions.

The project idea gives a link to the existing implementation of parallel
task processing in OpenCV. It's a good starting point since understanding
how OpenCV currently handles the parallelization is necessary to propose an
integration scheme for HPX. You may also want to take a look at the guide
to writing a successful proposal[1].

Best regards,
Marcin Copik

[1] -
https://github.com/STEllAR-GROUP/hpx/wiki/Hints-for-Successful-Proposals

śr., 14 lut 2018 o 16:38 użytkownik Ashish Jha 
napisał:

> Respected Sir,
> I am Ashish Jha, 3rd-year computer science undergrad at NIT Rourkela.
>
> I have worked on OpenMP project and has recently been introduced to
> OpenCV.  I will love to work on the project HPX Backend for OpenCV.
>
> Can you please guide me where to start?
> I am having exam until 27th February so I can devote up to 1 or 2 hour to
> the project and after that, I can devote full time on the project
>
> ​Attached is my CV.​
>
> -
> ​​
> -
> Thanking you,
> Ashish Kumar Jha
> 3rd year CSE Undergrad,
> National Institute of Technology, Rourkela.
> Contact: +91-9430682550 <+91%2094306%2082550>
> Facebook | LinkedIn
> 
>
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

Re: [hpx-users] GSoC '18 - Concurrent Data Structures

[Patrick Diehl]

Hi Shikar,

welcome to the community. Have you seen the hints for writing a
successful proposals [0]?

Best,

Patrick

[0] https://github.com/STEllAR-GROUP/hpx/wiki/Hints-for-Successful-Proposals

On 15/02/18 12:14 AM, Shikhar S wrote:
> Hi, 
> I am an engineering student doing my Bachelors from Bits Pilani,
> currently working as an intern for Microsoft Research and working on
> LevelDB (a concurrent API for persistent storage of string key-value
> pairs). I want to work on Concurrent Data Structures project. I went
> through issue #2235 and the code for concurrent vector in the Concurrent
> Data Structure branch. I have built hpx and ran tests successfully. What
> other data structures are expected to be implemented in hpx? Am I right
> in assuming that we will also need lockfree implementations for vector
> and unordered set/map ?
> 
> 
> Regards
> Shikhar Bharadwaj 
> 
> 
> ___
> hpx-users mailing list
> hpx-users@stellar.cct.lsu.edu
> https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
> 
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

Re: [hpx-users] Want to work on implementing "Coroutine-like Interface" and "Adding Lustre Backend to hpxio"

[Patrick Diehl]

Hi Nikunj,

 >> P.S: Please assign issue #3124
>>  to me.

We can not assign this issues to you right now, you have to apply for
GSoC and after you got selected, we can assign you this issue.

Please find attached here some hints for writing a successful proposal [0].

Best,

Patrick

[0]
https://github.com/STEllAR-GROUP/hpx/wiki/Hints-for-Successful-Proposals



On 14/02/18 05:06 PM, Nikunj Gupta wrote:
> ​Hi!
> 
> I'm Nikunj Gupta.
> I have quite a bit of working knowledge on many programming languages
> and have worked on projects but haven't worked on the open source front yet.
> I would like to start my open source journey through hpx.
> 
> I'm interested in implementing "Coroutine-like Interface" and "Adding
> Lustre Backend to hpxio" as my GSoC 2018 Projects. I'm interested and
> enthusiastic about working with an international team of developers on
> the cutting edge of C++ parallel, task-based runtime systems.
> 
> P.S: Please assign issue #3124
>  to me.
> 
> Regards
> Nikunj Gupta
> 1st Year B.Tech (Computer Sci. Engg. )
> IIT (Indian Institute of Technology) Roorkee
> 
> 
> ___
> hpx-users mailing list
> hpx-users@stellar.cct.lsu.edu
> https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
> 
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

[hpx-users] Want to work on implementing "Coroutine-like Interface" and "Adding Lustre Backend to hpxio"

[Nikunj Gupta]

​Hi!

I'm Nikunj Gupta.
I have quite a bit of working knowledge on many programming languages and
have worked on projects but haven't worked on the open source front yet.
I would like to start my open source journey through hpx.

I'm interested in implementing "Coroutine-like Interface" and "Adding
Lustre Backend to hpxio" as my GSoC 2018 Projects. I'm interested and
enthusiastic about working with an international team of developers on the
cutting edge of C++ parallel, task-based runtime systems.

P.S: Please assign issue #3124
 to me.

Regards
Nikunj Gupta
1st Year B.Tech (Computer Sci. Engg. )
IIT (Indian Institute of Technology) Roorkee
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users

[hpx-users] GSoC '18 - Concurrent Data Structures

[Shikhar S]

Hi,
I am an engineering student doing my Bachelors from Bits Pilani, currently
working as an intern for Microsoft Research and working on LevelDB (a
concurrent API for persistent storage of string key-value pairs). I want to
work on Concurrent Data Structures project. I went through issue #2235 and
the code for concurrent vector in the Concurrent Data Structure branch. I
have built hpx and ran tests successfully. What other data structures are
expected to be implemented in hpx? Am I right in assuming that we will also
need lockfree implementations for vector and unordered set/map ?


Regards
Shikhar Bharadwaj
___
hpx-users mailing list
hpx-users@stellar.cct.lsu.edu
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users