Andrey,
> I'm trying to learn how HPX works and whether it is applicable to my
> needs. Basically, what I'm trying to do is to parallelize parts of my
> code that are currently running sequentially, within a number of OS
> threads. I've written the following simple code to see how HPX can be
> used to achieve what I want:
>
> #include <unistd.h>
> #include <iostream>
> #include <thread>
> #include <vector>
> #include <iterator>
> #include <hpx/hpx_init.hpp>
> #include <hpx/parallel/execution_policy.hpp>
> #include <hpx/parallel/algorithms/for_each.hpp>
>
> unsigned int values[1000] = {};
>
> void process(unsigned int& value)
> {
> // Some useful work is performed here
> sleep(1000);
> }
>
> void my_thread()
> {
> // Assume this thread is getting data
> // from some kind of blocking IO.
> // I want to process this data in multiple
> // threads.
> hpx::parallel::for_each_n(hpx::parallel::par,
> std::begin(values), 1000, &process);
> }
>
> int hpx_main(int argc, char* argv[])
> {
> std::thread th(&my_thread);
>
> th.join();
>
> return hpx::finalize();
> }
>
> int main(int argc, char *argv[])
> {
> std::vector< char* > args;
> for (int i = 0; i < argc; ++i)
> args.push_back(argv[i]);
>
> // Use all hardware threads
> args.push_back(const_cast< char* >("--hpx:threads"));
> args.push_back(const_cast< char* >("all"));
>
> return hpx::init(args.size(), args.data());
> }
>
> My expectation of this code is that hpx::parallel::for_each_n would
> offload some iterations to the threads HPX spawns in its thread pool,
> and the rest is done in the my_thread thread. I would expect all
> iterations to be done in the my_thread thread if all HPX threads are
> busy doing something else, but in this example there's no other work to
> do, so this should not happen.
>
> However, that is exactly what I'm observing. The process function is
> only called in one thread - my_thread. There are multiple HPX threads
> started, and they are doing something (i.e. all but one CPU logical
> cores are heavilly loaded), but they are not calling process.
>
> So my questions are:
>
> 1. Is what I'm trying to do implementable with HPX?
Yes.
> 2. Am I doing something wrong in the above code?
See below.
> 3. What are HPX threads doing while they apparently are not doing any
> useful work? Is it normal for them to load the CPU for no apparent reason?
You told it to run on all cores, HPX is quite eager to grab all it can get.
> I'm using HPX 0.9.11 with Boost 1.60 on Kubuntu 15.10 x86_64 (gcc 5.2).
> I'm running the test app with no command line arguments and no ini files.
I'm amazed you got it to work without actually blowing up, it does not work
for me ;)
First, HPX threads and std::thread don't mix well. HPX threads are
lightweight user level threads and almost all of HPX's API functions (such
like for_each_n) expect to run on HPX threads. If you want to run work on a
kernel thread, you can do that by marshalling it over to one of the
OS-thread pools HPX is managing. This is turn is best done by creating an
executor and let the for_each_n() use that to schedule work. Note, that you
will need to use HPX from the master branch. Many of the executor facilities
have been made available only recently.
#include <hpx/hpx_init.hpp>
#include <hpx/include/parallel_executors.hpp>
#include <hpx/include/parallel_for_each.hpp>
#include <iostream>
#include <thread>
#include <vector>
#include <iterator>
unsigned int values[1000] = {};
// this will run on one of the kernel threads from HPX's IO thread pool
void process(unsigned int& value)
{
// Some useful work is performed here
std::this_thread::sleep_for(std::chrono::microseconds(1000));
}
int hpx_main(int argc, char* argv[])
{
// Assume this thread is getting data
// from some kind of blocking IO.
// I want to process this data in multiple
// threads.
hpx::threads::executors::io_pool_executor exec;
hpx::parallel::for_each_n(
hpx::parallel::par.on(exec),
std::begin(values), 1000, &process);
return hpx::finalize();
}
int main(int argc, char *argv[])
{
std::vector< char* > args;
for (int i = 0; i < argc; ++i)
args.push_back(argv[i]);
// Use all cores for HPX threads
args.push_back(const_cast< char* >("--hpx:threads"));
args.push_back(const_cast< char* >("all"));
args.push_back(0);
return hpx::init(args.size()-1, args.data());
}
Please note also, that in your case you don't need to let HPX run on all
cores, the number of cores you specify applies to the scheduler for HPX
threads only. The number of kernel threads spawned for the io-thread-pool is
by default 2, but can be increased by setting
hpx.threadpools.io_pool_size=N, either via command line
--hpx:ini=hpx.threadpools.io_pool_size=N, or via an explicit configuration
setting:
std::vector<std::string> cfg;
cfg.push_back("hpx.threadpools.io_pool_size=N");
hpx::init(argc, argv, cfg);
HTH
Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu
>
> Thanks.
> _______________________________________________
> hpx-users mailing list
> [email protected]
> https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
_______________________________________________
hpx-users mailing list
[email protected]
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
