Re: [R] PSOCK cluster and renice

[Henrik Bengtsson]

As a follow up, future 1.7.0 was just released on CRAN allowing you
specify 'renice' as expected.  Example (skip 'dryrun = TRUE' for
actually usage):

> cl <- future::makeClusterPSOCK(2L, renice = 19, dryrun = TRUE)

--
Manually start worker #1 on 'localhost' with:
  nice --adjustment=19 '/usr/lib/R/bin/Rscript'
--default-packages=datasets,utils,grDevices,graphics,stats,methods -e
'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11414 OUT=/dev/null
TIMEOUT=2592000 XDR=TRUE
--
Manually start worker #2 on 'localhost' with:
  nice --adjustment=19 '/usr/lib/R/bin/Rscript'
--default-packages=datasets,utils,grDevices,graphics,stats,methods -e
'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11414 OUT=/dev/null
TIMEOUT=2592000 XDR=TRUE

/Henrik

On Sun, Dec 3, 2017 at 9:06 PM, Andreas Leha
 wrote:
> Hi Henrik,
>
> Thanks for the detailed in fast reply!
>
> My guess would be that the confusion comes from the different use of nice and 
> renice.
>
> The workraund you provided work fine!  Thanks a lot.
>
> Best,
> Andreas
>
>
>
> Henrik Bengtsson  writes:
>
>> Looks like a bug to me due to wrong assumptions about 'nice'
>> arguments, but could be because a "non-standard" 'nice' is used.  If
>> we do:
>>
>>> trace(system, tracer = quote(print(command)))
>> Tracing function "system" in package "base"
>>
>> we see that the system call used is:
>>
>>> cl <- parallel::makePSOCKcluster(2L, renice = 19)
>> Tracing system(cmd, wait = FALSE) on entry
>> [1] "nice +19 '/usr/lib/R/bin/Rscript'
>> --default-packages=datasets,utils,grDevices,graphics,stats,methods -e
>> 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11146 OUT=/dev/null
>> TIMEOUT=2592000 XDR=TRUE"
>> nice: ‘+19’: No such file or directory
>> ^C
>>
>> The code that prepends that 'nice +19' is in parallel:::newPSOCKnode:
>>
>> if (!is.na(renice) && renice)
>> cmd <- sprintf("nice +%d %s", as.integer(renice), cmd)
>>
>> I don't know where that originates from and on what platform it was
>> tests/validated.  On Ubuntu 16.04, CentOS 6.6, and CentOS 7.4, I have
>> 'nice' from "GNU coreutils" and they all complain about using '+',
>> e.g.
>>
>> $ nice +19 date
>> nice: +19: No such file or directory
>>
>> but '-' works:
>>
>> $ nice -19 date
>> Sun Dec  3 20:01:31 PST 2017
>>
>> Neither 'nice --help' nor 'man help' mention the use of a +n option.
>>
>>
>> WORKAROUND:  As a workaround, you can use:
>>
>> cl <- future::makeClusterPSOCK(2L, rscript = c("nice",
>> "--adjustment=10", file.path(R.home("bin"), "Rscript")))
>>
>> which is backward compatible with parallel::makePSOCKcluster() but
>> provides you with more detailed control.  Try adding verbose = TRUE to
>> see what the exact call looks like.
>>
>> /Henrik
>>
>>
>> On Sun, Dec 3, 2017 at 7:35 PM, Andreas Leha
>>  wrote:
>>> Hi all,
>>>
>>> Is it possible to use the 'renice' option together with parallel
>>> clusters of type 'PSOCK'?  The help page for parallel::makeCluster is
>>> not specific about which options are supported on which types and I am
>>> getting the following message when passing renice = 19 :
>>>
 cl <- parallel::makeCluster(2, renice = 19)
>>> nice: ‘+19’: No such file or directory
>>>
>>> Kind regards,
>>> Andreas
>>>
>>> __
>>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] PSOCK cluster and renice

[Andreas Leha]

Hi Henrik,

Thanks for the detailed in fast reply!

My guess would be that the confusion comes from the different use of nice and 
renice.

The workraund you provided work fine!  Thanks a lot.

Best,
Andreas



Henrik Bengtsson  writes:

> Looks like a bug to me due to wrong assumptions about 'nice'
> arguments, but could be because a "non-standard" 'nice' is used.  If
> we do:
>
>> trace(system, tracer = quote(print(command)))
> Tracing function "system" in package "base"
>
> we see that the system call used is:
>
>> cl <- parallel::makePSOCKcluster(2L, renice = 19)
> Tracing system(cmd, wait = FALSE) on entry
> [1] "nice +19 '/usr/lib/R/bin/Rscript'
> --default-packages=datasets,utils,grDevices,graphics,stats,methods -e
> 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11146 OUT=/dev/null
> TIMEOUT=2592000 XDR=TRUE"
> nice: ‘+19’: No such file or directory
> ^C
>
> The code that prepends that 'nice +19' is in parallel:::newPSOCKnode:
>
> if (!is.na(renice) && renice)
> cmd <- sprintf("nice +%d %s", as.integer(renice), cmd)
>
> I don't know where that originates from and on what platform it was
> tests/validated.  On Ubuntu 16.04, CentOS 6.6, and CentOS 7.4, I have
> 'nice' from "GNU coreutils" and they all complain about using '+',
> e.g.
>
> $ nice +19 date
> nice: +19: No such file or directory
>
> but '-' works:
>
> $ nice -19 date
> Sun Dec  3 20:01:31 PST 2017
>
> Neither 'nice --help' nor 'man help' mention the use of a +n option.
>
>
> WORKAROUND:  As a workaround, you can use:
>
> cl <- future::makeClusterPSOCK(2L, rscript = c("nice",
> "--adjustment=10", file.path(R.home("bin"), "Rscript")))
>
> which is backward compatible with parallel::makePSOCKcluster() but
> provides you with more detailed control.  Try adding verbose = TRUE to
> see what the exact call looks like.
>
> /Henrik
>
>
> On Sun, Dec 3, 2017 at 7:35 PM, Andreas Leha
>  wrote:
>> Hi all,
>>
>> Is it possible to use the 'renice' option together with parallel
>> clusters of type 'PSOCK'?  The help page for parallel::makeCluster is
>> not specific about which options are supported on which types and I am
>> getting the following message when passing renice = 19 :
>>
>>> cl <- parallel::makeCluster(2, renice = 19)
>> nice: ‘+19’: No such file or directory
>>
>> Kind regards,
>> Andreas
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] PSOCK cluster and renice

[Henrik Bengtsson]

Looks like a bug to me due to wrong assumptions about 'nice'
arguments, but could be because a "non-standard" 'nice' is used.  If
we do:

> trace(system, tracer = quote(print(command)))
Tracing function "system" in package "base"

we see that the system call used is:

> cl <- parallel::makePSOCKcluster(2L, renice = 19)
Tracing system(cmd, wait = FALSE) on entry
[1] "nice +19 '/usr/lib/R/bin/Rscript'
--default-packages=datasets,utils,grDevices,graphics,stats,methods -e
'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11146 OUT=/dev/null
TIMEOUT=2592000 XDR=TRUE"
nice: ‘+19’: No such file or directory
^C

The code that prepends that 'nice +19' is in parallel:::newPSOCKnode:

if (!is.na(renice) && renice)
cmd <- sprintf("nice +%d %s", as.integer(renice), cmd)

I don't know where that originates from and on what platform it was
tests/validated.  On Ubuntu 16.04, CentOS 6.6, and CentOS 7.4, I have
'nice' from "GNU coreutils" and they all complain about using '+',
e.g.

$ nice +19 date
nice: +19: No such file or directory

but '-' works:

$ nice -19 date
Sun Dec  3 20:01:31 PST 2017

Neither 'nice --help' nor 'man help' mention the use of a +n option.


WORKAROUND:  As a workaround, you can use:

cl <- future::makeClusterPSOCK(2L, rscript = c("nice",
"--adjustment=10", file.path(R.home("bin"), "Rscript")))

which is backward compatible with parallel::makePSOCKcluster() but
provides you with more detailed control.  Try adding verbose = TRUE to
see what the exact call looks like.

/Henrik


On Sun, Dec 3, 2017 at 7:35 PM, Andreas Leha
 wrote:
> Hi all,
>
> Is it possible to use the 'renice' option together with parallel
> clusters of type 'PSOCK'?  The help page for parallel::makeCluster is
> not specific about which options are supported on which types and I am
> getting the following message when passing renice = 19 :
>
>> cl <- parallel::makeCluster(2, renice = 19)
> nice: ‘+19’: No such file or directory
>
> Kind regards,
> Andreas
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] PSOCK cluster and renice

[Andreas Leha]

Hi all,

Is it possible to use the 'renice' option together with parallel
clusters of type 'PSOCK'?  The help page for parallel::makeCluster is
not specific about which options are supported on which types and I am
getting the following message when passing renice = 19 :

> cl <- parallel::makeCluster(2, renice = 19)
nice: ‘+19’: No such file or directory

Kind regards,
Andreas

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.