Re: [Rd] Faster sorting algorithm...

[frederik]

I think it is "Professor Neal" :)

I also appreciate the pqR comparisons.

On Wed, Mar 17, 2021 at 09:23:15AM +, Morgan Morgan wrote:

Thank you Neal. This is interesting. I will have a look at pqR.
Indeed radix only does C collation, I believe that is why it is not the
default choice for character ordering and sorting.
Not sure but I believe it can help address the following bugzilla item:
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17400

On the same topic of collation, there is an experimental sorting function
"psort" in package kit that might help address this issue.


library(kit)

Attaching kit 0.0.7 (OPENMP enabled using 1 thread)

x <- c("b","A","B","a","\xe4")
Encoding(x) <- "latin1"
identical(psort(x, c.locale=FALSE), sort(x))

[1] TRUE

identical(psort(x, c.locale=TRUE), sort(x, method="radix"))

[1] TRUE

Coming back to the topic of fsort, I have just finished the implementation
for double, integer, factor and logical.
The implementation takes into account NA, Inf.. values. Values can be
sorted in a decreasing order or increasing order.
Comparing benchmark with the current implementation in data.table, it is
currently over 30% faster.
There might bugs but I am sure performance can be further improved as I did
not really try hard.
If there is interest in both the implementation and cross community
sharing, please let know

Best regards,
Morgan

On Wed, 17 Mar 2021, 00:37 Radford Neal,  wrote:


Those interested in faster sorting may want to look at the merge sort
implemented in pqR (see pqR-project.org).  It's often used as the
default, because it is stable, and does different collations, while
being faster than shell sort (except for small vectors).

Here are examples, with timings, for pqR-2020-07-23 and R-4.0.2,
compiled identically:

-
pqR-2020-07-23 in C locale:

> set.seed(1)
> N <- 100
> x <- as.character (sample(N,N,replace=TRUE))
> print(system.time (os <- order(x,method="shell")))
   user  system elapsed
  1.332   0.000   1.334
> print(system.time (or <- order(x,method="radix")))
   user  system elapsed
  0.092   0.004   0.096
> print(system.time (om <- order(x,method="merge")))
   user  system elapsed
  0.363   0.000   0.363
> print(identical(os,or))
[1] TRUE
> print(identical(os,om))
[1] TRUE
>
> x <- c("a","~")
> print(order(x,method="shell"))
[1] 1 2
> print(order(x,method="radix"))
[1] 1 2
> print(order(x,method="merge"))
[1] 1 2

-
R-4.0.2 in C locale:

> set.seed(1)
> N <- 100
> x <- as.character (sample(N,N,replace=TRUE))
> print(system.time (os <- order(x,method="shell")))
   user  system elapsed
  2.381   0.004   2.387
> print(system.time (or <- order(x,method="radix")))
   user  system elapsed
  0.138   0.000   0.137
> #print(system.time (om <- order(x,method="merge")))
> print(identical(os,or))
[1] TRUE
> #print(identical(os,om))
>
> x <- c("a","~")
> print(order(x,method="shell"))
[1] 1 2
> print(order(x,method="radix"))
[1] 1 2
> #print(order(x,method="merge"))


pqR-2020-07-23 in fr_CA.utf8 locale:

> set.seed(1)
> N <- 100
> x <- as.character (sample(N,N,replace=TRUE))
> print(system.time (os <- order(x,method="shell")))
utilisateur système  écoulé
  2.960   0.000   2.962
> print(system.time (or <- order(x,method="radix")))
utilisateur système  écoulé
  0.083   0.008   0.092
> print(system.time (om <- order(x,method="merge")))
utilisateur système  écoulé
  1.143   0.000   1.142
> print(identical(os,or))
[1] TRUE
> print(identical(os,om))
[1] TRUE
>
> x <- c("a","~")
> print(order(x,method="shell"))
[1] 2 1
> print(order(x,method="radix"))
[1] 1 2
> print(order(x,method="merge"))
[1] 2 1


R-4.0.2 in fr_CA.utf8 locale:

> set.seed(1)
> N <- 100
> x <- as.character (sample(N,N,replace=TRUE))
> print(system.time (os <- order(x,method="shell")))
utilisateur système  écoulé
  4.222   0.016   4.239
> print(system.time (or <- order(x,method="radix")))
utilisateur système  écoulé
  0.136   0.000   0.137
> #print(system.time (om <- order(x,method="merge")))
> print(identical(os,or))
[1] TRUE
> #print(identical(os,om))
>
> x <- c("a","~")
> print(order(x,method="shell"))
[1] 2 1
> print(order(x,method="radix"))
[1] 1 2
> #print(order(x,method="merge"))


pqR is faster in all the tests, but more relevant to this discussion
is that the "merge" method is substantially faster than "shell" for
these character vectors with a million elements, while retaining the
stability and collation properties of "shell" (whereas "radix" only
does C collation).

It would probably not be too hard to take the merge sort code from pqR
and use it in R core's implementation.

The merge sort in pqR doesn't exploit parallelism at the moment, but
merge sort is potentially quite parallelizable (though I think the
storage allocation strategy I use 

Re: [Rd] closing R graphics windows?

[frederik]

I use keyboard shortcuts to interact with my window manager. One of those 
shortcuts closes the currently focused window. This is on Linux.

The only annoying thing for me is that 'plot()' is not interruptible, so 
neither Ctrl-C nor the window manager can stop a plot once it has started - but 
I submitted a bug to fix this a long time ago. If I use the keyboard to close 
the window while a plot is being drawn, then it has to finish drawing before 
the window actually closes.

Frederick

On Tue, May 26, 2020 at 12:49:24PM -0400, Ben Bolker wrote:


   Does anyone have any idea how hard it would be/where to start if 
one wanted to hack/patch R to allow X11 graphics windows that had 
keyboard focus to be closed with standard keyboard shortcuts (e.g. 
Ctrl-W to close on Linux)?  Has this been suggested/tried before?


   cheers

    Ben Bolker

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] writeBin and short writes

[frederik]

Dear R-devel,

I'm curious why writeBin should always return NULL. Would anyone's code break 
if we made it return the number of bytes written? I'm not sure how else I'm 
supposed to use writeBin on a writable connection in non-blocking mode...

See attached.

Thank you,

Frederick
>From 3304e714f204c22d8e5c2ae93828683e04d1569d Mon Sep 17 00:00:00 2001
From: frede...@ofb.net
Date: Fri, 22 May 2020 15:42:37 -0700
Subject: [PATCH 1/1] Return number of bytes written

---
 src/main/connections.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/main/connections.c b/src/main/connections.c
index 88f082c696..2c47589018 100644
--- a/src/main/connections.c
+++ b/src/main/connections.c
@@ -4632,6 +4632,7 @@ SEXP attribute_hidden do_writebin(SEXP call, SEXP op, 
SEXP args, SEXP env)
} else {
size_t nwrite = con->write(buf, size, len, con);
if(nwrite < len) warning(_("problem writing to connection"));
+   ans = ScalarInteger(nwrite);
}
Free(buf);
 }
-- 
2.26.2

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] defining r audio connections

[frederik]

On Wed, May 06, 2020 at 02:26:34PM -0400, Duncan Murdoch wrote:

On 06/05/2020 1:09 p.m., frede...@ofb.net wrote:

I only want an object that I can use `readBin` and `writeBin` on, to read and 
write audio data using e.g. `snd_pcm_writei` which is part of the `alsa-lib` 
package.


I don't think R supports user-defined connections, but probably 
writing readBin and writeBin equivalents specific to your library 
wouldn't be any harder than creating a connection.  For those, you 
will probably want to work with an "external pointer" (see Writing R 
Extensions). Rcpp probably has support for these if you're working in 
C++.


Thank you, yes I was following

https://stackoverflow.com/questions/59384221/proper-way-to-return-a-pointer-to-a-new-object-from-an-rcpp-function

which uses XPtr.

I'll forget about making a connection, if it is not supported.

I had this idea that I would be able to use the same code for writing to a file 
as to an audio device, but I can create my own class for that.

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] defining r audio connections

[frederik]

Dear R Devel,

Since Linux moved away from using a file-system interface for audio, I think it 
is necessary to write special libraries to interface with audio hardware from 
various languages on Linux.

In R, it seems like the appropriate datatype for a `snd_pcm_t` handle pointing to an open ALSA source or sink would be a 
"connection". Connection types are already defined in R for "file", "url", "pipe", 
"fifo", "socketConnection", etc.

Is there a tutorial or an example package where a new type of connection is 
defined, so that I can see how to do this properly in a package?

I can see from the R source that, for example, `do_gzfile` is defined in 
`connections.c` and referenced in `names.c`. However, I thought I should ask 
here first in case there is a better place to start, than trying to copy this 
code.

I only want an object that I can use `readBin` and `writeBin` on, to read and 
write audio data using e.g. `snd_pcm_writei` which is part of the `alsa-lib` 
package.

Thank you very much,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] best way to build from Git

[frederik]

Thank you all for your replies.

On Thu, Dec 26, 2019 at 01:06:48PM +, Gábor Csárdi wrote:

On Thu, Dec 26, 2019 at 12:39 PM Martin Maechler

...
It would probably really be useful, if Lionel (or someone else)
updated his nice write-up and auxiliary {script / make} - files
so things work too when build with "src-dir != build-dir".


I am not sure what you mean, what is wrong with this build directory?
https://bookdown.org/lionel/contributing/building-r.html#sec:build


Yes it seems that Lionel's instructions successfully build a Git repo
containing R into a separate build directory. Thank you Lionel.

I've adapted these instructions, together with Lionel's GNUmakefile
and make-svn-revision script, to suit my (Linux-based) environment. I
put it in a little GitHub repository. Currently the following commands
work for me to download and install R in my home directory:

git clone https://github.com/navarum/tweaks
PREFIX=~/.local ./tweaks/r/BUILD install

For now it applies one example patch on top of a fixed recent commit
to the upstream repo https://github.com/wch/r-source/ . I noticed that
the first commit hash I selected from 'git log' was not stable, so I'm
not sure if this script will work tomorrow or next week. However, I
thought I ought to share it in case others find it useful. I tried to
make the (Bash) code concise enough that it would be possible for
others to see what it is doing and adapt it to maintaining their own
patch series on top of some version of R. I've been using this tool to
maintain custom versions of other software packages as well, so there
may be a little "too much information" in there - but someone could
obviously fork it and keep just what they need. The system tries to be
somewhat intelligent about remembering whether configure was run,
checking if the patches need to be reapplied, not overwriting any
changes I might have made to the cloned r-source git repo, and so on.

Just for the record, my original question on this thread was about the
reasons for not having R's default build system work with different
revision control systems, as well as some apparent typos in the
Makefile that comes with R.

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] best way to build from Git

[frederik]

Dear R-devel,

I checked out a recent copy of R via Subversion and made a few changes to the 
code. I wanted to commit them locally to my repo, just to stay organized and 
keep them separate from other changes I plan to make. However, I was not able 
to commit them because I don't think SVN allows this?

Plan B, check out a Git mirror of R, https://github.com/wch/r-source/

It has some instructions for building: https://github.com/wch/r-source/wiki

However, there are some complicated workarounds due to a commit from 2013 
(https://github.com/wch/r-source/commit/4f13e5325dfbcb9fc8f55fc6027af9ae9c7750a3).
 And it only seems to succeed if the build directory is the same as the source 
directory.

Makefile.in has some odd stuff that might be related, like why is it checking for "$(srcdir)/doc/FAQ" rather than 
"$(top_builddir)/doc/FAQ"? Conversely, there is a check for "$(top_builddir)/.git" which I would expect to be in "$(srcdir)/.git" 
instead. And it runs "git svn info" which seems to take indefinitely long - I am not using "git svn", I'm just trying to build from a 
regular git repo. This is difficult to debug because it is inside an @if, so "make -n" still runs it. Also there are no comments to explain what e.g. 
the "svnonly" target is doing, or anything else.

After some time I still have little idea why it is necessary for the build system to 
depend on the revision control system. Clearly we are trying to do something fancy, but 
why not e.g. just failover to "Last Changed Date: unknown; not SVN" if 
SVN-REVISION is missing?

I am wondering if there is any plan to clean up this situation because it seems 
like a stumbling block for users who might want to contribute to R.

Thanks,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Halfway through writing an "IDE" with support for R; Proof of concept, and request for suggestions.

[frederik]

On Sat, Jun 15, 2019 at 01:13:07PM +1200, Abby Spurdle wrote:

What about Atom, VS Code and the like? Or what about taking a project
that meets most of the constraints and pushing to cover all of them,
or even forking it and modifying the part you don't like?


I'm not prepared to endorse GitHub affiliated software.

[[alternative HTML version deleted]]


I'm trying to figure out what (of many possibilities) is wrong with GitHub.

Also, do good programmers really use IDEs? For me the problem is the 
"integrated" part; the more stuff is bundled together in one package (terminal, 
editor, window manager, build system), the more annoying it's going to be that I can't 
use the terminal/editor/window-manager/build-system of my preference.

When I use R, I use it on the command-line with a general-purpose terminal 
multiplexer, terminal, text editor, (tiling) window manager, and so on. The 
text editor and R are both running in separate windows in a terminal 
multiplexer session. I have custom key-bindings in my terminal and text editor 
which help me move text back and forth between R and the editor, and to do 
common tasks like sourcing the current file that I'm editing. For me this is 
much more flexible than anything I could get in a coherent package like 
RStudio, or something like what Abby is working on. For example, in my setup I 
never have to use the mouse, which is great for me.

Although Abby's IDE looks awesome, and like a fun project, I felt obliged to 
weigh in with my own opinions. Not that I'm a very good programmer, but I feel 
an IDE would make me worse. It seems more like a valuable tool for introducing 
beginners to the language. We have RStudio, which already fills this niche 
pretty well (to echo Iñaki).

I'm not even quite sure why I'm writing this, but I hope it may be vaguely 
useful...

Best wishes,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [FORGED] src/modules/X11/devX11.c, can we remove "#if BUG" yet

[frederik]

Thanks again ... but by the way, I wonder if this would be a good time
to take another look at the use case that produced this error.

Consider that you have a complicated algorithm you are trying to
analyze, and you decide to display plots of certain data, as part of
the algorithm's main loop. Thanks to the recent fix, the plot window
will be correctly resized when it first opens and receives a
ConfigureNotify from the window manager. However, if the user decides
to resize it again while the algorithm is running, then the window
won't react until control comes back to the R read-eval-print loop,
since this is where R_ProcessX11Events gets called. Question: is it
possible to call R_ProcessX11Events manually from R code? That would
be another way to solve the problem. Or if R_ProcessX11Events is
called with every call to plot() or dev.flush() or something, might be
an even simpler solution.

Previously, R developers have been kind enough to add an "onIdle()"
handler to the getGraphicsEvent handlers, making it possible for me to
have interactive animated plots. I love this feature, but I think
there are some applications, like the above, where it would be more
convenient to keep a loop-based flow control.

I don't want to overwhelm you with requests, of course, so maybe this
goes on a queue to think about later.

Thanks,

Frederick

On Thu, May 02, 2019 at 02:07:19PM -0700, frede...@ofb.net wrote:

I tested it. It fixes the bug and didn't seem to produce any errors. Thank you 
Professor Dalgaard! I'm so glad this has finally been addressed. I will update 
the bug report.

(https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702)

On Thu, May 02, 2019 at 04:48:51PM +0200, peter dalgaard wrote:

OK, this is now in R-devel, but only superficially tested (b/c this is a Mac). 
Please check it out.

-pd


On 30 Apr 2019, at 23:09 , Paul Murrell  wrote:

Hi Peter

Yes, that looks roughly right to me.  I would be in favour of your option (b), 
partly because it is probably easiest and partly because that retains the basic 
graphics device startup logic pattern that is replicated across all(?) graphics 
devices.

Paul

On 28/04/19 11:39 AM, peter dalgaard wrote:

I had a look at the current code, and AFAICT it has essentially the same 
structure as it did back then. I think it may have finally dawned upon me what 
the issue really is:
The logic is that in Rf_addX11Device, we have
   if (!X11DeviceDriver(dev, display, width, height,
ps, gamma, colormodel, maxcubesize,
bgcolor, canvascolor, sfonts, res,
xpos, ypos, title, useCairo, antialias, family)) {
   free(dev);
   errorcall(call, _("unable to start device %s"), devname);
   }
   dd = GEcreateDevDesc(dev);
   GEaddDevice2(dd, devname);
i.e., we start the device driver, and if it fails, we throw away the "dev" 
structure and call it a day. If it succeeds, we proceed to create a device descriptor 
structure and add it to the list of open devices.
This approach means that X11DeviceDriver() cannot do anything that potentially 
accesses the dd structure because it isn't there yet, and the things it cannot 
do apparently includes calling R_ProcessX11Events(). [To be completely sure 
that this is actually still true, I'd need to have a closer look at what 
handleEvent() does.]
So to fix things, it would seem that you could (a) add the device before 
attempting to start the driver, preparing to back it out if the driver fails to 
start, or (b) add a call to R_ProcessX11Events() _after_ the GEaddDevice2(dd, 
devname). Option (b) is probably the easiest.
Paul: Does this analysis look roughly right?
-pd

On 26 Apr 2019, at 01:23 , frede...@ofb.net wrote:

Thanks Professor Dalgard.

If you have a different way to fix the bug then I'd be happy to test
it.

Or whatever. I understand that maybe some data was being referenced
before it had been initialized. I could also support moving the
R_ProcessEvents call in another place, but it seems one would also
like to generate some kind of warning message, at the location of the
bad reference, rather than segfaulting. Was it not possible to
identify this location? I'm guessing that Valgrind is a bit more
mature now than it was in 2001...?

Frederick

On Wed, Apr 24, 2019 at 03:12:55PM +0200, peter dalgaard wrote:

OK, so I did the archaeology anyway


This was the story, R-core November 29, 2001. Part of thread "X11 still 
segfaults".

>>
.
Gah. I've been too tired today. Why did that take me so long?

The culprit seems to be

R_ProcessEvents((void*) NULL)

in newX11DeviceDriver

This gets called *before* this stuff at the end of Rf_addX11Device

dd = GEcreateDevDesc(dev);
addDevice((DevDesc*) dd);
initDisplayList((DevDesc*) dd);

and it is that "dd" that gets called by Rf_playDisplayList. Removing
the offending line stops the segfaulting, seemingly with no ill
effects.

I'm 

Re: [Rd] [FORGED] src/modules/X11/devX11.c, can we remove "#if BUG" yet

[frederik]

I tested it. It fixes the bug and didn't seem to produce any errors. Thank you 
Professor Dalgaard! I'm so glad this has finally been addressed. I will update 
the bug report.

(https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702)

On Thu, May 02, 2019 at 04:48:51PM +0200, peter dalgaard wrote:

OK, this is now in R-devel, but only superficially tested (b/c this is a Mac). 
Please check it out.

-pd


On 30 Apr 2019, at 23:09 , Paul Murrell  wrote:

Hi Peter

Yes, that looks roughly right to me.  I would be in favour of your option (b), 
partly because it is probably easiest and partly because that retains the basic 
graphics device startup logic pattern that is replicated across all(?) graphics 
devices.

Paul

On 28/04/19 11:39 AM, peter dalgaard wrote:

I had a look at the current code, and AFAICT it has essentially the same 
structure as it did back then. I think it may have finally dawned upon me what 
the issue really is:
The logic is that in Rf_addX11Device, we have
if (!X11DeviceDriver(dev, display, width, height,
 ps, gamma, colormodel, maxcubesize,
 bgcolor, canvascolor, sfonts, res,
 xpos, ypos, title, useCairo, antialias, family)) {
free(dev);
errorcall(call, _("unable to start device %s"), devname);
}
dd = GEcreateDevDesc(dev);
GEaddDevice2(dd, devname);
i.e., we start the device driver, and if it fails, we throw away the "dev" 
structure and call it a day. If it succeeds, we proceed to create a device descriptor 
structure and add it to the list of open devices.
This approach means that X11DeviceDriver() cannot do anything that potentially 
accesses the dd structure because it isn't there yet, and the things it cannot 
do apparently includes calling R_ProcessX11Events(). [To be completely sure 
that this is actually still true, I'd need to have a closer look at what 
handleEvent() does.]
So to fix things, it would seem that you could (a) add the device before 
attempting to start the driver, preparing to back it out if the driver fails to 
start, or (b) add a call to R_ProcessX11Events() _after_ the GEaddDevice2(dd, 
devname). Option (b) is probably the easiest.
Paul: Does this analysis look roughly right?
-pd

On 26 Apr 2019, at 01:23 , frede...@ofb.net wrote:

Thanks Professor Dalgard.

If you have a different way to fix the bug then I'd be happy to test
it.

Or whatever. I understand that maybe some data was being referenced
before it had been initialized. I could also support moving the
R_ProcessEvents call in another place, but it seems one would also
like to generate some kind of warning message, at the location of the
bad reference, rather than segfaulting. Was it not possible to
identify this location? I'm guessing that Valgrind is a bit more
mature now than it was in 2001...?

Frederick

On Wed, Apr 24, 2019 at 03:12:55PM +0200, peter dalgaard wrote:

OK, so I did the archaeology anyway


This was the story, R-core November 29, 2001. Part of thread "X11 still 
segfaults".

>>
.
Gah. I've been too tired today. Why did that take me so long?

The culprit seems to be

R_ProcessEvents((void*) NULL)

in newX11DeviceDriver

This gets called *before* this stuff at the end of Rf_addX11Device

dd = GEcreateDevDesc(dev);
addDevice((DevDesc*) dd);
initDisplayList((DevDesc*) dd);

and it is that "dd" that gets called by Rf_playDisplayList. Removing
the offending line stops the segfaulting, seemingly with no ill
effects.

I'm not really sure what the use of that line ever was; it might be
necessary to make the call somewhere later, but it appears to have
been possible to race past it before receiving any events all the
time.

I also changed a couple of spots missing dd->newDevStruct=1

Will commit in a moment.
<<

And the following day, in "graphics saga part III", we had

->>
...

I can't make it happen in 1.3.1 but...

It is probably not unrelated to the R_ProcessEvents line that
I took out, but that was definitely wrong. However, one might reenable
it if one could change this bit of code

if (!(ptr_X11DeviceDriver)((DevDesc*)(dev), display, width, height, ps, 
gamma,
  colormodel, maxcubesize, canvascolor)) {
free(dev);
errorcall(gcall, "unable to start device %s", devname);
}
gsetVar(install(".Device"), mkString(devname), R_NilValue);
dd = GEcreateDevDesc(dev);
addDevice((DevDesc*) dd);
initDisplayList((DevDesc*) dd);


and put the if-clause last. A cursory clance through the three
functions that are being called didn't reveal anything that would rely
on having opened the device driver first.

Paul?

(I might try it locally, but I'm not sure I should commit anything.)

<<---

It seems that the suggestion was never followed up on?

-pd



On 24 Apr 2019, at 11:42 , 

Re: [Rd] patch to improve matrix conformability error message

[frederik]

I'm not really the one who makes these decisions; I'd just wait for
someone on the Core team to reply to this thread. Maybe they are all
just busy right now.

On Mon, Apr 29, 2019 at 10:08:43PM -0400, Joshua N Pritikin wrote:

On Mon, Apr 29, 2019 at 06:28:46PM -0700, frede...@ofb.net wrote:

I think this is a good idea. Is there a reason why it got no interest?
Slippery slope?


What's the next step? Can it get committed?


Or maybe others were also just occupied trying to figure out how
Joshua's second message had timestamp earlier than his first message?


As a new subscriber, I guess my messages got caught in moderation.

--
Joshua N. Pritikin, Ph.D.
Virginia Institute for Psychiatric and Behavioral Genetics
Virginia Commonwealth University
PO Box 980126
800 E Leigh St, Biotech One, Suite 1-133
Richmond, VA 23219
http://exuberant-island.surge.sh



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] patch to improve matrix conformability error message

[frederik]

I think this is a good idea. Is there a reason why it got no interest?
Slippery slope?

Or maybe others were also just occupied trying to figure out how
Joshua's second message had timestamp earlier than his first message?

On Fri, Apr 05, 2019 at 12:05:36PM -0400, Joshua N Pritikin wrote:

With this patch,


A <- matrix(1, 2, 2)
B <- matrix(2, 3, 2)
A %*% B

Error in A %*% B :
 non-conformable arguments of dimension (2, 2) and (3, 2)

From 205b591d4d14b5ff667325fb233a6deb08314726 Mon Sep 17 00:00:00 2001
From: Joshua Nathaniel Pritikin 
Date: Fri, 5 Apr 2019 12:03:58 -0400
Subject: [PATCH] Improve non-conformable arguments error message

---
src/main/array.c | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/src/main/array.c b/src/main/array.c
index 4b95e33..cedf627 100644
--- a/src/main/array.c
+++ b/src/main/array.c
@@ -1350,15 +1350,18 @@ SEXP attribute_hidden do_matprod(SEXP call, SEXP op, 
SEXP args, SEXP rho)
if (PRIMVAL(op) == 0) {
/* primitive, so use call */
if (ncx != nry)
-   errorcall(call, _("non-conformable arguments"));
+ errorcall(call, _("non-conformable arguments of dimension (%d, %d) and 
(%d, %d)"),
+   nrx, ncx, nry, ncy);
}
else if (PRIMVAL(op) == 1) {
if (nrx != nry)
-   error(_("non-conformable arguments"));
+ error(_("non-conformable arguments of dimension (%d, %d) and (%d, 
%d)"),
+   nrx, ncx, nry, ncy);
}
else {
if (ncx != ncy)
-   error(_("non-conformable arguments"));
+ error(_("non-conformable arguments of dimension (%d, %d) and (%d, 
%d)"),
+   nrx, ncx, nry, ncy);
}

if (isComplex(CAR(args)) || isComplex(CADR(args)))
--
2.19.1

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] [FORGED] src/modules/X11/devX11.c, can we remove "#if BUG" yet

[frederik]

Thanks Professor Dalgard.

If you have a different way to fix the bug then I'd be happy to test
it.

Or whatever. I understand that maybe some data was being referenced
before it had been initialized. I could also support moving the
R_ProcessEvents call in another place, but it seems one would also
like to generate some kind of warning message, at the location of the
bad reference, rather than segfaulting. Was it not possible to
identify this location? I'm guessing that Valgrind is a bit more
mature now than it was in 2001...?

Frederick

On Wed, Apr 24, 2019 at 03:12:55PM +0200, peter dalgaard wrote:

OK, so I did the archaeology anyway


This was the story, R-core November 29, 2001. Part of thread "X11 still 
segfaults".

>>
.
Gah. I've been too tired today. Why did that take me so long?

The culprit seems to be

R_ProcessEvents((void*) NULL)

in newX11DeviceDriver

This gets called *before* this stuff at the end of Rf_addX11Device

dd = GEcreateDevDesc(dev);
addDevice((DevDesc*) dd);
initDisplayList((DevDesc*) dd);

and it is that "dd" that gets called by Rf_playDisplayList. Removing
the offending line stops the segfaulting, seemingly with no ill
effects.

I'm not really sure what the use of that line ever was; it might be
necessary to make the call somewhere later, but it appears to have
been possible to race past it before receiving any events all the
time.

I also changed a couple of spots missing dd->newDevStruct=1

Will commit in a moment.
<<

And the following day, in "graphics saga part III", we had

->>
...

I can't make it happen in 1.3.1 but...

It is probably not unrelated to the R_ProcessEvents line that
I took out, but that was definitely wrong. However, one might reenable
it if one could change this bit of code

if (!(ptr_X11DeviceDriver)((DevDesc*)(dev), display, width, height, ps, 
gamma,
  colormodel, maxcubesize, canvascolor)) {
free(dev);
errorcall(gcall, "unable to start device %s", devname);
}
gsetVar(install(".Device"), mkString(devname), R_NilValue);
dd = GEcreateDevDesc(dev);
addDevice((DevDesc*) dd);
initDisplayList((DevDesc*) dd);


and put the if-clause last. A cursory clance through the three
functions that are being called didn't reveal anything that would rely
on having opened the device driver first.

Paul?

(I might try it locally, but I'm not sure I should commit anything.)

<<---

It seems that the suggestion was never followed up on?

-pd



On 24 Apr 2019, at 11:42 , peter dalgaard  wrote:

I don't recall exactly what I did 18 years ago eiher and I likely don't have 
the time to dig into the archives and reconstruct.

I can imagine that the issue had to do with the protocol around creating and 
mapping windows. Presumably the segfault comes from looking for events on a 
window that hasn't been created yet, or has already been destroyed, leading to 
a NULL reference somewhere. I have a vague recollection that the issue was 
window manager dependent (in 2001 probably not twm, more likely xvwm on RedHat 
if it was affecting me).

A proper fix should go via proper understanding of the X11 protocol - uncommenting a line 
is as bad as commenting it in the 1st place So more like "wait for window to 
exist THEN process events" -- but the 1st part may be WM specific, etc.

I recall docs being quite obtuse, and the X11 "mechanism not policy" credo 
doesn't help as WMs are not obliged to (say) send notifications, so you can end up 
stalling, waiting for events that never happen.

It is entirely possible that there is stuff in here that I didn't understand 
properly at the time, and still don't!

- pd


On 24 Apr 2019, at 02:30 , Paul Murrell  wrote:

Hi

Sorry, I can't offer an explanation for the commented-out line.
However, regarding your final question of avoiding the R-core bottleneck, you 
do have the option of creating a third-party graphics device package.  See, for 
example, the 'tikzDevice' and 'svglite' packages on CRAN.  Does that provide 
you with a way forward ?

Paul

On 20/04/2019 5:27 p.m., frede...@ofb.net wrote:

Dear R Devel,

I know that someone put this line in src/modules/X11/devX11.c:2824 for
a reason, because commenting it out causes R to miss an important
ConfigureNotify event in my window manager. The result is that plots
are initially drawn off the window borders, unreadable.

  R_ProcessX11Events((void*) NULL);

Unfortunately for me, this line is commented in the standard release
of R, it has "#if BUG ... #endif" around it.

I guess it is also unfortunate for anyone who uses the same window
manager as I do, namely i3, which I think is pretty popular among Unix
power users these days; not to mention other full-screen window
managers which probably exhibit the same bug in R.

Maybe everyone on the Core team uses twm as their window manager? Or
RStudio on Windows? 

Re: [Rd] [FORGED] src/modules/X11/devX11.c, can we remove "#if BUG" yet

[frederik]

Thanks Paul for answering the additional question.

I admit that I've only had experience with R's X11 code through work
on a couple of bugs, but for some reason I thought it might be
nontrivial to move it all into a self-contained module due to
interactions with various event loops. The two modules you listed
appear to be used for producing output in image and document formats,
so they don't really cast light on whether this is a problem.

I am not very serious about contributing my time to an effort like
this, but maybe it is good to have some discussion here anyway. I had
thought that maybe authors of alternative plotting interfaces would
have something to say about whether the current graphics design
provides sufficient modularity. Obviously it is modular enough for
RStudio to exist.

Other improvements aside, I think it would just be better to comment
out the old "#if BUG" line, and wait for someone to complain if it
breaks something. A lot has been changed since that line was added, as
I explained in the bug report. I would expect that the bug it was
attempting to fix no longer exists.

Otherwise, what is the next milestone on this bug?

Frederick

On Wed, Apr 24, 2019 at 12:30:44PM +1200, Paul Murrell wrote:

Hi

Sorry, I can't offer an explanation for the commented-out line.
However, regarding your final question of avoiding the R-core 
bottleneck, you do have the option of creating a third-party graphics 
device package.  See, for example, the 'tikzDevice' and 'svglite' 
packages on CRAN.  Does that provide you with a way forward ?


Paul

On 20/04/2019 5:27 p.m., frede...@ofb.net wrote:

Dear R Devel,

I know that someone put this line in src/modules/X11/devX11.c:2824 for
a reason, because commenting it out causes R to miss an important
ConfigureNotify event in my window manager. The result is that plots
are initially drawn off the window borders, unreadable.

   R_ProcessX11Events((void*) NULL);

Unfortunately for me, this line is commented in the standard release
of R, it has "#if BUG ... #endif" around it.

I guess it is also unfortunate for anyone who uses the same window
manager as I do, namely i3, which I think is pretty popular among Unix
power users these days; not to mention other full-screen window
managers which probably exhibit the same bug in R.

Maybe everyone on the Core team uses twm as their window manager? Or
RStudio on Windows? Which would be sad because then we're not
representing an important user demographic, namely those who prefer
software which is modern and powerful, yet simple to understand and
modify; fully configurable and interoperable and so on.

I first reported this bug 3 years ago. In doing research for my bug
report, I found that the line was commented out by Peter Dalgaard in
2001 with the explanation "X11 segfault fix - I hope".

I don't know what the way forward is. Obviously the Core Team has
reason to say, "look, this isn't very important, it's been broken
since 2001, maybe fixing it will cause the undocumented segfault bug
to reappear, clearly no one here uses your window manager". Do I have
to submit a correctness proof for the proposed change? What do I do?

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702

As mentioned in my bug report, I checked using gdb that
ConfigureNotify is indeed being received by the call to
R_ProcessX11Events() when it is uncommented. I haven't experienced any
segfaults.

It's good that Peter left evidence that "R_ProcessX11Events" was being
called 18 years ago from X11DeviceDriver(). If he had deleted the
line, rather than commenting it, then discovering the reason for the
window rendering bug would have been much harder for me.

However, the downside is that now it is not just a matter of inserting
the line where it belongs; I also feel a bit like I have to explain
why it was initially removed. But although I've given it some thought,
I still have no idea.

Somewhat tangentially, I am wondering if there is some way that we
could make the development of R's graphics code proceed at a faster
rate, for example by pulling it out into a separate module, so that
people could offer alternative implementations via CRAN etc., rather
than having R Core be the bottleneck. Would this make sense? Has it
already been done?

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


--
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
p...@stat.auckland.ac.nz
http://www.stat.auckland.ac.nz/~paul/



__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] src/modules/X11/devX11.c, can we remove "#if BUG" yet

[frederik]

Dear R Devel,

I know that someone put this line in src/modules/X11/devX11.c:2824 for
a reason, because commenting it out causes R to miss an important
ConfigureNotify event in my window manager. The result is that plots
are initially drawn off the window borders, unreadable.

R_ProcessX11Events((void*) NULL);

Unfortunately for me, this line is commented in the standard release
of R, it has "#if BUG ... #endif" around it.

I guess it is also unfortunate for anyone who uses the same window
manager as I do, namely i3, which I think is pretty popular among Unix
power users these days; not to mention other full-screen window
managers which probably exhibit the same bug in R.

Maybe everyone on the Core team uses twm as their window manager? Or
RStudio on Windows? Which would be sad because then we're not
representing an important user demographic, namely those who prefer
software which is modern and powerful, yet simple to understand and
modify; fully configurable and interoperable and so on.

I first reported this bug 3 years ago. In doing research for my bug
report, I found that the line was commented out by Peter Dalgaard in
2001 with the explanation "X11 segfault fix - I hope".

I don't know what the way forward is. Obviously the Core Team has
reason to say, "look, this isn't very important, it's been broken
since 2001, maybe fixing it will cause the undocumented segfault bug
to reappear, clearly no one here uses your window manager". Do I have
to submit a correctness proof for the proposed change? What do I do?

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702

As mentioned in my bug report, I checked using gdb that
ConfigureNotify is indeed being received by the call to
R_ProcessX11Events() when it is uncommented. I haven't experienced any
segfaults.

It's good that Peter left evidence that "R_ProcessX11Events" was being
called 18 years ago from X11DeviceDriver(). If he had deleted the
line, rather than commenting it, then discovering the reason for the
window rendering bug would have been much harder for me.

However, the downside is that now it is not just a matter of inserting
the line where it belongs; I also feel a bit like I have to explain
why it was initially removed. But although I've given it some thought,
I still have no idea.

Somewhat tangentially, I am wondering if there is some way that we
could make the development of R's graphics code proceed at a faster
rate, for example by pulling it out into a separate module, so that
people could offer alternative implementations via CRAN etc., rather
than having R Core be the bottleneck. Would this make sense? Has it
already been done?

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Documentation examples for lm and glm

[frederik]

I agree with Steve and Achim that we should keep some examples with no
data frame. That's Objectively Simpler, whether or not it leads to
clutter in the wrong hands. As Steve points out, we have attach()
which is an excellent language feature - not to mention with().

I would go even further and say that the examples that are in lm() now
should stay at the top. Because people may be used to referring to
them, and also because Historical Order is generally a good order in
which to learn things. However, if there is an important function
argument ("data=") not in the examples, then we should add examples
which use it. Likewise if there is a popular programming style
(putting things in a data frame). So let's do something along the
lines of what Thomas is requesting, but put it after the existing
documentation? Please?

On a bit of a tangent, I would like to see an example in lm() which
plots my data with a fitted line through it. I'm probably betraying my
ignorance here, but I was asked how to do this when showing R to a
friend and I thought it should be in lm(), after all it seems a bit
more basic than displaying a Normal Q-Q plot (whatever that is!
gasp...). Similarly for glm(). Perhaps all this can be accomplished
with merely doubling the size of the existing examples.

Thanks.

Frederick

On Sat, Dec 15, 2018 at 02:15:52PM +0100, Achim Zeileis wrote:
A pragmatic solution could be to create a simple linear regression 
example with variables in the global environment and then another 
example with a data.frame.


The latter might be somewhat more complex, e.g., with several 
regressors and/or mixed categorical and numeric covariates to 
illustrate how regression and analysis of (co-)variance can be 
combined. I like to use MASS's whiteside data for this:


data("whiteside", package = "MASS")
m1 <- lm(Gas ~ Temp, data = whiteside)
m2 <- lm(Gas ~ Insul + Temp, data = whiteside)
m3 <- lm(Gas ~ Insul * Temp, data = whiteside)
anova(m1, m2, m3)

Moreover, some binary response data.frame with a few covariates might 
be a useful addition to "datasets". For example a more granular 
version of the "Titanic" data (in addition to the 4-way tabel 
?Titanic). Or another relatively straightforward data set, popular in 
econometrics and social sciences is the "Mroz" data, see e.g., 
help("PSID1976", package = "AER").


I would be happy to help with these if such additions were considered 
for datasets/stats.



On Sat, 15 Dec 2018, David Hugh-Jones wrote:


I would argue examples should encourage good practice. Beginners ought to
learn to keep data in data frames and not to overuse attach(). Experts can
do otherwise at their own risk, but they have less need of explicit
examples.

On Fri, 14 Dec 2018 at 14:51, S Ellison  wrote:


FWIW, before all the examples are changed to data frame variants, I think
there's fairly good reason to have at least _one_ example that does _not_
place variables in a data frame.

The data argument in lm() is optional. And there is more than one way to
manage data in a project. I personally don't much like lots of stray
variables lurking about, but if those are the only variables out there and
we can be sure they aren't affected by other code, it's hardly essential to
create a data frame to hold something you already have.
Also, attach() is still part of R, for those folk who have a data frame
but want to reference the contents across a wider range of functions
without using with() a lot. lm() can reasonably omit the data argument
there, too.

So while there are good reasons to use data frames, there are also good
reasons to provide examples that don't.

Steve Ellison



-Original Message-
From: R-devel [mailto:r-devel-boun...@r-project.org] On Behalf Of Ben
Bolker
Sent: 13 December 2018 20:36
To: r-devel@r-project.org
Subject: Re: [Rd] Documentation examples for lm and glm


 Agree.  Or just create the data frame with those variables in it
directly ...

On 2018-12-13 3:26 p.m., Thomas Yee wrote:

Hello,

something that has been on my mind for a decade or two has
been the examples for lm() and glm(). They encourage poor style
because of mismanagement of data frames. Also, having the
variables in a data frame means that predict()
is more likely to work properly.

For lm(), the variables should be put into a data frame.
As 2 vectors are assigned first in the general workspace they
should be deleted afterwards.

For the glm(), the data frame d.AD is constructed but not used. Also,
its 3 components were assigned first in the general workspace, so they
float around dangerously afterwards like in the lm() example.

Rather than attached improved .Rd files here, they are put at
www.stat.auckland.ac.nz/~yee/Rdfiles
You are welcome to use them!

Best,

Thomas

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


__
R-devel@r-project.org mailing list

Re: [Rd] Poor documentation for "adj" and text()

[frederik]

Thanks Martin.

I wouldn't necessarily fault Ulrich for his subject line - unless you
want to propose a better one... I might fault him for not following up
and checking out the patch that I submitted at his prompting.

I noticed that you committed my patch last Friday, with some welcome
improvements, thanks! I'm assuming there is no easy way to get
notified about commits, I just have to do 'svn update'?

FWIW there are a lot of things that I would want to improve about R if
I had the skill and the ability to do so. The patch hints at what I
would improve about text rotation. Additionally, perhaps some other
documentation issues; the fact that large plots can take forever and
are not interruptable on the command line; I want better compatibility
with tiling window managers (plots shouldn't have to be redrawn every
time I switch to the window; they should come up with correct
dimensions); better readline history facilities. I wouldn't assume
that just because something has been the same way for decades that
people like it that way! On the other hand I can see from the logs
that there are at least several commits per day, so there must be a
trade-off in different directions that the project can take.

I made this graph:

https://i.imgur.com/l4uCOek.png

Best wishes,

Frederick

On Fri, Sep 14, 2018 at 09:38:42AM +0200, Martin Maechler wrote:
> >   
> > on Thu, 13 Sep 2018 23:33:41 -0700 writes:
> 
> > Hello Core Team, I sent this patch over a year ago. It
> > looks like it was sent in response to another user's
> > complaint which echoed some of my own observations about
> > problems in the documentation for 'text'. Did anyone have
> > a chance to look it over? 
> 
> I see it marked in my box as some (of too many !) thing I had
> wanted to look at.
> 
> OTOH,   R core  may not be terribly motivated by e-mail threads
> starting with "Poor " notably when that thing has
> been in R (and S before that) for decades.
> OTOH^2,  improving documentation is often a good and helpful
> thing and I will look at it now.
> 
> Thank you, Frederick, for trying to help making R better!
> Martin
> 
> 
> > I'd like to get it out of my queue.
> 
> > Thanks,
> 
> > Frederick
> 
> > On Tue, Apr 11, 2017 at 12:01:05PM -0700, frede...@ofb.net
> > wrote:
> >> Thanks Ulrich for sharing your experience.
> >> 
> >> I'm attaching a patch which tries to address the issues
> >> you raised.
> >> 
> >> I agree with you in principle, but I think it makes sense
> >> to leave some details under "Details". However, the
> >> descriptions in "Arguments" should give enough
> >> information that a user can get the function to do
> >> something predictable in at least one situation, and I
> >> feel this is not the case at present.
> >> 
> >> I tried to fix the wording so that 'adj' and 'offset' are
> >> no longer confusing to new users (or to me, every time I
> >> forget what they mean).
> >> 
> >> I also fixed the paragraph on rotated text; it is more
> >> correct now, at least for X11-cairo.
> >> 
> >> I hope that someone in the Core Team can look this over
> >> and apply it.
> >> 
> >> Thank you,
> >> 
> >> Frederick
> >> 
> >> On Tue, Apr 11, 2017 at 09:23:50AM +0200, Ulrich Windl
> >> wrote: > Hi!
> >> > 
> >> > (I'd like to be able to access your bugzilla, BTW) >
> >> The documentation for parameter "adj" of text() in R
> >> 3.3.3 is hard to understand (unless you know what it does
> >> already):
> >> > 
> >> > "adj > one or two values in [0, 1] which specify the x
> >> (and optionally y) adjustment of the labels. On most
> >> devices values outside that interval will also work."
> >> > 
> >> > What is the meaning of the values? I think the
> >> description ("adj allows adjustment of the text with
> >> respect to (x, y). Values of 0, 0.5, and 1 specify
> >> left/bottom, middle and right/top alignment,
> >> respectively. The default is for centered text, i.e., adj
> >> = c(0.5, NA). Accurate vertical centering needs character
> >> metric information on individual characters which is only
> >> available on some devices. Vertical alignment is done
> >> slightly differently for character strings and for
> >> expressions: adj = c(0,0) means to left-justify and to
> >> align on the baseline for strings but on the bottom of
> >> the bounding box for expressions. This also affects
> >> vertical centering: for strings the centering excludes
> >> any descenders whereas for expressions it includes
> >> them. Using NA for strings centers them, including
> >> descenders.") should be moved to the parameter.
> >> > 
> >> > In general I'd suggest to describe the range, meaning
> >> and default of every parameter where the parameter is
> >> listed. "Details" should only give 

Re: [Rd] Poor documentation for "adj" and text()

[frederik]

Hello Core Team,

I sent this patch over a year ago. It looks like it was sent in
response to another user's complaint which echoed some of my own
observations about problems in the documentation for 'text'. Did
anyone have a chance to look it over? I'd like to get it out of my
queue.

Thanks,

Frederick

On Tue, Apr 11, 2017 at 12:01:05PM -0700, frede...@ofb.net wrote:
> Thanks Ulrich for sharing your experience.
> 
> I'm attaching a patch which tries to address the issues you raised.
> 
> I agree with you in principle, but I think it makes sense to leave
> some details under "Details". However, the descriptions in "Arguments"
> should give enough information that a user can get the function to do
> something predictable in at least one situation, and I feel this is
> not the case at present.
> 
> I tried to fix the wording so that 'adj' and 'offset' are no longer
> confusing to new users (or to me, every time I forget what they mean).
> 
> I also fixed the paragraph on rotated text; it is more correct now, at
> least for X11-cairo.
> 
> I hope that someone in the Core Team can look this over and apply it.
> 
> Thank you,
> 
> Frederick
> 
> On Tue, Apr 11, 2017 at 09:23:50AM +0200, Ulrich Windl wrote:
> > Hi!
> > 
> > (I'd like to be able to access your bugzilla, BTW)
> > The documentation for parameter "adj" of text() in R 3.3.3 is hard to 
> > understand (unless you know what it does already):
> > 
> > "adj 
> > one or two values in [0, 1] which specify the x (and optionally y) 
> > adjustment of the labels. On most devices values outside that interval will 
> > also work."
> > 
> > What is the meaning of the values? I think the description ("adj allows 
> > adjustment of the text with respect to (x, y). Values of 0, 0.5, and 1 
> > specify left/bottom, middle and right/top alignment, respectively. The 
> > default is for centered text, i.e., adj = c(0.5, NA). Accurate vertical 
> > centering needs character metric information on individual characters which 
> > is only available on some devices. Vertical alignment is done slightly 
> > differently for character strings and for expressions: adj = c(0,0) means 
> > to left-justify and to align on the baseline for strings but on the bottom 
> > of the bounding box for expressions. This also affects vertical centering: 
> > for strings the centering excludes any descenders whereas for expressions 
> > it includes them. Using NA for strings centers them, including 
> > descenders.") should be moved to the parameter.
> > 
> > In general I'd suggest to describe the range, meaning and default of every 
> > parameter where the parameter is listed. "Details" should only give an 
> > overview of the functions.
> > 
> > Likewise "offset": Will the direction be influenced by "pos"? The 
> > description is quite silent on that.
> > 
> > Documentation should be structured to help the user to find the facts 
> > easily without having to read the whole page.
> > 
> > Regards,
> > Ulrich Windl
> > 
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> > 

> --- text.Rd   2016-11-27 18:33:26.541516325 -0800
> +++ new-text.Rd   2017-04-11 11:48:32.668926075 -0700
> @@ -26,16 +26,18 @@
>  If \code{labels} is longer than \code{x} and
>  \code{y}, the coordinates are recycled to the length of \code{labels}.}
>\item{adj}{one or two values in \eqn{[0, 1]} which specify the x
> -(and optionally y) adjustment of the labels.  On most devices values
> -outside that interval will also work.}
> +(and optionally y) justification of the labels, with 0 for
> +left/bottom, 1 for right/top, and 0.5 for centered.
> +On most devices values
> +outside \eqn{[0, 1]} will also work. See below.}
>\item{pos}{a position specifier for the text.  If specified this
>  overrides any \code{adj} value given.  Values of \code{1},
>  \code{2}, \code{3} and \code{4}, respectively indicate
>  positions below, to the left of, above and to the right of
> -the specified coordinates.}
> -  \item{offset}{when \code{pos} is specified, this value gives the
> -offset of the label from the specified coordinate in fractions
> -of a character width.}
> +\code{(x, y)}.}
> +  \item{offset}{when \code{pos} is specified, this value controls the
> +distance of the text label from \code{(x, y)}, in fractions of a
> +character width.}
>\item{vfont}{\code{NULL} for the current font family, or a character
>  vector of length 2 for Hershey vector fonts.  The first element of
>  the vector selects a typeface and the second element selects a
> @@ -62,10 +64,11 @@
>mathematical notation is available such as sub- and superscripts,
>greek letters, fractions, etc.
>  
> -  \code{adj} allows \emph{adj}ustment of the text with respect to
> +  \code{adj} allows \emph{adj}ustment of the text position with respect to
>\code{(x, y)}.
> -  Values of 

Re: [Rd] Modification-proposal for %% (modulo) when supplied with double

[frederik]

Duncan, I think Emil realizes that the floating point format isn't
able to represent certain numbers, that's why he is suggesting this
change rather than complaining about our arithmetic being broken.

However, I agree with you that we should not adopt his proposal. It
would not make things more "user friendly" for people. Everyone has a
different application and a different use of %% and they just need to
keep in mind that they are talking to a computer and not a blackboard.
Here is an example of a feature that was meant to help users get more
intuitive results with floating point numbers, but which actually
caused headaches instead:
https://github.com/Rdatatable/data.table/issues/1642 It is a slightly
different scenario to this one, but I think it is still a good example
of how we can end up creating unforeseen problems for people if we
change core functionality to do unsolicited rounding behind the
scenes.

Best wishes,

Frederick

On Tue, Sep 11, 2018 at 12:11:29PM -0400, Duncan Murdoch wrote:
> On 11/09/2018 11:23 AM, Emil Bode wrote:
> > Hi all,
> > 
> > 
> > 
> > Could we modify the "%%" (modulo)-operator to include some tolerance for 
> > rounding-errors when supplied with doubles?
> > 
> > It's not much work (patch supplied on the bottom), and I don't think it 
> > would break anything, only if you were really interested in analysing 
> > rounding differences.
> > 
> > Any ideas about implementing this and overwriting base::`%%`, or would we 
> > want another method (as I've done for the moment)?
> 
> I think this is a bad idea.  Your comments say "The
> \code{\link[base:Arithmetic]{`\%\%`}} operator calculates the modulo, but
> sometimes has rounding errors, e.g. "\code{(9.1/.1) \%\% 1}" gives ~ 1,
> instead of 0."
> 
> This is false.  The %% calculation is exactly correct.  The rounding error
> happened in your input:  9.1/0.1 is not equal to 91, it is a little bit
> less:
> 
> > options(digits=20)
> > 9.1/.1
> [1] 90.985789
> 
> And %% did not return 1, it returned the correct value:
> 
> > (9.1/.1) %% 1
> [1] 0.98578915
> 
> So it makes no sense to change %%.
> 
> You might argue that the division 9.1/.1 is giving the wrong answer, but in
> fact that answer is correct too.  The real problem is that in double
> precision floating point the numbers 9.1 and .1 can't be represented
> exactly.  This is well known, it's in the FAQ (question 7.31).
> 
> Duncan Murdoch
> 
> > 
> > 
> > 
> > Background
> > 
> > I was writing some code where something has to happen at a certain 
> > interval, with progress indicated, something like this:
> > 
> > interval <- .001
> > 
> > progress <- .1
> > 
> > for(i in 1:1000*interval) {myFun(i); Sys.sleep(interval); if(i %% progress, 
> > 0))) cat(i, '\n')}
> > 
> > without interval and progress being known in advance. I could work around 
> > it and make i integer, or do something like
> > 
> > isTRUE(all.equal(i %% progress,0)) || isTRUE(all.equal(i %% progress, 
> > progress),
> > 
> > but I think my code is clearer as it is. And I like the idea behind 
> > all.equal: we want double to approximately identical.
> > 
> > 
> > 
> > So my patch (with roxygen2-markup):
> > 
> > #' Modulo-operator with near-equality
> > 
> > #'
> > 
> > #' The \code{\link[base:Arithmetic]{`\%\%`}} operator calculates the 
> > modulo, but sometimes has rounding errors, e.g. "\code{(9.1/.1) \%\% 1}" 
> > gives ~ 1, instead of 0.\cr
> > 
> > #' Comparable to what all.equal does, this operator has some tolerance for 
> > small rounding errors.\cr
> > 
> > #' If the answer would be equal to the divisor within a small tolerance, 0 
> > is returned instead.
> > 
> > #'
> > 
> > #' For integer x and y, the normal \%\%-operator is used
> > 
> > #'
> > 
> > #' @usage `\%mod\%`(x, y, tolerance = sqrt(.Machine$double.eps))
> > 
> > #' x \%mod\% y
> > 
> > #' @param x,y numeric vectors, similar to those passed on to \%\%
> > 
> > #' @param tolerance numeric, maximum difference, see 
> > \code{\link[base]{all.equal}}. The default is ~ \code{1.5e-8}
> > 
> > #' @return identical to the result for \%\%, unless the answer would be 
> > really close to y, in which case 0 is returned
> > 
> > #' @note To specify tolerance, use the call \code{`\%mod\%`(x,y,tolerance)}
> > 
> > #' @note The precedence for \code{\%mod\%} is the same as that for 
> > \code{\%\%}
> > 
> > #'
> > 
> > #' @name mod
> > 
> > #' @rdname mod
> > 
> > #'
> > 
> > #' @export
> > 
> > `%mod%` <- function(x,y, tolerance = sqrt(.Machine$double.eps)) {
> > 
> >stopifnot(is.numeric(x), is.numeric(y), is.numeric(tolerance),
> > 
> >  !is.na(tolerance), length(tolerance)==1, tolerance>=0)
> > 
> >if(is.integer(x) && is.integer(y)) {
> > 
> >  return(x %% y)
> > 
> >} else {
> > 
> >  ans <- x %% y
> > 
> >  return(ifelse(abs(ans-y) > 
> >}
> > 
> > }
> > 
> > 
> > 
> > Best regards,
> > 
> > Emil Bode
> > 
> > [[alternative HTML version deleted]]
> > 
> > 

Re: [Rd] Duplicate column names created by base::merge() when by.x has the same name as a column in y

[frederik]

Hi Scott,

I tried the new patch and can confirm that it has the advertised
behavior on a couple of test cases. I think it makes sense to apply
it, because any existing code which refers to a second duplicate
data.frame column by name is already broken, while if the reference is
by numerical index then changing the column name shouldn't break it.

I don't know if you need to update the documentation as part of your
patch, or if whoever applies it would be happy to do that. Somebody
from R core want to weigh in on this?

I attach a file with the test example from your original email as well
as a second test case I added with two "by" columns.

Thanks,

Frederick

On Wed, Feb 21, 2018 at 10:06:21AM +1100, Scott Ritchie wrote:
> Hi Frederick,
> 
> It looks like I didn't overwrite the patch.diff file after the last edits.
> Here's the correct patch (attached and copied below):
> 
> Index: src/library/base/R/merge.R
> ===
> --- src/library/base/R/merge.R (revision 74280)
> +++ src/library/base/R/merge.R (working copy)
> @@ -157,6 +157,14 @@
>  }
> 
>  if(has.common.nms) names(y) <- nm.y
> +## If by.x %in% names(y) then duplicate column names still arise,
> +## apply suffixes to just y - this keeps backwards compatibility
> +## when referring to by.x in the resulting data.frame
> +dupe.keyx <- intersect(nm.by, names(y))
> +if(length(dupe.keyx)) {
> +  if(nzchar(suffixes[2L]))
> +names(y)[match(dupe.keyx, names(y), 0L)] <- paste(dupe.keyx,
> suffixes[2L], sep="")
> +}
>  nm <- c(names(x), names(y))
>  if(any(d <- duplicated(nm)))
>  if(sum(d) > 1L)
> 
> Best,
> 
> Scott
> 
> On 21 February 2018 at 08:23,  wrote:
> 
> > Hi Scott,
> >
> > I think that's a good idea and I tried your patch on my copy of the
> > repository. But it looks to me like the recent patch is identical to
> > the previous one, can you confirm this?
> >
> > Frederick
> >
> > On Mon, Feb 19, 2018 at 07:19:32AM +1100, Scott Ritchie wrote:
> > > Thanks Gabriel,
> > >
> > > I think your suggested approach is 100% backwards compatible
> > >
> > > Currently in the case of duplicate column names only the first can be
> > > indexed by its name. This will always be the column appearing in by.x,
> > > meaning the column in y with the same name cannot be accessed. Appending
> > > ".y" (suffixes[2L]) to this column means it can now be accessed, while
> > > keeping the current behaviour of making the key columns always accessible
> > > by using the names provided to by.x.
> > >
> > > I've attached a new patch that has this behaviour.
> > >
> > > Best,
> > >
> > > Scott
> > >
> > > On 19 February 2018 at 05:08, Gabriel Becker 
> > wrote:
> > >
> > > > It seems like there is a way that is backwards compatible-ish in the
> > sense
> > > > mentioned and still has the (arguably, but a good argument I think)
> > better
> > > > behavior:
> > > >
> > > > if by.x is 'name', (AND by.y is not also 'name'), then x's 'name'
> > column
> > > > is called name and y's 'name' column (not used int he merge) is
> > changed to
> > > > name.y.
> > > >
> > > > Now of course this would still change output, but it would change it to
> > > > something I think would be better, while retaining the 'merge columns
> > > > retain their exact names' mechanic as documented.
> > > >
> > > > ~G
> > > >
> > > > On Sat, Feb 17, 2018 at 6:50 PM, Scott Ritchie 
> > > > wrote:
> > > >
> > > >> Thanks Duncan and Frederick,
> > > >>
> > > >> I suspected as much - there doesn't appear to be any reason why
> > conflicts
> > > >> between by.x and names(y) shouldn't and cannot be checked, but I can
> > see
> > > >> how this might be more trouble than its worth given it potentially may
> > > >> break downstream packages (i.e. any cases where this occurs but they
> > > >> expect
> > > >> the name of the key column(s) to remain the same).
> > > >>
> > > >> Best,
> > > >>
> > > >> Scott
> > > >>
> > > >> On 18 February 2018 at 11:48, Duncan Murdoch <
> > murdoch.dun...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > On 17/02/2018 6:36 PM, frede...@ofb.net wrote:
> > > >> >
> > > >> >> Hi Scott,
> > > >> >>
> > > >> >> Thanks for the patch. I'm not really involved in R development; it
> > > >> >> will be up to someone in the R core team to apply it. I would
> > hazard
> > > >> >> to say that even if correct (I haven't checked), it will not be
> > > >> >> applied because the change might break existing code. For example
> > it
> > > >> >> seems like reasonable code might easily assume that a column with
> > the
> > > >> >> same name as "by.x" exists in the output of 'merge'. That's just my
> > > >> >> best guess... I don't participate on here often.
> > > >> >>
> > > >> >
> > > >> >
> > > >> > I think you're right.  If I were still a member of R Core, I would
> > want
> > 

Re: [Rd] Duplicate column names created by base::merge() when by.x has the same name as a column in y

[frederik]

Hi Scott,

I think that's a good idea and I tried your patch on my copy of the
repository. But it looks to me like the recent patch is identical to
the previous one, can you confirm this?

Frederick

On Mon, Feb 19, 2018 at 07:19:32AM +1100, Scott Ritchie wrote:
> Thanks Gabriel,
> 
> I think your suggested approach is 100% backwards compatible
> 
> Currently in the case of duplicate column names only the first can be
> indexed by its name. This will always be the column appearing in by.x,
> meaning the column in y with the same name cannot be accessed. Appending
> ".y" (suffixes[2L]) to this column means it can now be accessed, while
> keeping the current behaviour of making the key columns always accessible
> by using the names provided to by.x.
> 
> I've attached a new patch that has this behaviour.
> 
> Best,
> 
> Scott
> 
> On 19 February 2018 at 05:08, Gabriel Becker  wrote:
> 
> > It seems like there is a way that is backwards compatible-ish in the sense
> > mentioned and still has the (arguably, but a good argument I think) better
> > behavior:
> >
> > if by.x is 'name', (AND by.y is not also 'name'), then x's 'name' column
> > is called name and y's 'name' column (not used int he merge) is changed to
> > name.y.
> >
> > Now of course this would still change output, but it would change it to
> > something I think would be better, while retaining the 'merge columns
> > retain their exact names' mechanic as documented.
> >
> > ~G
> >
> > On Sat, Feb 17, 2018 at 6:50 PM, Scott Ritchie 
> > wrote:
> >
> >> Thanks Duncan and Frederick,
> >>
> >> I suspected as much - there doesn't appear to be any reason why conflicts
> >> between by.x and names(y) shouldn't and cannot be checked, but I can see
> >> how this might be more trouble than its worth given it potentially may
> >> break downstream packages (i.e. any cases where this occurs but they
> >> expect
> >> the name of the key column(s) to remain the same).
> >>
> >> Best,
> >>
> >> Scott
> >>
> >> On 18 February 2018 at 11:48, Duncan Murdoch 
> >> wrote:
> >>
> >> > On 17/02/2018 6:36 PM, frede...@ofb.net wrote:
> >> >
> >> >> Hi Scott,
> >> >>
> >> >> Thanks for the patch. I'm not really involved in R development; it
> >> >> will be up to someone in the R core team to apply it. I would hazard
> >> >> to say that even if correct (I haven't checked), it will not be
> >> >> applied because the change might break existing code. For example it
> >> >> seems like reasonable code might easily assume that a column with the
> >> >> same name as "by.x" exists in the output of 'merge'. That's just my
> >> >> best guess... I don't participate on here often.
> >> >>
> >> >
> >> >
> >> > I think you're right.  If I were still a member of R Core, I would want
> >> to
> >> > test this against all packages on CRAN and Bioconductor, and since that
> >> > test takes a couple of days to run on my laptop, I'd probably never get
> >> > around to it.
> >> >
> >> > There are lots of cases where "I would have done that differently", but
> >> > most of them are far too much trouble to change now that R is more than
> >> 20
> >> > years old.  And in many cases it will turn out that the way R does it
> >> > actually does make more sense than the way I would have done it.
> >> >
> >> > Duncan Murdoch
> >> >
> >> >
> >> >
> >> >> Cheers,
> >> >>
> >> >> Frederick
> >> >>
> >> >> On Sat, Feb 17, 2018 at 04:42:21PM +1100, Scott Ritchie wrote:
> >> >>
> >> >>> The attached patch.diff will make merge.data.frame() append the
> >> suffixes
> >> >>> to
> >> >>> columns with common names between by.x and names(y).
> >> >>>
> >> >>> Best,
> >> >>>
> >> >>> Scott Ritchie
> >> >>>
> >> >>> On 17 February 2018 at 11:15, Scott Ritchie 
> >> >>> wrote:
> >> >>>
> >> >>> Hi Frederick,
> >> 
> >>  I would expect that any duplicate names in the resulting data.frame
> >>  would
> >>  have the suffixes appended to them, regardless of whether or not they
> >>  are
> >>  used as the join key. So in my example I would expect "names.x" and
> >>  "names.y" to indicate their source data.frame.
> >> 
> >>  While careful reading of the documentation reveals this is not the
> >>  case, I
> >>  would argue the intent of the suffixes functionality should equally
> >> be
> >>  applied to this type of case.
> >> 
> >>  If you agree this would be useful, I'm happy to write a patch for
> >>  merge.data.frame that will add suffixes in this case - I intend to do
> >>  the
> >>  same for merge.data.table in the data.table package where I initially
> >>  encountered the edge case.
> >> 
> >>  Best,
> >> 
> >>  Scott
> >> 
> >>  On 17 February 2018 at 03:53,  wrote:
> >> 
> >>  Hi Scott,
> >> >
> >> > It seems like reasonable behavior to me. What result would you
> >> expect?
> >> > That the 

Re: [Rd] Duplicate column names created by base::merge() when by.x has the same name as a column in y

[frederik]

Hi Scott,

Thanks for the patch. I'm not really involved in R development; it
will be up to someone in the R core team to apply it. I would hazard
to say that even if correct (I haven't checked), it will not be
applied because the change might break existing code. For example it
seems like reasonable code might easily assume that a column with the
same name as "by.x" exists in the output of 'merge'. That's just my
best guess... I don't participate on here often.

Cheers,

Frederick

On Sat, Feb 17, 2018 at 04:42:21PM +1100, Scott Ritchie wrote:
> The attached patch.diff will make merge.data.frame() append the suffixes to
> columns with common names between by.x and names(y).
> 
> Best,
> 
> Scott Ritchie
> 
> On 17 February 2018 at 11:15, Scott Ritchie  wrote:
> 
> > Hi Frederick,
> >
> > I would expect that any duplicate names in the resulting data.frame would
> > have the suffixes appended to them, regardless of whether or not they are
> > used as the join key. So in my example I would expect "names.x" and
> > "names.y" to indicate their source data.frame.
> >
> > While careful reading of the documentation reveals this is not the case, I
> > would argue the intent of the suffixes functionality should equally be
> > applied to this type of case.
> >
> > If you agree this would be useful, I'm happy to write a patch for
> > merge.data.frame that will add suffixes in this case - I intend to do the
> > same for merge.data.table in the data.table package where I initially
> > encountered the edge case.
> >
> > Best,
> >
> > Scott
> >
> > On 17 February 2018 at 03:53,  wrote:
> >
> >> Hi Scott,
> >>
> >> It seems like reasonable behavior to me. What result would you expect?
> >> That the second "name" should be called "name.y"?
> >>
> >> The "merge" documentation says:
> >>
> >> If the columns in the data frames not used in merging have any
> >> common names, these have ‘suffixes’ (‘".x"’ and ‘".y"’ by default)
> >> appended to try to make the names of the result unique.
> >>
> >> Since the first "name" column was used in merging, leaving both
> >> without a suffix seems consistent with the documentation...
> >>
> >> Frederick
> >>
> >> On Fri, Feb 16, 2018 at 09:08:29AM +1100, Scott Ritchie wrote:
> >> > Hi,
> >> >
> >> > I was unable to find a bug report for this with a cursory search, but
> >> would
> >> > like clarification if this is intended or unavoidable behaviour:
> >> >
> >> > ```{r}
> >> > # Create example data.frames
> >> > parents <- data.frame(name=c("Sarah", "Max", "Qin", "Lex"),
> >> >   sex=c("F", "M", "F", "M"),
> >> >   age=c(41, 43, 36, 51))
> >> > children <- data.frame(parent=c("Sarah", "Max", "Qin"),
> >> >name=c("Oliver", "Sebastian", "Kai-lee"),
> >> >sex=c("M", "M", "F"),
> >> >age=c(5,8,7))
> >> >
> >> > # Merge() creates a duplicated "name" column:
> >> > merge(parents, children, by.x = "name", by.y = "parent")
> >> > ```
> >> >
> >> > Output:
> >> > ```
> >> >name sex.x age.x  name sex.y age.y
> >> > 1   Max M43 Sebastian M 8
> >> > 2   Qin F36   Kai-lee F 7
> >> > 3 Sarah F41Oliver M 5
> >> > Warning message:
> >> > In merge.data.frame(parents, children, by.x = "name", by.y = "parent") :
> >> >   column name ‘name’ is duplicated in the result
> >> > ```
> >> >
> >> > Kind Regards,
> >> >
> >> > Scott Ritchie
> >> >
> >> >   [[alternative HTML version deleted]]
> >> >
> >> > __
> >> > R-devel@r-project.org mailing list
> >> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >> >
> >>
> >
> >

> Index: src/library/base/R/merge.R
> ===
> --- src/library/base/R/merge.R(revision 74264)
> +++ src/library/base/R/merge.R(working copy)
> @@ -157,6 +157,15 @@
>  }
>  
>  if(has.common.nms) names(y) <- nm.y
> +## If by.x %in% names(y) then duplicate column names still arise,
> +## apply suffixes to these
> +dupe.keyx <- intersect(nm.by, names(y))
> +if(length(dupe.keyx)) {
> +  if(nzchar(suffixes[1L]))
> +names(x)[match(dupe.keyx, names(x), 0L)] <- paste(dupe.keyx, 
> suffixes[1L], sep="")
> +  if(nzchar(suffixes[2L]))
> +names(y)[match(dupe.keyx, names(y), 0L)] <- paste(dupe.keyx, 
> suffixes[2L], sep="")
> +}
>  nm <- c(names(x), names(y))
>  if(any(d <- duplicated(nm)))
>  if(sum(d) > 1L)

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Duplicate column names created by base::merge() when by.x has the same name as a column in y

[frederik]

Hi Scott,

It seems like reasonable behavior to me. What result would you expect?
That the second "name" should be called "name.y"?

The "merge" documentation says:

If the columns in the data frames not used in merging have any
common names, these have ‘suffixes’ (‘".x"’ and ‘".y"’ by default)
appended to try to make the names of the result unique.

Since the first "name" column was used in merging, leaving both
without a suffix seems consistent with the documentation...

Frederick

On Fri, Feb 16, 2018 at 09:08:29AM +1100, Scott Ritchie wrote:
> Hi,
> 
> I was unable to find a bug report for this with a cursory search, but would
> like clarification if this is intended or unavoidable behaviour:
> 
> ```{r}
> # Create example data.frames
> parents <- data.frame(name=c("Sarah", "Max", "Qin", "Lex"),
>   sex=c("F", "M", "F", "M"),
>   age=c(41, 43, 36, 51))
> children <- data.frame(parent=c("Sarah", "Max", "Qin"),
>name=c("Oliver", "Sebastian", "Kai-lee"),
>sex=c("M", "M", "F"),
>age=c(5,8,7))
> 
> # Merge() creates a duplicated "name" column:
> merge(parents, children, by.x = "name", by.y = "parent")
> ```
> 
> Output:
> ```
>name sex.x age.x  name sex.y age.y
> 1   Max M43 Sebastian M 8
> 2   Qin F36   Kai-lee F 7
> 3 Sarah F41Oliver M 5
> Warning message:
> In merge.data.frame(parents, children, by.x = "name", by.y = "parent") :
>   column name ‘name’ is duplicated in the result
> ```
> 
> Kind Regards,
> 
> Scott Ritchie
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Suggestion: Create On-Disk Dataframes; SparkR

[frederik]

What's wrong with SparkR? I never heard of either Spark or SparkR.

For on-disk dataframes there is a package called 'ff'. I looked into
using it, it works well but there are some drawbacks with the
implementation. I think that it should be possible to mmap an object
from disk and use it as a vector, but 'ff' is doing something else:

https://github.com/edwindj/ffbase/issues/52

I think you'd need something called a "weak reference" to do this
properly:

http://homepage.divms.uiowa.edu/~luke/R/references/weakfinex.html

I don't know what SparkR is doing under the hood.

Then again I was mostly interested in having large data sets which
persist across R sessions, while Juan seems to be interested in
supporting data which doesn't fit in RAM. But if something doesn't fit
in RAM, it can be swapped out to disk by the OS, no? So I'm not sure
why you'd want a special interface for that situation, aside from
giving the programmer more control.

Thanks,

Frederick

On Mon, Sep 04, 2017 at 07:43:50AM -0500, Dirk Eddelbuettel wrote:
> 
> On 4 September 2017 at 11:35, Suzen, Mehmet wrote:
> | It is not needed. There is a large community of developer using SparkR.
> | https://spark.apache.org/docs/latest/sparkr.html
> | It does exactly what you want.
> 
> I hope you are not going to mail a sparkr commercial to this list every day.
> As the count is now at two, this may be an excellent good time to stop it.
> 
> Dirk
> 
> -- 
> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] italic font on cairo devices in R 3.4

[frederik]

On Fri, Jul 07, 2017 at 07:08:52PM +0200, Ilia Kats wrote:
> Interesting. I did not have the package installed, but I did at some point
> extract Helvetica from some MacOSX font files and R was using that just fine
> until 3.3. This is how the plot looks in 3.4 (still using Helvetica):
> https://ptpb.pw/HikX.pdf . After removing Helvetica, installing the
> ttf-mscorefonts-installer package, and running fc-cache --force  the plot
> looks like this: https://ptpb.pw/CM8A.pdf

The second plot looks worse, in other words, and the Microsoft fonts
didn't help.

Maybe the Cairo device should be giving better warning messages.

Anyway it sounds like you are describing a regression so maybe someone
can track down the commit that created this problem.

Thanks,

Frederick


> Also note that the standard pdf device works fine: https://ptpb.pw/3Ml1.pdf
> , it's just the cairo devices (both pdf and svg) that have the issue.
> Unfortunately I need to use cairo_pdf due to unicode characters in axis
> labels.
> 
> Cheers, Ilia
> 
> 
>  Original Message 
> Subject: Re: [Rd] italic font on cairo devices in R 3.4
> Date: 2017-07-07 18:17:34 +0200
> From: frederik
> To: Ilia Kats
> CC: r-devel, r-help
> > Hi Ilia,
> > 
> > I'm running Arch Linux, R 3.4.0.
> > 
> > Here's my test.pdf from your minimal example: https://ptpb.pw/HxsA.pdf
> > 
> > It doesn't look pixelated to me...
> > 
> > Here's a post that I wrote when I solved my last font problem in R,
> > almost 2 years ago:
> > 
> > https://stackoverflow.com/a/40940331/5087283
> > 
> > I had to install some Microsoft font packages, which is sad, because
> > there are some perfectly good free fonts that R could be using
> > instead. It could be considered a bug that R requires Microsoft fonts,
> > at least by default. However, does this even fix your problem? I.e. if
> > you install the corresponding Debian Microsoft font packages, does the
> > text appear anti-aliased?
> > 
> > Frederick
> > 
> > On Fri, Jul 07, 2017 at 10:30:46AM +0200, Ilia Kats wrote:
> > > [cross-post from R-help]
> > > 
> > > Hi all,
> > > 
> > > I have the following problem: Since R 3.4.0, italic fonts rendered on 
> > > Cairo
> > > devices appear pixelated. Here's a minimal example:
> > > cairo_pdf('test.pdf')
> > > plot(1:10, ylab=expression(italic(test)))
> > > dev.off()
> > > 
> > > The same problem occurs with bolditalic, but not bold. I am using Debian
> > > Stretch. Several friends tried the same on their machines, another Debian
> > > machine has the same problem. On MacOSX the output was not pixelated, but 
> > > it
> > > wasn't italic either. Ubuntu 16.04.2 xenial works fine. My impression is
> > > that R somehow can't find the proper font to use and falls back to 
> > > something
> > > weird. Ideas?
> > > 
> > > Note that I'm not subscribed to the list, so please CC me in replies.
> > > 
> > > Cheers, Ilia
> > > 
> > > __
> > > R-devel@r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > > 
> 
> -- 
> The first is to ensure your partner understands that nature has root
> privileges - nature doesn't have to make sense.
> -- Telsa Gwynne
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] italic font on cairo devices in R 3.4

[frederik]

Hi Ilia,

I'm running Arch Linux, R 3.4.0.

Here's my test.pdf from your minimal example: https://ptpb.pw/HxsA.pdf

It doesn't look pixelated to me...

Here's a post that I wrote when I solved my last font problem in R,
almost 2 years ago:

https://stackoverflow.com/a/40940331/5087283

I had to install some Microsoft font packages, which is sad, because
there are some perfectly good free fonts that R could be using
instead. It could be considered a bug that R requires Microsoft fonts,
at least by default. However, does this even fix your problem? I.e. if
you install the corresponding Debian Microsoft font packages, does the
text appear anti-aliased?

Frederick

On Fri, Jul 07, 2017 at 10:30:46AM +0200, Ilia Kats wrote:
> [cross-post from R-help]
> 
> Hi all,
> 
> I have the following problem: Since R 3.4.0, italic fonts rendered on Cairo
> devices appear pixelated. Here's a minimal example:
> cairo_pdf('test.pdf')
> plot(1:10, ylab=expression(italic(test)))
> dev.off()
> 
> The same problem occurs with bolditalic, but not bold. I am using Debian
> Stretch. Several friends tried the same on their machines, another Debian
> machine has the same problem. On MacOSX the output was not pixelated, but it
> wasn't italic either. Ubuntu 16.04.2 xenial works fine. My impression is
> that R somehow can't find the proper font to use and falls back to something
> weird. Ideas?
> 
> Note that I'm not subscribed to the list, so please CC me in replies.
> 
> Cheers, Ilia
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] SUGGESTION: R Base Packages

[frederik]

Hello!

Thanks to Juan for his suggestions. I would like to voice my opinion
for and against some of these.

> o   Add in Bugzilla an Area in order to request membership for bug reporting, 
> which shall itself include another area to introduce a little dissertation 
> for justifying "Why someone would like to be a member of Bugzilla" for being 
> able to report proper bugs, and avoid spam.
> 
> o   Now, new R serious members cannot report properly bugs.

I too was annoyed by the state of the bug reporting system when I
first started using R, and that was even before automatic account
creation was disabled. I think the situation could be improved. I
don't think it's a huge burden to tell bug reporters to ask for an
account via the R-devel mailing list. But from what I can see the bug
tracker doesn't make it clear that this has to be done. We should at
least post a message on the main page explaining how people can create
an account. This would be less work than what Juan is suggesting, but
I think sufficient for our needs.

> * That, in the R Base distribution, the following packages are 
> included as default (pre-installed):
> 
> o   ggplot2.
> 
> o   dplyr.
> 
> o   tidyr.

I think this is a terrible idea. I've only been using R for a few
years and while I find these "tidyverse" packages interesting and use
them on occasion, I've also concluded that they change too quickly to
be used in code that I want to stay working for a long time. They are
largely based on experimental concepts. Being able to change and
evolve is part of the strength of these packages. So I think putting
them in the R Base distribution would be bad for all parties.

> * And that, as regards R Project:
> 
> o   Improve the CRAN & R Project web pages, giving them a more modern look 
> and feel (Such as the Python.org webpage), in order to "sell" better R to the 
> everybody.
> 
> o   Improve the basic GUI distributed with R, with some of the core features 
> of RStudio Desktop (GPL). Or even make RStudio Desktop to replace the R 
> default GUI, with another name, in its executable version.
> 
> o   Enable a Suggestion web page / forum in the R Project webpage in order to 
> boost core R Open Innovation.

I think the R project web page looks great. It's simple and it loads
quickly and doesn't try to mesmerize people. I like R's command line
interface. It completes on symbols and files and I can easily use it
with my favorite editor and run it in the terminal of my choice. I
don't think that more effort should be put into developing bloated
GUIs which try to enforce a standard way of interacting with R.

I think a "forum" or bulletin board system would be a detraction from
the project and a distraction for the project leaders. Users have
Stack Exchange - it's better than any forum we could create, and it
takes care of itself.

That's my two cents, more or less.

Thanks,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] tempdir() may be deleted during long-running R session

[frederik]

> > It might still be useful if R could sometimes detect when automated
> > cleanup happened and warn the user. Perhaps a simple way could be to
> > always create an empty file inside session directory, like
> > ".tmp_cleaner_trap". R would never touch this file, but check its
> > existence time-to-time. If it gets deleted, R would issue a warning and
> > ask the user to check tmp cleaner configuration. The idea is that this
> > file will be the oldest one in the session directory, so would get
> > cleaned up first.

It may kill two birds with one stone if we put the PID into such a
file. Then it could be used to automate the cleanup of /tmp/Rtmp
directories left around from crashed R sessions (since we're about to
tell Systemd not to touch them).

I'm not sure what is meant by "R would ... check its existence
time-to-time". It doesn't sound to me like an important enough task to
go into the main event loop of the interpreter. Maybe a warning/error
in 'tempdir()' to tell us that either the directory has been deleted
or that the PID file no longer exists.

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] tempdir() may be deleted during long-running R session

[frederik]

> Frederick posted the way to do this in systems using systemd.  We should be
> putting that in place, or the equivalent on systems using other tempfile
> cleanups.  This looks to me like something that "make install" should do, or
> perhaps it should be done by people putting together packages for specific
> systems.

For Arch's 'apache' package it is just a file in the package directory
that gets installed by the PKGBUILD script:

install -D -m644 "${srcdir}/apache.tmpfiles.conf" 
"${pkgdir}/usr/lib/tmpfiles.d/apache.conf"

Similarly for the 'screen' package.

I added a task in the Arch bug tracker to do this for R:

https://bugs.archlinux.org/task/53848

I don't get the sense that it is customary to put examples of such
files into the source distribution, so I think nothing more needs to
be done on your side (aside from alerting package maintainers for
other distributions).

Thanks,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] tempdir() may be deleted during long-running R session

[frederik]

Hi Gabriel,

Thanks for asking for a better solution, as far as actually preventing
temporary files from getting deleted in the first place.

I still don't know very much about other peoples' distributions, but
Arch uses Systemd which is the culprit on my system. Systemd's
'tmpfiles.d(5)' man page says we can create configuration files in
locations like

/usr/lib/tmpfiles.d/*.conf
/etc/tmpfiles.d/*.conf

which control when temporary files are deleted. There is an 'x'
specifier which accepts glob paths and can protect everything in
/tmp/Rtmp* ...:

$ mkdir /tmp/Rtmpaoeu
$ touch -d "12 days ago" /tmp/Rtmpaoeu
$ sudo systemd-tmpfiles --clean
$ ls /tmp/Rtmpaoeu
ls: cannot access '/tmp/Rtmpaoeu': No such file or directory

$ sudo sh -c "echo 'x /tmp/Rtmp*' > /etc/tmpfiles.d/R.conf"
$ mkdir /tmp/Rtmpaoeu
$ touch -d "12 days ago" /tmp/Rtmpaoeu
$ sudo systemd-tmpfiles --clean
$ ls /tmp/Rtmpaoeu
(still there)

I guess installing such a file is something that would be done by the
various distribution-specific R packages. Even though I run R from a
home-directory compiled version, I have my distribution's binary
package installed globally, and so I would get the benefit of this
protection from the distribution package. If this sounds like it makes
sense then I can ask the Arch package maintainer to do it. Of course I
don't need permission but it would be good to hear if I'm missing or
forgetting something.

Based on what other packages are doing the file should probably be
named:

/usr/lib/tmpfiles.d/R.conf

and contain:

x /tmp/Rtmp*

(For example on my system I have stuff like this owned by various
packages:

$ pacman -Qo /usr/lib/tmpfiles.d/*
/usr/lib/tmpfiles.d/apache.conf is owned by apache 2.4.25-1
/usr/lib/tmpfiles.d/bind.conf is owned by bind 9.11.0.P3-3
/usr/lib/tmpfiles.d/colord.conf is owned by colord 1.3.4-1
/usr/lib/tmpfiles.d/etc.conf is owned by systemd 232-8
/usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf is owned by gvfs 1.30.3-1
...

)

Thanks!

Frederick

On Tue, Apr 25, 2017 at 09:03:01AM -0700, Gabriel Becker wrote:
> Martin,
> 
> Thanks for your work on this.
> 
> One thing that seems to be missing from the conversation is that recreating
> the temp directory will prevent future failures when R wants to write a
> temp file, but the files will, of course, not be there. Any code written
> assuming the contract is that the temporary directory, and thus temporary
> files, will not be cleaned up before the R process exits (which was my
> naive assumption before this thread, and is the behavior AFAICT on all the
> systems I regularly use) will still break.
> 
> I'm not saying that's necessarily fixable (though the R keeping a permanent
> pointer to a file in the dir suggested by Malcom might? fix it.), but I
> would argue if it IS fixable, a fix that includes that would be preferable.
> 
> Best,
> ~G
> 
> On Tue, Apr 25, 2017 at 8:53 AM, Martin Maechler  > wrote:
> 
> > > Jeroen Ooms 
> > > on Tue, 25 Apr 2017 15:05:51 +0200 writes:
> >
> > > On Tue, Apr 25, 2017 at 1:00 PM, Martin Maechler
> > >  wrote:
> > >> As I've found it is not at all hard to add an option
> > >> which checks the existence and if the directory is no
> > >> longer "valid", tries to recreate it (and if it fails
> > >> doing that it calls the famous R_Suicide(), as it does
> > >> when R starts up and tempdir() cannot be initialized
> > >> correctly).
> >
> > > Perhaps this can also fix the problem with mcparallel
> > > deleting the tempdir() when one of its children dies:
> >
> >>   file.exists(tempdir()) #TRUE
> >>   parallel::mcparallel(q('no'))
> >>   file.exists(tempdir()) # FALSE
> >
> > Thank you, Jeroen, for the extra example.
> >
> > I now have comitted the new feature... (completely back
> > compatible: in R's code tempdir() is not yet called with an
> > argument and the default is  check = FALSE ),
> > actually in a "suicide-free" way ...  which needed only slightly
> > more code.
> >
> > In the worst case, one could save the R session by
> >Sys.setenv(TEMPDIR = "")
> > if for instance /tmp/ suddenly became unwritable for the user.
> >
> > What we could consider is making the default of 'check' settable
> > by an option, and experiment with setting the option to TRUE, so
> > all such problems would be auto-solved (says the incurable optimist ...).
> >
> > Martin
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> 
> 
> 
> -- 
> Gabriel Becker, PhD
> Associate Scientist (Bioinformatics)
> Genentech Research
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

Re: [Rd] tempdir() may be deleted during long-running R session

[frederik]

On Tue, Apr 25, 2017 at 02:41:58PM +, Cook, Malcolm wrote:
> Might this combination serve the purpose: 
>   * R session keeps an open handle on the tempdir it creates, 
>   * whatever tempdir harvesting cron job the user has be made sensitive 
> enough not to delete open files (including open directories)

Good suggestion but doesn't work with the (increasingly popular)
"Systemd":

$ mkdir /tmp/somedir
$ touch -d "12 days ago" /tmp/somedir/
$ cd /tmp/somedir/
$ sudo systemd-tmpfiles --clean
$ ls /tmp/somedir/
ls: cannot access '/tmp/somedir/': No such file or directory

I would advocate just changing 'tempfile()' so that it recreates the
directory where the file is (the "dirname") before returning the file
path. This would have fixed the issue I ran into. Changing 'tempdir()'
to recreate the directory is another option.

Thanks,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] tempdir() may be deleted during long-running R session

[frederik]

Dirk,

Your message felt a bit antagonistic to me, or maybe I'm not
understanding what you're trying to say. We all seem to agree that
different configurations exist, and that some Linux distributions are
configured to delete files in /tmp/ after a certain amount of time
(seems to be 10 days for Arch Linux, not sure about Ubuntu or Debian).

The question of how users of such distributions can individually work
around the problem Mikko identified has already been answered. The
question that remains is what we expect new users to do. It's not
really helpful to pretend that they will be reading the mailing list,
as exciting as it is, or that they'll read the "R Installation and
Administration" manual to make sure that their distribution did a good
job of packaging R. There are plenty more visible places where this
"gotcha" could be documented, than a manual I've never heard of until
now.

Even if a particular solution has to be implemented by the package
maintainers of various distributions, I think it is fitting to discuss
and solicit such solutions here on this mailing list. But it felt like
you were trying to stifle such discussion.

As it is, I don't even know what distributions are affected. I'm not
sure how to look up the contents of a "default" configuration on other
distributions.

Frederick


On Sun, Apr 23, 2017 at 09:15:18AM -0500, Dirk Eddelbuettel wrote:
> 
> On 21 April 2017 at 10:34, frede...@ofb.net wrote:
> | Hi Mikko,
> | 
> | I was bitten by this recently and I think some of the replies are
> | missing the point. As I understand it, the problem consists of these
> | elements:
> | 
> | 1. When R starts, it creates a directory like /tmp/RtmpVIeFj4
> | 
> | 2. Right after R starts I can create files in this directory with no
> |error
> | 
> | 3. After some hours or days I can no longer create files in this
> |directory, because it has been deleted
> 
> Nope. That is local to your system. Witness eg at my workstation:
> 
> /tmp$ ls -ltGd Rtmp* 
> drwx-- 3 edd 4096 Apr 21 16:12 Rtmp9K6bSN
> drwx-- 3 edd 4096 Apr 21 11:48 RtmpRRbaMP
> drwx-- 3 edd 4096 Apr 21 11:28 RtmpFlguFy
> drwx-- 3 edd 4096 Apr 20 13:06 RtmpWJDF3U
> drwx-- 3 edd 4096 Apr 18 15:58 RtmpY7ZIS1
> drwx-- 3 edd 4096 Apr 18 12:12 Rtmpzr9W0v
> drwx-- 2 edd 4096 Apr 16 16:02 RtmpeD27El
> drwx-- 2 edd 4096 Apr 16 15:57 Rtmp572FHk
> drwx-- 3 edd 4096 Apr 13 11:08 RtmpqP0JSf
> drwx-- 3 edd 4096 Apr 10 18:47 RtmpzRzyFb
> drwx-- 3 edd 4096 Apr  6 15:21 RtmpQhvAUb
> drwx-- 3 edd 4096 Apr  6 11:24 Rtmp2lFKPz
> drwx-- 3 edd 4096 Apr  5 20:57 RtmprCeWUS
> drwx-- 2 edd 4096 Apr  3 15:12 Rtmp8xviDl
> drwx-- 3 edd 4096 Mar 30 16:50 Rtmp8w9n5h
> drwx-- 3 edd 4096 Mar 28 11:33 RtmpjAg6iY
> drwx-- 2 edd 4096 Mar 28 09:26 RtmpYHSgZG
> drwx-- 2 edd 4096 Mar 27 11:21 Rtmp0gSV4e
> drwx-- 2 edd 4096 Mar 27 11:21 RtmpOnneiY
> drwx-- 2 edd 4096 Mar 27 11:17 RtmpIWeiTJ
> drwx-- 3 edd 4096 Mar 22 08:51 RtmpJkVsSJ
> drwx-- 3 edd 4096 Mar 21 10:33 Rtmp9a5KxL
> /tmp$ 
> 
> Clearly still there after a month. I tend to have some longer-running R
> sessions in either Emacs/ESS or RStudio.
> 
> So what I wrote in my last message here *clearly* applies to you: a local
> issue for which you have to take local action as R cannot know.  You also
> have a choice of setting variables to affect this.
>  
> | If R expected the directory to be deleted at random, and if we expect
> | users to call dir.create every time they access tempdir, then why did
> | R create the directory for us at the beginning of the session? That's
> | just setting people up to get weird bugs, which only appear in
> | difficult-to-reproduce situations (i.e. after the session has been
> | open for a long time).
> 
> I disagree. R has been doing this many years, possibly two decades.
>  
> | I think before we dismiss this we should think about possible in-R
> | solutions and why they are not feasible. Are there any packages which
> | would break if a call to 'tempdir' automatically recreated this
> | directory? (Or would it be too much of a performance hit to have
> | 'tempdir' check and even just issue a warning when the directory is
> | found not to exist?) Should we have a timer which periodically updates
> | the modification time of tempdir()? What do other long-running
> | programs do (e.g. screen, emacs)?
> 
> There are options you have right now.
> 
> Dirk
> 
> -- 
> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] tempdir() may be deleted during long-running R session

[frederik]

Hi Mikko,

I was bitten by this recently and I think some of the replies are
missing the point. As I understand it, the problem consists of these
elements:

1. When R starts, it creates a directory like /tmp/RtmpVIeFj4

2. Right after R starts I can create files in this directory with no
   error

3. After some hours or days I can no longer create files in this
   directory, because it has been deleted

If R expected the directory to be deleted at random, and if we expect
users to call dir.create every time they access tempdir, then why did
R create the directory for us at the beginning of the session? That's
just setting people up to get weird bugs, which only appear in
difficult-to-reproduce situations (i.e. after the session has been
open for a long time).

I think before we dismiss this we should think about possible in-R
solutions and why they are not feasible. Are there any packages which
would break if a call to 'tempdir' automatically recreated this
directory? (Or would it be too much of a performance hit to have
'tempdir' check and even just issue a warning when the directory is
found not to exist?) Should we have a timer which periodically updates
the modification time of tempdir()? What do other long-running
programs do (e.g. screen, emacs)?

Thank you,

Frederick

P.S. I noticed that dir.create does not seem to update the access or
modification time of the file. So there is also a remote possibility
that the directory could be "cleaned up" in between calling
'dir.create()' and putting a file in it. Maybe this is nitpicky, but
if we accept that the *really* correct practice is more complicated
than just calling 'dir.create()', this also argues for putting the
proper invocations into some kind of standard function - either
'tempdir()' or something else.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Poor documentation for "adj" and text()

[frederik]

Thanks Ulrich for sharing your experience.

I'm attaching a patch which tries to address the issues you raised.

I agree with you in principle, but I think it makes sense to leave
some details under "Details". However, the descriptions in "Arguments"
should give enough information that a user can get the function to do
something predictable in at least one situation, and I feel this is
not the case at present.

I tried to fix the wording so that 'adj' and 'offset' are no longer
confusing to new users (or to me, every time I forget what they mean).

I also fixed the paragraph on rotated text; it is more correct now, at
least for X11-cairo.

I hope that someone in the Core Team can look this over and apply it.

Thank you,

Frederick

On Tue, Apr 11, 2017 at 09:23:50AM +0200, Ulrich Windl wrote:
> Hi!
> 
> (I'd like to be able to access your bugzilla, BTW)
> The documentation for parameter "adj" of text() in R 3.3.3 is hard to 
> understand (unless you know what it does already):
> 
> "adj 
> one or two values in [0, 1] which specify the x (and optionally y) adjustment 
> of the labels. On most devices values outside that interval will also work."
> 
> What is the meaning of the values? I think the description ("adj allows 
> adjustment of the text with respect to (x, y). Values of 0, 0.5, and 1 
> specify left/bottom, middle and right/top alignment, respectively. The 
> default is for centered text, i.e., adj = c(0.5, NA). Accurate vertical 
> centering needs character metric information on individual characters which 
> is only available on some devices. Vertical alignment is done slightly 
> differently for character strings and for expressions: adj = c(0,0) means to 
> left-justify and to align on the baseline for strings but on the bottom of 
> the bounding box for expressions. This also affects vertical centering: for 
> strings the centering excludes any descenders whereas for expressions it 
> includes them. Using NA for strings centers them, including descenders.") 
> should be moved to the parameter.
> 
> In general I'd suggest to describe the range, meaning and default of every 
> parameter where the parameter is listed. "Details" should only give an 
> overview of the functions.
> 
> Likewise "offset": Will the direction be influenced by "pos"? The description 
> is quite silent on that.
> 
> Documentation should be structured to help the user to find the facts easily 
> without having to read the whole page.
> 
> Regards,
> Ulrich Windl
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
--- text.Rd	2016-11-27 18:33:26.541516325 -0800
+++ new-text.Rd	2017-04-11 11:48:32.668926075 -0700
@@ -26,16 +26,18 @@
 If \code{labels} is longer than \code{x} and
 \code{y}, the coordinates are recycled to the length of \code{labels}.}
   \item{adj}{one or two values in \eqn{[0, 1]} which specify the x
-(and optionally y) adjustment of the labels.  On most devices values
-outside that interval will also work.}
+(and optionally y) justification of the labels, with 0 for
+left/bottom, 1 for right/top, and 0.5 for centered.
+On most devices values
+outside \eqn{[0, 1]} will also work. See below.}
   \item{pos}{a position specifier for the text.  If specified this
 overrides any \code{adj} value given.  Values of \code{1},
 \code{2}, \code{3} and \code{4}, respectively indicate
 positions below, to the left of, above and to the right of
-the specified coordinates.}
-  \item{offset}{when \code{pos} is specified, this value gives the
-offset of the label from the specified coordinate in fractions
-of a character width.}
+\code{(x, y)}.}
+  \item{offset}{when \code{pos} is specified, this value controls the
+distance of the text label from \code{(x, y)}, in fractions of a
+character width.}
   \item{vfont}{\code{NULL} for the current font family, or a character
 vector of length 2 for Hershey vector fonts.  The first element of
 the vector selects a typeface and the second element selects a
@@ -62,10 +64,11 @@
   mathematical notation is available such as sub- and superscripts,
   greek letters, fractions, etc.
 
-  \code{adj} allows \emph{adj}ustment of the text with respect to
+  \code{adj} allows \emph{adj}ustment of the text position with respect to
   \code{(x, y)}.
-  Values of 0, 0.5, and 1 specify left/bottom, middle and
-  right/top alignment, respectively.  The default is for centered text, i.e.,
+  Values of 0, 0.5, and 1 specify that \code{(x, y)} should align with
+  the left/bottom, middle and
+  right/top of the text, respectively.  The default is for centered text, i.e.,
   \code{adj = c(0.5, NA)}.  Accurate vertical centering needs
   character metric information on individual characters which is
   only available on some devices.  Vertical alignment is done slightly
@@ -81,8 +84,17 @@
   labelled plot.
 
   Text can be rotated by using 

Re: [Rd] strptime("1","%m") returns NA

[frederik]

Hi Martyn,

Thanks for finding that stuff in the documentation, and apologies for
not reading the whole thing carefully. I guess when I got to the
minutiae about printing years before '999', I started to skim.

My vote is for more sensible / standard behavior, but I guess this has
probably been this way for a long time.

Frederick

On Tue, Jan 17, 2017 at 01:35:35PM +, Martyn Plummer wrote:
> Hi Frederik,
> 
> On Mon, 2017-01-16 at 18:20 -0800, frede...@ofb.net wrote:
> > Hi R Devel,
> > 
> > I wrote some code which depends on 'strptime' being able to parse an
> > incomplete date, like this:
> > 
> > > 
> > > base::strptime("2016","%Y")
> > [1] "2016-01-14 PST"
> > 
> > The above works - although it's odd that it gives the month and day
> > for Sys.time(). I might expect it to set them both to zero as the GNU
> > libc strptime does on my system, or to use January 1 which would also
> > be reasonable.
> 
> From the help page for strptime:
> 
> "For ‘strptime’ the input string need not specify the date completely:
> it is assumed that unspecified seconds, minutes or hours are zero, and
> an unspecified year, month or day is the current one."
>  
> > When I specify the month, however, I get NA:
> > 
> > > 
> > > base::strptime("2016-12","%Y-%m")
> > [1] NA
> > > 
> > > base::strptime("1", "%m")
> > [1] NA
> > 
> > Any reason for this to be the case?
> 
> Also from the help page:
> 
> "(However, if a month is specified, the day of that month has to be
> specified by ‘%d’ or ‘%e’ since the current day of the month need not
> be valid for the specified month.)"
> 
> If strptime("2016-2", "%Y-%m") just filled in the current day then it
> would give valid output when called on the 1st to the 28th of each
> month, but would give either invalid output or fail when called on the
> 29th to the 31st of any month. This would be a nightmare to debug. The
> current behaviour lets you know there is a logical problem with your
> input.
> 
> > I reported a bug here:
> > 
> > https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17212
> > 
> > but I don't think I'm getting emails from Bugzilla so maybe best to
> > ping me if anyone replies there instead.
> 
> See the general guidance on submitting bug reports:
> 
> "Code doing something unexpected is not necessarily a bug - make sure to 
> carefully review the documentation for the function you are calling to see if 
> the behaviour it exhibits is what it was designed to do, even if it’s not 
> what you want."
> 
> https://www.r-project.org/bugs.html
> 
> 
> Martyn
> 
> > I've just written a simple reimplementation of 'strptime' for my own
> > use; I hope this bug report may be useful to others.
> > 
> > Thank you,
> > 
> > Frederick
> > 
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Graphics Device API change

[frederik]

Thank you Paul for taking care of these patches. I'm very glad to see
them finally applied in r-devel.

Frederick

On Mon, Feb 27, 2017 at 01:31:39PM +1300, Paul Murrell wrote:
> Hi
> 
> r72261 in r-devel has modified the graphics device API and bumped
> R_GE_version in GraphicsEngine.h (to satisfy PR 16951).
> 
> This means that ...
> 
> (i) third-party graphics device packages need to be reinstalled.
> 
> (ii) third-party graphics device packages will ideally need a new version
> for r-devel.
> 
> (iii) any packages that make use of getGraphicsEvent() should test against
> r-devel (and let me know of any problems).
> 
> Thanks
> 
> Paul
> -- 
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> p...@stat.auckland.ac.nz
> http://www.stat.auckland.ac.nz/~paul/
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Update copyright year in manuals

[frederik]

On Thu, Feb 23, 2017 at 03:23:10PM +0100, Martin Maechler wrote:
> > Mikko Korpela 
> > on Thu, 23 Feb 2017 14:02:58 +0200 writes:
> 
> > With new R releases soon to come, I suggest updating the
> > Rcopyright macro in "doc/manual/R-defs.texi" to use year
> > 2017.
> 
> Now this is an e-mail that *REALLY* does not fit to the R-devel
> mailing list ... even though it is very very slightly related to
> the R sources.
> 
> We do *not* want noise on R-devel, please.
> 
> (and let's continue this issue in private if you want)
> Martin

Martin, I'm confused. Should Mikko have sent his suggestion to the
bugtracker, or just left it to the R Core Team to figure out for
themselves? Are we not grateful for valid suggestions? What's the
lesson here? Maybe I'm the only slow one on the list, but to me your
reply would have been a bit more helpful if it told us what to do, not
just what not to do...

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] strptime("1","%m") returns NA

[frederik]

Hi R Devel,

I wrote some code which depends on 'strptime' being able to parse an
incomplete date, like this:

> base::strptime("2016","%Y")
[1] "2016-01-14 PST"

The above works - although it's odd that it gives the month and day
for Sys.time(). I might expect it to set them both to zero as the GNU
libc strptime does on my system, or to use January 1 which would also
be reasonable.

When I specify the month, however, I get NA:

> base::strptime("2016-12","%Y-%m")
[1] NA
> base::strptime("1", "%m")
[1] NA

Any reason for this to be the case?

I reported a bug here:

https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17212

but I don't think I'm getting emails from Bugzilla so maybe best to
ping me if anyone replies there instead.

I've just written a simple reimplementation of 'strptime' for my own
use; I hope this bug report may be useful to others.

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] unlicense

[frederik]

I don't see why Charles' question should be taken as anything other
than an honest request for information.

As for me, I've never heard of this license, but if CRAN doesn't have
an option to license software in the public domain, then I would
support the inclusion of some such option.

FWIW, searching for "public domain software license" on Google turns
up unlicense.org as the second result.

Frederick

On Fri, Jan 13, 2017 at 07:19:47PM -0500, Duncan Murdoch wrote:
> On 13/01/2017 3:21 PM, Charles Geyer wrote:
> > I would like the unlicense (http://unlicense.org/) added to R
> > licenses.  Does anyone else think that worthwhile?
> > 
> 
> That's a question for you to answer, not to ask.  Who besides you thinks
> that it's a good license for open source software?
> 
> If it is recognized by the OSF or FSF or some other authority as a FOSS
> license, then CRAN would probably also recognize it.  If not, then CRAN
> doesn't have the resources to evaluate it and so is unlikely to recognize
> it.
> 
> Duncan Murdoch
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] bug with strptime, %OS, and "."

[frederik]

Thanks for the reply. That's a creative way to do it, but I wanted to
use the same format string for both parsing and formatting my file
name.

I figured I should just put this on the bug tracker in case someone is
editing strptime at some point in the distant future:

https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17208

I also made a related bug asking for e.g. "%OS6" to be accepted by
strptime.

https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17209

Cheers,

Frederick

On Wed, Jan 11, 2017 at 01:36:07PM +, Upton, Stephen (Steve) (CIV) wrote:
> Works for me:
> > strptime("17_35_14.01234.mp3","%H_%M_%OS")$sec
> [1] 14.01234
> > strptime("17_35_14.mp3","%H_%M_%OS")$sec
> [1] 14
> 
> Just leave off the ".mp3" in your time pattern.
> 
> Relevant section from the help ("Details") for strptime:
> strptime converts character vectors to class "POSIXlt": its input x is first
> converted by as.character. Each input string is processed as far as
> necessary for the format specified: any trailing characters are ignored.
> 
> R version 3.3.2 (2016-10-31)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> Running under: Windows 7 x64 (build 7601) Service Pack 1
> 
> Stephen C. Upton
> Faculty Associate - Research
> SEED (Simulation Experiments & Efficient Designs) Center
> Operations Research Department
> Naval Postgraduate School
> Mobile: 804-994-4257
> NIPR: scup...@nps.edu
> SIPR: upto...@nps.navy.smil.mil
> SEED Center web site: http://harvest.nps.edu
> -Original Message-
> From: R-devel [mailto:r-devel-boun...@r-project.org] On Behalf Of
> frede...@ofb.net
> Sent: Tuesday, January 10, 2017 9:59 PM
> To: Dirk Eddelbuettel
> Cc: R-devel
> Subject: Re: [Rd] bug with strptime, %OS, and "."
> 
> On Tue, Jan 10, 2017 at 08:13:21PM -0600, Dirk Eddelbuettel wrote:
> > 
> > On 10 January 2017 at 17:48, frede...@ofb.net wrote:
> > | Hi R Devel,
> > | 
> > | I just ran into a corner case with 'strptime'. Recall that the "%OS"
> > | conversion accepts fractional seconds:
> > | 
> > | > strptime("17_35_14.01234.mp3","%H_%M_%OS.mp3")$sec
> > | [1] 14.01234
> > | 
> > | Unfortunately for my application it seems to be "greedy", in that it 
> > | tries to parse a decimal point which might belong to the rest of the
> > | format:
> > | 
> > | > strptime("17_35_14.mp3","%H_%M_%OS.mp3")
> > | [1] NA
> > 
> > Maybe just don't use the optional O:
> > 
> >R> strptime("17_35_14.mp3","%H_%M_%S.mp3")$sec
> >[1] 14
> >R> 
> >R> strptime("17_35_14.mp3","%H_%M_%S.mp3")
> >[1] "2017-01-10 17:35:14 CST"
> >R>
> 
> For my application I wanted to be able to accept both formats, "14.mp3" and
> "14.01234.mp3". Since "14" and "14.01234" both parse as numbers, I thought
> "%OS" should accept both.
> 
> Yes, I can work around it fairly easily.
> 
> Frederick
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] bug with strptime, %OS, and "."

[frederik]

On Tue, Jan 10, 2017 at 08:13:21PM -0600, Dirk Eddelbuettel wrote:
> 
> On 10 January 2017 at 17:48, frede...@ofb.net wrote:
> | Hi R Devel,
> | 
> | I just ran into a corner case with 'strptime'. Recall that the "%OS"
> | conversion accepts fractional seconds:
> | 
> | > strptime("17_35_14.01234.mp3","%H_%M_%OS.mp3")$sec
> | [1] 14.01234
> | 
> | Unfortunately for my application it seems to be "greedy", in that it
> | tries to parse a decimal point which might belong to the rest of the
> | format:
> | 
> | > strptime("17_35_14.mp3","%H_%M_%OS.mp3")
> | [1] NA
> 
> Maybe just don't use the optional O:
> 
>R> strptime("17_35_14.mp3","%H_%M_%S.mp3")$sec
>[1] 14
>R> 
>R> strptime("17_35_14.mp3","%H_%M_%S.mp3")
>[1] "2017-01-10 17:35:14 CST"
>R>

For my application I wanted to be able to accept both formats,
"14.mp3" and "14.01234.mp3". Since "14" and "14.01234" both parse as
numbers, I thought "%OS" should accept both.

Yes, I can work around it fairly easily.

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] bug with strptime, %OS, and "."

[frederik]

Hi R Devel,

I just ran into a corner case with 'strptime'. Recall that the "%OS"
conversion accepts fractional seconds:

> strptime("17_35_14.01234.mp3","%H_%M_%OS.mp3")$sec
[1] 14.01234

Unfortunately for my application it seems to be "greedy", in that it
tries to parse a decimal point which might belong to the rest of the
format:

> strptime("17_35_14.mp3","%H_%M_%OS.mp3")
[1] NA

If I use "_" instead of ".", then it works:

> strptime("17_35_14_mp3","%H_%M_%OS_mp3")
[1] "2017-01-10 17:35:14 PST"

Perhaps a low priority, but seems like a bug to me...

Thanks,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] [RE: why does parent.frame() cycle when called from inside capture.output()?]

[frederik]

Thanks, Mark - I'm taking up your invitation to forward your message
to the list just because it gives us some valuable data on (1) how
long the behavior has been around, and (2) how many other people
(don't) understand the behavior, and (3) how we might fix or work
around it.

I notice some other people also seem to be diffident about posting on
R-devel; perhaps I should conclude that bugs like this are below the
radar for busy statisticians.

FWIW, after playing around a bit with mvbutils::mvb.parent.frame, I
now have a working "desubN" (see the attachment on my original
message, and this one). I don't really have a good understanding of
*why* it now works, and why the original version didn't work...

Thanks again Mark. Also, nice hacking.

Frederick

- Forwarded message from mark.braving...@data61.csiro.au -

Date: Mon, 12 Dec 2016 06:20:17 +
From: mark.braving...@data61.csiro.au
To: frede...@ofb.net
Subject: RE: [Rd] why does parent.frame() cycle when called from inside
capture.output()?
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,T_SPF_HELO_TEMPERROR,T_SPF_TEMPERROR 
autolearn=ham
autolearn_force=no version=3.4.1
X-Spam-Level: 
X-My-Tags: inbox r-devel

Hi Frederik

[ I'm replying off-list in case you, or the rest of R-devel, don't find this 
reply useful... please fwd to the list if it does help you ]

I'm the author of the 'debug' package. When I wrote it--- many years ago now--- 
I encountered some fairly strange behaviour with frames in the call stack, 
which is reminiscent of what you're seeing. The debug package still works fine, 
so I presume nothing has changed much.

If you load debug and then do ?mvb.sys.parent (and look at the code), you might 
get *some* idea of what's going on. TBH I can't remember the details now...

HTH
Mark

Mark Bravington
CSIRO Marine Lab
Hobart
Australia


From: R-devel [r-devel-boun...@r-project.org] on behalf of frede...@ofb.net 
[frede...@ofb.net]
Sent: 12 December 2016 16:54
To: r-devel@r-project.org
Cc: r-h...@r-project.org
Subject: [Rd] why does parent.frame() cycle when called from inside 
capture.output()?

Hello R devel/help,

I ran into this strange behavior:

# showstack is supposed to walk through the stack of parent
# environments when it is called:
showstack = function() {
  env = environment()
  for(i in 1:12) {
env = do.call(parent.frame, list(), env=env)
print(env)
  }
}

# a simple chain of functions:
g3=function(t) showstack()
g2=function(w) g3(w)
g1=function(z) g2(z)
g=function(y) g1(y)

g()
# outputs:
# 
# 
# 
# 
# 
# 
# 
# 
# 
# ...

cat(capture.output(g()),sep="\n")
# outputs:
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 

The strange thing of course is that the second call doesn't stay with
R_GlobalEnv, but in fact goes into a loop of period 4. I'm not so
surprised that the parent frame of the global environment is itself,
as in the first call, but it seems weird to have a loop of period 4...
Using `ls()` shows that two of the "loop" environments belong to
capture.output() and eval().

But if capture.output is really evaluating its input in the parent
frame, as it appears to be doing from its source code, then I would
have expected the output to be the same as the output I get by
evaluating the same expression in this frame.

I was trying to debug a function which attempts to be a multi-frame
version of deparse(substitute(...)). I'm attaching this function in
case anyone is curious. Perhaps my attachment can shed more light on
the problem I'm having.

Apologies if this is not a bug - I wasn't sure which mailing list to
send this to, and I took a guess.

Thanks,

Frederick


- End forwarded message -
# We can't just use `mvb.parent.frame` as a replacement for
# `parent.frame`, because the former throws an error when there is no
# parent frame - while we had relied on the latter gracefully giving
# us the global environment. So here's a wrapper which gives
# mvb.parent.frame the behavior we want:

my_mvb_parent=function() {
  tryCatch(
mvb.parent.frame(2),
error=function(e) { globalenv()})
}

desubN <- function(y,n=1) {
  env=environment();
  for(i in 1:n) {
y = do.call(substitute, list(substitute(y)), env=env)
env = do.call(my_mvb_parent, list(), env=env)
  }
  deparse(y)
}
g2=function(t) {
  for(i in 1:5) {
print(desubN(t,i))
print(capture.output(desubN(t,i)))
  }
}
g1=function(z) g2(z)
g=function(y) g1(y)

g(log)

# now capture.output seems to give the same results as the bare
# expression
## [1] "t"
## [1] "[1] \"t\""
## [1] "z"
## [1] "[1] \"z\""
## [1] "y"
## [1] "[1] \"y\""
## [1] "log"
## 

[Rd] why does parent.frame() cycle when called from inside capture.output()?

[frederik]

Hello R devel/help,

I ran into this strange behavior:

# showstack is supposed to walk through the stack of parent
# environments when it is called:
showstack = function() {
  env = environment()
  for(i in 1:12) {
env = do.call(parent.frame, list(), env=env)
print(env)
  }
}

# a simple chain of functions:
g3=function(t) showstack()
g2=function(w) g3(w)
g1=function(z) g2(z)
g=function(y) g1(y)

g()
# outputs:
# 
# 
# 
# 
# 
# 
# 
# 
# 
# ...

cat(capture.output(g()),sep="\n")
# outputs:
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 

The strange thing of course is that the second call doesn't stay with
R_GlobalEnv, but in fact goes into a loop of period 4. I'm not so
surprised that the parent frame of the global environment is itself,
as in the first call, but it seems weird to have a loop of period 4...
Using `ls()` shows that two of the "loop" environments belong to
capture.output() and eval().

But if capture.output is really evaluating its input in the parent
frame, as it appears to be doing from its source code, then I would
have expected the output to be the same as the output I get by
evaluating the same expression in this frame.

I was trying to debug a function which attempts to be a multi-frame
version of deparse(substitute(...)). I'm attaching this function in
case anyone is curious. Perhaps my attachment can shed more light on
the problem I'm having.

Apologies if this is not a bug - I wasn't sure which mailing list to
send this to, and I took a guess.

Thanks,

Frederick
desubN <- function(y,n=1) {
  env=environment();
  for(i in 1:n) {
y = do.call(substitute, list(substitute(y)), env=env)
env = do.call(parent.frame, list(), env=env)
  }
  deparse(y)
}
g2=function(t) {
  for(i in 1:5) {
print(desubN(t,i))
print(capture.output(desubN(t,i)))
  }
}
g1=function(z) g2(z)
g=function(y) g1(y)

g(log)
# output: (why does it stop at "z"?)
## [1] "t"
## [1] "[1] \"t\""
## [1] "z"
## [1] "[1] \"z\""
## [1] "y"
## [1] "[1] \"z\""
## [1] "log"
## [1] "[1] \"z\""
## [1] "log"
## [1] "[1] \"z\""
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] methods(`|`) lists all functions?

[frederik]

On Thu, Dec 08, 2016 at 09:37:59PM -0500, Martin Morgan wrote:
> On 12/08/2016 05:16 PM, frede...@ofb.net wrote:
> > Dear R-Devel,
> > 
> > I was attempting an exercise in Hadley Wickam's book "Advanced R". The
> > exercise is to find the generic with the greatest number of methods.
> > 
> > I found that 'methods(`|`)' produces a list of length 2506, in R
> > 3.3.1. Similar behavior is found in 3.4.0. It seems to include all
> > functions and methods. I imagine something is being passed to "grep"
> > without being escaped.
> 
> Exactly; I've fixed this in r71763 (R-devel).
> 
> Martin Morgan

Thank you.

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] getGraphicsEvent() alternative for cairo graphics device?

[frederik]

Hi Paul,

Thanks for keeping me posted and letting me know what I should do.

Are there regression tests for other graphics functions in R? To me
that sounds a bit unnecessary. I think you get more testing from
people who use R; and having a good turnaround for applying patches
(some have been waiting 5 years) would invite better participation.

Thank you,

Frederick

On Fri, Dec 09, 2016 at 12:01:55PM +1300, Paul Murrell wrote:
> Hi
> 
> Just taking a bit more of a look at this today (currently fixated on making
> sure I can build some good regression tests).
> 
> The best thing you can do is to keep reminding me like this :)
> 
> Paul
> 
> On 09/12/16 11:19, frede...@ofb.net wrote:
> > Hi Paul,
> > 
> > Thanks for your efforts. Do you have an idea when my patch(es) might
> > be committed? Is there anything I can do to help move this along?
> > 
> > Thanks,
> > 
> > Frederick
> > 
> > On Mon, Nov 14, 2016 at 08:55:20AM +1300, Paul Murrell wrote:
> > > Hi
> > > 
> > > The current status is that I am keen for people to contribute some testing
> > > code (see https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951)
> > > 
> > > There were also some getGraphicsEvent() changes/fixes suggested by Richard
> > > Bodewits (cc'ed), for which I am also seeking test code.
> > > 
> > > Paul
> > > 
> > > On 13/11/16 09:00, frede...@ofb.net wrote:
> > > > Hi Paul,
> > > > 
> > > > Just checking in to see what the status is.
> > > > 
> > > > From my perspective it seems logical to split off X11 into a separate
> > > > package, so development can proceed at a reasonable rate, but I
> > > > haven't yet tried to see if that's even possible.
> > > > 
> > > > Thanks,
> > > > 
> > > > Frederick
> > > > 
> > > > On Tue, Jul 26, 2016 at 09:23:35AM +1200, Paul Murrell wrote:
> > > > > Hi
> > > > > 
> > > > > Taking a look at those patches is now on my todo list, so I may be in 
> > > > > touch
> > > > > with both of you at some point to request some testing.
> > > > > 
> > > > > Paul
> > > > > 
> > > > > On 26/07/16 07:17, frede...@ofb.net wrote:
> > > > > > Dear Daniel Greenidge,
> > > > > > 
> > > > > > To enable getGraphicsEvent on Cairo, you have two patches to choose
> > > > > > from:
> > > > > > 
> > > > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=14364
> > > > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951
> > > > > > 
> > > > > > The second one is by me, and the first one is from five years ago by
> > > > > > Hugo Mildenberger.
> > > > > > 
> > > > > > Both patches are very simple, they move some lines enabling
> > > > > > getGrahpicsEvent outside of a if(!cairo) statement. My patch also 
> > > > > > adds
> > > > > > the ability to execute code (e.g. for animation) while the interface
> > > > > > is idle.
> > > > > > 
> > > > > > Top guy Duncan Murdoch has expressed that he doesn't have time to 
> > > > > > work
> > > > > > on applying these patches, and I haven't had any responses from the
> > > > > > rest of the R Core Team. I was thinking that perhaps your best bet 
> > > > > > is
> > > > > > to try to create a package called e.g. "X11-fixes" which people can
> > > > > > use to get a better X11 library (there is also a bug waiting to be
> > > > > > fixed from 2001:
> > > > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702).
> > > > > > 
> > > > > > I don't know if CRAN would accept such a package, or if you'd have 
> > > > > > to
> > > > > > distribute it via GitHub, but R has excellent tools to facilitate 
> > > > > > the
> > > > > > distribution of code via packages. Whether the R kernel exports 
> > > > > > enough
> > > > > > functions to allow a package to take over event handling, I'm not
> > > > > > sure. I was intending to look more into the details of this
> > > > > > possibility but haven't had time.
> > > > > > 
> > > > > > Best wishes,
> > > > > > 
> > > > > > Frederick
> > > > > > 
> > > > > > On Mon, Jul 25, 2016 at 02:15:59PM -0400, Daniel Greenidge wrote:
> > > > > > > Hi all,
> > > > > > > 
> > > > > > > I'm writing an interactive plotting function for viewing fMRI
> > > > > > > datasets. Currently, I get keypresses using
> > > > > > > grDevices::getGraphicsEvent().
> > > > > > > 
> > > > > > > Unfortunately getGraphicsEvent() only supports the 
> > > > > > > X11(type="Xlib")
> > > > > > > graphics device on Unix systems. The Xlib device doesn't support
> > > > > > > buffering (i.e. dev.hold() and dev.flush()), so redrawing the 
> > > > > > > plots
> > > > > > > causes lots of flickering.
> > > > > > > 
> > > > > > > Is there a way to get keypresses while using the cairo graphics
> > > > > > > device? Alternatively, is there a way to prevent flickering with 
> > > > > > > the
> > > > > > > Xlib graphics device?
> > > > > > > 
> > > > > > > Best,
> > > > > > > Daniel Greenidge
> > > > > > > 
> > > > > > > __
> > > > > > > R-devel@r-project.org mailing list
> > > > > > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > 

Re: [Rd] getGraphicsEvent() alternative for cairo graphics device?

[frederik]

Hi Paul,

Thanks for your efforts. Do you have an idea when my patch(es) might
be committed? Is there anything I can do to help move this along?

Thanks,

Frederick

On Mon, Nov 14, 2016 at 08:55:20AM +1300, Paul Murrell wrote:
> Hi
> 
> The current status is that I am keen for people to contribute some testing
> code (see https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951)
> 
> There were also some getGraphicsEvent() changes/fixes suggested by Richard
> Bodewits (cc'ed), for which I am also seeking test code.
> 
> Paul
> 
> On 13/11/16 09:00, frede...@ofb.net wrote:
> > Hi Paul,
> > 
> > Just checking in to see what the status is.
> > 
> > From my perspective it seems logical to split off X11 into a separate
> > package, so development can proceed at a reasonable rate, but I
> > haven't yet tried to see if that's even possible.
> > 
> > Thanks,
> > 
> > Frederick
> > 
> > On Tue, Jul 26, 2016 at 09:23:35AM +1200, Paul Murrell wrote:
> > > Hi
> > > 
> > > Taking a look at those patches is now on my todo list, so I may be in 
> > > touch
> > > with both of you at some point to request some testing.
> > > 
> > > Paul
> > > 
> > > On 26/07/16 07:17, frede...@ofb.net wrote:
> > > > Dear Daniel Greenidge,
> > > > 
> > > > To enable getGraphicsEvent on Cairo, you have two patches to choose
> > > > from:
> > > > 
> > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=14364
> > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951
> > > > 
> > > > The second one is by me, and the first one is from five years ago by
> > > > Hugo Mildenberger.
> > > > 
> > > > Both patches are very simple, they move some lines enabling
> > > > getGrahpicsEvent outside of a if(!cairo) statement. My patch also adds
> > > > the ability to execute code (e.g. for animation) while the interface
> > > > is idle.
> > > > 
> > > > Top guy Duncan Murdoch has expressed that he doesn't have time to work
> > > > on applying these patches, and I haven't had any responses from the
> > > > rest of the R Core Team. I was thinking that perhaps your best bet is
> > > > to try to create a package called e.g. "X11-fixes" which people can
> > > > use to get a better X11 library (there is also a bug waiting to be
> > > > fixed from 2001:
> > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702).
> > > > 
> > > > I don't know if CRAN would accept such a package, or if you'd have to
> > > > distribute it via GitHub, but R has excellent tools to facilitate the
> > > > distribution of code via packages. Whether the R kernel exports enough
> > > > functions to allow a package to take over event handling, I'm not
> > > > sure. I was intending to look more into the details of this
> > > > possibility but haven't had time.
> > > > 
> > > > Best wishes,
> > > > 
> > > > Frederick
> > > > 
> > > > On Mon, Jul 25, 2016 at 02:15:59PM -0400, Daniel Greenidge wrote:
> > > > > Hi all,
> > > > > 
> > > > > I'm writing an interactive plotting function for viewing fMRI
> > > > > datasets. Currently, I get keypresses using
> > > > > grDevices::getGraphicsEvent().
> > > > > 
> > > > > Unfortunately getGraphicsEvent() only supports the X11(type="Xlib")
> > > > > graphics device on Unix systems. The Xlib device doesn't support
> > > > > buffering (i.e. dev.hold() and dev.flush()), so redrawing the plots
> > > > > causes lots of flickering.
> > > > > 
> > > > > Is there a way to get keypresses while using the cairo graphics
> > > > > device? Alternatively, is there a way to prevent flickering with the
> > > > > Xlib graphics device?
> > > > > 
> > > > > Best,
> > > > > Daniel Greenidge
> > > > > 
> > > > > __
> > > > > R-devel@r-project.org mailing list
> > > > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > > > > 
> > > > 
> > > > __
> > > > R-devel@r-project.org mailing list
> > > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > > > 
> > > 
> > > --
> > > Dr Paul Murrell
> > > Department of Statistics
> > > The University of Auckland
> > > Private Bag 92019
> > > Auckland
> > > New Zealand
> > > 64 9 3737599 x85392
> > > p...@stat.auckland.ac.nz
> > > http://www.stat.auckland.ac.nz/~paul/
> > > 
> 
> -- 
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> p...@stat.auckland.ac.nz
> http://www.stat.auckland.ac.nz/~paul/
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] methods(`|`) lists all functions?

[frederik]

Dear R-Devel,

I was attempting an exercise in Hadley Wickam's book "Advanced R". The
exercise is to find the generic with the greatest number of methods.

I found that 'methods(`|`)' produces a list of length 2506, in R
3.3.1. Similar behavior is found in 3.4.0. It seems to include all
functions and methods. I imagine something is being passed to "grep"
without being escaped.

I hope I didn't miss something in the documentation, and that I'm good
to report this as a bug. I can send it to Bugzilla if that's better.

By the way, how do I produce such a list of functions (or variables)
in a "normal" way? I used 'ls("package:base")' for the exercise,
because I saw this call used somewhere as an example, but I couldn't
find that "package:" syntax documented under ls()... Also found this
confusing:

> environmentName(globalenv())
[1] "R_GlobalEnv"
> ls("R_GlobalEnv")
Error in as.environment(pos) :
  no item called "R_GlobalEnv" on the search list

So I'm not sure if "package:base" is naming an environment, or if
there are different ways to name environments and ls() is using one
form while environmentName is returning another ... It might be good
to add some clarifying examples under "?ls".

Thanks,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] segfault with POSIXlt zone=NULL zone=""

[frederik]

Thanks for the reply, Duncan.

It looks like Martin can commit a patch faster than I can open a bug
report...

Frederick

On Tue, Dec 06, 2016 at 12:39:32PM -0500, Duncan Murdoch wrote:
> I agree this is a bug; R should never segfault.  I wouldn't call it a high
> priority one, since you can avoid the problem by not messing with R's
> internal structures.
> 
> It's unlikely to get fixed unless someone posts it as a bug report to
> bugs.r-project.org (because low priority bugs reported only on mailing lists
> get forgotten).
> 
> So please post a minimal example there, possibly accompanied with a patch.
> If you don't have an account, you can write to me privately and I'll set one
> up for you.  (We no longer allow people to create their own accounts because
> of abuse by spammers.)
> 
> Duncan Murdoch

On Tue, Dec 06, 2016 at 06:49:05PM +0100, Martin Maechler wrote:
> > Joshua Ulrich 
> > on Tue, 6 Dec 2016 09:51:16 -0600 writes:
> 
> > On Tue, Dec 6, 2016 at 6:37 AM,   wrote:
> >> Hi all,
> >> 
> >> I ran into a segfault while playing with dates.
> >> 
> >> $ R --no-init-file
> >> ...
> >> > library(lubridate); d=as.POSIXlt(floor_date(Sys.time(),"year")); 
> d$zone=NULL; d$zone=""; d
> >> 
> > If you're asking about a bug in R, you should provide a *minimal*
> > reproducible example (i.e. one without any package dependencies).
> > This has nothing to do with lubridate, so you can reproduce the
> > behavior with:
> 
> > d <- as.POSIXlt(Sys.time())
> > d$zone <- NULL
> > d$zone <- ""
> > d
> 
> [..]
> 
> >> Hope I'm not doing something illegal...
> >> 
> > You are.  You're changing the internal structure of a POSIXlt object
> > by re-ordering the list elements.  You should not expect a malformed
> > POSIXlt object to behave as if it's correctly formed.  You can see
> > it's malformed by comparing it's unclass()'d output.
> 
> > d <- as.POSIXlt(Sys.time())
> > unclass(d)  # valid POSIXlt object
> > d$zone <- NULL
> > d$zone <- ""
> > unclass(d)  # your malformed POSIXlt object
> 
> Indeed, really illegal, i.e. "against the law" ... ;-)
> 
> Thank you, Joshua!
> 
> Still, if R segfaults without the user explicitly
> calling .Call(), .Internal()  or similar -- as here --
> we usually acknowledge there *is* a bug in R .. even if it is
> only triggered by a users "illegal" messing around.
> 
> an MRE for the above, where I really only re-order the "internal" list:
> 
> d <- as.POSIXlt("2016-12-06"); dz <- d$zone; d$zone <- NULL; d$zone <- dz; f 
> <- format(d)
> 
> >  *** caught segfault ***
> > address 0x8020, cause 'memory not mapped'
> 
> > Traceback:
> >  1: format.POSIXlt(d)
> >  2: format(d)
> 
> The current code is "optimized for speed" (not perfectly), and
> a patch should hopefully address the C code.
> 
> Note that a smaller MRE -- which does *not* re-order, but just
> invalidate the time zone is
> 
>   d <- as.POSIXlt("2016-12-06"); d$zone <- 1; f <- format(d)
> 
> --
> 
> I have now committed a "minimal" patch (to the C code) which for
> the above two cases gives a sensible error rather than a
> seg.fault :
> 
>   > d <- as.POSIXlt("2016-12-06"); d$zone <- 1 ; f <- format(d)
>   Error in format.POSIXlt(d) : 
> invalid 'zone' component in "POSIXlt" structure
> 
>   > d <- as.POSIXlt("2016-12-06"); dz <- d$zone; d$zone <- NULL; d$zone <- 
> dz; f <- format(d)
>   Error in format.POSIXlt(d) : 
> invalid 'zone' component in "POSIXlt" structure
>   > 
> 
> I guess that it should still be possible to produce a segfault
> with invalid 'POSIXlt' structures though.
> 
> Martin
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] ok to segfault with POSIXlt zone=NULL zone=""?

[frederik]

Hi all,

Here's a more minimal version of my earlier bug report (thanks, Joshua
Ulrich):

d=as.POSIXlt(Sys.time()); d$zone=NULL; d$zone=""; d

I got some helpful, if glib, feedback from Joshua that the segfault
may be caused by the changing of the order of the list elements in 'd'
(representing the "internal structure" of the POSIXlt object).

He seems to think that it's OK for R to segfault - I was wondering if
someone else could lend a second opinion. My understanding is that we
should try to avoid segfaulting as a way of handling errors, if only
because they become much more difficult to debug when the R session is
forced to quit.

I don't know exactly which line is causing the bug, but looking at the
code for do_formatPOSIXlt in "src/main/datetime.c", it seems that
there would not be a huge performance penalty to add an extra sanity
check to prevent this from occurring.

Thank you,

Frederick

On Tue, Dec 06, 2016 at 04:37:20AM -0800, frede...@ofb.net wrote:
> Hi all,
> 
> I ran into a segfault while playing with dates.
> 
> $ R --no-init-file
> ...
> > library(lubridate); d=as.POSIXlt(floor_date(Sys.time(),"year")); 
> d$zone=NULL; d$zone=""; d
> 
> Attaching package: ‘lubridate’
> 
> The following object is masked from ‘package:base’:
> 
> date
> 
> Warning message:
> package ‘lubridate’ was built under R version 3.4.0 
> 
>  *** caught segfault ***
> address (nil), cause 'unknown'
> 
> Traceback:
>  1: format.POSIXlt(x, usetz = TRUE)
>  2: format(x, usetz = TRUE)
>  3: print(format(x, usetz = TRUE), ...)
>  4: print.POSIXlt(x)
>  5: function (x, ...) UseMethod("print")(x)
> 
> Possible actions:
> ...
> 
> Hope I'm not doing something illegal...
> 
> Thanks,
> 
> Frederick
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] segfault with POSIXlt zone=NULL zone=""

[frederik]

Hi Joshua,

Thank you for minimizing my test case.

> > Hope I'm not doing something illegal...
> >
> You are.  You're changing the internal structure of a POSIXlt object
> by re-ordering the list elements.  You should not expect a malformed
> POSIXlt object to behave as if it's correctly formed.  You can see
> it's malformed by comparing it's unclass()'d output.
> 
> d <- as.POSIXlt(Sys.time())
> unclass(d)  # valid POSIXlt object
> d$zone <- NULL
> d$zone <- ""
> unclass(d)  # your malformed POSIXlt object

I don't know if these questions are not already obvious, but:

1. Is there a reasonable way to fail more elegantly when a user makes
this mistake?

2. Should we update the documentation for POSIXlt to warn people that
the optional "zone" list element must precede the optional "gmtoff"
list element, in cases where both are present?

Thanks,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Spam messages

[frederik]

I agree that no action should be taken.

It's somewhat mystifying that the robot known as "Amy Kristen"
responds so quickly after my post, and with such regularity (so far
twice per hour), using perhaps several email addresses - and using the
correct "Reply-To" headers. But more mystifying is that she keeps the
same name the whole time. And lucky, I guess, because otherwise I
wouldn't know how to filter her out...

On Tue, Dec 06, 2016 at 10:02:11AM -0600, Marc Schwartz wrote:
> Hi,
> 
> This topic has come up previously, across the R e-mail lists and the spammers 
> need not be subscribers (but could be), but simply reasonably competent HTML 
> scrapers.
> 
> If you look at the online archives of the R lists, for example R-Devel for 
> this month:
> 
> https://stat.ethz.ch/pipermail/r-devel/2016-December/thread.html 
> 
> 
> and look at the individual posts, there is only minimal munging of e-mail 
> addresses, which is easily overcome with basic scripting.
> 
> This is further compounded by there being a multitude of other places on the 
> web, where actively updated archives of the R lists exist, providing 
> additional sources of e-mail addresses for spammers.
> 
> Some of the prior discussions had considered that we might preserve only the 
> list address in the posts sent out (as some groups do) and fully scrape the 
> sender's e-mail address from the post when distributed and archived. However, 
> that approach is not without it's own limitations (e.g. would make cc's and 
> reply-all largely useless, except for off-list discussion) and so no action 
> has been taken since this seems to be a transient issue.
> 
> Regards,
> 
> Marc Schwartz
> 
> 
> > On Dec 6, 2016, at 7:52 AM, Mario Emmenlauer  wrote:
> > 
> > 
> > The problem is not specific to this list. Any kind of public
> > list may mean that other subscribers (or even the whole world)
> > can see your email address. So whenever you mail to a (public)
> > list there is a good chance that afterwards, you will get more
> > spam. Not really much can be done about it, at least not on
> > the side of the list, since any of the subscribers may be a
> > spammer, who can know...
> > 
> > All the best,
> > 
> >Mario
> > 
> > 
> > 
> > On 06.12.2016 14:13, frede...@ofb.net wrote:
> >> Yes, I just heard from Amy Kristen who is "looking to meet new
> >> guys"...
> >> 
> >> On Fri, Dec 02, 2016 at 11:31:56AM -0800, Kenny Bell wrote:
> >>> Have others received spam messages after posting to this list?
> >>> 
> >>> The two problem emails are hodgesdonna...@yahoo.com and
> >>> amykristen4...@octbm.com.
> >>> 
> >>>   [[alternative HTML version deleted]]
> >>> 
> >>> __
> >>> R-devel@r-project.org mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>> 
> >> 
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >> 
> > 
> > 
> > 
> > Viele Gruesse,
> > 
> >Mario Emmenlauer
> > 
> > 
> > --
> > BioDataAnalysis GmbH, Mario Emmenlauer  Tel. Buero: +49-89-74677203
> > Balanstr. 43   mailto: memmenlauer * biodataanalysis.de
> > D-81669 München  http://www.biodataanalysis.de/
> > 
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Spam messages

[frederik]

> new package, R_spam

Oh, you got my hopes up!

On Tue, Dec 06, 2016 at 12:24:20PM -0200, Marcelo Perlin wrote:
> Perhpas amy needs help with her new package, R_spam,which is not working
> well with R_scam.  
> 
> Em 6 de dez de 2016 11:16,  escreveu:
> 
> Yes, I just heard from Amy Kristen who is "looking to meet new
> guys"...
> 
> On Fri, Dec 02, 2016 at 11:31:56AM -0800, Kenny Bell wrote:
> > Have others received spam messages after posting to this list?
> >
> > The two problem emails are hodgesdonna...@yahoo.com and
> > amykristen4...@octbm.com.
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Spam messages

[frederik]

Yes, I just heard from Amy Kristen who is "looking to meet new
guys"...

On Fri, Dec 02, 2016 at 11:31:56AM -0800, Kenny Bell wrote:
> Have others received spam messages after posting to this list?
> 
> The two problem emails are hodgesdonna...@yahoo.com and
> amykristen4...@octbm.com.
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] segfault with POSIXlt zone=NULL zone=""

[frederik]

Hi all,

I ran into a segfault while playing with dates.

$ R --no-init-file
...
> library(lubridate); d=as.POSIXlt(floor_date(Sys.time(),"year")); 
d$zone=NULL; d$zone=""; d

Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

date

Warning message:
package ‘lubridate’ was built under R version 3.4.0 

 *** caught segfault ***
address (nil), cause 'unknown'

Traceback:
 1: format.POSIXlt(x, usetz = TRUE)
 2: format(x, usetz = TRUE)
 3: print(format(x, usetz = TRUE), ...)
 4: print.POSIXlt(x)
 5: function (x, ...) UseMethod("print")(x)

Possible actions:
...

Hope I'm not doing something illegal...

Thanks,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] getGraphicsEvent() alternative for cairo graphics device?

[frederik]

Hi Paul,

OK I tried not to make the examples too fancy.

Please let me know what you think. They should probably be amended to
support the Windows platform, but I think that task would be much
easier for someone with access to Windows...

By the way I'm Cc'ing Mark O'Connell who shared with me some great
getGraphicsEvent examples - well, I found them useful, perhaps if
these are going to the R distro somewhere, then his examples should be
included as well.

Thank you,

Frederick

On Mon, Nov 14, 2016 at 01:51:08PM +1300, Paul Murrell wrote:
> 
> Great.  Thanks!
> 
> Paul
> 
> On 14/11/16 13:41, frede...@ofb.net wrote:
> > Hi Paul,
> > 
> > Thank you, for some reason I didn't seem to get an email notification
> > for your bugzilla comment!
> > 
> > I will try to send you something shortly.
> > 
> > Frederick
> > 
> > On Mon, Nov 14, 2016 at 08:55:20AM +1300, Paul Murrell wrote:
> > > Hi
> > > 
> > > The current status is that I am keen for people to contribute some testing
> > > code (see https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951)
> > > 
> > > There were also some getGraphicsEvent() changes/fixes suggested by Richard
> > > Bodewits (cc'ed), for which I am also seeking test code.
> > > 
> > > Paul
> > > 
> > > On 13/11/16 09:00, frede...@ofb.net wrote:
> > > > Hi Paul,
> > > > 
> > > > Just checking in to see what the status is.
> > > > 
> > > > From my perspective it seems logical to split off X11 into a separate
> > > > package, so development can proceed at a reasonable rate, but I
> > > > haven't yet tried to see if that's even possible.
> > > > 
> > > > Thanks,
> > > > 
> > > > Frederick
> > > > 
> > > > On Tue, Jul 26, 2016 at 09:23:35AM +1200, Paul Murrell wrote:
> > > > > Hi
> > > > > 
> > > > > Taking a look at those patches is now on my todo list, so I may be in 
> > > > > touch
> > > > > with both of you at some point to request some testing.
> > > > > 
> > > > > Paul
> > > > > 
> > > > > On 26/07/16 07:17, frede...@ofb.net wrote:
> > > > > > Dear Daniel Greenidge,
> > > > > > 
> > > > > > To enable getGraphicsEvent on Cairo, you have two patches to choose
> > > > > > from:
> > > > > > 
> > > > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=14364
> > > > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951
> > > > > > 
> > > > > > The second one is by me, and the first one is from five years ago by
> > > > > > Hugo Mildenberger.
> > > > > > 
> > > > > > Both patches are very simple, they move some lines enabling
> > > > > > getGrahpicsEvent outside of a if(!cairo) statement. My patch also 
> > > > > > adds
> > > > > > the ability to execute code (e.g. for animation) while the interface
> > > > > > is idle.
> > > > > > 
> > > > > > Top guy Duncan Murdoch has expressed that he doesn't have time to 
> > > > > > work
> > > > > > on applying these patches, and I haven't had any responses from the
> > > > > > rest of the R Core Team. I was thinking that perhaps your best bet 
> > > > > > is
> > > > > > to try to create a package called e.g. "X11-fixes" which people can
> > > > > > use to get a better X11 library (there is also a bug waiting to be
> > > > > > fixed from 2001:
> > > > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702).
> > > > > > 
> > > > > > I don't know if CRAN would accept such a package, or if you'd have 
> > > > > > to
> > > > > > distribute it via GitHub, but R has excellent tools to facilitate 
> > > > > > the
> > > > > > distribution of code via packages. Whether the R kernel exports 
> > > > > > enough
> > > > > > functions to allow a package to take over event handling, I'm not
> > > > > > sure. I was intending to look more into the details of this
> > > > > > possibility but haven't had time.
> > > > > > 
> > > > > > Best wishes,
> > > > > > 
> > > > > > Frederick
> > > > > > 
> > > > > > On Mon, Jul 25, 2016 at 02:15:59PM -0400, Daniel Greenidge wrote:
> > > > > > > Hi all,
> > > > > > > 
> > > > > > > I'm writing an interactive plotting function for viewing fMRI
> > > > > > > datasets. Currently, I get keypresses using
> > > > > > > grDevices::getGraphicsEvent().
> > > > > > > 
> > > > > > > Unfortunately getGraphicsEvent() only supports the 
> > > > > > > X11(type="Xlib")
> > > > > > > graphics device on Unix systems. The Xlib device doesn't support
> > > > > > > buffering (i.e. dev.hold() and dev.flush()), so redrawing the 
> > > > > > > plots
> > > > > > > causes lots of flickering.
> > > > > > > 
> > > > > > > Is there a way to get keypresses while using the cairo graphics
> > > > > > > device? Alternatively, is there a way to prevent flickering with 
> > > > > > > the
> > > > > > > Xlib graphics device?
> > > > > > > 
> > > > > > > Best,
> > > > > > > Daniel Greenidge
> > > > > > > 
> > > > > > > __
> > > > > > > R-devel@r-project.org mailing list
> > > > > > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > > > > > > 
> > > > > > 
> > > > > > 

Re: [Rd] getGraphicsEvent() alternative for cairo graphics device?

[frederik]

Hi Paul,

Thank you, for some reason I didn't seem to get an email notification
for your bugzilla comment!

I will try to send you something shortly.

Frederick

On Mon, Nov 14, 2016 at 08:55:20AM +1300, Paul Murrell wrote:
> Hi
> 
> The current status is that I am keen for people to contribute some testing
> code (see https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951)
> 
> There were also some getGraphicsEvent() changes/fixes suggested by Richard
> Bodewits (cc'ed), for which I am also seeking test code.
> 
> Paul
> 
> On 13/11/16 09:00, frede...@ofb.net wrote:
> > Hi Paul,
> > 
> > Just checking in to see what the status is.
> > 
> > From my perspective it seems logical to split off X11 into a separate
> > package, so development can proceed at a reasonable rate, but I
> > haven't yet tried to see if that's even possible.
> > 
> > Thanks,
> > 
> > Frederick
> > 
> > On Tue, Jul 26, 2016 at 09:23:35AM +1200, Paul Murrell wrote:
> > > Hi
> > > 
> > > Taking a look at those patches is now on my todo list, so I may be in 
> > > touch
> > > with both of you at some point to request some testing.
> > > 
> > > Paul
> > > 
> > > On 26/07/16 07:17, frede...@ofb.net wrote:
> > > > Dear Daniel Greenidge,
> > > > 
> > > > To enable getGraphicsEvent on Cairo, you have two patches to choose
> > > > from:
> > > > 
> > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=14364
> > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951
> > > > 
> > > > The second one is by me, and the first one is from five years ago by
> > > > Hugo Mildenberger.
> > > > 
> > > > Both patches are very simple, they move some lines enabling
> > > > getGrahpicsEvent outside of a if(!cairo) statement. My patch also adds
> > > > the ability to execute code (e.g. for animation) while the interface
> > > > is idle.
> > > > 
> > > > Top guy Duncan Murdoch has expressed that he doesn't have time to work
> > > > on applying these patches, and I haven't had any responses from the
> > > > rest of the R Core Team. I was thinking that perhaps your best bet is
> > > > to try to create a package called e.g. "X11-fixes" which people can
> > > > use to get a better X11 library (there is also a bug waiting to be
> > > > fixed from 2001:
> > > > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702).
> > > > 
> > > > I don't know if CRAN would accept such a package, or if you'd have to
> > > > distribute it via GitHub, but R has excellent tools to facilitate the
> > > > distribution of code via packages. Whether the R kernel exports enough
> > > > functions to allow a package to take over event handling, I'm not
> > > > sure. I was intending to look more into the details of this
> > > > possibility but haven't had time.
> > > > 
> > > > Best wishes,
> > > > 
> > > > Frederick
> > > > 
> > > > On Mon, Jul 25, 2016 at 02:15:59PM -0400, Daniel Greenidge wrote:
> > > > > Hi all,
> > > > > 
> > > > > I'm writing an interactive plotting function for viewing fMRI
> > > > > datasets. Currently, I get keypresses using
> > > > > grDevices::getGraphicsEvent().
> > > > > 
> > > > > Unfortunately getGraphicsEvent() only supports the X11(type="Xlib")
> > > > > graphics device on Unix systems. The Xlib device doesn't support
> > > > > buffering (i.e. dev.hold() and dev.flush()), so redrawing the plots
> > > > > causes lots of flickering.
> > > > > 
> > > > > Is there a way to get keypresses while using the cairo graphics
> > > > > device? Alternatively, is there a way to prevent flickering with the
> > > > > Xlib graphics device?
> > > > > 
> > > > > Best,
> > > > > Daniel Greenidge
> > > > > 
> > > > > __
> > > > > R-devel@r-project.org mailing list
> > > > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > > > > 
> > > > 
> > > > __
> > > > R-devel@r-project.org mailing list
> > > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > > > 
> > > 
> > > --
> > > Dr Paul Murrell
> > > Department of Statistics
> > > The University of Auckland
> > > Private Bag 92019
> > > Auckland
> > > New Zealand
> > > 64 9 3737599 x85392
> > > p...@stat.auckland.ac.nz
> > > http://www.stat.auckland.ac.nz/~paul/
> > > 
> 
> -- 
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> p...@stat.auckland.ac.nz
> http://www.stat.auckland.ac.nz/~paul/
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] getGraphicsEvent() alternative for cairo graphics device?

[frederik]

Hi Paul,

Just checking in to see what the status is.

>From my perspective it seems logical to split off X11 into a separate
package, so development can proceed at a reasonable rate, but I
haven't yet tried to see if that's even possible.

Thanks,

Frederick

On Tue, Jul 26, 2016 at 09:23:35AM +1200, Paul Murrell wrote:
> Hi
> 
> Taking a look at those patches is now on my todo list, so I may be in touch
> with both of you at some point to request some testing.
> 
> Paul
> 
> On 26/07/16 07:17, frede...@ofb.net wrote:
> > Dear Daniel Greenidge,
> > 
> > To enable getGraphicsEvent on Cairo, you have two patches to choose
> > from:
> > 
> > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=14364
> > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951
> > 
> > The second one is by me, and the first one is from five years ago by
> > Hugo Mildenberger.
> > 
> > Both patches are very simple, they move some lines enabling
> > getGrahpicsEvent outside of a if(!cairo) statement. My patch also adds
> > the ability to execute code (e.g. for animation) while the interface
> > is idle.
> > 
> > Top guy Duncan Murdoch has expressed that he doesn't have time to work
> > on applying these patches, and I haven't had any responses from the
> > rest of the R Core Team. I was thinking that perhaps your best bet is
> > to try to create a package called e.g. "X11-fixes" which people can
> > use to get a better X11 library (there is also a bug waiting to be
> > fixed from 2001:
> > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702).
> > 
> > I don't know if CRAN would accept such a package, or if you'd have to
> > distribute it via GitHub, but R has excellent tools to facilitate the
> > distribution of code via packages. Whether the R kernel exports enough
> > functions to allow a package to take over event handling, I'm not
> > sure. I was intending to look more into the details of this
> > possibility but haven't had time.
> > 
> > Best wishes,
> > 
> > Frederick
> > 
> > On Mon, Jul 25, 2016 at 02:15:59PM -0400, Daniel Greenidge wrote:
> > > Hi all,
> > > 
> > > I'm writing an interactive plotting function for viewing fMRI
> > > datasets. Currently, I get keypresses using
> > > grDevices::getGraphicsEvent().
> > > 
> > > Unfortunately getGraphicsEvent() only supports the X11(type="Xlib")
> > > graphics device on Unix systems. The Xlib device doesn't support
> > > buffering (i.e. dev.hold() and dev.flush()), so redrawing the plots
> > > causes lots of flickering.
> > > 
> > > Is there a way to get keypresses while using the cairo graphics
> > > device? Alternatively, is there a way to prevent flickering with the
> > > Xlib graphics device?
> > > 
> > > Best,
> > > Daniel Greenidge
> > > 
> > > __
> > > R-devel@r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > > 
> > 
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> > 
> 
> -- 
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> p...@stat.auckland.ac.nz
> http://www.stat.auckland.ac.nz/~paul/
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] On implementing zero-overhead code reuse

[frederik]

Hi Kynn,

Thanks for expanding.

I wrote a function like yours when I first started using R. It's
basically the same up to your "new.env()" line, I don't do anything
with environmentns. I just called my function "mysource" and it's
essentially a "source with path". That allows me to find code I reuse
in standard locations.

I don't know why R does not have built-in support for such a thing.
You can get it in C compilers with CPATH, and as you say in Perl with
PERL5LIB, in Python, etc. Obviously when I use my "mysource" I have to
remember that my code is now not portable without copying over some
files from other locations in my home directory. However, as a
beginner I find this tool to be indispensable, as R lacks several
functions which I use regularly, and I'm not necessarily ready to
confront the challenges associated with creating a package.

However, I guess since we can get your functionality pretty easily
using some lines in .Rprofile, that makes it seem less important to
have it built-in. In fact, if everyone has to implement their own
version of your "import", this almost guarantees that the function
won't appear by accident in any public code. My choice of name
"mysource" was meant to serve as a more visible lexical reminder that
the function is not meant to be seen by the public.

By the way, why do you do the stuff with environments in your "import"
function?

Dirk's take is interesting. I don't use version control for my
personal projects, just backing-up. Obviously not all R users are
interested in becoming package maintainers, in fact I think it would
clutter things a bit if this were the case. Or maybe it would be good
to have everyone publish their personal utility functions, who knows?
Anyway I appreciate Dirk's arguments, but I'm also a bit surprised
that Kynn and I seem to be the only ones who have written personal
functions to do what Kynn calls "zero-overhead code reuse". FWIW.

Cheers,

Frederick

On Sun, Oct 02, 2016 at 08:01:58PM -0400, Kynn Jones wrote:
> Hi Frederick,
> 
> I described what I meant in the post I sent to R-help
> (https://stat.ethz.ch/pipermail/r-help/2016-September/442174.html),
> but in brief, by "zero overhead" I mean that the only thing needed for
> library code to be accessible to client code is for it to be located
> in a designated directory.  No additional meta-files, packaging/compiling,
> etc. are required.
> 
> Best,
> 
> G.
> 
> On Sun, Oct 2, 2016 at 7:09 PM,   wrote:
> > Hi Kynn,
> >
> > Do you mind defining the term "zero-overhead model of code reuse"?
> >
> > I think I understand what you're getting at, but not sure.
> >
> > Thank you,
> >
> > Frederick
> >
> > On Sun, Oct 02, 2016 at 01:29:52PM -0400, Kynn Jones wrote:
> >> I'm looking for a way to approximate the "zero-overhead" model of code
> >> reuse available in languages like Python, Perl, etc.
> >>
> >> I've described this idea in more detail, and the motivation for this
> >> question in an earlier post to R-help
> >> (https://stat.ethz.ch/pipermail/r-help/2016-September/442174.html).
> >>
> >> (One of the responses I got advised that I post my question here instead.)
> >>
> >> The best I have so far is to configure my PROJ_R_LIB environment
> >> variable to point to the directory with my shared code, and put a
> >> function like the following in my .Rprofile file:
> >>
> >> import <- function(name){
> >> ## usage:
> >> ## import("foo")
> >> ## foo$bar()
> >> path <- file.path(Sys.getenv("PROJ_R_LIB"),paste0(name,".R"))
> >> if(!file.exists(path)) stop('file "',path,'" does not exist')
> >> mod <- new.env()
> >> source(path,local=mod)
> >> list2env(setNames(list(mod),list(name)),envir=parent.frame())
> >> invisible()
> >> }
> >>
> >> (NB: the idea above is an elaboration of the one I showed in my first 
> >> post.)
> >>
> >> But this is very much of an R noob's solution.  I figure there may
> >> already be more solid ways to achieve "zero-overhead" code reuse.
> >>
> >> I would appreciate any suggestions/critiques/pointers/comments.
> >>
> >> TIA!
> >>
> >> kj
> >>
> >> __
> >> R-devel@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
> 

On Sun, Oct 02, 2016 at 08:05:53PM -0400, Kynn Jones wrote:
> On Sun, Oct 2, 2016 at 8:01 PM, Kynn Jones  wrote:
> > Hi Frederick,
> >
> > I described what I meant in the post I sent to R-help
> > (https://stat.ethz.ch/pipermail/r-help/2016-September/442174.html),
> > but in brief, by "zero overhead" I mean that the only thing needed for
> > library code to be accessible to client code is for it to be located
> > in designed directory.  No additional meta-files, packaging/compiling,
>  
> 
> Sorry, I meant to write "designated".
> 
> > etc. are required.
> 

On Sun, Oct 02, 2016 at 07:18:41PM -0500, Dirk Eddelbuettel wrote:
> 
> Kynn,
> 
> How much 

Re: [Rd] On implementing zero-overhead code reuse

[frederik]

Hi Kynn,

Do you mind defining the term "zero-overhead model of code reuse"?

I think I understand what you're getting at, but not sure.

Thank you,

Frederick

On Sun, Oct 02, 2016 at 01:29:52PM -0400, Kynn Jones wrote:
> I'm looking for a way to approximate the "zero-overhead" model of code
> reuse available in languages like Python, Perl, etc.
> 
> I've described this idea in more detail, and the motivation for this
> question in an earlier post to R-help
> (https://stat.ethz.ch/pipermail/r-help/2016-September/442174.html).
> 
> (One of the responses I got advised that I post my question here instead.)
> 
> The best I have so far is to configure my PROJ_R_LIB environment
> variable to point to the directory with my shared code, and put a
> function like the following in my .Rprofile file:
> 
> import <- function(name){
> ## usage:
> ## import("foo")
> ## foo$bar()
> path <- file.path(Sys.getenv("PROJ_R_LIB"),paste0(name,".R"))
> if(!file.exists(path)) stop('file "',path,'" does not exist')
> mod <- new.env()
> source(path,local=mod)
> list2env(setNames(list(mod),list(name)),envir=parent.frame())
> invisible()
> }
> 
> (NB: the idea above is an elaboration of the one I showed in my first post.)
> 
> But this is very much of an R noob's solution.  I figure there may
> already be more solid ways to achieve "zero-overhead" code reuse.
> 
> I would appreciate any suggestions/critiques/pointers/comments.
> 
> TIA!
> 
> kj
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] vector finalizers

[frederik]

Dear R Devel,

In a thread this morning Luke Tierney mentioned that R's way of
garbage collecting is going to change soon in 3.4.0. I couldn't find
this info on Google but I wanted to share what I had been discussing
in another forum, in case now is not too late to raise considerations
which could affect the design of planned changes to R's garbage
collection facilities.

I ran into a problem when trying to get R to quickly load some vectors
from disk. R should be able to do this efficiently using memory
mapping. There is a package 'ff' which implements efficient loading of
disk-based vectors using memory mapping. It works pretty well, but the
problem is that it creates a separate data type - the vectors are not
"native" R vectors. There are some wrapper functions in a package
'ffbase' which allow people to use common functions like 'sum' on
these 'ff' vectors. However, a new wrapper has to be written for every
such function, and I guess the 'ffbase' authors do not have time to
write wrappers that are as efficient as the native R functions - in my
testing, there was a 10x slow-down for 'sum'.

The situation is a bit wistful because an 'ff' vector and a native R
vector are basically the same data type, they both store elements
contiguously in memory. Apparently, what prevents 'ffbase' and 'ff'
from creating native R vectors is the fact that it is impossible to
assign a "finalizer" to a native R vector. We need a finalizer so that
R can tell us when a vector is being freed, so we can unmap the
associated memory/file. Ffbase maintainer Edwin de Jonge was even
skeptical that CRAN would accept a package implementing the hack I had
proposed to simulate native R vectors from mmap'ed 'ff' vectors. The
issue is discussed here:

https://github.com/edwindj/ffbase/issues/52

Of course, weak references and external pointers allow finalizers to
be assigned to objects, but as I understand it, such objects are
separate types from vectors - there is no way in R to synthesize a
native vector endowed with a finalizer - something which could be
passed directly to built-in functions like 'sum'.

I think a finalizer facility for vectors would be useful because it
would allow us to take advantage of the memory mapping architecture
present in all modern processors, to do fast copy-free operations on
large disk-based data structures, without having to re-implement
internal functions like 'sum' which are essentially the same algorithm
no matter where the data is stored.

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] getGraphicsEvent() alternative for cairo graphics device?

[frederik]

Dear Daniel Greenidge,

To enable getGraphicsEvent on Cairo, you have two patches to choose
from:

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=14364
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951

The second one is by me, and the first one is from five years ago by
Hugo Mildenberger.

Both patches are very simple, they move some lines enabling
getGrahpicsEvent outside of a if(!cairo) statement. My patch also adds
the ability to execute code (e.g. for animation) while the interface
is idle.

Top guy Duncan Murdoch has expressed that he doesn't have time to work
on applying these patches, and I haven't had any responses from the
rest of the R Core Team. I was thinking that perhaps your best bet is
to try to create a package called e.g. "X11-fixes" which people can
use to get a better X11 library (there is also a bug waiting to be
fixed from 2001:
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16702).

I don't know if CRAN would accept such a package, or if you'd have to
distribute it via GitHub, but R has excellent tools to facilitate the
distribution of code via packages. Whether the R kernel exports enough
functions to allow a package to take over event handling, I'm not
sure. I was intending to look more into the details of this
possibility but haven't had time.

Best wishes,

Frederick

On Mon, Jul 25, 2016 at 02:15:59PM -0400, Daniel Greenidge wrote:
> Hi all,
> 
> I'm writing an interactive plotting function for viewing fMRI
> datasets. Currently, I get keypresses using
> grDevices::getGraphicsEvent().
> 
> Unfortunately getGraphicsEvent() only supports the X11(type="Xlib")
> graphics device on Unix systems. The Xlib device doesn't support
> buffering (i.e. dev.hold() and dev.flush()), so redrawing the plots
> causes lots of flickering.
> 
> Is there a way to get keypresses while using the cairo graphics
> device? Alternatively, is there a way to prevent flickering with the
> Xlib graphics device?
> 
> Best,
> Daniel Greenidge
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] getGraphicsEvent on X11 and event queuing

[frederik]

Hi R-Devel and Duncan,

Any ideas about my patch? I'm including Duncan Murdoch since he was
actually the one who made the "fix" to Hugo's bug in September 2010.
Duncan chose to "fix" Cairo support by formally disabling it, rather
than by enabling it as Hugo had proposed. I don't see a reply to
Hugo's January 2011 question, asking why this was done. Hugo's patch
was pretty simple:

if(!using cairo) {
xlib stuff
-set up for getGraphicsEvent
}
+set up for getGraphicsEvent

Sorry to be annoying, but I'm not sure how long I should wait for an
acknowledgment. Perhaps Duncan is waiting for someone else to take the
lead in this? I think it would be great if R maintainers could
delegate commit powers to more people so that time isn't wasted by
eager contributors duplicating each other's no-brainer patches which
were ignored many years ago. I guess there is a lot more interest in
Windows than Linux, but maybe that means that someone should be
assigned to handle Linux-specific changes who cares about them. (I see
a lot of Word-document, Windows-character-set, and RStudio stuff on
this list, so I sympathize with any disinterest there may be in Linux
Xlib problems)

Let me know if you'd like to divorce the "onIdle" part of my patch
from the Cairo-enabling part and discuss those separately, perhaps
that would make sense if time is short.

Thanks for handling my recent Readline patches.

Frederick

On Tue, Jun 14, 2016 at 08:34:13AM -0700, frede...@ofb.net wrote:
> Hi Martin,
> 
> > Also, as you are studying the C code and the issues anyway, it
> > may be worthwhile to consider (as small as possible!) patches to
> > the source you could also post to the bugzilla site if you
> > prefer; attaching as text/plain to R-devel does work fine, too.
> 
> Well, after your kind solicitation I went through all the touble of
> locally fixing getGraphicsEvent Cairo support and implementing an
> onIdle handler.
> 
> However, I noticed that the first part actually already has a patch on
> the R bug tracker, which is exactly the same as the fix I implemented!
> (minus documentation)
> 
> https://bugs.r-project.org/bugzilla/show_bug.cgi?id=14364
> 
> Problem: somebody ignored the patch and closed the bug. Who closed it?
> YOU!
> 
> So, I guess since that patch (fixing Cairo support) was sent in 2010,
> and my patch (fixing both problems) is about 8 times longer, I can
> expect you to ignore my patch until at least 2063 (a tight lower bound
> if you apply Hugo's patch this year). :)
> 
> I posted my patch here:
> 
> https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951
> 
> Frederick
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] getGraphicsEvent on X11 and event queuing

[frederik]

Hi Martin,

> Also, as you are studying the C code and the issues anyway, it
> may be worthwhile to consider (as small as possible!) patches to
> the source you could also post to the bugzilla site if you
> prefer; attaching as text/plain to R-devel does work fine, too.

Well, after your kind solicitation I went through all the touble of
locally fixing getGraphicsEvent Cairo support and implementing an
onIdle handler.

However, I noticed that the first part actually already has a patch on
the R bug tracker, which is exactly the same as the fix I implemented!
(minus documentation)

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=14364

Problem: somebody ignored the patch and closed the bug. Who closed it?
YOU!

So, I guess since that patch (fixing Cairo support) was sent in 2010,
and my patch (fixing both problems) is about 8 times longer, I can
expect you to ignore my patch until at least 2063 (a tight lower bound
if you apply Hugo's patch this year). :)

I posted my patch here:

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16951

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] getGraphicsEvent on X11 and event queuing; Makefile.in

[frederik]

Hi Martin,

Thanks for getting back about getGraphicsEvent.

I got an off-list reply from someone who echoed the annoyance with
lack of Cairo support, and also provided some interesting examples of
his own use of getGraphicsEvent. Thank you for the invitation to
contribute patches; I don't know what my upcoming priorities will be
like but it's good to know that you would consider using my code.

Speaking of which, I just did an 'svn update' and noticed that
Makefile.in has the same line that I asked about earlier:

@if test ! -f "$(srcdir)/doc/FAQ" || test -f non-tarball ; then \

But, my earlier suggestion to change this to $(srcdir)/non-tarball was
wrong because it doesn't seem to fix anything. In fact the
"non-tarball" exists in the build directory so it was OK to test for
it there. Sorry about that!

Apparently building in a separate directory, when a build has already
been started in the source directory, is still broken. I just checked
this with a clean "svn checkout". Even if I start a successful build
in a separate build directory, cancel it with SIGINT, start a build in
the source directory, cancel that, and then try to continue the
build-directory build, I get an error:

make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/doc/manual'
make[1]: Nothing to be done for 'front-matter'.
make[1]: Nothing to be done for 'html-non-svn'.
make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/doc/manual'
SVN-REVISION is unchanged
make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/m4'
make[1]: Nothing to be done for 'R'.
make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/m4'
make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/tools'
make[1]: Nothing to be done for 'R'.
make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/tools'
make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/doc'
/usr/bin/install: cannot stat '../../R-svn/doc/NEWS': No such file or 
directory
/usr/bin/install: cannot stat '../../R-svn/doc/NEWS.pdf': No such file or 
directory
Makefile:164: recipe for target 'svnonly' failed
make[1]: *** [svnonly] Error 1
make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/doc'
Makefile:60: recipe for target 'R' failed
make: *** [R] Error 1

I'm somewhat stumped... I had thought that removing doc/FAQ would fix
it, but it doesn't. I don't know what else could have been done by
'make' in the source directory which is confusing the build-directory
build.

Thanks,

Frederick

On Thu, Jun 09, 2016 at 04:52:26PM +0200, Martin Maechler wrote:
> Hi Frederik,
> 
> >>>>>   <frede...@ofb.net>
> >>>>> on Tue, 7 Jun 2016 15:20:05 -0700 writes:
> 
> > ... I just realized that setGraphicsEventHandlers or
> > getGraphicsEvent could have an 'onIdle' callback, to be
> > called somewhere in the polling loop of gevents.c:163 - I
> > think this would solve my problem #2 in a minimally
> > disruptive way.
> 
> I hope you will get some feedback about this by one of the
> graphics/devices experts from within R core.
> I'm not among them at all, but they may be busy or travelling
> currently.
> 
> Also, as you are studying the C code and the issues anyway, it
> may be worthwhile to consider (as small as possible!) patches to
> the source you could also post to the bugzilla site if you
> prefer; attaching as text/plain to R-devel does work fine, too.
> 
> Best regards,
> Martin
> 
> > On Mon, Jun 06, 2016 at 06:38:45PM -0700, frede...@ofb.net
> > wrote:
> >> Hi R-Devel,
> >> 
> >> I've been working on an oscilloscope project using an
> >> Arduino microcontroller board. I found that it's quite
> >> easy to get realtime updates, e.g. 30+ frames per second,
> >> if I read data from the board in a little Rcpp library. I
> >> have to use dev.hold() and dev.flush() to keep the plot
> >> from flickering, which restricts me to the "cairo" X11
> >> device.
> >> 
> >> I'd like to be able to add interactivity to the
> >> oscilloscope display, for instance to bind a key to save
> >> the current plot to a file, or to bind keys for adjusting
> >> the time scale etc.
> >> 
> >> However, I ran into two problems:
> >> 
> >> (1) setGraphicsEventHandlers only works on the "Xlib" X11
> >> device, which doesn't support buffering via dev.hold() -
> >> it flickers.
> >> 
> >> (2) getGraphicsEvent and friends lack some interface
> >> features which are needed to use the fu

Re: [Rd] getGraphicsEvent on X11 and event queuing

[frederik]

... I just realized that setGraphicsEventHandlers or getGraphicsEvent
could have an 'onIdle' callback, to be called somewhere in the polling
loop of gevents.c:163 - I think this would solve my problem #2 in a
minimally disruptive way.

On Mon, Jun 06, 2016 at 06:38:45PM -0700, frede...@ofb.net wrote:
> Hi R-Devel,
> 
> I've been working on an oscilloscope project using an Arduino
> microcontroller board. I found that it's quite easy to get realtime
> updates, e.g. 30+ frames per second, if I read data from the board in
> a little Rcpp library. I have to use dev.hold() and dev.flush() to
> keep the plot from flickering, which restricts me to the "cairo" X11
> device.
> 
> I'd like to be able to add interactivity to the oscilloscope display,
> for instance to bind a key to save the current plot to a file, or to
> bind keys for adjusting the time scale etc.
> 
> However, I ran into two problems:
> 
> (1) setGraphicsEventHandlers only works on the "Xlib" X11 device,
> which doesn't support buffering via dev.hold() - it flickers.
> 
> (2) getGraphicsEvent and friends lack some interface features which
> are needed to use the functions in an asynchronous fashion. Typically,
> event listener library functions have a "timeout" parameter, and the
> ability to return already-queued events. But getGraphicsEvent() has
> neither - it waits indefinitely (you can't set a timeout), and it
> seems to ignore events which occurred before it was called (I can't
> figure out why, from the code, I guess the normal R event processing
> grabs events which occur between calls to getGraphicsEvent?).
> 
> It seems like it should be possible to set a handler for keyboard
> events and have it execute in between plot updates when the user
> presses a key - yet without blocking further updates if no key has
> been pressed.
> 
> Is anyone interested in fixing (1) and (2)? Or is there some other
> library or workaround to solve my problems?
> 
> Here is some code I used to play around with these functions:
> 
> X11(type="Xlib");
> 
> keydown = function(key) {
>   cat("Got key: ",key);
> lastkey <<- key
> }
> 
> setGraphicsEventHandlers(onKeybd = keydown);
> 
> plot(c(0,0));
> getGraphicsEvent();
> 
> Well, I think it's great that getGraphicsEvent exists at all, and want
> to salute Duncan Murdoch who is listed as the author. I hope I may
> have helped by describing some new possible uses for these functions.
> 
> Thank you,
> 
> Frederick
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] getGraphicsEvent on X11 and event queuing

[frederik]

Hi R-Devel,

I've been working on an oscilloscope project using an Arduino
microcontroller board. I found that it's quite easy to get realtime
updates, e.g. 30+ frames per second, if I read data from the board in
a little Rcpp library. I have to use dev.hold() and dev.flush() to
keep the plot from flickering, which restricts me to the "cairo" X11
device.

I'd like to be able to add interactivity to the oscilloscope display,
for instance to bind a key to save the current plot to a file, or to
bind keys for adjusting the time scale etc.

However, I ran into two problems:

(1) setGraphicsEventHandlers only works on the "Xlib" X11 device,
which doesn't support buffering via dev.hold() - it flickers.

(2) getGraphicsEvent and friends lack some interface features which
are needed to use the functions in an asynchronous fashion. Typically,
event listener library functions have a "timeout" parameter, and the
ability to return already-queued events. But getGraphicsEvent() has
neither - it waits indefinitely (you can't set a timeout), and it
seems to ignore events which occurred before it was called (I can't
figure out why, from the code, I guess the normal R event processing
grabs events which occur between calls to getGraphicsEvent?).

It seems like it should be possible to set a handler for keyboard
events and have it execute in between plot updates when the user
presses a key - yet without blocking further updates if no key has
been pressed.

Is anyone interested in fixing (1) and (2)? Or is there some other
library or workaround to solve my problems?

Here is some code I used to play around with these functions:

X11(type="Xlib");

keydown = function(key) {
  cat("Got key: ",key);
lastkey <<- key
}

setGraphicsEventHandlers(onKeybd = keydown);

plot(c(0,0));
getGraphicsEvent();

Well, I think it's great that getGraphicsEvent exists at all, and want
to salute Duncan Murdoch who is listed as the author. I hope I may
have helped by describing some new possible uses for these functions.

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] configure / make problems with R-devel

[frederik]

Hi Martin,

Thanks for the Makefile clue. The file 'non-tarball' was present in
the source directory, but not in the build directory (should the
Makefile be checking for 'non-tarball' in the source directory
instead?). However, 'doc/FAQ' was present in the source directory, so
the first clause of '||' failed.

svnonly:
@if test ! -f "$(srcdir)/doc/FAQ" || test -f non-tarball ; then \
...

It works if I remove 'doc/FAQ'. Incidentally, this file is created by
'make' but not removed by 'make clean'. So the problem is that I
accidentally ran 'make' in the source directory before taking your
advice. I ran 'make clean' out of habit, not even realizing that
'make' in the source directory would break your
separate-build-directory technique.

STRIP=true is highly counterintuitive, but I'm surprised you don't
recognise it - it just replaces the 'strip' command with 'true' so
that my installed binaries get to keep their debugging symbols. Maybe
there is a better way of doing it - I guess you use the binaries in
place so perhaps that is one solution. I'm not sure where I came up
with STRIP=true.




In case anyone skimmed, I think there may be two "action items" here:

1. in Makefile, change

@if test ! -f "$(srcdir)/doc/FAQ" || test -f non-tarball ; then \

to

@if test ! -f "$(srcdir)/doc/FAQ" || test -f 
"$(srcdir)/non-tarball" ; then \

(I haven't tested it)

2. Figure out how to remove doc/FAQ with 'make clean' (if it makes
sense to do so)



Thanks,

Frederick

On Wed, May 25, 2016 at 12:46:53PM +0200, Martin Maechler wrote:
> >>>>>   <frede...@ofb.net>
> >>>>> on Tue, 24 May 2016 15:15:17 -0700 writes:
> 
> > Thank you, Martin. I linked to your message in a comment here so maybe
> > other people will know about that useful technique:
> 
> > http://singmann.org/installing-r-devel-on-linux/#comment-161
> 
> > However, when I try it, I get an error:
> 
> > $ make 
> > make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/m4'
> > make[1]: Nothing to be done for 'R'.
> > make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/m4'
> > make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/tools'
> > make[1]: Nothing to be done for 'R'.
> > make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/tools'
> > make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/doc'
> > /usr/bin/install: cannot stat '../../R-svn/doc/NEWS': No such file or 
> directory
> > /usr/bin/install: cannot stat '../../R-svn/doc/NEWS.pdf': No such file 
> or directory
> > Makefile:164: recipe for target 'svnonly' failed
> > make[1]: *** [svnonly] Error 1
> > make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/doc'
> > Makefile:60: recipe for target 'R' failed
> > make: *** [R] Error 1
> 
> This is strange:  Did you accidentally delete the 'non-tarball'
> file in your build directory which should have been created by
> 'configure' ?
> 
> I ask because your 'make' seems to be using the 'else' clause in
> the 'svnonly' target in the R-svn-build/Makefile
> but it should really use the first branch which does install
> things in ./doc/ (such as NEWS or NEWS.pdf).
> 
> I have never used  'STRIP=true' -- maybe that did remove the
> 'non-tarball' file ?
> 
> Why not rather do it the way I told you, i.e., *with* recommended
> packages, and no arguments to 'configure'
> (if that does work, you may try variants.. I agree that
> --without-recommended-packages should work as well, I just never
> use that).
> 
> Martin
> 
> 
> > I configured like this:
> 
> > $ cd ../R-svn-build/  
> > $ ../R-svn/configure --without-recommended-packages 
> --prefix=$HOME/r-svn-test STRIP=true
> 
> > I guess I can try to debug it myself but thought I should report back
> > to you. It works when I 'configure' and 'make' in the source
> > directory.
> 
> > Cheers,
> 
> > Frederick
> 
> 
> 
> > On Tue, May 24, 2016 at 07:20:18PM +0200, Martin Maechler wrote:
> >> >>>>> Keith O'Hara <keith.oh...@nyu.edu>
> >> >>>>> on Tue, 24 May 2016 12:47:43 -0400 writes:
> >> 
> >> > svn checkout https://svn.r-project.org/R/trunk/ 
> >> 
> >> yes, indeed. thank you, Keith.
> >> 
> >> and from then on only
> >> 
> >> cd 
> >> svn up
> >> 
> >> (which is short for 'sv

Re: [Rd] Latest R-devel build failing on OS X

[frederik]

Thank you, Martin. I linked to your message in a comment here so maybe
other people will know about that useful technique:

http://singmann.org/installing-r-devel-on-linux/#comment-161

However, when I try it, I get an error:

$ make 
make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/m4'
make[1]: Nothing to be done for 'R'.
make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/m4'
make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/tools'
make[1]: Nothing to be done for 'R'.
make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/tools'
make[1]: Entering directory '/home/frederik/pkg-tmp/R-svn-build/doc'
/usr/bin/install: cannot stat '../../R-svn/doc/NEWS': No such file or 
directory
/usr/bin/install: cannot stat '../../R-svn/doc/NEWS.pdf': No such file or 
directory
Makefile:164: recipe for target 'svnonly' failed
make[1]: *** [svnonly] Error 1
make[1]: Leaving directory '/home/frederik/pkg-tmp/R-svn-build/doc'
Makefile:60: recipe for target 'R' failed
make: *** [R] Error 1

I configured like this:

$ cd ../R-svn-build/  
$ ../R-svn/configure --without-recommended-packages 
--prefix=$HOME/r-svn-test STRIP=true

I guess I can try to debug it myself but thought I should report back
to you. It works when I 'configure' and 'make' in the source
directory.

Cheers,

Frederick



On Tue, May 24, 2016 at 07:20:18PM +0200, Martin Maechler wrote:
> >>>>> Keith O'Hara <keith.oh...@nyu.edu>
> >>>>> on Tue, 24 May 2016 12:47:43 -0400 writes:
> 
> > svn checkout https://svn.r-project.org/R/trunk/ 
> 
> yes, indeed. thank you, Keith.
> 
> and from then on only
> 
> cd 
> svn up
> 
> (which is short for 'svn update').
> 
> Another hint: Then do *not* build in the source directory but in
> what we called a "build directory"; i.e., something like
> (from scratch; including the only-once needed "checkout") :
> 
>  svn checkout https://svn.r-project.org/R/trunk/ R
>  cd R
>  tools/rsync-recommended
>  mkdir ../build-R
>  cd ../build-R
>  ../R/configure
>  make
>  make check
> 
> and I then never run 'make install', but rather use symbolic
> link from
>  /build-R/bin/R  to something like  ~/bin/R-devel 
> i.e.,
>  cd ~/bin
>  ln -s /build-R/bin/R R-devel
> 
> Martin
> 
> >> On May 24, 2016, at 12:45 PM, frede...@ofb.net wrote:
> >> 
> >> I agree with Martin's summary of the situation, and with the updated
> >> NEWS entry.
> >> 
> >> I'm not familiar with Subversion, can you tell me the command to use?
> >> 
> >> (I tried "svn co https://svn.r-project.org/R/; but it seems to be
> >> downloading all branches)
> >> 
> >> Frederick
> >> 
> >> On Tue, May 24, 2016 at 04:30:11PM +0200, Martin Maechler wrote:
> >>>>>>>> peter dalgaard <pda...@gmail.com>
> >>>>>>>> on Tue, 24 May 2016 13:47:27 +0200 writes:
> >>> 
> >>>> I had a regression in config.site so the nightly build didn't. 
> Retrying
> >>>> Looks like it will build, but the ctl-R, ctl-C bug is still present 
> on OSX (w/Simon's libs). This _was_ fixed for a while, was it not?
> >>> 
> >>> I thought it was never fixed, for readline versions 5.x (or all
> >>> of readline_version < 6.3 ?) because the patch assumed features
> >>> not available, e.g., for Frederik (who got compilation errors
> >>> which I think you confirmed on pre-6 readline).
> >>> 
> >>> I remember you having two different readlines installed on OSX
> >>> but the standard Mac binary (from CRAN, i.e. Simon) would use
> >>> the old readline version ?
> >>> 
> >>> so that whole  resetReadline() solution is now conditionalized inside
> >>> 
> >>> #if defined(RL_READLINE_VERSION) && RL_READLINE_VERSION >= 0x0603
> >>> ...
> >>> ...
> >>> #endif
> >>> 
> >>> and hence the previous code (which is buggy) is used for
> >>> readline versions < 6.3.
> >>> As a consequence the bug is only fixed for readline >= 6.3,
> >>> because the current patch did not compile and hence seemed not
> >>> appropriate for  readline < 6.3 (and hence the above 
> conditionaliz

Re: [Rd] Latest R-devel build failing on OS X

[frederik]

I agree with Martin's summary of the situation, and with the updated
NEWS entry.

I'm not familiar with Subversion, can you tell me the command to use?

(I tried "svn co https://svn.r-project.org/R/; but it seems to be
downloading all branches)

Frederick

On Tue, May 24, 2016 at 04:30:11PM +0200, Martin Maechler wrote:
> >>>>> peter dalgaard <pda...@gmail.com>
> >>>>> on Tue, 24 May 2016 13:47:27 +0200 writes:
> 
> > I had a regression in config.site so the nightly build didn't. 
> Retrying
> > Looks like it will build, but the ctl-R, ctl-C bug is still present on 
> OSX (w/Simon's libs). This _was_ fixed for a while, was it not?
> 
> I thought it was never fixed, for readline versions 5.x (or all
> of readline_version < 6.3 ?) because the patch assumed features
> not available, e.g., for Frederik (who got compilation errors
> which I think you confirmed on pre-6 readline).
> 
> I remember you having two different readlines installed on OSX
> but the standard Mac binary (from CRAN, i.e. Simon) would use
> the old readline version ?
> 
> so that whole  resetReadline() solution is now conditionalized inside
> 
> #if defined(RL_READLINE_VERSION) && RL_READLINE_VERSION >= 0x0603
>...
>...
> #endif
> 
> and hence the previous code (which is buggy) is used for
> readline versions < 6.3.
> As a consequence the bug is only fixed for readline >= 6.3,
> because the current patch did not compile and hence seemed not
> appropriate for  readline < 6.3 (and hence the above conditionalization).
> 
> 
> > (The NEWS entry is also wrong: The issue existed before readline 6.3)
> 
> Aah.. you are right. The API change with 6.3 was for the other, the
> "SIGWINCH" bug.
> 
> Here's a an update proposal for that NEWS entry :
> 
>  • The API for readline libraries >= 6.3 had changed such
>terminal window resizes where no longer properly signalled
>(PR#16604).  Also, ‘Ctrl C’ in incremental search behaved
>confusingly in R (unix) consoles (PR#16603) also for older
>readline versions. These have been fixed (for readline >=
>6.3 only), thanks to patches by Frederick Eaton.
> 
> 
> Martin
> 
> > -pd
> 
> > On 24 May 2016, at 12:55 , Martin Maechler <maech...@stat.math.ethz.ch> 
> wrote:
> 
> >> 
> >> Can you (Frederick, Peter, Keith, but ideally others, too)
> >> confirm that you don't see any problems anymore, when building a
> >> version of R-devel from sources that are newer 
> >> than (or equal to)  svn revision 70632 (2016-05-19 10:59:51, see 
> below)?
> >> 
> >> I'm asking because the question is open if these should be
> >> "back ported" to R 3.3.0 patched or not.
> >> 
> >> Best regards,
> >> Martin
> >> 
> >>>>>>> Martin Maechler <maech...@stat.math.ethz.ch>
> >>>>>>> on Thu, 19 May 2016 11:02:48 +0200 writes:
> >> 
> >>>>>>> <frede...@ofb.net>
> >>>>>>> on Wed, 18 May 2016 15:03:31 -0700 writes:
> >> 
> >>>>> Readline <= 6.2 shouldn't require the SIGWINCH patch, so
> >>>>> if older versions have trouble finding rl_resize_terminal
> >>>>> then you could wrap a macro around that part.
> >> 
> >>>> I find python related patches that use
> >> 
> >>>> #ifdef HAVE_RL_RESIZE_TERMINAL
> >> 
> >>>> so they must have configured for that.  We could and
> >>>> probably should do the same, but as a Linux_only guy
> >>>> currently (even basically only one flavor of Linux), I'd
> >>>> appreciate others to produce code for that.
> >> 
> >>> Actually that was easy (in hindsight.. I took too long!)
> >>> enough, so I've now committed
> >> 
> >>> 
> 
> >>> r70632 | maechler | 2016-05-19 10:59:51 +0200 (Thu, 19 May 2016) | 1 
> line
> >>> Changed paths:
> >>> M configure
> >>> M configure.ac
> >>> M src/include/config.h.in
> >>> M src/unix/sys-std.c
> >> 
> >>> check for rl_resize_terminal() now
> >>> 
> 
> >> 
> >>> ... and Keith should even not see the warning anymore
> >>> (nor Peter the error, when compiling using readline 5.x instead of 
> 6.[23]).
> >> 
> >> 
> >> [...]
> 
> > -- 
> > Peter Dalgaard, Professor,
> > Center for Statistics, Copenhagen Business School
> > Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> > Phone: (+45)38153501
> > Office: A 4.23
> > Email: pd@cbs.dk  Priv: pda...@gmail.com
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Latest R-devel build failing on OS X

[frederik]

Readline <= 6.2 shouldn't require the SIGWINCH patch, so if older
versions have trouble finding rl_resize_terminal then you could wrap a
macro around that part.

The isearch C-c bug has existed forever, according to Chet Ramey. Yes,
I had to retrain myself to use C-g to exit isearch but it's confusing.
It would be nice to fix this C-c bug on older versions too, but my
solution used some global variables and I'm not sure which Readline
version they date from.

Out of curiosity, did you delete the HAVE_READLINE_READLINE_H block?

Thanks so much for working this out!

Frederick

P.S. I do use Emacs too, but I have some bindings which stuff the
current region into a numbered 'GNU screen' window (where R is
running) - I was never a fan of inferior Emacs processes. Also a
keybinding which stuffs "source('CURRENT_FILE')\n". And a rxvt-unicode
'url-select' extension which copies text after a "> " prompt, to get
commands I edited back into Emacs. That's why I still use the basic
interface.

On Wed, May 18, 2016 at 10:58:59PM +0200, peter dalgaard wrote:
> Spoke too soon, both systems now build, but neither has the original bugs 
> fixed
> 
> (Incidentally, I realized why the ctl-R...ctl-C bug never bit me: The emacs 
> habit is to exit isearch with ctl-G and that works flawlessly.) 
> 
> -pd
> 
> > On 18 May 2016, at 22:40 , peter dalgaard  wrote:
> > 
> > Ah, got it. For some ancient reason config.site on that machine had 
> > 
> > LDFLAGS=-L/usr/X11R6/lib
> > 
> > in config.site, and that prevented configure from inserting -L 
> > /usr/local/lib, so linked /usr/lib/libreadline.dylib, which is the 
> > Apple-supplied one, which possibly is really libedit...
> > 
> > -p
> > 
> > 
> >> On 18 May 2016, at 22:01 , peter dalgaard  wrote:
> >> 
> >> gcc   -L/usr/X11R6/lib -o R.bin Rmain.o CommandLineArgs.o Rdynload.o 
> >> Renviron.o RNG.o agrep.o apply.o arithmetic.o array.o attrib.o bind.o 
> >> builtin.o character.o coerce.o colors.o complex.o connections.o context.o 
> >> cum.o dcf.o datetime.o debug.o deparse.o devices.o dotcode.o dounzip.o 
> >> dstruct.o duplicate.o edit.o engine.o envir.o errors.o eval.o format.o 
> >> gevents.o gram.o gram-ex.o graphics.o grep.o identical.o inlined.o 
> >> inspect.o internet.o iosupport.o lapack.o list.o localecharset.o logic.o 
> >> main.o mapply.o match.o memory.o names.o objects.o options.o paste.o 
> >> platform.o plot.o plot3d.o plotmath.o print.o printarray.o printvector.o 
> >> printutils.o qsort.o radixsort.o random.o raw.o registration.o relop.o 
> >> rlocale.o saveload.o scan.o seq.o serialize.o sort.o source.o split.o 
> >> sprintf.o startup.o subassign.o subscript.o subset.o summary.o sysutils.o 
> >> times.o unique.o util.o version.o g_alab_her.o g_cntrlify.o g_fontdb.o 
> >> g_her_glyph.o xxxpr.o   `ls ../unix/*.o ../appl/*.o ../nmath/!
 *.o` ../extra/tre/libtre.a ../extra/intl/libintl.a ../extra/tzone/libtz.a 
-L../../lib/x86_64 -lRblas -L/usr/lib/gcc/i686-apple-darwin11/4.2.1/x86_64 
-L/usr/lib/gcc/i686-apple-darwin11/4.2.1 -L/usr/lib -lgfortran
-Wl,-framework -Wl,CoreFoundation -lreadline  -lpcre -llzma -lbz2 -lz -licucore 
-lm -llzma -liconv
> >> Undefined symbols for architecture x86_64:
> >> "_rl_resize_terminal", referenced from:
> >> _Rstd_ReadConsole in sys-std.o
> >> ld: symbol(s) not found for architecture x86_64
> >> clang: error: linker command failed with exit code 1 (use -v to see 
> >> invocation)
> >> make[3]: *** [R.bin] Error 1
> >> make[2]: *** [R] Error 2
> >> make[1]: *** [R] Error 1
> >> make: *** [R] Error 1
> >> 
> >> and the MBAir still builds... 
> >> 
> >> Confused!,
> >> 
> >> -pd
> > 
> > -- 
> > Peter Dalgaard, Professor,
> > Center for Statistics, Copenhagen Business School
> > Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> > Phone: (+45)38153501
> > Office: A 4.23
> > Email: pd@cbs.dk  Priv: pda...@gmail.com
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> 
> -- 
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
> 
> 
> 
> 
> 
> 
> 
> 
> 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] please apply my patches

[frederik]

Thank you, Martin. Sorry if I sounded impatient.

> Sure (it was me, not Peter, who mentioned ESS etc).

Sorry, Peter!

> Be assured that we will deal with this before the next release of R

Thank you, that works for me.

Frederick

On Tue, May 17, 2016 at 10:57:28AM +0200, Martin Maechler wrote:
> >   
> > on Mon, 16 May 2016 18:40:50 -0700 writes:
> 
> > Dear R Devel, Sorry for sending so many emails, but I
> > haven't heard that anyone is working on the patches I
> > created.
> 
> > A few weeks ago I wanted to do something to contribute to
> > R because I use it a lot. I picked the most annoying bugs
> > I could find, and created two of the simplest patches I
> > could create,
> 
> > but all I've had in response from the R core
> > team (aside from briefly banning my Bugzilla account) is a
> > lengthy explanation for why power users don't use the
> > basic command line interface.
> 
> Well, I feel slightly irritated by this picture of what
> happened, as the very first sentence I wrote started with
> "Thank you" ... and I did mean it.
> Also, I apologize if you+ only got critique from what I wrote.
> 
> > I'm grateful to Peter for the list of tools like ESS that
> > people use to interface with R; but if I wanted to submit
> > a bug to ESS then I would have sent it to a different mailing list.
> 
> If you reread what I said (or tried to say): I gave you a reason
> why I (personally) was waiting/reluctant applying your patches:
> I would not notice if I introduced a new bug, etc... and I also told
> you that I thought that (almost) everybody else from the core team would
> be in the same situation: all just reasons, Peter and I had hoped somebody
> else would step up and confirm and apply the bug fixes.
> 
> Be assured that we will deal with this before the next release of R,
> (R 3.3.1, which has not yet been scheduled). I'm sorry if this
> was not clear from what we said, we did imply it.
> 
> We also have other obligations, not the least teaching,
> students, and we have a life off screen-&-keyboard, so do have
> tad more patience, please.
> 
> Best regards,
> Martin
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] please apply my patches

[frederik]

Dear R Devel,

Sorry for sending so many emails, but I haven't heard that anyone is
working on the patches I created.

A few weeks ago I wanted to do something to contribute to R because I
use it a lot. I picked the most annoying bugs I could find, and
created two of the simplest patches I could create, but all I've had
in response from the R core team (aside from briefly banning my
Bugzilla account) is a lengthy explanation for why power users don't
use the basic command line interface. I'm grateful to Peter for the
list of tools like ESS that people use to interface with R; but if I
wanted to submit a bug to ESS then I would have sent it to a different
mailing list.

Thank you Göran for confirming that you experience these bugs. I've
Cc'ed two other users, Chris Smowton and "Terran", who commented on
the Bugzilla bugs. Do you guys want to weigh in on the relative
importance of getting these fixed?

Nobody asked me to describe the patches in more detail, but I'll give
a brief summary in case it helps: One of my patches adds a signal
handler for SIGWINCH (which GNU Readline stopped handling a few years
ago without telling anyone). There is no other use of SIGWINCH in the
R code so I don't think this will cause problems. This makes
terminal-resizing possible. The other one adds some additional cleanup
to popReadline. Now, popReadline is called every time a command is
entered at the prompt, or when R catches SIGINT, to reset the terminal
settings and take control back from Readline. My second patch fixes
incremental-search by amending popReadline to clear the line state as
well. I invite creative suggestions as to how these might lead to
unforeseen trouble for Unix or non-Unix users - what I might have
missed in my testing, etc.

Otherwise, I would appreciate some kind of real input as to what I
should do or who I should contact to get this rolling. We're all short
on time; if we can get this taken care of quickly and efficiently then
we'll have more time left for other stuff.

Thank you,

Frederick


On Fri, May 13, 2016 at 10:02:54AM +0200, Göran Broström wrote:
> Dear Frederik,
> 
> I can confirm that especially the first issue on your list (resizing the
> terminal) has "bothered me" badly for some time now: I'm on Ubuntu
> (usually).
> 
> Göran Broström
> 
> 
> On 2016-05-13 00:23, frede...@ofb.net wrote:
> > Hi Peter, Martin, and others,
> > 
> > Thanks for your replies.
> > 
> > - The bugs apply to all systems that use GNU Readline, not just Linux
> >or Arch Linux.
> > 
> > - Readline version 6.3 changed the signal handling so that SIGWINCH is
> >no longer handled automatically by the library. This means it's not
> >currently possible for people using R on e.g. Linux to resize the
> >terminal, or at least when they do so they have to make sure that
> >all their commands fit in one line and don't wrap.
> > 
> > - There is also a long-standing bug in Readline where the callback
> >interface didn't properly clear the line on SIGINT (^C). This means
> >that "exiting" reverse-incremental-search with ^C would give an
> >apparently empty prompt which still had some pending input, so if
> >you hit ^C-Return then an unintended command would get executed.
> > 
> > If they're not "bothering all that many people", then perhaps it's
> > because everyone uses Windows or Mac OS X or RStudio. For me these are
> > pretty significant bugs. The second one causes unintended code to be
> > executed. Random code could delete files, for example, or worse. The
> > first one bites me every time I want to change the size of a window,
> > which is pretty often.
> > 
> > I tried to get Readline maintainer Chet Ramey to fix these on the
> > Readline side, but he disagreed with my proposal:
> > 
> > https://lists.gnu.org/archive/html/bug-readline/2016-04/threads.html
> > 
> > I'm glad that my message here at least was seen and I hope that
> > someone who uses the R command line on Linux will have time to verify
> > that the patches work correctly. They are basically Chet-approved
> > workarounds for bugs/changes in Readline, not very complicated.
> > 
> > Do either of you know a Linux R person you could ping to get these
> > patches checked out? I'm not overly frustrated, and I'm not in a major
> > hurry, but from what we've observed it seems like waiting for someone
> > concerned to come along and finally read Bugzilla or the R-Devel
> > archives is not going to result in a very dense Poisson process...
> > 
> > Thanks,
> > 
> > Frederick Eaton
> > 
> > On Thu, May 12, 2016 at 03:42:59PM +0200, peter dalgaard wrote:
> > > 
> > >

Re: [Rd] where to send patches to R source code

[frederik]

Hi Peter, Martin, and others,

Thanks for your replies.

- The bugs apply to all systems that use GNU Readline, not just Linux
  or Arch Linux.

- Readline version 6.3 changed the signal handling so that SIGWINCH is
  no longer handled automatically by the library. This means it's not
  currently possible for people using R on e.g. Linux to resize the
  terminal, or at least when they do so they have to make sure that
  all their commands fit in one line and don't wrap.

- There is also a long-standing bug in Readline where the callback
  interface didn't properly clear the line on SIGINT (^C). This means
  that "exiting" reverse-incremental-search with ^C would give an
  apparently empty prompt which still had some pending input, so if
  you hit ^C-Return then an unintended command would get executed.

If they're not "bothering all that many people", then perhaps it's
because everyone uses Windows or Mac OS X or RStudio. For me these are
pretty significant bugs. The second one causes unintended code to be
executed. Random code could delete files, for example, or worse. The
first one bites me every time I want to change the size of a window,
which is pretty often.

I tried to get Readline maintainer Chet Ramey to fix these on the
Readline side, but he disagreed with my proposal:

https://lists.gnu.org/archive/html/bug-readline/2016-04/threads.html

I'm glad that my message here at least was seen and I hope that
someone who uses the R command line on Linux will have time to verify
that the patches work correctly. They are basically Chet-approved
workarounds for bugs/changes in Readline, not very complicated.

Do either of you know a Linux R person you could ping to get these
patches checked out? I'm not overly frustrated, and I'm not in a major
hurry, but from what we've observed it seems like waiting for someone
concerned to come along and finally read Bugzilla or the R-Devel
archives is not going to result in a very dense Poisson process...

Thanks,

Frederick Eaton

On Thu, May 12, 2016 at 03:42:59PM +0200, peter dalgaard wrote:
> 
> > On 12 May 2016, at 10:03 , Martin Maechler  
> > wrote:
> > 
> >>  
> >>on Wed, 11 May 2016 23:00:20 -0700 writes:
> > 
> >> Dear R Developers,
> >> I wrote to this list a week ago with some patches that fix bugs in R's
> >> GNU Readline interface, but I haven't had a reply. I'm not in a hurry
> >> but I'd like to make sure that my message is getting read by the right
> >> people. Should I be sending my patches somewhere else?
> > 
> > Thank you Frederick for your reports and patches.
> > You did send them to the correct place, https://bugs.r-project.org/
> > 
> > Sometimes (as here) a combination of circumstances do lead to
> > nobody picking them up quickly.
> > In this case,
> > 
> > - probably none of R-core use or even have easy access to Arch Linux
> >  so we cannot easily veryify that there is a bug at all
> >  nor -consequently- veryify that your patch does fix the bug.
> 
> Actually, the bugs look like they should apply fairly generally, just maybe 
> not bothering all that many people. But there could be portability issues 
> with the fixes, so I suspect some of us were waiting for "a readline expert" 
> to check them out.
> 
> -pd
> 
> BTW: Anyone with a fix for the stuck-at-eol issue? (aaabbbccc) 
> 
> > 
> > - no other user has confirmed the bug on his/her platform, so
> >  there did not seem a huge demand...
> > 
> > - Accidentally many in R core may be busy with other bugs, teaching, .
> >  and just lack the extra resources to delve into these problems
> >  at the current moment.
> > 
> > Hence, there was not even an 'Acknowledged' change to your
> > reports--indeed as nobody had been able to see there is a problem
> > existing outside of your personal computer.
> > 
> > I agree that this must seem a bit frustrating to you.
> > 
> > --
> > Martin
> > 
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> 
> -- 
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
> 
> 
> 
> 
> 
> 
> 
> 
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] where to send patches to R source code

[frederik]

Dear R Developers,

I wrote to this list a week ago with some patches that fix bugs in R's
GNU Readline interface, but I haven't had a reply. I'm not in a hurry
but I'd like to make sure that my message is getting read by the right
people. Should I be sending my patches somewhere else?

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] unix readline signal handling patches

[frederik]

Hello R Developers,

I posted some patches yesterday to your bug tracking system but I'm
not sure if I should have notified the mailing list as well. I haven't
gotten any responses yet.

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16603

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16604

Thank you,

Frederick

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] problem submitting R bug; bug plotting in tiling window manager

[frederik]

That was an amusing bug.

I wish I had time to help more directly with this project, which has
been very helpful to me.

Frederick

On Thu, Feb 25, 2016 at 08:04:40PM +0100, peter dalgaard wrote:
> In case you still care, see 
> 
> https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16726
> 
> which even our very human spam detector hasn't decided to label as spam (yet).
> 
> -pd
> 
> > On 08 Feb 2016, at 18:34 , frede...@ofb.net wrote:
> > 
> > Ah, thank you for that explanation. I somehow didn't catch that my
> > Bugzilla account had been disabled by a human.
> > 
> > "Common pattern is to post ... something copied from a generic bug
> > report" - that sounds very annoying.
> > 
> > Frederick
> > 
> > On Sun, Feb 07, 2016 at 11:54:11AM +0100, peter dalgaard wrote:
> >> Unfortunately, the spammers in question appear to be human (of sorts). 
> >> 
> >> We're not sure what they're up to, but a common pattern is to post random 
> >> text, or something copied from a generic bug report (like "able to add 6 
> >> item"), later followed by a comment containing a link or a file 
> >> attachment. 
> >> 
> >> Presumably, it is some sort of click-bait scheme, but it could also be a 
> >> covert channel for contrabande files. At any rate, it is very hard to 
> >> distinguish by mechanical means. So it is done by eye, with some risk of 
> >> Type-I error. Thus, the Bugzilla maintainers are pretty vigilant to stamp 
> >> out spammers, sometimes edging on being ham-fisted (er, -footed?).
> >> 
> >> -pd
> >> 
> >> 
> >>> On 07 Feb 2016, at 00:25 , frede...@ofb.net wrote:
> >>> 
> >>> No problem.
> >>> 
> >>> Another suggestion would be to simply validate user input like most
> >>> websites, and reject invalid submissions immediately, rather than
> >>> blocking the user's account. I don't know what kind of spambots you
> >>> are up against, but unless they are very intelligent I doubt they'll
> >>> be able to understand a message like "You submitted a bug with no body
> >>> text, please enter something and try again." There may also be the
> >>> option of using Captchas.
> >>> 
> >>> Not sure how hard it is to get Bugzilla to do these things.
> >>> 
> >>> Thanks,
> >>> 
> >>> Frederick
> >>> 
> >>> P.S. (I now see that all errors on the bug tracker are displayed with
> >>> a red background)
> >>> 
> >>> On Sat, Feb 06, 2016 at 03:00:21AM -0500, Duncan Murdoch wrote:
> >>>> Thanks for the suggestions.
> >>>> 
> >>>> Duncan Murdoch
> >>>> 
> >>>> On 05/02/2016 10:07 PM, frede...@ofb.net wrote:
> >>>>> Hi Duncan Murdoch,
> >>>>> 
> >>>>> Thanks for your time. I apologize for not telling you that my email
> >>>>> address on the bug tracker is slightly different -
> >>>>> "frederik-rproj...@ofb.net" vs "frede...@ofb.net". I was going to
> >>>>> follow up with this information, but then I thought, he probably knows
> >>>>> how to find a tagged email address.
> >>>>> 
> >>>>> I do hope that you are able to fix the bug tracker. In particular,
> >>>>> people should be made aware that their account is blocked before being
> >>>>> invited to submit a bug. The error they get should be less rude - no
> >>>>> need to make it red - and the email address in the error should be
> >>>>> filled in. You complained about wasting time having to look for my
> >>>>> email address - well, I wasted time looking for yours. The error
> >>>>> message could even hint at what triggered the ban. I don't think that
> >>>>> you're going to get very far by trying to scare off actual spammers
> >>>>> with a big red accusation - I imagine they all have pretty thick skin.
> >>>>> 
> >>>>> Reading the first line of my bug report was generous of you, but if
> >>>>> you read the rest, you'll see that, indeed, after checking with the
> >>>>> knowledgeable i3 guys, it appears to be an R bug. So I would like to
> >>>>> submit it. What appears at the top of my bug report is a copy of the
> >>>>> original bug I posted to i3, at the linked URL (are 

Re: [Rd] problem submitting R bug; bug plotting in tiling window manager

[frederik]

Ah, thank you for that explanation. I somehow didn't catch that my
Bugzilla account had been disabled by a human.

"Common pattern is to post ... something copied from a generic bug
report" - that sounds very annoying.

Frederick

On Sun, Feb 07, 2016 at 11:54:11AM +0100, peter dalgaard wrote:
> Unfortunately, the spammers in question appear to be human (of sorts). 
> 
> We're not sure what they're up to, but a common pattern is to post random 
> text, or something copied from a generic bug report (like "able to add 6 
> item"), later followed by a comment containing a link or a file attachment. 
> 
> Presumably, it is some sort of click-bait scheme, but it could also be a 
> covert channel for contrabande files. At any rate, it is very hard to 
> distinguish by mechanical means. So it is done by eye, with some risk of 
> Type-I error. Thus, the Bugzilla maintainers are pretty vigilant to stamp out 
> spammers, sometimes edging on being ham-fisted (er, -footed?).
> 
> -pd
> 
> 
> > On 07 Feb 2016, at 00:25 , frede...@ofb.net wrote:
> > 
> > No problem.
> > 
> > Another suggestion would be to simply validate user input like most
> > websites, and reject invalid submissions immediately, rather than
> > blocking the user's account. I don't know what kind of spambots you
> > are up against, but unless they are very intelligent I doubt they'll
> > be able to understand a message like "You submitted a bug with no body
> > text, please enter something and try again." There may also be the
> > option of using Captchas.
> > 
> > Not sure how hard it is to get Bugzilla to do these things.
> > 
> > Thanks,
> > 
> > Frederick
> > 
> > P.S. (I now see that all errors on the bug tracker are displayed with
> > a red background)
> > 
> > On Sat, Feb 06, 2016 at 03:00:21AM -0500, Duncan Murdoch wrote:
> >> Thanks for the suggestions.
> >> 
> >> Duncan Murdoch
> >> 
> >> On 05/02/2016 10:07 PM, frede...@ofb.net wrote:
> >>> Hi Duncan Murdoch,
> >>> 
> >>> Thanks for your time. I apologize for not telling you that my email
> >>> address on the bug tracker is slightly different -
> >>> "frederik-rproj...@ofb.net" vs "frede...@ofb.net". I was going to
> >>> follow up with this information, but then I thought, he probably knows
> >>> how to find a tagged email address.
> >>> 
> >>> I do hope that you are able to fix the bug tracker. In particular,
> >>> people should be made aware that their account is blocked before being
> >>> invited to submit a bug. The error they get should be less rude - no
> >>> need to make it red - and the email address in the error should be
> >>> filled in. You complained about wasting time having to look for my
> >>> email address - well, I wasted time looking for yours. The error
> >>> message could even hint at what triggered the ban. I don't think that
> >>> you're going to get very far by trying to scare off actual spammers
> >>> with a big red accusation - I imagine they all have pretty thick skin.
> >>> 
> >>> Reading the first line of my bug report was generous of you, but if
> >>> you read the rest, you'll see that, indeed, after checking with the
> >>> knowledgeable i3 guys, it appears to be an R bug. So I would like to
> >>> submit it. What appears at the top of my bug report is a copy of the
> >>> original bug I posted to i3, at the linked URL (are links OK or will I
> >>> get banned again?).
> >>> 
> >>> The reason a bug appeared with the subject "til" is because I noticed
> >>> that when typing into the subject field, some "related bugs" come up.
> >>> However, this "suggestion" process appeared to be stalled when I typed
> >>> "til" (for "tiling" or "tilable"). I tried hitting "enter" and it
> >>> ended up opening a bug with that subject, which I never submitted,
> >>> because I clicked "back" and figured out that *four* characters are
> >>> actually necesary to start getting suggestions. The whole point of
> >>> doing this was to see if another bug had been submitted with the same
> >>> topic, and thereby save you time! I'm not going to try to reproduce
> >>> this error, because you said it will get me banned again, but I think
> >>> somebody should try to fix the site so that people don't get banned
&

Re: [Rd] problem submitting R bug; bug plotting in tiling window manager

[frederik]

No problem.

Another suggestion would be to simply validate user input like most
websites, and reject invalid submissions immediately, rather than
blocking the user's account. I don't know what kind of spambots you
are up against, but unless they are very intelligent I doubt they'll
be able to understand a message like "You submitted a bug with no body
text, please enter something and try again." There may also be the
option of using Captchas.

Not sure how hard it is to get Bugzilla to do these things.

Thanks,

Frederick

P.S. (I now see that all errors on the bug tracker are displayed with
a red background)

On Sat, Feb 06, 2016 at 03:00:21AM -0500, Duncan Murdoch wrote:
> Thanks for the suggestions.
> 
> Duncan Murdoch
> 
> On 05/02/2016 10:07 PM, frede...@ofb.net wrote:
> >Hi Duncan Murdoch,
> >
> >Thanks for your time. I apologize for not telling you that my email
> >address on the bug tracker is slightly different -
> >"frederik-rproj...@ofb.net" vs "frede...@ofb.net". I was going to
> >follow up with this information, but then I thought, he probably knows
> >how to find a tagged email address.
> >
> >I do hope that you are able to fix the bug tracker. In particular,
> >people should be made aware that their account is blocked before being
> >invited to submit a bug. The error they get should be less rude - no
> >need to make it red - and the email address in the error should be
> >filled in. You complained about wasting time having to look for my
> >email address - well, I wasted time looking for yours. The error
> >message could even hint at what triggered the ban. I don't think that
> >you're going to get very far by trying to scare off actual spammers
> >with a big red accusation - I imagine they all have pretty thick skin.
> >
> >Reading the first line of my bug report was generous of you, but if
> >you read the rest, you'll see that, indeed, after checking with the
> >knowledgeable i3 guys, it appears to be an R bug. So I would like to
> >submit it. What appears at the top of my bug report is a copy of the
> >original bug I posted to i3, at the linked URL (are links OK or will I
> >get banned again?).
> >
> >The reason a bug appeared with the subject "til" is because I noticed
> >that when typing into the subject field, some "related bugs" come up.
> >However, this "suggestion" process appeared to be stalled when I typed
> >"til" (for "tiling" or "tilable"). I tried hitting "enter" and it
> >ended up opening a bug with that subject, which I never submitted,
> >because I clicked "back" and figured out that *four* characters are
> >actually necesary to start getting suggestions. The whole point of
> >doing this was to see if another bug had been submitted with the same
> >topic, and thereby save you time! I'm not going to try to reproduce
> >this error, because you said it will get me banned again, but I think
> >somebody should try to fix the site so that people don't get banned
> >for any content which is not submitted. Especially people with
> >months-old accounts, like me.
> >
> >I definitely sympathize with the spam problem, and thank you for your
> >hard work. Best wishes,
> >
> >Frederick
> >
> >On Fri, Feb 05, 2016 at 08:19:40PM -0500, Duncan Murdoch wrote:
> >>On 05/02/2016 7:26 PM, frede...@ofb.net wrote:
> >>>Dear Dirk Eddelbuettel and Duncan Murdoch,
> >>>
> >>>Thank you for your work on the wonderful R project!
> >>>
> >>>I recently attempted to submit a bug with your Bugzilla interface:
> >>>
> >>>https://bugs.r-project.org/bugzilla/enter_bug.cgi
> >>>
> >>>I created an account, typed in all my information, first checking
> >>>details with another project. Then I clicked submit, and was taken to
> >>>a web page with a big red banner, it said
> >>>
> >>> Spammer
> >>> If you believe your account should be restored, please send email to 
> >>> explaining why.
> >>>
> >>>What a hostile thing to say to your users! I tried resubmitting my
> >>>bug, but removing any links, and I still get the message - so it looks
> >>>like my account has really been blocked. Please do something to warn
> >>>your users about this so they can avoid the upset.
> >>
> >>Your account isn't blocked now, but it wasn't easy to unblock it: you used a
> >>different email address in the submission, not the same one you used in this
> >