Re: [R-pkg-devel] [Learning] the secret of Win[dows C-backed packages]

2016-08-18 Thread Jim Hester 
It has been a few months since I was doing this for the 4.9.3 windows
toolchain, but the best way I found IIRC of getting everything working was
to install msys2 and use that as the build environment, but put the proper
Rtools toolchain first in the PATH.

On Mon, Aug 15, 2016 at 9:13 AM, Hadley Wickham  wrote:

> One other option is to include the src for libmagic inside your
> package, and use that as a fallback if it's not installed on the
> system. That gives linux users what they want (linking to the system
> package), and windows users what they want (it just works). It only
> works for simple libraries, but it looks like libmagic might qualify.
>
> I could have sworn I did that in a package, but I can't remember which one
> :(
>
> Hadley
>
> On Sat, Aug 13, 2016 at 11:34 AM, Bob Rudis  wrote:
> > Hey folks,
> >
> > I usually stare in awe at the C-backed packages that rely on eternal
> > libraries which are super-easy to get working on macOS & *nix _but_ that
> > also work perfectly on Windows. I fire up Windows (*maybe*) once a month
> to
> > test some of my packages but I'm curious as to what I need to do to
> setup a
> > more extended toolchain to create libraries such as are found in
> > https://github.com/rwinlib.
> >
> > Jeroen (I think others, too) seem to effortlessly package those up and I
> > fully grok his "anticonf" setup and the 32-bit & 64-bit Windows library
> > setup. I just can't seem to actually do the building.
> >
> > Why I'm asking what I'm asking (below) is that I'm working on something
> for
> > the $DAYJOB now - https://github.com/hrbrmstr/wand - that requires
> libmagic
> > and I kinda, unfortunately, actually need it to also work on Windows (it
> > usually doesn't matter, but this time some folks who use Windows systems
> at
> > work need to use this forthcoming package).
> >
> > Rather than b[eu]g Jeroen or others for library building/packaging, I'd
> be
> > very, keenly interested in learning what else besides Rtools I need to
> > install to be able to make similar binary bundles to include during
> > installation (and to have my compiled pkgs work on Windows).
> >
> > My disdain for Windows and my frustration with the super-limited
> toolchain
> > components installed with Rtools is probably contributing to my inability
> > to solve this on my own (no bash.exe...rly?). I used to use cygwin back
> in
> > the day but there are so many cautions in the main R docs abt compatible
> > versions that I'd rather not spend hours or days figuring out something
> > that others already have.
> >
> > One example here is that I actually managed to get one of the libmagic
> > support packages (the regex library) built and installed on Windows (in
> the
> > toolchain /usr/local) but can't get libmagic's `configure` script to
> > find/use it.
> >
> > I'm more than willing to write up any copious pointers folks provide
> into a
> > detailed, attributed blog post or R Journal article since this is
> seriously
> > the only bit holding me back from working on and releasing a ton of other
> > compilation-required packages to CRAN. I also know I'm not the only one
> who
> > gets stalled when trying to get similar things (especially those with
> full
> > Boost dependencies) to work on Windows.
> >
> > thx,
> >
> > -Bob
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-package-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-package-devel
>
>
>
> --
> http://hadley.nz
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] [Learning] the secret of Win[dows C-backed packages]

2016-08-15 Thread Hadley Wickham 
One other option is to include the src for libmagic inside your
package, and use that as a fallback if it's not installed on the
system. That gives linux users what they want (linking to the system
package), and windows users what they want (it just works). It only
works for simple libraries, but it looks like libmagic might qualify.

I could have sworn I did that in a package, but I can't remember which one :(

Hadley

On Sat, Aug 13, 2016 at 11:34 AM, Bob Rudis  wrote:
> Hey folks,
>
> I usually stare in awe at the C-backed packages that rely on eternal
> libraries which are super-easy to get working on macOS & *nix _but_ that
> also work perfectly on Windows. I fire up Windows (*maybe*) once a month to
> test some of my packages but I'm curious as to what I need to do to setup a
> more extended toolchain to create libraries such as are found in
> https://github.com/rwinlib.
>
> Jeroen (I think others, too) seem to effortlessly package those up and I
> fully grok his "anticonf" setup and the 32-bit & 64-bit Windows library
> setup. I just can't seem to actually do the building.
>
> Why I'm asking what I'm asking (below) is that I'm working on something for
> the $DAYJOB now - https://github.com/hrbrmstr/wand - that requires libmagic
> and I kinda, unfortunately, actually need it to also work on Windows (it
> usually doesn't matter, but this time some folks who use Windows systems at
> work need to use this forthcoming package).
>
> Rather than b[eu]g Jeroen or others for library building/packaging, I'd be
> very, keenly interested in learning what else besides Rtools I need to
> install to be able to make similar binary bundles to include during
> installation (and to have my compiled pkgs work on Windows).
>
> My disdain for Windows and my frustration with the super-limited toolchain
> components installed with Rtools is probably contributing to my inability
> to solve this on my own (no bash.exe...rly?). I used to use cygwin back in
> the day but there are so many cautions in the main R docs abt compatible
> versions that I'd rather not spend hours or days figuring out something
> that others already have.
>
> One example here is that I actually managed to get one of the libmagic
> support packages (the regex library) built and installed on Windows (in the
> toolchain /usr/local) but can't get libmagic's `configure` script to
> find/use it.
>
> I'm more than willing to write up any copious pointers folks provide into a
> detailed, attributed blog post or R Journal article since this is seriously
> the only bit holding me back from working on and releasing a ton of other
> compilation-required packages to CRAN. I also know I'm not the only one who
> gets stalled when trying to get similar things (especially those with full
> Boost dependencies) to work on Windows.
>
> thx,
>
> -Bob
>
> [[alternative HTML version deleted]]
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel



-- 
http://hadley.nz

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] [Learning] the secret of Win[dows C-backed packages]

2016-08-13 Thread Dirk Eddelbuettel 

On 13 August 2016 at 18:19, Bob Rudis wrote:
| Aye. I rly need to get back to my security & privacy "R" post. The
| slipstreaming in of these binaries is somewhat frightening. Almost as
| frightening as being stuck on Windows 😀

Right.

And the Windows side of that so-called "anticonf" [1] pattern more-or-less
establishes itself as "best practice" -- which may be defensible realizing
[2] that Windows is a fairly terrible development environment lacking any
kind of modularity or packages [3].  But then we all need to wake up, realize
it is 2016 and we may be providing binaries for maybe up to a millions users
on that platform.  By what you call "slipstreaming". Seriously?

Dirk

[1] I personally dislike that name a lot, even though I generally enjoy puns.
[2] See my earlier post in this thread.
[3] At the system level; I am unfamiliar with nu-get but add-ons failed before

| On Sat, Aug 13, 2016 at 13:09 Dirk Eddelbuettel  wrote:
| 
| 
| I don't think there is a good "generally applicable" solution.  We have a
| bunch of hacks. Below are three quickly picked examples from packages of
| mine:
| 
| RQuantLib relies on environment variables and a 'library that needs to 
just
| be there' on the build host:
| 
|    ## The environment variable QUANTLIB_ROOT has to point to an existing
| build of QuantLib
|    ## With R 2.12.0 and later, we also support 32 and 64 bit builds and
| need to differentiate
|    PKG_CPPFLAGS=-I$(QUANTLIB_ROOT) -I../inst/include -I. -I$(BOOSTLIB)
|    PKG_CXXFLAGS=$(SHLIB_OPENMP_CFLAGS) -fpermissive
|    PKG_LIBS=-L$(QUANTLIB_ROOT)/lib${R_ARCH} -lQuantLib $
| (SHLIB_OPENMP_CFLAGS)
| 
| RcppGSL does the same
| 
|    ## This assumes that the LIB_GSL variable points to working GSL
| libraries
|    PKG_CPPFLAGS=-I$(LIB_GSL)/include -I../inst/include
|    PKG_LIBS=-L$(LIB_GSL)/lib -lgsl -lgslcblas
| 
| Rblpapi copies a library (for which we have 'free' binaries but no source)
| in
| from GitHub:
| 
|    ## target to ensure tar.gz files get unpacked
|    ## the opening '@' ensures operations are executed 'quietly'
|    ## in order to see commands as they happens add a 'v' to the tar and cp
| commands
|    ## curl has '-k' flag to suppress certificate warnings
|    blpLibrary:
|                 @if [ ! -d ../inst ]; then mkdir -p ../inst; fi
|                 @if [ ! -d ../blp/win/${FLV} ]; then mkdir -p ../blp/win/$
| {FLV}; fi
|                 @if [ ! -f ../blp/win/${FLV}/blpHeaders.tar.gz ]; then 
curl
| -s -k -L -O https://github.com/Rblp/blp/raw/master/headers/windows/
| blpHeaders.tar.gz; mv blpHeaders.tar.gz ../blp/win/${FLV}; tar xfz ../blp/
| win/${FLV}/blpHeaders.tar.gz -C ../inst; fi
|                 @if [ ! -f ../blp/win/${FLV}/blpLibrary.tar.gz ]; then 
curl
| -s -k -L -O https://github.com/Rblp/blp/raw/master/win${WIN}/
| blpLibrary.tar.gz; mv blpLibrary.tar.gz ../blp/win/${FLV}; tar xfz ../blp/
| win/${FLV}/blpLibrary.tar.gz; fi
|                 @if [ ! -d ${FLV} ]; then mkdir -p ${FLV}; fi
|                 @cp blpapi3_${WIN}.dll ${FLV}
|                 @if [ ! -d ../inst/libs/${FLV} ]; then mkdir -p ../inst/
| libs/${FLV}; fi
|                 @cp blpapi3_${WIN}.dll ../inst/libs/${FLV}
| 
| 
| None of that really generalizes well.  The last bit, much like the
| so-called
| anticonf pattern, is more or less a non-standard practice exposing a
| security
| risk by copying in code from a semi-random location which could be 
hijacked
| or man-in-the-middle'd.
| 
| Really all this is just pretending that Windows is an envrionment on which
| you can deploy proper build systems -- similar to what we are used from 
the
| likes of dpkg/apt/yum or brew.  But in all honesty you just can't.
| 
| So I would not try to aim for something overly general.  It is Windows
| after all.
| 
| Dirk
| 
| --
| http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
| 

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] [Learning] the secret of Win[dows C-backed packages]

2016-08-13 Thread Bob Rudis 
Aye. I rly need to get back to my security & privacy "R" post. The
slipstreaming in of these binaries is somewhat frightening. Almost as
frightening as being stuck on Windows 😀
On Sat, Aug 13, 2016 at 13:09 Dirk Eddelbuettel  wrote:

>
> I don't think there is a good "generally applicable" solution.  We have a
> bunch of hacks. Below are three quickly picked examples from packages of
> mine:
>
> RQuantLib relies on environment variables and a 'library that needs to just
> be there' on the build host:
>
>## The environment variable QUANTLIB_ROOT has to point to an existing
> build of QuantLib
>## With R 2.12.0 and later, we also support 32 and 64 bit builds and
> need to differentiate
>PKG_CPPFLAGS=-I$(QUANTLIB_ROOT) -I../inst/include -I. -I$(BOOSTLIB)
>PKG_CXXFLAGS=$(SHLIB_OPENMP_CFLAGS) -fpermissive
>PKG_LIBS=-L$(QUANTLIB_ROOT)/lib${R_ARCH} -lQuantLib
> $(SHLIB_OPENMP_CFLAGS)
>
> RcppGSL does the same
>
>## This assumes that the LIB_GSL variable points to working GSL
> libraries
>PKG_CPPFLAGS=-I$(LIB_GSL)/include -I../inst/include
>PKG_LIBS=-L$(LIB_GSL)/lib -lgsl -lgslcblas
>
> Rblpapi copies a library (for which we have 'free' binaries but no source)
> in
> from GitHub:
>
>## target to ensure tar.gz files get unpacked
>## the opening '@' ensures operations are executed 'quietly'
>## in order to see commands as they happens add a 'v' to the tar and cp
> commands
>## curl has '-k' flag to suppress certificate warnings
>blpLibrary:
> @if [ ! -d ../inst ]; then mkdir -p ../inst; fi
> @if [ ! -d ../blp/win/${FLV} ]; then mkdir -p
> ../blp/win/${FLV}; fi
> @if [ ! -f ../blp/win/${FLV}/blpHeaders.tar.gz ]; then
> curl -s -k -L -O
> https://github.com/Rblp/blp/raw/master/headers/windows/blpHeaders.tar.gz;
> mv blpHeaders.tar.gz ../blp/win/${FLV}; tar xfz
> ../blp/win/${FLV}/blpHeaders.tar.gz -C ../inst; fi
> @if [ ! -f ../blp/win/${FLV}/blpLibrary.tar.gz ]; then
> curl -s -k -L -O
> https://github.com/Rblp/blp/raw/master/win${WIN}/blpLibrary.tar.gz; mv
> blpLibrary.tar.gz ../blp/win/${FLV}; tar xfz
> ../blp/win/${FLV}/blpLibrary.tar.gz; fi
> @if [ ! -d ${FLV} ]; then mkdir -p ${FLV}; fi
> @cp blpapi3_${WIN}.dll ${FLV}
> @if [ ! -d ../inst/libs/${FLV} ]; then mkdir -p
> ../inst/libs/${FLV}; fi
> @cp blpapi3_${WIN}.dll ../inst/libs/${FLV}
>
>
> None of that really generalizes well.  The last bit, much like the
> so-called
> anticonf pattern, is more or less a non-standard practice exposing a
> security
> risk by copying in code from a semi-random location which could be hijacked
> or man-in-the-middle'd.
>
> Really all this is just pretending that Windows is an envrionment on which
> you can deploy proper build systems -- similar to what we are used from the
> likes of dpkg/apt/yum or brew.  But in all honesty you just can't.
>
> So I would not try to aim for something overly general.  It is Windows
> after all.
>
> Dirk
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] [Learning] the secret of Win[dows C-backed packages]

2016-08-13 Thread Dirk Eddelbuettel 

I don't think there is a good "generally applicable" solution.  We have a
bunch of hacks. Below are three quickly picked examples from packages of mine:

RQuantLib relies on environment variables and a 'library that needs to just
be there' on the build host:

   ## The environment variable QUANTLIB_ROOT has to point to an existing build 
of QuantLib
   ## With R 2.12.0 and later, we also support 32 and 64 bit builds and need to 
differentiate
   PKG_CPPFLAGS=-I$(QUANTLIB_ROOT) -I../inst/include -I. -I$(BOOSTLIB) 
   PKG_CXXFLAGS=$(SHLIB_OPENMP_CFLAGS) -fpermissive 
   PKG_LIBS=-L$(QUANTLIB_ROOT)/lib${R_ARCH} -lQuantLib $(SHLIB_OPENMP_CFLAGS) 

RcppGSL does the same

   ## This assumes that the LIB_GSL variable points to working GSL libraries
   PKG_CPPFLAGS=-I$(LIB_GSL)/include -I../inst/include
   PKG_LIBS=-L$(LIB_GSL)/lib -lgsl -lgslcblas 

Rblpapi copies a library (for which we have 'free' binaries but no source) in
from GitHub:

   ## target to ensure tar.gz files get unpacked
   ## the opening '@' ensures operations are executed 'quietly'
   ## in order to see commands as they happens add a 'v' to the tar and cp 
commands
   ## curl has '-k' flag to suppress certificate warnings
   blpLibrary:
@if [ ! -d ../inst ]; then mkdir -p ../inst; fi
@if [ ! -d ../blp/win/${FLV} ]; then mkdir -p 
../blp/win/${FLV}; fi
@if [ ! -f ../blp/win/${FLV}/blpHeaders.tar.gz ]; then curl -s 
-k -L -O 
https://github.com/Rblp/blp/raw/master/headers/windows/blpHeaders.tar.gz; mv 
blpHeaders.tar.gz ../blp/win/${FLV}; tar xfz 
../blp/win/${FLV}/blpHeaders.tar.gz -C ../inst; fi
@if [ ! -f ../blp/win/${FLV}/blpLibrary.tar.gz ]; then curl -s 
-k -L -O https://github.com/Rblp/blp/raw/master/win${WIN}/blpLibrary.tar.gz; mv 
blpLibrary.tar.gz ../blp/win/${FLV}; tar xfz 
../blp/win/${FLV}/blpLibrary.tar.gz; fi
@if [ ! -d ${FLV} ]; then mkdir -p ${FLV}; fi
@cp blpapi3_${WIN}.dll ${FLV}
@if [ ! -d ../inst/libs/${FLV} ]; then mkdir -p 
../inst/libs/${FLV}; fi
@cp blpapi3_${WIN}.dll ../inst/libs/${FLV}


None of that really generalizes well.  The last bit, much like the so-called
anticonf pattern, is more or less a non-standard practice exposing a security
risk by copying in code from a semi-random location which could be hijacked
or man-in-the-middle'd.

Really all this is just pretending that Windows is an envrionment on which
you can deploy proper build systems -- similar to what we are used from the
likes of dpkg/apt/yum or brew.  But in all honesty you just can't.

So I would not try to aim for something overly general.  It is Windows after 
all.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] [Learning] the secret of Win[dows C-backed packages]

2016-08-13 Thread Bob Rudis 
Hey folks,

I usually stare in awe at the C-backed packages that rely on eternal
libraries which are super-easy to get working on macOS & *nix _but_ that
also work perfectly on Windows. I fire up Windows (*maybe*) once a month to
test some of my packages but I'm curious as to what I need to do to setup a
more extended toolchain to create libraries such as are found in
https://github.com/rwinlib.

Jeroen (I think others, too) seem to effortlessly package those up and I
fully grok his "anticonf" setup and the 32-bit & 64-bit Windows library
setup. I just can't seem to actually do the building.

Why I'm asking what I'm asking (below) is that I'm working on something for
the $DAYJOB now - https://github.com/hrbrmstr/wand - that requires libmagic
and I kinda, unfortunately, actually need it to also work on Windows (it
usually doesn't matter, but this time some folks who use Windows systems at
work need to use this forthcoming package).

Rather than b[eu]g Jeroen or others for library building/packaging, I'd be
very, keenly interested in learning what else besides Rtools I need to
install to be able to make similar binary bundles to include during
installation (and to have my compiled pkgs work on Windows).

My disdain for Windows and my frustration with the super-limited toolchain
components installed with Rtools is probably contributing to my inability
to solve this on my own (no bash.exe...rly?). I used to use cygwin back in
the day but there are so many cautions in the main R docs abt compatible
versions that I'd rather not spend hours or days figuring out something
that others already have.

One example here is that I actually managed to get one of the libmagic
support packages (the regex library) built and installed on Windows (in the
toolchain /usr/local) but can't get libmagic's `configure` script to
find/use it.

I'm more than willing to write up any copious pointers folks provide into a
detailed, attributed blog post or R Journal article since this is seriously
the only bit holding me back from working on and releasing a ton of other
compilation-required packages to CRAN. I also know I'm not the only one who
gets stalled when trying to get similar things (especially those with full
Boost dependencies) to work on Windows.

thx,

-Bob

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel