Re: [R] Package httr::GET() question

[Bob Rudis]

curl::curl_escape() —
https://github.com/jeroen/curl/search?q=curl_escape_q=curl_escape
— uses the underlying libcurl curl_easy_escape() which does proper
escaping b/c it's, well, curl.

{httr} uses curl::curl_escape() —
https://github.com/r-lib/httr/search?q=curl_escape_q=curl_escape

The use it's `url-query.r` is the function compose_query().

compose_query()  is called from build_url() in url.r. handle_url()
(from handle-url.r) uses build_url().

All the "verbs" use handle_url() —
https://github.com/r-lib/httr/search?q=handle_url_q=handle_url

So {httr} relies on the quintessential standard in URL escaping —
which is libcurl's — for all URL machinations.

-boB

On Wed, Feb 19, 2020 at 10:36 AM Roy Mendelssohn - NOAA Federal via
R-help  wrote:
>
> Thanks.  Yes.  I did that,  it also has a verbose mode so that I could see 
> what it was doing.  What I needed was not just escaping but strict escaping.  
> My memory forma number of years back was that I had issues with urlencode 
> from base not being strict.  And of course you don't what to encode twice.
>
> Thanks,
>
> -Roy
>
>
> > On Feb 19, 2020, at 7:08 AM, Ben Tupper  wrote:
> >
> > Hi,
> >
> > Perhaps you could test it out by using httr::GET() with and without
> > escaping using xml2::url_escape()?
> >
> > https://www.rdocumentation.org/packages/xml2/versions/1.2.2/topics/url_escape
> >
> > Cheers,
> > Ben
> >
> > On Tue, Feb 18, 2020 at 1:29 PM Roy Mendelssohn - NOAA Federal via
> > R-help  wrote:
> >>
> >> Hi All:
> >>
> >> I hav been trying to go through the code for httr::GET() but it is 
> >> somewhat beyond what I know.  What I am trying to find out is if all urls 
> >> are automatically percent encoded,  or whether the user needs to do that.
> >>
> >> Thanks,
> >>
> >> -Roy
> >>
> >> **
> >> "The contents of this message do not reflect any position of the U.S. 
> >> Government or NOAA."
> >> **
> >> Roy Mendelssohn
> >> Supervisory Operations Research Analyst
> >> NOAA/NMFS
> >> Environmental Research Division
> >> Southwest Fisheries Science Center
> >> ***Note new street address***
> >> 110 McAllister Way
> >> Santa Cruz, CA 95060
> >> Phone: (831)-420-3666
> >> Fax: (831) 420-3980
> >> e-mail: roy.mendelss...@noaa.gov www: https://www.pfeg.noaa.gov/
> >>
> >> "Old age and treachery will overcome youth and skill."
> >> "From those who have been given much, much will be expected"
> >> "the arc of the moral universe is long, but it bends toward justice" -MLK 
> >> Jr.
> >>
> >> __
> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide 
> >> http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >
> >
> >
> > --
> > Ben Tupper
> > Bigelow Laboratory for Ocean Science
> > West Boothbay Harbor, Maine
> > http://www.bigelow.org/
> > https://eco.bigelow.org
>
> **
> "The contents of this message do not reflect any position of the U.S. 
> Government or NOAA."
> **
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: roy.mendelss...@noaa.gov www: https://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] [FORGED] Re: Regarding R licensing usage guidance

[Bob Rudis]

Hey Anamika,

I only caught the tail end of what became an off-topic thread, but backed up a 
bit to your original q. If I'm duplicating anything previous, apologies.

If you are going to ship your "product" to end users directly (vs provide via 
an API or web application) I'm not sure how you get around the "you have to 
ship your source code" problem (regardless of license issues). Even the best 
obfuscation is fairly easily overcome these days.

On a different note, apart from {RTextTools} and {tabulizer} (both of which 
aren't on CRAN so I couldn't just run a script against my mirror) you have the 
follow package licensing dependencies as well:

   License n
 1 GPL-3   7
 2 MIT + file LICENSE  7
 3 GPL (>= 2)  5
 4 GPL-2   4
 5 GPL-2 | GPL-3   4
 6 BSD 2-clause License + file LICENSE 1
 7 BSD_3_clause + file LICENSE 1
 8 file LICENSE1
 9 GPL-2 | file LICENSE1
10 MPL-2.0 | file LICENSE  1

So, if you do end up distributing something you're going to need to plan for 
adhering to the requirements of each of those when it comes to how you have to 
message the use of them (and license for them) to end-users.

If this is a pay-for piece of software you're hoping to distribute directly to 
end users then I'd highly suggest seeking legal assistance from a firm that 
specializes in reviewing licensing situations. They abound these days and it'll 
ultimately be worth the expense (it shouldn't be too bad).

-Bob

> On Jul 24, 2019, at 6:07 PM, Andrew Robinson  wrote:
> 
> 1.
> *R-3.4.4 *
> 2. *'spacyr'*
> 3.
> *'jsonlite' *
> 4.
> *'lubridate' *
> 5.
> *'data.table' *
> 6.
> *'png' *
> 7.
> *'maps' *
> 8.
> *'countrycode' *
> 9.
> *'humaniformat' *
> 10.
> *'ngram' *
> 11.
> *'stringr' *
> 12.
> *'slam' *
> 13.
> *'tm' *
> 14.
> *'lsa' *
> 15.
> *'RTextTools' *
> 16.
> *'stringi' *
> 17.
> *'plumber' *
> 18. *"Rook"*
> 19. *"pdftools"*
> 20. *'tokenizers'*
> 21. *'zoo'*
> 22. *"tidyr"*
> 23. *"reqres"*
> 24. *"rJava"*
> 25. *"tiff"*
> 26. *"splitstackshape"*
> 27. *"stringdist"*
> 28. *"RJSONIO"*
> 29. *"ropensci/tabulizer"*
> 30. *"staplr"*
> 31. *"SparseM"*
> 32. *"randomForest"*
> 33. *"e1071"*
> 34. *"ipred"*
> 35. *"caTools"*
> 36. *RCMD INSTALL maxent_1.3.3.1.tar.gz*
> 37. *RCMD INSTALL tree_1.0-39.tar.gz*
> 38. *RCMD INSTALL RTextTools_1.4.2.tar.gz*

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] [R-pkgs] New CRAN Package Announcement: splashr

[Bob Rudis]

I'm pleased to announce that splashr is now on CRAN:
https://CRAN.R-project.org/package=splashr

The package is an R interface to the Splash javascript rendering
service. It works in a similar fashion to Selenium but is fear more
geared to web scraping and has quite a bit of power under the hood.

There are three vignettes that should help folks get started using it
(like Selenium it really requires Docker for the best experience) and
there are a few blog posts about it kicking around on the internet.

Many thanks to all (if you're on here) who took it for a spin and
provided suggestions and even more thanks to the CRAN team for a
speedy onboarding.

-Bob

___
R-packages mailing list
r-packa...@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-packages

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] [R-pkgs] New package: hrbrthemes

[Bob Rudis]

Hey folks,

I'm pleased to announce the inaugural release of my hrbrthemes (0.1.0)
on CRAN: https://CRAN.R-project.org/package=hrbrthemes

The primary goal of said package is to provide opinionated
typographical and other aesthetic defaults for ggplot2 charts.

Two core themes are included:

- theme_ipsum() – an Arial Narrow-based theme
- theme_ipsum_rc() – a Roboto Condensed-based theme.

The Roboto Condensed Google Font comes with the package along with an
installer for said font.

Other niceties include:

- scale_[xy]_comma() — shortcut for expand=c(0,0), labels=scales::comma
- scale_[xy]_percent() — shortcut for expand=c(0,0), labels=scales::percent
- scale_[color|fill]_ipsum() — discrete scale with 9 colors
- gg_check() — pass-through spell checker for ggplot2 label elements

Source version is tracked on GitHub: https://github.com/hrbrmstr/hrbrthemes

Critiques, bug reports and enhancement requests are most welcome as
GitHub issues.

-Bob

___
R-packages mailing list
r-packa...@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-packages
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] [R-pkgs] New package: epidata

[Bob Rudis]

Hey folks,

epidata — https://cran.r-project.org/package=epidata — hit CRAN a few
days ago. It provides tools to retrieve Economic Policy Institute data
library extracts from their "hidden"-but-well-conceived API, returning
pristine data frames.

EPI  provides researchers, media, and the public
with easily accessible, up-to-date, and comprehensive historical data
on the American labor force. It is compiled from Economic Policy
Institute analysis of [U.S.] government data sources.

It has data on wages, unemployment, inequality, and other economic
indicators over time and among demographic groups. Their data is
usually updated monthly.

Code (with extended examples in the README) is at:
https://github.com/hrbrmstr/epidata

Issues, enhancements (etc) are — as always — welcome.

-Bob

___
R-packages mailing list
r-packa...@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-packages
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Match ISO 8601 week-of-year numbers to month-of-year numbers on Windows with German locale

[Bob Rudis]

Aye, but this:

  some_dates <- as.POSIXct(c("2015-12-24", "2015-12-31", "2016-01-01",
"2016-01-08"))

  (year_week <- format(some_dates, "%Y-%U"))
  ## [1] "2015-51" "2015-52" "2016-00" "2016-01"

  (year_week_day <- sprintf("%s-1", year_week))
  ## [1] "2015-51-1" "2015-52-1" "2016-00-1" "2016-01-1"

  (as.POSIXct(year_week_day, format = "%Y-%U-%u"))
  ## [1] "2015-12-21 EST" "2015-12-28 EST" "2016-01-04 EST" "2016-01-04 EST"

works fine on macOS & Linux (Ubuntu, anyway), but it fails on Windows
(10, 64bit, R 3.3.2):

  (as.POSIXct(year_week_day, format = "%Y-%U-%u"))
  ## [1] "2015-12-21 PST" "2015-12-28 PST" NA   "2016-01-04 PST"

On 1/12/17, David Winsemius  wrote:
>
>> On Jan 12, 2017, at 8:14 AM, Janko Thyson  wrote:
>>
>> Dear list,
>>
>> I'm experiencing problems with converting strings of the format
>> "-" (e.g. 2016-01, 2016-52) to proper POSIX dates which
>> (I
>> think) I need in order to retrieve the month-of-the-year number.
>>
>> Simpler put: I'd like to match week-of-the-year numbers to
>> month-of-the-year numbers. Ideally, the week-of-the-year number would
>> follow the ISO 8601 convention (i.e. format argument "%V") instead of the
>> US (format argument "%U") or UK (format argument "%W") convention.
>>
>> After posting this to Stackoverflow, I have strong reasons to believe
>> that
>> the issue is caused by Windows:
>> http://stackoverflow.com/questions/41616407/match-iso-8601-week-numbers-to-month-of-year-on-windows-with-german-locale/41617215?noredirect=1#comment70436768_41617215
>>
>> Example:
>>
>> # ISO 8601 convention:
>>
>> (yw <- format(posix, "%Y-%V"))
>
> The documentation for R datetime format parameters ?strptime says %V is
> ignored on input.
>
>
>> # [1] "2015-52" "2015-53" "2016-53" "2016-01"
>> ywd <- sprintf("%s-1", yw)(as.POSIXct(ywd, format = "%Y-%V-%u"))
>
> The documentation for R datetime format parameters ( = ?strptime) says %V is
> ignored on input.
>
> You should leartn to post plain text to r-help.
>
> --
> David.
>
>
>> # [1]
>> "2015-01-12 CET" "2015-01-12 CET" "2016-01-12 CET" "2016-01-12 CET"#
>> -> utterly wrong!!!
>>
>> # US convention:
>> (yw <- format(posix, "%Y-%U"))# [1] "2015-51" "2015-52" "2016-00"
>> "2016-01"
>> ywd <- sprintf("%s-1", yw)(as.POSIXct(ywd, format = "%Y-%U-%u"))# [1]
>> "2015-12-21 CET" "2015-12-28 CET" NA   "2016-01-04 CET"#
>> -> NA problem for week 00A fellow R user tested this on both macOS and
>> Ubuntu and he didn't encounter the issue:
>>
>> some_dates <- as.POSIXct(c("2015-12-24", "2015-12-31", "2016-01-01",
>> "2016-01-08"))
>> (year_week <- format(some_dates, "%Y %U"))## [1] "2015 51" "2015 52"
>> "2016 00" "2016 01"
>> (year_week_day <- sprintf("%s 1", year_week))## [1] "2015 51 1" "2015
>> 52 1" "2016 00 1" "2016 01 1"
>> (as.POSIXct(year_week_day, format = "%Y %U %u"))## [1] "2015-12-21
>> EST" "2015-12-28 EST" "2016-01-04 EST" "2016-01-04 EST"
>>
>> My session info:
>>
>>> sessionInfo()
>> R version 3.3.2 (2016-10-31)
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> Running under: Windows >= 8 x64 (build 9200)
>>
>> locale:[1] LC_COLLATE=German_Germany.1252
>> LC_CTYPE=German_Germany.1252   LC_MONETARY=German_Germany.1252
>> [4] LC_NUMERIC=C   LC_TIME=English_United
>> States.1252
>>
>> attached base packages:[1] stats graphics  grDevices utils
>> datasets  methods   base
>>
>> other attached packages:
>> [1] fva_0.1.0   digest_0.6.10   readxl_0.1.1dplyr_0.5.0
>> plyr_1.8.4  magrittr_1.5
>> [7] memoise_1.0.0   testthat_1.0.2  roxygen2_5.0.1  devtools_1.12.0
>>
>> loaded via a namespace (and not attached):
>> [1] Rcpp_0.12.8 lubridate_1.6.0 assertthat_0.1  packrat_0.4.8-1
>> crayon_1.3.2withr_1.0.2
>> [7] R6_2.2.0DBI_0.5-1   stringi_1.1.2   rstudioapi_0.6
>> tools_3.3.2 stringr_1.1.0  [13] tibble_1.2
>>
>> Any idea on how to workaround this issue on Windows?
>>
>> Thanks and best regards,
>>
>> Janko Thyson
>>
>>  [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> David Winsemius
> Alameda, CA, USA
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, 

Re: [R] Help with decrypting

[Bob Rudis]

Perhaps https://cran.r-project.org/web/packages/bcrypt/index.html
might be of assistance.

If not, drop a note back to the list as it'll be trivial to expand on
that to give you an R alternative to Perl.

On Mon, Nov 7, 2016 at 5:47 PM, MacQueen, Don  wrote:
> I have a file containing encrypted contents. The contents can be decrypted
> using perl, like this:
>
> open (FILEHANDLE, "/path/to/file")
> chomp ($ciphertext = );
>
>
> use Crypt::CBC;
> $cipher = Crypt::CBC->new( -key=> 'my secret key',
>-cipher => 'Blowfish'
>   );
>
> $plaintext  = $cipher->decrypt($ciphertext);
>
>
> (See http://search.cpan.org/~lds/Crypt-CBC-2.33/CBC.pm)
>
> M goal is to have the value of $plaintext in an R object, so, is there an
> R equivalent to this decrypt() perl function?
>
> I've found R packages
>   bcrypt
>   sodium
> that appear to have potential, but I don't understand this business well
> enough to figure out how to use them, if indeed they can be used, for
> this. Help would be much appreciated.
>
> Thanks
> -Don
>
> --
> Don MacQueen
>
> Lawrence Livermore National Laboratory
> 7000 East Ave., L-627
> Livermore, CA 94550
> 925-423-1062
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] The equivalent of which() when accessing slots in an object

[Bob Rudis]

Aye, Bill. I figured others wld include a `vapply()` example (didn't
want to "hog" the answer :-).

I went with bland `sapply()` as an alternative since I made an
assumption Thomas (like the large % of R users I've come in contact
with - albeit a biased sample) aren't really familiar with `vapply()`.

On Mon, Oct 31, 2016 at 12:29 PM, William Dunlap <wdun...@tibco.com> wrote:
> vapply(buylist, slot, "reqstock", FUN.VALUE=0.0)
> is closer in spirit than sapply() to purrr::map_dbl() - FUN.VALUE
> gives the expected type and size of f's output.  vapply() is usually
> quicker than sapply(), uses less memory, gives the right results
> when given a vector of length 0, and gives an error when FUN does
> not return the specified sort of result.
>
>
> Bill Dunlap
> TIBCO Software
> wdunlap tibco.com
>
> On Mon, Oct 31, 2016 at 7:31 AM, Bob Rudis <b...@rud.is> wrote:
>>
>> which(purrr::map_dbl(buylist, slot, "reqstock") > 100)
>>
>> or
>>
>> which(sapply(buylist, slot, "reqstock") > 100)
>>
>> ought to do the trick.
>>
>> On Mon, Oct 31, 2016 at 10:09 AM, Thomas Chesney
>> <thomas.ches...@nottingham.ac.uk> wrote:
>> > I have the following object
>> >
>> > setClass("buyer",
>> > representation(
>> > reqstock="numeric",
>> > buyout="numeric"),
>> > )
>> >
>> > nBuy <- 5
>> >
>> > #Set buyer parameters here
>> > buylist <- list()
>> > for (i in 1:nBuy){
>> > buylist[[i]] <- new("buyer")
>> > buylist[[i]]@reqstock <- sample(c(50:200),1)
>> > }
>> >
>> > and want to count the number of objects in buylist that have reqstock
>> > greater than 100. Something like this if it was a vector:
>> >
>> > length(which(buylist[[]]@reqstock > 100))
>> >
>> > How could I do this please and where could I find more information about
>> > manipulating slots? When I try to search for this I just keep finding info
>> > on ls() and similar functions.
>> >
>> > Thank you
>> >
>> > Thomas Chesney
>> > http://www.nottingham.ac.uk/~liztc/Personal/index.html
>> >
>> >
>> >
>> > This message and any attachment are intended solely for the addressee
>> > and may contain confidential information. If you have received this
>> > message in error, please send it back to me, and immediately delete it.
>> >
>> > Please do not use, copy or disclose the information contained in this
>> > message or in any attachment.  Any views or opinions expressed by the
>> > author of this email do not necessarily reflect the views of the
>> > University of Nottingham.
>> >
>> > This message has been checked for viruses but the contents of an
>> > attachment may still contain software viruses which could damage your
>> > computer system, you are advised to perform your own checks. Email
>> > communications with the University of Nottingham may be monitored as
>> > permitted by UK legislation.
>> >
>> > __
>> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide
>> > http://www.R-project.org/posting-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] The equivalent of which() when accessing slots in an object

[Bob Rudis]

which(purrr::map_dbl(buylist, slot, "reqstock") > 100)

or

which(sapply(buylist, slot, "reqstock") > 100)

ought to do the trick.

On Mon, Oct 31, 2016 at 10:09 AM, Thomas Chesney
 wrote:
> I have the following object
>
> setClass("buyer",
> representation(
> reqstock="numeric",
> buyout="numeric"),
> )
>
> nBuy <- 5
>
> #Set buyer parameters here
> buylist <- list()
> for (i in 1:nBuy){
> buylist[[i]] <- new("buyer")
> buylist[[i]]@reqstock <- sample(c(50:200),1)
> }
>
> and want to count the number of objects in buylist that have reqstock greater 
> than 100. Something like this if it was a vector:
>
> length(which(buylist[[]]@reqstock > 100))
>
> How could I do this please and where could I find more information about 
> manipulating slots? When I try to search for this I just keep finding info on 
> ls() and similar functions.
>
> Thank you
>
> Thomas Chesney
> http://www.nottingham.ac.uk/~liztc/Personal/index.html
>
>
>
> This message and any attachment are intended solely for the addressee
> and may contain confidential information. If you have received this
> message in error, please send it back to me, and immediately delete it.
>
> Please do not use, copy or disclose the information contained in this
> message or in any attachment.  Any views or opinions expressed by the
> author of this email do not necessarily reflect the views of the
> University of Nottingham.
>
> This message has been checked for viruses but the contents of an
> attachment may still contain software viruses which could damage your
> computer system, you are advised to perform your own checks. Email
> communications with the University of Nottingham may be monitored as
> permitted by UK legislation.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] About converting files in R

[Bob Rudis]

Can you tell us where you got the file from and perhaps even send a
link to the file? I know of at least 11 types of files that use `.bin`
as an extension which are all different types of data with different
binary formats.

On Tue, Oct 25, 2016 at 5:40 PM, Bob Rudis <b...@rud.is> wrote:
> I'm afraid we'll need more information that that since the answer from
> many folks on the list to such a generic question is going to be a
> generic "yes".
>
> What's the source of the binary files? If you know the type, there may
> even be an R package for it already.
>
> On Tue, Oct 25, 2016 at 5:28 PM, lily li <chocol...@gmail.com> wrote:
>> Hi R users,
>>
>> Do any of you have any experience about converting binary files to ascii or
>> txt files in R? Thanks a lot for your help.
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] About converting files in R

[Bob Rudis]

I'm afraid we'll need more information that that since the answer from
many folks on the list to such a generic question is going to be a
generic "yes".

What's the source of the binary files? If you know the type, there may
even be an R package for it already.

On Tue, Oct 25, 2016 at 5:28 PM, lily li  wrote:
> Hi R users,
>
> Do any of you have any experience about converting binary files to ascii or
> txt files in R? Thanks a lot for your help.
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] rsync: failed to connect to cran.r-project.org (137.208.57.37): No route to host (113)

[Bob Rudis]

I ran traceroutes & BGP traces from Marseille & Paris routers to that
CRAN IPv4 address (it's 10hrs after your mail, tho) and there's no
network errors. You can use any CRAN mirror, though. You aren't
limited to that one.

On Mon, Oct 24, 2016 at 9:49 AM, Etienne Borocco
 wrote:
> I still have the same problem today. Has the issue been fixed in Vienna?
>
>
> Le 19/10/2016 à 22:46, Duncan Murdoch a écrit :
>> On 19/10/2016 11:36 AM, Etienne Borocco wrote:
>>> Hello,
>>>
>>> I folowed this tutorial here:
>>> http://singmann.org/installing-r-devel-on-linux/
>>>
>>> I tried to install r-dev to compile gstoos that I can't manage to
>>> compile now on my ubuntu 16.04 distribution.
>>>
>>> I get an error with rsync:
>>>
>>>  1. bash ./tools/rsync-recommended
>>>
>>> There is the output of the shell:
>>>
>>> rsync: failed to connect to cran.r-project.org (137.208.57.37): No route
>>> to host (113)
>>> rsync error: error in socket IO (code 10) at clientserver.c(128)
>>> [Receiver=3.1.1]
>>> *** rsync failed to update Recommended files ***
>>>
>>> I saw this old post from 2008 but it does not seem to adress my issue:
>>> https://stat.ethz.ch/pipermail/r-devel/2008-October/050973.html
>>
>> Other people have been having trouble today getting through to some of
>> the Vienna machines.  I've got no idea if it's a problem in Vienna or
>> somewhere else, but it will probably be fixed pretty quickly.
>> However, it's now after office hours, so if the problem is actually in
>> Vienna, it might not be fixed until tomorrow.
>>
>> Duncan Murdoch
>>
>
> --
> Etienne Borocco
> PhD Student in Economics - Paris Dauphine University
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] gtools Gator infected...

[Bob Rudis]

I think your tool is a bit overzealous. VirusTotal -
https://virustotal.com/en/file/5fd1b2fc5c061c0836a70cbad620893a89a27d9251358a5c42c3e49113c9456c/analysis/
& 
https://virustotal.com/en/file/e133ebf5001e1e991f1f6b425adcfbab170fe3c02656e3a697a5ebea961e909c/analysis/
- shows no sign of any malware in the 32-bit DLLor 64-bit DLL (I
tested  r-release: gtools_3.5.0.zip)

On Sat, Oct 22, 2016 at 4:50 AM, Troels Ring  wrote:
> Hi friends - just installed gtools to make rstan run. Was rapidly informed
> by Spyhunter 4 that gtools.dll harboured Gator. Spyhunter 4 then
> aggressively removed Gator - and gtools were gone and rstan out of function.
> Kind of sorry about this.
>
> Best wishes
>
> Troels Ring
>
> Aalborg, Denmark
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] function which returns number of occurrences of a pattern in string

[Bob Rudis]

`stringi::stri_count()`

I know that the `stringr` pkg saves some typing (it wraps the
`stringi` pkg), but you should really just use the `stringi` package.
It has many more very useful functions with not too much more typing.

On Thu, Oct 20, 2016 at 5:47 PM, Jan Kacaba  wrote:
> Hello dear R-help
>
> I tried to find function which returns number of occurrences of a pattern
> in string. The closest match I've found is str_locate_all in stringr
> package. I can use str_locate_all but write my function but I don't want
> reinvent wheel.
>
> JK
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] create n suffixes of length 1:n from string of length n

[Bob Rudis]

purrr::map(paste0(letters, collapse=""), ~purrr::map2_chr(.,
1:nchar(.), ~substr(.x, 1, .y)))[[1]]

seems to crank really fast at least on my system

what did you try that was slow?

On Wed, Oct 19, 2016 at 11:01 AM, Witold E Wolski  wrote:
> Is there a build in function, which creates n suffixes of length 1:n
> from string of length n?
> e.g given abcd
>
> produces
> "a"
> "ab"
> "abc"
>
>
> FAST.
>
> equally nice to have would be:
>
> e.g.
> given c("a", "b", "c")
> produces
> "a"
> "a","b"
> "a","b","c"
>
> Thank you
> Witold
>
>
> --
> Witold Eryk Wolski
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] JSON to Dataframe

[Bob Rudis]

If those are in "ndjson" files or are indeed single records, `ndjson`
functions will be a few orders of magnitude faster and will produce
perfectly "flat" data frames. It's not intended to be a replacement
for `jsonlite` (a.k.a. the quintessential JSON pkg for R) but it's
tailor made for making quick work of (potentially deeply nested)
ndjson records/files.

On Tue, Oct 18, 2016 at 7:49 AM, K. Elo  wrote:
> Hi!
>
> 18.10.2016, 14:38, Abhinaba Roy wrote:
>>
>> Hi R helpers,
>>
>> I have json inputs from an app which I want to convert to dataframes.
>> Below
>> are the two inputs. Can someone help me in converting these to dataframes
>>
> [...]
>
> IMHO, the best way is to use the package 'jsonlite', see:
>
> * https://cran.r-project.org/web/packages/jsonlite/index.html
>
> A good documentation comes with the package, but the links listed on the
> CRAN site provide useful information, too.
>
> HTH,
> Kimmo
>
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Reg : R : How to capture cpu usage, memory usage and disks info using R language

[Bob Rudis]

You can do something like:
https://www.simple-talk.com/sql/performance/collecting-performance-data-into-a-sql-server-table/
and avoid the R step. Let the log perf data directly.

On Mon, Oct 17, 2016 at 6:03 AM, jim holtman  wrote:
> within the VBS script you can easily access remote computers.
>
> Jim Holtman
> Data Munger Guru
>
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
>
>
> On Mon, Oct 17, 2016 at 5:58 AM, Manohar Reddy  wrote:
>> Thanks Jim.
>>
>>
>>
>>Actually my requirement is I have ~ 20 servers which are running on
>> windows server family OS,if I want to check any server cpu usge,memory usage
>> or disk info I need to log into every server instead of doing like this if I
>> get that kind of information using R then I can save it in some RDBMS
>> database then I will populate this live data on some dashboard like which
>> are made by R using shiny,so that I can view/get the all the information on
>> single page.
>>
>>
>>
>>Here for me challenging work is how to capture cpu,memory,disk info using
>> R .
>>
>>
>> On Sun, Oct 16, 2016 at 8:37 PM, jim holtman  wrote:
>>>
>>> Here is a start on the solution.  This will create a VBS script that
>>> will gather the CPU data and return it in a character vector that you
>>> can extract the data from.  You can add to it to get the other data
>>> you are looking for.
>>>
>>> 
>>> > temp <- tempfile(fileext = '.vbs')  # get a temp file
>>> >
>>> > # create the VBS file to collect processor data
>>> > writeLines('Set objWMIService =
>>> > GetObject("winmgmts:localhost\\root\\CIMV2")
>>> + Set CPUInfo = objWMIService.ExecQuery("SELECT * FROM
>>> Win32_PerfFormattedData_PerfOS_Processor",,48)
>>> + For Each Item in CPUInfo
>>> + Wscript.Echo "PercentProcessorTime: " & Item.PercentProcessorTime &
>>> _
>>> +  "  processor:" & Item.Name
>>> + Next',
>>> +  temp)
>>> >
>>> > results <- shell(paste("cscript", temp), intern = TRUE)  # execute using
>>> > 'cscript'
>>> > results # all the data
>>> [1] "Microsoft (R) Windows Script Host Version 5.8"
>>> [2] "Copyright (C) Microsoft Corporation. All rights reserved."
>>> [3] ""
>>> [4] "PercentProcessorTime: 18  processor:0"
>>> [5] "PercentProcessorTime: 6  processor:1"
>>> [6] "PercentProcessorTime: 6  processor:2"
>>> [7] "PercentProcessorTime: 0  processor:3"
>>> [8] "PercentProcessorTime: 7  processor:_Total"
>>> > grep("processor:", results, value = TRUE)  # get just processor data
>>> [1] "PercentProcessorTime: 18  processor:0" "PercentProcessorTime:
>>> 6  processor:1"
>>> [3] "PercentProcessorTime: 6  processor:2"  "PercentProcessorTime:
>>> 0  processor:3"
>>> [5] "PercentProcessorTime: 7  processor:_Total"
>>> >
>>> >
>>> #
>>>
>>>
>>> Jim Holtman
>>> Data Munger Guru
>>>
>>> What is the problem that you are trying to solve?
>>> Tell me what you want to do, not how you want to do it.
>>>
>>>
>>> On Fri, Oct 14, 2016 at 5:37 AM, Manohar Reddy 
>>> wrote:
>>> > Hi,
>>> >
>>> > Is there any possibility that we can capture cpu usage ,memory usage and
>>> > disks info using R language on *windows family OS* ?
>>> >
>>> >
>>> >
>>> >   I would like to see data that’s looks like
>>> > a
>>> >  below
>>> >
>>> >
>>> >
>>> >Cpu usage : 70 %
>>> >
>>> >Memory usage  : 80 %
>>> >
>>> >Disks: C drive – 40 % full,D dive – 60 %,full E drive – 30 %
>>> > full
>>> >
>>> >
>>> >for more info please find the attachement.
>>> >
>>> >
>>> >  Thanks in Advance ,Manu.
>>> >
>>> > __
>>> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> > https://stat.ethz.ch/mailman/listinfo/r-help
>>> > PLEASE do read the posting guide
>>> > http://www.R-project.org/posting-guide.html
>>> > and provide commented, minimal, self-contained, reproducible code.
>>
>>
>>
>>
>> --
>>
>>
>> Manu.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Compatible version of R software for OEL v6.5 Linux OS

[Bob Rudis]

Having worked in big pharma for over 10 years, I'm _fairly_ certain
AstraZeneca can afford some paid R consulting.

On Fri, Oct 14, 2016 at 2:14 PM, David Winsemius  wrote:
>
>> On Oct 14, 2016, at 12:05 AM, Vijayakumar, Sowmya 
>>  wrote:
>>
>> Hi R-Help team,
>>
>>
>> Greeting from AstraZeneca India!!
>>
>>
>>
>> We are currently using *R 3.1.1* in Windows machine for one of our 
>> application. We have a plan to upgrade the application. Please let us know 
>> the steps to download the R which is compatible with *Oracle Enterprise 
>> Linux (OEL) v6.5.*
>
> I'm curious why you think we are supposed to know the answer to that 
> question? I would think it should be asked of the people at Oracle.
>
>
>>
>> Thanks,
>> Sowmya
>>
>> 
>>
>> Confidentiality Notice: This message is private and may ...{{dropped:10}}
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> David Winsemius
> Alameda, CA, USA
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Share R.net dll without having to share R script code?

[Bob Rudis]

Ugly idea/option, but you could base64 encode the R script (solely to
avoid the need to do string quoting) and have that string in the
source of the R.net code, then pass it in to the eval portion or write
it out to a temp dir and pass that to the eval portion of the code.
That way the script is embedded with the DLL and not an extra asset
that needs to be managed.

On Fri, Oct 14, 2016 at 10:43 AM, Narendra Modi  wrote:
> Thanks Duncan. That's useful to know.
>
> On Fri, Oct 14, 2016 at 9:18 AM, Duncan Murdoch
>  wrote:
>> On 14/10/2016 10:00 AM, Narendra Modi wrote:
>>>
>>> Hello Gurus,
>>>
>>> I have built a code snippet using R.net wherein I call couple of R
>>> scripts to run optimization packages and use the output in C# code.
>>> The way I call the R scripts is just by providing its location in the
>>> C# code.
>>>
>>> So, if I have to share the .dll of the complete program, I will also
>>> have to share the R scripts; actual code. Is there anyway to avoid it;
>>> not having to share the r script code with users/testers.
>>> I am considerably new to R. Any suggestion in this direction is
>>> appreciated!
>>>
>>
>> I don't know what the R.net .dll is, but if it includes R, you need to share
>> the complete source code of anything you distribute that includes it.  The R
>> scripts will be only a small part of that.
>>
>> Sharing less than that is a copyright violation, since you are only licensed
>> to distribute R under the GPL license, and it requires that you share code
>> of the original and your modifications.
>>
>> Duncan Murdoch
>>
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] turning comma separated string from multiple choices into

[Bob Rudis]

Take a look at tidyr::separate()

On Fri, Oct 7, 2016 at 12:57 PM, silvia giussani
 wrote:
> Hi all,
>
>
>
> could you please tell me if you find a solution to this problem (in
> Subject)?
>
>
>
> June Kim wrote:
>
>>* Hello,*
>
>>
>
>>* I use google docs' Forms to conduct surveys online. Multiple choices*
>
>>* questions are coded as comma separated values.*
>
>>
>
>>* For example,*
>
>>
>
>>* if the question is like:*
>
>>
>
>>* 1. What magazines do you currently subscribe to? (you can choose*
>
>>* multiple choices)*
>
>>* 1) Fast Company*
>
>>* 2) Havard Business Review*
>
>>* 3) Business Week*
>
>>* 4) The Economist*
>
>>
>
>>* And if the subject chose 1) and 3), the data is coded as a cell in a*
>
>>* spreadsheet as,*
>
>>
>
>>* "Fast Company, Business Week"*
>
>>
>
>>* I read the data with read.csv into R. To analyze the data, I have to*
>
>>* change that string into something like flags(indicator variables?).*
>
>>* That is, there should be 4 variables, of which values are either 1 or*
>
>>* 0, indicating chosen or not-chosen respectively.*
>
>>
>
>>* Suppose the data is something like,*
>
>>
>
>>
>
>>>* survey1*
>
>>>
>
>>*   agefavorite_magazine*
>
>>* 1  29 Fast Company*
>
>>* 2  31  Fast Company, Business Week*
>
>>* 3  32 Havard Business Review, Business Week, The Economist*
>
>>
>
>>
>
>>* Then I have to chop the string in favorite_magazine column to turn*
>
>>* that data into something like,*
>
>>
>
>>
>
>>>* survey1transformed*
>
>>>
>
>>*   age Fast Company Havard Business Review Business Week The Economist*
>
>>* 1  291  0 0 0*
>
>>* 2  311  0 1 0*
>
>>* 3  320  1 1 1*
>
>>
>
>>
>
>>* Actually I have many more multiple choice questions in the survey.*
>
>>
>
>>* What is the easy elegant and natural way in R to do the job?*
>
>>
>
>
>
> I'd look into something like as.data.frame(lapply(strings, grep,
>
> x=favorite_magazine, fixed=TRUE)), where strings <- c("Fast Company",
>
> "Havard Business Review", ...).
>
>
>
> (I take it that the mechanism is such that you can rely on at least
>
> having everything misspelled in the same way? If it is alternatingly
>
> "Havard" and "Harvard", then things get a bit trickier.)
>
>
>
> Thank you and regards,
>
> Silvia Giussani
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Antwort: RE: How to plot a bunch of dichotomous code variables in one plot using ggplot2

[Bob Rudis]

No need to bring in so many dependencies for a simple ggplot2 marplot:

ds <- stack(ds)
ggplot(ds[ds$values==1,], aes(ind)) + geom_bar()

On Wed, Oct 5, 2016 at 10:17 AM, Thierry Onkelinx 
wrote:

> Here is a ggplot2, tidyr, dplyr solution
>
> library(tidyr)
> library(dplyr)
> library(ggplot2)
> ds %>%
>   gather() %>%
>   group_by(key) %>%
>   summarize(total = sum(value)) %>%
>   ggplot(aes(x = key, y = total)) +
>   geom_bar(stat = "identity")
>
>
> ir. Thierry Onkelinx
> Instituut voor natuur- en bosonderzoek / Research Institute for Nature and
> Forest
> team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance
> Kliniekstraat 25
> 1070 Anderlecht
> Belgium
>
> To call in the statistician after the experiment is done may be no more
> than asking him to perform a post-mortem examination: he may be able to say
> what the experiment died of. ~ Sir Ronald Aylmer Fisher
> The plural of anecdote is not data. ~ Roger Brinner
> The combination of some data and an aching desire for an answer does not
> ensure that a reasonable answer can be extracted from a given body of data.
> ~ John Tukey
>
> 2016-10-05 15:55 GMT+02:00 :
>
>> Hi Bob,
>> Hi John,
>> Hi readers,
>>
>> many thanks for your reply.
>>
>> I did
>>
>> barplot(colSums(dataset %>% select(FirstVar:LastVar)))
>>
>> and it worked fine.
>>
>> How would I do it with ggplot2?
>>
>> Kind regards
>>
>> Georg
>>
>>
>>
>>
>> Von:"Fox, John" 
>> An: "g.maub...@weinwolf.de" ,
>> Kopie:  "r-help@r-project.org" 
>> Datum:  05.10.2016 15:01
>> Betreff:RE: [R] How to plot a bunch of dichotomous code variables
>> in one plot using ggplot2
>>
>>
>>
>> Dear Georg,
>>
>> How about barplot(colSums(ds)) ?
>>
>> Best,
>>  John
>>
>> -
>> John Fox, Professor
>> McMaster University
>> Hamilton, Ontario
>> Canada L8S 4M4
>> Web: socserv.mcmaster.ca/jfox
>>
>>
>> > -Original Message-
>> > From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of
>> > g.maub...@weinwolf.de
>> > Sent: October 5, 2016 8:47 AM
>> > To: r-help@r-project.org
>> > Subject: [R] How to plot a bunch of dichotomous code variables in one
>> plot
>> > using ggplot2
>> >
>> > Hi All,
>> >
>> > I have a bunch of dichotomous code variables which shall be plotted in
>> one
>> > graph using one of their values, this is "1" in this case.
>> >
>> > The dataset looks like this:
>> >
>> > -- cut --
>> > var1 <- c(1,0,1,0,0,1,1,1,0,1)
>> > var2 <- c(0,1,1,1,1,0,0,0,0,0)
>> > var3 <- c(1,1,1,1,1,1,1,1,0,1)
>> >
>> > ds <- data.frame(var1, var2, var3)
>> > -- cut --
>> >
>> > I would like to have a bar plot like this
>> >
>> >
>> >
>> >   *
>> >   *
>> >   *
>> >   *
>> > * *
>> > * *
>> > *  *  *
>> > *  *  *
>> > *  *  *
>> > *  *  *
>> > -
>> > var1  var2   var3
>> >
>> > If this possible in R? If so, how can I achieve this?
>> >
>> > Kind regards
>> >
>> > Georg
>> >
>> > __
>> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide http://www.R-project.org/posting-
>> > guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posti
>> ng-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Antwort: RE: How to plot a bunch of dichotomous code variables in one plot using ggplot2

[Bob Rudis]

(s/marplot/barplot)

On Wed, Oct 5, 2016 at 10:35 AM, Bob Rudis <b...@rud.is> wrote:

> No need to bring in so many dependencies for a simple ggplot2 marplot:
>
> ds <- stack(ds)
> ggplot(ds[ds$values==1,], aes(ind)) + geom_bar()
>
> On Wed, Oct 5, 2016 at 10:17 AM, Thierry Onkelinx <
> thierry.onkel...@inbo.be> wrote:
>
>> Here is a ggplot2, tidyr, dplyr solution
>>
>> library(tidyr)
>> library(dplyr)
>> library(ggplot2)
>> ds %>%
>>   gather() %>%
>>   group_by(key) %>%
>>   summarize(total = sum(value)) %>%
>>   ggplot(aes(x = key, y = total)) +
>>   geom_bar(stat = "identity")
>>
>>
>> ir. Thierry Onkelinx
>> Instituut voor natuur- en bosonderzoek / Research Institute for Nature
>> and Forest
>> team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance
>> Kliniekstraat 25
>> 1070 Anderlecht
>> Belgium
>>
>> To call in the statistician after the experiment is done may be no more
>> than asking him to perform a post-mortem examination: he may be able to say
>> what the experiment died of. ~ Sir Ronald Aylmer Fisher
>> The plural of anecdote is not data. ~ Roger Brinner
>> The combination of some data and an aching desire for an answer does not
>> ensure that a reasonable answer can be extracted from a given body of data.
>> ~ John Tukey
>>
>> 2016-10-05 15:55 GMT+02:00 <g.maub...@weinwolf.de>:
>>
>>> Hi Bob,
>>> Hi John,
>>> Hi readers,
>>>
>>> many thanks for your reply.
>>>
>>> I did
>>>
>>> barplot(colSums(dataset %>% select(FirstVar:LastVar)))
>>>
>>> and it worked fine.
>>>
>>> How would I do it with ggplot2?
>>>
>>> Kind regards
>>>
>>> Georg
>>>
>>>
>>>
>>>
>>> Von:"Fox, John" <j...@mcmaster.ca>
>>> An: "g.maub...@weinwolf.de" <g.maub...@weinwolf.de>,
>>> Kopie:  "r-help@r-project.org" <r-help@r-project.org>
>>> Datum:  05.10.2016 15:01
>>> Betreff:RE: [R] How to plot a bunch of dichotomous code variables
>>> in one plot using ggplot2
>>>
>>>
>>>
>>> Dear Georg,
>>>
>>> How about barplot(colSums(ds)) ?
>>>
>>> Best,
>>>  John
>>>
>>> -
>>> John Fox, Professor
>>> McMaster University
>>> Hamilton, Ontario
>>> Canada L8S 4M4
>>> Web: socserv.mcmaster.ca/jfox
>>>
>>>
>>> > -Original Message-
>>> > From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of
>>> > g.maub...@weinwolf.de
>>> > Sent: October 5, 2016 8:47 AM
>>> > To: r-help@r-project.org
>>> > Subject: [R] How to plot a bunch of dichotomous code variables in one
>>> plot
>>> > using ggplot2
>>> >
>>> > Hi All,
>>> >
>>> > I have a bunch of dichotomous code variables which shall be plotted in
>>> one
>>> > graph using one of their values, this is "1" in this case.
>>> >
>>> > The dataset looks like this:
>>> >
>>> > -- cut --
>>> > var1 <- c(1,0,1,0,0,1,1,1,0,1)
>>> > var2 <- c(0,1,1,1,1,0,0,0,0,0)
>>> > var3 <- c(1,1,1,1,1,1,1,1,0,1)
>>> >
>>> > ds <- data.frame(var1, var2, var3)
>>> > -- cut --
>>> >
>>> > I would like to have a bar plot like this
>>> >
>>> >
>>> >
>>> >   *
>>> >   *
>>> >   *
>>> >   *
>>> > * *
>>> > * *
>>> > *  *  *
>>> > *  *  *
>>> > *  *  *
>>> > *  *  *
>>> > -
>>> > var1  var2   var3
>>> >
>>> > If this possible in R? If so, how can I achieve this?
>>> >
>>> > Kind regards
>>> >
>>> > Georg
>>> >
>>> > __
>>> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> > https://stat.ethz.ch/mailman/listinfo/r-help
>>> > PLEASE do read the posting guide http://www.R-project.org/posting-
>>> > guide.html
>>> > and provide commented, minimal, self-contained, reproducible code.
>>>
>>> __
>>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posti
>>> ng-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Problem installing rgdal.

[Bob Rudis]

​Hey Ron,

I (literally, in the correct use of the term) fired up an Ubuntu 16.04
vagrant box - https://atlas.hashicorp.com/bento/boxes/ubuntu-16.04 - and
then did:

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial

and, then:

sudo echo "deb http://cran.rstudio.com/bin/linux/ubuntu xenial/" | sudo tee
-a /etc/apt/sources.list
gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9
gpg -a --export E084DAB9 | sudo apt-key add -
sudo apt-get update
sudo apt-get install r-base r-base-dev
sudo apt-get install libgdal-dev
sudo apt-get install libproj4-dev

and then did

install.packages("rgdal")

in an R session and it's working fine:

> library(rgdal)
Loading required package: sp
rgdal: version: 1.1-10, (SVN revision 622)
 Geospatial Data Abstraction Library extensions to R successfully loaded
 Loaded GDAL runtime: GDAL 1.11.3, released 2015/09/16
 Path to GDAL shared files: /usr/share/gdal/1.11
 Loaded PROJ.4 runtime: Rel. 4.9.2, 08 September 2015, [PJ_VERSION: 492]
 Path to PROJ.4 shared files: (autodetected)
 Linking to sp version: 1.2-3

I wish I could have run into errors and helped debug your issue, but it
went in flawlessly.

-Bob
​


On Tue, Oct 4, 2016 at 8:31 PM, Rolf Turner  wrote:

> On 05/10/16 12:56, ProfJCNash wrote:
>
>> Can you build/install the source package? I had a problem once where my
>> libraries were "too recent" for the R package,
>> but I could build against my installed base. In any event, it may point
>> out the source of the problem.
>>
>> I can appreciate your frustration -- been there, but wish I hadn't.
>>
>
> Essentially install.packages() builds from source.
>
> I also tried to install from the source tarball; same error resulted.
>
> Surely there *must* be somebody out there who understands what's going on
> and how to fix it.  Mustn't there?
>
> cheers,
>
> Rolf
>
> --
> Technical Editor ANZJS
> Department of Statistics
> University of Auckland
> Phone: +64-9-373-7599 ext. 88276
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posti
> ng-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] [R-pkgs] A few new packages on CRAN

[Bob Rudis]

- ndjdon : Wicked Fast ndjson Reader

  Reads in ndjson significantly faster than jsonlite::stream_in(), flattens
each
  JSON record and returns a data.table.

  https://cran.r-project.org/web/packages/ndjson/index.html


- htmltidy : Clean Up or Pretty Print Gnarly HTML and XHTML

  C-backed package that includes the HTML Tidy library. Useful for cleaning
up
  HTML beyond what you get with the HTML parsing in the libxml2-based
packages.

  https://cran.r-project.org/web/packages/htmltidy/index.html

  (v0.3.0 on github is a tad more robust and will be in CRAN later in
October)


- wand : Retrieve 'Magic' Attributes from Files and Directories

  Uses libmagic (file.exe on Windows for the time being) to discern file
types.

  https://cran.r-project.org/web/packages/wand/index.html

  (100% libmagic version coming later in October)


- gdns : Tools to work with the Google DNS over HTTPS API

  Provides full access to the Google DNS HTTPS API and also toold to work
with
  SPF records. Great for validating your local provider DNS lookups and for
  generating features for cybersecurity machine learning.

  https://cran.r-project.org/web/packages/gdns/index.html


- qrencoder : Quick Response Code (QR Code) / Matrix Barcode Creator

  C-backed package to generate QR codes (it's being used in some bitcoin
Shiny
  projects).

  https://cran.r-project.org/web/packages/qrencoder/index.html


- darksky : Tools to Work with the Dark Sky API
  Data retrieval and some default plotting for this weather API.

  https://cran.r-project.org/web/packages/darksky/index.html

Issues/enhancement requests are most welcome at each pkg's GH issues page.

-Bob

[[alternative HTML version deleted]]

___
R-packages mailing list
r-packa...@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-packages

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Convert a list with NULL to a dataframe with NA

[Bob Rudis]

It's fairly straightforward with help from the purrr package:

library(purrr)

map_df(OB1, function(x) {
  if (length(x) == 0) {
data.frame(id=NA_character_, nam=NA_character_, stringsAsFactors=FALSE)
  } else {
data.frame(id=x[1], nam=names(x), stringsAsFactors=FALSE)
  }
}, .id="V1")



On Sun, Oct 2, 2016 at 6:40 AM, Mohammad Tanvir Ahamed via R-help <
r-help@r-project.org> wrote:

> Hi,
>
> I have a list like below.
>
> OB1 <- structure(list(aa0 = NULL,
> aa1 = structure("23403", .Names = "BB10"),
> aa2 = structure("54904", .Names = "BB20"),
> aa3 = structure("22897", .Names = "BB30"),
> aa4 = structure("3751", .Names = "BB40"),
> aa5 = NULL,
> aa6 = structure("3679", .Names = "BB50"),
> aa7 = structure("440193", .Names = "BB60"),
> aa8 = structure("23144", .Names = "BB70"),
> aa9 = structure("84667", .Names = "BB80"),
> aa10 = structure("130540", .Names = "BB90")),
> .Names = c("aa0", "aa1", "aa2", "aa3",
> "aa4", "aa5", "aa6", "aa7",
> "aa8", "aa9", "aa10"))
>
> I am expecting an output like below
> OB2 <- structure(list(V1 = structure(c(3L, 5L, 8L, 1L, 4L, 2L, 10L,
> 7L, 9L, 11L, 6L), .Label = c("aa3", "aa5", "aa0",
> "aa4", "aa1", "aa10", "aa7", "aa2",
> "aa8", "aa6", "aa9"), class = "factor"),
> id = structure(c(NA, 4L, 8L, 2L, 6L, NA, 5L, 7L, 3L, 9L,
> 1L), .Label = c("130540", "22897", "23144", "23403", "3679",
> "3751", "440193", "54904", "84667"), class = "factor"), nam =
> structure(c(NA,
> 4L, 8L, 3L, 7L, NA, 6L, 2L, 9L, 5L, 1L), .Label = c("BB90",
> "BB60", "BB30", "BB10", "BB80", "BB50", "BB40",
> "BB20", "BB70"), class = "factor")), .Names = c("V1",
> "id", "nam"), row.names = c(NA, -11L), class = "data.frame")
>
> Problems :
> 1. Get OB1 to OB2
> 2. Get OB2 to OB1
>
> I  will be great-full if anyone can share idea how to solve the problem .
> Thanks in advance !!
>
>
>
>
> Tanvir Ahamed
> Göteborg, Sweden  |  mashra...@yahoo.com
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Opening or activating a URL to access data, alternative to browseURL

[Bob Rudis]

The rvest/httr/curl trio can do the cookie management pretty well. Make the
initial connection via rvest::html_session() and then hopefully be able to
use other rvest function calls, but curl and httr calls will use the cached
in-memory handle info seamlessly. You'd need to store and retrieve cookies
if you need them preserved between R sessions.

Failing the above and assuming this would not need to be lightning fast,
use the phantomjs or firefox web driver (either with RSelenium or some new
stuff rOpenSci is cooking up) which will then do what browsers do best and
maintain all this state for you. You can still slurp the page contents up
with xml2::read_html() and use the super handy processing idioms in the
scraping tidyverse (it needs it's own name).

A concrete example (assuming the URLs aren't sensitive) would enable me or
someone else to mock up something for you.


On Thu, Sep 29, 2016 at 4:59 PM, Duncan Murdoch 
wrote:

> On 29/09/2016 3:29 PM, Ryan Utz wrote:
>
>> Hi all,
>>
>> I've got a situation that involves activating a URL so that a link to some
>> data becomes available for download. I can easily use 'browseURL' to do
>> so,
>> but I'm hoping to make this batch-process-able, and I would prefer to not
>> have 100s of browser windows open when I go to download multiple data
>> sets.
>>
>> Here's the example:
>>
>> #1
>> browseURL('
>> http://pick18.discoverlife.org/mp/20m?plot=2=Hypoprepia
>> +fucosa=33.9+-83.3=2011,2012,2013=build_txt:
>> ')
>> # This opens the URL and creates a link to machine-readable data on the
>> page, which I can then download by simply doing this:
>>
>> #2
>> read.delim('
>> http://pick18.discoverlife.org/tmp/Hypoprepia_fucosa_33.9_-
>> 83.3_2011,2012,2013.txt
>> ')
>>
>> However, I can only get the second line above to work if the thing in line
>> #1 has been opened in a browser already. Is there any way to allow me to
>> either 1) close the browser after it's been opened or 2) execute the line
>> #2 above without having to open a browser? We have hundreds of species
>> that
>> you can see after the '=' bit of the URL, so I'm trying to keep the
>> browsing situation sane.
>>
>> Thanks!
>> R
>>
>>
> You'll need to figure out what happens when you open the first page. Does
> it set a cookie?  Does it record your IP address?  Does it just build the
> file but record nothing about you?
>
> If it's one of the simpler versions, you can just read the first page,
> wait a bit, then read the second one.
>
> If you need to manage cookies, you'll need something more complicated. I
> don't know the easiest way to do that.
>
> Duncan Murdoch
>
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posti
> ng-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Accelerating binRead

[Bob Rudis]

You should probably pick a forum — here or SO :
http://stackoverflow.com/questions/39547398/faster-reading-of-binary-files-in-r
: - vs cross-post to all of them.

On Sat, Sep 17, 2016 at 11:04 AM, Ismail SEZEN 
wrote:

> I noticed same issue but didnt care much :)
>
> On Sat, Sep 17, 2016, 18:01 jim holtman  wrote:
>
> > Your example was not reproducible.  Also how do you "break" out of the
> > "while" loop?
> >
> >
> > Jim Holtman
> > Data Munger Guru
> >
> > What is the problem that you are trying to solve?
> > Tell me what you want to do, not how you want to do it.
> >
> > On Sat, Sep 17, 2016 at 8:05 AM, Philippe de Rochambeau 
> > wrote:
> >
> > > Hello,
> > > the following function, which stores numeric values extracted from a
> > > binary file, into an R matrix, is very slow, especially when the said
> > file
> > > is several MB in size.
> > > Should I rewrite the function in inline C or in C/C++ using Rcpp? If
> the
> > > latter case is true, how do you « readBin »  in Rcpp (I’m a total Rcpp
> > > newbie)?
> > > Many thanks.
> > > Best regards,
> > > phiroc
> > >
> > >
> > > -
> > >
> > > # inputPath is something like http://myintranet/getData?
> > > pathToFile=/usr/lib/xxx/yyy/data.bin  > > pathToFile=/usr/lib/xxx/yyy/data.bin>
> > >
> > > PLTreader <- function(inputPath){
> > > URL <- file(inputPath, "rb")
> > > PLT <- matrix(nrow=0, ncol=6)
> > > compteurDePrints = 0
> > > compteurDeLignes <- 0
> > > maxiPrints = 5
> > > displayData <- FALSE
> > > while (TRUE) {
> > > periodIndex <- readBin(URL, integer(), size=4, n=1,
> > > endian="little") # int (4 bytes)
> > > eventId <- readBin(URL, integer(), size=4, n=1,
> > > endian="little") # int (4 bytes)
> > > dword1 <- readBin(URL, integer(), size=4, signed=FALSE,
> > > n=1, endian="little") # int
> > > dword2 <- readBin(URL, integer(), size=4, signed=FALSE,
> > > n=1, endian="little") # int
> > > if (dword1 < 0) {
> > > dword1 = dword1 + 2^32-1;
> > > }
> > > eventDate = (dword2*2^32 + dword1)/1000
> > > repNum <- readBin(URL, integer(), size=2, n=1,
> > > endian="little") # short (2 bytes)
> > > exp <- readBin(URL, numeric(), size=4, n=1,
> > > endian="little") # float (4 bytes, strangely enough, would expect 8)
> > > loss <- readBin(URL, numeric(), size=4, n=1,
> > > endian="little") # float (4 bytes)
> > > PLT <- rbind(PLT, c(periodIndex, eventId, eventDate,
> > > repNum, exp, loss))
> > > } # end while
> > > return(PLT)
> > > close(URL)
> > > }
> > >
> > > 
> > > [[alternative HTML version deleted]]
> > >
> > > __
> > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide http://www.R-project.org/
> > > posting-guide.html
> > > and provide commented, minimal, self-contained, reproducible code.
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Better use of regex

[Bob Rudis]

Base:

Filter(Negate(is.na), sapply(regmatches(dimInfo, regexec("HS_(.{1})",
dimInfo)), "[", 2))

Modernverse:

library(stringi)
library(purrr)

stri_match_first_regex(dimInfo, "HS_(.{1})")[,2] %>%
  discard(is.na)


They both use capture groups to find the matches and return just the
matches. The "{1}" isn't really necessary but I include to show that you
can match whatever lengths you want, in this case just 1 char.

On Thu, Sep 15, 2016 at 12:17 PM, Doran, Harold  wrote:

> I have produced a terribly inefficient piece of codes. In the end, it
> gives exactly what I need, but clumsily steps through multiple steps which
> I'm sure could be more efficiently reduced.
>
> Below is a reproducible example. What I have to begin with is character
> vector, dimInfo. What I want to do is parse this vector 1) find the
> elements containing 'HS' and 2) grab *only* the first character after the
> "HS_". The final line of code in the example gives what I need.
>
> Any suggestions on a better approach?
>
> Harold
>
>
> dimInfo <- c("RecordID", "oppID", "position", "key", "operational",
> "IsSelected",
> "score", "item_1_HS_conv_ovrl_scr", "item_1_HS_elab_ovrl_scr",
> "item_1_HS_org_ovrl_scr")
>
> ff <- dimInfo[grep('HS', dimInfo)]
> gg <- strsplit(ff, 'HS_')
> hh <- sapply(1:3, function(i) gg[[i]][2])
> substr(hh, 1, 1)
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] impossible # of errors in a simple code

[Bob Rudis]

pretty sure you just missed the `{` at the beginning of the `function`
definition block.

On Sun, Sep 4, 2016 at 7:38 AM, Michael Dewey 
wrote:

> A useful rule is to fix the first error you understand and hope that the
> others go away.
>
> On 04/09/2016 04:05, Tamar Michaeli wrote:
>
>> Any help in resolving the following errors will be appreciated:
>>
>> pollutantmean <- function(directory, pollutant, id=1:332)
>>>
>> + file_inc <- list.files("specdata", full.names=TRUE)
>>
>
> So what did you hope your function was going to do, if you called it?
>
> dat <- data.frame()
>>> for(i in 1:10) {
>>>
>> + dat <- rbind(dat, read.csv(file_inc[i]))
>> + }
>> Error in read.table(file = file, header = header, sep = sep, quote =
>> quote,
>>  :
>>   object 'file_inc' not found
>>
>
> Why did you think at this point that you had an object called file_inc?
>
> sulfate <- subset(dat( , 2))
>>>
>> Error in subset(dat(, 2)) : could not find function "dat"
>>
>
> So you did not manage anywhere to define a function called dat.
>
> nitrate <- subset(dat( , 3))
>>>
>> Error in subset(dat(, 3)) : could not find function "dat"
>>
>>> mean(pollutant)
>>>
>> Error in mean(pollutant) : object 'pollutant' not found
>>
>
> You have never defined it.
>
> save"pollutantmean.R"
>>>
>> Error: unexpected string constant in "save"pollutantmean.R""
>>
>
> ?save might help you here.
>
>
>
>> [[alternative HTML version deleted]]
>>
>>
>
> Please do not post in HTML as it makes your post unreadable.
>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posti
>> ng-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
> --
> Michael
> http://www.dewey.myzen.co.uk/home.html
>
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posti
> ng-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] paste0 in file path

[Bob Rudis]

if the files are supposed to be "1r.xlsx", "2r.xlsx" (etc) then you need to
ensure there's a "/" before it.

It's better to use `file.path()` to, well, build file paths since it will
help account for differences between directory separators on the various
operating systems out there.

On Wed, Aug 31, 2016 at 11:54 AM, Uwe Ligges <
lig...@statistik.tu-dortmund.de> wrote:

>
>
> On 31.08.2016 17:50, Leslie Rutkowski wrote:
>
>> Hi,
>>
>> I'm trying to reshape and output 8 simple tables into excel files. This is
>> the code I'm using
>>
>>   for (i in 1:8) {
>>   count <- table(mydata$ctry, mydata[,paste0("q0",i,"r")])
>>   dat <- as.data.frame(q01count)
>>
>>   wide <- reshape(dat,
>>   timevar="Var2",
>>   idvar="Var1",
>>   direction="wide")
>>write.xlsx(wide, file=paste0(i, 'C:/temp/q0',i,'r.xlsx'))
>>
>
>   ^^
> remove the i?
>
> Best,
> Uwe Ligges
>
>
>
>   }
>>
>> All goes well until the write.xlsx, which produces the error
>>
>> Error in .jnew("java/io/FileOutputStream", jFile) :
>>   java.io.FileNotFoundException: 1C:\temp\q01r.xlsx (The filename,
>> directory name, or volume label syntax is incorrect)
>>
>> Among other things, I'm puzzled about why a "1" is getting tacked on to
>> the
>> file path.
>>
>> Any hints?
>>
>> Thanks,
>> Leslie
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posti
>> ng-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posti
> ng-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Retrieving data from survey in R Studio

[Bob Rudis]

Ulrik: you can absolutely read from a URL in read.csv() with that syntax.

The error `## Error in attach(survey): object 'survey' not found` suggests
that the OP mis-typed something in the `survey` name in the assignment from
`read.csv()`.

However, the OP has quite a bit more to be concerned about than the missing
variable in the environment if their stats program is encouraging both no
spaces between operators and the use of `attach()`.

On Thu, Aug 18, 2016 at 1:26 PM, Ulrik Stervbo 
wrote:

> I don't think you can read files into R like that. This answer
> http://stackoverflow.com/questions/3236651/read-data-from-internet on
> Stack
> Overflow tells you, you could use the RCurl package.
>
> Here are more examples on getting data from online sources
> https://www.r-bloggers.com/getting-data-from-an-online-source/
>
> You could also just download the file manually and open it using read.scv
>
> Hope this helps,
> Ulrik
>
> On Thu, 18 Aug 2016 at 19:10 Barathan Roy Pinas <
> bpin3...@uni.sydney.edu.au>
> wrote:
>
> > Hello,
> >
> >
> > I have been given a .csv file and it is not loading. This is what I did.
> >
> >
> > survey=read.csv("
> > http://www.maths.usyd.edu.au/u/UG/IM/STAT2012/r/survey.csv;)
> > attach(survey)
> > pulse.sf=pulse[smoke==1 & sex==2]
> > pulse.sf
> >
> > Template link here Rnw
> > >.
> >
> > I insert the file into the R commands section for question 1. Doing that
> > produces the following below:
> >
> >
> > survey <- read.csv(file="
> > http://www.maths.usyd.edu.au/u/UG/IM/STAT2012/r/survey.csv;)
> > attach(survey)
> >
> > ## Error in attach(survey): object 'survey' not found
> > pulse.sf=pulse[smoke==1 & sex==2]
> > ## Error in eval(expr, envir, enclos): object 'pulse' not found
> > pulse.sf
> > ## Error in eval(expr, envir, enclos): object 'pulse.sf' not
> > found
> >
> > What's going on? I can actually download the survey and view it on Excel.
> >
> > Appreciate any help that I can get.
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] [R-pkgs] New package uaparserjs 0.1.0 - Slice up browser user agent strings

[Bob Rudis]

I keep forgetting I can announce things here.

[Insert witty/standard boilerplate introductory verbiage here]

CRAN: 
GitHub: 

Until Oliver and/or I figure out a way to get uap-r
 working w/o Boost, this package
provides a way to parse browser user agent strings that are found in
web logs, proxy logs, PCAPs, etc.

This is about 100x slower than uap-r as it's based on javascript
modules that I've built a V8-wrapper around.

It doesn't work on i386 Windows due to v8-library (not the V8-package)
limitations but it works on 64-bit Windows (prbly better off
installing from github for that as CRAN is marking this non-Windows
due to the i386 incompatibility).

If you're on Linux and can deal with a full Boost install and have
need of user agent parsing, use uap-r (it still won't work on Windows
or macOS). Otherwise, give this a go.

I've tested in on a number of OSes but welcome feedback and I'm sure
both Oliver & I would welcome hints on alternatives to Boost regex
that work on all OSes for uap-r since I'd eventually like to replace
this with that.

-Bob

___
R-packages mailing list
r-packa...@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-packages

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Ocr

[boB Rudis]

https://cran.rstudio.com/web/packages/abbyyR/index.html

https://github.com/greenore/ocR

https://electricarchaeology.ca/2014/07/15/doing-ocr-within-r/

that was from a Google "r ocr" search. So, yes, there are options.

On Tue, Jul 26, 2016 at 6:43 PM, Achim Zeileis  wrote:
> On Wed, 27 Jul 2016, Shane Carey wrote:
>
>> Cool, thanks Jim!!
>> I would love to be able to write my own script for this as I have many
>> images/ pdf's in a folder and would like to batch process them using an R
>> script!!
>
>
> The underlying engine is "tesseract" which is also available as a
> command-line tool and on other OSs. In principle, it is not hard to call it
> with a system() command and then readLines() the resulting text. However, it
> might be useful to play with the available options in the GUI first to see
> what works best for your images.
>
>
>> Thanks
>>
>> On Tuesday, July 26, 2016, Jim Lemon  wrote:
>>
>>> Hi Shane,
>>> FreeOCR is a really good place to start.
>>>
>>> http://www.paperfile.net/
>>>
>>> Jim
>>>
>>>
>>> On Wed, Jul 27, 2016 at 6:11 AM, Shane Carey >> > wrote:

 Hi,

 Has anyone ever done any ocr in R?? I have some scanned images that I
>>>
>>> would

 like to convert to text!!
 Thanks


 --
 Le gach dea ghui,
 Shane

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org  mailing list -- To UNSUBSCRIBE and
>>>
>>> more, see

 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
>>>
>>> http://www.R-project.org/posting-guide.html

 and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>>
>>
>> --
>> Le gach dea ghui,
>> Shane
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Please assist me to download this data

[boB Rudis]

Valid parameters for the form would be super-helpful.

On Mon, Jul 25, 2016 at 3:52 PM, Ulrik Stervbo  wrote:
> Hi Christofer,
>
> If you can load all the data into R you don't need to query the website -
> you simply filter the data by your dates.
>
> I think that's the easiest solution.
>
> Best wishes,
> Ulrik
>
> Christofer Bogaso  schrieb am Mo., 25. Juli
> 2016 21:30:
>
>> Hi Ulrik, Thanks for your reply. I am aware of that link and this is a
>> good option. However with this approach, I can not get Data
>> historically. I would like to create some TS in R for each MF there.
>>
>> Any other idea?
>>
>> Thanks,
>>
>> On Tue, Jul 26, 2016 at 12:53 AM, Ulrik Stervbo 
>> wrote:
>> > The easiest might be to download the entire dataset and filter it
>> > appropriately. If I follow your link and press Download, I get the
>> option to
>> > "Download Complete NAV Report in Text Format" and will result in this:
>> > http://portal.amfiindia.com/NAVReport.aspx?type=0
>> >
>> > It is not the easiest format to get into R, and you might have to
>> > pre-process it.
>> >
>> > Best,
>> > Ulrik
>> >
>> > On Mon, 25 Jul 2016 at 20:37 Christofer Bogaso <
>> bogaso.christo...@gmail.com>
>> > wrote:
>> >>
>> >> Hi again,
>> >>
>> >> I am trying to find some way to download all data historically from
>> >> this website "https://www.amfiindia.com/net-asset-value/nav-history;.
>> >>
>> >> Basically what I am trying to do is, I shall have a range of dates and
>> >> for each date I need to download entire dataset programmatically.
>> >>
>> >> Really appreciate if experts here help.
>> >>
>> >> Thanks,
>> >>
>> >> __
>> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> >> https://stat.ethz.ch/mailman/listinfo/r-help
>> >> PLEASE do read the posting guide
>> >> http://www.R-project.org/posting-guide.html
>> >> and provide commented, minimal, self-contained, reproducible code.
>>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Has R recently made performance improvements in accumulation?

[boB Rudis]

Ideally, you would use a more functional programming approach:

minimal <- function(rows, cols){
  x <- matrix(NA_integer_, ncol = cols, nrow = 0)
  for (i in seq_len(rows)){
x <- rbind(x, rep(i, 10))
  }
  x
}

minimaly <- function(rows, cols){
  x <- matrix(NA_integer_, ncol = cols, nrow = 0)
  do.call(rbind, lapply(seq_len(rows), rep, cols))
}

identical(minimal(100, 100), minimaly(100, 100))
# [1] TRUE

microbenchmark(
  .for=minimal(100, 100),
  .lap=minimaly(100, 100)
)

## Unit: microseconds
##  expr minlq  mean   median   uq  max neval cld
## .for 943.936 1062.3710 1416.1399 1120.259 1366.860 4655.322   100   b
## .lap 111.566  118.1945  160.9058  124.520  146.991 2862.391   100  a

On Tue, Jul 19, 2016 at 10:27 AM, Thierry Onkelinx
 wrote:
> Dear Brent,
>
> I can confirm your timings with
>
> library(microbenchmark)
> microbenchmark(
>   mkFrameForLoop(100, 10),
>   mkFrameForLoop(200, 10),
>   mkFrameForLoop(400, 10)
> )
>
> but profiling your code shown that rbind only uses a small fraction on the
> cpu time used by the function.
>
> profvis::profvis({mkFrameForLoop(100, 10)})
>
> So I cleaned your example further into the function below. Now rbind is
> using most of cpu time. And the timings indicate an O(n^2) relation.
>
> minimal <- function(rows, cols){
>   x <- matrix(NA_integer_, ncol = cols, nrow = 0)
>   for (i in seq_len(rows)){
> x <- rbind(x, rep(i, 10))
>   }
> }
>
> profvis::profvis({minimal(1000, 100)})
>
> timing <- microbenchmark(
>   X50 = minimal(50, 50),
>   X100 = minimal(100, 50),
>   X200 = minimal(200, 50),
>   X400 = minimal(400, 50),
>   X800 = minimal(800, 50),
>   X1600 = minimal(1600, 50)
> )
> timing
> Unit: microseconds
>   exprmin lqmean median  uqmax
> neval cld
>X50199.006212.278233.8444235.728247.3770296.987
>   100 a
>   X100565.693593.957827.8733618.835640.1925   2950.139
>   100 a
>   X200   1804.059   1876.390   2166.1106   1903.370   1938.7115   4263.967
>   100 a
>   X400   6453.913   8755.848   8546.4339   8890.884   8961.7535  13259.024
>   100 a
>   X800  30575.048  32913.186  36555.0118  33093.243  34620.5895 178740.765
>   100  b
>  X1600 130976.429 133674.679 151494.6492 135197.087 137327.1235 292291.385
>   100   c
> timing$N <- as.integer(gsub("X", "", levels(timing$expr)))[timing$expr]
> model <- lm(time ~ poly(N, 4), data = timing)
> summary(model)
>
> Call:
> lm(formula = time ~ poly(N, 4), data = timing)
>
> Residuals:
>   Min1QMedian3Q   Max
> -20518162  -3378940   -130815-45881 142183951
>
> Coefficients:
>   Estimate Std. Error t value Pr(>|t|)
> (Intercept)   33303987 843350  39.490   <2e-16 ***
> poly(N, 4)1 1286962014   20657783  62.299   <2e-16 ***
> poly(N, 4)2  338770077   20657783  16.399   <2e-16 ***
> poly(N, 4)3 222734   20657783   0.0110.991
> poly(N, 4)4   -2260902   20657783  -0.1090.913
> ---
> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> Residual standard error: 2066 on 595 degrees of freedom
> Multiple R-squared:  0.8746, Adjusted R-squared:  0.8738
> F-statistic:  1038 on 4 and 595 DF,  p-value: < 2.2e-16
> newdata <- data.frame(N = pretty(timing$N, 40))
> newdata$time <- predict(model, newdata = newdata)
> plot(newdata$N, newdata$time, type = "l")
> plot(newdata$N, sqrt(newdata$time), type = "l")
>
> model2 <- lm(sqrt(time) ~ poly(N, 4), data = timing)
> summary(model2)
> Call:
> lm(formula = sqrt(time) ~ poly(N, 4), data = timing)
>
> Residuals:
>Min 1Q Median 3QMax
> -756.3 -202.8  -54.7   -5.5 7416.5
>
> Coefficients:
>  Estimate Std. Error t value Pr(>|t|)
> (Intercept)   3980.36  33.13 120.160  < 2e-16 ***
> poly(N, 4)1 100395.40 811.41 123.730  < 2e-16 ***
> poly(N, 4)2   2191.34 811.41   2.701  0.00712 **
> poly(N, 4)3   -803.54 811.41  -0.990  0.32243
> poly(N, 4)4 82.09 811.41   0.101  0.91945
> ---
> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> Residual standard error: 811.4 on 595 degrees of freedom
> Multiple R-squared:  0.9626, Adjusted R-squared:  0.9624
> F-statistic:  3829 on 4 and 595 DF,  p-value: < 2.2e-16
>
>
> ir. Thierry Onkelinx
> Instituut voor natuur- en bosonderzoek / Research Institute for Nature and
> Forest
> team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance
> Kliniekstraat 25
> 1070 Anderlecht
> Belgium
>
> To call in the statistician after the experiment is done may be no more
> than asking him to perform a post-mortem examination: he may be able to say
> what the experiment died of. ~ Sir Ronald Aylmer Fisher
> The plural of anecdote is not data. ~ Roger Brinner
> The combination of some data and an aching desire for an answer does not
> ensure that a reasonable answer can be extracted from a given body of data.
> ~ John Tukey
>
> 2016-07-19 15:40 GMT+02:00 Brent via 

Re: [R] Aggregate rainfall data

[boB Rudis]

use `gsub()` after the `as.character()` conversion to remove
everything but valid numeric components from the strings.

On Wed, Jul 13, 2016 at 6:21 AM, roslinazairimah zakaria
 wrote:
> Dear David,
>
> I got your point.  How do I remove the data that contain "0.0?".
>
> I tried : balok <- cbind(balok3[,-5], balok3$Rain.mm[balok3$Rain.mm==0.0?]
> <- NA)
>
> However all the Rain.mm column all become NA.
>
>day month year Time balok3$Rain.mm[balok3$Rain.mm == "0.0?"] <- NA
> 1   30 7 2008  9:00:00 NA
> 2   30 7 2008 10:00:00 NA
> 3   30 7 2008 11:00:00 NA
> 4   30 7 2008 12:00:00 NA
> 5   30 7 2008 13:00:00 NA
> 6   30 7 2008 14:00:00 NA
> 7   30 7 2008 15:00:00 NA
> 8   30 7 2008 16:00:00 NA
> 9   30 7 2008 17:00:00 NA
> 10  30 7 2008 18:00:00 NA
>
> Thank you so much.
>
>
> On Wed, Jul 13, 2016 at 9:42 AM, David Winsemius 
> wrote:
>
>>
>> > On Jul 12, 2016, at 3:45 PM, roslinazairimah zakaria <
>> roslina...@gmail.com> wrote:
>> >
>> > Dear R-users,
>> >
>> > I have these data:
>> >
>> > head(balok, 10); tail(balok, 10)
>> >Date Time Rain.mm
>> > 1  30/7/2008  9:00:00   0
>> > 2  30/7/2008 10:00:00   0
>> > 3  30/7/2008 11:00:00   0
>> > 4  30/7/2008 12:00:00   0
>> > 5  30/7/2008 13:00:00   0
>> > 6  30/7/2008 14:00:00   0
>> > 7  30/7/2008 15:00:00   0
>> > 8  30/7/2008 16:00:00   0
>> > 9  30/7/2008 17:00:00   0
>> > 10 30/7/2008 18:00:00   0
>> >   Date Time Rain.mm
>> > 63667 4/11/2015  3:00:00   0
>> > 63668 4/11/2015  4:00:00   0
>> > 63669 4/11/2015  5:00:00   0
>> > 63670 4/11/2015  6:00:00   0
>> > 63671 4/11/2015  7:00:00   0
>> > 63672 4/11/2015  8:00:00   0
>> > 63673 4/11/2015  9:00:00 0.1
>> > 63674 4/11/2015 10:00:00 0.1
>> > 63675 4/11/2015 11:00:00 0.1
>> > 63676 4/11/2015 12:00:000.1?
>> >
>> >> str(balok)
>> > 'data.frame':   63676 obs. of  3 variables:
>> > $ Date   : Factor w/ 2654 levels "1/1/2009","1/1/2010",..: 2056 2056 2056
>> > 2056 2056 2056 2056 2056 2056 2056 ...
>> > $ Time   : Factor w/ 24 levels "1:00:00","10:00:00",..: 24 2 3 4 5 6 7 8
>> 9
>> > 10 ...
>> > $ Rain.mm: Factor w/ 352 levels "0","0.0?","0.1",..: 1 1 1 1 1 1 1 1 1 1
>>
>> Thar's your problem:
>>
>>   Rain.mm: Factor w/ 352 levels "0","0.0?","0.1"
>>
>> Need to use the standard fix for the screwed-up-factor-on-input-problem
>>
>>   balok$Rain.mm2 <- as.numeric( as.character(balok$Rain.mm) )
>>
>> Cannot just do as.numeric because factors are actually already numeric.
>>
>> --
>> David.
>>
>>
>> > ...
>> >
>> > and I have change the data as follows:
>> >
>> > realdate <- as.Date(balok$Date,format="%d/%m/%Y")
>> > dfdate <- data.frame(date=realdate)
>> > year=as.numeric (format(realdate,"%Y"))
>> > month=as.numeric (format(realdate,"%m"))
>> > day=as.numeric (format(realdate,"%d"))
>> >
>> > balok2 <-cbind(dfdate,day,month,year,balok[,2:3])
>> > colnames(balok2)
>> > head(balok2)
>> >date day month year Time Rain.mm
>> > 1 2008-07-30  30 7 2008  9:00:00   0
>> > 2 2008-07-30  30 7 2008 10:00:00   0
>> > 3 2008-07-30  30 7 2008 11:00:00   0
>> > 4 2008-07-30  30 7 2008 12:00:00   0
>> > 5 2008-07-30  30 7 2008 13:00:00   0
>> > 6 2008-07-30  30 7 2008 14:00:00   0
>> > ...
>> >
>> >> balok3 <- balok2[,-1]; head(balok3, n=100)
>> >day month year Time Rain.mm
>> > 130 7 2008  9:00:00   0
>> > 230 7 2008 10:00:00   0
>> > 330 7 2008 11:00:00   0
>> > 430 7 2008 12:00:00   0
>> > 530 7 2008 13:00:00   0
>> > 630 7 2008 14:00:00   0
>> > 730 7 2008 15:00:00   0
>> > 830 7 2008 16:00:00   0
>> > 930 7 2008 17:00:00   0
>> > 10   30 7 2008 18:00:00   0
>> > 11   30 7 2008 19:00:00   0
>> > 12   30 7 2008 20:00:00   0
>> > 13   30 7 2008 21:00:00   0
>> > 14   30 7 2008 22:00:00   0
>> > 15   30 7 2008 23:00:00   0
>> > 16   30 7 2008 24:00:00   0
>> > 17   31 7 2008  1:00:00   0
>> > 18   31 7 2008  2:00:00   0
>> > 19   31 7 2008  3:00:00   0
>> > 20   31 7 2008  4:00:00   0
>> > 21   31 7 2008  5:00:00   0
>> > 22   31 7 2008  6:00:00   0
>> > 23   31 7 2008  7:00:00   0
>> > 24   31 7 2008  8:00:00   0
>> > 25   31 7 2008  9:00:00   0
>> > 26   31 7 2008 10:00:00   0
>> > 27   31 

Re: [R] Can R read Word fonts and comments?

[boB Rudis]

I'll dig into that (was hoping the small feature addition wld cause
enhanced feature requests :-)

On Tue, Jul 5, 2016 at 1:02 PM, John <miao...@gmail.com> wrote:
> Thank you, David and Bert, for the info.
> Thank you, Bob, for this excellent function. Allow me to request a feature:
> You highlighted the following text, and comment "This is the first comment".
>
> "Lorem ipsum dolor sit amet, cu sit modus voluptua accommodare, meis
> disputando voluptatibus eu nec, qui te modo solum delicata. Eam scripta
> maluisset urbanitas et, numquam disputationi in pri, vis tibique deserunt
> accusamus ut. Vis movet admodum probatus cu, ex pri ludus possit. Molestiae
> efficiendi at vix, eu labore elaboraret deterruisset mei, et eos persius
> nominati."
>
> Could you let the function output the above text (with the comments, of
> course), which you highlighted for comment?
>
> Thanks,
>
> John
>
>
>
> 2016-07-02 14:12 GMT-07:00 boB Rudis <b...@rudis.net>:
>>
>> I just added `docx_extract_all_cmnts()` (and a cpl other
>> comments-related things) to the dev version of `docxtractr`
>> (https://github.com/hrbrmstr/docxtractr). You can use
>> `devtools::install_github("hrbrmstr/docxtractr")` to install it.
>> There's an example in the help for that function.
>>
>> Give it a go and file detailed issues for other functionality you need.
>>
>> On Fri, Jul 1, 2016 at 11:14 PM, David Winsemius <dwinsem...@comcast.net>
>> wrote:
>> > It’s my understanding that docx and xlsx files are zipped containers
>> > that have their data in XML files. You should try unzipping one and
>> > examining it with a viewer. You may then be able to use pkg:XML.
>> >
>> > —
>> > David.
>> >
>> >> On Jul 1, 2016, at 3:13 PM, Bert Gunter <bgunter.4...@gmail.com> wrote:
>> >>
>> >> No, sorry -- all I would do is search.
>> >>
>> >> -- Bert
>> >>
>> >>
>> >> Bert Gunter
>> >>
>> >> "The trouble with having an open mind is that people keep coming along
>> >> and sticking things into it."
>> >> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>> >>
>> >>
>> >> On Fri, Jul 1, 2016 at 2:33 PM, John <miao...@gmail.com> wrote:
>> >>> Yes, I have done some search (e.g., tm, markdown, etc), but I can't
>> >>> find
>> >>> this function.
>> >>> If you know any package that works for this purpose, that would be
>> >>> quite
>> >>> helpful.
>> >>> Thanks,
>> >>>
>> >>> John
>> >>>
>> >>> 2016-06-28 16:50 GMT-07:00 Bert Gunter <bgunter.4...@gmail.com>:
>> >>>>
>> >>>> Did you try searching before posting here? -- e.g. a web search or on
>> >>>> rseek.org ?
>> >>>>
>> >>>> Cheers,
>> >>>> Bert
>> >>>>
>> >>>>
>> >>>> Bert Gunter
>> >>>>
>> >>>> "The trouble with having an open mind is that people keep coming
>> >>>> along
>> >>>> and sticking things into it."
>> >>>> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>> >>>>
>> >>>>
>> >>>> On Tue, Jun 28, 2016 at 3:53 PM, John <miao...@gmail.com> wrote:
>> >>>>> Hi,
>> >>>>>
>> >>>>>   From time to time I highlight the word documents with red/blue
>> >>>>> color
>> >>>>> or
>> >>>>> italic/bold fonts, and I also add comments to a file. Is there a
>> >>>>> package/function to let R extract the italic/bold blue/red words and
>> >>>>> comments from a docx/doc file?
>> >>>>>
>> >>>>>   I am aware that there are a few packages reading Word, but don't
>> >>>>> know
>> >>>>> which one is able to do it.
>> >>>>>
>> >>>>>   Thanks,
>> >>>>>
>> >>>>> John
>> >>>>>
>> >>>>>[[alternative HTML version deleted]]
>> >>>>>
>> >>>>> __
>> >>>>> R-help@r-project.org mailing l

Re: [R] Can R read Word fonts and comments?

[boB Rudis]

I just added `docx_extract_all_cmnts()` (and a cpl other
comments-related things) to the dev version of `docxtractr`
(https://github.com/hrbrmstr/docxtractr). You can use
`devtools::install_github("hrbrmstr/docxtractr")` to install it.
There's an example in the help for that function.

Give it a go and file detailed issues for other functionality you need.

On Fri, Jul 1, 2016 at 11:14 PM, David Winsemius  wrote:
> It’s my understanding that docx and xlsx files are zipped containers that 
> have their data in XML files. You should try unzipping one and examining it 
> with a viewer. You may then be able to use pkg:XML.
>
> —
> David.
>
>> On Jul 1, 2016, at 3:13 PM, Bert Gunter  wrote:
>>
>> No, sorry -- all I would do is search.
>>
>> -- Bert
>>
>>
>> Bert Gunter
>>
>> "The trouble with having an open mind is that people keep coming along
>> and sticking things into it."
>> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>>
>>
>> On Fri, Jul 1, 2016 at 2:33 PM, John  wrote:
>>> Yes, I have done some search (e.g., tm, markdown, etc), but I can't find
>>> this function.
>>> If you know any package that works for this purpose, that would be quite
>>> helpful.
>>> Thanks,
>>>
>>> John
>>>
>>> 2016-06-28 16:50 GMT-07:00 Bert Gunter :

 Did you try searching before posting here? -- e.g. a web search or on
 rseek.org ?

 Cheers,
 Bert


 Bert Gunter

 "The trouble with having an open mind is that people keep coming along
 and sticking things into it."
 -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


 On Tue, Jun 28, 2016 at 3:53 PM, John  wrote:
> Hi,
>
>   From time to time I highlight the word documents with red/blue color
> or
> italic/bold fonts, and I also add comments to a file. Is there a
> package/function to let R extract the italic/bold blue/red words and
> comments from a docx/doc file?
>
>   I am aware that there are a few packages reading Word, but don't know
> which one is able to do it.
>
>   Thanks,
>
> John
>
>[[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] merging df with world map

[boB Rudis]

you also don't need to do a merger if you use a base `geom_map()`
layer with the polygons and another using the fill (or points, lines,
etc).

On Fri, Jun 17, 2016 at 5:08 PM, MacQueen, Don  wrote:
> And you can check what David and Jeff suggested like this:
>
> intersect( df$COUNTRY, world_map$region )
>
> If they have any values in common, that command will show them. (Note that
> I said values in common, not countries in common.)
>
> WARNING:
> It appears that you have each country appearing more than once in both of
> the data frames. Even if the country names were spelled the same (which
> they are not in the first few rows), I would not care to predict the
> outcome of a many-to-many merge. It probably won't make sense for showing
> the data on a map.
>
> -Don
>
> --
> Don MacQueen
>
> Lawrence Livermore National Laboratory
> 7000 East Ave., L-627
> Livermore, CA 94550
> 925-423-1062
>
>
>
>
>
> On 6/17/16, 1:06 PM, "R-help on behalf of ch.elahe via R-help"
>  wrote:
>
>>Hi all,
>>I want to use world map in ggplot2 and show my data on world map. my df
>>is:
>>
>>
>>$ COUNTRY   : chr  "DE" "DE" "FR" "FR" ..
>>
>>$ ContrastColor : int  9 9 9 9 13 9 9 9 9 ..
>>
>>$ quant : Factor w/ 4 levels "FAST","SLOW",..I need to
>>merge my df with world_map data which is like this:
>>
>>
>>world_map=map_data("world")
>>data.frame':   99338 obs. of  6 variables:
>>$ long : num  -69.9 -69.9 -69.9 -70 -70.1 ...
>>$ lat  : num  12.5 12.4 12.4 12.5 12.5 ...
>>$ group: num  1 1 1 1 1 1 1 1 1 1 ...
>>$ order: int  1 2 3 4 5 6 7 8 9 10 ...
>>$ region   : chr  "Aruba" "Aruba" "Aruba" "Aruba" ...
>>$ subregion: chr  NA NA NA NA ...
>>but by merging my df with world map data I get a data frame with zero
>>observation in it,I use this command for merging:
>>
>>
>>world_map=merge(world_map,df,by.x="region",by.y="COUNTRY")
>>str(world_map)
>>
>>'data.frame':   0 obs. of  133 variables:
>>$ region: chr
>>$ long  : num
>>$ lat   : num
>>$ group : num
>>$ order : int
>>$ subregion : chr
>>does anyone know what is the problem of this merging that I am currently
>>using?
>>thanks for any help!
>>Elahe
>>
>>__
>>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide
>>http://www.R-project.org/posting-guide.html
>>and provide commented, minimal, self-contained, reproducible code.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Kendall heat map

[boB Rudis]

Did you try:

cor(mat, method="kendall", use="pairwise")

That only provides the matrix (so the equiv of the $r list component),
but that seems to be all you need.

On Fri, Jun 17, 2016 at 5:47 AM, Shane Carey  wrote:
> Hi,
>
> I was hoping someone could help me. I was wondering are there any libraries
> available to undertake a kendall correlation on a matrix of data, in the
> same way as what can be undertaken with the rcorr function:
>
> cormatrix = rcorr(as.matrix(A), type='spearman')
> cordata = melt(cormatrix$r)
> ggplot(cordata, aes(x=Var1, y=Var2, fill=value)) +
>   geom_tile() + xlab("") + ylab("")
>
> Thanks
>
> --
> Le gach dea ghui,
> Shane
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Help: How to Convert Binary Data into Text Using R

[boB Rudis]

Welcome to R and R-help!

It would help others help you if you provided a minimal example and
explained your situation with a bit more details. It's pretty vague as
it stands.

Base R has both a `readBin()` and `rawConnection()` functions (amongst
other tools for such things) and there are a few packages that also
help with reading "binary" data. But, without knowing more specifics,
that's about as much direction as any of us wld be able to give.

-Bob

On Sat, Jun 11, 2016 at 9:06 PM, Fahman Khan via R-help
 wrote:
> Good Evening,
> Just started learning R and one of the task given to me is to convert Binary 
> Data into text. I'm not sure what package i'm suppose to use. Can i get an 
> idea of how i can convert binary into text.
> Regards,Fahman Khan
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Map of Italy data filled at the level of the province

[boB Rudis]

This should help you get started:

  library(maptools)
  library(ggplot2)
  library(ggalt)
  library(ggthemes)
  library(tibble)
  library(viridis)

  # get italy region map
  italy_map <- map_data("italy")

  # your data will need to have these region names
  print(unique(italy_map$region))

  # we'll simulate some data for this
  set.seed(1492)
  choro_dat <- data_frame(region=unique(italy_map$region),
  value=sample(100, length(region)))

  # we'll use this in a bit
  italy_proj <- "+proj=aea +lat_1=38.15040684902542
+lat_2=44.925490198742295 +lon_0=12.7880859375"

  gg <- ggplot()

  # lay down the base layer
  gg <- gg + geom_map(data=italy_map, map=italy_map,
  aes(long, lat, map_id=region),
  color="#b2b2b2", size=0.1, fill=NA)

  # fill in the regions with the data
  gg <- gg + geom_map(data=choro_dat, map=italy_map,
  aes(fill=value, map_id=region),
  color="#b2b2b2", size=0.1)

  # great color palette (use a better legend title)
  gg <- gg + scale_fill_viridis(name="Scale title")

  # decent map projection for italy choropleth
  gg <- gg + coord_proj(italy_proj)

  # good base theme for most maps
  gg <- gg + theme_map()

  # move the legend
  gg <- gg + theme(legend.position=c(0.95, 0.3))

  gg

This uses a continuous color palette for the region fill. You may want
to consider binning data and using a discrete fill (IMO that's usually
a better choice for most choropleths).

-Bob

On Thu, Jun 2, 2016 at 5:37 AM, francesca Pancotto
 wrote:
> Dear Users
> I am very new to the use of ggplot. I am supposed to make a plot of
> Italian provinces in which I have to fill the color of some provinces
> with the values of a variable(I do not provide the data because it is 
> irrelevant which data to use).
>
> Right now I explored the function map in maps package thanks to which I 
> managed to plot
> the map of Italy with provinces borders and select only those provinces 
> contained in the
> vector nomi(which is just a list of character elements with the names of the 
> provinces which are
> just like counties in the US).
>
> map("italy",col=1:20, regions=nomi)
>
> The problem is to fill the provinces level with the values of a variable that 
> is the variable of interest:
> I found a series of examples based on US data extracted from very hard to get 
> databases.
>
> Can anyone provide an easy example where to start from?
>
> Thanks in advance
> Francesca
>
> --
> Francesca Pancotto
> Professore Associato di Politica Economica
> Università degli Studi di Modena e Reggio Emilia
> Palazzo Dossetti - Viale Allegri, 9 - 42121 Reggio Emilia
> Office: +39 0522 523264
> Web: https://sites.google.com/site/francescapancotto/
> --
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to replace all commas with semicolon in a string

[boB Rudis]

You can use gsub() instead of sub()

On Fri, May 27, 2016 at 11:10 AM, Jun Shen  wrote:
> Dear list,
>
> Say I have a data frame
>
> test <- data.frame(C1=c('a,b,c,d'),C2=c('g,h,f'))
>
> I want to replace the commas with semicolons
>
> sub(',',';',test$C1) -> test$C1 will only replace the first comma of a string.
>
> How do I replace them all in one run? Thanks.
>
> Jun
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R help - Web Scraping of Google News using R

[boB Rudis]

What you are doing wrong is both trying yourself and asking others to
violate Google's Terms of Service and (amongst other things) get your
IP banned along with anyone who aids you (or worse). Please don't.
Just because something can be done does not mean it should be done.

On Tue, May 24, 2016 at 11:21 AM, Kumar Gauraw  wrote:
> Hello Experts,
>
> I am trying to scrap data from Google news for a particular topic using XML
> and Curl Package of R. I am able to extract the summary part of the news
> through *XPath* but in a similar way, I am trying to extract title and
> Links of news which is not working.Please note this work is just for POC
> purpose and I would make maximum of 500 requests per day so that Google TOS
> remains intact.
>
>
> library(XML)
>
> library(RCurl)
>
> getGoogleURL <- function(search.term, domain = '.co.in', quotes=TRUE)
>
> {
>
>   search.term <- gsub(' ', '%20', search.term)
>
>   if(quotes) search.term <- paste('%22', search.term, '%22', sep='')
>
>   getGoogleURL <- paste('http://www.google', domain,
> '/search?hl=en=in=nws=0=',search.term, sep='')
>
> }
>
> search.term <- "IPL 2016"
>
> quotes <- "FALSE"
>
> search.url <- getGoogleURL(search.term=search.term, quotes=quotes)
>
> getGoogleSummary <- function(google.url) {
>
>   doc <- getURL(google.url, httpheader = c("User-Agent" = "R(2.10.0)"))
>
>   html <- htmlTreeParse(doc, useInternalNodes = TRUE, error=function(...){})
>
>   nodes <- getNodeSet(html, "//div[@class='st']")
>
>   return(sapply(nodes, function(x) x <- xmlValue(x)))
>
> }
>
> *#Problem is with this part of code*
>
> getGoogleTitle <- function(google.url) {
>
>   doc <- getURL(google.url, httpheader = c("User-Agent" = "R(2.10.0)"))
>
>   html <- htmlTreeParse(doc, useInternalNodes = TRUE, error=function(...){})
>
>  * nodes <- getNodeSet(html, "//a[@class='l _HId']")*
>
>   return(sapply(nodes, function(x) x <- xmlValue(x)))
>
> }
>
> Kindly help me to understand where I am getting wrong so that I can rectify
> the code and get the correct output.
>
> Thank you.
>
> With Regards,
> Kumar Gauraw
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] web scraping tables generated in multiple server pages

[boB Rudis]

I upgraded ffox to the 46-series and intermittently received the same
error. But by adding a `Sys.sleep(1)` to the final `if`:

  if ((i %% 10) == 0) {
ref <- remDr$findElements("xpath", ".//a[.='...']")
ref[[length(ref)]]$clickElement()
Sys.sleep(1)
  }

I was able to reproduce my original, successful outcome. I think it
has something to do with the page not being fully loaded when the the
driver tries to get the page content. Go multithreading! My choice of
1s was arbitrary. Longer == better chance of it working more often.

This 
<http://stackoverflow.com/questions/27080920/how-to-check-if-page-finished-loading-in-rselenium>
would probably also be better (waiting for a full page load signal),
but I try to not use [R]Selenium at all if it can be helped.

-Bob



On Wed, May 11, 2016 at 2:00 PM, boB Rudis <b...@rudis.net> wrote:
> Hey David,
>
> I'm on a Mac as well but have never had to tweak anything to get
> [R]Selenium to work (but this is one reason I try to avoid solutions
> involving RSelenium as they are pretty fragile IMO).
>
> The site itself has "Página 1 de 69" at the top which is where i got
> the "69" from and I just re-ran the code in a 100% clean env (on a
> completely different Mac) and it worked fine.
>
> I did neglect to put my session info up before (apologies):
>
> Session info
> 
>  setting  value
>  version  R version 3.3.0 RC (2016-05-01 r70572)
>  system   x86_64, darwin13.4.0
>  ui   RStudio (0.99.1172)
>  language (EN)
>  collate  en_US.UTF-8
>  tz   America/New_York
>  date 2016-05-11
>
> Packages 
> 
>  package* version  date   source
>  assertthat   0.1  2013-12-06 CRAN (R 3.3.0)
>  bitops * 1.0-62013-08-17 CRAN (R 3.3.0)
>  caTools  1.17.1   2014-09-10 CRAN (R 3.3.0)
>  DBI  0.4  2016-05-02 CRAN (R 3.3.0)
>  devtools   * 1.11.1   2016-04-21 CRAN (R 3.3.0)
>  digest   0.6.92016-01-08 CRAN (R 3.3.0)
>  dplyr  * 0.4.32015-09-01 CRAN (R 3.3.0)
>  httr 1.1.02016-01-28 CRAN (R 3.3.0)
>  magrittr 1.5  2014-11-22 CRAN (R 3.3.0)
>  memoise  1.0.02016-01-29 CRAN (R 3.3.0)
>  pbapply* 1.2-12016-04-19 CRAN (R 3.3.0)
>  R6   2.1.22016-01-26 CRAN (R 3.3.0)
>  Rcpp 0.12.4   2016-03-26 CRAN (R 3.3.0)
>  RCurl  * 1.95-4.8 2016-03-01 CRAN (R 3.3.0)
>  RJSONIO* 1.3-02014-07-28 CRAN (R 3.3.0)
>  RSelenium  * 1.3.52014-10-26 CRAN (R 3.3.0)
>  rvest  * 0.3.12015-11-11 CRAN (R 3.3.0)
>  selectr  0.2-32014-12-24 CRAN (R 3.3.0)
>  stringi  1.0-12015-10-22 CRAN (R 3.3.0)
>  stringr  1.0.02015-04-30 CRAN (R 3.3.0)
>  withr1.0.12016-02-04 CRAN (R 3.3.0)
>  XML* 3.98-1.4 2016-03-01 CRAN (R 3.3.0)
>  xml2   * 0.1.22015-09-01 CRAN (R 3.3.0)
>
> (and, wow, does that tiny snippet of code end up using alot of pkgs)
>
> I had actually started with smaller snippets to test. The code got
> uglier due to the way the site paginates (it loads 10-entries worth of
> data on to a single page but requires a server call for the next 10).
>
> I also keep firefox scarily out-of-date (back in the 33's rev) b/c I
> only use it with RSelenium (not a big fan of the browser). Let me
> update to the 46-series and see if I can replicate.
>
> -Bob
>
> On Wed, May 11, 2016 at 1:48 PM, David Winsemius <dwinsem...@comcast.net> 
> wrote:
>>
>>> On May 10, 2016, at 1:11 PM, boB Rudis <b...@rudis.net> wrote:
>>>
>>> Unfortunately, it's a wretched, vile, SharePoint-based site. That
>>> means it doesn't use traditional encoding methods to do the pagination
>>> and one of the only ways to do this effectively is going to be to use
>>> RSelenium:
>>>
>>>library(RSelenium)
>>>library(rvest)
>>>library(dplyr)
>>>library(pbapply)
>>>
>>>URL <- 
>>> "http://outorgaonerosa.prefeitura.sp.gov.br/relatorios/RelSituacaoGeralProcessos.aspx;
>>>
>>>checkForServer()
>>>startServer()
>>>remDr <- remoteDriver$new()
>>>remDr$open()
>>
>> Thanks Bob/hrbrmstr;
>>
>> At this point I got an error:
>>
>>>startServer()
>>>remDr <- remoteDriver$new()
>>>remDr$open()
>> [1] "Connecting 

Re: [R] web scraping tables generated in multiple server pages

[boB Rudis]

Hey David,

I'm on a Mac as well but have never had to tweak anything to get
[R]Selenium to work (but this is one reason I try to avoid solutions
involving RSelenium as they are pretty fragile IMO).

The site itself has "Página 1 de 69" at the top which is where i got
the "69" from and I just re-ran the code in a 100% clean env (on a
completely different Mac) and it worked fine.

I did neglect to put my session info up before (apologies):

Session info

 setting  value
 version  R version 3.3.0 RC (2016-05-01 r70572)
 system   x86_64, darwin13.4.0
 ui   RStudio (0.99.1172)
 language (EN)
 collate  en_US.UTF-8
 tz   America/New_York
 date 2016-05-11

Packages 

 package* version  date   source
 assertthat   0.1  2013-12-06 CRAN (R 3.3.0)
 bitops * 1.0-62013-08-17 CRAN (R 3.3.0)
 caTools  1.17.1   2014-09-10 CRAN (R 3.3.0)
 DBI  0.4  2016-05-02 CRAN (R 3.3.0)
 devtools   * 1.11.1   2016-04-21 CRAN (R 3.3.0)
 digest   0.6.92016-01-08 CRAN (R 3.3.0)
 dplyr  * 0.4.32015-09-01 CRAN (R 3.3.0)
 httr 1.1.02016-01-28 CRAN (R 3.3.0)
 magrittr 1.5  2014-11-22 CRAN (R 3.3.0)
 memoise  1.0.02016-01-29 CRAN (R 3.3.0)
 pbapply* 1.2-12016-04-19 CRAN (R 3.3.0)
 R6   2.1.22016-01-26 CRAN (R 3.3.0)
 Rcpp 0.12.4   2016-03-26 CRAN (R 3.3.0)
 RCurl  * 1.95-4.8 2016-03-01 CRAN (R 3.3.0)
 RJSONIO* 1.3-02014-07-28 CRAN (R 3.3.0)
 RSelenium  * 1.3.52014-10-26 CRAN (R 3.3.0)
 rvest  * 0.3.12015-11-11 CRAN (R 3.3.0)
 selectr  0.2-32014-12-24 CRAN (R 3.3.0)
 stringi  1.0-12015-10-22 CRAN (R 3.3.0)
 stringr  1.0.02015-04-30 CRAN (R 3.3.0)
 withr1.0.12016-02-04 CRAN (R 3.3.0)
 XML* 3.98-1.4 2016-03-01 CRAN (R 3.3.0)
 xml2   * 0.1.22015-09-01 CRAN (R 3.3.0)

(and, wow, does that tiny snippet of code end up using alot of pkgs)

I had actually started with smaller snippets to test. The code got
uglier due to the way the site paginates (it loads 10-entries worth of
data on to a single page but requires a server call for the next 10).

I also keep firefox scarily out-of-date (back in the 33's rev) b/c I
only use it with RSelenium (not a big fan of the browser). Let me
update to the 46-series and see if I can replicate.

-Bob

On Wed, May 11, 2016 at 1:48 PM, David Winsemius <dwinsem...@comcast.net> wrote:
>
>> On May 10, 2016, at 1:11 PM, boB Rudis <b...@rudis.net> wrote:
>>
>> Unfortunately, it's a wretched, vile, SharePoint-based site. That
>> means it doesn't use traditional encoding methods to do the pagination
>> and one of the only ways to do this effectively is going to be to use
>> RSelenium:
>>
>>library(RSelenium)
>>library(rvest)
>>library(dplyr)
>>library(pbapply)
>>
>>URL <- 
>> "http://outorgaonerosa.prefeitura.sp.gov.br/relatorios/RelSituacaoGeralProcessos.aspx;
>>
>>checkForServer()
>>startServer()
>>remDr <- remoteDriver$new()
>>remDr$open()
>
> Thanks Bob/hrbrmstr;
>
> At this point I got an error:
>
>>startServer()
>>remDr <- remoteDriver$new()
>>remDr$open()
> [1] "Connecting to remote server"
> Undefined error in RCurl call.Error in queryRD(paste0(serverURL, "/session"), 
> "POST", qdata = toJSON(serverOpts)) :
>
> Running R 3.0.0 on a Mac (El Cap) in the R.app GUI.
> $ java -version
> java version "1.8.0_65"
> Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)
>
> I asked myself: What additional information is needed to debug this? But then 
> I thought I had a responsibility to search for earlier reports of this error 
> on a Mac, and there were many. After reading this thread: 
> https://github.com/ropensci/RSelenium/issues/54  I decided to try creating an 
> "alias", mac-speak for a symlink, and put that symlink in my working 
> directory (with no further chmod security efforts). I restarted R and re-ran 
> the code which opened a Firefox browser window and then proceeded to page 
> through many pages. Eventually, however it errors out with this message:
>
>>pblapply(1:69, function(i) {
> +
> +  if (i %in% seq(1, 69, 10)) {
> +pg <- read_html(remDr$getPageSource()[[1]])
> +ret <- html_table(html_nodes(pg, "table")[[3]], header=TRUE)
> +
> + 

Re: [R] web scraping tables generated in multiple server pages

[boB Rudis]

Unfortunately, it's a wretched, vile, SharePoint-based site. That
means it doesn't use traditional encoding methods to do the pagination
and one of the only ways to do this effectively is going to be to use
RSelenium:

library(RSelenium)
library(rvest)
library(dplyr)
library(pbapply)

URL <- 
"http://outorgaonerosa.prefeitura.sp.gov.br/relatorios/RelSituacaoGeralProcessos.aspx;

checkForServer()
startServer()
remDr <- remoteDriver$new()
remDr$open()

remDr$navigate(URL)

pblapply(1:69, function(i) {

  if (i %in% seq(1, 69, 10)) {

# the first item on the page is not a link but we can just grab the page

pg <- read_html(remDr$getPageSource()[[1]])
ret <- html_table(html_nodes(pg, "table")[[3]], header=TRUE)

  } else {

# we can get the rest of them by the link text directly

ref <- remDr$findElements("xpath",
sprintf(".//a[contains(@href, 'javascript:__doPostBack') and .='%s']",
i))
ref[[1]]$clickElement()
pg <- read_html(remDr$getPageSource()[[1]])
ret <- html_table(html_nodes(pg, "table")[[3]], header=TRUE)

  }

  # we have to move to the next actual page of data after every 10 links

  if ((i %% 10) == 0) {
ref <- remDr$findElements("xpath", ".//a[.='...']")
ref[[length(ref)]]$clickElement()
  }

  ret

}) -> tabs

final_dat <- bind_rows(tabs)
final_dat <- final_dat[, c(1, 2, 5, 7, 8, 13, 14)] # the cols you want
final_dat <- final_dat[complete.cases(final_dat),] # take care of NAs

remDr$quit()


Prbly good ref code to have around, but you can grab the data & code
here: https://gist.github.com/hrbrmstr/ec35ebb32c3cf0aba95f7bad28df1e98

(anything to help a fellow parent out :-)

-Bob

On Tue, May 10, 2016 at 2:45 PM, Michael Friendly  wrote:
> This is my first attempt to try R web scraping tools, for a project my
> daughter is working on.  It concerns a data base of projects in Sao
> Paulo, Brazil, listed at
> http://outorgaonerosa.prefeitura.sp.gov.br/relatorios/RelSituacaoGeralProcessos.aspx,
> but spread out over 69 pages accessed through a javascript menu at the
> bottom of the page.
>
> Each web page contains 3 HTML tables, of which only the last contains
> the relevant data.  In this, only a subset of columns are of interest.
> I tried using the XML package as illustrated on several tutorial pages,
> as shown below.  I have no idea how to automate this to extract these
> tables from multiple web pages.  Is there some other package better
> suited to this task?  Can someone help me solve this and other issues?
>
> # Goal: read the data tables contained on 69 pages generated by the link
> below, where
> # each page is generated by a javascript link in the menu of the bottom
> of the page.
> #
> # Each "page" contains 3 html tables, with names "Table 1", "Table 2",
> and the only one
> # of interest with the data, "grdRelSitGeralProcessos"
> #
> # From each such table, extract the following columns:
> #- Processo
> #- Endereço
> #- Distrito
> #- Area terreno (m2)
> #- Valor contrapartida ($)
> #- Area excedente (m2)
>
> # NB: All of the numeric fields use "." as comma-separator and "," as
> the decimal separator,
> #   but because of this are read in as character
>
>
> library(XML)
> link <-
> "http://outorgaonerosa.prefeitura.sp.gov.br/relatorios/RelSituacaoGeralProcessos.aspx;
>
> saopaulo <- htmlParse(link)
> saopaulo.tables <- readHTMLTable(saopaulo, stringsAsFactors = FALSE)
> length(saopaulo.tables)
>
> # its the third table on this page we want
> sp.tab <- saopaulo.tables[[3]]
>
> # columns wanted
> wanted <- c(1, 2, 5, 7, 8, 13, 14)
> head(sp.tab[, wanted])
>
>  > head(sp.tab[, wanted])
>Proposta Processo EndereçoDistrito
> 11 2002-0.148.242-4 R. DOMINGOS LOPES DA SILVA X R. CORNÉLIO
> VAN CLEVEVILA ANDRADE
> 22 2003-0.129.667-3  AV. DR. JOSÉ HIGINO,
> 200 E 216   AGUA RASA
> 33 2003-0.065.011-2   R. ALIANÇA LIBERAL,
> 980 E 990 VILA LEOPOLDINA
> 44 2003-0.165.806-0   R. ALIANÇA LIBERAL,
> 880 E 886 VILA LEOPOLDINA
> 55 2003-0.139.053-0R. DR. JOSÉ DE ANDRADE
> FIGUEIRA, 111VILA ANDRADE
> 66 2003-0.200.692-0R. JOSÉ DE
> JESUS, 66  VILA SONIA
>Ã rea Terreno (m2) Ã rea Excedente (m2) Valor Contrapartida (R$)
> 1   0,00 1.551,14 127.875,98
> 2   0,00 3.552,13 267.075,77
> 3   0,00   624,99 70.212,93
> 4   0,00   395,64 44.447,18
> 5   0,00   719,68 41.764,46
> 6   0,00   446,52 85.152,92
>
> thanks,
>
>
> --
> Michael Friendly Email: friendly AT yorku DOT ca
> Professor, Psychology Dept. & Chair, Quantitative Methods
> York University  Voice: 416 736-2100 x66249 

Re: [R] Assistance with httr package with R version 3.3.0

[boB Rudis]

I don't fully remember, but I doubt httr::content() ever returned a
character vector without using the `as="text"` parameter. Try
switching that line to:

html <- content(r, as="text")



On Tue, May 10, 2016 at 3:27 AM, Luca Meyer  wrote:
> Hi Jim,
>
> Thank you for your suggestion. I have actually tried to upload XML and xml2
> but nothing changed...any other suggestion?
>
> Kind regards,
>
> Luca
>
>> rm(list=ls())
>> library(httr)
>> library(XML)
>> library(xml2)
>>
>> #carico i dati da Google spreadsheets
>> url <- "
> https://docs.google.com/spreadsheets/d/102-jJ7x1YfIe4Kkvb9olQ4chQ_TS90jxoU0vAbFZewc/pubhtml?gid=0=true
> "
>> readSpreadsheet <- function(url, sheet = 1){
> +   r <- GET(url)
> +   html <- content(r)
> +   sheets <- readHTMLTable(html, header=FALSE, stringsAsFactors=FALSE)
> +   df <- sheets[[sheet]]
> +   dfClean <- function(df){
> + nms <- t(df[1,])
> + names(df) <- nms
> + df <- df[-1,-1]
> + row.names(df) <- seq(1,nrow(df))
> + df
> +   }
> +   dfClean(df)
> + }
>> dati <- readSpreadsheet(url)
> Error in (function (classes, fdef, mtable)  :
>   unable to find an inherited method for function ‘readHTMLTable’ for
> signature ‘"xml_document"’
>> rm(readSpreadsheet,url)
>
> 2016-05-10 8:52 GMT+02:00 Jim Lemon :
>
>> Hi Luca,
>> The function readHTMLtable is in the XML package, not httr. Perhaps
>> that is the problem as I don't see a dependency in httr for XML
>> (although xml2 is suggested).
>>
>> Jim
>>
>>
>> On Tue, May 10, 2016 at 2:58 PM, Luca Meyer  wrote:
>> > Hello,
>> >
>> > I am trying to run a code I have been using for a few years now after
>> > downloading the new R version 3.3.0 and I get the following error:
>> >
>> >> rm(list=ls())
>> >> library(httr)
>> >>
>> >> #carico i dati da Google spreadsheets
>> >> url <- "
>> >
>> https://docs.google.com/spreadsheets/d/102-jJ7x1YfIe4Kkvb9olQ4chQ_TS90jxoU0vAbFZewc/pubhtml?gid=0=true
>> > "
>> >> readSpreadsheet <- function(url, sheet = 1){
>> > +   r <- GET(url)
>> > +   html <- content(r)
>> > +   sheets <- readHTMLTable(html, header=FALSE, stringsAsFactors=FALSE)
>> > +   df <- sheets[[sheet]]
>> > +   dfClean <- function(df){
>> > + nms <- t(df[1,])
>> > + names(df) <- nms
>> > + df <- df[-1,-1]
>> > + row.names(df) <- seq(1,nrow(df))
>> > + df
>> > +   }
>> > +   dfClean(df)
>> > + }
>> >> dati <- readSpreadsheet(url)
>> > Error in (function (classes, fdef, mtable)  :
>> >   unable to find an inherited method for function ‘readHTMLTable’ for
>> > signature ‘"xml_document"’
>> >> rm(readSpreadsheet,url)
>> >
>> > Can anyone suggest a solution to it?
>> >
>> > Thanks,
>> >
>> > Luca
>> >
>> > [[alternative HTML version deleted]]
>> >
>> > __
>> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] CRAN package check results tabulated ... wasRe: Number of package in Ubuntu

[boB Rudis]

Or grab https://cran.r-project.org/web/checks/check_results.rds and
read it w/o the need for scraping.

On Sat, Apr 23, 2016 at 10:43 AM, David Winsemius
 wrote:
>
>> On Apr 23, 2016, at 6:56 AM, David Winsemius  wrote:
>>
>>
>>> On Apr 22, 2016, at 11:51 AM, mylistt...@gmail.com wrote:
>>>
>>> Dear Experts ,
>>>
>>> I am using R with Spark on Windows and now there is a need to move to 
>>> Ubuntu. I wanted to know if most of the packages that are available on 
>>> windows , would they be available on Ubuntu/Linux? If not can I compile the 
>>> source code of those package ? Has any one of you used the packages on 
>>> Ubuntu ?
>>
>> You can get the status of efforts to compile packages on the various machine 
>> avaialbe to CRAN at the CRAN package checks page:
>>
>> https://cran.r-project.org/web/checks/check_summary.html
>>
>
> After scraping that page with rvest::read_html I then used `table` to 
> summarize. I posted the full output at the end of this but here are the 
> relevant rows for Debian (for the Ubuntu option) and the Windows platforms 
> tested:
>
>> res_tbl
> $`r-develLinuxx86_64(Debian GCC)`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   26159  415028  39943525
>
> $`r-develWindowsix86+x86_64`
>
> ERROR ERROR*   NOTE  NOTE* OKOK*   WARN
>295124  1   3962 25   4064 36 45
>
> $`r-patchedLinuxx86_64`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   25954  415328  39983525
>
> $`r-releaseLinuxx86_64`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   26854  257818  55694520
>
> $`r-releaseWindowsix86+x86_64`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   32470  218716  58854624
>
> $`r-oldrelWindowsix86+x86_64`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN WARN*
>   532   159  160513  602845   168 2
>
> I think the various "NOTE" and "WARN" categories in most cases should be 
> interpreted as "probably OK". It's those initial 's and ERROR 
> categories that would most likely be the ones affecting users.
>
> --
>
> David Winsemius
> Alameda, CA, USA
>
> - full tables
> Tabulate on just the first letter of hte result:
>
>> t(res_tbl2)
>  blank   ENO   W
> r-develLinuxx86_64(Debian GCC) 261  59 4178 4029  25
> r-develLinuxx86_64(Fedora Clang)   265  66 4191 4001  29
> r-develLinuxx86_64(Fedora GCC) 265  60 4204 3991  32
> r-develOS Xx86_64(Clang)   294  70 4075 4081  32
> r-develWindowsix86+x86_64  295 125 3987 4100  45
> r-patchedLinuxx86_64   259  54 4181 4033  25
> r-patchedSolarissparc  364 131 4082 3909  66
> r-patchedSolarisx86343 106 4091 3965  47
> r-releaseLinuxx86_64   268  54 2596 5614  20
> r-releaseOS Xx86_64(Mavericks) 255 174 2337 5718  68
> r-releaseWindowsix86+x86_64324  70 2203 5931  24
> r-oldrelWindowsix86+x86_64 532 159 1618 6073 170
>
> Tablulate on full message:
>> res_tbl
> $`r-develLinuxx86_64(Debian GCC)`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   26159  415028  39943525
>
> $`r-develLinuxx86_64(Fedora Clang)`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   26566  417912  39821929
>
> $`r-develLinuxx86_64(Fedora GCC)`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   26560  419113  39721932
>
> $`r-develOS Xx86_64(Clang)`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   29470  4066 9  40681332
>
> $`r-develWindowsix86+x86_64`
>
> ERROR ERROR*   NOTE  NOTE* OKOK*   WARN
>295124  1   3962 25   4064 36 45
>
> $`r-patchedLinuxx86_64`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   25954  415328  39983525
>
> $`r-patchedSolarissparc`
>
> ERROR ERROR*   NOTE  NOTE* OKOK*   WARN  WARN*
>364129  2   4006 76   3839 70 65  1
>
> $`r-patchedSolarisx86`
>
> ERROR ERROR*   NOTE  NOTE* OKOK*   WARN
>343105  1   4062 29   3945 20 47
>
> $`r-releaseLinuxx86_64`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   26854  257818  55694520
>
> $`r-releaseOS Xx86_64(Mavericks)`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   255   174  2335 2  5716 268
>
> $`r-releaseWindowsix86+x86_64`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN
>   32470  218716  58854624
>
> $`r-oldrelWindowsix86+x86_64`
>
>   ERROR  NOTE NOTE*OK   OK*  WARN WARN*
>   532   159  160513  602845   168 2
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Re: [R] Mean of hexadecimal numbers

[boB Rudis]

grDevices has `convertColor()` and the `colorspace` has other
functions that can convert from RBG to Lab space. You should convert
the RGB colors to Lab and average them that way (or us other functions
to convert to HSL or HSV). It all depends on what you are trying to
accomplish with the "average" color determination.

-Bob

On Sat, Apr 16, 2016 at 12:03 PM, Duncan Murdoch
 wrote:
> On 16/04/2016 8:47 AM, Atte Tenkanen wrote:
>>
>> Hi,
>>
>> How would you calculate the "mean colour" of several colours, for
>> example c("#FF7C00","#00BF40","#00")?
>>
>
> Bert answered your subject line question.  Your text is asking something
> else:  if those are colours, you don't want to treat each of them as a
> single integer.
>
> A simple-minded approach would split them into 3 hex numbers, and average
> those (using Bert's solution).
>
> A more sophisticated approach would take into account that they are really
> colours.  You could probably put together something using the colorRamp or
> colorRampPalette functions to average in perception space.  For example,
>
> # Average the 1st two by taking the middle colour of a 3 colour palette
> x <- colorRampPalette(c("#FF7C00","#00BF40"), space = "Lab")(3)[2]
>
> # Average in the third by taking the 2nd of a 4 colour palette, so x
> # gets twice the weight
> colorRampPalette(c(x, "#00"), space = "Lab")(4)[2]
>
> Duncan Murdoch
>
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Microsoft R Server

[boB Rudis]

Yes. Yes. That info is on their site. That info is on their site. They
have paid support for their customers and
non-Microsoft-R-platform-dependent packages will (most likely) still
be answered by the community.

This is just a re-branding and expansion of what was Revolution R
which has been around for ages and a really great supporter and
champion of the R community & ecosystem.

On Thu, Apr 14, 2016 at 8:33 AM, John Sorkin
 wrote:
> Has anyone ever heard of or used Microsoft R server? Does the product work? 
> What are requirements for running it? How much does it cost and is it 
> supported by the R community?
> Thank you,
> John
> Confidentiality Statement:
> This email message, including any attachments, is for ...{{dropped:8}}

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Solution to communicating with UDP and other interfaces (under Linux) using R

[boB Rudis]

Hey Bob,

If you're interested, I'd be glad to see what I can do to make doing
UDP comms from R accessible across platforms without the need for a
`system()` call. Mind shooting me a private e-mail to see what your
needs are so I can try to generalize a solution from them?

-Bob

On Sat, Apr 9, 2016 at 12:11 PM, Robert D. Bowers  wrote:
> I'd spent hours with Google trying to find how to control a separate
> software package from R via a UDP interface (I could run the package under a
> "system" command, but that was too slow).  I finally figured out a way to
> communicate with UDP through R, and it works fine (using the "system"
> command) - it's also fast.
>
> (1) install software "socat" (available on most of the Linux repositories).
> It's potent software for communication stuff, btw.
>
> (2) set up the UDP 'port' to communicate with (like in a separate software
> package for passing commands and responses back and forth).
>
> (3) use the "system" command to send messages - as an example:
> system("echo \"quit\"| socat - UDP4-DATAGRAM:0.0.0.0:19004")
>
> I haven't tried receiving data from that software yet... but according to
> the writeup on socat, it should be no problem.
> You can also use the socat software to redirect from one type of connection
> to another.
>
> (I'm studying R - both for programming/running 'real world' interfaces and
> then doing the statistics on the data.)
>
> Hope this helps someone - the (tiny) few messages I've found indicated that
> it couldn't be done (they were old).  It can be done and rather simply.
>
> Bob
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] issue -- Packages unavailable for R version 3.2.3

[boB Rudis]

>'It's not unlikely that you will need a copy of "Writing R Extensions" at 
>hand.'

+ a few bottles of Scotch.

It might be worth approaching rOpenSci https://ropensci.org/ to take
over resurrection/maintenance of this.

But, it seems others are in your predicament:

https://www.researchgate.net/post/Does_anyone_know_of_an_alternative_R_package_for_sentiment_analysis

so there may be others you can reach out to for alternatives.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] issue -- Packages unavailable for R version 3.2.3

[boB Rudis]

Will you be able to fix the issues that crop up (or even notice the
issues) for these unsupported packages? (There _is_ a reason they
aren't in CRAN anymore.) That particular one (which is, indeed,
archived in CRAN) also depends on Rstem, which is also archived on
CRAN, and now (according to CRAN) "only available on OmegaHat" (though
older versions in the archive). OmegaHat has been down for a bit, too
(including today).

Basing a business practice on these packages seems to be--at best--an
unwise idea.

I'd suggest moving back to a version of R that those pkgs are able to
run on (probably best in a VM) and use Ista's suggestion or something
like MRAN to get specific versions then freeze that VM for future use.

On Wed, Feb 24, 2016 at 6:57 AM, Ista Zahn  wrote:
> Installing unsupported packages is usually not a good idea (there is a
> reason they were removed...).
>
> But if you must:
>
> install.packages("devtools")
> install_version("sentiment", '0.2')
>
> Best,
> Ista
>
> On Wed, Feb 24, 2016 at 2:49 AM, Sandeep Rana
>  wrote:
>> Hi,
>>
>> I have newly installed R version 3.2.3 and experiencing an issue where the
>> packages that I had been using in previous release aren't compatible in the
>> latest release.
>> I need you help to suggest how we could force the installation even if its
>> not supported or what is the workaround to move ahead with it.
>>
>> Right now, I need to install the package 'sentiment'. Please help.
>>
>> Regards,
>>
>> Sandeep S. Rana
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Password-Shiny

[boB Rudis]

What would cause you to think this mailing list is a free code-writing
service? Perhaps post your question on Amazon's Mechanical Turk
service?

Alternatively: purchase a license for Shiny Server Pro.

On Tue, Feb 23, 2016 at 12:45 AM, Venky  wrote:
> Hi R users,
>
> Please anyone help me how to create password access in shiny
>
>
> Thanks and Regards
> Venkatesan
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] qplot Error Message

[boB Rudis]

Assuming that's qplot from ggplot2, it's trying to pass span to the
Point Geom which doesn't recognize it. I highly suggest moving away
from using qplot and working with the stat_s and geom_s directly with
ggplot().

On Sat, Jan 23, 2016 at 8:46 AM, Jeff Reichman  wrote:
> R-Users
>
>
>
> Anyone see what maybe wrong with the following command, other than R doesn't
> seem to recognize the "span" parameter - it should must be my syntax.
>
>
>
>> qplot(seq, count,geom=c("point","smooth"), span=0.8)
>
> Error: Unknown parameters: span
>
>
>
> Jeff
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error opening SHP file

[boB Rudis]

Look at the choroplethr package.

On Fri, Jan 22, 2016 at 5:31 PM, Amoy Yang <amo...@yahoo.com> wrote:
> Let me just make this case simple and quick to address what I need.
>
> Giving a data-file that includes tow columns: zip5 and population. How do I
> put pop (colored with different segments) by zips (with boundary) on the USA
> map with R?
>
> Amoy
>
>
> On Friday, January 22, 2016 12:45 PM, David Winsemius
> <dwinsem...@comcast.net> wrote:
>
>
>
>> On Jan 22, 2016, at 9:24 AM, Amoy Yang via R-help <r-help@r-project.org>
>> wrote:
>>
>> This is the results that addresses David's advice.
>>> library(maptools)
>>> library(maps)
>>> state.map <- readShapeSpatial("maps/st24_d00.shp")
>> Error in getinfo.shape(fn) : Error opening SHP file
>>> # David question: What does list.files('maps') return? Is there a
>>> 'st24_d00.shp' value in there?
>>> list.files('maps')
>> character(0)
>> I actually use the link below to learn how to load state/zip-data on USA
>> map.
>
> That link says:
>
> "For example (assumes you have the maryland shapefiles in the map
> subdirectory):"
>
> So it did not purport to tell you how to put shape files in that
> subdirectory. That statement assumed you understood basic OS path naming
> conventions in your unstated OS and how to move files around, which is not a
> topic for rhelp.
>
> Try working through the examples in the documents that come with the
> `sp`-package.
>
>
> --
> David.
>>
>> #
>> http://stackoverflow.com/questions/1441717/plotting-color-map-with-zip-codes-in-r-or-python
>> Plotting color map with zip codes in R or Python
>> |  |
>> |  |  |  |  |  |  |  |
>> | Plotting color map with zip codes in R or PythonI have some US
>> demographic and firmographic data. I would like to plot zipcode areas in a
>> state or a smaller region (e.g. city). Each area would be annotated by
>> col... |
>> |  |
>> | View on stackoverflow.com | Preview by Yahoo |
>> |  |
>> |  |
>>
>>On Friday, January 22, 2016 10:03 AM, Barry Rowlingson
>> <b.rowling...@lancaster.ac.uk> wrote:
>>
>>
>> We can duplicate the error by giving a path to a non-existent
>> shapefile, which is probably the original problem:
>>
>>> require(maptools)
>> Loading required package: maptools
>> Loading required package: sp
>> Checking rgeos availability: TRUE
>>> foo=readShapeSpatial("fnord.shp")
>> Error in getinfo.shape(fn) : Error opening SHP file
>>
>> The error message there isn't totally explicit, and might cover a
>> range of other possibilities such as a corrupted shapefile, or a
>> missing .shx component of the shapefile or whatever.
>>
>> BUT you probably shouldn't be using readShapeSpatial anyway, as it has
>> a habit of not reading the coordinate system in the .prj file. I find
>> it much easier to use `raster::shapefile` which *does* read the
>> coordinate system *and* gives a more explicit error message for a
>> missing shapefile:
>>
>>> require(raster)
>> Loading required package: raster
>>> foo=shapefile("fnord.shp")
>> Error in normalizePath(x, winslash = "/", mustWork = TRUE) :
>>  path[1]="fnord.shp": No such file or directory
>>
>> "No such file or directory"
>>
>> Barry
>>
>>
>>
>> On Fri, Jan 22, 2016 at 2:25 AM, boB Rudis <b...@rudis.net> wrote:
>>> Agreed with the others. After finding that shapefile and getting it to
>>> work you are definitely not in the proper working directory.
>>>
>>> On Thu, Jan 21, 2016 at 8:40 PM, David Winsemius <dwinsem...@comcast.net>
>>> wrote:
>>>>
>>>>> On Jan 21, 2016, at 4:39 PM, Amoy Yang via R-help
>>>>> <r-help@r-project.org> wrote:
>>>>>
>>>>> Any advice for the following errors?
>>>>> state.map <- readShapeSpatial("maps/st24_d00.shp")
>>>>> Error in getinfo.shape(fn) : Error opening SHP file
>>>>
>>>> What does list.files('maps') return? Is there a 'st24_d00.shp' value in
>>>> there?
>>>>
>>>>
>>>> --
>>>> David.
>>>>
>>>>>
>>
>>
>>
>> [[alternative HTML version deleted]]
>   ^^
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> David Winsemius
> Alameda, CA, USA
>
>

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R editor for Mac

[boB Rudis]

Aye. You can make source/editor windows consume the entire area or
have them as separate windows and can define a consistent line-ending
vs platform native (I run RStudio Preview and [sometimes] dailies and
can confirm these are in there). The addition of full R
(C/C++/HTML/javascript/etc) code diagnostics (optional) is also a
pretty compelling feature.

On Thu, Jan 21, 2016 at 1:06 PM, Duncan Murdoch
 wrote:
> On 20/01/2016 1:28 PM, Duncan Murdoch wrote:
>>
>> On 20/01/2016 1:22 PM, Christofer Bogaso wrote:
>> > Hi,
>> >
>> > Could you please suggest a good R editor for Mac OS X (10.7.5)
>> > Previously my operating system was Windows and there I used Notepad++,
>> > I really had very nice experience with it. However I dont see any Mac
>> > version is available for Mac.
>> >
>> > Appreciate your positive feedback.
>>
>> RStudio is probably best on both OS X and Windows.A nice advantage
>> is that it looks the same on both, so you can move back and forth.
>>
>> I only know two negatives:
>>
>>- I still don't like the tiled window.  I often work on a small
>> screen, and it's not enough space.
>>
>>- The editor still changes file endings to native format whenever it
>> saves.  It would be better if it handled both Windows and Unix line
>> endings in both systems, and left them alone unless the user asked them
>> to be changed.
>
>
> I've just heard offline from JJ Allaire that both negatives above have been
> addressed in a version soon to be released.   Excellent news!
>
>
> Duncan Murdoch
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R editor for Mac

[boB Rudis]

Here you go Ista: https://atom.io/packages/repl (Atom rly isn't bad
for general purpose data sci needs, I still think RStudio is the best
environment for working with R projects).

On Thu, Jan 21, 2016 at 12:48 PM, Ista Zahn  wrote:
> On Jan 21, 2016 12:01 PM, "Philippe Massicotte" 
> wrote:
>>
>> On 01/20/2016 07:22 PM, Christofer Bogaso wrote:
>>>
>>> Hi,
>>>
>>> Could you please suggest a good R editor for Mac OS X (10.7.5)
>>> Previously my operating system was Windows and there I used Notepad++,
>>> I really had very nice experience with it. However I dont see any Mac
>>> version is available for Mac.
>>>
>>> Appreciate your positive feedback.
>>>
>>> Thanks and regards,
>>>
>> Atom seems to be a good choice also.
>
> Is it? Which package(s) should I install to write and run R code in Atom?
> Certainly I don't see anything useful out of the box.
>
> Best,
> Ista
>>
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error opening SHP file

[boB Rudis]

Agreed with the others. After finding that shapefile and getting it to
work you are definitely not in the proper working directory.

On Thu, Jan 21, 2016 at 8:40 PM, David Winsemius  wrote:
>
>> On Jan 21, 2016, at 4:39 PM, Amoy Yang via R-help  
>> wrote:
>>
>> Any advice for the following errors?
>> state.map <- readShapeSpatial("maps/st24_d00.shp")
>> Error in getinfo.shape(fn) : Error opening SHP file
>
> What does list.files('maps') return? Is there a 'st24_d00.shp' value in there?
>
>
> --
> David.
>
>>
>>   [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> David Winsemius
> Alameda, CA, USA
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R editor for Mac

[boB Rudis]

If you don't want to run RStudio, Sublime Text has both great R code
syntax highlighting/formatting and a REPL mode for an interactive
console in-editor.

Atom also has decent R support.

They both play well with "Dash" which is an alternative way (separate
app) to lookup R docs on OS X.

On Wed, Jan 20, 2016 at 2:22 PM, Franklin Bretschneider
 wrote:
> Dear Christofer Bogaso,
>
>
> Re:
>
>
>> Could you please suggest a good R editor for Mac OS X (10.7.5)
>
>
>
> Indeed, as Roy Mendelssohn wrote, the editor built into "R.app", the GUI 
> program which is part of the standard R for OS X, has a beautiful editor, 
> complete with syntax colouring and bracket balancing. And one can run only 
> one or a few lines from a script at wish.
> I couldn't wish myself more.
>
> Success and best wishes,
>
> Frank
> --
>
>
>
>
>
> Franklin Bretschneider
> Dept of Biology
> Utrecht University
> brets...@xs4all.nl
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] printing a data.frame that contains a list-column of S4 objects

[boB Rudis]

Martin, I'm pretty sure the use of Matrix here (actually by someone
else than Dr Bryan) was to make an easy, inline, reproducible example.
The actual "ugh" column comes from using git2r. I'm assuming there's
an API call returning some pretty gnarly structures that are getting
shoehorned into a data.frame. That happens more often than I'd like in
modern API calls (really complex/nested JSON being returned).

On Thu, Jan 14, 2016 at 3:34 AM, Martin Maechler
<maech...@stat.math.ethz.ch> wrote:
>>>>>> boB Rudis <b...@rudis.net>
>>>>>> on Tue, 12 Jan 2016 13:51:50 -0500 writes:
>
> > I wonder if something like:
> > format.list <- function(x, ...) {
> > rep(class(x[[1]]), length(x))
> > }
>
> > would be sufficient? (prbly needs more 'if's though)
>
> Dear Jenny,
> for a different perspective (and a lot of musings), see inline below
>
> > On Tue, Jan 12, 2016 at 12:15 PM, Jenny Bryan <je...@stat.ubc.ca> wrote:
> >> Is there a general problem with printing a data.frame when it has a
> >> list-column of S4 objects? Or am I just unlucky in my life choices?
> >>
> >> I ran across this with objects from the git2r package but maintainer
> >> Stefan Widgren points out this example below from Matrix as well. I 
> note
> >> that the offending object can be printed if sent through
> >> dplyr::tbl_df(). I accept that that printing doesn't provide much info
> >> on S4 objects. I'd just like those vars to not prevent data.frame-style
> >> inpsection of the entire object.
> >>
> >> I asked this on stack overflow, where commenter provided the lead to 
> the
> >> workaround below. Is that the best solution?
> >>
> >> library(Matrix)
> >>
> >> m <- new("dgCMatrix")
> >> isS4(m)
> >> #> [1] TRUE
> >> df <- data.frame(id = 1:2)
> >> df$matrices <- list(m, m)
>
> This only works by accident (I think), and fails for
>
>   df <- data.frame(id = 1)
>   df$matrices <- list(m, m)
>
> > df <- data.frame(id = 1)
> > df$matrices <- list(m, m)
> Error in `$<-.data.frame`(`*tmp*`, "matrices", value = list( class "dgCMatrix">,  :
> replacement has 2 rows, data has 1
> >
>
>
> >> df
> >> #> Error in prettyNum(.Internal(format(x, trim, digits, nsmall, width, 
> 3L, : first argument must be atomic
> >> #> Error in prettyNum(.Internal(format(x, trim, digits, nsmall, width, 
> 3L, : first argument must be atomic
>
> Hmm,
> As 'data.frame' is just an S3 class there is no formal
> definition to go with and in this sense you are of course entitled
> to all expectations. ;-)
> Even though data frames are internally coded as lists, I
> strongly believe data frames should be taught as (and thought of)
>  "generalized matrices"
> in the sense that data frames should be thought of n (say) rows
> and p (say) columns.
>
> The help pages  for  data.frame()  and as.data.frame()
> should make it clear that you can *not* put all kinds of entries
> into data frame columns, but I agree the documentation is vague
> and probably has to remain vague,
> because if you provide  as.data.frame()  methods for your class
> you should be able to go quite far.
>
> In addition, the data frame columns need to fulfill properties, e.g.,
> subsetting (aka "indexing") and also subassignment ( df[i,j] <- v )
>
> Now the real "problem" here is that the '$<-' and '[<-'  methods
> for data frames which you call via  df$m <- v  or  df[,co] <- V
> are too "forgiving". They only check that NROW(.) of the new
> entry corresponds to the nrow().
> Currently they allow very easy construction of illegal data
> frames(*), as in your present case.
>
> --
> *) Yes, it is hard to say when a data.frame is illegal, as there
>is no formal definition
>
> There is more to be said and thought about if you really want
> sparse matrices in a data frame, and as 'Matrix' maintainers,
> I'm quite interested *why* you'd want that, but I won't go there
> now.
>
> One last issue though: The idea of allowing to put 'matrix' or
> 'array' into data frames is that each column of the matrix
> becomes a separate column of the data frame
>
>> data.frame(D = diag(3), M = matrix(1:12, 3,4))
>   D.1 D.2 D.3 M.1 M.2 M.3 M.4
> 1   1   0   0   1   4   7  10
> 2   0   1   0   2   5   8  11
> 3   0   0   1   3   6   9  12
>
> 

Re: [R] printing a data.frame that contains a list-column of S4 objects

[boB Rudis]

I wonder if something like:

format.list <- function(x, ...) {
  rep(class(x[[1]]), length(x))
}

would be sufficient? (prbly needs more 'if's though)

On Tue, Jan 12, 2016 at 12:15 PM, Jenny Bryan  wrote:
> Is there a general problem with printing a data.frame when it has a
> list-column of S4 objects? Or am I just unlucky in my life choices?
>
> I ran across this with objects from the git2r package but maintainer
> Stefan Widgren points out this example below from Matrix as well. I note
> that the offending object can be printed if sent through
> dplyr::tbl_df(). I accept that that printing doesn't provide much info
> on S4 objects. I'd just like those vars to not prevent data.frame-style
> inpsection of the entire object.
>
> I asked this on stack overflow, where commenter provided the lead to the
> workaround below. Is that the best solution?
>
> library(Matrix)
>
> m <- new("dgCMatrix")
> isS4(m)
> #> [1] TRUE
> df <- data.frame(id = 1:2)
> df$matrices <- list(m, m)
> df
> #> Error in prettyNum(.Internal(format(x, trim, digits, nsmall, width, 3L, : 
> first argument must be atomic
> #> Error in prettyNum(.Internal(format(x, trim, digits, nsmall, width, 3L, : 
> first argument must be atomic
>
> ## fairly costly workaround
> df2 <- df
> df2[] <- lapply(df2, as.character)
> df2
> #>   id matrices
> #> 1  1 
> #> 2  2 
>
> ## dplyr handles original object better but not as well as workaround
> library(dplyr)
> ## use select to force dplyr to show the tricky column
> tbl_df(select(df, matrices))
> #> Source: local data frame [2 x 1]
> #>
> #>  
> matrices
> #>
> (list)
> #> 1  dCsparseMatrix,
> #> 2  dCsparseMatrix,
>
> Thanks,
> Jenny
>
> Jennifer Bryan
> Associate Professor
> Department of Statistics and
>the Michael Smith Laboratories
> University of British Columbia
> Vancouver, BC Canada
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error: could not find function "VectorSource" in package tm

[boB Rudis]

Do you have any code? Any more logs from the error? It's hard to help
when you've provided little more than an error message. What does the
output of:

   library(tm)
   docs <- c("This is a text.", "This another one.")
   (vs <- VectorSource(docs))

generate?

On Wed, Dec 30, 2015 at 2:32 PM, David Brand <9rogr4m...@gmail.com> wrote:
> Error: could not find function "VectorSource" in package tm
>
> Windows 64bit using Rstudio
>
> please advise
>
> thanks
> David
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Modifying graphs in 'survrec' package

[boB Rudis]

As answered here: http://stackoverflow.com/a/1444/1457051

palette(c("red", "blue", "orange"))
par(lty=3)
plot(fit,ylim=c(0,1),xlim=c(0,2000))

though, as indicated in that post, you'll need to customize the
survrec:::plot.survfitr function to do more detailed customization.

On Sun, Oct 25, 2015 at 10:17 AM, Anna Cecilia Lawson McLean
 wrote:
> Dear all,
>
> I am new to R and I am using the package 'survrec'. I would like to modify
> the colors and lines in graphs with multiple groups. The package includes
> this example:
>
> data(colon)
> fit<-survfitr(Survr(hc,time,event)~as.factor(dukes),data=colon,type="pena")
>
> Using the arguments "col" or "lty" has not worked:
>
> plot(fit,ylim=c(0,1),xlim=c(0,2000), col=c("red", "blue", "orange"), lty=3)
>
>
> I would be very grateful for any help! Thank you very much in advance!
>
> Kind regards,
>
> Anna McLean
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Does R work on Mac OS 10.11?

[boB Rudis]

I use it daily (hourly, really) on 10.11 (including the new betas). No issues.

On Mon, Oct 5, 2015 at 10:03 AM, R Martinez  wrote:
> Has anyone tried to use R 3.2.2 on a Mac running OS 10.11 El Capitan? Did it 
> work? Were any problems installing and running it?
>
> Thanks in advance,
>
> Raul Martinez
>
>
>
>
> Sent from my iPad 4
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] extracting a value from XML

[boB Rudis]

The "
>  observation_start="2015-09-01" observation_end="2015-09-01"
> units="lin" output_type="1" file_type="xml"
> order_by="observation_date" sort_order="asc" count="1" offset="0"
> limit="10">
>  date="2015-09-01" value="0.46"/>
> '
>
> doc <- read_xml(txt)
> xml_attr(xml_find_all(doc, "//observation"), "value")
>
> doc1 <- xmlParse(txt)
> xpathSApply(doc1, "//observation", xmlGetAttr, "value")
>
>
>
> On Mon, Sep 21, 2015 at 2:01 PM, Glenn Schultz  wrote:
>
> 
>
> 
> observation_start="2015-09-01" observation_end="2015-09-01" units="lin"
>
> output_type="1" file_type="xml" order_by="observation_date" sort_order="asc"
>
> count="1" offset="0" limit="10">
>
> 
> date="2015-09-01" value="0.46"/>
>
> 

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] extracting a value from XML

[boB Rudis]

This is how (one way) in both the xml2 package and XML package:

library(xml2)
library(XML)

txt <- '

  
'

doc <- read_xml(txt)
xml_attr(xml_find_all(doc, "//observation"), "value")

doc1 <- xmlParse(txt)
xpathSApply(doc1, "//observation", xmlGetAttr, "value")



On Mon, Sep 21, 2015 at 2:01 PM, Glenn Schultz  wrote:
> 
>  observation_start="2015-09-01" observation_end="2015-09-01" units="lin"
> output_type="1" file_type="xml" order_by="observation_date" sort_order="asc"
> count="1" offset="0" limit="10">
>date="2015-09-01" value="0.46"/>
> 

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Problem R markdown document

[boB Rudis]

Try increasing the memory for pandoc via knitr YAML options:

--
title: TITLE
output:
  html_document:
pandoc_args: [
  +RTS, -K64m,
  -RTS
]
---

ref: http://stackoverflow.com/a/28015894/1457051

you can bump up those #'s IIRC, too, if they don't work at first.

On Thu, Aug 27, 2015 at 1:55 PM, Conklin, Mike (GfK)
mike.conk...@gfk.com wrote:
 I have successfully done this many times using RStudio's rmarkdown 
 capabilities and knitting the document to HTML or Word. However, I am running 
 into this error today.

 C:/Program Files/RStudio/bin/pandoc/pandoc FusionTestsAugust25.utf8.md --to 
 docx --from 
 markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash-implicit_figures
  --output FusionTestsAugust25.docx --highlight-style tango
 pandoc.exe: getMBlocks: VirtualAlloc MEM_COMMIT failed
 Error: pandoc document conversion failed with error 1

 Same error occurs whether to knitting to Word or to HTML.  It looks like the 
 a memory issue but this whole process is of limited use if the entire 
 analysis runs but the document can't be output.  If anyone has any ideas on 
 how to deal with memory issues in the final step of pandoc conversion or can 
 point me to where to look for this I would appreciate it.

 Best regards,

 Mike


 --
 W. Michael Conklin
 Executive Vice President
 Marketing  Data Sciences - North America
 GfK | 8401 Golden Valley Road | Minneapolis | MN | 55427
 mike.conk...@gfk.com
 T +1 763 417 4545 | M +1 612 567 8287
 www.gfk.com

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] How to download this data

[boB Rudis]

Looks like you can get what you need from
http://www.nseindia.com/homepage/Indices1.json on that page.

On Tue, Aug 25, 2015 at 2:23 PM, Bert Gunter bgunter.4...@gmail.com wrote:
 This is not a simple question. The data are in an html-formatted web
 page. You must scrape the html for the data and read it into an R
 table (or other appropriate R data structure). Searching (the web) on
 scrape data from html into R  listed several packages that claim to
 enable you to do this easily. Choose what seems best for you.

 You should also install and read the documentation for the XML
 package, which is also used for this purpose, though those you find
 above may be slicker.

 Disclaimer: I have no direct experience with this. I'm just pointing
 out what I believe are relevant resources.

 Cheers,
 Bert
 Bert Gunter

 Data is not information. Information is not knowledge. And knowledge
 is certainly not wisdom.
-- Clifford Stoll


 On Tue, Aug 25, 2015 at 11:10 AM, Christofer Bogaso
 bogaso.christo...@gmail.com wrote:
 Hi,

 I would like to download data from below page directly onto R.

 http://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm

 Could you please assist me how can I do that programmatically.

 Thanks for your time.

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] data frame formatting

[boB Rudis]

Here's one way in base R:

df - data.frame(id=c(A,A,B,B),
 first=c(BX,NA,NA,LF),
 second=c(NA,TD,BZ,NA),
 third=c(NA,NA,RB,BT),
 fourth=c(LG,QR,NA,NA))


new_df - data.frame(do.call(rbind, by(df, df$id, function(x) {
  sapply(x[,-1], function(y) {
if (all(is.na(y))) return(NA)
if (all(!is.na(y))) return(clash)
return(as.character(y[which(!is.na(y))]))
  })
})))

new_df$id - rownames(new_df)
rownames(new_df) - NULL

new_df

##   first second third fourth id
## 1BX TD  NA  clash  A
## 2LF BZ clash   NA  B


On Tue, Aug 18, 2015 at 3:06 PM, Jon BR jonsle...@gmail.com wrote:
 df -
 data.frame(id=c(A,A,B,B),first=c(BX,NA,NA,LF),second=c(NA,TD,BZ,NA),third=c(NA,NA,RB,BT),fourth=c(LG,QR,NA,NA))
 df

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Knit R and Book Publishing

[boB Rudis]

https://github.com/hadley/adv-r is how it was done.

On Thu, Aug 6, 2015 at 8:33 AM, Bert Gunter bgunter.4...@gmail.com wrote:
 I would have thought that the first place to look was R Studio support
 site. You will find a lot of (Imo well done) docs there as well as links to
 Hadley's and Yihui's books and online docs.

 Bert

 On Wednesday, August 5, 2015, Glenn Schultz glennmschu...@me.com wrote:

 Hello All,

 I have finished my first R package, BondLab, and it will be available in
 Feb 2016.   I wrote the book and R package concurrently.  Prior, I had very
 little programming experience other than VBA and using others R packages.


 http://www.amazon.com/Investing-Mortgage-Backed-Securities-Website/dp/1118944003/ref=sr_1_1?ie=UTF8qid=1438822386sr=8-1keywords=glenn+m+schultz

 During the learning process I came to know literate programming and
 reproducible research.  Now,  I have a viable package and I would like
 re-write the book in R Studio with knitr.  I have had some success with LYX
 but I think I can do the whole thing in R studio.  The first edition was
 done in TexMaker.

 I really want to set the next project up in R Studio, I would really like
 to open source the next edition like Hadley's recent books but I really
 don't know how it was all done.  Any starting points to set this up are
 appreciated.

 -Glenn
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



 --
 Bert Gunter

 Data is not information. Information is not knowledge. And knowledge is
 certainly not wisdom.
-- Clifford Stoll

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] help_ReverseGeocoding

[boB Rudis]

You should use ggmap::revgeocode (it calls google's api) and google
will rate-limit you. There are also packages to use HERE maps
geo/revgeo lookups
http://blog.corynissen.com/2014/10/making-r-package-to-use-here-geocode-api.html
and the geocode package has GNfindNearestAddress, so tons of options
to choose from.

On Tue, Jul 28, 2015 at 11:30 AM, MacQueen, Don macque...@llnl.gov wrote:
 My first guess, after a quick glance, is that Google only lets you do a
 limited number of lookups within some period of time.

 -Don

 --
 Don MacQueen

 Lawrence Livermore National Laboratory
 7000 East Ave., L-627
 Livermore, CA 94550
 925-423-1062





 On 7/27/15, 10:14 PM, R-help on behalf of shreya ghosh
 r-help-boun...@r-project.org on behalf of shreya@gmail.com wrote:

Hi,
I'm trying to do reversegeocoding on a large dataset. I'm using RJSONIO
library and using Google map API to get the location of the given lat-lon
in the dataset. After 100 or 150 successful displaying location
information
it is showing
 Warning message - In readLines(con) : cannot open: HTTP status was '0
(null)'
and Error : Error in fromJSON(paste(readLines(con), collapse = )) :
  error in evaluating the argument 'content' in selecting a method for
function 'fromJSON': Error in readLines(con) : cannot open the connection

Please help me to solve the issue.

location function is as follows :

location-function(latlng){
 latlngStr -  gsub(' ','%20', paste(latlng, collapse=,))
  library(RJSONIO) #Load Library
  #Open Connection
  connectStr - paste('
http://maps.google.com/maps/api/geocode/json?sensor=falselatlng=',latlngS
tr,
sep=)
  con - url(connectStr)
  data.json - fromJSON(paste(readLines(con), collapse=))
  close(con)

  data.json - unlist(data.json)
  if(data.json[status]==OK)
address - data.json[results.formatted_address]
  print (address)
}

I'm using R version 3.2.1 and Ubuntu 14.10 OS.

Thank you.



--

Shreya Ghosh

*9007448845*

-- The mind is not a vessel to be filled, but a fire to be kindled

   [[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] modifying a package installed via GitHub

[boB Rudis]

You can go to the package directory:

cd /some/path/to/package

and do

R CMD install .

from a command-line there.

Many github-based packages are also made using RStudio and you can
just open the .Rproj file (i.e. load it into R studio) and build the
package there which will install it.

The same-named package will overwrite what you have previously installed.

Just:

   devtools::install_github(owner/package)

to go back to the original.

On Fri, Jul 17, 2015 at 8:12 PM, Steve E. se...@vt.edu wrote:
 Hi Folks,

 I am working with a package installed via GitHub that I would like to
 modify. However, I am not sure how I would go about loading a 'local'
 version of the package after I have modified it, and whether that process
 would including uninstalling the original unmodified package (and,
 conversely, how to uninstall my local, modified version if I wanted to go
 back to the unmodified version available on GitHub).

 Any advice would be appreciated.


 Thanks,
 Steve



 --
 View this message in context: 
 http://r.789695.n4.nabble.com/modifying-a-package-installed-via-GitHub-tp4710016.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Web crawling amazon website using R

[boB Rudis]

You might want to read Amazon's terms of service before crawling their
site: 
http://www.amazon.in/gp/help/customer/display.html/ref=footer_cou/276-8549425-3823542?ie=UTF8nodeId=200545940

On Tue, Jun 30, 2015 at 3:33 AM, Abhinaba Roy abhinabaro...@gmail.com wrote:
 Hi R helpers,

 I want to crawl the amazon.in website to extract the customer feedbacks for
 a particular product, and then use the texts for word cloud and sentiment
 analysis.

 For example, if I want to extract the feedback texts from

 http://www.amazon.in/Mi-4-White-16GB/product-reviews/B00VEB0F22/ref=dpx_acr_txt?showViewpoints=1

 How do I achieve this using R?


 Regards,
 Abhinaba Roy

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Call to a function

[boB Rudis]

You can do something like:

aaa - function(data, w=w) {
  if (class(w) %in% c(integer, numeric, double)) {
out - mean(w)
  } else {
out - mean(data[, w])
  }
  return(out)
}

(there are some typos in your function you may want to double check, too)

On Tue, Jun 23, 2015 at 5:39 PM, Steven Yen sye...@gmail.com wrote:
 mydata-data.frame(matrix(1:20,ncol=2))
 colnames(mydata) -c(v1,v2)
 summary(mydata)

 aaa-function(data,w=w){
   if(is.vector(w)){
 out-mean(w)
   } else {
 out-mean(data[wt])
   }
 return(out)
 }

 aaa(mydata,mydata$v1)
 aaa(mydata,v1)  # want this call to work

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] [R-pkgs] Version 0.8.5 of metricsgraphics is on CRAN

[boB Rudis]

Version 0.8.5 of metricsgraphics is now on CRAN.

It provides an 'htmlwidgets' interface to the 'MetricsGraphics.js'
('D3'-based) charting
library which is geared towards displaying time-series data. There are
routines for scatterplots, histograms and even 'grid.arrange'-like
functionality for laying out multiple charts.

-Bob

___
R-packages mailing list
r-packa...@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-packages

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] reading daily snow depth data

[boB Rudis]

This look similar to snow data I used last year:
https://github.com/hrbrmstr/snowfirst/blob/master/R/snowfirst.R

All the data worked pretty well.

On Tue, Jun 16, 2015 at 3:21 PM, jim holtman jholt...@gmail.com wrote:
 Here is an example of reading in the data.  After that it is a data frame
 and should be able to process it with dplyr/data.table without much trouble:

 x - readLines(
 http://www1.ncdc.noaa.gov/pub/data/snowmonitoring/fema/06-2015-dlysndpth.txt
 )
 writeLines(x, '/temp/snow.txt')  # save for testing
 head(x)
 [1]
 

 [2] State:
 AL

 [3]Lat Lon  COOP# StnID State City/Station Name
 County Elev  Jun 1  Jun 2  Jun 3  Jun
 4  Jun 5  Jun 6  Jun 7  Jun 8  Jun 9  Jun10
 Jun11  Jun12  Jun13  Jun14  Jun15  Jun16
 [4]  33.59  -85.86 010272  AL ANNISTON ARPT ASOS
 CALHOUN  594  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000  -.000
 [5]  33.83  -85.78 014209  AL JACKSONVILLE
 CALHOUN  608  -.000  -.000  -.000
 -.000  -.000  0.000  0.000  -.000  -.000
 -.000  -.000  -.000  -.000  -.000  -.000  -.000
 [6]  34.74  -87.60 015749  AL MUSCLE SHOALS AP
 COLBERT  540  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000  -.000
 z - grepl((^$)|(^State)|(^   Lat), x)  # get lines to discard
 xm - x[!z]  # remove info lines
 head(xm)
 [1]  33.59  -85.86 010272  AL ANNISTON ARPT ASOS
 CALHOUN  594  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000  -.000
 [2]  33.83  -85.78 014209  AL JACKSONVILLE
 CALHOUN  608  -.000  -.000  -.000
 -.000  -.000  0.000  0.000  -.000  -.000
 -.000  -.000  -.000  -.000  -.000  -.000  -.000
 [3]  34.74  -87.60 015749  AL MUSCLE SHOALS AP
 COLBERT  540  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000  -.000
 [4]  31.32  -85.45 012372  AL DOTHAN FAA AIRPORT
 DALE 374  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000  -.000
 [5]  32.70  -87.58 013511  AL GREENSBORO
 HALE 220  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000  -.000
 [6]  33.57  -86.74 010831  AL BIRMINGHAM AP ASOS
 JEFFERSON615  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000
 0.000  0.000  0.000  0.000  0.000  0.000  -.000

 # read in the data
 xf - textConnection(xm)
 snow - read.fwf(xf
 + , width = c(6,8,7,10,3,32,26,6,rep(11,16))
 + , comment.char = ''
 + , as.is = TRUE
 + )
 str(snow)
 'data.frame':   3067 obs. of  24 variables:
  $ V1 : num  33.6 33.8 34.7 31.3 32.7 ...
  $ V2 : num  -85.9 -85.8 -87.6 -85.5 -87.6 ...
  $ V3 : int  10272 14209 15749 12372 13511 10831 11225 14064 12245 15478 ...
  $ V4 : chr  ...
  $ V5 : chr  AL  AL  AL  AL  ...
  $ V6 : chr  ANNISTON ARPT ASOS  
 JACKSONVILLE MUSCLE SHOALS AP
 DOTHAN FAA AIRPORT   ...
  $ V7 : chr  CALHOUNCALHOUN   
 COLBERTDALE   ...
  $ V8 : int  594 608 540 374 220 615 461 624 100 215 ...
  $ V9 : num  0 - 0 0 0 ...
  $ V10: num  0 - 0 0 0 ...
  $ V11: num  0 - 0 0 0 ...
  $ V12: num  0 - 0 0 0 ...
  $ V13: num  0 - 0 0 0 ...
  $ V14: num  0 0 0 0 0 ...
  $ V15: num  0 0 0 0 0 ...
  $ V16: num  0 - 0 0 0 ...
  $ V17: num  0 - 0 0 0 ...
  $ V18: num  0 - 0 0 0 ...
  $ V19: num  0 - 0 0 0 ...
  $ V20: num  0 - 0 0 0 ...
  $ V21: num  0 - 0 0 0 ...
  $ V22: num  0 - 0 0 0 ...
  $ V23: num  0 - 0 0 0 ...
  $ V24: num  - - - - - ...
 table(snow$V5)  # tally up the states
 AK  AL  AR  AZ  CA  CO  CT  DE  FL  GA  HI  IA  ID  IL  IN  KS  KY  LA  MA
 MD  ME  MI  MN  MO  MS  MT
  72  18  65  55  99 128  10   1  30  33   6 112  57 103  85  90  49  29
 35  14  40  86  90 124  27 113
 NC  ND  NE  NH  NJ  NM  NV  NY  OH  OK  OR  PA  RI  SC  SD  TN  TX  UT  VA
 VT  WA  WI  WV  WY
  45  19 136  22  13  53  65 

Re: [R] Change Julian function in SPlus to R date code

[boB Rudis]

The parameters for the built-in julian() are a bit different. This
should be what you can use:

start - julian(as.Date(sprintf(%d-%d-%d, nn, mt1, 1)),
origin=as.Date(1971-01-01))
end - julian(as.Date(sprintf(%d-%d-%d, nn, mt2, 1)),
  origin=as.Date(1971-01-01))

On Wed, Jun 10, 2015 at 8:08 AM, Adams, Jean jvad...@usgs.gov wrote:
 Try looking at the julian() function in base ...

 ?julian

 Jean

 On Wed, Jun 10, 2015 at 2:46 AM, roslinazairimah zakaria 
 roslina...@gmail.com wrote:

 Dear r-users,

 I have a code in SPlus which use julian function.  What is the similar code
 used in R?

 define.date1-function(dt1,mt1,mt2,nn,da)
 { mt2-mt2+1
 start-julian(mt1, 1, nn, origin=c(month=1, day=1, year=1971))+1
 end-julian(mt2, 1, nn, origin=c(month=1, day=1, year=1971))+da
 a-dt1[start:end,]
 am-as.matrix(a[,5])
 }

 I have check the Date package in R but not so sure how to adjust it.

 Thank you for any help given.

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] web scraping image

[boB Rudis]

You can also do it with rvest  httr (but that does involve some parsing):

library(httr)
library(rvest)

url - 
http://nwis.waterdata.usgs.gov/nwis/peak?site_no=12144500agency_cd=USGSformat=img;
html(url) %%
  html_nodes(img) %%
  html_attr(src) %%
  paste0(http://nwis.waterdata.usgs.gov;, .) %%
  GET(write_disk(12144500.gif)) - status

Very readable and can be made programmatic pretty easily, too. Plus:
avoids direct use of the XML library. Future versions will no doubt
swap xml2 for XML as well.

-Bob


On Mon, Jun 8, 2015 at 2:09 PM, Curtis DeGasperi
curtis.degasp...@gmail.com wrote:
 Thanks to Jim's prompting, I think I came up with a fairly painless way to
 parse the HTML without having to write any parsing code myself using the
 function getHTMLExternalFiles in the XML package. A working version of the
 code follows:

 ## Code to process USGS peak flow data

 require(dataRetrieval)
 require(XML)

 ## Need to start with list of gauge ids to process

 siteno - c('12142000','12134500','12149000')

 lstas -length(siteno) #length of locator list

 print(paste('Processsing...',siteno[1],' ',siteno[1], sep = ))

 datall -  readNWISpeak(siteno[1])

 for (a in 2:lstas) {
   # Print station being processed
   print(paste('Processsing...',siteno[a], sep = ))

   dat-  readNWISpeak(siteno[a])

   datall - rbind(datall,dat)

 }

 write.csv(datall, file = usgs_peaks.csv)

 # Retrieve ascii text files and graphics
 for (a in 1:lstas) {

   print(paste('Processsing...',siteno[a], sep = ))

   graphic.url -
 paste('http://nwis.waterdata.usgs.gov/nwis/peak?site_no=',siteno[a],'agency_cd=USGSformat=img',
 sep = )
   usgs.img - getHTMLExternalFiles(graphic.url)
   graphic.img - paste('http://nwis.waterdata.usgs.gov',usgs.img, sep = )

   peakfq.url -
 paste('http://nwis.waterdata.usgs.gov/nwis/peak?site_no=',siteno[a],'agency_cd=USGSformat=hn2',
 sep = )
   tab.url  - 
 paste('http://nwis.waterdata.usgs.gov/nwis/peak?site_no=',siteno[a],'agency_cd=USGSformat=rdb',
 sep = )

   graphic.fn - paste('graphic_',siteno[a],'.gif', sep = )
   peakfq.fn - paste('peakfq_',siteno[a],'.txt', sep = )
   tab.fn  - paste('tab_',siteno[a],'.txt', sep = )
   download.file(graphic.img,graphic.fn,mode='wb')
   download.file(peakfq.url,peakfq.fn)
   download.file(tab.url,tab.fn)
 }

 --

 Message: 34
 Date: Fri, 5 Jun 2015 08:59:04 +1000
 From: Jim Lemon drjimle...@gmail.com
 To: Curtis DeGasperi curtis.degasp...@gmail.com
 Cc: r-help mailing list r-help@r-project.org
 Subject: Re: [R] web scraping image
 Message-ID:
 
 ca+8x3fv0ajw+e22jayv1gfm6jr_tazua5fwgd3t_mfgfqy2...@mail.gmail.com
 Content-Type: text/plain; charset=UTF-8

 Hi Chris,
 I don't have the packages you are using, but tracing this indicates
 that the page source contains the relative path of the graphic, in
 this case:

 /nwisweb/data/img/USGS.12144500.19581112.20140309..0.peak.pres.gif

 and you already have the server URL:

 nwis.waterdata.usgs.gov

 getting the path out of the page source isn't difficult, just split
 the text at double quotes and get the token following img src=. If I
 understand the arguments of download.file correctly, the path is the
 graphic.fn argument and the server URL is the graphic.url argument. I
 would paste them together and display the result to make sure that it
 matches the image you want. When I did this, the correct image
 appeared in my browser. I'm using Google Chrome, so I don't have to
 prepend the http://

 Jim

 On Fri, Jun 5, 2015 at 2:31 AM, Curtis DeGasperi
 curtis.degasp...@gmail.com wrote:
 I'm working on a script that downloads data from the USGS NWIS server.
 dataRetrieval makes it easy to quickly get the data in a neat tabular
 format, but I was also interested in getting the tabular text files -
 also fairly easy for me using download.file.

 However, I'm not skilled enough to work out how to download the nice
 graphic files that can be produced dynamically from the USGS NWIS
 server (for example:

 http://nwis.waterdata.usgs.gov/nwis/peak?site_no=12144500agency_cd=USGSformat=img
 )

 My question is how do I get the image from this web page and save it
 to a local directory? scrapeR returns the information from the page
 and I suspect this is a possible solution path, but I don't know what
 the next step is.

 My code provided below works from a list I've created of USGS flow
 gauging stations.

 Curtis

 ## Code to process USGS daily flow data for high and low flow analysis
 ## Need to start with list of gauge ids to process
 ## Can't figure out how to automate download of images

 require(dataRetrieval)
 require(data.table)
 require(scrapeR)

 df - read.csv(usgs_stations.csv, header=TRUE)

 lstas -length(df$siteno) #length of locator list

 print(paste('Processsing...',df$name[1],' ',df$siteno[1], sep = ))

 datall -  readNWISpeak(df$siteno[1])

 for (a in 2:lstas) {
   # Print station being processed
   print(paste('Processsing...',df$name[a],' ',df$siteno[a], sep = ))

   dat-