Re: [R] Can someone please have a look at my query on stackoverflow ?

[Ashim Kapoor]

Dear Uwe and others,

Here is my query:

I am looking for a function which can:-

take a seasonal ARMA model

and return :-

the spectrum of the seasonal ARMA model.

I have seen:

library(astsa)
?arma.spec

However it's a bit clumsy to use arma.spec when my model has seasonal
ar / ma terms.

For example (from the help page of arma.spec) we have that, we may
compute the spectrum of a seasonal AR model like this:

Notice: How we have a seasonal AR term of order one,
which is being fed to arma.spec by converting the seasonal lag to a
non-seasonal lag:

arma.spec(ar=c(rep(0,11),.4), ma=.5, col=5, lwd=3, frequency=12)

I will illustrate my difficulty as follows:

> plot(AirPassengers)
> auto.arima(log(AirPassengers))
Series: log(AirPassengers)
ARIMA(0,1,1)(0,1,1)[12]

Coefficients:
  ma1 sma1
  -0.4018  -0.5569
s.e.   0.0896   0.0731

sigma^2 = 0.001371:  log likelihood = 244.7
AIC=-483.4   AICc=-483.21   BIC=-474.77

I do not know how to programmatically feed the above model to arma.spec.

I have tried:

auto.arima(log(AirPassengers))$coef
arma.spec(auto.arima(log(AirPassengers))$coef,frequency=12)

I manually convert the seasonal MA to non-seasonal MA model and give
it to arma.spec and I get a different picture.

arma.spec(ma=c(-.4,rep(0,10),-.55))

Hence I conclude that arma.spec does not understand seasonal models.

Query : How can I conveniently plot the spectrum of a seasonal ARMA process ?

Note : I wonder if there is a library which converts seasonal model to
non-seasonal model which perhaps can then be used with arma.spec.

Many thanks,
Ashim

On Tue, May 28, 2024 at 12:28 PM Uwe Ligges
 wrote:
>
>
>
> On 28.05.2024 06:41, Ashim Kapoor wrote:
> > Dear Sir,
> >
> > OK. I will migrate this query to stats.stackexchange.com.
>
> In any case, if you ask on the list, the question should be on the list.
> Personally, I do read mails, but rather not websites.
>
> Best,
> Uwe Ligges
>
>
>
>
>
>
> > Best,
> > Ashim
> >
> >
> > On Mon, May 27, 2024 at 8:28 PM David Winsemius  
> > wrote:
> >>
> >> If I had seen that post before it got a bounty, I would have voted to 
> >> migrate it to https://stats.stackexchange.com since it is a request for 
> >> advice on methods as well as an implicit request for a package 
> >> recommendation.
> >>
> >> —
> >> David
> >> Sent from my iPhone
> >>
> >>> On May 27, 2024, at 5:40 AM, Ashim Kapoor  wrote:
> >>>
> >>> Dear experts,
> >>>
> >>> I am having difficulty with computing spectrum of seasonal ARMA models.
> >>>
> >>> My query is posted here :
> >>>
> >>> https://stackoverflow.com/questions/78526800/computing-the-spectrum-of-a-seasonal-model-in-r
> >>>
> >>> Will be grateful if someone has a look at it and responds.
> >>>
> >>> Many thanks,
> >>> Ashim.
> >>>
> >>> [[alternative HTML version deleted]]
> >>>
> >>> __
> >>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >>> https://stat.ethz.ch/mailman/listinfo/r-help
> >>> PLEASE do read the posting guide 
> >>> http://www.R-project.org/posting-guide.html
> >>> and provide commented, minimal, self-contained, reproducible code.
> >>
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can someone please have a look at my query on stackoverflow ?

[Ashim Kapoor]

Dear Sir,

OK. I will migrate this query to stats.stackexchange.com.

Best,
Ashim


On Mon, May 27, 2024 at 8:28 PM David Winsemius  wrote:
>
> If I had seen that post before it got a bounty, I would have voted to migrate 
> it to https://stats.stackexchange.com since it is a request for advice on 
> methods as well as an implicit request for a package recommendation.
>
> —
> David
> Sent from my iPhone
>
> > On May 27, 2024, at 5:40 AM, Ashim Kapoor  wrote:
> >
> > Dear experts,
> >
> > I am having difficulty with computing spectrum of seasonal ARMA models.
> >
> > My query is posted here :
> >
> > https://stackoverflow.com/questions/78526800/computing-the-spectrum-of-a-seasonal-model-in-r
> >
> > Will be grateful if someone has a look at it and responds.
> >
> > Many thanks,
> > Ashim.
> >
> >[[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Can someone please have a look at my query on stackoverflow ?

[Ashim Kapoor]

Dear experts,

I am having difficulty with computing spectrum of seasonal ARMA models.

My query is posted here :

https://stackoverflow.com/questions/78526800/computing-the-spectrum-of-a-seasonal-model-in-r

Will be grateful if someone has a look at it and responds.

Many thanks,
Ashim.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can someone please have a look at this query on stackoverflow?

[Ashim Kapoor]

Dear Olivier,

Many thanks.

Best Regards,
Ashim

On Sat, Nov 18, 2023 at 2:36 PM Olivier Crouzet
 wrote:
>
> And indeed again (I did not understand your previous question exactly
> at first), the 'hard-coded' definition of a landscape a4 sheet would
> therefore be:
>
> paperwidth=29.7cm, paperheight=21cm
>
> Olivier.
>
> On Sat, 18 Nov 2023 13:20:49 +0530
> Ashim Kapoor  wrote:
>
> > Dear Olivier,
> >
> > Many thanks for your reply.
> >
> > Very cool.
> >
> > You know what I thought ? I thought you had modified the A4 sheet size
> > to compute the paperheight and paperwidth ?
> >
> > I wonder if that's another way of proceeding.
> >
> > Best,
> > Ashim
> >
> > On Sat, Nov 18, 2023 at 12:51 PM Olivier Crouzet
> >  wrote:
> > >
> > > Dear Ashim,
> > >
> > > these are documented in the LaTeX 'geometry' package (see for
> > > example on CTAN: https://ctan.org/pkg/geometry). As I added in my
> > > response on Stackoverflow, several parts in the RMarkdown header
> > > actually concern information that are processed by LaTeX to
> > > actually generate the PDF, among which the 'geometry' line. For
> > > someone who is used to working with LaTeX, it is relatively natural
> > > to 'identify' LaTeX options in these lines even though they are not
> > > structured exactly this way in a LaTeX document (but their names
> > > are the same).
> > >
> > > Concerning the choice for these specific numbers, it is relatively
> > > arbitrary. I've been doing this on LaTeX for years when I want to
> > > generate slides without using the 'beamer' document class and the
> > > things to take into account are:
> > >
> > > - These dimensions express the physical sizes of the pdf page (in a
> > > sense, what their size would be if you print the document without
> > > adapting to the paper in your printer)
> > > - If these sizes are reduced... the relative font size will increase
> > > because LaTeX will project the same font on a smaller virtual paper,
> > > - And if these sizes are increased... the relative font size will
> > > decrease because LaTeX will project the same font on a larger
> > > virtual paper,
> > > - I processed starting from 16x9cm sizes, resp. for paperwidth and
> > > paperheight, and estimated that the fonts were too large to my
> > > taste, then doubled them and felt that the fonts were too small for
> > > slides, then I ended up trying multiplying 16x9cm by a factor of
> > > 1.5, which gave me 24x13.5cm and I found it was ok.
> > > - But one may vary these sizes arbitrary (even using smaller
> > > steps and ratios differing from 16/9) depending on the aims to be
> > > reached.
> > > - It is also possible to specify different units (pt = points, in =
> > > inches...)
> > >
> > > Yours.
> > > Olivier.
> > >
> > > On Sat, 18 Nov 2023 09:53:19 +0530 Ashim Kapoor
> > >  wrote:
> > >
> > > > Dear Olivier,
> > > >
> > > > Many thanks for your reply.
> > > >
> > > > This works well for me.
> > > >
> > > > How did you come up with the pagewidth / pageheight numbers?  I do
> > > > understand that their ratio = 16:9,
> > > > but how did you choose these numbers?
> > > >
> > > > Best Regards,
> > > > Ashim
> > > >
> > > > On Fri, Nov 17, 2023 at 9:25 PM Olivier Crouzet
> > > >  wrote:
> > > > >
> > > > > Dear Ashim,
> > > > >
> > > > > I don't think the aspectratio is appropriate in this context
> > > > > because it would imply that the beamer (LaTeX) class is used
> > > > > but you're actually using the article (LaTeX) class.
> > > > >
> > > > > You may use specifications of the geometry package rather than
> > > > > specifying options to the class:
> > > > >
> > > > > e.g. replace your current header:
> > > > >
> > > > > ---
> > > > > title: "Testing landscape and aspect ratio"
> > > > > output:
> > > > >   pdf_document:
> > > > > number_sections: true
> > > > > classoption:
> > > > >   - landscape
> > > > >   - "aspectratio=169"
> > > > > header-includes:
> > > > >- \usepack

Re: [R] Can someone please have a look at this query on stackoverflow?

[Ashim Kapoor]

Dear Olivier,

Many thanks for your reply.

Very cool.

You know what I thought ? I thought you had modified the A4 sheet size
to compute the paperheight and paperwidth ?

I wonder if that's another way of proceeding.

Best,
Ashim

On Sat, Nov 18, 2023 at 12:51 PM Olivier Crouzet
 wrote:
>
> Dear Ashim,
>
> these are documented in the LaTeX 'geometry' package (see for example
> on CTAN: https://ctan.org/pkg/geometry). As I added in my response on
> Stackoverflow, several parts in the RMarkdown header actually concern
> information that are processed by LaTeX to actually generate the PDF,
> among which the 'geometry' line. For someone who is used to working with
> LaTeX, it is relatively natural to 'identify' LaTeX options in these
> lines even though they are not structured exactly this way in a LaTeX
> document (but their names are the same).
>
> Concerning the choice for these specific numbers, it is relatively
> arbitrary. I've been doing this on LaTeX for years when I want to
> generate slides without using the 'beamer' document class and the
> things to take into account are:
>
> - These dimensions express the physical sizes of the pdf page (in a
> sense, what their size would be if you print the document without
> adapting to the paper in your printer)
> - If these sizes are reduced... the relative font size will increase
> because LaTeX will project the same font on a smaller virtual paper,
> - And if these sizes are increased... the relative font size will
> decrease because LaTeX will project the same font on a larger
> virtual paper,
> - I processed starting from 16x9cm sizes, resp. for paperwidth and
> paperheight, and estimated that the fonts were too large to my taste,
> then doubled them and felt that the fonts were too small for slides,
> then I ended up trying multiplying 16x9cm by a factor of 1.5, which
> gave me 24x13.5cm and I found it was ok.
> - But one may vary these sizes arbitrary (even using smaller
> steps and ratios differing from 16/9) depending on the aims to be
> reached.
> - It is also possible to specify different units (pt = points, in =
> inches...)
>
> Yours.
> Olivier.
>
> On Sat, 18 Nov 2023 09:53:19 +0530 Ashim Kapoor 
> wrote:
>
> > Dear Olivier,
> >
> > Many thanks for your reply.
> >
> > This works well for me.
> >
> > How did you come up with the pagewidth / pageheight numbers?  I do
> > understand that their ratio = 16:9,
> > but how did you choose these numbers?
> >
> > Best Regards,
> > Ashim
> >
> > On Fri, Nov 17, 2023 at 9:25 PM Olivier Crouzet
> >  wrote:
> > >
> > > Dear Ashim,
> > >
> > > I don't think the aspectratio is appropriate in this context because
> > > it would imply that the beamer (LaTeX) class is used but you're
> > > actually using the article (LaTeX) class.
> > >
> > > You may use specifications of the geometry package rather than
> > > specifying options to the class:
> > >
> > > e.g. replace your current header:
> > >
> > > ---
> > > title: "Testing landscape and aspect ratio"
> > > output:
> > >   pdf_document:
> > > number_sections: true
> > > classoption:
> > >   - landscape
> > >   - "aspectratio=169"
> > > header-includes:
> > >- \usepackage{dcolumn}
> > > documentclass: article
> > > geometry: margin=1.5cm
> > > ---
> > >
> > > with this one:
> > >
> > > ---
> > > title: "Testing landscape and aspect ratio"
> > > output:
> > >   pdf_document:
> > > number_sections: true
> > > header-includes:
> > >- \usepackage{dcolumn}
> > > documentclass: article
> > > geometry: margin=1.5cm, paperwidth=24cm, paperheight=13.5cm
> > > ---
> > >
> > > Of course, you may change the exact dimensions and it will impact
> > > the relative font sizes. I've tested it and it generates what you
> > > want.
> > >
> > > Yours.
> > > Olivier.
> > >
> > >
> > >
> > >
> > >
> > >  On Tue, 14 Nov 2023 10:03:23
> > > +0530 Ashim Kapoor  wrote:
> > >
> > > > Dear all,
> > > >
> > > > I have posted a query which has received a response but that is
> > > > not working on my computer.
> > > >
> > > > Here is the query:
> > > >
> > > > https://stackoverflow.com/questions/77387434/pdf-from-rmarkdown-landscape-and-aspectratio-169
> > > 

Re: [R] Can someone please have a look at this query on stackoverflow?

[Ashim Kapoor]

Dear Olivier,

Many thanks for your reply.

This works well for me.

How did you come up with the pagewidth / pageheight numbers?  I do
understand that their ratio = 16:9,
but how did you choose these numbers?

Best Regards,
Ashim

On Fri, Nov 17, 2023 at 9:25 PM Olivier Crouzet
 wrote:
>
> Dear Ashim,
>
> I don't think the aspectratio is appropriate in this context because
> it would imply that the beamer (LaTeX) class is used but you're actually
> using the article (LaTeX) class.
>
> You may use specifications of the geometry package rather than
> specifying options to the class:
>
> e.g. replace your current header:
>
> ---
> title: "Testing landscape and aspect ratio"
> output:
>   pdf_document:
> number_sections: true
> classoption:
>   - landscape
>   - "aspectratio=169"
> header-includes:
>- \usepackage{dcolumn}
> documentclass: article
> geometry: margin=1.5cm
> ---
>
> with this one:
>
> ---
> title: "Testing landscape and aspect ratio"
> output:
>   pdf_document:
> number_sections: true
> header-includes:
>- \usepackage{dcolumn}
> documentclass: article
> geometry: margin=1.5cm, paperwidth=24cm, paperheight=13.5cm
> ---
>
> Of course, you may change the exact dimensions and it will impact the
> relative font sizes. I've tested it and it generates what you want.
>
> Yours.
> Olivier.
>
>
>
>
>
>  On Tue, 14 Nov 2023 10:03:23
> +0530 Ashim Kapoor  wrote:
>
> > Dear all,
> >
> > I have posted a query which has received a response but that is not
> > working on my computer.
> >
> > Here is the query:
> >
> > https://stackoverflow.com/questions/77387434/pdf-from-rmarkdown-landscape-and-aspectratio-169
> >
> > Can someone please help me ?
> >
> > Best Regards,
> > Ashim
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html and provide commented,
> > minimal, self-contained, reproducible code.
>
>
> --
>   Olivier Crouzet, PhD
>   http://olivier.ghostinthemachine.space
>   /Maître de Conférences/
>   @LLING - Laboratoire de Linguistique de Nantes
> UMR6310 CNRS / Université de Nantes
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can someone please have a look at this query on stackoverflow?

[Ashim Kapoor]

Hello,

> 1366/768
[1] 1.778646
> 16/9
[1] 1.78
>

There is a mild difference. Is that causing the problem?

Best,
Ashim

On Thu, Nov 16, 2023 at 1:42 PM Ashim Kapoor  wrote:
>
> Dear Bert,
>
> Following your suggestion:
>
> I checked my device resolution is : 1366 x 768 = 16:9
>
> which is the same as is specified in the Rmd file.
>
> I am out of ideas now. Any other idea?
>
> Best Regards,
> Ashim
>
> On Wed, Nov 15, 2023 at 10:32 PM Bert Gunter  wrote:
> >
> > Well, as no one else has offered an answer ...
> >
> > I haven't looked at this closely, but could it not simply be the case that 
> > the aspect ratio set by "Landscape Mode" just differs from that of your 
> > display device? -- i.e., it is impossible to have the figure displayed in 
> > landscape ratio *and* simultaneously fill your display device?
> >
> > If this is obviously wrong, feel free to ignore without replying.
> >
> > Cheers,
> > Bert
> >
> > On Tue, Nov 14, 2023 at 8:48 PM Ashim Kapoor  wrote:
> >>
> >> Dear John,
> >>
> >> Many thanks for your reply.
> >>
> >> I wish 2 things :
> >> 1. Landscape mode
> >> 2. No wasted space on the sides when I maximise the pdf.
> >>
> >> When I download an open and maximise the pdf it wastes space. Please
> >> see the attached screenshot.
> >>
> >> Query: When you maximise the PDF does it occupy the full screen ?
> >>
> >> Best,
> >> Ashim
> >>
> >> On Tue, Nov 14, 2023 at 5:14 PM John Kane  wrote:
> >> >
> >> > I ran the code from the answer and it seems to work well. It, 
> >> > definitely, is giving a landscape output.
> >> >
> >> > ---
> >> > title: "Testing landscape and aspect ratio"
> >> > output:
> >> >   pdf_document:
> >> >     number_sections: true
> >> > classoption:
> >> >   - landscape
> >> >   - "aspectratio=169"
> >> > header-includes:
> >> >- \usepackage{dcolumn}
> >> > documentclass: article
> >> > geometry: margin=1.5cm
> >> > ---
> >> >
> >> > ```{r, out.extra='keepaspectratio=true', out.height='100%', 
> >> > out.width="100%"}
> >> > plot(rnorm(100))
> >> > ```
> >> >
> >> >
> >> >
> >> >
> >> > On Mon, 13 Nov 2023 at 23:33, Ashim Kapoor  wrote:
> >> >>
> >> >> Dear all,
> >> >>
> >> >> I have posted a query which has received a response but that is not
> >> >> working on my computer.
> >> >>
> >> >> Here is the query:
> >> >>
> >> >> https://stackoverflow.com/questions/77387434/pdf-from-rmarkdown-landscape-and-aspectratio-169
> >> >>
> >> >> Can someone please help me ?
> >> >>
> >> >> Best Regards,
> >> >> Ashim
> >> >>
> >> >> __
> >> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> >> PLEASE do read the posting guide 
> >> >> http://www.R-project.org/posting-guide.html
> >> >> and provide commented, minimal, self-contained, reproducible code.
> >> >
> >> >
> >> >
> >> > --
> >> > John Kane
> >> > Kingston ON Canada
> >> __
> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide 
> >> http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can someone please have a look at this query on stackoverflow?

[Ashim Kapoor]

Dear Bert,

Following your suggestion:

I checked my device resolution is : 1366 x 768 = 16:9

which is the same as is specified in the Rmd file.

I am out of ideas now. Any other idea?

Best Regards,
Ashim

On Wed, Nov 15, 2023 at 10:32 PM Bert Gunter  wrote:
>
> Well, as no one else has offered an answer ...
>
> I haven't looked at this closely, but could it not simply be the case that 
> the aspect ratio set by "Landscape Mode" just differs from that of your 
> display device? -- i.e., it is impossible to have the figure displayed in 
> landscape ratio *and* simultaneously fill your display device?
>
> If this is obviously wrong, feel free to ignore without replying.
>
> Cheers,
> Bert
>
> On Tue, Nov 14, 2023 at 8:48 PM Ashim Kapoor  wrote:
>>
>> Dear John,
>>
>> Many thanks for your reply.
>>
>> I wish 2 things :
>> 1. Landscape mode
>> 2. No wasted space on the sides when I maximise the pdf.
>>
>> When I download an open and maximise the pdf it wastes space. Please
>> see the attached screenshot.
>>
>> Query: When you maximise the PDF does it occupy the full screen ?
>>
>> Best,
>> Ashim
>>
>> On Tue, Nov 14, 2023 at 5:14 PM John Kane  wrote:
>> >
>> > I ran the code from the answer and it seems to work well. It, definitely, 
>> > is giving a landscape output.
>> >
>> > ---
>> > title: "Testing landscape and aspect ratio"
>> > output:
>> >   pdf_document:
>> > number_sections: true
>> > classoption:
>> >   - landscape
>> >   - "aspectratio=169"
>> > header-includes:
>> >- \usepackage{dcolumn}
>> > documentclass: article
>> > geometry: margin=1.5cm
>> > ---
>> >
>> > ```{r, out.extra='keepaspectratio=true', out.height='100%', 
>> > out.width="100%"}
>> > plot(rnorm(100))
>> > ```
>> >
>> >
>> >
>> >
>> > On Mon, 13 Nov 2023 at 23:33, Ashim Kapoor  wrote:
>> >>
>> >> Dear all,
>> >>
>> >> I have posted a query which has received a response but that is not
>> >> working on my computer.
>> >>
>> >> Here is the query:
>> >>
>> >> https://stackoverflow.com/questions/77387434/pdf-from-rmarkdown-landscape-and-aspectratio-169
>> >>
>> >> Can someone please help me ?
>> >>
>> >> Best Regards,
>> >> Ashim
>> >>
>> >> __
>> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> >> https://stat.ethz.ch/mailman/listinfo/r-help
>> >> PLEASE do read the posting guide 
>> >> http://www.R-project.org/posting-guide.html
>> >> and provide commented, minimal, self-contained, reproducible code.
>> >
>> >
>> >
>> > --
>> > John Kane
>> > Kingston ON Canada
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can someone please have a look at this query on stackoverflow?

[Ashim Kapoor]

Dear John,

Many thanks for your reply.

I wish 2 things :
1. Landscape mode
2. No wasted space on the sides when I maximise the pdf.

When I download an open and maximise the pdf it wastes space. Please
see the attached screenshot.

Query: When you maximise the PDF does it occupy the full screen ?

Best,
Ashim

On Tue, Nov 14, 2023 at 5:14 PM John Kane  wrote:
>
> I ran the code from the answer and it seems to work well. It, definitely, is 
> giving a landscape output.
>
> ---
> title: "Testing landscape and aspect ratio"
> output:
>   pdf_document:
> number_sections: true
> classoption:
>   - landscape
>   - "aspectratio=169"
> header-includes:
>- \usepackage{dcolumn}
> documentclass: article
> geometry: margin=1.5cm
> ---
>
> ```{r, out.extra='keepaspectratio=true', out.height='100%', out.width="100%"}
> plot(rnorm(100))
> ```
>
>
>
>
> On Mon, 13 Nov 2023 at 23:33, Ashim Kapoor  wrote:
>>
>> Dear all,
>>
>> I have posted a query which has received a response but that is not
>> working on my computer.
>>
>> Here is the query:
>>
>> https://stackoverflow.com/questions/77387434/pdf-from-rmarkdown-landscape-and-aspectratio-169
>>
>> Can someone please help me ?
>>
>> Best Regards,
>> Ashim
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>
>
> --
> John Kane
> Kingston ON Canada
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Can someone please have a look at this query on stackoverflow?

[Ashim Kapoor]

Dear all,

I have posted a query which has received a response but that is not
working on my computer.

Here is the query:

https://stackoverflow.com/questions/77387434/pdf-from-rmarkdown-landscape-and-aspectratio-169

Can someone please help me ?

Best Regards,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] olsrr and substitute lm -> lmRob

[Ashim Kapoor]

Dear all,

I wish to use backward selection from the package olsrr.
?ols_step_backward_p
This gives me a model of class lm.
I wish to then run lmRob from the robust package on the SAME variables
ie I want to substitute lm -> lmRob in the function call which created
the lm model.

Here is a reprex:

library(olsrr)
library(robust)

myformula <- lm(mpg~.,data = mtcars)

# This is how I do backward selection.

chosen.model = ols_step_backward_p(myformula,details=F)

# Chosen model

chosen.model$model

# Not sure if this is the correct thing to do

lmRob(chosen.model$model)

--

My query is : is the above correct ? Do I need to do some meta
programming and substitute lm with lmRob ?
I am not sure how I would do that in this context using the
?substitute function perhaps ?

Many thanks,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] leaps for plm - looking for variable selection software for panel data

[Ashim Kapoor]

Dear all,

I have a small query.

Just like we have package leaps which can do variable selection for a
linear model, do we have any technique/software for variable selection
for panel data?

Here are the links to the mentioned packages:

https://cran.r-project.org/web/packages/leaps/leaps.pdf
https://cran.r-project.org/web/packages/plm/plm.pdf

Many thanks,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Cryptic error from stargazer

[Ashim Kapoor]

Dear Enrico,

--- snip --
> library(stargazer)

Please cite as:

 Hlavac, Marek (2022). stargazer: Well-Formatted Regression and
Summary Statistics Tables.
 R package version 5.2.3. https://CRAN.R-project.org/package=stargazer

>
--- snip--

Yes I do think I now have 5.2.3

Many thanks,
Ashim


On Thu, Jun 8, 2023 at 1:13 PM Enrico Schumann  wrote:
>
> On Thu, 08 Jun 2023, Ashim Kapoor writes:
>
> > Dear All,
> >
> > I had done an automatic upgrade of my Debian 10  system which had also
> > upgraded R.
> >
> > I reinstalled the stargazer package and the error went away.
> >
> > Query : Do I need to reinstall all packages with each upgrade of R ?
> >
> > Best,
> > Ashim
>
> Your session info says "stargazer 5.2.2".  What version do
> you have now?  CRAN has 5.2.3. It seems likely to me that
>
>   i) with R 4.3-0, certain warnings in the package have
>  become errors (the NEWS file for R 4.3-0,
>  https://cloud.r-project.org/doc/manuals/r-release/NEWS.html ,
>  has a prominent first item about using && and errors.)
>
> and
>
>  ii) the package maintainer fixed those warnings/errors in 5.2.3.
>
> kind regards
> Enrico
>
>
> > On Thu, Jun 8, 2023 at 11:11 AM Ashim Kapoor  wrote:
> >>
> >> Dear All,
> >>
> >> Here is my reproducible example:
> >>
> >> > library(stargazer)
> >>
> >> Please cite as:
> >>
> >>  Hlavac, Marek (2018). stargazer: Well-Formatted Regression and
> >> Summary Statistics Tables.
> >>  R package version 5.2.2. https://CRAN.R-project.org/package=stargazer
> >>
> >> > x1=1:1000 ; y = 2 * x1+ rnorm(1000)
> >> > stargazer(lm(y~x1))
> >> Error in (.format.s.statistics.list != "p25") &&
> >> (.format.s.statistics.list !=  :
> >>   'length = 7' in coercion to 'logical(1)'
> >>
> >> The error returned is cryptic and I am not able to google and find the 
> >> solution.
> >>
> >> Here is my sessionInfo:
> >>
> >> > sessionInfo()
> >> R version 4.3.0 (2023-04-21)
> >> Platform: x86_64-pc-linux-gnu (64-bit)
> >> Running under: Debian GNU/Linux 10 (buster)
> >>
> >> Matrix products: default
> >> BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
> >> LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.5.so;  LAPACK version 
> >> 3.8.0
> >>
> >> locale:
> >>  [1] LC_CTYPE=en_IN   LC_NUMERIC=C LC_TIME=en_IN
> >>  [4] LC_COLLATE=en_IN LC_MONETARY=en_INLC_MESSAGES=en_IN
> >>  [7] LC_PAPER=en_IN   LC_NAME=CLC_ADDRESS=C
> >> [10] LC_TELEPHONE=C   LC_MEASUREMENT=en_IN LC_IDENTIFICATION=C
> >>
> >> time zone: Asia/Kolkata
> >> tzcode source: system (glibc)
> >>
> >> attached base packages:
> >> [1] stats graphics  grDevices utils datasets  methods   base
> >>
> >> other attached packages:
> >> [1] stargazer_5.2.2
> >>
> >> loaded via a namespace (and not attached):
> >> [1] compiler_4.3.0
> >>
> >> Any help will be appreciated.
> >>
> >> Thank you,
> >> Ashim
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> --
> Enrico Schumann
> Lucerne, Switzerland
> http://enricoschumann.net

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Cryptic error from stargazer

[Ashim Kapoor]

Dear All,

I had done an automatic upgrade of my Debian 10  system which had also
upgraded R.

I reinstalled the stargazer package and the error went away.

Query : Do I need to reinstall all packages with each upgrade of R ?

Best,
Ashim

On Thu, Jun 8, 2023 at 11:11 AM Ashim Kapoor  wrote:
>
> Dear All,
>
> Here is my reproducible example:
>
> > library(stargazer)
>
> Please cite as:
>
>  Hlavac, Marek (2018). stargazer: Well-Formatted Regression and
> Summary Statistics Tables.
>  R package version 5.2.2. https://CRAN.R-project.org/package=stargazer
>
> > x1=1:1000 ; y = 2 * x1+ rnorm(1000)
> > stargazer(lm(y~x1))
> Error in (.format.s.statistics.list != "p25") &&
> (.format.s.statistics.list !=  :
>   'length = 7' in coercion to 'logical(1)'
>
> The error returned is cryptic and I am not able to google and find the 
> solution.
>
> Here is my sessionInfo:
>
> > sessionInfo()
> R version 4.3.0 (2023-04-21)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Debian GNU/Linux 10 (buster)
>
> Matrix products: default
> BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
> LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.5.so;  LAPACK version 
> 3.8.0
>
> locale:
>  [1] LC_CTYPE=en_IN   LC_NUMERIC=C LC_TIME=en_IN
>  [4] LC_COLLATE=en_IN LC_MONETARY=en_INLC_MESSAGES=en_IN
>  [7] LC_PAPER=en_IN   LC_NAME=CLC_ADDRESS=C
> [10] LC_TELEPHONE=C   LC_MEASUREMENT=en_IN LC_IDENTIFICATION=C
>
> time zone: Asia/Kolkata
> tzcode source: system (glibc)
>
> attached base packages:
> [1] stats graphics  grDevices utils datasets  methods   base
>
> other attached packages:
> [1] stargazer_5.2.2
>
> loaded via a namespace (and not attached):
> [1] compiler_4.3.0
>
> Any help will be appreciated.
>
> Thank you,
> Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Cryptic error from stargazer

[Ashim Kapoor]

Dear All,

Here is my reproducible example:

> library(stargazer)

Please cite as:

 Hlavac, Marek (2018). stargazer: Well-Formatted Regression and
Summary Statistics Tables.
 R package version 5.2.2. https://CRAN.R-project.org/package=stargazer

> x1=1:1000 ; y = 2 * x1+ rnorm(1000)
> stargazer(lm(y~x1))
Error in (.format.s.statistics.list != "p25") &&
(.format.s.statistics.list !=  :
  'length = 7' in coercion to 'logical(1)'

The error returned is cryptic and I am not able to google and find the solution.

Here is my sessionInfo:

> sessionInfo()
R version 4.3.0 (2023-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 10 (buster)

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.5.so;  LAPACK version 3.8.0

locale:
 [1] LC_CTYPE=en_IN   LC_NUMERIC=C LC_TIME=en_IN
 [4] LC_COLLATE=en_IN LC_MONETARY=en_INLC_MESSAGES=en_IN
 [7] LC_PAPER=en_IN   LC_NAME=CLC_ADDRESS=C
[10] LC_TELEPHONE=C   LC_MEASUREMENT=en_IN LC_IDENTIFICATION=C

time zone: Asia/Kolkata
tzcode source: system (glibc)

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

other attached packages:
[1] stargazer_5.2.2

loaded via a namespace (and not attached):
[1] compiler_4.3.0

Any help will be appreciated.

Thank you,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] prcomp - arbitrary direction of the returned principal components

[Ashim Kapoor]

Dear All,

Many thanks for your replies.

My PC1 loading turns out to be :

1/sqrt(2) , -1/sqrt(2)

In simple words : I had 2 variables and I ran prcomp on them. I got my
PC1 as :  .7071068 var1 - .7071068 var2

PC2 turned out to be the same as PC1 with a PLUS replacing the minus,
ie. .7071068 var1 + .7071068 var2

But forget PC2 for the time being.

Now my question is : I am not able to use the rule that : choose the
variable with a bigger magnitude of loading and multiply PC1 by -1 if
needed (to flip the PC1 since any vector x and it's flipped version -x
 are the same vector but with opposite direction) if the variable with
bigger magnitude is of negative sign.

I have an alternative measure of stress which is trending UP and has 2
peaks during 2 recessions and I can see that PC1 is trending DOWN and
has 2 TROUGHS during the same recessions. That's how I wish to FLIP
PC1 with a negative sign.

The data is not mine and I am not at liberty to share it. I can
construct an artificial example but I would need time to do that.

That's what's happening.

Best Regards and
Many thanks.
Ashim









On Thu, Oct 13, 2022 at 5:38 PM Ebert,Timothy Aaron  wrote:
>
> I still do not understand. However, the general approach would be to identify 
> a specific value to test. If the test is TRUE then do "this" otherwise do 
> nothing. Once the test condition is properly identified, the coding easily 
> follows.
>
>  abs() is the same as
> if x<0 then x = -x   (non-R code, just idea)
> The R code might look something more like
> for (number in 1:ncol(x)){
>if (x[3,2] < 0) {
>  x[number, number] = -x[number, number] #only change the diagonal
>}
> }
>
> Depending on what values need to be changed you may need a nested for loop to 
> go through all values of x[number1, number2].
>
> Your words: " I can forcefully use a NEGATIVE sign to FLIP the index when it 
> is LOW." Where it appeared that "low" was defined as values that are 
> negative. You still will have low values (close to zero) and high values (far 
> from zero).
>
> You could make the condition some other value:
>
> if x< -4 then x = -x
>
> If you just want to rotate about zero then
> x = -x
> In this case the positive values will become negative and the negative values 
> positive.
> Add an if test to selectively rotate based on the value of a single test 
> element in x (as in x[3,2]).
>
> In debugging or trouble shooting setting seed is useful. For actual data 
> analysis you should not set seed, or possibly better yet use set.seed(NULL).
>
> Tim
>
>
>
> -Original Message-
> From: Ashim Kapoor 
> Sent: Thursday, October 13, 2022 12:28 AM
> To: Ebert,Timothy Aaron 
> Cc: R Help 
> Subject: Re: [R] prcomp - arbitrary direction of the returned principal 
> components
>
> [External Email]
>
> Dear Aaron,
>
> Many thanks for your reply.
>
> Please allow me to illustrate my query a bit.
>
> I take some data, throw it to prcomp and extract the x data frame from prcomp.
>
> From ?prcomp:
>
>x: if 'retx' is true the value of the rotated data (the centred
>   (and scaled if requested) data multiplied by the 'rotation'
>   matrix) is returned.  Hence, 'cov(x)' is the diagonal matrix
>   'diag(sdev^2)'.  For the formula method, 'napredict()' is
>   applied to handle the treatment of values omitted by the
>   'na.action'.
>
> I consider x[,1] as my index. This makes sense as x[,1] is the projection of 
> the data on the FIRST principal component.
> Now this x[,1] can be a high +ve number or a low -ve number. I can't ignore 
> the sign.
>
> If I ignore the sign by taking the absolute value, the HIGH / LOW stress 
> values will be indistinguishable.
>
> Hence I do not think using absolute values of x[,1] is the solution.
> Yes it will make the results REPRODUCIBLE but that will be at the cost of 
> losing information.
>
> Any other idea ?
>
> Many thanks,
> Ashim
>
> On Wed, Oct 12, 2022 at 5:23 PM Ebert,Timothy Aaron  wrote:
> >
> > Use absolute value
> >
> > Tim
> >
> > -Original Message-
> > From: R-help  On Behalf Of Ashim Kapoor
> > Sent: Wednesday, October 12, 2022 7:48 AM
> > To: R Help 
> > Subject: [R] prcomp - arbitrary direction of the returned principal
> > components
> >
> > [External Email]
> >
> > Dear R experts,
> >
> > From ?prcomp,
> >
> >  snip -
> > Note:
> >
> >  The signs of the columns of the rotation matrix are arbitrary, and
> >  so may differ between different programs for PCA, and even between
> >  different builds

Re: [R] prcomp - arbitrary direction of the returned principal components

[Ashim Kapoor]

Dear Aaron,

Many thanks for your reply.

Please allow me to illustrate my query a bit.

I take some data, throw it to prcomp and extract the x data frame from prcomp.

>From ?prcomp:

   x: if ‘retx’ is true the value of the rotated data (the centred
  (and scaled if requested) data multiplied by the ‘rotation’
  matrix) is returned.  Hence, ‘cov(x)’ is the diagonal matrix
  ‘diag(sdev^2)’.  For the formula method, ‘napredict()’ is
  applied to handle the treatment of values omitted by the
  ‘na.action’.

I consider x[,1] as my index. This makes sense as x[,1] is the
projection of the data on the FIRST principal component.
Now this x[,1] can be a high +ve number or a low -ve number. I can't
ignore the sign.

If I ignore the sign by taking the absolute value, the HIGH / LOW
stress values will be indistinguishable.

Hence I do not think using absolute values of x[,1] is the solution.
Yes it will make the results REPRODUCIBLE but that will be at the cost
of losing information.

Any other idea ?

Many thanks,
Ashim

On Wed, Oct 12, 2022 at 5:23 PM Ebert,Timothy Aaron  wrote:
>
> Use absolute value
>
> Tim
>
> -Original Message-
> From: R-help  On Behalf Of Ashim Kapoor
> Sent: Wednesday, October 12, 2022 7:48 AM
> To: R Help 
> Subject: [R] prcomp - arbitrary direction of the returned principal components
>
> [External Email]
>
> Dear R experts,
>
> From ?prcomp,
>
>  snip -
> Note:
>
>  The signs of the columns of the rotation matrix are arbitrary, and
>  so may differ between different programs for PCA, and even between
>  different builds of R.
>  snip --
>
> My problem is that I am building an index based on Principal Components 
> Analysis.
> When the index is high it should indicate stress in the market. Due to the 
> arbitrary sign sometimes I get an index which is HIGH when there is stress 
> and sometimes I get  the OPPOSITE - an index which is LOW when there is 
> stress.
> This program is shared with other people who may have a different build of R.
>
> I can forcefully use a NEGATIVE sign to FLIP the index when it is LOW.
> That works.
>
> Now my query is : Just like we do set.seed(1234) and force the pattern of 
> generation of random number and make it REPRODUCIBLE, can I do something like 
> :
>
> set.direction.for.vector.in.pca(1234)
>
> Now each time I do prcomp it should choose the SAME ( high or low ) direction 
> of the principle component on ANY computer having ANY version of R installed.
>
> That's what I want. I don't want the the returned principal component to be 
> HIGH(LOW) on my computer and LOW(HIGH) on someone else's computer.
> That would confuse the people the code is shared with.
>
> Is this possible ? How do people deal with this ?
>
> Many thanks,
> Ashim
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-helpdata=05%7C01%7Ctebert%40ufl.edu%7C258ecdf67d1342e9785508daac47cdf3%7C0d4da0f84a314d76ace60a62331e1b84%7C0%7C0%7C638011721656997427%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=Jh00DHZnx%2FbRGgsdqkgEp7qcMzzqcjhxYfJGF1d13PI%3Dreserved=0
> PLEASE do read the posting guide 
> https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.r-project.org%2Fposting-guide.htmldata=05%7C01%7Ctebert%40ufl.edu%7C258ecdf67d1342e9785508daac47cdf3%7C0d4da0f84a314d76ace60a62331e1b84%7C0%7C0%7C638011721656997427%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=p%2BYrpIUZTD1msNJFsE34J1iLCt8yAPsCe334GKm%2BAtk%3Dreserved=0
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] prcomp - arbitrary direction of the returned principal components

[Ashim Kapoor]

Dear R experts,

>From ?prcomp,

 snip -
Note:

 The signs of the columns of the rotation matrix are arbitrary, and
 so may differ between different programs for PCA, and even between
 different builds of R.
 snip --

My problem is that I am building an index based on Principal
Components Analysis.
When the index is high it should indicate stress in the market. Due to
the arbitrary sign sometimes I get an index which is HIGH when there
is stress and sometimes I get  the OPPOSITE - an index which is LOW
when there is stress.
This program is shared with other people who may have a different build of R.

I can forcefully use a NEGATIVE sign to FLIP the index when it is LOW.
That works.

Now my query is : Just like we do set.seed(1234) and force the pattern
of generation of random number and make it REPRODUCIBLE, can I do
something like :

set.direction.for.vector.in.pca(1234)

Now each time I do prcomp it should choose the SAME ( high or low )
direction of the principle component on ANY computer having ANY
version of R installed.

That's what I want. I don't want the the returned principal component
to be HIGH(LOW) on my computer and LOW(HIGH) on someone else's
computer.
That would confuse the people the code is shared with.

Is this possible ? How do people deal with this ?

Many thanks,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Is there a package that can do Fuzzy name matching to standardize names in a single column

[Ashim Kapoor]

Dear Gregg,

This is what I  meant :-

> df1
 Names
1John Good
2  Joe Jackson
3Bob A. Barker
4 John B. Good
5   Joe J. Jackson
6 Bob Allen Barker
7John Good
8 Joe Jack Johnson
9   Bob Barker

> stringdist_left_join(df1,df1,by="Names",max_dist = 3)
Names.x  Names.y
1 John GoodJohn Good
2 John Good John B. Good
3 John GoodJohn Good
4   Joe Jackson  Joe Jackson
5   Joe Jackson   Joe J. Jackson
6 Bob A. BarkerBob A. Barker
7 Bob A. Barker   Bob Barker
8  John B. GoodJohn Good
9  John B. Good John B. Good
10 John B. GoodJohn Good
11   Joe J. Jackson  Joe Jackson
12   Joe J. Jackson   Joe J. Jackson
13 Bob Allen Barker Bob Allen Barker
14John GoodJohn Good
15John Good John B. Good
16John GoodJohn Good
17 Joe Jack Johnson Joe Jack Johnson
18   Bob BarkerBob A. Barker
19   Bob Barker   Bob Barker
>


You can join a table to itself while tinkering with the max_distance function..
Please notice the clusters that have formed. This has to be cleaned up.

This is similar to the answer by Jan van der Laan.

Best Regards,
Ashim

On Wed, Jun 15, 2022 at 9:13 PM Gregg Powell  wrote:
>
>
> Hello Ashim and kind regards for you taking the time to answer back.
>
>
> > library(fuzzyjoin)
> > ?stringdist_left_join
>
> -this will join two tables, but what I am trying to do is just standardize 
> the similarly spelled duplicate names in just the first column of a single 
> table.
>
> I don't think fuzzyjoin will help me in that regard.
>
> Thanks.
> Gregg
> Arizona, USA
>
> --- Original Message ---
> On Wednesday, June 15th, 2022 at 8:04 AM, Ashim Kapoor 
>  wrote:
>
>
> >
>
> >
>
> > Dear Gregg,
> >
>
> > Check this out:
> >
>
> > library(fuzzyjoin)
> > ?stringdist_left_join
> >
>
> > Best Regards,
> > Ashim
> >
>
> > On Wed, Jun 15, 2022 at 8:28 PM Gregg Powell via R-help
> > r-help@r-project.org wrote:
> >
>
> > > Have data sets where there are names, in the first column, client names 
> > > in the second, and Client start date in the third.
> > >
>
> > > There are thousands of these records with thousands of 
> > > names/clients/client start dates. The name is entered each time the 
> > > person begins with a new client such that each person has many entries in 
> > > the name column. Often the names were not entered in a consistent way. 
> > > With and without middle initial, middle name, or various abbreviations 
> > > such as ",RN" at the end of the name.
> > >
>
> > > Is there a package that can do fuzzy name matching so that the names in 
> > > name column get replaced with a "standardized" format - where some type 
> > > of machine learning can pick the most common spelling of each repeat name 
> > > and replace the different variations with the common spelling?
> > >
>
> > > I included an example below. First table includes the names with the 
> > > various spellings. Second table depicts what I hope to achieve.
> > >
>
> > > Again - this is on a large scale - there are something like 10,000 
> > > records with names that need to be standardized.
> > >
>
> > > Name
> > >
>
> > > Client
> > >
>
> > > Client Start Date
> > >
>
> > > John Good
> > >
>
> > > Client 1
> > >
>
> > > 1/1/2020
> > >
>
> > > Joe Jackson
> > >
>
> > > Client 2
> > >
>
> > > 6/1/2020
> > >
>
> > > Bob A. Barker
> > >
>
> > > Client 3
> > >
>
> > > 8/1/2020
> > >
>
> > > John B. Good
> > >
>
> > > Client 4
> > >
>
> > > 10/1/2020
> > >
>
> > > Joe J. Jackson
> > >
>
> > > Client 5
> > >
>
> > > 12/1/2020
> > >
>
> > > Bob Allen Barker
> > >
>
> > > Client 6
> > >
>
> > > 1/1/2021
> > >
>
> > > John Good
> > >
>
> > > Client 7
> > >
>
> > > 5/1/2021
> > >
>
> > > Joe Jack Jackson
> > >
>
> > > Client 8
> > >
>
> > > 8/1/2021
> > >
>
> > > Bob Barker
> > >
>
> > > Client 9
> > >
>
> > >

Re: [R] Is there a package that can do Fuzzy name matching to standardize names in a single column

[Ashim Kapoor]

Dear Gregg,

Check this out:

library(fuzzyjoin)
?stringdist_left_join

Best Regards,
Ashim

On Wed, Jun 15, 2022 at 8:28 PM Gregg Powell via R-help
 wrote:
>
> Have data sets where there are names, in the first column, client names in 
> the second, and Client start date in the third.
>
> There are thousands of these records with thousands of names/clients/client 
> start dates. The name is entered each time the person begins with a new 
> client such that each person has many entries in the name column. Often the 
> names were not entered in a consistent way. With and without middle initial, 
> middle name, or various abbreviations such as ",RN" at the end of the name.
>
> Is there a package that can do fuzzy name matching so that the names in name 
> column get replaced with a "standardized" format - where some type of machine 
> learning can pick the most common spelling of each repeat name and replace 
> the different variations with the common spelling?
>
> I included an example below. First table includes the names with the various 
> spellings. Second table depicts what I hope to achieve.
>
> Again - this is on a large scale - there are something like 10,000 records 
> with names that need to be standardized.
>
>
> Name
>
> Client
>
> Client Start Date
>
> John Good
>
> Client 1
>
> 1/1/2020
>
> Joe Jackson
>
> Client 2
>
> 6/1/2020
>
> Bob A. Barker
>
> Client 3
>
> 8/1/2020
>
> John B. Good
>
> Client 4
>
> 10/1/2020
>
> Joe J. Jackson
>
> Client 5
>
> 12/1/2020
>
> Bob Allen Barker
>
> Client 6
>
> 1/1/2021
>
> John Good
>
> Client 7
>
> 5/1/2021
>
> Joe Jack Jackson
>
> Client 8
>
> 8/1/2021
>
> Bob Barker
>
> Client 9
>
> 12/1/2021
>
>
>
>
>
>
>
> Name
>
> Client
>
> Client Start Date
>
> John Good
>
> Client 1
>
> 1/1/2020
>
> Joe J. Jackson
>
> Client 2
>
> 6/1/2020
>
> Bob A. Barker
>
> Client 3
>
> 8/1/2020
>
> John Good
>
> Client 4
>
> 10/1/2020
>
> Joe J. Jackson
>
> Client 5
>
> 12/1/2020
>
> Bob A. Barker
>
> Client 6
>
> 1/1/2021
>
> John Good
>
> Client 7
>
> 5/1/2021
>
> Joe J. Jackson
>
> Client 8
>
> 8/1/2021
>
> Bob A. Barker
>
> Client 9
>
> 12/1/2021
>
>
>
> THANKS!
>
> Gregg Powell
>
> Arizona, USA__
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R_LIBS var needed to be set after upgrade to R 4.2.2

[Ashim Kapoor]

Dear Sir,

Many thanks for the reply.

Best,
Ashim

On Thu, Jun 2, 2022 at 12:56 PM Martin Maechler
 wrote:
>
> >>>>> Ashim Kapoor
> >>>>> on Wed, 1 Jun 2022 14:30:58 +0530 writes:
>
> > Dear Sir,
> >> > I upgraded to R 4.2.2  on Debian 10 today.
> >>
> >> Well, I assume you mean R 4.2.0 .. at least that one exists.
>
> > My bad, yes I made a typo. I did mean R 4.2.0.
>
> >> > The R shell incantation worked fine and all libraries would load but,
> >> > I needed to point the R_LIBS variable to
> >> > /usr/local/lib/R/site-library/ in order for the R --vanilla < 
> myfile.R
> >> > incantation to find the libraries.
> >>
> >> you mean other installed *packages*
> >>
> >> > May I ask, why was this ? I never needed to do this on any previous
> >> > upgrade to R.
> >>
> >> Well,  for me, the R --vanilla   form also only sees the
> >> 29 (14 "base" + 15 "Recommended") packages that come with R.
>
> > Has this ALWAYS been the case for you ? Even with prior versions of R ?
>
> Yes; I'm sorry this was unclear.
>
> >> Debian (& Ubuntu etc)  have used a similar setup where the
> >> default {R-level}  .libPaths() has contained three libraries,
> >> via
> >>
> >> 
> R_LIBS_SITE=${R_LIBS_SITE-'/usr/local/lib/R/site-library:/usr/lib/R/site-library:/usr/lib/R/library'}
> >>
> >> see
> >>
> >> https://cloud.r-project.org/bin/linux/debian/#pathways-to-r-packages
> >>
> >> also for much more.
> >> Note (also from the above CRAN page
> >> https://cloud.r-project.org/bin/linux/debian/
> >> [ Remember "menu"  "Binaries" -> "Linux" -> "Debian" ]
> >>
> >> The good thing about the Debian (and derivatives) setup is that
> >> it also separates (as I do) the  "packages that come with R" in
> >> one library (= /usr/lib/R/library) from packages that are
> >> installed differently.
>
> > My confusion is : Earlier R --vanilla incantation was working fine,
> > even without my intervening and
> > adding to R_LIBS. That is why I was confused.
>
> > My main query is : Is there anything special to R 4.2.0 which needs
> > R_LIBS to be setup seperately?
>
> to which Jeroen Ooms answered nicely -- thank you, Jeroen!
>
> > I was hit by this problem as well a few weeks ago. You may get a more
> > detailed answer in r-sig-debian or from Dirk directly, but from what I
> > understood, this is now expected behavior: indeed if you start R
> > --vanilla then /usr/local/lib is no longer included in the library path.
>
> > The reason is that R-core made a change in 4.2.0 to pre-set values for
> > R_LIBS_USER and R_LIBS_SITE in Renviron [1] which would take
> > precedence over the proper distro defaults (that include
> > /usr/local/lib) as configured by the r-base deb/rpm packages. To
> > mitigate the problem the r-base deb package moved the appropriate
> > R_LIBS_USER and R_LIBS_SITE definitions into Renviron.site, which
> > takes precedence over Renviron. However a side effect of this solution
> > is that Renviron.site is ignored in --vanilla mode. At least this is
> > my best understanding of the problem.
>
> > [1] https://cran.r-project.org/doc/manuals/r-release/NEWS.html
>
> Yes, as
>
>  R --help | grep -A1 -e --vanilla | head -2
>
> has given
>
>   --vanilla Combine --no-save, --no-restore, --no-site-file,
> --no-init-file and --no-environ
>
> (unchanged for many years).
>
>
> Being a Linux user of more than 25 years, I've not been much of a
> Debian-or-derivatives user in recent years (apart from setting up
> and maintaining Ubuntu LTS on my wife's computer) mainly because
> of lazyness as our IT staff helps me solve all problems with
> Fedora quickly, including lowelevel device-related ones,
> I think that Debian(+derivatives) has always been the exception
> among the Linux distros and for all the others, '--vanilla'
> really meant "vanilla" in a loose sense, i.e., "just base R".
>
> I agree that I had wanted a "vanilla+strawberry" version myself, really.
> which would be just not have the  --no-site-file  switch,
> so we could call it
>--vanilla+site
>
> {Others may hate  "feature creep" and yet more output from
> 'R --help'  ..}
>
> Martin

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R_LIBS var needed to be set after upgrade to R 4.2.2

[Ashim Kapoor]

Dear Sir,

Many thanks.

Best Regards,
Ashim

On Wed, Jun 1, 2022 at 5:36 PM Jeroen Ooms  wrote:
>
> On Wed, Jun 1, 2022 at 11:02 AM Ashim Kapoor  wrote:
> >
> > My confusion is : Earlier R --vanilla incantation was working fine,
> > even without my intervening and
> > adding to R_LIBS. That is why I was confused.
> >
> > My main query is : Is there anything special to R 4.2.0 which needs
> > R_LIBS to be setup seperately?
>
> I was hit by this problem as well a few weeks ago. You may get a more
> detailed answer in r-sig-debian or from Dirk directly, but from what I
> understood, this is now expected behavior: indeed if you start R
> --vanilla then /usr/local/lib is no longer included in the library
> path.
>
> The reason is that R-core made a change in 4.2.0 to pre-set values for
> R_LIBS_USER and R_LIBS_SITE in Renviron [1] which would take
> precedence over the proper distro defaults (that include
> /usr/local/lib) as configured by the r-base deb/rpm packages. To
> mitigate the problem the the r-base deb package moved the appropriate
> R_LIBS_USER and R_LIBS_SITE definitions into Renviron.site, which
> takes precedence over Renviron. However a side effect of this solution
> is that Renviron.site is ignored in --vanilla mode. At least this is
> my best understanding of the problem.
>
> [1] https://cran.r-project.org/doc/manuals/r-release/NEWS.html

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R_LIBS var needed to be set after upgrade to R 4.2.2

[Ashim Kapoor]

Dear Sir,

> > I upgraded to R 4.2.2  on Debian 10 today.
>
> Well, I assume you mean R 4.2.0 .. at least that one exists.

My bad, yes I made a typo. I did mean R 4.2.0.

> > The R shell incantation worked fine and all libraries would load but,
> > I needed to point the R_LIBS variable to
> > /usr/local/lib/R/site-library/ in order for the R --vanilla < myfile.R
> > incantation to find the libraries.
>
> you mean other installed *packages*
>
> > May I ask, why was this ? I never needed to do this on any previous
> > upgrade to R.
>
> Well,  for me, the R --vanilla   form also only sees the
> 29 (14 "base" + 15 "Recommended") packages that come with R.

Has this ALWAYS been the case for you ? Even with prior versions of R ?

> Debian (& Ubuntu etc)  have used a similar setup where the
> default {R-level}  .libPaths() has contained three libraries,
> via
>
> R_LIBS_SITE=${R_LIBS_SITE-'/usr/local/lib/R/site-library:/usr/lib/R/site-library:/usr/lib/R/library'}
>
> see
>
>https://cloud.r-project.org/bin/linux/debian/#pathways-to-r-packages
>
> also for much more.
> Note (also from the above CRAN page
>   https://cloud.r-project.org/bin/linux/debian/
>[ Remember "menu"  "Binaries" -> "Linux" -> "Debian" ]
>
> The good thing about the Debian (and derivatives) setup is that
> it also separates (as I do) the  "packages that come with R" in
> one library (= /usr/lib/R/library) from packages that are
> installed differently.

My confusion is : Earlier R --vanilla incantation was working fine,
even without my intervening and
adding to R_LIBS. That is why I was confused.

My main query is : Is there anything special to R 4.2.0 which needs
R_LIBS to be setup seperately?

THAT is my query.

Best Regards,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can browseURL open a lynx browser?

[Ashim Kapoor]

Dear Sir,

> You mention reading the ?browseURL help page, but seem to make no reference 
> to the available options for invoking the browser of your choice. If the 
> browser you choose isn't behaving when you do this, then isn't this a 
> question for a forum related to that browser?

OK.

Thank you,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Can browseURL open a lynx browser?

[Ashim Kapoor]

Dear Sir,

> I haven't used lynx in a while, but
>
> browseURL("http://www.google.com;, browser = "elinks -remote")
>
> works for me as long as an elinks session is already running.

The above elinks incantation is working for me.

Many thanks,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error with RMySQL

[Ashim Kapoor]

Dear Jim and others,

(One of my colleagues pointed me in this direction)

I think the following is the problem that I am experiencing:-

https://dev.mysql.com/doc/mysql-security-excerpt/5.7/en/problems-connecting.html

The above says :

Some permanent solutions are:

Determine what is wrong with your DNS server and fix it.

Specify IP addresses rather than host names in the MySQL grant tables.

I could NOT determine what is wrong with my DNS server. If someone
here knows how to do that please do share with me. However, I
specified the IP address in place of the host name and that seems to
be working fine.

Best,
Ashim






On Thu, Oct 28, 2021 at 2:24 PM Ashim Kapoor  wrote:
>
> Dear Jim,
>
> I restarted the mariadb.service but I am STILL not able to access the server.
>
> I can access one database from one user but not from another user. The
> credentials seem to be OK to me.
>
> If I can figure this out I will post it here.
>
> Many thanks,
> Ashim
>
> On Thu, Oct 28, 2021 at 1:38 PM Ashim Kapoor  wrote:
> >
> > Dear Jim,
> >
> > Many thanks. I will try your suggestion and come back to you.
> >
> > Many thanks,
> > Ashim
> >
> > On Thu, Oct 28, 2021 at 1:25 PM Jim Lemon  wrote:
> > >
> > > Hi Ashim,
> > > I was out for a while. I would first try to restart MySQL, then if
> > > that didn't fix it, try logging in as root and accessing the database.
> > > While I can't debug this at a distance, I'm pretty sure that the
> > > database thinks that you aren't authorized to access it. If the
> > > restart works, your problem is solved. If you have to log in as root,
> > > it's more difficult to know how this happened.
> > >
> > > Jim
> > >
> > > On Thu, Oct 28, 2021 at 6:13 PM Ashim Kapoor  
> > > wrote:
> > > >
> > > > Dear Jim,
> > > >
> > > > Can you please help me? I am a little confused here.
> > > >
> > > > Best,
> > > > Ashim
> > > >
> > > > On Thu, Oct 28, 2021 at 11:30 AM Ashim Kapoor  
> > > > wrote:
> > > > >
> > > > > Dear Jim,
> > > > >
> > > > > > That error means that your user number or group is not allowed to
> > > > > > access it. Did you create the new one as a different user, maybe as
> > > > > > root?
> > > > >
> > > > > Here is what I did:
> > > > >
> > > > > I switched to root in the MySQL Server, created a new user called
> > > > > user2. I GRANTed user2 all permissions to access db2. I did NOT
> > > > > restart the mysql server as I think the GRANT  does not need a restart
> > > > > of the MySQL server.
> > > > >
> > > > > > Jim
> > > > > >
> > > > > > On Thu, Oct 28, 2021 at 4:42 PM Ashim Kapoor 
> > > > > >  wrote:
> > > > > > >
> > > > > > > Dear R - users,
> > > > > > >
> > > > > > > I have 2 databases on a MySQL server. I am able to access the old 
> > > > > > > one
> > > > > > > but not the freshly created one.
> > > > > > >
> > > > > > > library(RMySQL)
> > > > > > > # This one is the old one, I am able to access it,
> > > > > > > > con1<- dbConnect(MySQL(),host= "myserver", db="db1", user= 
> > > > > > > > "user1",password ="mypass1")
> > > > > > >
> > > > > > > # This is the new one. I am getting an error.
> > > > > > > > con2 <- dbConnect(MySQL(),host= "myserver", db="db2", user= 
> > > > > > > > "user2",password="mypass2")
> > > > > > > Error in .local(drv, ...) :
> > > > > > >   Failed to connect to database: Error: Access denied for user
> > > > > > > 'user2'@'myserver' (using password: YES)
> > > > > > > >
> > > > > > >
> > > > > > > Can someone please point me in the right direction? The error is 
> > > > > > > not
> > > > > > > clear. I do not think I have mistyped the credentials.
> > > > > > >
> > > > > > > Many thanks,
> > > > > > > Ashim
> > > > > > >
> > > > > > > __
> > > > > > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > > > > > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > > > > > PLEASE do read the posting guide 
> > > > > > > http://www.R-project.org/posting-guide.html
> > > > > > > and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error with RMySQL

[Ashim Kapoor]

Dear Jim,

I restarted the mariadb.service but I am STILL not able to access the server.

I can access one database from one user but not from another user. The
credentials seem to be OK to me.

If I can figure this out I will post it here.

Many thanks,
Ashim

On Thu, Oct 28, 2021 at 1:38 PM Ashim Kapoor  wrote:
>
> Dear Jim,
>
> Many thanks. I will try your suggestion and come back to you.
>
> Many thanks,
> Ashim
>
> On Thu, Oct 28, 2021 at 1:25 PM Jim Lemon  wrote:
> >
> > Hi Ashim,
> > I was out for a while. I would first try to restart MySQL, then if
> > that didn't fix it, try logging in as root and accessing the database.
> > While I can't debug this at a distance, I'm pretty sure that the
> > database thinks that you aren't authorized to access it. If the
> > restart works, your problem is solved. If you have to log in as root,
> > it's more difficult to know how this happened.
> >
> > Jim
> >
> > On Thu, Oct 28, 2021 at 6:13 PM Ashim Kapoor  wrote:
> > >
> > > Dear Jim,
> > >
> > > Can you please help me? I am a little confused here.
> > >
> > > Best,
> > > Ashim
> > >
> > > On Thu, Oct 28, 2021 at 11:30 AM Ashim Kapoor  
> > > wrote:
> > > >
> > > > Dear Jim,
> > > >
> > > > > That error means that your user number or group is not allowed to
> > > > > access it. Did you create the new one as a different user, maybe as
> > > > > root?
> > > >
> > > > Here is what I did:
> > > >
> > > > I switched to root in the MySQL Server, created a new user called
> > > > user2. I GRANTed user2 all permissions to access db2. I did NOT
> > > > restart the mysql server as I think the GRANT  does not need a restart
> > > > of the MySQL server.
> > > >
> > > > > Jim
> > > > >
> > > > > On Thu, Oct 28, 2021 at 4:42 PM Ashim Kapoor  
> > > > > wrote:
> > > > > >
> > > > > > Dear R - users,
> > > > > >
> > > > > > I have 2 databases on a MySQL server. I am able to access the old 
> > > > > > one
> > > > > > but not the freshly created one.
> > > > > >
> > > > > > library(RMySQL)
> > > > > > # This one is the old one, I am able to access it,
> > > > > > > con1<- dbConnect(MySQL(),host= "myserver", db="db1", user= 
> > > > > > > "user1",password ="mypass1")
> > > > > >
> > > > > > # This is the new one. I am getting an error.
> > > > > > > con2 <- dbConnect(MySQL(),host= "myserver", db="db2", user= 
> > > > > > > "user2",password="mypass2")
> > > > > > Error in .local(drv, ...) :
> > > > > >   Failed to connect to database: Error: Access denied for user
> > > > > > 'user2'@'myserver' (using password: YES)
> > > > > > >
> > > > > >
> > > > > > Can someone please point me in the right direction? The error is not
> > > > > > clear. I do not think I have mistyped the credentials.
> > > > > >
> > > > > > Many thanks,
> > > > > > Ashim
> > > > > >
> > > > > > __
> > > > > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > > > > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > > > > PLEASE do read the posting guide 
> > > > > > http://www.R-project.org/posting-guide.html
> > > > > > and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error with RMySQL

[Ashim Kapoor]

Dear Jim,

Many thanks. I will try your suggestion and come back to you.

Many thanks,
Ashim

On Thu, Oct 28, 2021 at 1:25 PM Jim Lemon  wrote:
>
> Hi Ashim,
> I was out for a while. I would first try to restart MySQL, then if
> that didn't fix it, try logging in as root and accessing the database.
> While I can't debug this at a distance, I'm pretty sure that the
> database thinks that you aren't authorized to access it. If the
> restart works, your problem is solved. If you have to log in as root,
> it's more difficult to know how this happened.
>
> Jim
>
> On Thu, Oct 28, 2021 at 6:13 PM Ashim Kapoor  wrote:
> >
> > Dear Jim,
> >
> > Can you please help me? I am a little confused here.
> >
> > Best,
> > Ashim
> >
> > On Thu, Oct 28, 2021 at 11:30 AM Ashim Kapoor  wrote:
> > >
> > > Dear Jim,
> > >
> > > > That error means that your user number or group is not allowed to
> > > > access it. Did you create the new one as a different user, maybe as
> > > > root?
> > >
> > > Here is what I did:
> > >
> > > I switched to root in the MySQL Server, created a new user called
> > > user2. I GRANTed user2 all permissions to access db2. I did NOT
> > > restart the mysql server as I think the GRANT  does not need a restart
> > > of the MySQL server.
> > >
> > > > Jim
> > > >
> > > > On Thu, Oct 28, 2021 at 4:42 PM Ashim Kapoor  
> > > > wrote:
> > > > >
> > > > > Dear R - users,
> > > > >
> > > > > I have 2 databases on a MySQL server. I am able to access the old one
> > > > > but not the freshly created one.
> > > > >
> > > > > library(RMySQL)
> > > > > # This one is the old one, I am able to access it,
> > > > > > con1<- dbConnect(MySQL(),host= "myserver", db="db1", user= 
> > > > > > "user1",password ="mypass1")
> > > > >
> > > > > # This is the new one. I am getting an error.
> > > > > > con2 <- dbConnect(MySQL(),host= "myserver", db="db2", user= 
> > > > > > "user2",password="mypass2")
> > > > > Error in .local(drv, ...) :
> > > > >   Failed to connect to database: Error: Access denied for user
> > > > > 'user2'@'myserver' (using password: YES)
> > > > > >
> > > > >
> > > > > Can someone please point me in the right direction? The error is not
> > > > > clear. I do not think I have mistyped the credentials.
> > > > >
> > > > > Many thanks,
> > > > > Ashim
> > > > >
> > > > > __
> > > > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > > > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > > > PLEASE do read the posting guide 
> > > > > http://www.R-project.org/posting-guide.html
> > > > > and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error with RMySQL

[Ashim Kapoor]

Dear Jim,

Can you please help me? I am a little confused here.

Best,
Ashim

On Thu, Oct 28, 2021 at 11:30 AM Ashim Kapoor  wrote:
>
> Dear Jim,
>
> > That error means that your user number or group is not allowed to
> > access it. Did you create the new one as a different user, maybe as
> > root?
>
> Here is what I did:
>
> I switched to root in the MySQL Server, created a new user called
> user2. I GRANTed user2 all permissions to access db2. I did NOT
> restart the mysql server as I think the GRANT  does not need a restart
> of the MySQL server.
>
> > Jim
> >
> > On Thu, Oct 28, 2021 at 4:42 PM Ashim Kapoor  wrote:
> > >
> > > Dear R - users,
> > >
> > > I have 2 databases on a MySQL server. I am able to access the old one
> > > but not the freshly created one.
> > >
> > > library(RMySQL)
> > > # This one is the old one, I am able to access it,
> > > > con1<- dbConnect(MySQL(),host= "myserver", db="db1", user= 
> > > > "user1",password ="mypass1")
> > >
> > > # This is the new one. I am getting an error.
> > > > con2 <- dbConnect(MySQL(),host= "myserver", db="db2", user= 
> > > > "user2",password="mypass2")
> > > Error in .local(drv, ...) :
> > >   Failed to connect to database: Error: Access denied for user
> > > 'user2'@'myserver' (using password: YES)
> > > >
> > >
> > > Can someone please point me in the right direction? The error is not
> > > clear. I do not think I have mistyped the credentials.
> > >
> > > Many thanks,
> > > Ashim
> > >
> > > __
> > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide 
> > > http://www.R-project.org/posting-guide.html
> > > and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error with RMySQL

[Ashim Kapoor]

Dear Jim,

> That error means that your user number or group is not allowed to
> access it. Did you create the new one as a different user, maybe as
> root?

Here is what I did:

I switched to root in the MySQL Server, created a new user called
user2. I GRANTed user2 all permissions to access db2. I did NOT
restart the mysql server as I think the GRANT  does not need a restart
of the MySQL server.

> Jim
>
> On Thu, Oct 28, 2021 at 4:42 PM Ashim Kapoor  wrote:
> >
> > Dear R - users,
> >
> > I have 2 databases on a MySQL server. I am able to access the old one
> > but not the freshly created one.
> >
> > library(RMySQL)
> > # This one is the old one, I am able to access it,
> > > con1<- dbConnect(MySQL(),host= "myserver", db="db1", user= 
> > > "user1",password ="mypass1")
> >
> > # This is the new one. I am getting an error.
> > > con2 <- dbConnect(MySQL(),host= "myserver", db="db2", user= 
> > > "user2",password="mypass2")
> > Error in .local(drv, ...) :
> >   Failed to connect to database: Error: Access denied for user
> > 'user2'@'myserver' (using password: YES)
> > >
> >
> > Can someone please point me in the right direction? The error is not
> > clear. I do not think I have mistyped the credentials.
> >
> > Many thanks,
> > Ashim
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Error with RMySQL

[Ashim Kapoor]

Dear R - users,

I have 2 databases on a MySQL server. I am able to access the old one
but not the freshly created one.

library(RMySQL)
# This one is the old one, I am able to access it,
> con1<- dbConnect(MySQL(),host= "myserver", db="db1", user= "user1",password 
> ="mypass1")

# This is the new one. I am getting an error.
> con2 <- dbConnect(MySQL(),host= "myserver", db="db2", user= 
> "user2",password="mypass2")
Error in .local(drv, ...) :
  Failed to connect to database: Error: Access denied for user
'user2'@'myserver' (using password: YES)
>

Can someone please point me in the right direction? The error is not
clear. I do not think I have mistyped the credentials.

Many thanks,
Ashim

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R does not start from (Debian) linux command line - error with doWithOneRestart() - segmentation fault

[Ashim Kapoor]

Dear Henrik,

We upgraded R and rebooted the computer and the error went away. Hence
I do not think we will get any strange results anymore.

Still, I could be mistaken and I am still curious as to what happened
to my R installation. Please find the answers inline below.

On Thu, Apr 8, 2021 at 9:59 PM Henrik Bengtsson
 wrote:
>
> Ashim, as Martin says, there's something really weird going on with
> your core R installation.  This is definitely not expected, and I
> don't know think if I've every seen this reported before.  Here are
> some questions/comments that might help you move forward and for
> others to pitch in:
>
> 1. How did you install R?
>

IIRC I installed R by doing:

apt-get install r-base r-base-dev

( following the instructions on this page :
https://cloud.r-project.org/ ) I went to this page and clicked on
"Download R for Linux" and then clicked on  "debian". I think, since
the page uses frames, the url does not change when I do those 2 steps.

> 2. What happens if you call:
>
> $ Rscript --vanilla -e "1+2"
>

$ Rscript --vanilla -e "1+2"
[1] 3


> 3. What happens if you call:
>
> $ Rscript --vanilla --default-packages=base -e "1+2"
>

$ Rscript --vanilla --default-packages=base -e "1+2"
[1] 3

> 4. What happens if you call:
>
> $ Rscript --vanilla --default-packages=base,methods -e "1+2"
>
$ Rscript --vanilla --default-packages=base,methods -e "1+2"
[1] 3

> If any of 2-4 gives a different result, that's a first clue.
>
> /Henrik
>

Many thanks for your help,
Best Regards,
Ashim

> On Wed, Apr 7, 2021 at 1:17 AM Ashim Kapoor  wrote:
> >
> > On Wed, Apr 7, 2021 at 12:51 PM Martin Maechler
> >  wrote:
> > >
> > > >>>>> Ashim Kapoor
> > > >>>>> on Wed, 7 Apr 2021 10:35:14 +0530 writes:
> > >
> > > > Dear R experts,
> > >
> > > > Here is my problem :
> > >
> > > > R startup FAILS with an error message. The error message
> > > > is more meaningful when I do invoke R via sudo OR as
> > > > root. I attach the startup messages when I invoke R as :
> > >
> > > > 1. as non root user 2. with sudo 3. as Root user.
> > >
> > > > The error messages ( mentioned in snippets below ) are
> > > > more meaningful to me in the above mentioned order.
> > >
> > > Thank you, Ashim.
> > >
> > > Yes, the messages point to something really bad.
> > >
> > > OTOH ("On the other hand"), what we *can* see is that you try to
> > > start R version 3.6.3.
> > >
> >
> > My apologies for not being clear.
> >
> > I would like to clarify :
> >
> > 1. The error message from the incantation `R` is not very informative.
> >
> > 2. The error message from the incantation `sudo R` says (among other
> > things which don't seem significant to me):
> > Error in doWithOneRestart(return(` \\x82\\x0ccPV`), restart) :
> >   not a proper file name
> >
> > 3. The error message from invoking R as a root user says:
> >
> > Error: not a proper file name
> > Fatal error: unable to initialize the JIT
> >
> > I found this on the internet :
> > https://stackoverflow.com/questions/19512165/error-in-dowithonerestart
> >
> > The error messages and the above link point to issues with JAVA, but
> > it does not say how to fix them.
> >
> > That is why I thought : there is an .xlsx file which has non english
> > characters which is messing with
> > Java.
> >
> > Query : is there a way to do JAVA garbage collection when R is not starting 
> > ?
> >
> > > While that is not extremely old, it may well be older than
> > > several other pieces of software (or even hardware) that you are
> > > running with.
> > >
> > > I very very  *strongly* recommend to use an R version 4.0.x ... and why 
> > > not
> > > use the latest  4.0.5 ?
> > >
> >
> > We upgraded to 4.0.5 but it did not make a difference. We restarted
> > the computer and that fixed the error.
> > I think the JAVA garbage collection kicked in when we reset the
> > computer and that is why it was fixed on rebooting. Not sure though.
> >
> > > Then, it may also be caused by a mismatch of system libraries
> > > and your oldish version of R. ... but there I'd strongly
> > > recommend consulting with other Debian users, notably as there
> > > is a dedicated  mailing list  R-SIG-Debian --> 

Re: [R] R does not start from (Debian) linux command line - error with doWithOneRestart() - segmentation fault

[Ashim Kapoor]

On Wed, Apr 7, 2021 at 12:51 PM Martin Maechler
 wrote:
>
> >>>>> Ashim Kapoor
> >>>>> on Wed, 7 Apr 2021 10:35:14 +0530 writes:
>
> > Dear R experts,
>
> > Here is my problem :
>
> > R startup FAILS with an error message. The error message
> > is more meaningful when I do invoke R via sudo OR as
> > root. I attach the startup messages when I invoke R as :
>
> > 1. as non root user 2. with sudo 3. as Root user.
>
> > The error messages ( mentioned in snippets below ) are
> > more meaningful to me in the above mentioned order.
>
> Thank you, Ashim.
>
> Yes, the messages point to something really bad.
>
> OTOH ("On the other hand"), what we *can* see is that you try to
> start R version 3.6.3.
>

My apologies for not being clear.

I would like to clarify :

1. The error message from the incantation `R` is not very informative.

2. The error message from the incantation `sudo R` says (among other
things which don't seem significant to me):
Error in doWithOneRestart(return(` \\x82\\x0ccPV`), restart) :
  not a proper file name

3. The error message from invoking R as a root user says:

Error: not a proper file name
Fatal error: unable to initialize the JIT

I found this on the internet :
https://stackoverflow.com/questions/19512165/error-in-dowithonerestart

The error messages and the above link point to issues with JAVA, but
it does not say how to fix them.

That is why I thought : there is an .xlsx file which has non english
characters which is messing with
Java.

Query : is there a way to do JAVA garbage collection when R is not starting ?

> While that is not extremely old, it may well be older than
> several other pieces of software (or even hardware) that you are
> running with.
>
> I very very  *strongly* recommend to use an R version 4.0.x ... and why not
> use the latest  4.0.5 ?
>

We upgraded to 4.0.5 but it did not make a difference. We restarted
the computer and that fixed the error.
I think the JAVA garbage collection kicked in when we reset the
computer and that is why it was fixed on rebooting. Not sure though.

> Then, it may also be caused by a mismatch of system libraries
> and your oldish version of R. ... but there I'd strongly
> recommend consulting with other Debian users, notably as there
> is a dedicated  mailing list  R-SIG-Debian --> do subscribe
> there, and ask -- with more details on how you got your R: Is it
> the default R on your Debian, which version of Debian,  etc.
>
> Last but not least, Dirk Eddelbuettel, the maintainer of the
> official R Debian package maintains a nice web page -- part of
> the official CRAN web pages, but unfortunately a bit hidden
> nowadays, (not the least because CRAN still uses frames (würg!!)):
>
>   https://cloud.r-project.org/bin/linux/debian/
>
> A very nice and useful page,  much underrated and underused,
> probably.
>

Thank you for this.

> Best regards,
>
> Martin Maechler
> ETH Zurich  and  R Core Team
>
>
> > When I google around for the error message, it looks like
> > there is an .xlsx file which has non english characters
> > which is messing with Java.
>
> > I do not know how to fix this. I tried :-
>
> > R --vanilla
>
> > so that it would not use any startup scripts but that also
> > does not work.
>
> > - snip
> > 
> 
>
> > When I try to start R from the command line :
>
> > ~$ R
>
> >  *** caught segfault *** address (nil), cause 'unknown'
>
> > Traceback: 1: NextMethod(.Generic) 2:
> > Ops.numeric_version(R_version_built_under, "3.0.0") 3:
> > testRversion(pkgInfo, package, pkgpath) 4:
> > library(package, lib.loc = lib.loc, character.only = TRUE,
> > logical.return = TRUE, warn.conflicts = warn.conflicts,
> > quietly = quietly, mask.ok = mask.ok, exclude = exclude,
> > include.only = include.only, attach.required =
> > attach.required) 5: doTryCatch(return(expr), name,
> > parentenv, handler) 6: tryCatchOne(expr, names, parentenv,
> > handlers[[1L]]) 7: tryCatchList(expr, classes, parentenv,
> > handlers) 8: tryCatch(library(package, lib.loc = lib.loc,
> > character.only = TRUE, logical.return = TRUE,
> > warn.conflicts = warn.conflicts, quietly = quietly,
> > mask.ok = mask.ok, exclude = exclude, include.only =
> &

[R] R does not start from (Debian) linux command line - error with doWithOneRestart() - segmentation fault

[Ashim Kapoor]

Dear R experts,

Here is my problem :

R startup FAILS with an error message. The error message is more
meaningful when I do invoke R via sudo OR as root. I attach the
startup messages when I invoke R as :

1. as non root user
2. with sudo
3. as Root user.

The error messages ( mentioned in snippets below ) are more meaningful
to me in the above mentioned order.

When I google around for the error message, it looks like there is an
.xlsx file which has non english characters which is messing with
Java.

I do not know how to fix this. I tried :-

R --vanilla

so that it would not use any startup scripts but that also does not work.

- snip


When I try to start R from the command line :

~$ R

 *** caught segfault ***
address (nil), cause 'unknown'

Traceback:
 1: NextMethod(.Generic)
 2: Ops.numeric_version(R_version_built_under, "3.0.0")
 3: testRversion(pkgInfo, package, pkgpath)
 4: library(package, lib.loc = lib.loc, character.only = TRUE,
logical.return = TRUE, warn.conflicts = warn.conflicts, quietly =
quietly, mask.ok = mask.ok, exclude = exclude, include.only =
include.only, attach.required = attach.required)
 5: doTryCatch(return(expr), name, parentenv, handler)
 6: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 7: tryCatchList(expr, classes, parentenv, handlers)
 8: tryCatch(library(package, lib.loc = lib.loc, character.only =
TRUE, logical.return = TRUE, warn.conflicts = warn.conflicts,
quietly = quietly, mask.ok = mask.ok, exclude = exclude,
include.only = include.only, attach.required = attach.required),
error = function(e) e)
 9: require(pkg, quietly = TRUE, warn.conflicts = FALSE, character.only = TRUE)
10: .OptRequireMethods()

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: Segmentation fault

- snip

When I try to start R with sudo it gives a more clear message :-

~$ sudo R

 *** caught segfault ***
address (nil), cause 'unknown'

Traceback:
 1: NextMethod(.Generic)
 2: Ops.numeric_version(R_version_built_under, "3.0.0")
 3: testRversion(pkgInfo, package, pkgpath)
 4: library(package, lib.loc = lib.loc, character.only = TRUE,
logical.return = TRUE, warn.conflicts = warn.conflicts, quietly =
quietly, mask.ok = mask.ok, exclude = exclude, include.only =
include.only, attach.required = attach.required)
 5: doTryCatch(return(expr), name, parentenv, handler)
 6: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 7: tryCatchList(expr, classes, parentenv, handlers)
 8: tryCatch(library(package, lib.loc = lib.loc, character.only =
TRUE, logical.return = TRUE, warn.conflicts = warn.conflicts,
quietly = quietly, mask.ok = mask.ok, exclude = exclude,
include.only = include.only, attach.required = attach.required),
error = function(e) e)
 9: require(pkg, quietly = TRUE, warn.conflicts = FALSE, character.only = TRUE)
10: .OptRequireMethods()

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: Error in doWithOneRestart(return(` \\x82\\x0ccPV`), restart) :
  not a proper file name

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Error: not a proper file name
Error in doWithOneRestart(return(` \\x82\\x0ccPV`), restart) :
  promise already under evaluation: recursive default argument
reference or earlier problems?
Segmentation fault

- snip


Invoking R as a root user :-

$ sudo -i
root@crayshrimp ~ # R

 *** caught segfault ***
address (nil), cause 'unknown'

Traceback:
 1: NextMethod(.Generic)
 2: Ops.numeric_version(R_version_built_under, "3.0.0")
 3: testRversion(pkgInfo, package, pkgpath)
 4: library(package, lib.loc = lib.loc, character.only = TRUE,
logical.return = TRUE, warn.conflicts = warn.conflicts, quietly =
quietly, mask.ok = mask.ok, exclude = exclude, include.only =
include.only, attach.required = 

[R] tidyquant package

[Ashim Kapoor]

Dear all,

I am trying to follow along/ recreate this page ( but I do not get the
same results) :-

https://bookdown.org/sstoeckl/Tidy_Portfoliomanagement_in_R/s-2Data.html

Here is a reprex / what I have done till now / my queries ( 3 in
number ) are as comments :-

> library(tidyquant)
Loading required package: lubridate

Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

date

Loading required package: PerformanceAnalytics
Loading required package: xts
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

as.Date, as.Date.numeric


Attaching package: ‘PerformanceAnalytics’

The following object is masked from ‘package:graphics’:

legend

Loading required package: quantmod
Loading required package: TTR
Registered S3 method overwritten by 'quantmod':
  methodfrom
  as.zoo.data.frame zoo
Version 0.4-0 included new data defaults. See ?getSymbols.
══ Need to Learn tidyquant? 
Business Science offers a 1-hour course - Learning Lab #9: Performance
Analysis & Portfolio Optimization with tidyquant!
 Learn more at:
https://university.business-science.io/p/learning-labs-pro 
> library(tidyverse)
── Attaching packages ─── tidyverse 1.3.0 ──
✔ ggplot2 3.2.1 ✔ purrr   0.3.3
✔ tibble  3.0.3 ✔ dplyr   1.0.1
✔ tidyr   1.1.1 ✔ stringr 1.4.0
✔ readr   1.3.1 ✔ forcats 0.4.0
── Conflicts ── tidyverse_conflicts() ──
✖ lubridate::as.difftime() masks base::as.difftime()
✖ lubridate::date()masks base::date()
✖ dplyr::filter()  masks stats::filter()
✖ dplyr::first()   masks xts::first()
✖ lubridate::intersect()   masks base::intersect()
✖ dplyr::lag() masks stats::lag()
✖ dplyr::last()masks xts::last()
✖ lubridate::setdiff() masks base::setdiff()
✖ lubridate::union()   masks base::union()
> tq_exchange_options()
[1] "AMEX"   "NASDAQ" "NYSE"
> tq_index_options()
[1] "DOW"   "DOWGLOBAL" "SP400" "SP500" "SP600"

# Query 1 : The above should also show RUSSELL3000. Why is it not showing?

> tq_get_options()
 [1] "stock.prices"   "stock.prices.japan" "dividends"
 [4] "splits" "economic.data"  "quandl"
 [7] "quandl.datatable"   "tiingo" "tiingo.iex"
[10] "tiingo.crypto"  "alphavantager"  "alphavantage"
[13] "rblpapi"
> glimpse(sp500)
Error in glimpse(sp500) : object 'sp500' not found

# Query 2 : This is working fine in the page mentioned on top. What
mistake have I made?

> tq_exchange("NYSE")
Getting data...

[1] NA
Warning messages:
1: In download.file(url, destfile = tmp, quiet = TRUE) :
  URL 
'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0=nyse=download':
status was 'Failure when receiving data from the peer'
2: In value[[3L]](cond) : Error at nyse during call to tq_exchange.
>
# Query 3 : Is this a networking issue? Should I try after some time ?


Here is my session info:-

> sessionInfo()
R version 3.6.2 (2019-12-12)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS/LAPACK: 
/opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64_lin/libmkl_rt.so

locale:
 [1] LC_CTYPE=en_IN   LC_NUMERIC=C LC_TIME=en_IN
 [4] LC_COLLATE=en_IN LC_MONETARY=en_INLC_MESSAGES=en_IN
 [7] LC_PAPER=en_IN   LC_NAME=CLC_ADDRESS=C
[10] LC_TELEPHONE=C   LC_MEASUREMENT=en_IN LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

other attached packages:
 [1] forcats_0.4.0  stringr_1.4.0
 [3] dplyr_1.0.1purrr_0.3.3
 [5] readr_1.3.1tidyr_1.1.1
 [7] tibble_3.0.3   ggplot2_3.2.1
 [9] tidyverse_1.3.0tidyquant_1.0.1
[11] quantmod_0.4-15TTR_0.23-6
[13] PerformanceAnalytics_2.0.4 xts_0.12-0
[15] zoo_1.8-7  lubridate_1.7.4

loaded via a namespace (and not attached):
 [1] tidyselect_1.1.0 haven_2.2.0  lattice_0.20-38  colorspace_1.4-1
 [5] vctrs_0.3.2  generics_0.0.2   rlang_0.4.7  pillar_1.4.6
 [9] withr_2.1.2  glue_1.4.1   DBI_1.1.0dbplyr_1.4.2
[13] modelr_0.1.5 readxl_1.3.1 lifecycle_0.2.0  Quandl_2.10.0
[17] cellranger_1.1.0 munsell_0.5.0gtable_0.3.0 rvest_0.3.5
[21] curl_4.3 fansi_0.4.1  broom_0.5.3  Rcpp_1.0.3
[25] backports_1.1.5  scales_1.1.0 jsonlite_1.6 fs_1.3.1
[29] hms_0.5.2stringi_1.4.6grid_3.6.2   quadprog_1.5-8
[33] cli_2.0.1tools_3.6.2  magrittr_1.5 lazyeval_0.2.2
[37] crayon_1.3.4 pkgconfig_2.0.3  ellipsis_0.3.0   xml2_1.2.2
[41] reprex_0.3.0 assertthat_0.2.1 httr_1.4.1   rstudioapi_0.10
[45] R6_2.4.1 nlme_3.1-143 compiler_3.6.2
>

Many thanks,
Ashim

Re: [R] The best way for making speciall matrix

[Ashim Kapoor]

Dear Vahid,

Would this help?

> row1<- c(1,0,0,4)
> row2<- c(0,2,0,5)
> row3<- c(0,0,3,6)
> mymatrix <- rbind(row1,row2,row3)
> mymatrix
 [,1] [,2] [,3] [,4]
row11004
row20205
row30036
>

Best Regards,
Ashim

On Sat, May 23, 2020 at 3:16 PM Vahid Borji  wrote:
>
> Hi my friends,
>
> I want to make the below matrix in r:
>
> 1 0 0 4
>
> 0 2 0 5
>
> 0 0 3 6
>
> I used the below code:
>
> matrix(c(1,0,0,0,2,0,0,0,3,4,5,6),nrow=3)
>
> My code works. But I do not like my solution way. I am thinking to find the
> simplest way for making this matrix. Do you think my code is the simplest
> code for making this matrix? If not, could anyone writes a simpler code
> than my one?
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] 2 docker containers with R ?

[Ashim Kapoor]

Dear Bogdan,

Perhaps

https://rstudio.github.io/packrat/

can be of help?

Best,
Ashim

On Sat, Apr 11, 2020 at 4:47 PM Ivan Krylov  wrote:

> On Sat, 11 Apr 2020 03:44:51 -0700
> Bogdan Tanasa  wrote:
>
> > how could I have Seurat2 and Seurat3 on the same machine
>
> What I would try first is to install Seurat2 and Seurat3 in separate
> library directories: add the `lib` argument to install.packages when
> installing a given version and `lib.loc` when loading it using
> library().
>
> While Docker is definitely able to solve this problem, in my opinion,
> virtualising a whole operating system is overkill in this case.
>
> --
> Best regards,
> Ivan
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R Markdown & chunk extraction in R

[Ashim Kapoor]

Dear Yihui,

Can we not 1st use read_chunk to import an Rmd and then do knit_code()$get
? I could be mistaken. Please correct me if I am wrong.

Best,
Ashim

On Wed, Apr 8, 2020 at 8:47 PM Yihui Xie  wrote:

> And please note that knitr::knit_code$get() only works (i.e. returns a
> named list of code chunks) inside a knitr document when the document
> _is being knitted_. It doesn't work outside the document. Ideally, you
> should use the document parser of knitr, but it is not exported.
>
> Regards,
> Yihui
> --
> https://yihui.org
>
> On Wed, Apr 8, 2020 at 2:54 AM Ashim Kapoor  wrote:
> >
> > Dear Sigbert,
> >
> > Also see this :-
> >
> >
> https://www.rdocumentation.org/packages/knitr/versions/1.28/topics/knit_code
> >
> > Best,
> > Ashim
> >
> > On Wed, Apr 8, 2020 at 1:13 PM Ashim Kapoor 
> wrote:
> >
> > > Dear Sigbert,
> > >
> > > Please see this.
> > >
> > > https://bookdown.org/yihui/rmarkdown-cookbook/purl.html
> > >
> > > Best,
> > > Ashim
> > >
> > > On Wed, Apr 8, 2020 at 1:02 PM Sigbert Klinke <
> sigb...@wiwi.hu-berlin.de>
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> exists a possibility to extract chunks from a R Markdown file and to
> > >> return them as (named) list in R?
> > >>
> > >> Thanks Sigbert
> > >>
> > >> --
> > >> https://hu.berlin/sk
> > >> https://hu.berlin/mmstat3
> > >>
> > >> __
> > >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > >> https://stat.ethz.ch/mailman/listinfo/r-help
> > >> PLEASE do read the posting guide
> > >> http://www.R-project.org/posting-guide.html
> > >> and provide commented, minimal, self-contained, reproducible code.
> > >>
> > >
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R Markdown & chunk extraction in R

[Ashim Kapoor]

Dear Sigbert,

Also see this :-

https://www.rdocumentation.org/packages/knitr/versions/1.28/topics/knit_code

Best,
Ashim

On Wed, Apr 8, 2020 at 1:13 PM Ashim Kapoor  wrote:

> Dear Sigbert,
>
> Please see this.
>
> https://bookdown.org/yihui/rmarkdown-cookbook/purl.html
>
> Best,
> Ashim
>
> On Wed, Apr 8, 2020 at 1:02 PM Sigbert Klinke 
> wrote:
>
>> Hi,
>>
>> exists a possibility to extract chunks from a R Markdown file and to
>> return them as (named) list in R?
>>
>> Thanks Sigbert
>>
>> --
>> https://hu.berlin/sk
>> https://hu.berlin/mmstat3
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R Markdown & chunk extraction in R

[Ashim Kapoor]

Dear Sigbert,

Please see this.

https://bookdown.org/yihui/rmarkdown-cookbook/purl.html

Best,
Ashim

On Wed, Apr 8, 2020 at 1:02 PM Sigbert Klinke 
wrote:

> Hi,
>
> exists a possibility to extract chunks from a R Markdown file and to
> return them as (named) list in R?
>
> Thanks Sigbert
>
> --
> https://hu.berlin/sk
> https://hu.berlin/mmstat3
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Program to produce multiple plots

[Ashim Kapoor]

Dear David,

Try this :-
par(mfrow=c(1,2))
hist(rnorm(100))
hist(rnorm(100,100,1))

Best Regards,
Ashim


On Mon, Mar 30, 2020 at 7:16 PM David  wrote:

> I wrote a little program that asked to produce two plots, but when I ran
> it, I saw only the second plot.  Is there a way to write a program that
> will produce multiple visible plots?
>
> David
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Correct place to put an import directive

[Ashim Kapoor]

Dear All,

This is a cross post of the following query:-

https://stackoverflow.com/questions/59946803/import-directive-in-r-which-file-and-where-in-that-file-should-we-put-this

Since no one answered this query, so I am posting this here.

Can someone please help  me ?

Thank you,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Tables from Rmarkdown to Word Document - using huxtables

[Ashim Kapoor]

Dear Rui,

No problem and Many thanks.

Best Regards,
Ashim

On Fri, Dec 13, 2019 at 12:09 AM Rui Barradas  wrote:

> Hello,
>
> You're right, none of the two seems to be working.
>
> set_width gives me tables of the same width no matter what value I pass
> as argument.
>
> And set_caption is asking for a flextable object, doesn't work with
> huxtable.
>
> I don't know how to solve it right now, I will try later.
>
> Rui Barradas
>
> Às 10:51 de 12/12/19, Ashim Kapoor escreveu:
> > Hello once again,
> >
> > for me,set_caption is not working as well. Here is my Rmd file :-
> >
> > ---
> > title: Testing Huxtables
> > author: Ashim Kapoor
> > output: word_document
> > ---
> >
> > ```{r}
> > library(dplyr)
> > library(huxtable)
> >
> > hx <- iris %>%
> > group_by(Species) %>%
> > summarise_if(is.numeric, mean) %>%
> > as_hux() %>%
> > add_colnames() %>%
> > set_bold(1, , TRUE) %>%
> > set_bottom_border(1, , 1) %>%
> > set_width(0.3) %>%
> > set_col_width(1:5, 1.5) %>%
> > set_number_format(2) %>%
> > set_caption("Table 1:")
> >
> > hx
> > ```
> >
> > Thank you,
> > Ashim
> >
> > On Thu, Dec 12, 2019 at 12:57 PM Ashim Kapoor  > <mailto:ashimkap...@gmail.com>> wrote:
> >
> >
> >
> > On Wed, Dec 11, 2019 at 9:11 PM Rui Barradas  > <mailto:ruipbarra...@sapo.pt>> wrote:
> >
> > Hello,
> >
> > This works for me:
> >
> >
> > library(dplyr)
> > library(huxtable)
> > library(flextable)
> > library(officer)
> >
> > hx <- iris %>%
> > group_by(Species) %>%
> > summarise_if(is.numeric, mean) %>%
> > as_hux() %>%
> > add_colnames() %>%
> > set_bold(1, , TRUE) %>%
> > set_bottom_border(1, , 1) %>%
> > set_width(0.99) %>%
> > set_col_width(1:5, 0.99) %>%
> > set_number_format(2)
> >
> > hx
> >
> > quick_docx(hx, file = "test.docx")
> >
> > Does set_width work for you ? For me modifying the argument to
> > set_width is NOT working.
> >
> > There are ways of doing the same without pipes, those functions
> > don't
> > have the prefix 'set_'. But I believe that what's important is
> > function
> > ?quick_docx.
> >
> > Hope this helps,
> >
> > Rui Barradas
> >
> > Às 11:37 de 11/12/19, Ashim Kapoor escreveu:
> >  > Dear All,
> >  >
> >  > I am reading this :-
> >  >
> >  > https://hughjonesd.github.io/huxtable/huxtable.html
> >  >
> >  > I quote from the above:
> >  >
> >  > If you want to create Word or Powerpoint documents, install
> > the flextable
> >  > package <https://cran.r-project.org/package=flextable> from
> > CRAN. Huxtables
> >  > can then be automatically printed in Word documents. Or you
> > can convert
> >  > them to flextable objects and include them in Word or
> > Powerpoint documents.
> >  >
> >  > My query is how do I do the former ? How do I do this --->
> > Huxtables can
> >  > then be automatically printed in Word documents.
> >  >
> >  > I do understand how to do this ---> Or you can convert them
> > to flextable
> >  > objects and include them in Word or Powerpoint documents.
> >  >
> >  > Thank you,
> >  > Ashim
> >  >
> >  >   [[alternative HTML version deleted]]
> >  >
> >  > __
> >  > R-help@r-project.org <mailto:R-help@r-project.org> mailing
> > list -- To UNSUBSCRIBE and more, see
> >  > https://stat.ethz.ch/mailman/listinfo/r-help
> >  > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> >  > and provide commented, minimal, self-contained, reproducible
> > code.
> >  >
> >
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Tables from Rmarkdown to Word Document - using huxtables

[Ashim Kapoor]

Hello once again,

for me,set_caption is not working as well. Here is my Rmd file :-

---
title: Testing Huxtables
author: Ashim Kapoor
output: word_document
---

```{r}
library(dplyr)
library(huxtable)

hx <- iris %>%
   group_by(Species) %>%
   summarise_if(is.numeric, mean) %>%
   as_hux() %>%
   add_colnames() %>%
   set_bold(1, , TRUE) %>%
   set_bottom_border(1, , 1) %>%
   set_width(0.3) %>%
   set_col_width(1:5, 1.5) %>%
   set_number_format(2) %>%
   set_caption("Table 1:")

hx
```

Thank you,
Ashim

On Thu, Dec 12, 2019 at 12:57 PM Ashim Kapoor  wrote:

>
>
> On Wed, Dec 11, 2019 at 9:11 PM Rui Barradas  wrote:
>
>> Hello,
>>
>> This works for me:
>>
>>
>> library(dplyr)
>> library(huxtable)
>> library(flextable)
>> library(officer)
>>
>> hx <- iris %>%
>>group_by(Species) %>%
>>summarise_if(is.numeric, mean) %>%
>>as_hux() %>%
>>add_colnames() %>%
>>set_bold(1, , TRUE) %>%
>>set_bottom_border(1, , 1) %>%
>>set_width(0.99) %>%
>>set_col_width(1:5, 0.99) %>%
>>set_number_format(2)
>>
>> hx
>>
>> quick_docx(hx, file = "test.docx")
>>
>>
> Does set_width work for you ? For me modifying the argument to set_width
> is NOT working.
>
>
>> There are ways of doing the same without pipes, those functions don't
>> have the prefix 'set_'. But I believe that what's important is function
>> ?quick_docx.
>>
>> Hope this helps,
>>
>> Rui Barradas
>>
>> Às 11:37 de 11/12/19, Ashim Kapoor escreveu:
>> > Dear All,
>> >
>> > I am reading this :-
>> >
>> > https://hughjonesd.github.io/huxtable/huxtable.html
>> >
>> > I quote from the above:
>> >
>> > If you want to create Word or Powerpoint documents, install the
>> flextable
>> > package <https://cran.r-project.org/package=flextable> from CRAN.
>> Huxtables
>> > can then be automatically printed in Word documents. Or you can convert
>> > them to flextable objects and include them in Word or Powerpoint
>> documents.
>> >
>> > My query is how do I do the former ? How do I do this --->  Huxtables
>> can
>> > then be automatically printed in Word documents.
>> >
>> > I do understand how to do this ---> Or you can convert them to flextable
>> > objects and include them in Word or Powerpoint documents.
>> >
>> > Thank you,
>> > Ashim
>> >
>> >   [[alternative HTML version deleted]]
>> >
>> > __
>> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>> >
>>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Tables from Rmarkdown to Word Document - using huxtables

[Ashim Kapoor]

On Wed, Dec 11, 2019 at 9:11 PM Rui Barradas  wrote:

> Hello,
>
> This works for me:
>
>
> library(dplyr)
> library(huxtable)
> library(flextable)
> library(officer)
>
> hx <- iris %>%
>group_by(Species) %>%
>summarise_if(is.numeric, mean) %>%
>as_hux() %>%
>add_colnames() %>%
>set_bold(1, , TRUE) %>%
>set_bottom_border(1, , 1) %>%
>set_width(0.99) %>%
>set_col_width(1:5, 0.99) %>%
>set_number_format(2)
>
> hx
>
> quick_docx(hx, file = "test.docx")
>
>
Does set_width work for you ? For me modifying the argument to set_width is
NOT working.


> There are ways of doing the same without pipes, those functions don't
> have the prefix 'set_'. But I believe that what's important is function
> ?quick_docx.
>
> Hope this helps,
>
> Rui Barradas
>
> Às 11:37 de 11/12/19, Ashim Kapoor escreveu:
> > Dear All,
> >
> > I am reading this :-
> >
> > https://hughjonesd.github.io/huxtable/huxtable.html
> >
> > I quote from the above:
> >
> > If you want to create Word or Powerpoint documents, install the flextable
> > package <https://cran.r-project.org/package=flextable> from CRAN.
> Huxtables
> > can then be automatically printed in Word documents. Or you can convert
> > them to flextable objects and include them in Word or Powerpoint
> documents.
> >
> > My query is how do I do the former ? How do I do this --->  Huxtables can
> > then be automatically printed in Word documents.
> >
> > I do understand how to do this ---> Or you can convert them to flextable
> > objects and include them in Word or Powerpoint documents.
> >
> > Thank you,
> > Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Tables from Rmarkdown to Word Document - using huxtables

[Ashim Kapoor]

Dear All,

I am reading this :-

https://hughjonesd.github.io/huxtable/huxtable.html

I quote from the above:

If you want to create Word or Powerpoint documents, install the flextable
package  from CRAN. Huxtables
can then be automatically printed in Word documents. Or you can convert
them to flextable objects and include them in Word or Powerpoint documents.

My query is how do I do the former ? How do I do this --->  Huxtables can
then be automatically printed in Word documents.

I do understand how to do this ---> Or you can convert them to flextable
objects and include them in Word or Powerpoint documents.

Thank you,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Orthogonal polynomials used by R

[Ashim Kapoor]

 raw = T)18  5.181e-23  1.076e-22   0.4820.631
poly(x, 20, raw = T)19 NA NA  NA   NA
poly(x, 20, raw = T)20 -7.173e-28  1.480e-27  -0.4850.629

Residual standard error: 641000 on 76 degrees of freedom
Multiple R-squared:  0.9982, Adjusted R-squared:  0.9979
F-statistic:  2690 on 16 and 76 DF,  p-value: < 2.2e-16

>

Note,however, once the orthogonal polynomials have suggested a number, 4 in
this case, I can do this :-

 summary(lm(gdp ~ poly(x,4,raw=T)))

Call:
lm(formula = gdp ~ poly(x, 4, raw = T))

Residuals:
 Min   1Q   Median   3Q  Max
-1278673  -424315   -22357   310977  1731813

Coefficients:
   Estimate Std. Error t value Pr(>|t|)
(Intercept)   3.022e+06  3.676e+05   8.220 1.64e-12 ***
poly(x, 4, raw = T)1  1.741e+05  5.357e+04   3.249  0.00164 **
poly(x, 4, raw = T)2 -6.434e+03  2.300e+03  -2.797  0.00633 **
poly(x, 4, raw = T)3  1.878e+02  3.667e+01   5.123 1.76e-06 ***
poly(x, 4, raw = T)4 -8.682e-01  1.935e-01  -4.486 2.19e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 663700 on 88 degrees of freedom
Multiple R-squared:  0.9978, Adjusted R-squared:  0.9977
F-statistic: 1.003e+04 on 4 and 88 DF,  p-value: < 2.2e-16

>

Although due to correlations they may not be significant jointly, but in
this case all 4 powers come out significant.


Second, the model formula gdp~1+x+x^2 and other similar formulas in your
> message don't do what you think. Like + and *, the ^ operator has special
> meaning on the right-hand side of an R model formula. See ?Formula and
> perhaps read something about statistical models in R. For example:
>
> > x <- 1:93
> > y <- 1 + x + x^2 + x^3 + x^4 + rnorm(93)
> > (m <- lm(y ~ x + x^2))
>
> Call:
> lm(formula = y ~ x + x^2)
>
> Coefficients:
> (Intercept)x
>   -15781393   667147
>
> While gpp ~ x + I(x^2) would work, a better way to fit a raw quadratic is
> as gdp ~ poly(x, 2, raw=TRUE), as I suggested in my earlier message.
>

My bad. Yes, I have some idea of the Wilkinson-Rogers notation. I have seen
it in books, although it slipped my mind that I had to use I( ).


> Finally, as to what you should do, I generally try to avoid statistical
> consulting by email. If you can find competent statistical help locally,
> such as at a nearby university, I'd recommend talking to someone about the
> purpose of your research and the nature of your data. If that's not
> possible, then others have suggested where you might find help, but to get
> useful advice you'll have to provide much more information about your
> research.
>

My original query was about the polynomials used by R which I think is ON
topic. My apologies that this query turned into a statistics query.


> Best,
>  John
>
>   -----
>   John Fox, Professor Emeritus
>   McMaster University
>   Hamilton, Ontario, Canada
>   Web: http::/socserv.mcmaster.ca/jfox
>
> > On Nov 28, 2019, at 12:46 AM, Ashim Kapoor 
> wrote:
> >
> > Dear Peter and John,
> >
> > Many thanks for your prompt replies.
> >
> > Here is what I was trying to do.  I was trying to build a statistical
> model of a given time series using Box Jenkins methodology. The series has
> 93 data points. Before I analyse the ACF and PACF, I am required to
> de-trend the series. The series seems to have an upward trend. I wanted to
> find out what order polynomial should I fit the series
> > without overfitting.  For this I want to use orthogonal polynomials(I
> think someone on the internet was talking about preventing overfitting by
> using orthogonal polynomials) . This seems to me as a poor man's cross
> validation.
> >
> > So my plan is to keep increasing the degree of the orthogonal
> polynomials till the coefficient of the last orthogonal polynomial becomes
> insignificant.
> >
> > Note : If I do NOT use orthogonal polynomials, I will overfit the data
> set and I don't think that is a good way to detect the true order of the
> polynomial.
> >
> > Also now that I have detrended the series and built an ARIMA model of
> the residuals, now I want to forecast. For this I need to use the original
> polynomials and their coefficients.
> >
> > I hope I was clear and that my methodology is ok.
> >
> > I have another query here :-
> >
> > Note : If I used cross-validation to determine the order of the
> polynomial, I don't get a clear answer.
> >
> > See here :-
> > library(boot)
> > mydf = data.frame(cbind(gdp,x))
> > d<-(c(
> > cv.glm(data = mydf,glm(gdp~x),K=10)$delta[1],
> > cv.glm(data = mydf,glm(gdp~poly(x,2)),K=10)$delta[1],
> > cv.glm(data = mydf,

Re: [R] Orthogonal polynomials used by R

[Ashim Kapoor]

Dear Bert,

OK and thank you.

@Fox, John  will be grateful for an offline reply.

Best,
Ashim

On Thu, Nov 28, 2019 at 11:43 AM Bert Gunter  wrote:

> Statistical questions are generally off topic on this list. Try
> stats.stackexchange.com instead.
>
> But FWIW, I recommend that you work with someone with expertise in time
> series analysis, as your efforts to shake and bake your own methodology
> seem rather unwise to me.
>
> Cheers,
> Bert
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along and
> sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Wed, Nov 27, 2019 at 9:47 PM Ashim Kapoor 
> wrote:
>
>> Dear Peter and John,
>>
>> Many thanks for your prompt replies.
>>
>> Here is what I was trying to do.  I was trying to build a statistical
>> model
>> of a given time series using Box Jenkins methodology. The series has 93
>> data points. Before I analyse the ACF and PACF, I am required to de-trend
>> the series. The series seems to have an upward trend. I wanted to find out
>> what order polynomial should I fit the series
>> without overfitting.  For this I want to use orthogonal polynomials(I
>> think
>> someone on the internet was talking about preventing overfitting by using
>> orthogonal polynomials) . This seems to me as a poor man's cross
>> validation.
>>
>> So my plan is to keep increasing the degree of the orthogonal polynomials
>> till the coefficient of the last orthogonal polynomial becomes
>> insignificant.
>>
>> Note : If I do NOT use orthogonal polynomials, I will overfit the data set
>> and I don't think that is a good way to detect the true order of the
>> polynomial.
>>
>> Also now that I have detrended the series and built an ARIMA model of the
>> residuals, now I want to forecast. For this I need to use the original
>> polynomials and their coefficients.
>>
>> I hope I was clear and that my methodology is ok.
>>
>> I have another query here :-
>>
>> Note : If I used cross-validation to determine the order of the
>> polynomial,
>> I don't get a clear answer.
>>
>> See here :-
>> library(boot)
>> mydf = data.frame(cbind(gdp,x))
>> d<-(c(
>> cv.glm(data = mydf,glm(gdp~x),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~poly(x,2)),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~poly(x,3)),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~poly(x,4)),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~poly(x,5)),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~poly(x,6)),K=10)$delta[1]))
>> print(d)
>> ## [1] 2.178574e+13 7.303031e+11 5.994783e+11 4.943586e+11 4.596648e+11
>> ## [6] 4.980159e+11
>>
>> # Here it chooses 5. (but 4 and 5 are kind of similar).
>>
>>
>> d1 <- (c(
>> cv.glm(data = mydf,glm(gdp~1+x),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~1+x+x^2),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~1+x+x^2+x^3),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~1+x+x^2+x^3+x^4),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~1+x+x^2+x^3+x^4+x^5),K=10)$delta[1],
>> cv.glm(data = mydf,glm(gdp~1+x+x^2+x^3+x^4+x^5+x^6),K=10)$delta[1]))
>>
>> print(d1)
>> ## [1] 2.149647e+13 2.253999e+13 2.182175e+13 2.177170e+13 2.198675e+13
>> ## [6] 2.145754e+13
>>
>> # here it chooses 1 or 6
>>
>> Query : Why does it choose 1? Notice : Is this just round off noise /
>> noise
>> due to sampling error created by Cross Validation when it creates the K
>> folds? Is this due to the ill conditioned model matrix?
>>
>> Best Regards,
>> Ashim.
>>
>>
>>
>>
>>
>> On Wed, Nov 27, 2019 at 10:41 PM Fox, John  wrote:
>>
>> > Dear Ashim,
>> >
>> > Orthogonal polynomials are used because they tend to produce more
>> accurate
>> > numerical computations, not because their coefficients are
>> interpretable,
>> > so I wonder why you're interested in the coefficients.
>> >
>> > The regressors produced are orthogonal to the constant regressor and are
>> > orthogonal to each other (and in fact are orthonormal), as it's simple
>> to
>> > demonstrate:
>> >
>> > --- snip ---
>> >
>> > > x <- 1:93
>> > > y <- 1 + x + x^2 + x^3 + x^4 + rnorm(93)
>> > > (m <- lm(y ~ poly(x, 4)))
>> >
>> > Call:
>> > lm(formula = y ~ poly(x, 4))
>> >
>> > Coefficients:
>> > (Intercept

Re: [R] Orthogonal polynomials used by R

[Ashim Kapoor]

0037
>  1.
> poly(x, 4, raw = TRUE)4
>  1.
>
> --- snip ---
>
> These coefficients are simply interpretable but the model matrix is more
> poorly conditioned:
>
> --- snip ---
>
> > head(X1)
>   (Intercept) poly(x, 4, raw = TRUE)1 poly(x, 4, raw = TRUE)2 poly(x, 4,
> raw = TRUE)3
> 1   1   1   1
>  1
> 2   1   2   4
>  8
> 3   1   3   9
> 27
> 4   1   4  16
> 64
> 5   1   5  25
>125
> 6   1   6  36
>216
>   poly(x, 4, raw = TRUE)4
> 1   1
> 2  16
> 3  81
> 4 256
> 5 625
> 61296
> > round(cor(X1[, -1]), 2)
> poly(x, 4, raw = TRUE)1 poly(x, 4, raw = TRUE)2
> poly(x, 4, raw = TRUE)3
> poly(x, 4, raw = TRUE)11.000.97
> 0.92
> poly(x, 4, raw = TRUE)20.971.00
> 0.99
> poly(x, 4, raw = TRUE)30.920.99
> 1.00
> poly(x, 4, raw = TRUE)40.870.96
> 0.99
> poly(x, 4, raw = TRUE)4
> poly(x, 4, raw = TRUE)10.87
> poly(x, 4, raw = TRUE)20.96
> poly(x, 4, raw = TRUE)30.99
> poly(x, 4, raw = TRUE)41.00
>
> --- snip ---
>
> The two parametrizations are equivalent, however, in that they represent
> the same regression surface, and so, e.g., produce the same fitted values:
>
> --- snip ---
>
> > all.equal(fitted(m), fitted(m1))
> [1] TRUE
>
> --- snip ---
>
> Because one is usually not interested in the individual coefficients of a
> polynomial there usually isn't a reason to prefer one parametrization to
> the other on the grounds of interpretability, so why do you need to
> interpret the regression equation?
>
> I hope this helps,
>  John
>
>   -
>   John Fox, Professor Emeritus
>   McMaster University
>   Hamilton, Ontario, Canada
>   Web: http::/socserv.mcmaster.ca/jfox
>
> > On Nov 27, 2019, at 10:17 AM, Ashim Kapoor 
> wrote:
> >
> > Dear Petr,
> >
> > Many thanks for the quick response.
> >
> > I also read this:-
> > https://en.wikipedia.org/wiki/Discrete_orthogonal_polynomials
> >
> > Also I read  in ?poly:-
> > The orthogonal polynomial is summarized by the coefficients, which
> > can be used to evaluate it via the three-term recursion given in
> > Kennedy & Gentle (1980, pp. 343-4), and used in the ‘predict’ part
> > of the code.
> >
> > I don't have access to the mentioned book.
> >
> > Out of curiosity, what is the name of the discrete orthogonal polynomial
> > used by R ?
> > What discrete measure is it orthogonal with respect to ?
> >
> > Many thanks,
> > Ashim
> >
> >
> >
> >
> > On Wed, Nov 27, 2019 at 6:11 PM PIKAL Petr 
> wrote:
> >
> >> You could get answer quickly by searching net.
> >>
> >>
> >>
> https://stackoverflow.com/questions/39031172/how-poly-generates-orthogonal-p
> >> olynomials-how-to-understand-the-coefs-ret/39051154#39051154
> >> <
> https://stackoverflow.com/questions/39031172/how-poly-generates-orthogonal-polynomials-how-to-understand-the-coefs-ret/39051154#39051154
> >
> >>
> >> Cheers
> >> Petr
> >>
> >>> -Original Message-
> >>> From: R-help  On Behalf Of Ashim Kapoor
> >>> Sent: Wednesday, November 27, 2019 12:55 PM
> >>> To: R Help 
> >>> Subject: [R] Orthogonal polynomials used by R
> >>>
> >>> Dear All,
> >>>
> >>> I have created a time trend by doing x<-1:93 because I have a time
> series
> >>> with 93 data points. Next I did :-
> >>>
> >>> y = lm(series ~ poly(x,4))$residuals
> >>>
> >>> to detrend series.
> >>>
> >>> I choose this 4 as the order of my polynomial using cross validation/
> >>> checking the absence of trend in the residuals so I think I ha

Re: [R] Orthogonal polynomials used by R

[Ashim Kapoor]

Dear Petr,

Many thanks for the quick response.

I also read this:-
https://en.wikipedia.org/wiki/Discrete_orthogonal_polynomials

Also I read  in ?poly:-
 The orthogonal polynomial is summarized by the coefficients, which
 can be used to evaluate it via the three-term recursion given in
 Kennedy & Gentle (1980, pp. 343-4), and used in the ‘predict’ part
 of the code.

I don't have access to the mentioned book.

Out of curiosity, what is the name of the discrete orthogonal polynomial
used by R ?
What discrete measure is it orthogonal with respect to ?

Many thanks,
Ashim




On Wed, Nov 27, 2019 at 6:11 PM PIKAL Petr  wrote:

> You could get answer quickly by searching net.
>
>
> https://stackoverflow.com/questions/39031172/how-poly-generates-orthogonal-p
> olynomials-how-to-understand-the-coefs-ret/39051154#39051154
> <https://stackoverflow.com/questions/39031172/how-poly-generates-orthogonal-polynomials-how-to-understand-the-coefs-ret/39051154#39051154>
>
> Cheers
> Petr
>
> > -Original Message-----
> > From: R-help  On Behalf Of Ashim Kapoor
> > Sent: Wednesday, November 27, 2019 12:55 PM
> > To: R Help 
> > Subject: [R] Orthogonal polynomials used by R
> >
> > Dear All,
> >
> > I have created a time trend by doing x<-1:93 because I have a time series
> > with 93 data points. Next I did :-
> >
> > y = lm(series ~ poly(x,4))$residuals
> >
> > to detrend series.
> >
> > I choose this 4 as the order of my polynomial using cross validation/
> > checking the absence of trend in the residuals so I think I have not
> overfit
> > this series.
> >
> > I wish to document the formula of poly(x,4). I am not able to find it in
> ?poly
> >
> > Can someone please tell me what the formula for the orthogonal
> > polynomial used by R is ?
> >
> > Thank you,
> > Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-
> > guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Orthogonal polynomials used by R

[Ashim Kapoor]

Dear All,

I have created a time trend by doing x<-1:93 because I have a time series
with 93 data points. Next I did :-

y = lm(series ~ poly(x,4))$residuals

to detrend series.

I choose this 4 as the order of my polynomial using cross validation/
checking the absence of trend in the residuals so I think I have not
overfit this series.

I wish to document the formula of poly(x,4). I am not able to find it in
?poly

Can someone please tell me what the formula for the orthogonal polynomial
used by R is ?

Thank you,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Double / single backticks for inline R code in an Rmarkdown file

[Ashim Kapoor]

Dear Yihui,

Thank you and Best Regards,
Ashim

On Mon, Oct 14, 2019 at 7:30 PM Yihui Xie  wrote:

> params$data is "hawaii" in this case, so the output of
>
>   ``r params$data``
>
> is
>
>   `hawaii`
>
> The double backticks don't have any special meaning here. Only the
> inside pair of backticks (i.e. `r `) makes sense to R Markdown (or
> precisely speaking, knitr). The outside pair will be left untouched
> after knitr evaluates the inline R expression `r params$data`. After
> evaluation, there won't be double backticks. This has nothing to do
> with the left curly double quote LaTeX. The result `hawaii` will be
> converted to hawaii if the output format is HTML (or
> \texttt{hawaii} if the output format is LaTeX), but the extra
> backticks in the original Rmd document is completely optional. It all
> depends on whether you want to place the result in a code tag or
> command.
>
> Regards,
> Yihui
> --
> https://yihui.name
>
> On Mon, Oct 14, 2019 at 8:42 AM Jeff Newmiller 
> wrote:
> >
> > No, you don't misunderstand. But you might want to be aware that
> targeting PDF output uses LaTeX which regards double backticks as a method
> of encoding a left curly double quote mark, so this error could derive from
> an incomplete adaptation of the Rmd file for HTML output even though such a
> symbol doesn't make sense there in PDF either.
> >
> > On October 14, 2019 4:35:26 AM PDT, Ashim Kapoor 
> wrote:
> > >Dear All,
> > >
> > >I am reading this file :-
> > >
> > >https://rmarkdown.rstudio.com/lesson-6.html
> > >
> > >My query is : In line 14 of the above file, ie.
> > >
> > >The [marmap](https://cran.r-project.org/web/packages/marmap/index.html)
> > >package provides tools and data for visualizing the ocean floor. Here
> > >is an
> > >example contour plot of marmap's ``r params$data`` dataset.
> > >
> > >Should not the phrase rparams$data be within SINGLE backticks ?
> > >
> > >Do I misunderstand?
> > >
> > >Thank you,
> > >Ashim
> > >
> > >   [[alternative HTML version deleted]]
> > >
> > >__
> > >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > >https://stat.ethz.ch/mailman/listinfo/r-help
> > >PLEASE do read the posting guide
> > >http://www.R-project.org/posting-guide.html
> > >and provide commented, minimal, self-contained, reproducible code.
> >
> > --
> > Sent from my phone. Please excuse my brevity.
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Double / single backticks for inline R code in an Rmarkdown file

[Ashim Kapoor]

Dear Jeff,

Many thanks for your reply.

Best,
Ashim

On Mon, Oct 14, 2019 at 7:07 PM Jeff Newmiller 
wrote:

> No, you don't misunderstand. But you might want to be aware that targeting
> PDF output uses LaTeX which regards double backticks as a method of
> encoding a left curly double quote mark, so this error could derive from an
> incomplete adaptation of the Rmd file for HTML output even though such a
> symbol doesn't make sense there in PDF either.
>
> On October 14, 2019 4:35:26 AM PDT, Ashim Kapoor 
> wrote:
> >Dear All,
> >
> >I am reading this file :-
> >
> >https://rmarkdown.rstudio.com/lesson-6.html
> >
> >My query is : In line 14 of the above file, ie.
> >
> >The [marmap](https://cran.r-project.org/web/packages/marmap/index.html)
> >package provides tools and data for visualizing the ocean floor. Here
> >is an
> >example contour plot of marmap's ``r params$data`` dataset.
> >
> >Should not the phrase rparams$data be within SINGLE backticks ?
> >
> >Do I misunderstand?
> >
> >Thank you,
> >Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> >__
> >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> --
> Sent from my phone. Please excuse my brevity.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Double / single backticks for inline R code in an Rmarkdown file

[Ashim Kapoor]

Dear All,

I am reading this file :-

https://rmarkdown.rstudio.com/lesson-6.html

My query is : In line 14 of the above file, ie.

The [marmap](https://cran.r-project.org/web/packages/marmap/index.html)
package provides tools and data for visualizing the ocean floor. Here is an
example contour plot of marmap's ``r params$data`` dataset.

Should not the phrase rparams$data be within SINGLE backticks ?

Do I misunderstand?

Thank you,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Correct way to use a lemmatizer in package tm

[Ashim Kapoor]

Dear All,

I want to do lemmatization using the tm package and textstem package.

The following is how I am doing it currently :-

library("tm")
library("wordcloud")
library("RColorBrewer")

filePath = < Path to any text file >
text <- readLines(filePath)
docs <- Corpus(VectorSource(text))

# Convert the text to lower case
docs <- tm_map(docs,content_transformer(tolower))

# Remove numbers
docs <- tm_map(docs, removeNumbers)
# Remove english common stopwords
docs <- tm_map(docs, removeWords, stopwords("english"))
# Remove punctuations
docs <- tm_map(docs, removePunctuation)
# Eliminate extra white spaces
docs <- tm_map(docs, stripWhitespace)
# Text Lemmatization
library(textstem)
docs <- tm_map(docs, content_transformer(lemmatize_words))

My query : Is the above line the correct way to do lemmatization ? Can
someone please confirm?

For the sake of giving a complete example I am giving the following code as
well.

dtm <- TermDocumentMatrix(docs)
m <- as.matrix(dtm)
v <- sort(rowSums(m),decreasing=TRUE)
d <- data.frame(word = names(v),freq=v)
head(d, 10)
set.seed(1234)
wordcloud(words = d$word, freq = d$freq, min.freq = 1,
  max.words=200, random.order=FALSE, rot.per=0.35,
  colors=brewer.pal(8, "Dark2"))
Thank you,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Package sf, unit for dist in st_buffer

[Ashim Kapoor]

Dear All,

map3 <- st_read("My Shapefile") %>% st_as_sf %>% st_transform(crs = 7755)
# Please see any shapefile here :
https://github.com/datameet/maps/tree/master/Districts

map_buffer <- st_buffer(mapHR, dist = -1000)
# CRS 7755 is for India, please see here, https://epsg.io/7755

My query is : The unit for distance is meters ?
On this page : https://epsg.io/7755,it says unit is metre.

On this page :
https://www.rdocumentation.org/packages/sf/versions/0.7-4/topics/geos_unary
it says :
dist numeric; buffer distance for all, or for each of the elements in x; in
case dist is a units object, it should be convertible to arc_degree if x
has geographic coordinates, and to st_crs(x)$units otherwise

I just want to be sure that, I am using a distance of  1 kilo metre ( 1000
metres) . Can someone please confirm ?

Thank you,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Error in installing pbmcapply / WhatIf

[Ashim Kapoor]

Dear Ivan,

OK and thank you for your help.

Best Regards,
Ashim

On Thu, Jun 27, 2019 at 3:34 PM Ivan Krylov  wrote:

> On Thu, 27 Jun 2019 12:48:50 +0530
> Ashim Kapoor  wrote:
>
> > R version 3.3.3 (2017-03-06)
>
> > process.c:10:23: error: unknown type name ‘DllInfo’
>
> DllInfo is defined in R_ext/Rdynload.h [1].
>
> process.c only includes Rinternals.h [2], which has been including
> R_ext/Rdynload.h since September 2016 [3]. Unfortunately, R 3.3.3 from
> Debian Stretch does not contain this change [4].
>
> On the one hand, a cursory look at latest "Writing R extensions" doesn't
> seem to indicate that Rinternals.h is guaranteed to include
> R_ext/Rdynload.h, so this may turn out to be a bug in pbmcapply (easily
> fixable by adding an #include). But that discussion is better suited
> for R-pkg-devel.
>
> On the other hand, there is an excellent Debian repo [5] containing the
> latest version of R which should be able to build pbmcapply without a
> problem.
>
> --
> Best regards,
> Ivan
>
> [1]
>
> https://github.com/wch/r-source/blob/22441d30384507d9d358a1e1c13f311d7a1b3627/src/include/R_ext/Rdynload.h#L70
>
> [2]
>
> https://github.com/kvnkuang/pbmcapply/blob/ef95ca6a23e003b45ec97b2aa55b7cb01d21363b/src/process.c#L3
>
> [3]
>
> https://github.com/wch/r-source/blame/22441d30384507d9d358a1e1c13f311d7a1b3627/src/include/Rinternals.h#L51
>
> [4]
> https://github.com/wch/r-source/blob/tags/R-3-3-3/src/include/Rinternals.h
>
> [5] https://cloud.r-project.org/bin/linux/debian/
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Error in installing pbmcapply / WhatIf

[Ashim Kapoor]

Dear All,

I am not able to install pbmcapply which I need for package WhatIf.

Here are my sessionInfo and the error message :-

> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 8 (jessie)

locale:
 [1] LC_CTYPE=en_IN   LC_NUMERIC=C LC_TIME=en_IN
 [4] LC_COLLATE=en_IN LC_MONETARY=en_INLC_MESSAGES=en_IN
 [7] LC_PAPER=en_IN   LC_NAME=CLC_ADDRESS=C
[10] LC_TELEPHONE=C   LC_MEASUREMENT=en_IN LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] tools_3.3.3 tcltk_3.3.3
> install.packages("pbmcapply")
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/pbmcapply_1.4.1.tar.gz'
Content type 'unknown' length 8224 bytes
==
downloaded 8224 bytes

* installing *source* package ‘pbmcapply’ ...
** package ‘pbmcapply’ successfully unpacked and MD5 sums checked
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG  -fpic  -g -O2
-fdebug-prefix-map=/build/r-base-3.3.3=. -fstack-protector-strong -Wformat
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c process.c -o
process.o
process.c:10:23: error: unknown type name ‘DllInfo’
 void R_init_pbmcapply(DllInfo* info) {
   ^
/usr/lib/R/etc/Makeconf:132: recipe for target 'process.o' failed
make: *** [process.o] Error 1
ERROR: compilation failed for package ‘pbmcapply’
* removing ‘/usr/local/lib/R/site-library/pbmcapply’

The downloaded source packages are in
‘/tmp/RtmpgOn4Vx/downloaded_packages’
Warning message:
In install.packages("pbmcapply") :
  installation of package ‘pbmcapply’ had non-zero exit status
>


Thank you,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Intuition behind time varying parameters regression model.

[Ashim Kapoor]

Dear All,

I have a query which I have posted on Stack Exchange:-

https://stats.stackexchange.com/questions/404353/do-all-parameters-have-to-have-the-same-nature-in-a-structural-change-test

Can some one here please have a look at this ? I posted but I did not
receive any reply.

Many thanks,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] magick package says format is PNG when we load an SVG.

[Ashim Kapoor]

Dear Peter,

So when I convert from PDF to SVG it will first convert to raster and then
back to vector ? Is'nt there a way of preventing this ?

Many thanks,
Ashim

On Mon, Feb 25, 2019 at 3:10 PM peter dalgaard  wrote:

> The image format in magick is raster based, not vector based, no? So I'd
> expect that image_read_svg() converts the file on the fly.
>
> -pd
>
> > On 25 Feb 2019, at 08:27 , Ashim Kapoor  wrote:
> >
> > Dear All,
> >
> > Here is a MWE :
> >
> > library(magick)
> > tiger <- image_read_svg('http://jeroen.github.io/images/tiger.svg',
> width = 400)
> > print(tiger)
> >
> > ##   format width height colorspace matte filesize density
> > ## 1PNG   400400   sRGB  TRUE0   72x72
> >
> > Why does the format say PNG ? Should it not say SVG? We just read an SVG
> file.
> >
> > Can someone please explain?
> >
> > Best Regards,
> >
> > Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
>
>
>
>
>
>
>
>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] magick package says format is PNG when we load an SVG.

[Ashim Kapoor]

Dear All,

Here is a MWE :

library(magick)
tiger <- image_read_svg('http://jeroen.github.io/images/tiger.svg', width = 400)
print(tiger)

##   format width height colorspace matte filesize density
## 1PNG   400400   sRGB  TRUE0   72x72

Why does the format say PNG ? Should it not say SVG? We just read an SVG file.

Can someone please explain?

Best Regards,

Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Unable to compute Confidence Intervals from output from MARSS package

[Ashim Kapoor]

Dear Bill,

Appreciate all your effort. I hope some one here can respond to this query.

Many thanks,
Ashim




On Thu, Jan 24, 2019 at 12:49 AM Bill Poling  wrote:

>
> Ashim.
>
> I see where I was mistaken, using MARSSparamsCIs(fit) <--Somehow I got an
> s in between param & Cis.
>
> I now get new error similarly as you, my apologies.
>
> final <- MARSSparamCIs(fit)
> Error in dpari[time.varying] <- dparmat(MLEobj, time.varying, t = t) :
>  replacement has length zero
>
> WHP
>
> From: R-help  On Behalf Of Ashim Kapoor
> Sent: Wednesday, January 23, 2019 1:38 AM
> To: r-help@r-project.org
> Subject: [R] Unable to compute Confidence Intervals from output from MARSS
> package
>
> Dear All,
>
> I am trying to use this package --->
> https://cran.r-project.org/web/packages/MARSS/index.html
> I am reading this book which shows some examples based on the above package
> ---> https://nwfsc-timeseries.github.io/atsa-labs/
>
> In a few words, the incantation MARSS(...) estimates the parameters and
> MARSSparamCIs should return the confidence intervals. My problem is that
> MARSSparamCIs returns this error :-
>
> > MARSSparamCIs(fit)
> Error in MARSSharveyobsFI(MLEobj) : replacement has length zero
> >
>
> ### This page and the next page, explains the jargon used :
>
> https://nwfsc-timeseries.github.io/atsa-labs/sec-dlm-example-of-a-univariate-dlm.html
>
> ### Here is a MWE to recreate the error :-
>
> # This is a Time Varying Parameters regression
> # Here a is fixed and b is doing a RW.
>
> x1 <- rnorm(1000)
> b <- cumsum(rnorm(1000, sd = sqrt(.6)))
> y <- 1 + b*x1 + rnorm(1000)
>
> Z = array(NA,c(1,2,1000))
> Z[1,1,] = rep(1,1000)
> Z[1,2,] = x1
>
> mod2.list = list ( B = matrix(c(1,0,0,1),nrow = 2 , byrow = T), U =
> matrix(0,2,1),
> Q = matrix(list(0,0,0,"s2b"),2,2), Z= Z, A = matrix(0), R = matrix("r"))
>
> fit = MARSS(as.vector(y), model = mod2.list,inits =
> list(x0=matrix(0,nrow=2,ncol=1)))
>
> MARSSparamsCIs(fit)
>
> # The above will give this error :
> Error in MARSSharveyobsFI(MLEobj) : replacement has length zero
>
> Best Regards,
> Ashim
>
> [[alternative HTML version deleted]]
>
> __
> mailto:R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
> Confidentiality Notice This message is sent from Zelis. This transmission
> may contain information which is privileged and confidential and is
> intended for the personal and confidential use of the named recipient only.
> Such information may be protected by applicable State and Federal laws from
> this disclosure or unauthorized use. If the reader of this message is not
> the intended recipient, or the employee or agent responsible for delivering
> the message to the intended recipient, you are hereby notified that any
> disclosure, review, discussion, copying, or taking any action in reliance
> on the contents of this transmission is strictly prohibited. If you have
> received this transmission in error, please contact the sender immediately.
> Zelis, 2018.
>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Unable to compute Confidence Intervals from output from MARSS package

[Ashim Kapoor]

Dear All,

I am trying to use this package --->
https://cran.r-project.org/web/packages/MARSS/index.html
I am reading this book which shows some examples based on the above package
---> https://nwfsc-timeseries.github.io/atsa-labs/

In a few words, the incantation MARSS(...) estimates the parameters and
MARSSparamCIs should return the confidence intervals. My problem is that
MARSSparamCIs returns this  error :-

> MARSSparamCIs(fit)
Error in MARSSharveyobsFI(MLEobj) : replacement has length zero
>

### This page and the next page, explains the jargon used :
https://nwfsc-timeseries.github.io/atsa-labs/sec-dlm-example-of-a-univariate-dlm.html

### Here is a MWE to recreate the error :-

# This is a Time Varying Parameters regression
# Here a is fixed and b is doing a RW.

x1 <- rnorm(1000)
b <- cumsum(rnorm(1000, sd = sqrt(.6)))
y <- 1 + b*x1 + rnorm(1000)

Z = array(NA,c(1,2,1000))
Z[1,1,]  = rep(1,1000)
Z[1,2,] = x1

mod2.list = list ( B = matrix(c(1,0,0,1),nrow = 2 , byrow = T), U  =
matrix(0,2,1),
Q = matrix(list(0,0,0,"s2b"),2,2), Z= Z, A = matrix(0), R = matrix("r"))

fit = MARSS(as.vector(y), model = mod2.list,inits =
list(x0=matrix(0,nrow=2,ncol=1)))

MARSSparamsCIs(fit)

# The above will give this error :
Error in MARSSharveyobsFI(MLEobj) : replacement has length zero

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Reproducing the results from package dlm

[Ashim Kapoor]

Dear all,

I have a query with regards to the package dlm. My query is : will dlm
return the same results if I give it the same data set ?

Here is a MWE ( created from :
https://sites.ualberta.ca/~sfossati/e509/files/other/dlm_ex.R )

library(dlm)

# simulate AR(1) process
# phi = .8, sig2 = .25
nobs <- 250
yt <- arima.sim(n=nobs,list(ar=.8,ma=0),sd=.5)

# estimate AR(1) for comparison
model10 <- Arima(yt,order=c(1,0,0),method="ML",include.mean=FALSE)
model10

# set parameter restrictions
parm_rest <- function(parm){
 return( c(exp(parm[1])/(1+exp(parm[1])),exp(parm[2])) )
}

# set up SS model
ssm1 <- function(parm){
parm <- parm_rest(parm)
return( dlm(FF=1,V=0,GG=parm[1],W=parm[2],
m0=0,C0=solve(1-parm[1]^2)*parm[2]) )
}
# estimate parameters
fit1 <- dlmMLE(y=yt,parm=c(0,1),build=ssm1,hessian=T)

# get estimates
coef <- parm_rest(fit1$par)
# get standard errors using delta method
dg1 <- exp(fit1$par[1])/(1+exp(fit1$par[1]))^2
dg2 <- exp(fit1$par[2])
dg <- diag(c(dg1,dg2))
var <- dg%*%solve(fit1$hessian)%*%dg
# print results
coef; sqrt(diag(var))

# Here are 2 runs of the latter part of the code: -
> fit1 <- dlmMLE(y=yt,parm=c(0,1),build=ssm1,hessian=T)
>
> # get estimates
> coef <- parm_rest(fit1$par)
> # get standard errors using delta method
> dg1 <- exp(fit1$par[1])/(1+exp(fit1$par[1]))^2
> dg2 <- exp(fit1$par[2])
> dg <- diag(c(dg1,dg2))
> var <- dg%*%solve(fit1$hessian)%*%dg
> # print results
> coef; sqrt(diag(var))
[1] 0.8442885 0.2513462
[1] 0.03361245 0.02248196
> fit1 <- dlmMLE(y=yt,parm=c(0,1),build=ssm1,hessian=T)
>
> # get estimates
> coef <- parm_rest(fit1$par)
> # get standard errors using delta method
> dg1 <- exp(fit1$par[1])/(1+exp(fit1$par[1]))^2
> dg2 <- exp(fit1$par[2])
> dg <- diag(c(dg1,dg2))
> var <- dg%*%solve(fit1$hessian)%*%dg
> # print results
> coef; sqrt(diag(var))
[1] 0.8442885 0.2513462
[1] 0.03361245 0.02248196
>

Seems to me that even without set.seed, dlm reports the SAME results. Is
that true? I have written a big program where I have slightly different
results from dlm and I wonder if I have made a mistake. I have created a
MWE above to verify my doubt.

Do we need a set.seed for reproducing results from dlm ? Please clarify.

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Help needed regarding dlm model on stats.stackexchange

[Ashim Kapoor]

Dear all,

I have created a time varying parameters regression. When I do that I have
a parameter which is AR1. I am not able to recover this parameter.

My query is posted here :

https://stats.stackexchange.com/questions/377295/unable-to-recover-time-varying-ar1-parameter-from-state-space-model

I did not receive a reply so I am posting here.

Many thanks,
Ashim.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Output of arima

[Ashim Kapoor]

Dear Eric,

Many thanks for your reply.

Best Regards,
Ashim

On Wed, Nov 14, 2018 at 4:05 PM Eric Berger  wrote:

> Hi Ashim,
> Per the help page for arima(), it fits an ARIMA model to the specified
> time series - but the caller has to specify the order - i.e. (p,d,q) - of
> the model.
> The default order is (0,0,0) (per the help page). Hence your two calls are
> different. The first call is equivalent to order=c(0,0,0) and the second
> specifies order=c(1,0,0).
> In the first, since there is no auto-regression, all the variance is
> "assigned" to the innovations, hence sigma^2 = 5.233.
> The second case you understand.
> A clue that this was happening is that the first call only returns a
> single coefficient (where is the autoregressive coefficient? - not there
> because you didn't ask for it).
> The second call returns two coefficients, as requested/expected.
>
> HTH,
> Eric
>
>
> On Wed, Nov 14, 2018 at 12:08 PM Ashim Kapoor 
> wrote:
>
>> Dear Eric and William,
>>
>> Why do the 1st and 2nd incantation of arima return sigma^2 as 5.233 vs
>> .?
>> The help for arima says  --->  sigma2: the MLE of the innovations
>> variance.
>> By that account the 1st result is incorrect. I am a little confused.
>>
>> set.seed(123)
>> b <- arima.sim(list(order = c(1,0,0),ar= .9),n=100,sd=1)
>>
>> # Variance of the innovations, e_t = 1
>>
>> # Variance of b = Var(e_t)/(1-Phi^2) = 1 / (1-.81) = 5.263158
>>
>> arima(b)
>>
>> > arima(b)
>>
>> Call:
>> arima(x = b)
>>
>> Coefficients:
>>   intercept
>> -0.0051
>> s.e. 0.0023
>>
>> sigma^2 estimated as 5.233:  log likelihood = -2246450,  aic = 4492903
>> >
>>
>>
>> arima(b,order= c(1,0,0))
>>
>> Call:
>> arima(x = b, order = c(1, 0, 0))
>>
>> Coefficients:
>>  ar1  intercept
>>   0.8994-0.0051
>> s.e.  0.0004 0.0099
>>
>> sigma^2 estimated as 0.:  log likelihood = -1418870,  aic = 2837747
>> >
>>
>> On Tue, Nov 13, 2018 at 11:07 PM William Dunlap 
>> wrote:
>>
>> > Try supplying the order argument to arima.  It looks like the default is
>> > to estimate only the mean.
>> >
>> > > arima(b, order=c(1,0,0))
>> >
>> > Call:
>> > arima(x = b, order = c(1, 0, 0))
>> >
>> > Coefficients:
>> >  ar1  intercept
>> >   0.8871 0.2369
>> > s.e.  0.0145 0.2783
>> >
>> > sigma^2 estimated as 1.002:  log likelihood = -1420.82,  aic = 2847.63
>> >
>> >
>> > Bill Dunlap
>> > TIBCO Software
>> > wdunlap tibco.com
>> >
>> > On Tue, Nov 13, 2018 at 4:02 AM, Ashim Kapoor 
>> > wrote:
>> >
>> >> Dear All,
>> >>
>> >> Here is a reprex:
>> >>
>> >> set.seed(123)
>> >> b <- arima.sim(list(order = c(1,0,0),ar= .9),n=1000,sd=1)
>> >> arima(b)
>> >>
>> >> Call:
>> >> arima(x = b)
>> >>
>> >> Coefficients:
>> >>   intercept
>> >>  0.2250
>> >> s.e. 0.0688
>> >>
>> >> sigma^2 estimated as 4.735:  log likelihood = -2196.4,  aic = 4396.81
>> >> >
>> >>
>> >> Should sigma^2 not be equal to 1 ? Where do I misunderstand ?
>> >>
>> >> Many thanks,
>> >> Ashim
>> >>
>> >> [[alternative HTML version deleted]]
>> >>
>> >> __
>> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> >> https://stat.ethz.ch/mailman/listinfo/r-help
>> >> PLEASE do read the posting guide
>> >> http://www.R-project.org/posting-guide.html
>> >> and provide commented, minimal, self-contained, reproducible code.
>> >>
>> >
>> >
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Output of arima

[Ashim Kapoor]

Dear Eric and William,

Why do the 1st and 2nd incantation of arima return sigma^2 as 5.233 vs
.?
The help for arima says  --->  sigma2: the MLE of the innovations variance.
By that account the 1st result is incorrect. I am a little confused.

set.seed(123)
b <- arima.sim(list(order = c(1,0,0),ar= .9),n=100,sd=1)

# Variance of the innovations, e_t = 1

# Variance of b = Var(e_t)/(1-Phi^2) = 1 / (1-.81) = 5.263158

arima(b)

> arima(b)

Call:
arima(x = b)

Coefficients:
  intercept
-0.0051
s.e. 0.0023

sigma^2 estimated as 5.233:  log likelihood = -2246450,  aic = 4492903
>


arima(b,order= c(1,0,0))

Call:
arima(x = b, order = c(1, 0, 0))

Coefficients:
 ar1  intercept
  0.8994-0.0051
s.e.  0.0004 0.0099

sigma^2 estimated as 0.:  log likelihood = -1418870,  aic = 2837747
>

On Tue, Nov 13, 2018 at 11:07 PM William Dunlap  wrote:

> Try supplying the order argument to arima.  It looks like the default is
> to estimate only the mean.
>
> > arima(b, order=c(1,0,0))
>
> Call:
> arima(x = b, order = c(1, 0, 0))
>
> Coefficients:
>  ar1  intercept
>   0.8871 0.2369
> s.e.  0.0145 0.2783
>
> sigma^2 estimated as 1.002:  log likelihood = -1420.82,  aic = 2847.63
>
>
> Bill Dunlap
> TIBCO Software
> wdunlap tibco.com
>
> On Tue, Nov 13, 2018 at 4:02 AM, Ashim Kapoor 
> wrote:
>
>> Dear All,
>>
>> Here is a reprex:
>>
>> set.seed(123)
>> b <- arima.sim(list(order = c(1,0,0),ar= .9),n=1000,sd=1)
>> arima(b)
>>
>> Call:
>> arima(x = b)
>>
>> Coefficients:
>>   intercept
>>  0.2250
>> s.e. 0.0688
>>
>> sigma^2 estimated as 4.735:  log likelihood = -2196.4,  aic = 4396.81
>> >
>>
>> Should sigma^2 not be equal to 1 ? Where do I misunderstand ?
>>
>> Many thanks,
>> Ashim
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Output of arima

[Ashim Kapoor]

Dear All,

Here is a reprex:

set.seed(123)
b <- arima.sim(list(order = c(1,0,0),ar= .9),n=1000,sd=1)
arima(b)

Call:
arima(x = b)

Coefficients:
  intercept
 0.2250
s.e. 0.0688

sigma^2 estimated as 4.735:  log likelihood = -2196.4,  aic = 4396.81
>

Should sigma^2 not be equal to 1 ? Where do I misunderstand ?

Many thanks,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Does R version 3.4.4 work on Ubuntu 18.04.1

[Ashim Kapoor]

Dear All,

Okay and thank you.

Best Regards,
Ashim

On Wed, Sep 19, 2018 at 3:32 AM Benoit Vaillant 
wrote:

> Hello,
>
> On Tue, Sep 18, 2018 at 09:08:18AM -0400, Ista Zahn wrote:
> > This is really the wrong place for this discussion. Please post ubuntu
> > specific questions to r-sig-debian.
>
> While i guess your asking to get on to r-sig-debian is a true start,
> the package that seems to be causing troubles is "in house".
>
> No matter the mailing-list the user should end up, the appropriate
> consideration should be to get back at who made that package.
>
> Cheers, :)
>
> --
> Benoît Vaillant
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Does R version 3.4.4 work on Ubuntu 18.04.1

[Ashim Kapoor]

Dear Rui,

I tried R 3.4.4 on Ubuntu 18.04.1. It runs FINE except for an inhouse
package created by us. That is why I asked.

Best,
Ashim

On Tue, Sep 18, 2018 at 4:12 PM Rui Barradas  wrote:

> Hello,
>
> R 3.4.0 was released 2017-04-21 09:14 and R 3.4.4 2018-03-15 09:04. This
> is before the release of Ubuntu 18.04 LTS so that version of Ubuntu was
> not supported by any sub-version of R 3.4.
>
> At least this is how I understand it. If you want to give R 3.4/Ubuntu
> 18.04 a try, you can download older versions of R from CRAN and install
> them.
>
> Do you have any practical reason for asking this?
>
> Rui Barradas
>
> Às 08:24 de 18/09/2018, Ashim Kapoor escreveu:
> > Dear Rui,
> >
> > I am a little confused.
> >
> > See this ---> : R 3.4 packages for Ubuntu on i386 and amd64 are
> > available for all stable Desktop releases of Ubuntu prior to Bionic
> > Beaver (18.04) until their official end of life date. However, only the
> > latest Long Term Support (LTS) release is fully supported. As of June
> > 11, 2018 the supported releases are Artful Aardvark (17.10), Xenial
> > Xerus (16.04; LTS), and Trusty Tahr (14.04; LTS).
> >
> > Bionic Beaver ( Ubuntu 18.04 ) is NOT in the list of supported releases
> > for R 3.4  What does this mean ? Can you please clarify ?
> >
> > Many thanks,
> > Ashim
> >
> > On Tue, Sep 18, 2018 at 11:49 AM Rui Barradas  > <mailto:ruipbarra...@sapo.pt>> wrote:
> >
> > Hello,
> >
> > I am not completely sure but I think I installed Ubuntu 18.04 LTS
> first
> > and R 3.5 days later, so yes, if I'm right it is possible to run R
> 3.4
> > on 18.04.
> >
> > (You ask whether we can *install* R 3.4 on Ubuntu 18.04.1, I'm
> > saying it
> > can be *run* on Ubuntu 18.04.1.)
> >
> > Hope this helps,
> >
> > Rui Barradas
> >
> >
> > Às 06:51 de 18/09/2018, Ashim Kapoor escreveu:
> >  > Dear All,
> >  >
> >  > I was reading this page --->
> >  > https://cran.r-project.org/bin/linux/ubuntu/README.html
> >  >
> >  > It says: R 3.4 packages for Ubuntu on i386 and amd64 are
> > available for all
> >  > stable Desktop releases of Ubuntu prior to Bionic Beaver (18.04)
> > until
> >  > their official end of life date.
> >  >
> >  > The page also shows how to install R 3.5 on Ubuntu 18.04.1
> >  >
> >  > My query is : Can we install R 3.4 on Ubuntu 18.04.1 ? Or can we
> only
> >  > install R 3.5 ?
> >  >
> >  > Many thanks,
> >  > Ashim
> >  >
> >  >   [[alternative HTML version deleted]]
> >  >
> >  > __
> >  > R-help@r-project.org <mailto:R-help@r-project.org> mailing list
> > -- To UNSUBSCRIBE and more, see
> >  > https://stat.ethz.ch/mailman/listinfo/r-help
> >  > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> >  > and provide commented, minimal, self-contained, reproducible code.
> >  >
> >
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Does R version 3.4.4 work on Ubuntu 18.04.1

[Ashim Kapoor]

Dear Rui,

I am a little confused.

See this ---> : R 3.4 packages for Ubuntu on i386 and amd64 are available
for all stable Desktop releases of Ubuntu prior to Bionic Beaver (18.04)
until their official end of life date. However, only the latest Long Term
Support (LTS) release is fully supported. As of June 11, 2018 the supported
releases are Artful Aardvark (17.10), Xenial Xerus (16.04; LTS), and Trusty
Tahr (14.04; LTS).

Bionic Beaver ( Ubuntu 18.04 ) is NOT in the list of supported releases for
R 3.4  What does this mean ? Can you please clarify ?

Many thanks,
Ashim

On Tue, Sep 18, 2018 at 11:49 AM Rui Barradas  wrote:

> Hello,
>
> I am not completely sure but I think I installed Ubuntu 18.04 LTS first
> and R 3.5 days later, so yes, if I'm right it is possible to run R 3.4
> on 18.04.
>
> (You ask whether we can *install* R 3.4 on Ubuntu 18.04.1, I'm saying it
> can be *run* on Ubuntu 18.04.1.)
>
> Hope this helps,
>
> Rui Barradas
>
>
> Às 06:51 de 18/09/2018, Ashim Kapoor escreveu:
> > Dear All,
> >
> > I was reading this page --->
> > https://cran.r-project.org/bin/linux/ubuntu/README.html
> >
> > It says: R 3.4 packages for Ubuntu on i386 and amd64 are available for
> all
> > stable Desktop releases of Ubuntu prior to Bionic Beaver (18.04) until
> > their official end of life date.
> >
> > The page also shows how to install R 3.5 on Ubuntu 18.04.1
> >
> > My query is : Can we install R 3.4 on Ubuntu 18.04.1 ? Or can we only
> > install R 3.5 ?
> >
> > Many thanks,
> > Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Does R version 3.4.4 work on Ubuntu 18.04.1

[Ashim Kapoor]

Dear All,

I was reading this page --->
https://cran.r-project.org/bin/linux/ubuntu/README.html

It says: R 3.4 packages for Ubuntu on i386 and amd64 are available for all
stable Desktop releases of Ubuntu prior to Bionic Beaver (18.04) until
their official end of life date.

The page also shows how to install R 3.5 on Ubuntu 18.04.1

My query is : Can we install R 3.4 on Ubuntu 18.04.1 ? Or can we only
install R 3.5 ?

Many thanks,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Unclear about the output from summary of ca.jo from package urca

[Ashim Kapoor]

Dear Bert,

I have read some of the references. I do understand what the 2 matrices(
the cointegrating relationships and the alpha / loading matrix which gives
the speed of the mean reversion)  are. What I do not understand is the
format of the output of the package. My main query is that why do we have
.l2 in the cointegrating relationships. They are contemporaneous
relationships , they should not have .l2 in the end. That's my query.

Many thanks,
Ashim

On Thu, Aug 23, 2018 at 7:52 PM Bert Gunter  wrote:

> This is about statistics , not R programming, and so is off topic here.
> Your first port of call for this sort of thing should be the package docs,
> **including any references** . There are references given. Have you studied
> them??
>
> Cheers,
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along and
> sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Thu, Aug 23, 2018 at 2:12 AM Ashim Kapoor 
> wrote:
>
>> Dear All,
>>
>> I am not sure about the summary of the function ca.jo. I have posted my
>> query here :-
>>
>>
>> https://stats.stackexchange.com/questions/363188/interpreting-the-names-used-in-the-output-of-johansen-test-in-package-urca-in-r
>>
>> I did not receive any reply so I am posting my query here.
>>
>> Many thanks and best regards,
>> Ashim
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Unclear about the output from summary of ca.jo from package urca

[Ashim Kapoor]

Dear All,

I am not sure about the summary of the function ca.jo. I have posted my
query here :-

https://stats.stackexchange.com/questions/363188/interpreting-the-names-used-in-the-output-of-johansen-test-in-package-urca-in-r

I did not receive any reply so I am posting my query here.

Many thanks and best regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Last page on a multi page, faceted graph created using ggforce, loses its layout under a certain condition

[Ashim Kapoor]

Dear All,

I posted this query on stack overflow but I received no replies so I am
posting it here so that someone here can take a look at this.

My query is that suppose the variable I am faceting by has 40 categories
and I want 3 categories per page( 1 row x 3 columns),then the last category
fills the entire second page. I want the last category to fill only the
left - most position in the grid and the remaining 2 places to be empty on
the last page.

Here is the link to the stackoverflow query which has a reprex in it as
well:

https://stackoverflow.com/questions/50604211/facetting-a-graph-across-multiple-pages-and-keeping-the-same-layout-on-all-pag

Many thanks and Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Understanding which

[Ashim Kapoor]

Dear Jeff and Eric,

Okay and many thanks.
Best Regards,
Ashim

On Wed, Apr 18, 2018 at 4:56 PM, Jeff Newmiller <jdnew...@dcn.davis.ca.us>
wrote:

> Look at
>
> which(x>100)
>
> This is a zero-length vector. The negative of nothing is nothing, not a
> list of all possible index values.
>
> Do you want
>
> x[ !( x > 100 ) ]
>
> ?
>
> On April 18, 2018 6:13:30 AM CDT, Ashim Kapoor <ashimkap...@gmail.com>
> wrote:
> >Dear All,
> >
> >Here is a reprex:
> >
> >> x<- 1:100
> >> x[-which(x>100)]
> >integer(0)
> >
> >In words, I am finding out which indices correspond to values in x
> >which
> >are  greater than  100 ( there are no such items ) . Then I remove
> >those
> >indices. I should get back the x that I started with since there are no
> >items in x which are bigger than 100 . Instead, it is returning an
> >empty
> >vector.
> >
> >Why is this ? What am I misunderstanding?
> >
> >Best Regards,
> >Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> >__
> >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> --
> Sent from my phone. Please excuse my brevity.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Understanding which

[Ashim Kapoor]

Dear All,

Here is a reprex:

> x<- 1:100
> x[-which(x>100)]
integer(0)

In words, I am finding out which indices correspond to values in x which
are  greater than  100 ( there are no such items ) . Then I remove those
indices. I should get back the x that I started with since there are no
items in x which are bigger than 100 . Instead, it is returning an empty
vector.

Why is this ? What am I misunderstanding?

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] grepAMModelLib in package AMML not working as expected

[Ashim Kapoor]

Dear All,

Here is a minimal working example :

library(AMModels)

mymodels <- amModelLib(description = "Simple AM Model library")

dataset1 <- 1:100
dataset2 <- seq(2,200,2)

model1 <- lm(dataset2~1)
model2 <- lm(dataset2~ dataset1)

m1 <- amModel(model = model1,comment  = "Initial model.")
m2 <- amModel(model = model2,comment  = "Final model.")

mymodels <- insertAMModelLib(model = list(m1=m1,m2=m2),amml = mymodels)


> grepAMModelLib(pattern= "m1",amml = mymodels,search = "model")

Description:
[1] Simple AM Model library

Info:
** no informative metadata **


Models:
  name class package
1   m1lm  NA

Data:

 --- There are no datasets ---


Here I am searching for all models which have the letter "1".

> grepAMModelLib(pattern= "1",amml = mymodels,search = "model")

Description:
[1] Simple AM Model library

Info:
** no informative metadata **


Models:
  name class package
1   m1lm  NA
2   m2lm  NA

Data:

 --- There are no datasets ---
>

The above should return only model m1. Why does the result of grepping
return both m1 and m2 ? Where do I misunderstand ?

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] SpreadLevelPlot for more than one factor

[Ashim Kapoor]

Dear Sir,

Many thanks and Best Regards,
Ashim.

On Sun, Jan 14, 2018 at 9:05 PM, Fox, John <j...@mcmaster.ca> wrote:

> Dear Ashim,
>
> I’ll address your questions briefly but they’re really not appropriate for
> this list, which is for questions about using R, not general statistical
> questions.
>
> (1) The relevant distribution is within cells of the wool x tension
> cross-classification because it’s the deviations from the cell means that
> are supposed to be normally distributed with equal variance. In the
> warpbreaks data there are only 9 cases per cell. If you examine all of
> these deviations simultaneously, that’s equivalent to examining the
> residuals from the two-way ANOVA model fit to the data.
>
> (2) Yes, (d) and (e) visualize simple effects, and (a) and (b) visualize
> main effects, the latter only because the data are balanced.
>
> Best,
>  John
>
> -
> John Fox, Professor Emeritus
> McMaster University
> Hamilton, Ontario, Canada
> Web: http://socserv.mcmaster.ca/jfox/
>
>
>
>
> On 2018-01-09, 10:18 AM, "Ashim Kapoor" <ashimkap...@gmail.com> wrote:
>
> >Dear Sir,
> >
> >
> >Many thanks for your reply.
> >
> >
> >I have a query.
> >
> >
> >
> >I have a whole set of distributions which should be made normal /
> >homoscedastic. Take for instance the warpbreaks data set.
> >
> >
> >
> >We have the following boxplots for the warpbreaks dataset:
> >
> >
> >a. boxplot(breaks ~ wool)
> >
> >b. boxplot(breaks ~ tension)
> >
> >c. boxplot(breaks ~ interaction(wool,tension))
> >d. boxplot(breaks ~ wool @ each level of tension)
> >e. boxplot(breaks ~ tension @ each level of wool)
> >
> >
> >Now should we not be making a-e normal and homoscedastic? Should we not
> >make a giant collection of boxplots from a-e and use the SpreadLevelPlot
> >on this entire collection?
> >
> >
> >A second query : (d) and (e) are the distribution of the simple effects
> >of factor wool and tension @ each level of the other. Is that correct?
> >Are (a) and (b) the distribution of the main effect of wool and tension?
> >Please confirm.
> >
> >
> >
> >Best Regards,
> >Ashim
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >On Sun, Jan 7, 2018 at 8:05 PM, Fox, John
> ><j...@mcmaster.ca> wrote:
> >
> >Dear Ashim,
> >
> >Try spreadLevelPlot(breaks ~ interaction(tension, wool), data=warpbreaks)
> >.
> >
> >I hope this helps,
> > John
> >
> >-
> >John Fox, Professor Emeritus
> >McMaster University
> >Hamilton, Ontario, Canada
> >Web:
> >socialsciences.mcmaster.ca/jfox/ <http://socialsciences.mcmaster.ca/jfox/
> >
> >
> >
> >
> >> -Original Message-
> >> From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Ashim
> >> Kapoor
> >> Sent: Sunday, January 7, 2018 12:08 AM
> >> To: r-help@r-project.org
> >> Subject: [R] SpreadLevelPlot for more than one factor
> >>
> >> Dear All,
> >>
> >> I want a transformation which will make the spread of the response at
> >>all
> >> combinations of  2 factors the same.
> >>
> >> See for example :
> >>
> >> boxplot(breaks ~ tension * wool, warpbreaks)
> >>
> >> The closest I  can do is :
> >>
> >> spreadLevelPlot(breaks ~tension , warpbreaks) spreadLevelPlot(breaks ~
> >>wool ,
> >> warpbreaks)
> >>
> >> I want to do :
> >>
> >> spreadLevelPlot(breaks ~tension * wool, warpbreaks)
> >>
> >> But I get :
> >>
> >> > spreadLevelPlot(breaks ~tension * wool , warpbreaks)
> >> Error in spreadLevelPlot.formula(breaks ~ tension * wool, warpbreaks) :
> >>   right-hand side of model has more than one variable
> >>
> >> What is the corresponding appropriate function for 2 factors ?
> >>
> >> Many thanks,
> >> Ashim
> >>
> >>   [[alternative HTML version deleted]]
> >>
> >> __
> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >>
> >https://stat.ethz.ch/mailman/listinfo/r-help
> ><https://stat.ethz.ch/mailman/listinfo/r-help>
> >> PLEASE do read the posting guide
> >http://www.R-project.org/posting- <http://www.R-project.org/posting->
> >> guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >
> >
> >
> >
> >
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] SpreadLevelPlot for more than one factor

[Ashim Kapoor]

Dear Sir,

Many thanks for your reply.

I have a query.

I have a whole set of distributions which should be made normal /
homoscedastic. Take for instance the warpbreaks data set.

We have the following boxplots for the warpbreaks dataset:

a. boxplot(breaks ~ wool)
b. boxplot(breaks ~ tension)
c. boxplot(breaks ~ interaction(wool,tension))
d. boxplot(breaks ~ wool @ each level of tension)
e. boxplot(breaks ~ tension @ each level of wool)

Now should we not be making a-e normal and homoscedastic? Should we not
make a giant collection of boxplots from a-e and use the SpreadLevelPlot on
this entire collection?

A second query : (d) and (e) are the distribution of the simple effects of
factor wool and tension @ each level of the other. Is that correct?  Are
(a) and (b) the distribution of the main effect of wool and tension? Please
confirm.

Best Regards,
Ashim


On Sun, Jan 7, 2018 at 8:05 PM, Fox, John <j...@mcmaster.ca> wrote:

> Dear Ashim,
>
> Try spreadLevelPlot(breaks ~ interaction(tension, wool), data=warpbreaks) .
>
> I hope this helps,
>  John
>
> -
> John Fox, Professor Emeritus
> McMaster University
> Hamilton, Ontario, Canada
> Web: socialsciences.mcmaster.ca/jfox/
>
>
>
> > -Original Message-
> > From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Ashim
> > Kapoor
> > Sent: Sunday, January 7, 2018 12:08 AM
> > To: r-help@r-project.org
> > Subject: [R] SpreadLevelPlot for more than one factor
> >
> > Dear All,
> >
> > I want a transformation which will make the spread of the response at
> all
> > combinations of  2 factors the same.
> >
> > See for example :
> >
> > boxplot(breaks ~ tension * wool, warpbreaks)
> >
> > The closest I  can do is :
> >
> > spreadLevelPlot(breaks ~tension , warpbreaks) spreadLevelPlot(breaks ~
> wool ,
> > warpbreaks)
> >
> > I want to do :
> >
> > spreadLevelPlot(breaks ~tension * wool, warpbreaks)
> >
> > But I get :
> >
> > > spreadLevelPlot(breaks ~tension * wool , warpbreaks)
> > Error in spreadLevelPlot.formula(breaks ~ tension * wool, warpbreaks) :
> >   right-hand side of model has more than one variable
> >
> > What is the corresponding appropriate function for 2 factors ?
> >
> > Many thanks,
> > Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-
> > guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] SpreadLevelPlot for more than one factor

[Ashim Kapoor]

Dear All,

I did this :

> v = wool:tension
> spreadLevelPlot(breaks ~ v)
LowerHinge Median UpperHinge Hinge-Spread
B:H 15 17 216
A:M 18 21 30   12
A:H 18 24 28   10
B:M 21 28 39   18
B:L 20 29 31   11
A:L 26 51 54   28

Suggested power transformation:  -0.2756176

So the ans is approximately  = -.2

My query is:corresponding to  p , what is the corresponding power
transformation defined as? Is it x^p  or (x^p -1) / p ?

Which one of the following is the final transformation ? Please clarify.
> boxplot(breaks ^ -.2 ~v)
> boxplot((breaks^-.2 -1)/(-.2)~v)
>

Best Regards,
Ashim

On Sun, Jan 7, 2018 at 10:59 AM, Ashim Kapoor <ashimkap...@gmail.com> wrote:

> Dear All,
>
> we need to do :
>
> library(car) for the spreadLevelPlot function
>
> I forgot to say that.
>
> Apologies,
> Ashim
>
> On Sun, Jan 7, 2018 at 10:37 AM, Ashim Kapoor <ashimkap...@gmail.com>
> wrote:
>
>> Dear All,
>>
>> I want a transformation which will make the spread of the response at
>> all combinations
>> of  2 factors the same.
>>
>> See for example :
>>
>> boxplot(breaks ~ tension * wool, warpbreaks)
>>
>> The closest I  can do is :
>>
>> spreadLevelPlot(breaks ~tension , warpbreaks)
>> spreadLevelPlot(breaks ~ wool , warpbreaks)
>>
>> I want to do :
>>
>> spreadLevelPlot(breaks ~tension * wool, warpbreaks)
>>
>> But I get :
>>
>> > spreadLevelPlot(breaks ~tension * wool , warpbreaks)
>> Error in spreadLevelPlot.formula(breaks ~ tension * wool, warpbreaks) :
>>   right-hand side of model has more than one variable
>>
>> What is the corresponding appropriate function for 2 factors ?
>>
>> Many thanks,
>> Ashim
>>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] SpreadLevelPlot for more than one factor

[Ashim Kapoor]

Dear All,

we need to do :

library(car) for the spreadLevelPlot function

I forgot to say that.

Apologies,
Ashim

On Sun, Jan 7, 2018 at 10:37 AM, Ashim Kapoor <ashimkap...@gmail.com> wrote:

> Dear All,
>
> I want a transformation which will make the spread of the response at  all
> combinations
> of  2 factors the same.
>
> See for example :
>
> boxplot(breaks ~ tension * wool, warpbreaks)
>
> The closest I  can do is :
>
> spreadLevelPlot(breaks ~tension , warpbreaks)
> spreadLevelPlot(breaks ~ wool , warpbreaks)
>
> I want to do :
>
> spreadLevelPlot(breaks ~tension * wool, warpbreaks)
>
> But I get :
>
> > spreadLevelPlot(breaks ~tension * wool , warpbreaks)
> Error in spreadLevelPlot.formula(breaks ~ tension * wool, warpbreaks) :
>   right-hand side of model has more than one variable
>
> What is the corresponding appropriate function for 2 factors ?
>
> Many thanks,
> Ashim
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] SpreadLevelPlot for more than one factor

[Ashim Kapoor]

Dear All,

I want a transformation which will make the spread of the response at  all
combinations
of  2 factors the same.

See for example :

boxplot(breaks ~ tension * wool, warpbreaks)

The closest I  can do is :

spreadLevelPlot(breaks ~tension , warpbreaks)
spreadLevelPlot(breaks ~ wool , warpbreaks)

I want to do :

spreadLevelPlot(breaks ~tension * wool, warpbreaks)

But I get :

> spreadLevelPlot(breaks ~tension * wool , warpbreaks)
Error in spreadLevelPlot.formula(breaks ~ tension * wool, warpbreaks) :
  right-hand side of model has more than one variable

What is the corresponding appropriate function for 2 factors ?

Many thanks,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] clustering levels using Tukey HSD in a one way anova

[Ashim Kapoor]

Dear all,

I am doing a one way between subjects anova in an unbalanced data set.
Suppose we have "a" levels of the one factor. I want to merge the not so
significantly different  levels into the same cluster.

Can I do a Tukey Kramer HSD and then use the following algorithm:

For i in 2 : "a"
 For j in 1 : i-1
if mean of level i is not significantly different to the mean
of level j,then put i and j in the same cluster. After the first time mean
of level i is not different to the mean of level j , just goto the next i ,
no need to compare with remaining j's.

Alternately,

I do not do Tukey Kramer HSD.  I only run the above algorithm. At each
iteration of the inner loop compute the contrast : mean of level i  = mean
j. At the first match I come out of the inner loop. To control for the (at
most)  1+ 2 + ... + (n-1) comparisons we can use bonferroni/scheffe / some
other technique.

Since this is a statistics query I have posted on stackexchange.  I have
not received a reply so I am posting my query here. Can some one please
answer my query here or on stackexchange?

The link to the query on stackexchange is:

https://stats.stackexchange.com/questions/320930/one-way-
anova-clustering-levels-using-tukey-kramer-hsd

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] How to read PMML data from a text file and convert it to a model ?

[Ashim Kapoor]

Dear All,

I want to save the XML representation of a model using PMML. Then I want to
read the model and predict using the model and a new  dataset.

This is described in this blog post :
https://www.r-bloggers.com/predictive-modeling-using-r-and-the-openscoring-engine-a-pmml-approach/

I am able to save the PMML representation of the model. I am not able to
read this representation convert it into  a model.  How can I do that ?

Here is a MWE :

library(pmml)
mydata<- iris

# Creating a model

mymodel <- lm(Sepal.Length ~ Sepal.Width, data = mydata)
#Computing its pmml representation and storing it to a file
saveXML(pmml(mymodel),file = "myfile.xml")

testingReading <- fileToXMLNode("myfile.txt")

> testingReading
http://www.dmg.org/PMML-4_3;>
 
  
  2017-11-15T10:37:29Z
 
 
  
  
 
 
  
   
   
  
  
   
  
 

Warning messages:
1: In structure(x$children, class = "XMLNodeList") :
  Calling 'structure(NULL, *)' is deprecated, as NULL cannot have
attributes.
  Consider 'structure(list(), *)' instead.
2: In structure(x$children, class = "XMLNodeList") :
  Calling 'structure(NULL, *)' is deprecated, as NULL cannot have
attributes.
  Consider 'structure(list(), *)' instead.
3: In structure(x$children, class = "XMLNodeList") :
  Calling 'structure(NULL, *)' is deprecated, as NULL cannot have
attributes.
  Consider 'structure(list(), *)' instead.
4: In structure(x$children, class = "XMLNodeList") :
  Calling 'structure(NULL, *)' is deprecated, as NULL cannot have
attributes.
  Consider 'structure(list(), *)' instead.
5: In structure(x$children, class = "XMLNodeList") :
  Calling 'structure(NULL, *)' is deprecated, as NULL cannot have
attributes.
  Consider 'structure(list(), *)' instead.
6: In structure(x$children, class = "XMLNodeList") :
  Calling 'structure(NULL, *)' is deprecated, as NULL cannot have
attributes.
  Consider 'structure(list(), *)' instead.
>

Can someone show me how to extract the model from the variable
testingReading and predict using this model at the values of rnorm(100) ?

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] regex in maps package

[Ashim Kapoor]

Dear All,

>From :

?maps::map

we have :

 map(database = "world", regions = ".", exact = FALSE, boundary = TRUE,
   interior = TRUE, projection = "", parameters = NULL, orientation =
NULL,
   fill = FALSE, col = 1, plot = TRUE, add = FALSE, namesonly = FALSE,
   xlim = NULL, ylim = NULL, wrap = FALSE, resolution = if (plot) 1
else 0,
   type = "l", bg = par("bg"), mar = c(4.1, 4.1, par("mar")[3], 0.1),
   myborder = 0.01, namefield="name", lforce="n", ...)

My query is the regex regions="." will match any single character. Should
it not be regions =".*" to be precise? Although I do realize that "." will
also match any string except "" (the empty string)? To be precise it should
be ".*" which can match an empty string as well ANY arbitrary string. I was
confused for a short while when I first read this. Look forward to
clarification on this.

Many thanks,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Default value of the option initial in the ses function in the forecast package.

[Ashim Kapoor]

Dear Peter,

Many thanks,
Ashim

On Mon, Oct 2, 2017 at 3:30 PM, peter dalgaard <pda...@gmail.com> wrote:

> The first one, i.e. "optimal"; check help for match.arg() for the idiom.
>
> -pd
>
>
> > On 2 Oct 2017, at 11:48 , Ashim Kapoor <ashimkap...@gmail.com> wrote:
> >
> > Dear All,
> >
> > I am trying to use the function ses from the forecast package.
> >
> > From its help I have :
> >
> > Usage:
> >
> > ses(y, h = 10, level = c(80, 95), fan = FALSE, initial = c("optimal",
> >   "simple"), alpha = NULL, lambda = NULL, biasadj = FALSE, x = y,
> ...)
> >
> > My query is that if I do not mention the initial value will its default
> > value be "optimal".
> >
> > A MWE would be  :
> >
> > library(fpp)
> > oildata <- window(oil,start=1996,end=2007)
> > fit3 <- ses(oildata, h=3)
> >
> > In the above is the default value of initial "optimal" or "simple" ?
> >
> > Note I have taken this example from https://www.otexts.org/fpp/7/1
> >
> > Best Regards,
> > Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
>
>
>
>
>
>
>
>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Default value of the option initial in the ses function in the forecast package.

[Ashim Kapoor]

Dear All,

I am trying to use the function ses from the forecast package.

>From its help I have :

Usage:

 ses(y, h = 10, level = c(80, 95), fan = FALSE, initial = c("optimal",
   "simple"), alpha = NULL, lambda = NULL, biasadj = FALSE, x = y, ...)

My query is that if I do not mention the initial value will its default
value be "optimal".

A MWE would be  :

library(fpp)
oildata <- window(oil,start=1996,end=2007)
fit3 <- ses(oildata, h=3)

In the above is the default value of initial "optimal" or "simple" ?

Note I have taken this example from https://www.otexts.org/fpp/7/1

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] ggmap + geom_raster

[Ashim Kapoor]

Dear all,

I want to :

1. Estimate a weighted 2D kernel.
2. Paint a heatmap on a ggmap.

Here is some reproducible data / code (I got it from the internet) :

s_rit <- structure(list(score = c(45, 60, 38, 98, 98, 53, 90, 42, 96,
45, 89, 18, 66, 2, 45, 98, 6, 83, 63, 86, 63, 81, 70, 8, 78,
15, 7, 86, 15, 63, 55, 13, 83, 76, 78, 70, 64, 88, 61, 78, 4,
7, 1, 70, 88, 58, 70, 58, 11, 45, 28, 42, 45, 73, 85, 86, 25,
17, 53, 95, 49, 80, 70, 35, 94, 61, 39, 76, 28, 1, 18, 93, 73,
67, 56, 38, 45, 66, 18, 76, 91, 76, 52, 60, 2, 38, 73, 95, 1,
76, 6, 25, 76, 81, 35, 49, 85, 55, 66, 90), lat = c(28.040961,
27.321633, 27.457342, 26.541129, 27.889476, 26.365284, 28.555024,
26.541129, 26.272728, 28.279994, 27.889476, 28.279994, 26.6674,
26.272728, 25.776045, 26.541129, 30.247658, 26.365284, 25.450123,
27.889476, 26.541129, 27.264513, 26.718652, 28.044369, 28.251435,
27.264513, 26.272728, 26.272728, 28.040961, 30.312239, 27.889476,
26.541129, 26.6674, 27.321633, 26.365284, 28.279994, 26.718652,
30.23286, 28.040961, 30.193704, 30.312239, 28.044369, 27.457342,
25.450123, 30.23286, 30.312239, 30.193704, 28.279994, 30.247658,
26.541129, 26.365284, 28.279994, 27.321633, 25.776045, 26.272728,
30.23286, 30.312239, 26.718652, 26.541129, 25.450123, 28.251435,
28.185751, 25.450123, 28.040961, 27.321633, 28.279994, 27.321633,
27.321633, 27.321633, 28.279994, 26.718652, 28.362308, 27.264513,
26.365284, 28.279994, 30.23286, 25.450123, 28.362308, 25.450123,
25.776045, 30.193704, 28.251435, 27.457342, 27.321633, 28.185751,
27.457342, 27.889476, 26.541129, 26.541129, 30.23286, 30.312239,
26.718652, 25.450123, 26.139258, 28.040961, 30.23286, 26.718652,
28.185751, 28.044369, 28.555024), lon = c(-82.5498, -80.376729,
-82.525985, -81.843986, -82.317701, -81.796389, -81.276464, -81.843986,
-80.207508, -81.331178, -82.317701, -81.331178, -80.072089, -80.207508,
-80.199437, -81.843986, -81.808664, -81.796389, -80.433557, -82.317701,
-81.843986, -80.432125, -80.091078, -82.394639, -81.490407, -80.432125,
-80.207508, -80.207508, -82.5498, -81.575916, -82.317701, -81.843986,
-80.072089, -80.376729, -81.796389, -81.331178, -80.091078, -81.585975,
-82.5498, -81.579846, -81.575916, -82.394639, -82.525985, -80.433557,
-81.585975, -81.575916, -81.579846, -81.331178, -81.808664, -81.843986,
-81.796389, -81.331178, -80.376729, -80.199437, -80.207508, -81.585975,
-81.575916, -80.091078, -81.843986, -80.433557, -81.490407, -81.289394,
-80.433557, -82.5498, -80.376729, -81.331178, -80.376729, -80.376729,
-80.376729, -81.331178, -80.091078, -81.428494, -80.432125, -81.796389,
-81.331178, -81.585975, -80.433557, -81.428494, -80.433557, -80.199437,
-81.579846, -81.490407, -82.525985, -80.376729, -81.289394, -82.525985,
-82.317701, -81.843986, -81.843986, -81.585975, -81.575916, -80.091078,
-80.433557, -80.238901, -82.5498, -81.585975, -80.091078, -81.289394,
-82.394639, -81.276464)), .Names = c("score", "lat", "lon"), row.names =
c(3205L,
8275L, 4645L, 8962L, 9199L, 340L, 5381L, 8998L, 5476L, 4956L,
9256L, 4940L, 6681L, 5586L, 1046L, 9017L, 1878L, 323L, 4175L,
9236L, 8968L, 6885L, 5874L, 9412L, 6434L, 7168L, 5420L, 5680L,
3202L, 1486L, 9255L, 9009L, 6833L, 8271L, 261L, 5024L, 8028L,
1774L, 3329L, 1824L, 1464L, 9468L, 4643L, 4389L, 1506L, 1441L,
1826L, 4968L, 1952L, 8803L, 339L, 4868L, 8266L, 1334L, 5483L,
1727L, 1389L, 7944L, 8943L, 4416L, 6440L, 526L, 4478L, 3117L,
8308L, 4891L, 8290L, 8299L, 8233L, 4848L, 7922L, 5795L, 6971L,
179L, 4990L, 1776L, 4431L, 5718L, 4268L, 1157L, 1854L, 6433L,
4637L, 8194L, 560L, 4694L, 9274L, 8903L, 8877L, 1586L, 1398L,
5865L, 4209L, 6075L, 3307L, 1634L, 8108L, 514L, 9453L, 5210L), class =
"data.frame")


library(ggmap)
library(RColorBrewer)
MyMap <- get_map(location= "Orlando, FL",
 source="google",
 maptype="roadmap", crop=FALSE, zoom=7)
YlOrBr <- c("#D4", "#FED98E", "#FE9929", "#D95F0E", "#993404")
ggmap(MyMap) + stat_density_2d(data=s_rit, aes(x=lon, y=lat,
fill=..level.., alpha=..level..),
  geom="polygon", size=0.01, bins=16) +
  scale_fill_gradient(low="red", high="green") +
  scale_alpha(range = c(0,0.3), guide=FALSE)

The above computes 2d density but it does not take the score (the weight
into account). To do a weighted KDE I do :

library(ks)

mydensity <- kde(x = s_rit[,c(2,3)],w = s_rit$score)

This computes a weighted KDE,but the data can't be passed to geom_raster
because the x,y's are 151 each while the estimated density is a matrix 151
x 151. However this can be accepted by image as shown below.


library(reshape2)
library(ggplot2)

x <- mydensity$eval.points[[1]]
y <- mydensity$eval.points[[2]]
z <- mydensity$estimate

image(x,y,z)

How can I convert this data so that I can do ggmap + geom_raster?

Also can I define eval.points in kde in a way that is relevant to the
underlying ggmap? How can I tweak the eval.points in the context of a map ?

Best Regards,
Ashim

[[alternative HTML version deleted]]

[R] Keep only those values in a row in a data frame which occur only once.

[Ashim Kapoor]

Dear All,

I have a file data.txt as follows:

Name_1,A,B,C
Name_2,E,F
Name_3,I,J,I,K,L,M

I will read this with:
my_data<- read.csv("data.txt",header=FALSE,col.names=paste0("V",
seq(1:10)),fill=TRUE)

Then the file will have 10 columns. I am assuming that each row in data.txt
will have at the max 10 entries.

Note: Here each row will have a different number of columns in data.txt but
each row will have 10 ( some trailing blank columns ) columns.

My query is how can I keep only the unique elements in each row? For
example: I want the row 3 to be Name_3,I,J,K,L,M

Please note I don't want the 2nd I to appear.

How can I do this?

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] regular expression help

[Ashim Kapoor]

Dear Enrico,

Many thanks and Best Regards,

Ashim.

On Thu, Jun 8, 2017 at 5:11 PM, Enrico Schumann <e...@enricoschumann.net>
wrote:

>
> Zitat von Ashim Kapoor <ashimkap...@gmail.com>:
>
>
> Dear All,
>>
>> My query is:
>>
>> Do we always need to use perl = TRUE option when doing ignore.case=TRUE?
>>
>> A small example :
>>
>> my_text =
>> "RECOVERY OFFICER-II\nDEBTS RECOVERY TRIBUNAL-III\n  RC No. 162/2015\nSBI
>> VS RAMESH GUPTA.\nDated: 01.03.2016   Item no.01\n
>> Present:   Ms. Sonakshi, the proxy counsel for Ms. Usha Singh, the counsel
>> for ARCIL.\nNone for the CDs.\n  The counsel for the CHFI
>> submitted that the matter has been assigned to ARCIL and deed of
>> assignment, application for substituting the name and vakalatnama has been
>> filed vide diary no. 1454 dated 08.02.2016\nIn the application it has been
>> prayed that ARCIL may be substituted in place of SBI for the purpose of
>> further proceedings in the matter. Request allowed.\nThe proxy counsel for
>> CHFI further requested to issue demand notice thereby mentioning the name
>> of ARCIL. Request allowed.\nRegistry is directed to issue fresh demand
>> notice mentioning the name of ARCIL.\nCHFI is directed to file status of
>> the mortgaged property as well as other assets of the CDs.\nList the case
>> on 28.03.2016.\n  (SUJEET KUMAR)\nRECOVERY OFFICER-II."
>>
>> My regular expression is:
>>
>> parties_present_start_1=
>> regexpr("\n.*Present.*\n.*\n",my_text,ignore.case=TRUE,perl=T)
>>
>> parties_present_start_2=
>> regexpr("\n.*Present.*\n.*\n",my_text,ignore.case=TRUE)
>>
>> parties_present_start_1
>>>
>> [1] 138
>> attr(,"match.length")
>> [1] 123
>> attr(,"useBytes")
>> [1] TRUE
>>
>>> parties_present_start_2
>>>
>> [1] 20
>> attr(,"match.length")
>> [1] 949
>> attr(,"useBytes")
>> [1] TRUE
>>
>>>
>>>
>> Why do I see the correct result only in the first case?
>>
>> Best Regards,
>> Ashim
>>
>>
> In Perl, '.' matches anything but a newline.
>
> In R, '.' matches any character.
>
>   test <- "hello\n1"
>   regexpr(".*[0-9]", test)
>   ## [1] 1
>   ## attr(,"match.length")
>   ## [1] 7
>   ## attr(,"useBytes")
>   ## [1] TRUE
>
>   regexpr(".*[0-9]", test, perl = TRUE)
>   ## [1] 7
>   ## attr(,"match.length")
>   ## [1] 1
>   ## attr(,"useBytes")
>   ## [1] TRUE
>
>
> --
> Enrico Schumann
> Lucerne, Switzerland
> http://enricoschumann.net
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] regular expression help

[Ashim Kapoor]

Dear All,

My query is:

Do we always need to use perl = TRUE option when doing ignore.case=TRUE?

A small example :

my_text =
"RECOVERY OFFICER-II\nDEBTS RECOVERY TRIBUNAL-III\n  RC No. 162/2015\nSBI
VS RAMESH GUPTA.\nDated: 01.03.2016   Item no.01\n
Present:   Ms. Sonakshi, the proxy counsel for Ms. Usha Singh, the counsel
for ARCIL.\nNone for the CDs.\n  The counsel for the CHFI
submitted that the matter has been assigned to ARCIL and deed of
assignment, application for substituting the name and vakalatnama has been
filed vide diary no. 1454 dated 08.02.2016\nIn the application it has been
prayed that ARCIL may be substituted in place of SBI for the purpose of
further proceedings in the matter. Request allowed.\nThe proxy counsel for
CHFI further requested to issue demand notice thereby mentioning the name
of ARCIL. Request allowed.\nRegistry is directed to issue fresh demand
notice mentioning the name of ARCIL.\nCHFI is directed to file status of
the mortgaged property as well as other assets of the CDs.\nList the case
on 28.03.2016.\n  (SUJEET KUMAR)\nRECOVERY OFFICER-II."

My regular expression is:

parties_present_start_1=
regexpr("\n.*Present.*\n.*\n",my_text,ignore.case=TRUE,perl=T)

parties_present_start_2=
regexpr("\n.*Present.*\n.*\n",my_text,ignore.case=TRUE)

> parties_present_start_1
[1] 138
attr(,"match.length")
[1] 123
attr(,"useBytes")
[1] TRUE
> parties_present_start_2
[1] 20
attr(,"match.length")
[1] 949
attr(,"useBytes")
[1] TRUE
>

Why do I see the correct result only in the first case?

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Warning from reshape2 when melting a data frame with uneven number of columns.

[Ashim Kapoor]

Is this the solution?

> d1<- as.data.frame(lapply(data,as.character),stringsAsFactors=FALSE)
> str(d1)
'data.frame':3 obs. of  5 variables:
 $ V1: chr  "Name1" "Name2" "Name3"
 $ V2: chr  "nam1" "name_12" "name-1"
 $ V3: chr  "nam2" "nam_34" "name-2"
 $ V4: chr  "nam3" "nam_56" ""
 $ V5: chr  "" "name_78" ""
> melt(d1,id.vars="V1")
  V1 variable   value
1  Name1   V2nam1
2  Name2   V2 name_12
3  Name3   V2  name-1
4  Name1   V3nam2
5  Name2   V3  nam_34
6  Name3   V3  name-2
7  Name1   V4nam3
8  Name2   V4  nam_56
9  Name3   V4
10 Name1   V5
11 Name2   V5 name_78
12 Name3   V5
>

On Sun, Jun 4, 2017 at 8:14 PM, John Kane <jrkrid...@yahoo.ca> wrote:

> I am not really sure what the warning means but I think your underlying
> problem is that all your variables are factors. Did you intend the values
> in each variable to be character?
> data.frame':3 obs. of  5 variables:
>  $ V1: Factor w/ 3 levels "Name1","Name2",..: 1 2 3
>  $ V2: Factor w/ 3 levels "nam1","name-1",..: 1 3 2
>  $ V3: Factor w/ 3 levels "nam2","nam_34",..: 1 2 3
>  $ V4: Factor w/ 3 levels "","nam3","nam_56": 2 3 1
>  $ V5: Factor w/ 2 levels "","name_78": 1 2 1
>
>
>
> On Sunday, June 4, 2017 6:48 AM, Ashim Kapoor <ashimkap...@gmail.com>
> wrote:
>
>
> Here is a small reproducible example:
>
> data <-
> structure(list(V1 = structure(1:3, .Label = c("Name1", "Name2",
> "Name3"), class = "factor"), V2 = structure(c(1L, 3L, 2L), .Label =
> c("nam1",
> "name-1", "name_12"), class = "factor"), V3 = structure(1:3, .Label =
> c("nam2",
> "nam_34", "name-2"), class = "factor"), V4 = structure(c(2L,
> 3L, 1L), .Label = c("", "nam3", "nam_56"), class = "factor"),
> V5 = structure(c(1L, 2L, 1L), .Label = c("", "name_78"), class =
> "factor")), .Names = c("V1",
> "V2", "V3", "V4", "V5"), class = "data.frame", row.names = c(NA,
> -3L))
>
> library(reshape2)
> data_long <- melt(data,id.vars="V1")
>
> Warning message:
> attributes are not identical across measure variables; they will be dropped
>
>
> Can someone please tell me how to correct this?
>
> Best Regards,
> Ashim
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html <http://www.r-project.org/posting-guide.html>
> and provide commented, minimal, self-contained, reproducible code.
>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Warning from reshape2 when melting a data frame with uneven number of columns.

[Ashim Kapoor]

Here is a small reproducible example:

data <-
structure(list(V1 = structure(1:3, .Label = c("Name1", "Name2",
"Name3"), class = "factor"), V2 = structure(c(1L, 3L, 2L), .Label =
c("nam1",
"name-1", "name_12"), class = "factor"), V3 = structure(1:3, .Label =
c("nam2",
"nam_34", "name-2"), class = "factor"), V4 = structure(c(2L,
3L, 1L), .Label = c("", "nam3", "nam_56"), class = "factor"),
V5 = structure(c(1L, 2L, 1L), .Label = c("", "name_78"), class =
"factor")), .Names = c("V1",
"V2", "V3", "V4", "V5"), class = "data.frame", row.names = c(NA,
-3L))

library(reshape2)
data_long <- melt(data,id.vars="V1")

Warning message:
attributes are not identical across measure variables; they will be dropped


Can someone please tell me how to correct this?

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Analysing the output from skmeans/clustering

[Ashim Kapoor]

Dear All,

Here is a small example:

library(skmeans)
library(tm)
data("crude")
#Examine the first document
inspect(crude[[1]])

dtm <- DocumentTermMatrix(crude, control =
   list(removePunctuation = TRUE,
   removeNumbers = TRUE,
stopwords = TRUE))
clus <- skmeans(dtm,3)
names(clus)

Is there any way I can get the document number of the  prototypes ? Also
can I get the 3 closest documents to each prototype ? By prototype I mean
the cluster centers.

I know can compare each row of the DocumentTermMatrix with the prototypes
to test for equality and I can manually compute the distance of each
Document from a prototypes,but I was wondering if such a tool already
exists.

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] qqplot for binomial distribution

[Ashim Kapoor]

Dear All,

when I do :

set.seed(123)

expected_distribution<-rbinom(1000,100,.05)

#Without jitter

qqplot(jitter(expected_distribution),count1_vector, xlab="Expected
distribution",ylab="Observed values")
qqline(count1_vector,distribution = function(probs) { qbinom(probs,
size=100, prob=0.05) },col = "red",lwd = 2)

I get a line through the middle. Is this satisfactory ?

#With jitter
qqplot(jitter(expected_distribution),jitter(count1_vector),xlab="Expected
distribution",ylab="Observed values")
qqline(jitter(count1_vector),distribution = function(probs) { qbinom(probs,
size=100, prob=0.05) },col = "red",lwd = 2)

Now I can see the line is  not exactly through the middle. Can I think that
this is due to the way the data is discretized?

count1_vector <-
c(2, 6, 5, 8, 8, 6, 8, 3, 5, 8, 7, 6, 4, 7, 5, 2, 3, 3, 6, 3,
7, 3, 4, 8, 6, 6, 3, 5, 5, 3, 4, 5, 4, 2, 3, 6, 7, 7, 5, 4, 4,
7, 9, 4, 4, 4, 8, 9, 3, 5, 4, 7, 4, 3, 8, 6, 5, 5, 7, 3, 6, 7,
8, 7, 9, 3, 5, 5, 9, 8, 7, 7, 2, 3, 5, 2, 4, 14, 7, 7, 7, 3,
5, 4, 2, 12, 3, 6, 9, 4, 4, 3, 4, 4, 4, 6, 7, 4, 6, 10, 8, 5,
3, 3, 1, 3, 4, 3, 7, 3, 9, 7, 3, 3, 7, 5, 1, 2, 2, 3, 5, 4, 3,
8, 7, 0, 5, 3, 3, 4, 9, 2, 7, 5, 5, 5, 7, 7, 5, 4, 7, 2, 3, 5,
4, 5, 2, 10, 6, 3, 6, 2, 11, 2, 5, 5, 2, 5, 2, 10, 4, 5, 9, 1,
5, 6, 3, 4, 7, 7, 2, 2, 3, 6, 6, 6, 7, 3, 3, 6, 1, 4, 4, 4, 10,
4, 7, 4, 3, 4, 6, 5, 6, 7, 3, 7, 3, 5, 6, 6, 4, 5, 1, 7, 5, 7,
6, 7, 5, 3, 6, 7, 10, 5, 5, 4, 9, 6, 3, 9, 8, 4, 2, 8, 10, 4,
6, 7, 4, 4, 8, 4, 5, 4, 5, 6, 6, 5, 8, 2, 2, 6, 5, 3, 7, 7, 4,
9, 6, 5, 7, 8, 6, 1, 2, 3, 4, 4, 6, 8, 5, 8, 5, 7, 7, 4, 6, 3,
4, 5, 3, 4, 5, 3, 3, 4, 5, 5, 7, 8, 6, 5, 3, 3, 4, 3, 8, 6, 6,
3, 0, 4, 2, 7, 3, 5, 4, 6, 7, 4, 7, 6, 5, 8, 6, 7, 4, 5, 3, 6,
7, 6, 3, 5, 3, 3, 6, 3, 3, 2, 7, 5, 10, 2, 4, 5, 2, 4, 10, 5,
2, 7, 8, 5, 3, 7, 4, 2, 4, 3, 5, 6, 8, 10, 3, 7, 5, 8, 5, 2,
6, 8, 6, 7, 8, 2, 4, 2, 4, 3, 2, 4, 4, 2, 4, 3, 12, 2, 11, 5,
8, 8, 3, 6, 2, 6, 3, 5, 4, 8, 4, 5, 7, 2, 5, 3, 5, 3, 7, 6, 5,
2, 8, 6, 3, 3, 5, 3, 2, 6, 5, 8, 7, 4, 2, 3, 5, 2, 6, 4, 9, 5,
4, 4, 2, 1, 3, 3, 2, 5, 7, 6, 4, 4, 5, 6, 7, 4, 4, 4, 5, 9, 7,
5, 3, 5, 5, 2, 11, 9, 6, 8, 6, 6, 8, 6, 3, 6, 3, 7, 3, 3, 7,
4, 7, 4, 8, 3, 4, 8, 8, 8, 7, 4, 6, 1, 3, 7, 5, 13, 8, 1, 8,
5, 1, 3, 5, 4, 6, 5, 4, 3, 3, 7, 5, 5, 5, 3, 5, 5, 1, 8, 6, 4,
5, 9, 3, 8, 6, 4, 7, 6, 7, 5, 5, 6, 2, 7, 8, 11, 10, 4, 8, 5,
5, 4, 5, 4, 2, 8, 3, 3, 4, 5, 7, 12, 4, 7, 5, 4, 9, 8, 4, 5,
9, 4, 6, 5, 5, 2, 3, 4, 7, 7, 7, 7, 1, 6, 6, 6, 4, 8, 8, 5, 7,
3, 4, 6, 2, 6, 6, 4, 8, 4, 3, 7, 1, 4, 6, 2, 3, 5, 5, 9, 7, 1,
4, 1, 5, 3, 5, 4, 3, 5, 10, 4, 8, 6, 4, 3, 5, 6, 4, 6, 6, 4,
7, 7, 6, 5, 4, 4, 6, 10, 6, 5, 3, 8, 3, 4, 3, 5, 5, 2, 6, 6,
8, 2, 5, 9, 6, 5, 5, 4, 10, 7, 3, 5, 6, 8, 5, 3, 3, 7, 3, 4,
6, 2, 9, 2, 6, 5, 3, 6, 2, 4, 3, 4, 5, 5, 1, 5, 4, 11, 4, 1,
9, 5, 4, 7, 2, 11, 4, 9, 6, 5, 5, 6, 6, 7, 9, 4, 4, 4, 4, 3,
7, 3, 3, 4, 2, 6, 6, 6, 4, 6, 2, 5, 6, 5, 4, 3, 4, 7, 8, 7, 3,
5, 4, 4, 4, 4, 4, 4, 2, 7, 3, 5, 7, 1, 5, 5, 2, 7, 3, 3, 5, 3,
5, 4, 9, 5, 7, 8, 7, 7, 4, 5, 3, 5, 6, 5, 1, 6, 5, 5, 8, 7, 3,
6, 8, 1, 12, 1, 7, 6, 6, 3, 4, 4, 2, 2, 3, 2, 8, 4, 3, 7, 9,
10, 5, 5, 6, 7, 3, 7, 4, 7, 7, 3, 5, 9, 7, 3, 6, 6, 2, 5, 4,
3, 5, 8, 5, 6, 3, 4, 5, 2, 4, 3, 4, 5, 2, 7, 2, 7, 5, 5, 6, 8,
4, 8, 6, 5, 4, 5, 1, 6, 6, 2, 4, 8, 5, 7, 6, 10, 6, 4, 4, 4,
9, 5, 3, 1, 10, 7, 5, 6, 4, 7, 5, 6, 4, 2, 4, 6, 5, 3, 3, 6,
5, 9, 3, 7, 9, 4, 1, 4, 2, 4, 5, 4, 4, 2, 7, 11, 3, 3, 5, 8,
3, 5, 7, 9, 6, 11, 6, 5, 3, 7, 5, 3, 7, 4, 5, 4, 4, 8, 3, 3,
5, 4, 3, 7, 4, 2, 10, 2, 4, 3, 8, 4, 4, 5, 3, 3, 6, 2, 7, 2,
2, 11, 1, 6, 3, 6, 5, 7, 3, 3, 1, 7, 9, 8, 7, 2, 5, 4, 3, 7,
7, 2, 5, 4, 3, 3, 6, 10, 4, 9, 6, 5, 3, 4, 5, 5, 6, 6, 7, 3,
4, 8, 6, 4, 5, 1, 5, 9, 3, 6, 2, 4, 5, 5, 3, 3, 3, 3, 5, 4, 4,
5, 5, 1, 4, 5, 8, 7, 4, 3, 3, 5, 5, 4, 6, 5, 4, 7, 4, 4, 3, 3,
8, 4, 6, 7, 3, 4, 3, 5, 5, 7, 3, 6, 9, 7, 4, 3, 2, 6)








On Wed, Apr 19, 2017 at 12:32 PM, Ashim Kapoor <ashimkap...@gmail.com>
wrote:

> Dear Boris,
>
> Many thanks,
> Ashim
>
> On Tue, Apr 18, 2017 at 7:56 PM, Boris Steipe <boris.ste...@utoronto.ca>
> wrote:
>
>> As per the help pages, the data samples are expected in the second
>> argument, "y".
>>
>> So try
>>   qqplot(rbinom(n=100, size=100, p=0.05), count1_vector)
>>
>> ... and then plot your qqline()
>>
>> Alternatively, try
>>
>> qqline(count1_vector,
>>distribution = function(probs) { qbinom(probs, size=100,
>> prob=0.05) },
>>datax = TRUE, # <- logical. Should data values be on the x-axis?
>>col = "red",
>>lwd = 0.5)
>> ... and use your original qqplot()
>>
>>
>> B.
>>
>>
>> > On Apr 18, 2017, at 12:47 AM, Ashim Kapoor <ashimkap...@gmail.com>
>> wrote:
>> >
>> > Dear Boris,
>> >
>> > Thank you for your reply.
>> >
>

Re: [R] Perfect prediction of AR1 series using package dlm, posted on stack exchange

[Ashim Kapoor]

Dear Peter,

Many thanks,
Ashim.

On Thu, May 4, 2017 at 4:54 PM, peter dalgaard <pda...@gmail.com> wrote:

> I am not an expert on dlm, but it seems to me that you are getting perfect
> _filtering_ not _prediction_. If you cast an AR model as a state space
> model, there is no measurement error on the state values, hence the
> conditional distribution of theta_t given y_t is just the point value of
> y_t...
>
> -pd
>
> > On 4 May 2017, at 12:05 , Ashim Kapoor <ashimkap...@gmail.com> wrote:
> >
> > Dear all,
> >
> > I have made a dlm model,where I am getting a perfect prediction.
> >
> > Here is a link to the output:
> >
> > http://pasteboard.co/9IxVQwjm6.png
> >
> > The query and code is on:
> >
> > https://stats.stackexchange.com/questions/276449/perfect-
> prediction-in-case-of-a-univariate-ar1-model-using-dlm
> >
> > Can someone here be kind enough to answer my query?
> >
> > Best Regards,
> > Ashim
> >
> >   [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> --
> Peter Dalgaard, Professor,
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Office: A 4.23
> Email: pd@cbs.dk  Priv: pda...@gmail.com
>
>
>
>
>
>
>
>
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Perfect prediction of AR1 series using package dlm, posted on stack exchange

[Ashim Kapoor]

Dear all,

I have made a dlm model,where I am getting a perfect prediction.

Here is a link to the output:

http://pasteboard.co/9IxVQwjm6.png

The query and code is on:

https://stats.stackexchange.com/questions/276449/perfect-prediction-in-case-of-a-univariate-ar1-model-using-dlm

Can someone here be kind enough to answer my query?

Best Regards,
Ashim

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] The effect of tolerance in all.equal()

[Ashim Kapoor]

On Sun, Apr 30, 2017 at 10:05 PM, Duncan Murdoch <murdoch.dun...@gmail.com>
wrote:

> On 30/04/2017 12:26 PM, Ashim Kapoor wrote:
>
>> Dear All,
>>
>> This answer is very clear. Many thanks.
>>
>> I am now confused about how str*ucture works. Where can I read more about
>> when does it  return language / logical / chr ? I would want to read that
>> so I can interpret the result of structure. I don't think ?str contains
>> this.To me, logical and chr make sense, what does language mean? I think I
>> need to read some more.
>>
>
> I would read the R Language Definition manual, and then bits and pieces of
> R Internals, as necessary.  These are both included with R.  There are also
> books separate from R that talk about these things, but I don't know which
> to recommend.
>
> Can you please name those books ?


> Duncan Murdoch
>
>
>> Many thanks,
>> Ashim
>>
>> On Tue, Apr 25, 2017 at 3:14 PM, Martin Maechler <
>> maech...@stat.math.ethz.ch
>>
>>> wrote:
>>>
>>
>> Ashim Kapoor <ashimkap...@gmail.com>
>>>>>>>> on Tue, 25 Apr 2017 14:02:18 +0530 writes:
>>>>>>>>
>>>>>>>
>>> > Dear all,
>>> > I am not able to understand the interplay of absolute vs relative
>>> and
>>> > tolerance in the use of all.equal
>>>
>>> > If I want to find out if absolute differences between 2
>>> numbers/vectors are
>>> > bigger than a given tolerance I would do:
>>>
>>> > all.equal(1,1.1,scale=1,tol= .1)
>>>
>>> > If I want to find out if relative differences between 2
>>> numbers/vectors are
>>> > bigger than a given tolerance I would do :
>>>
>>> > all.equal(1,1.1,tol=.1)
>>>
>>> > 
>>> ##
>>>
>>> > I can also do :
>>>
>>> > all.equal(1,3,tol=1)
>>>
>>> > to find out if the absolute difference is bigger than 1.But here I
>>> won't be
>>> > able to detect absolute differences smaller than 1 in this case,so
>>> I
>>> don't
>>> > think that this is a good way.
>>>
>>> > My query is: what is the reasoning behind all.equal returning the
>>> absolute
>>> > difference if the tolerance >= target and relative difference if
>>> tolerance
>>> > < target?
>>> (above, it istol  >/<=  |target|  ie. absolute value)
>>>
>>>
>>> The following are desiderata / restrictions :
>>>
>>> 1) Relative tolerance is needed to keep things scale-invariant
>>>i.e.,  all.equal(x, y)  and  all.equal(1000 * x, 1000 * y)
>>>should typically be identical for (almost) all (x,y).
>>>
>>>==> "the typical behavior should use relative error tolerance"
>>>
>>> 2) when x or y (and typically both!) are very close to zero it
>>>is typically undesirable to keep relative tolerances (in the
>>>boundary case, they _are_ zero exactly, and "relative error" is
>>> undefined).
>>>E.g., for most purposes, 3.45e-15 and 1.23e-17 should be counted as
>>>equal to zero and hence to themselves.
>>>
>>> 1) and 2) are typically reconciled by switching from relative to absolute
>>> when the arguments are close to zero (*).
>>>
>>> The exact cutoff at which to switch from relative to absolute
>>> (or a combination of the two) is somewhat arbitrary(*2) and for
>>> all.equal() has been made in the 1980's (or even slightly
>>> earlier?) when all.equal() was introduced into the S language at
>>> Bell labs AFAIK. Maybe John Chambers (or Rick Becker or ...,
>>> but they may not read R-help) knows more.
>>> *2) Then, the choice for all.equal() is in some way "least arbitrary",
>>> using c = 1 in the more general   tolerance >= c*|target|  framework.
>>>
>>> *) There have been alternatives in "the (applied numerical
>>>  analysis / algorithm) literature" seen in published algorithms,
>>>  but I don't have any example ready.
>>>  Notably some of these alternatives are _symmetric_ in (x,y)
>>>  where all.equal() was designed to be asymmetric using names
>>>  'target' and 

Re: [R] The effect of tolerance in all.equal()

[Ashim Kapoor]

Dear All,

This answer is very clear. Many thanks.

I am now confused about how str*ucture works. Where can I read more about
when does it  return language / logical / chr ? I would want to read that
so I can interpret the result of structure. I don't think ?str contains
this.To me, logical and chr make sense, what does language mean? I think I
need to read some more.

Many thanks,
Ashim

On Tue, Apr 25, 2017 at 3:14 PM, Martin Maechler <maech...@stat.math.ethz.ch
> wrote:

> >>>>> Ashim Kapoor <ashimkap...@gmail.com>
> >>>>> on Tue, 25 Apr 2017 14:02:18 +0530 writes:
>
> > Dear all,
> > I am not able to understand the interplay of absolute vs relative and
> > tolerance in the use of all.equal
>
> > If I want to find out if absolute differences between 2
> numbers/vectors are
> > bigger than a given tolerance I would do:
>
> > all.equal(1,1.1,scale=1,tol= .1)
>
> > If I want to find out if relative differences between 2
> numbers/vectors are
> > bigger than a given tolerance I would do :
>
> > all.equal(1,1.1,tol=.1)
>
> > 
> ##
>
> > I can also do :
>
> > all.equal(1,3,tol=1)
>
> > to find out if the absolute difference is bigger than 1.But here I
> won't be
> > able to detect absolute differences smaller than 1 in this case,so I
> don't
> > think that this is a good way.
>
> > My query is: what is the reasoning behind all.equal returning the
> absolute
> > difference if the tolerance >= target and relative difference if
> tolerance
> > < target?
> (above, it istol  >/<=  |target|  ie. absolute value)
>
>
> The following are desiderata / restrictions :
>
> 1) Relative tolerance is needed to keep things scale-invariant
>i.e.,  all.equal(x, y)  and  all.equal(1000 * x, 1000 * y)
>should typically be identical for (almost) all (x,y).
>
>==> "the typical behavior should use relative error tolerance"
>
> 2) when x or y (and typically both!) are very close to zero it
>is typically undesirable to keep relative tolerances (in the
>boundary case, they _are_ zero exactly, and "relative error" is
> undefined).
>E.g., for most purposes, 3.45e-15 and 1.23e-17 should be counted as
>equal to zero and hence to themselves.
>
> 1) and 2) are typically reconciled by switching from relative to absolute
> when the arguments are close to zero (*).
>
> The exact cutoff at which to switch from relative to absolute
> (or a combination of the two) is somewhat arbitrary(*2) and for
> all.equal() has been made in the 1980's (or even slightly
> earlier?) when all.equal() was introduced into the S language at
> Bell labs AFAIK. Maybe John Chambers (or Rick Becker or ...,
> but they may not read R-help) knows more.
> *2) Then, the choice for all.equal() is in some way "least arbitrary",
> using c = 1 in the more general   tolerance >= c*|target|  framework.
>
> *) There have been alternatives in "the (applied numerical
>  analysis / algorithm) literature" seen in published algorithms,
>  but I don't have any example ready.
>  Notably some of these alternatives are _symmetric_ in (x,y)
>  where all.equal() was designed to be asymmetric using names
>  'target' and 'current'.
>
> The alternative idea is along the following thoughts:
>
> Assume that for "equality" we want _both_ relative and
> absolute (e := tolerance) "equality"
>
>|x - y| < e (|x|+|y|)/2  (where you could use |y| or |x|
>  instead of their mean; all.equal()
>  uses |target|)
>|x - y| < e * e1  (where e1 = 1, or e1 = 10^-7..)
>
> If you add the two inequalities you get
>
>|x - y| < e (e1 + |x+y|/2)
>
> as check which is a "mixture" of relative and absolute tolerance.
>
> With a somewhat long history, my gut feeling would nowadays
> actually prefer this (I think with a default of e1 = e) - which
> does treat x and y symmetrically.
>
> Note that convergence checks in good algorithms typically check
> for _both_ relative and absolute difference (each with its
> tolerance providable by the user), and the really good ones for
> minimization do  check for (approximate) gradients also being
> close to zero - as old timers among us should have learned from
> Doug Bates ... but now I'm really diverging.
>
> Last but not least some  R  code at the end,  showing that the *asy