Thanks for the request for feedback on this change.
I share Josiah's view that "In the case of an isolated node, I think the best
thing to do is to return NA rather than 0. For example consider administrative
boundaries where the lagged variable is median household income. If we returned
a 0 in that case, we'd likely be introducing quite the outlier!" Currently
geostan's Moran scatter plot uses NA, so observations with no neighbors are
neutral, i.e., they are dropped when calculating Moran's I or creating a Moran
scatter plot. It also shows a kind of confusing error message about how this
impacts a row-standardized spatial weights matrix---if you just drop one or
more observations, the weights are no longer row-standardized relative to the
values included in the calculation.
Also, "creating nb neighbour objects should themselves report that an output
object (graph) is not connected"---that's a good suggestion.
Thanks,
Connor
________________________________
From: R-sig-Geo <r-sig-geo-boun...@r-project.org> on behalf of
r-sig-geo-requ...@r-project.org <r-sig-geo-requ...@r-project.org>
Sent: Monday, November 6, 2023 5:00 AM
To: r-sig-geo@r-project.org <r-sig-geo@r-project.org>
Subject: R-sig-Geo Digest, Vol 243, Issue 3
Send R-sig-Geo mailing list submissions to
r-sig-geo@r-project.org
To subscribe or unsubscribe via the World Wide Web, visit
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754639883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Zv6cmTUDnTT3WBgX%2Bwg%2FNFc%2Bx8BK84YatH%2Fjalf6grY%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-sig-geo>
or, via email, send a message with subject or body 'help' to
r-sig-geo-requ...@r-project.org
You can reach the person managing the list at
r-sig-geo-ow...@r-project.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of R-sig-Geo digest..."
Today's Topics:
1. Re: spdep: new zero.policy attribute (Roger Bivand)
2. Re: spdep: new zero.policy attribute (Josiah Parry)
----------------------------------------------------------------------
Message: 1
Date: Sun, 5 Nov 2023 15:31:16 +0000
From: Roger Bivand <roger.biv...@nhh.no>
To: "r-sig-geo@r-project.org" <r-sig-geo@r-project.org>
Subject: Re: [R-sig-Geo] spdep: new zero.policy attribute
Message-ID:
<sv0p279mb0475841dfe21ee66fbcbd252ee...@sv0p279mb0475.norp279.prod.outlook.com>
Content-Type: text/plain; charset="iso-8859-1"
And a question: in nb2listw() and similar functions creating spatial weights
listw objects, would it be sensible to guess that the presence of no-neighbour
observations in the input nb neighbour implies the choice of a spatially lagged
value of zero (zero.policy=TRUE), lx = Wx, rather than NA (zero.policy=FALSE)?
That is, use by default zero.policy=any(card(nb) == 0L) rather than
zero.policy=NULL and look in the spdep option set by default on package load to
FALSE but settable by the user?
Would this be taking trying to be helpful too far, given that the analyst is
creating the neighbour object and presumably should take responsibility for
choices made?
Context: polygons not sharing boundaries with other polygons do exist
legitimately in data sources, but setting spatially lagged values to zero for
those polygons is quite an invasive imputation. It may be better to oblige the
user to make the choice when the spatial weights listw object is created.
Little is known about the problem, for a recent treatment for CAR models see:
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F1705.04854&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754639883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=G8%2B714lKy1JUjezV0C%2FlRmSGYsq8WCj0iPkwk%2FQjMQI%3D&reserved=0<https://arxiv.org/abs/1705.04854>,
published as
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1016%2Fj.sste.2018.04.002&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754639883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nUkXfGSH%2BX8Y0RJSL1MTch67EdzW861xbHZiWuQRJ9w%3D&reserved=0<https://doi.org/10.1016/j.sste.2018.04.002>,
where: "The specification of a CAR model on a disconnected graph is undefined
... [t]here are essentially two types of disconnected graphs: first, a graph
containing an island (a singleton node with no neighbours), second, a graph
split in different sub-graphs (each of them being a connected graph)".
This question concerns the former, singleton, case, but adding sub-graph counts
if greater than unity to summary.nb and print.nb address the second . Very
possibly, functions creating nb neighbour objects should themselves report that
an output object (graph) is not connected, bigDM CARBayes CARBayesST geostan
spatialreg stampr do call spdep::n.comp.nb themselves to check the subgraph
count.
Interested in feedback,
Roger
--
Roger Bivand
Emeritus Professor
Norwegian School of Economics
Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
roger.biv...@nhh.no
________________________________________
From: R-sig-Geo <r-sig-geo-boun...@r-project.org> on behalf of Roger Bivand
<roger.biv...@nhh.no>
Sent: 04 November 2023 18:53
To: r-sig-geo@r-project.org
Subject: [R-sig-Geo] spdep: new zero.policy attribute
In forthcoming spdep 1.3-1, spatial weight listw objects get a new zero.policy
attribute. The attribute is added as objects are created to record the status
of the zero.policy argument in the function creating the object, see:
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fr-spatial%2Fspdep%2Fcommit%2Fe159de922c61713529a4075b0dfc2966eb8f9ad6&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754639883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=uy25DfhupOiu78MrTTjDn6nMPUDkHdYehSHimuMVoJk%3D&reserved=0<https://github.com/r-spatial/spdep/commit/e159de922c61713529a4075b0dfc2966eb8f9ad6>.
Reverse dependency checks only show problems from over-eager unit testing in
SpatialFeatureExperiment, a Bioconductor package, but other workflows may be
impacted. The new attribute is used in tests for spatial autocorrelation to set
the zero.policy argument in those tests (the arguments were zero.policy=NULL,
are now zero.policy=attr(listw, "zero.policy") where listw is the spatial
weights object argument to the test function.
This will be extended to spatialreg and friends if nobody reports negative
impacts here soon. I'll wait before releasing 1.3-1 for a few days to see if
any feedback is forthcoming.
Hope this long-overdue change is helpful,
Roger
--
Roger Bivand
Emeritus Professor
Norwegian School of Economics
Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
roger.biv...@nhh.no
_______________________________________________
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754639883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Zv6cmTUDnTT3WBgX%2Bwg%2FNFc%2Bx8BK84YatH%2Fjalf6grY%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-sig-geo>
------------------------------
Message: 2
Date: Sun, 5 Nov 2023 12:01:38 -0500
From: Josiah Parry <josiah.pa...@gmail.com>
To: Roger Bivand <roger.biv...@nhh.no>
Cc: "r-sig-geo@r-project.org" <r-sig-geo@r-project.org>
Subject: Re: [R-sig-Geo] spdep: new zero.policy attribute
Message-ID:
<CAL3ufULghGnSos1J1kMVGVVWdhiawoz+ec=QbraQRY-x2h=k...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
My take is generally that less is more. In the case of an isolated node, I
think the best thing to do is to return NA rather than 0. For example
consider administrative boundaries where the lagged variable is median
household income. If we returned a 0 in that case, we'd likely be
introducing quite the outlier!
My preference would be to _always_ default to NA lagged values when no
neighbors are present. Additionally, it would be quite nice to instruct
users on how to impute these values with other lagged variables. Say we
have an isolated node, but we don't want an NA value. We can impute that
missing value with the spatial lag of the variable using a different
neighborhood construction—e.g. using KNN with k = 3 to *ensure* that the
node always has values to lag.
Here is an example gist imputing k=3 lag
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2FJosiahParry%2Feb7878fc375fb931ddd6675a2c591a2b&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754639883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2FcHiYgOC3o0dU1NqiP4SbHH1B4JEejGvfi1qyO62iWM%3D&reserved=0<https://gist.github.com/JosiahParry/eb7878fc375fb931ddd6675a2c591a2b>
Another option could be to use the focal feature's value *as* the lag
itself. If a location has no neighborhood could we argue that the location *is
*its own neighborhood?
I'm not too familiar with the models to comment on them, though. I do see
this more as a pre-processing / data imputation issue. I suspect that's
more of a machine learning paradigm though!
On Sun, Nov 5, 2023 at 10:31 AM Roger Bivand <roger.biv...@nhh.no> wrote:
> And a question: in nb2listw() and similar functions creating spatial
> weights listw objects, would it be sensible to guess that the presence of
> no-neighbour observations in the input nb neighbour implies the choice of a
> spatially lagged value of zero (zero.policy=TRUE), lx = Wx, rather than NA
> (zero.policy=FALSE)?
>
> That is, use by default zero.policy=any(card(nb) == 0L) rather than
> zero.policy=NULL and look in the spdep option set by default on package
> load to FALSE but settable by the user?
>
> Would this be taking trying to be helpful too far, given that the analyst
> is creating the neighbour object and presumably should take responsibility
> for choices made?
>
> Context: polygons not sharing boundaries with other polygons do exist
> legitimately in data sources, but setting spatially lagged values to zero
> for those polygons is quite an invasive imputation. It may be better to
> oblige the user to make the choice when the spatial weights listw object is
> created.
>
> Little is known about the problem, for a recent treatment for CAR models
> see:
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F1705.04854&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754639883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=G8%2B714lKy1JUjezV0C%2FlRmSGYsq8WCj0iPkwk%2FQjMQI%3D&reserved=0<https://arxiv.org/abs/1705.04854>,
> published as
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1016%2Fj.sste.2018.04.002&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754796114%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zsDOSppZjxGiSppcmXCSYb1lWlIh8RgettJucnm4RNg%3D&reserved=0<https://doi.org/10.1016/j.sste.2018.04.002>,
> where: "The specification of
> a CAR model on a disconnected graph is undefined ... [t]here are
> essentially two types of disconnected graphs: first, a graph containing an
> island (a singleton node with no neighbours), second, a graph split in
> different sub-graphs (each of them being a connected graph)".
>
> This question concerns the former, singleton, case, but adding sub-graph
> counts if greater than unity to summary.nb and print.nb address the second
> . Very possibly, functions creating nb neighbour objects should themselves
> report that an output object (graph) is not connected, bigDM CARBayes
> CARBayesST geostan spatialreg stampr do call spdep::n.comp.nb themselves to
> check the subgraph count.
>
> Interested in feedback,
>
> Roger
>
> --
> Roger Bivand
> Emeritus Professor
> Norwegian School of Economics
> Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
> roger.biv...@nhh.no
>
> ________________________________________
> From: R-sig-Geo <r-sig-geo-boun...@r-project.org> on behalf of Roger
> Bivand <roger.biv...@nhh.no>
> Sent: 04 November 2023 18:53
> To: r-sig-geo@r-project.org
> Subject: [R-sig-Geo] spdep: new zero.policy attribute
>
> In forthcoming spdep 1.3-1, spatial weight listw objects get a new
> zero.policy attribute. The attribute is added as objects are created to
> record the status of the zero.policy argument in the function creating the
> object, see:
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fr-spatial%2Fspdep%2Fcommit%2Fe159de922c61713529a4075b0dfc2966eb8f9ad6&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754796114%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=E3ixQ1qYnfd6iqsSSZzwmHbvhayd4S04EtYfh0wENTc%3D&reserved=0<https://github.com/r-spatial/spdep/commit/e159de922c61713529a4075b0dfc2966eb8f9ad6>
> .
>
> Reverse dependency checks only show problems from over-eager unit testing
> in SpatialFeatureExperiment, a Bioconductor package, but other workflows
> may be impacted. The new attribute is used in tests for spatial
> autocorrelation to set the zero.policy argument in those tests (the
> arguments were zero.policy=NULL, are now zero.policy=attr(listw,
> "zero.policy") where listw is the spatial weights object argument to the
> test function.
>
> This will be extended to spatialreg and friends if nobody reports negative
> impacts here soon. I'll wait before releasing 1.3-1 for a few days to see
> if any feedback is forthcoming.
>
> Hope this long-overdue change is helpful,
>
> Roger
>
> --
> Roger Bivand
> Emeritus Professor
> Norwegian School of Economics
> Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
> roger.biv...@nhh.no
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754796114%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iebpGpErkLKR7eWsKVSU9znpLElbqqa1QUjjKCH9yZ8%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-sig-geo>
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754796114%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iebpGpErkLKR7eWsKVSU9znpLElbqqa1QUjjKCH9yZ8%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-sig-geo>
>
[[alternative HTML version deleted]]
------------------------------
Subject: Digest Footer
_______________________________________________
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C01%7Cconnor.donegan%40utdallas.edu%7C981c03aaa25a43db669d08dbdeb7b093%7C8d281d1d9c4d4bf7b16e032d15de9f6c%7C0%7C0%7C638348652754796114%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iebpGpErkLKR7eWsKVSU9znpLElbqqa1QUjjKCH9yZ8%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-sig-geo>
------------------------------
End of R-sig-Geo Digest, Vol 243, Issue 3
*****************************************
[[alternative HTML version deleted]]
_______________________________________________
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
