Re: [spatial] Cartesian Tiers nomenclature
Right, NRQ is able to translate any requested range into the union (OR) of brackets (from the trie) created during indexing. Can spatial do the same thing, just with 2D instead of 1D? Ie, reconstruct any expressible shape (created at query time) as the union of some number of grids/tiers, at finer finer levels, created during indexing? Spatial, today, seems to do this, except it must also do precise filtering on each matching doc, because some of the grids may contain hits outside of the requested shape. In fact, NRQ could also borrow from spatial's current approach -- ie, create the union of some smallish number of coarse brackets. Some of the brackets will fall entirely within the requested range, and so require no further filtering, while others will fall part inside / part outside of the requested range, and so will require precise filtering. If NRQ did this, it should have much fewer postings to enum, at the cost of having to do precise filtering on some of them (and we'd have to somehow encode the orig value in the index). Mike On Tue, Dec 29, 2009 at 8:42 PM, Yonik Seeley yo...@lucidimagination.com wrote: On Tue, Dec 29, 2009 at 7:13 PM, Marvin Humphrey mar...@rectangular.com wrote: ... but for this algorithm, different rasterization resolutions need not proceed by powers-of-two. Indeed - one way to further generalize would be to use something like Lucene's trie-based Numeric field, but with a square instead of a line. That would allow to tweak the space/speed tradeoff. -Yonik http://www.lucidimagination.com
Re: [spatial] Cartesian Tiers nomenclature
Marvin Humphrey wrote: [ Click on Package org.apache.lucene.spatial.tier ] Lots of red text -- guess they're serious about this not being a stable API. We put all that red in after realizing all the other issues existed - lest someone think this was a polished, finished contrib. When we were about to release 2.9, Spatial was in a brain dump format of its own. -- - Mark http://www.lucidimagination.com
Re: [spatial] Cartesian Tiers nomenclature
Actually that's just with the DistanceQueryBuilder that needs the precision boolean (wheither to do just a bounding box or also filter on radius), the original name of the project was locallucene, to me local and spatial are 2 completely different things. - Local is a search from point x within y miles. (hence the radius filter) - Spatial on the other hand is well everything else. I have yet to update the repository with that code, but a working example of what I see spatial as is http://www.knowledgenowsystems.com/polySpatialSolrUI/ The polygon tool works better than the line tool, you have to double click the link to get it to kick off a search. But essentially it's capable of taking almost arbitrary shapes and just doing convex hull searching or line string searching. Future work would include sorting based upon proximity to a point, so you don't just get a bag of results back, you can say starting at point x give me all the closest ones along this line or within this shape (trajectory sort? in my mind), then an edge proximity search would be closest to this line / vertex. After that I think determining sub-shapes, where your search vertexes cross each other, determine the point of intersection and create smaller shapes from the pieces around it. There's a whole bunch more i plan on getting in there. Including a new SRID based on beer mat diagrams with Ryan on having a prefixed id, that doesn't require additional fields, and can stop descending once a shape has been fully enveloped, reducing the number of terms within an index. On Wed, Dec 30, 2009 at 4:01 AM, Michael McCandless luc...@mikemccandless.com wrote: Right, NRQ is able to translate any requested range into the union (OR) of brackets (from the trie) created during indexing. Can spatial do the same thing, just with 2D instead of 1D? Ie, reconstruct any expressible shape (created at query time) as the union of some number of grids/tiers, at finer finer levels, created during indexing? Spatial, today, seems to do this, except it must also do precise filtering on each matching doc, because some of the grids may contain hits outside of the requested shape. In fact, NRQ could also borrow from spatial's current approach -- ie, create the union of some smallish number of coarse brackets. Some of the brackets will fall entirely within the requested range, and so require no further filtering, while others will fall part inside / part outside of the requested range, and so will require precise filtering. If NRQ did this, it should have much fewer postings to enum, at the cost of having to do precise filtering on some of them (and we'd have to somehow encode the orig value in the index). Mike On Tue, Dec 29, 2009 at 8:42 PM, Yonik Seeley yo...@lucidimagination.com wrote: On Tue, Dec 29, 2009 at 7:13 PM, Marvin Humphrey mar...@rectangular.com wrote: ... but for this algorithm, different rasterization resolutions need not proceed by powers-of-two. Indeed - one way to further generalize would be to use something like Lucene's trie-based Numeric field, but with a square instead of a line. That would allow to tweak the space/speed tradeoff. -Yonik http://www.lucidimagination.com
RE: [spatial] Cartesian Tiers nomenclature
Hi Mike, Right, NRQ is able to translate any requested range into the union (OR) of brackets (from the trie) created during indexing. Can spatial do the same thing, just with 2D instead of 1D? Ie, reconstruct any expressible shape (created at query time) as the union of some number of grids/tiers, at finer finer levels, created during indexing? Spatial, today, seems to do this, except it must also do precise filtering on each matching doc, because some of the grids may contain hits outside of the requested shape. The problem in 2D is, that the number of small cart tiers (rect tiles) at the corners of the BBOX can get very large. For NRQ it's very limited on precisionStep, but here, the number of tiles can get very large for highest precision (you can draw a picture to see this). So the number of terms will get large, too, and so you can simply use NRQ in two dimensions to achieve the same for non quadratic shaped bounding boxes. I tried to use spatial tiers for large and non quadratic bounding boxes (like all datasets around Africa). NRQ performed much better. Spatial is good for things like find all McBurger around position xy with distance d, but not for bounding box searches. Because of this the precision is limited for tiers, you filter the rest of the docs. If the precision would be higher, the number of cart tiers explodes at the corner of the BBOX. In fact, NRQ could also borrow from spatial's current approach -- ie, create the union of some smallish number of coarse brackets. Some of the brackets will fall entirely within the requested range, and so require no further filtering, while others will fall part inside / part outside of the requested range, and so will require precise filtering. If NRQ did this, it should have much fewer postings to enum, at the cost of having to do precise filtering on some of them (and we'd have to somehow encode the orig value in the index). We thought about that for NRQ, too. It is still needed that you store the full precision in the index. The idea is to use flexible indexing attributes and/or payloads for this. You select the larger brackets and then go through TermPositions and only enumerate docs, which payload/flexindex attribute match. Another idea was provided by Yonik who said, the position could be used instead of payload for the missing bits. The posincr is currently not used by NRQ (its 1 for full prec and 0 for lower prec, so all trie terms have pos=1). Mike On Tue, Dec 29, 2009 at 8:42 PM, Yonik Seeley yo...@lucidimagination.com wrote: On Tue, Dec 29, 2009 at 7:13 PM, Marvin Humphrey mar...@rectangular.com wrote: ... but for this algorithm, different rasterization resolutions need not proceed by powers-of-two. Indeed - one way to further generalize would be to use something like Lucene's trie-based Numeric field, but with a square instead of a line. That would allow to tweak the space/speed tradeoff. -Yonik http://www.lucidimagination.com
Re: [spatial] Cartesian Tiers nomenclature
Because of this the precision is limited for tiers, you filter the rest of the docs. If the precision would be higher, the number of cart tiers explodes at the corner of the BBOX. Hmm, depends on the order of the search, if you use bestFit is should not explode. The strategy for bestFit is to find the smallest number of boxes between the corner of the bounding box and the circle. CartesianTierPlotter.bestFit(double miles) /** * Find the tier with the best fit for a bounding box * Best fit is defined as the ceiling of * log2 (circumference of earth / distance) * distance is defined as the smallest box fitting * the corner between a radius and a bounding box. Now in the new structure which include polygon searching you supply vertex points, and cartesian tiers fill in the rest, if it's a convex hull it will switch to range searches, if a line string it simply pulls the individual boxes. So it removes the need for a regular box shape. On Wed, Dec 30, 2009 at 10:27 AM, Uwe Schindler u...@thetaphi.de wrote: Hi Mike, Right, NRQ is able to translate any requested range into the union (OR) of brackets (from the trie) created during indexing. Can spatial do the same thing, just with 2D instead of 1D? Ie, reconstruct any expressible shape (created at query time) as the union of some number of grids/tiers, at finer finer levels, created during indexing? Spatial, today, seems to do this, except it must also do precise filtering on each matching doc, because some of the grids may contain hits outside of the requested shape. The problem in 2D is, that the number of small cart tiers (rect tiles) at the corners of the BBOX can get very large. For NRQ it's very limited on precisionStep, but here, the number of tiles can get very large for highest precision (you can draw a picture to see this). So the number of terms will get large, too, and so you can simply use NRQ in two dimensions to achieve the same for non quadratic shaped bounding boxes. I tried to use spatial tiers for large and non quadratic bounding boxes (like all datasets around Africa). NRQ performed much better. Spatial is good for things like find all McBurger around position xy with distance d, but not for bounding box searches. Because of this the precision is limited for tiers, you filter the rest of the docs. If the precision would be higher, the number of cart tiers explodes at the corner of the BBOX. In fact, NRQ could also borrow from spatial's current approach -- ie, create the union of some smallish number of coarse brackets. Some of the brackets will fall entirely within the requested range, and so require no further filtering, while others will fall part inside / part outside of the requested range, and so will require precise filtering. If NRQ did this, it should have much fewer postings to enum, at the cost of having to do precise filtering on some of them (and we'd have to somehow encode the orig value in the index). We thought about that for NRQ, too. It is still needed that you store the full precision in the index. The idea is to use flexible indexing attributes and/or payloads for this. You select the larger brackets and then go through TermPositions and only enumerate docs, which payload/flexindex attribute match. Another idea was provided by Yonik who said, the position could be used instead of payload for the missing bits. The posincr is currently not used by NRQ (its 1 for full prec and 0 for lower prec, so all trie terms have pos=1). Mike On Tue, Dec 29, 2009 at 8:42 PM, Yonik Seeley yo...@lucidimagination.com wrote: On Tue, Dec 29, 2009 at 7:13 PM, Marvin Humphrey mar...@rectangular.com wrote: ... but for this algorithm, different rasterization resolutions need not proceed by powers-of-two. Indeed - one way to further generalize would be to use something like Lucene's trie-based Numeric field, but with a square instead of a line. That would allow to tweak the space/speed tradeoff. -Yonik http://www.lucidimagination.com
Re: [spatial] Cartesian Tiers nomenclature
It's great that there's such a sudden burst of energy to improve spatial in both Solr and Lucene! Isn't this concept the same as trie (for Lucene's numeric fields), but in 2D not 1D? If so, I think tiles doesn't convey that they recursively subdivide. Also: why does this notion even need naming so badly? Why does this concept leak out of the abstraction? Shouldn't this (cartesian tier, cartesian tier plotter) all be under the hood? I make a SpatialField, I index it, I can then make SpatialShapeQuery, a SpatialDistanceSort, etc.? Ie, trie is known within Lucene, but doesn't leak out -- the outside world knows it as Numeric*. Trie is an implementation detail, inside Lucene. (NOTE: I only know just enough about spatial to be dangerous...) Mike On Tue, Dec 29, 2009 at 2:49 AM, patrick o'leary pj...@pjaol.com wrote: Ah the language of math is the ultimate lingua franca - Nice ! When you look at the coordinates entity from KML, ask why are the lat / longs reversed to long/ lat? Answer because the folks working on the display thought in terms of *display not GIS*, the point is over Y degrees of longitude and down X degrees of latitude. But again that's not a convention used outside a little part of GeoTools or KML, GML / GeoRSS are again just the regular lat,long (NS,EW), or projected EPSG or other standard projections in OGC 05-011. To my knowledge google are the only real pushers of (EW,NS) these days. So what does this diatribe mean? We're kind of at the bleeding edge of defining the standard, hence the difficulty of finding data on it. This is one reason why locallucene and localsolr became popular, it solved a problem simply. Doc's about it exist on gissearch.com dzone are doing articles on it http://java.dzone.com/articles/spatial-search-hibernate?utm_source=feedburnerutm_medium=feedutm_campaign=Feed%3A+javalobby%2Ffrontpage+%28Javalobby+%2F+Java+Zone%29 Locallucene in google has over 8,000 results http://www.google.com/search?q=locallucene Localsolr has over 4,000 results http://www.google.com/search?q=localsolr I've seen and help with installations all over the place, heck even codehaus use it, as do folks on github with geonames db. I see named it mathematically scientifically correct, and gaining enough traction and popularity to start becoming part of the standard, not just duplicating one. I can't honestly see how a refactoring is bringing anything positive to this, when there isn't a good standard out there yet. On Mon, Dec 28, 2009 at 10:22 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Patrick, Interesting. It seems like there is a precedent already in the Local Lucene and Local SOLR packages that define CartesianTier as lingua franca. Like I said in an earlier email it depends on who you talk to regarding the preference of what to call these Tiles/Grids/Tiers, etc., and that seems to be further evidenced by your research. I for one don¹t really have a preference but precedent matters to me and if Tiers have been used to date then there should be strong consideration to use that nomenclature and +1 from me. Cheers, Chris On 12/28/09 9:25 PM, patrick o'leary pj...@pjaol.com wrote: So trying no to drag this out, the most frequent generic term used in GIS software is SRID http://en.wikipedia.org/wiki/SRID Again this provides just a basic nomenclature for the high level element, somewhat the blackbird of objects rather than the defining the magpie (sorry for the CS 101 reference) But it should show that every implementation is unique in some format. Perhaps as unique as CartesianTier's ( sorry Ted ! ) On Mon, Dec 28, 2009 at 5:26 PM, patrick o'leary pj...@pjaol.com wrote: Hmm, depends, tiles indicate to me a direct correlation between the id and a map tile, which will depend upon using the right projection with the cartesian plotter On Mon, Dec 28, 2009 at 2:56 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 4:19 PM, patrick o'leary wrote: Hmm, but when you say grid, to me that's just a bunch of regularly spaced lines.. Yeah, I hear you. I chose spatial tiles for the Solr patch, but spatial grid would work too. Or map tiles/map grids. That anchors it into the spatial world, since we're calling Lucene's spatial contrib/spatial and Solr's Solr Spatial. On Mon, Dec 28, 2009 at 1:16 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 3:51 PM, patrick o'leary wrote: So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've
Re: [spatial] Cartesian Tiers nomenclature
On Dec 29, 2009, at 6:28 AM, Michael McCandless wrote: It's great that there's such a sudden burst of energy to improve spatial in both Solr and Lucene! Isn't this concept the same as trie (for Lucene's numeric fields), but in 2D not 1D? If so, I think tiles doesn't convey that they recursively subdivide. Also: why does this notion even need naming so badly? Why does this concept leak out of the abstraction? Shouldn't this (cartesian tier, cartesian tier plotter) all be under the hood? I make a SpatialField, I index it, I can then make SpatialShapeQuery, a SpatialDistanceSort, etc.? To some extent, but there are tradeoffs between the Lucene numerics and the tiles/tiers in terms of performance, etc. such that one at least needs to understand some high level notion of what the data structure is in order to make an informed choice about when to use it. Ie, trie is known within Lucene, but doesn't leak out -- the outside world knows it as Numeric*. Trie is an implementation detail, inside Lucene. Not entirely true, in Solr, it's called a Trie*Field (TrieDoubleField, TrieIntField, etc.) as numeric is too generic since Solr has other numerics. (NOTE: I only know just enough about spatial to be dangerous...) Mike On Tue, Dec 29, 2009 at 2:49 AM, patrick o'leary pj...@pjaol.com wrote: Ah the language of math is the ultimate lingua franca - Nice ! When you look at the coordinates entity from KML, ask why are the lat / longs reversed to long/ lat? Answer because the folks working on the display thought in terms of *display not GIS*, the point is over Y degrees of longitude and down X degrees of latitude. But again that's not a convention used outside a little part of GeoTools or KML, GML / GeoRSS are again just the regular lat,long (NS,EW), or projected EPSG or other standard projections in OGC 05-011. To my knowledge google are the only real pushers of (EW,NS) these days. So what does this diatribe mean? We're kind of at the bleeding edge of defining the standard, hence the difficulty of finding data on it. This is one reason why locallucene and localsolr became popular, it solved a problem simply. Doc's about it exist on gissearch.com dzone are doing articles on it http://java.dzone.com/articles/spatial-search-hibernate?utm_source=feedburnerutm_medium=feedutm_campaign=Feed%3A+javalobby%2Ffrontpage+%28Javalobby+%2F+Java+Zone%29 Locallucene in google has over 8,000 results http://www.google.com/search?q=locallucene Localsolr has over 4,000 results http://www.google.com/search?q=localsolr I've seen and help with installations all over the place, heck even codehaus use it, as do folks on github with geonames db. I see named it mathematically scientifically correct, and gaining enough traction and popularity to start becoming part of the standard, not just duplicating one. I can't honestly see how a refactoring is bringing anything positive to this, when there isn't a good standard out there yet. On Mon, Dec 28, 2009 at 10:22 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Patrick, Interesting. It seems like there is a precedent already in the Local Lucene and Local SOLR packages that define CartesianTier as lingua franca. Like I said in an earlier email it depends on who you talk to regarding the preference of what to call these Tiles/Grids/Tiers, etc., and that seems to be further evidenced by your research. I for one don¹t really have a preference but precedent matters to me and if Tiers have been used to date then there should be strong consideration to use that nomenclature and +1 from me. Cheers, Chris On 12/28/09 9:25 PM, patrick o'leary pj...@pjaol.com wrote: So trying no to drag this out, the most frequent generic term used in GIS software is SRID http://en.wikipedia.org/wiki/SRID Again this provides just a basic nomenclature for the high level element, somewhat the blackbird of objects rather than the defining the magpie (sorry for the CS 101 reference) But it should show that every implementation is unique in some format. Perhaps as unique as CartesianTier's ( sorry Ted ! ) On Mon, Dec 28, 2009 at 5:26 PM, patrick o'leary pj...@pjaol.com wrote: Hmm, depends, tiles indicate to me a direct correlation between the id and a map tile, which will depend upon using the right projection with the cartesian plotter On Mon, Dec 28, 2009 at 2:56 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 4:19 PM, patrick o'leary wrote: Hmm, but when you say grid, to me that's just a bunch of regularly spaced lines.. Yeah, I hear you. I chose spatial tiles for the Solr patch, but spatial grid would work too. Or map tiles/map grids. That anchors it into the spatial world, since we're calling Lucene's spatial contrib/spatial and Solr's Solr Spatial. On Mon, Dec 28, 2009 at 1:16 PM, Grant Ingersoll
Re: [spatial] Cartesian Tiers nomenclature
On Dec 29, 2009, at 2:49 AM, patrick o'leary wrote: Doc's about it exist on gissearch.com dzone are doing articles on it http://java.dzone.com/articles/spatial-search-hibernate?utm_source=feedburnerutm_medium=feedutm_campaign=Feed%3A+javalobby%2Ffrontpage+%28Javalobby+%2F+Java+Zone%29 Locallucene in google has over 8,000 results http://www.google.com/search?q=locallucene Localsolr has over 4,000 results http://www.google.com/search?q=localsolr Like I said earlier, they are all self referential. Find me a link that mentions Cartesian Tier that isn't also talking about Local Lucene/Solr. That, of course, is valuable in that some people have already been trained on it. However, now that spatial is a part of Lucene/Solr, I think it's valuable to make sure we are using the terminology that most people are familiar with, not just those who have used LocalLucene/Solr. I've seen and help with installations all over the place, heck even codehaus use it, as do folks on github with geonames db. I see named it mathematically scientifically correct, and gaining enough traction and popularity to start becoming part of the standard, not just duplicating one. But aren't they calling it that b/c LocalLucene called it that? Not saying that makes it wrong, I just want to call it what the majority of other people call it so that we can take advantage of and attract more people to contribute and maintain it. Perhaps there is no standard, which makes it moot, but the fact that I've seen a number of people call it tiles or grids at much larger sites than Lucene makes me think those are better names, not too mention nearly everyone else on this thread. I can't honestly see how a refactoring is bringing anything positive to this, when there isn't a good standard out there yet. It's not about a standard, b/c as you say it doesn't exist. It's about what most people are going to be familiar with. On Mon, Dec 28, 2009 at 10:22 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Patrick, Interesting. It seems like there is a precedent already in the Local Lucene and Local SOLR packages that define CartesianTier as lingua franca. Like I said in an earlier email it depends on who you talk to regarding the preference of what to call these Tiles/Grids/Tiers, etc., and that seems to be further evidenced by your research. I for one don¹t really have a preference but precedent matters to me and if Tiers have been used to date then there should be strong consideration to use that nomenclature and +1 from me. Cheers, Chris On 12/28/09 9:25 PM, patrick o'leary pj...@pjaol.com wrote: So trying no to drag this out, the most frequent generic term used in GIS software is SRID http://en.wikipedia.org/wiki/SRID Again this provides just a basic nomenclature for the high level element, somewhat the blackbird of objects rather than the defining the magpie (sorry for the CS 101 reference) But it should show that every implementation is unique in some format. Perhaps as unique as CartesianTier's ( sorry Ted ! ) On Mon, Dec 28, 2009 at 5:26 PM, patrick o'leary pj...@pjaol.com wrote: Hmm, depends, tiles indicate to me a direct correlation between the id and a map tile, which will depend upon using the right projection with the cartesian plotter On Mon, Dec 28, 2009 at 2:56 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 4:19 PM, patrick o'leary wrote: Hmm, but when you say grid, to me that's just a bunch of regularly spaced lines.. Yeah, I hear you. I chose spatial tiles for the Solr patch, but spatial grid would work too. Or map tiles/map grids. That anchors it into the spatial world, since we're calling Lucene's spatial contrib/spatial and Solr's Solr Spatial. On Mon, Dec 28, 2009 at 1:16 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 3:51 PM, patrick o'leary wrote: So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've talked to they have quad something or other. - To math folks Cartesian levels makes sense. Can't make all the people happy all the time, Right, but as far as I can tell (and I've only done, say an hour of research), I can't find anyone who calls them Cartesian Tiers other than us. Personally, I think web users are the largest group (after all, aren't we all web users?) out there and therefore will be the most familiar with either grid or tile. FWIW, I have tentatively called the Solr FieldType to support this SpatialTileField as in it represents a tile in
Re: [spatial] Cartesian Tiers nomenclature
On Tue, Dec 29, 2009 at 08:54:21AM -0800, patrick o'leary wrote: But at the same time, the rational behind finding a name that most folks are familiar with, is kind of like sales / marketing talk You probably mean that to be derogatory, but it's related and not in a bad way. This is about effective communication, which marketing people understand. In addition to working on Lucy, I'm the primary author of KinoSearch, a loose port of Lucene to Perl/C. KinoSearch's main index writer class used to be called InvIndexer, partly to highlight the fact that it worked with inverted indexes, partly to lessen the overload burden on the word index, and partly because it was easy to say. And some people actually like the term and have adopted it: http://www.rectangular.com/pipermail/kinosearch/2009-December/007164.html invindex? Interesting choice. I always thought it was a good word, but I moved away from that because people didn't grok it right away. I always like the word. So I stole it. :) It is unambiguous, even if people do not know what it means right away. So, it worked as intended within a small, elite group. But let me tell you, it has mostly been a pain in the neck. I've trained people on KinoSearch, I've given talks, etc, and the specialized terminology has just slowed most people down. It has annoyed more than it has clarified. Since that class's name has switched to Indexer, things have gotten easier. People remember that class easily, it rolls off the tongue. They learn it quickly, and there are no pauses while they try to recall jargon. I'm very pleased with how the change has worked out. This is open source, not a commercial entity that needs to have features that can fit on a brochure. As an open source author, I take great pride crafting intutive, simple APIs, and in communicating effectively in general. At the micro-level, that includes writing good email, good documentation, good comments... always using all communication channels to the best advantage. (For instance by aggressively pruning quoted material and rewrapping it so that may even be clearer than in the poorly formatted original). At the high level, that means designing good class hierarchies that people can navigate easily. Using intuitive names -- so long as they are accurate -- is an important part of that. If CartesianTier more accurately describes the class than alternatives such as CartesianGrid or CartesianTile, please make that case. I have to say, I'm a little confused. I assumed that a CartesianTier corresponded to a single zoom level: one tier. CartesianGrid seemed like an equivalent, using more popular terminology. But in other parts of this conversation, people have made references to trees. This isn't an R-Tree implementation, is it? Please pick a name that will make it as easy as possible for me to understand. As a consumer of your API, I will thank you for working hard at the design stage to save me time and effort down the road. Cheers, Marvin Humphrey
Re: [spatial] Cartesian Tiers nomenclature
You probably mean that to be derogatory, but it's related and not in a bad way. This is about effective communication, which marketing people understand. Hence the term spatial-luence, or as it was originally called locallucene- We are discussing an internal component, where folks want to change the name of the methodology of how it works. To me it's like renaming the crank shaft in a car to the Spinning Wheel Turner. If CartesianTier more accurately describes the class than alternatives such as CartesianGrid or CartesianTile, please make that case. CartesianTier's adequately describes what the design does- Layer one cartesian coordinate system on top of another It's not a grid system, grids describe the bounding lines - where a point is x,y : x1,y1, the intersection of 2 grid lines. Cartesian Tiles ? again a web mapping concept ... that's dropping the concept of tiers. A single cartesian tier on it's own is of limited use. As a consumer of your API, I will thank you for working hard at the design stage to save me time and effort down the road. I appreciate your thanks and ask for your patience in following the train of thought . What effort do you have down the road, and how does the name of it create problems for you? If I were calling this PatrickGeoSolutions or MagicalGeoStuff then I would understand the problem, What I'm doing is calling it by what it's doing generating a Cartesian Tiered system could I put the word coordinate in there? Sure why not, CartesianCoordinateTiers ? But is that really worth breaking all the existing references to this? What value is that for the users? On Tue, Dec 29, 2009 at 10:32 AM, Marvin Humphrey mar...@rectangular.comwrote: On Tue, Dec 29, 2009 at 08:54:21AM -0800, patrick o'leary wrote: But at the same time, the rational behind finding a name that most folks are familiar with, is kind of like sales / marketing talk You probably mean that to be derogatory, but it's related and not in a bad way. This is about effective communication, which marketing people understand. In addition to working on Lucy, I'm the primary author of KinoSearch, a loose port of Lucene to Perl/C. KinoSearch's main index writer class used to be called InvIndexer, partly to highlight the fact that it worked with inverted indexes, partly to lessen the overload burden on the word index, and partly because it was easy to say. And some people actually like the term and have adopted it: http://www.rectangular.com/pipermail/kinosearch/2009-December/007164.html invindex? Interesting choice. I always thought it was a good word, but I moved away from that because people didn't grok it right away. I always like the word. So I stole it. :) It is unambiguous, even if people do not know what it means right away. So, it worked as intended within a small, elite group. But let me tell you, it has mostly been a pain in the neck. I've trained people on KinoSearch, I've given talks, etc, and the specialized terminology has just slowed most people down. It has annoyed more than it has clarified. Since that class's name has switched to Indexer, things have gotten easier. People remember that class easily, it rolls off the tongue. They learn it quickly, and there are no pauses while they try to recall jargon. I'm very pleased with how the change has worked out. This is open source, not a commercial entity that needs to have features that can fit on a brochure. As an open source author, I take great pride crafting intutive, simple APIs, and in communicating effectively in general. At the micro-level, that includes writing good email, good documentation, good comments... always using all communication channels to the best advantage. (For instance by aggressively pruning quoted material and rewrapping it so that may even be clearer than in the poorly formatted original). At the high level, that means designing good class hierarchies that people can navigate easily. Using intuitive names -- so long as they are accurate -- is an important part of that. If CartesianTier more accurately describes the class than alternatives such as CartesianGrid or CartesianTile, please make that case. I have to say, I'm a little confused. I assumed that a CartesianTier corresponded to a single zoom level: one tier. CartesianGrid seemed like an equivalent, using more popular terminology. But in other parts of this conversation, people have made references to trees. This isn't an R-Tree implementation, is it? Please pick a name that will make it as easy as possible for me to understand. As a consumer of your API, I will thank you for working hard at the design stage to save me time and effort down the road. Cheers, Marvin Humphrey
Re: [spatial] Cartesian Tiers nomenclature
patrick o'leary: CartesianTier's adequately describes what the design does- Layer one cartesian coordinate system on top of another So CartesianTier objects actually represent *multiple* tiers? Would CartesianTierSet be more accurate, albeit cumbersome? I'm not suggesting that as an alternative, I'm just trying to understand what the class does. Obviously I can go browse the JavaDocs and read the source code, and I will eventually. In Lucyland, we've adopted a tradition of recording brainlogs while browsing unfamiliar documentation as a form of UI testing -- I'll do one of those later. For the moment, though, I'm your trainee test case. Use the places where I'm confused to help you refine your API. (; Or to confirm to yourself that it is perfect already. ;) It's not a grid system, grids describe the bounding lines - where a point is x,y : x1,y1, the intersection of 2 grid lines. I don't fully understand what you mean, but I think I disagree. It's common to overlay two grids of differing resolutions on top of each other. So if a tier is a zoom level, it seems to me that the word grid conveys that same concept pretty well. Cartesian Tiles ? again a web mapping concept ... that's dropping the concept of tiers. Yes, I think the word tile suggests a single rectangular cell. But it's common to rasterize a surface into multiple tile sets at different zoom levels and then search on tiles as terms. Coming at this from a web guy perspective, I figured we were talking about something like that -- and *not* bounding boxes a la R-trees. A single cartesian tier on it's own is of limited use. Interesting. I guess a zoom level without any geographical data isn't useful, but it still seems like something important that could be encapsulated within an object. In fact that's exactly what the name CartesianTier seemed to suggest, since it's singular. If a single cartesian tier is of limited use, why do you have a class called CartesianTier at all? What effort do you have down the road, and how does the name of it create problems for you? I have to learn how to use your tool, obviously. Presumably you want me to use your tool, too, or you wouldn't have published it. I would like to minimize the effort it takes to learn your tool. Good class names make that easier. If I were calling this PatrickGeoSolutions or MagicalGeoStuff then I would understand the problem, What I'm doing is calling it by what it's doing generating a Cartesian Tiered system All these messages have gone by and I *still* don't understand what a CartesianTier is. I'm artificially preserving my ignorance by avoiding the JavaDocs and source code, but IMO, I ought to have grokked that by now. Or you can blame the user... But is that really worth breaking all the existing references to this? What value is that for the users? Breaking all the references is obviously a negative. You only do such a thing when the gains outweigh the costs. Marvin Humphrey
Re: [spatial] Cartesian Tiers nomenclature
patrick o'leary wrote: CartesianTier's adequately describes what the design does- Layer one cartesian coordinate system on top of another So, you all agree that it is Cartesian and that it evokes the idea of layers, but it's not a grid and it's not tiles; the original author is very attached to the name of tiers, but most others seem to think that it does not really roll of the tongue easily nor is it very commercial. It may also be somewhat confusing : Wikipedia : Tiers, also known as Ultra Checkers, is a complex variant of checkers that allows players to upgrade their pieces beyond kings. It is played on a standard eight by eight checkers board with two opposing players. The game has gained popularity amongst many northeastern collegiate students in the US. Given this, how about CartesianBranes ? Wikipedia again : In theoretical physics, a membrane, brane, or p-brane is a spatially extended mathematical concept that appears in string theory and its relatives (M-theory and brane cosmology) that exists in a static number of dimensions. I find that it rolls better off the tongue, it has a connotation with layers and a mysterious but highly intellectual flavor, it is not limited in its number of dimensions, it extends over time as well as space and should please physicists and mathematicians alike. And the occasional but unavoidable misspellings by future users and forum contributors should provide much enjoyment for years to come.
Re: [spatial] Cartesian Tiers nomenclature
On Dec 29, 2009, at 2:14 PM, patrick o'leary wrote: You probably mean that to be derogatory, but it's related and not in a bad way. This is about effective communication, which marketing people understand. Hence the term spatial-luence, or as it was originally called locallucene- We are discussing an internal component, where folks want to change the name of the methodology of how it works. To me it's like renaming the crank shaft in a car to the Spinning Wheel Turner. I totally understand where cartesian tier comes from and I get the math and all the explanation. It's a perfectly reasonable name and I don't have anything against it as far as the meaning it invokes. My only issue is that I can't find a single reference to that phrase outside of Local Lucene whereas I can find lots of references to the concept under names like: map tiles, map grids, spatial tiles, or just plain tiles/grids. That doesn't mean cartesian tiers isn't a better description and that if Lucene was the first to implement such a thing we wouldn't even be having this conversation. But the fact is, we're not the first and it appears to me like the majority of people out their use grid/tiles to describe this concept, for better or worse.
Re: [spatial] Cartesian Tiers nomenclature
But is that really worth breaking all the existing references to this? What value is that for the users? Just to clarify... your concern is two fold: 1. No term is perfect, Cartesian Tier is as good as any, lets stick with it. 2. There are already references to cartesian tiers (like this thread :), lets stick with it. The counter arguments are: 1. 'Tile' or 'Grid' immediately make more sense to the uninitiated 2. If we are going to pick a term, now is the time. The difference in opinion (as far as I can tell) is that everyone except Patrick is looking at this as a new API, not as an existing one. The spatial bits of lucene/solr, it is now quite different then localsolr, as such, i'm not sure concern about breaking references makes much sense. ryan
Re: [spatial] Cartesian Tiers nomenclature
No just that- We are looking at this API as a web maps tile solution- which it isn't. Spatial - lucene as it was originally proposed was meant to be a tool box of solutions- Where CartesianTiers was one of the tools, as was GeoHash, and anything else that others wanted to contribute. Now we're saying, rename it to make it more attractive to people who are familiar with a web map... Is this really the right thing? If you want that, then I'll happily contribute the code to do that, it gives you full Google Maps Tiles, TMS, and MS Quad Key's, which makes faceted overlays attractive. It's really handy. But again that doesn't solve search for you. And that is the key to this point, the two things (map tiles / grids and cartesian tiers) are mutually independent On Tue, Dec 29, 2009 at 2:21 PM, Ryan McKinley ryan...@gmail.com wrote: But is that really worth breaking all the existing references to this? What value is that for the users? Just to clarify... your concern is two fold: 1. No term is perfect, Cartesian Tier is as good as any, lets stick with it. 2. There are already references to cartesian tiers (like this thread :), lets stick with it. The counter arguments are: 1. 'Tile' or 'Grid' immediately make more sense to the uninitiated 2. If we are going to pick a term, now is the time. The difference in opinion (as far as I can tell) is that everyone except Patrick is looking at this as a new API, not as an existing one. The spatial bits of lucene/solr, it is now quite different then localsolr, as such, i'm not sure concern about breaking references makes much sense. ryan
Re: [spatial] Cartesian Tiers nomenclature
On Dec 29, 2009, at 4:27 PM, patrick o'leary wrote: My only issue is that I can't find a single reference to that phrase outside of Local Lucene whereas I can find lots of references to the concept under names like: map tiles, map grids, spatial tiles, or just plain tiles/grids. That doesn't mean cartesian tiers isn't a better description and that if Lucene was the first to implement such a thing we wouldn't even be having this conversation Then perhaps spatial-lucene isn't the product for you? That is always an option Not really. The community has made a commitment to provide spatial capabilities in Lucene/Solr. It's important the community provide code/documentation that people can understand and easily find other information about the concept. -Grant
Re: [spatial] Cartesian Tiers nomenclature
Not really. The community has made a commitment to provide spatial capabilities in Lucene/Solr. It's important the community provide code/documentation that people can understand and easily find other information about the concept. Even if it's incorrect... I see, well it is Apache licensed, and the license allows you to do as you wish On Tue, Dec 29, 2009 at 2:42 PM, Grant Ingersoll gsi...@gmail.com wrote: On Dec 29, 2009, at 4:27 PM, patrick o'leary wrote: My only issue is that I can't find a single reference to that phrase outside of Local Lucene whereas I can find lots of references to the concept under names like: map tiles, map grids, spatial tiles, or just plain tiles/grids. That doesn't mean cartesian tiers isn't a better description and that if Lucene was the first to implement such a thing we wouldn't even be having this conversation Then perhaps spatial-lucene isn't the product for you? That is always an option Not really. The community has made a commitment to provide spatial capabilities in Lucene/Solr. It's important the community provide code/documentation that people can understand and easily find other information about the concept. -Grant
Re: [spatial] Cartesian Tiers nomenclature
On Tue, Dec 29, 2009 at 12:29:47PM -0800, Marvin Humphrey wrote: In Lucyland, we've adopted a tradition of recording brainlogs while browsing unfamiliar documentation as a form of UI testing -- I'll do one of those later. OK, here's the brainlog I recorded while trying to figure out how spatial contrib works. [ BEGIN BRAINLOG ] [ surf to contrib-spatial Javadocs for Lucene 3.0 ] Support for filtering based upon geographic location. OK, I assume that means we can match a tile and create a posting list for it, then AND the resulting doc id set against other search results. No sample code. Looks like pure reference documentation rather than tutorial style documentation. [ Click on Package org.apache.lucene.spatial.tier ] Lots of red text -- guess they're serious about this not being a stable API. Not clear what to click on next, I'll try DistanceQueryBuilder since Patrick mentioned that. [ Click on Class DistanceQueryBuilder ] Documentation is sparse. The only real meat is in the method documetation: a single sentence plus parameter names. I don't know GeoHashes. I sort of think I understand tierFieldPrefix. (?) And what is needPrecise? Hmm, maybe I really need to go find a tutorial somewhere. Let's try the wiki... [ Go to Lucene Java wiki, search for spatial, get two hits: SpatialLucene, SpatialSearch. ] [ SpatialLucene wiki page ] Hmm, there's a big warning which says refers to content not yet committed... is that true? Nope, LUCENE-1387 is closed, so this wiki page is out of date. Pff, whatever... OK, I see links to Patrick's white paper. Seems like it will probably be heavier than I want. [ SpatialSearch wiki page ] Lots of GeoHash links, should be handy when I try to learn that. And another link to Patrick's whitepaper for the cartesian stuff. I'll try the full text search the Wiki search recommends. [ Search Lucene Java wiki for spatial, this time as full text ] Bleah, the Wiki search's performance sucked: Results 1 - 6 of 6 results out of about 1005 pages. (9.49 seconds). No interesting results. I'm reluctant to look at a PDF white paper, it will probably be too technical. [ google lucene spatial tutorial ] Crap, first hit is an article on Hibernate/Lucene integration. I just want how to use Lucene spatial. Looks like Lucid's got a webinar from Grant, but I don't feel like sitting down for an hour, I just want some frikkin' sample code. [ google lucene spatial ] Blog post at http://sujitpal.blogspot.com/2008/02/spatial-search-with-lucene.html looks promising... Crap, it's not using the spatial contrib. :( OK, there's stuff from Mike McCandless at http://www.manning-sandbox.com/thread.jspa?threadID=35203tstart=0... naturally all the indentation is stripped. :( I'll look anyway... OK, I think I basically follow that despite the sucky formatting, but it's not easy. Guess I'm really stuck reading a white paper. :( [ Surf to Patrick's white paper at http://www.nsshutdown.com/projects/lucene/whitepaper/locallucene_v2.html ] Jeez, that's a lot shorter than I expected. Formatting's all messed up and I see some Unicode replacement character glyphs, guess Patrick's not a web guy ;) ... But it's probably what I want in terms of content and depth. [ read through first section ] Inclusion, reductionism... sure sure, that's easy enough, it's just query optimization like I deal with all the time. But wait, no code samples. Dammit. :( [ read through Boundary Box ] Jeeze, the giant cross approach to finding intersection of lat/lon was actually part of a formal spatial package in the past? [ read through Cartesian Grid ] The formatting for this section is really messed up. OK, *finally* I see that a cartesian tier is in fact a zoom level. And even Patrick uses the word grid extensively. Turns out that algorithmically speaking, local Lucene works almost exactly like I expected it to. I wonder if it's faster to filter by saving the doc ids to a bitset first and filtering off of that, or if it's faster just to use an ANDQuery to join the result set from the matching tiles and the result set for the rest of the query. [ read through Box ID's ] Do I really need to know any of this? Box/Tile ID names are arbitrary. Only the query builder that figures out which boxes match a given geographical constraint needs to know. [ END BRAINLOG ] Conclusion # 1: I don't spend a lot of time immersed in Java culture so maybe I missed something, but there seems to be a dearth of high-quality tutorial-style documentation for spatial contrib. I'll save conclusion #2 for a separate email. Marvin Humphrey
Re: [spatial] Cartesian Tiers nomenclature
On Tue, Dec 29, 2009 at 03:55:22PM -0800, Marvin Humphrey wrote: I'll save conclusion #2 for a separate email. Conclusion # 2 == The concepts used in spatial contrib are easy -- much easier than I'd come to assume, given how drawn out this conversation has gotten. (Projections are hard, but I already grok them, and they play a minor role.) In fact, spatial contrib works almost exactly like I would have expected. Some places I've worked have implemented almost exactly the same algorithms independently. I continue to disagree with Patrick about elevating the word tier. There doesn't seem to be anything special about it. In fact, the tiers in spatial contrib refer to zoom levels which are always powers of two... tier 0: 1x1 = 1 tile tier 1: 2x2 = 4 tiles tier 2: 4x4 = 16 tiles ... ... but for this algorithm, different rasterization resolutions need not proceed by powers-of-two. IMO CartesianGrid is better, as it encapsulates the same data using more popular terminology. You could also say that a CartesianGrid has a resolution zoom level, and that in this implementation it's tied to a power-of-two tier. However, assigning such a meaning to the word tier is arbitrary. I might even suggest ZoomLevel as an alternative to CartesianTier, and bounce the term Rasterization out there for kicks. On the other hand, Cartesian Tile would not be appropriate at all as a substitute for Cartesian Tier. My sense is that tile, box, and cell are all basically the same and refer to a single rectangle, while Tier/Grid/ZoomLevel/etc. are composites encapsulating multiple tiles/boxes/cells. Marvin Humphrey
Re: [spatial] Cartesian Tiers nomenclature
Marvin, then by all means write your own sir On Tue, Dec 29, 2009 at 4:13 PM, Marvin Humphrey mar...@rectangular.comwrote: On Tue, Dec 29, 2009 at 03:55:22PM -0800, Marvin Humphrey wrote: I'll save conclusion #2 for a separate email. Conclusion # 2 == The concepts used in spatial contrib are easy -- much easier than I'd come to assume, given how drawn out this conversation has gotten. (Projections are hard, but I already grok them, and they play a minor role.) In fact, spatial contrib works almost exactly like I would have expected. Some places I've worked have implemented almost exactly the same algorithms independently. I continue to disagree with Patrick about elevating the word tier. There doesn't seem to be anything special about it. In fact, the tiers in spatial contrib refer to zoom levels which are always powers of two... tier 0: 1x1 = 1 tile tier 1: 2x2 = 4 tiles tier 2: 4x4 = 16 tiles ... ... but for this algorithm, different rasterization resolutions need not proceed by powers-of-two. IMO CartesianGrid is better, as it encapsulates the same data using more popular terminology. You could also say that a CartesianGrid has a resolution zoom level, and that in this implementation it's tied to a power-of-two tier. However, assigning such a meaning to the word tier is arbitrary. I might even suggest ZoomLevel as an alternative to CartesianTier, and bounce the term Rasterization out there for kicks. On the other hand, Cartesian Tile would not be appropriate at all as a substitute for Cartesian Tier. My sense is that tile, box, and cell are all basically the same and refer to a single rectangle, while Tier/Grid/ZoomLevel/etc. are composites encapsulating multiple tiles/boxes/cells. Marvin Humphrey
Re: [spatial] Cartesian Tiers nomenclature
On Tue, Dec 29, 2009 at 04:54:06PM -0800, patrick o'leary wrote: Marvin, then by all means write your own sir Well, that's a possibility. Of course it would work with Lucy, not Lucene. One of my motivations for studying contrib-spatial today was to understand how you'd done things so that if in fact publishing a public spacial package for Lucy becomes a priority, we'd have learned from your experiences and start off with a better API than I would have dreamed up myself from scratch. Alternatively, if someone else gets to it before I do, after today's conversation I will be in a better position to iterate with them and offer constructive criticism regarding their contribution. In the mean time, you are free to use the feedback I've contributed here to nip and tuck at the docs for Java Lucene contrib-spatial. Or not. I happen to believe that such reviews are valuable. In Lucyland, we actively solicit them and use them aggressively to refine APIs, documentation, class hierarchies and code. And historically, we have had a lot of success cross-pollinating Lucy and Lucene. Maybe this will be another area of fruitful collaboration. I hope so. Best, Marvin Humphrey
Re: [spatial] Cartesian Tiers nomenclature
On Tue, Dec 29, 2009 at 7:13 PM, Marvin Humphrey mar...@rectangular.com wrote: ... but for this algorithm, different rasterization resolutions need not proceed by powers-of-two. Indeed - one way to further generalize would be to use something like Lucene's trie-based Numeric field, but with a square instead of a line. That would allow to tweak the space/speed tradeoff. -Yonik http://www.lucidimagination.com
[spatial] Cartesian Tiers nomenclature
As some of you may know, I've been working pretty heavily on spatial stuff lately. One of the things that has bothered me for a while is the use of the terminology: cartesian tiers. The thing is, I can't find any reference to such a thing in any place other than Local Lucene and Patrick's white paper on it. Most GIS systems seem to either talk about grids or tiles when describing this capability. Do you think it is worth a name change? This is about to get baked into Solr and I would really prefer we choose names that the rest of the world seems to understand. Thanks, Grant
AW: [spatial] Cartesian Tiers nomenclature
I also know the name Quad Tree or Trie, but not sure, he it is really the same. Uwe Mit einem Mobiltelefon von Sony Ericsson gesendet Originalnachricht Von: Grant Ingersoll gsing...@apache.org Gesendet: An: general@lucene.apache.org Betreff: [spatial] Cartesian Tiers nomenclature As some of you may know, I've been working pretty heavily on spatial stuff lately. One of the things that has bothered me for a while is the use of the terminology: cartesian tiers. The thing is, I can't find any reference to such a thing in any place other than Local Lucene and Patrick's white paper on it. Most GIS systems seem to either talk about grids or tiles when describing this capability. Do you think it is worth a name change? This is about to get baked into Solr and I would really prefer we choose names that the rest of the world seems to understand. Thanks, Grant
Re: [spatial] Cartesian Tiers nomenclature
I would extremely prefer a common well know name instead of Cartensian tiers. While the API is still in flux changing the name is not that much of a deal either. Either grid or tiles is fine for me though while I would prefer the most common of the two - grid seems to be the better choice though. Yet, should we stick to Cartesian?! simon On Mon, Dec 28, 2009 at 5:31 PM, Grant Ingersoll gsing...@apache.org wrote: As some of you may know, I've been working pretty heavily on spatial stuff lately. One of the things that has bothered me for a while is the use of the terminology: cartesian tiers. The thing is, I can't find any reference to such a thing in any place other than Local Lucene and Patrick's white paper on it. Most GIS systems seem to either talk about grids or tiles when describing this capability. Do you think it is worth a name change? This is about to get baked into Solr and I would really prefer we choose names that the rest of the world seems to understand. Thanks, Grant
Re: [spatial] Cartesian Tiers nomenclature
On Mon, Dec 28, 2009 at 11:31:14AM -0500, Grant Ingersoll wrote: As some of you may know, I've been working pretty heavily on spatial stuff lately. Been watching from a distance, glad to see it. :) One of the things that has bothered me for a while is the use of the terminology: cartesian tiers. The thing is, I can't find any reference to such a thing in any place other than Local Lucene and Patrick's white paper on it. Most GIS systems seem to either talk about grids or tiles when describing this capability. I like cartesian tiles. Tiling is a familiar concept for anybody who's worked with Google Maps, and a little web searching tells me that tiles are also a concept in ArcGIS. But I don't think CartesianTile would substitute properly for a CartesianTier, would it? A CartesianTier is like a set of zoom levels on an interactive map So maybe use CartesianGrid for the zoom level, and CartesianTile as a single rectangle at whatever zoom (if such a class is needed)? Anecdotally, as a dabbler but not an expert, I can say that if you talked about cartesian tiles or cartesian grid, I'd grok right away. Cartesian tier would require an explanation. Do you think it is worth a name change? This is about to get baked into Solr and I would really prefer we choose names that the rest of the world seems to understand. If it hasn't been baked in yet, then +1. I do agree that it's important to use names that are already present in the hivemind rather than invent new ones. Been there, done that, got sick of having to explain myself, went back to popular names... Marvin Humphrey
Re: [spatial] Cartesian Tiers nomenclature
On Dec 28, 2009, at 1:00 PM, Ryan McKinley wrote: Do you think it is worth a name change? This is about to get baked into Solr and I would really prefer we choose names that the rest of the world seems to understand. If it hasn't been baked in yet, then +1. I do agree that it's important to use names that are already present in the hivemind rather than invent new ones. Been there, done that, got sick of having to explain myself, went back to popular names... It's semi-baked into Lucene already and people familiar w/ LocalLucene and LocalSolr. Although it is 'semi-baked', the spatial contrib in 2.9 is clearly marked experimental and subject to change. Also, the tier stuff in 2.9 does not have enough substance to stand on its own -- any change will break APIs. If folks think tile (or grid) make more sense, now is an easy time to change. In my book it seems better to use the most common terms, but I can also see the advantage to knowing that if people are talking about cartesian tiers then they are referring to lucene. (That can also be useful to distinguish spatial lucene/solr from LocalLucene/Solr) I'm +1 for Tile Ditto. I'm +1 on tiles.
Re: [spatial] Cartesian Tiers nomenclature
Grant Ingersoll wrote: On Dec 28, 2009, at 1:00 PM, Ryan McKinley wrote: Do you think it is worth a name change? This is about to get baked into Solr and I would really prefer we choose names that the rest of the world seems to understand. If it hasn't been baked in yet, then +1. I do agree that it's important to use names that are already present in the hivemind rather than invent new ones. Been there, done that, got sick of having to explain myself, went back to popular names... It's semi-baked into Lucene already and people familiar w/ LocalLucene and LocalSolr. Although it is 'semi-baked', the spatial contrib in 2.9 is clearly marked experimental and subject to change. Also, the tier stuff in 2.9 does not have enough substance to stand on its own -- any change will break APIs. If folks think tile (or grid) make more sense, now is an easy time to change. In my book it seems better to use the most common terms, but I can also see the advantage to knowing that if people are talking about cartesian tiers then they are referring to lucene. (That can also be useful to distinguish spatial lucene/solr from LocalLucene/Solr) I'm +1 for Tile Ditto. I'm +1 on tiles. It's not so often that I find on this forum a topic to which I can contribute. So as a total outsider, french-speaking, barely acquainted with Lucene and Solr, and who'se closest claim to knowledge in the matter is to have been an interested spectator at the Lucene/Solr presentations at ApacheCON Europe 2009, I vote - for tiles or grid, because they are terms I would search for in association with (geo)spatial stuff, while I would never think of tiers - for Cartesian because it comes from René Descartes, an icon of french-speaking philosophy and rationalism, and because it intuitively connects to the concept of coordinates
Re: [spatial] Cartesian Tiers nomenclature
Most of the atmospheric/climate/earth scientists that I work with refer to these tiers as grid boxes. I think you¹ll find different answers though, depending on who you ask. The scientific community is a bit different that GIS/decision support folks... Chris On 12/28/09 9:49 AM, Ryan McKinley ryan...@gmail.com wrote: I like tile best -- this has a direct mapping to common map caching systems (google/bing/tms/WorldWind) 'Grid' is also good. In OpenLayers, 'grid' is the parent, and tile based variations extend 'grid'. Tier is interesting since it implies various levels, but i think using a more common term is better for a wider audience. Cartesian? The common tiling schemes are all cartesian (planar), however i think much of the same mechanics can be used to to tile spherical space. Consider something like: HEALPix http://healpix.jpl.nasa.gov/ Uwe mentioned Quad Tree or Trie -- the big difference I see is that tiles or grids have sizes that are defined independent of the data. Quadtree, RTree, etc typically resize themselves as data is added. I like Tile, TilePlotter, FindBestTile, etc best. Grid also works, but seems to refer to the whole system rather then the cell. ryan On Dec 28, 2009, at 12:22 PM, Simon Willnauer wrote: I would extremely prefer a common well know name instead of Cartensian tiers. While the API is still in flux changing the name is not that much of a deal either. Either grid or tiles is fine for me though while I would prefer the most common of the two - grid seems to be the better choice though. Yet, should we stick to Cartesian?! simon On Mon, Dec 28, 2009 at 5:31 PM, Grant Ingersoll gsing...@apache.org wrote: As some of you may know, I've been working pretty heavily on spatial stuff lately. One of the things that has bothered me for a while is the use of the terminology: cartesian tiers. The thing is, I can't find any reference to such a thing in any place other than Local Lucene and Patrick's white paper on it. Most GIS systems seem to either talk about grids or tiles when describing this capability. Do you think it is worth a name change? This is about to get baked into Solr and I would really prefer we choose names that the rest of the world seems to understand. Thanks, Grant ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.mattm...@jpl.nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: [spatial] Cartesian Tiers nomenclature
So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've talked to they have quad something or other. - To math folks Cartesian levels makes sense. Can't make all the people happy all the time, Some folks view tiers/tiles as the mind killer, ahh I could quote Dune all day, but anyway's I digress. r-trees are the way forward, nope Hilbert curves are the way, then someday I expect someone to come along with the biometric butt print of sputnik as the ultimate way to map something. None of which, including CartesianTiers are a full solution, but all represent a solution to something that's GIS in nature. Web-ers think in the concept of google maps/zoom levels, chuck a mercator projector in there and you have google maps. I have a TMS, Google, and MS projector all which can fit into this stuff, but I also have more that aren't web map service projectors. I won't give you a vote as obviously I'm bias. semantic naming for standardization is something I do agree with, but I don't feel there is a good standard out there yet. If we pick something that's OGC in nature and makes sense then you have my support But if it's just a google / MS bunch of blogs then I don't think it's worth while. On Mon, Dec 28, 2009 at 9:22 AM, Simon Willnauer simon.willna...@googlemail.com wrote: I would extremely prefer a common well know name instead of Cartensian tiers. While the API is still in flux changing the name is not that much of a deal either. Either grid or tiles is fine for me though while I would prefer the most common of the two - grid seems to be the better choice though. Yet, should we stick to Cartesian?! simon On Mon, Dec 28, 2009 at 5:31 PM, Grant Ingersoll gsing...@apache.org wrote: As some of you may know, I've been working pretty heavily on spatial stuff lately. One of the things that has bothered me for a while is the use of the terminology: cartesian tiers. The thing is, I can't find any reference to such a thing in any place other than Local Lucene and Patrick's white paper on it. Most GIS systems seem to either talk about grids or tiles when describing this capability. Do you think it is worth a name change? This is about to get baked into Solr and I would really prefer we choose names that the rest of the world seems to understand. Thanks, Grant
Re: [spatial] Cartesian Tiers nomenclature
On Dec 28, 2009, at 3:51 PM, patrick o'leary wrote: So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've talked to they have quad something or other. - To math folks Cartesian levels makes sense. Can't make all the people happy all the time, Right, but as far as I can tell (and I've only done, say an hour of research), I can't find anyone who calls them Cartesian Tiers other than us. Personally, I think web users are the largest group (after all, aren't we all web users?) out there and therefore will be the most familiar with either grid or tile. FWIW, I have tentatively called the Solr FieldType to support this SpatialTileField as in it represents a tile in the spatial sense. I'd be fine with SpatialGridField as well (GridField seems a bit too generic).
Re: [spatial] Cartesian Tiers nomenclature
Hmm, but when you say grid, to me that's just a bunch of regularly spaced lines.. On Mon, Dec 28, 2009 at 1:16 PM, Grant Ingersoll gsing...@apache.orgwrote: On Dec 28, 2009, at 3:51 PM, patrick o'leary wrote: So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've talked to they have quad something or other. - To math folks Cartesian levels makes sense. Can't make all the people happy all the time, Right, but as far as I can tell (and I've only done, say an hour of research), I can't find anyone who calls them Cartesian Tiers other than us. Personally, I think web users are the largest group (after all, aren't we all web users?) out there and therefore will be the most familiar with either grid or tile. FWIW, I have tentatively called the Solr FieldType to support this SpatialTileField as in it represents a tile in the spatial sense. I'd be fine with SpatialGridField as well (GridField seems a bit too generic).
Re: [spatial] Cartesian Tiers nomenclature
Hmm, depends, tiles indicate to me a direct correlation between the id and a map tile, which will depend upon using the right projection with the cartesian plotter On Mon, Dec 28, 2009 at 2:56 PM, Grant Ingersoll gsing...@apache.orgwrote: On Dec 28, 2009, at 4:19 PM, patrick o'leary wrote: Hmm, but when you say grid, to me that's just a bunch of regularly spaced lines.. Yeah, I hear you. I chose spatial tiles for the Solr patch, but spatial grid would work too. Or map tiles/map grids. That anchors it into the spatial world, since we're calling Lucene's spatial contrib/spatial and Solr's Solr Spatial. On Mon, Dec 28, 2009 at 1:16 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 3:51 PM, patrick o'leary wrote: So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've talked to they have quad something or other. - To math folks Cartesian levels makes sense. Can't make all the people happy all the time, Right, but as far as I can tell (and I've only done, say an hour of research), I can't find anyone who calls them Cartesian Tiers other than us. Personally, I think web users are the largest group (after all, aren't we all web users?) out there and therefore will be the most familiar with either grid or tile. FWIW, I have tentatively called the Solr FieldType to support this SpatialTileField as in it represents a tile in the spatial sense. I'd be fine with SpatialGridField as well (GridField seems a bit too generic).
Re: [spatial] Cartesian Tiers nomenclature
So trying no to drag this out, the most frequent generic term used in GIS software is SRID http://en.wikipedia.org/wiki/SRID Again this provides just a basic nomenclature for the high level element, somewhat the blackbird of objects rather than the defining the magpie (sorry for the CS 101 reference) But it should show that every implementation is unique in some format. Perhaps as unique as CartesianTier's ( sorry Ted ! ) On Mon, Dec 28, 2009 at 5:26 PM, patrick o'leary pj...@pjaol.com wrote: Hmm, depends, tiles indicate to me a direct correlation between the id and a map tile, which will depend upon using the right projection with the cartesian plotter On Mon, Dec 28, 2009 at 2:56 PM, Grant Ingersoll gsing...@apache.orgwrote: On Dec 28, 2009, at 4:19 PM, patrick o'leary wrote: Hmm, but when you say grid, to me that's just a bunch of regularly spaced lines.. Yeah, I hear you. I chose spatial tiles for the Solr patch, but spatial grid would work too. Or map tiles/map grids. That anchors it into the spatial world, since we're calling Lucene's spatial contrib/spatial and Solr's Solr Spatial. On Mon, Dec 28, 2009 at 1:16 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 3:51 PM, patrick o'leary wrote: So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've talked to they have quad something or other. - To math folks Cartesian levels makes sense. Can't make all the people happy all the time, Right, but as far as I can tell (and I've only done, say an hour of research), I can't find anyone who calls them Cartesian Tiers other than us. Personally, I think web users are the largest group (after all, aren't we all web users?) out there and therefore will be the most familiar with either grid or tile. FWIW, I have tentatively called the Solr FieldType to support this SpatialTileField as in it represents a tile in the spatial sense. I'd be fine with SpatialGridField as well (GridField seems a bit too generic).
Re: [spatial] Cartesian Tiers nomenclature
Hi Patrick, Interesting. It seems like there is a precedent already in the Local Lucene and Local SOLR packages that define CartesianTier as lingua franca. Like I said in an earlier email it depends on who you talk to regarding the preference of what to call these Tiles/Grids/Tiers, etc., and that seems to be further evidenced by your research. I for one don¹t really have a preference but precedent matters to me and if Tiers have been used to date then there should be strong consideration to use that nomenclature and +1 from me. Cheers, Chris On 12/28/09 9:25 PM, patrick o'leary pj...@pjaol.com wrote: So trying no to drag this out, the most frequent generic term used in GIS software is SRID http://en.wikipedia.org/wiki/SRID Again this provides just a basic nomenclature for the high level element, somewhat the blackbird of objects rather than the defining the magpie (sorry for the CS 101 reference) But it should show that every implementation is unique in some format. Perhaps as unique as CartesianTier's ( sorry Ted ! ) On Mon, Dec 28, 2009 at 5:26 PM, patrick o'leary pj...@pjaol.com wrote: Hmm, depends, tiles indicate to me a direct correlation between the id and a map tile, which will depend upon using the right projection with the cartesian plotter On Mon, Dec 28, 2009 at 2:56 PM, Grant Ingersoll gsing...@apache.orgwrote: On Dec 28, 2009, at 4:19 PM, patrick o'leary wrote: Hmm, but when you say grid, to me that's just a bunch of regularly spaced lines.. Yeah, I hear you. I chose spatial tiles for the Solr patch, but spatial grid would work too. Or map tiles/map grids. That anchors it into the spatial world, since we're calling Lucene's spatial contrib/spatial and Solr's Solr Spatial. On Mon, Dec 28, 2009 at 1:16 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 3:51 PM, patrick o'leary wrote: So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've talked to they have quad something or other. - To math folks Cartesian levels makes sense. Can't make all the people happy all the time, Right, but as far as I can tell (and I've only done, say an hour of research), I can't find anyone who calls them Cartesian Tiers other than us. Personally, I think web users are the largest group (after all, aren't we all web users?) out there and therefore will be the most familiar with either grid or tile. FWIW, I have tentatively called the Solr FieldType to support this SpatialTileField as in it represents a tile in the spatial sense. I'd be fine with SpatialGridField as well (GridField seems a bit too generic). ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.mattm...@jpl.nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: [spatial] Cartesian Tiers nomenclature
Ah the language of math is the ultimate lingua franca - Nice ! When you look at the coordinates entity from KML, ask why are the lat / longs reversed to long/ lat? Answer because the folks working on the display thought in terms of *display not GIS*, the point is over Y degrees of longitude and down X degrees of latitude. But again that's not a convention used outside a little part of GeoTools or KML, GML / GeoRSS are again just the regular lat,long (NS,EW), or projected EPSG or other standard projections in OGC 05-011. To my knowledge google are the only real pushers of (EW,NS) these days. So what does this diatribe mean? We're kind of at the bleeding edge of defining the standard, hence the difficulty of finding data on it. This is one reason why locallucene and localsolr became popular, it solved a problem simply. Doc's about it exist on gissearch.com dzone are doing articles on it http://java.dzone.com/articles/spatial-search-hibernate?utm_source=feedburnerutm_medium=feedutm_campaign=Feed%3A+javalobby%2Ffrontpage+%28Javalobby+%2F+Java+Zone%29 Locallucene in google has over 8,000 results http://www.google.com/search?q=locallucene Localsolr has over 4,000 results http://www.google.com/search?q=localsolr I've seen and help with installations all over the place, heck even codehaus use it, as do folks on github with geonames db. I see named it mathematically scientifically correct, and gaining enough traction and popularity to start becoming part of the standard, not just duplicating one. I can't honestly see how a refactoring is bringing anything positive to this, when there isn't a good standard out there yet. On Mon, Dec 28, 2009 at 10:22 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Patrick, Interesting. It seems like there is a precedent already in the Local Lucene and Local SOLR packages that define CartesianTier as lingua franca. Like I said in an earlier email it depends on who you talk to regarding the preference of what to call these Tiles/Grids/Tiers, etc., and that seems to be further evidenced by your research. I for one don¹t really have a preference but precedent matters to me and if Tiers have been used to date then there should be strong consideration to use that nomenclature and +1 from me. Cheers, Chris On 12/28/09 9:25 PM, patrick o'leary pj...@pjaol.com wrote: So trying no to drag this out, the most frequent generic term used in GIS software is SRID http://en.wikipedia.org/wiki/SRID Again this provides just a basic nomenclature for the high level element, somewhat the blackbird of objects rather than the defining the magpie (sorry for the CS 101 reference) But it should show that every implementation is unique in some format. Perhaps as unique as CartesianTier's ( sorry Ted ! ) On Mon, Dec 28, 2009 at 5:26 PM, patrick o'leary pj...@pjaol.com wrote: Hmm, depends, tiles indicate to me a direct correlation between the id and a map tile, which will depend upon using the right projection with the cartesian plotter On Mon, Dec 28, 2009 at 2:56 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 4:19 PM, patrick o'leary wrote: Hmm, but when you say grid, to me that's just a bunch of regularly spaced lines.. Yeah, I hear you. I chose spatial tiles for the Solr patch, but spatial grid would work too. Or map tiles/map grids. That anchors it into the spatial world, since we're calling Lucene's spatial contrib/spatial and Solr's Solr Spatial. On Mon, Dec 28, 2009 at 1:16 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 28, 2009, at 3:51 PM, patrick o'leary wrote: So Grant here's the deal behind the name. Cartesian because it's a simple x.y coordinate system Tier because there are multiple tiers, levels of resolution. If you look at it closer: - To programmers there's a quadtree implementation - To web users who use maps these are grids / tiles. - To GIS experts this is a form of multi-resolution raster-ing. - To astrophysicists these are tiers. - To the MS folks I've talked to they have quad something or other. - To math folks Cartesian levels makes sense. Can't make all the people happy all the time, Right, but as far as I can tell (and I've only done, say an hour of research), I can't find anyone who calls them Cartesian Tiers other than us. Personally, I think web users are the largest group (after all, aren't we all web users?) out there and therefore will be the most familiar with either grid or tile. FWIW, I have tentatively called the Solr FieldType to support this SpatialTileField as in it represents a tile in the spatial sense. I'd be fine with SpatialGridField as well (GridField seems a bit too generic). ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory
