Justin Deoliveira ha scritto:
Andrea Aime wrote:Then I am not sure I can avoid an expensive lookup. If someone gives me a shapefile with no authority in its .prj what can I do?Justin Deoliveira ha scritto:Unfortunately I have run into this issue in a few places, and the only time the answer is to do a full lookup. I always to try to set fullScan to false, but it seems to only work some of the time. I understand the cost of the lookup but I see no other alternative in a case where you *need* an SRS code.Can we perhaps on startup of geoserver force a full lookup and cache it so that we can be 100% sure that lookupEpsgCode with fullScan set to false will return an answer?Nope, that would not help. The fast version works only if your random WKT contains the AUTHORITY specification with the code the wkt is supposed to be, in that case the fast path reads the official code and compares the two, returning the official one if equal
That SRS xstream converter falls back on the WKT representation when anything else fails no? It also means the REST client will have to be ready and parse both syntaxes no?
I was referring to saving out the WKT of the CRS, rather that first looking up the epsg code and then saving that out.I could amend the code to use the WKT, but that would be a bit of a pain. It will also be problematic with rest, because an EPSG code is needed in order to get at data. Making the client parse WKT will be problematic.I've lost you there. The SRS is a string and is stored so should be always available anyways, what's taking long is trying to represent the native CRS as an epsg code (at least, that is my guess... I'll check better). No OGC request uses the native CRS no?
You said: > I could amend the code to use the WKT, but that would be a bit of a > pain. It will also be problematic with rest, because an EPSG code is > needed in order to get at data. Making the client parse WKT will be > problematic. The client will have to be ready and parse WKT anyways, even the extensive lookup can fail, in that case the SRSConverter will use WKT syntax. That's why I was suggesting to just use the WKT converter or at least to avoid the expensive scan. When you say "the EPSG code is needed in order to get at data" I think you mean when using OGC services? But in that case you have to use the declared CRS that is stored in the SRS code, not the native CRS, no?
I am also curious as to under what circumstances a look up can be as expensive as you refer to. A quick test of a lookup with full scan of some random WKT I time at ~800ms on my machine. The non full scan takes ~650ms. While I realize this adds up in a loop that has to iterate many times. Is that where you are running up against it?
Nope, as stated in my first mail, I'm in the order of seconds. The full lookup goes through 3 stages: - fast lookup by authority - quick query against the EPSG database to find a handful of possible candidates (non exaustive list) - full scan against any and all codes contained in the EPSG database The code that I was trying to lookup is a British Columbia one, see the attached code sample. If I try to run that from GeoServer itblows up due to one of our extra custom definitions (which is lacking the authority bit), if you run it inside Geotools, or remove our extra definitions, on my PC, it reports a time of 6 seconds, and then will return nothing, as an equivalent official code cannot be found.
I also made a little test as to whether a successful lookup, even in full scan mode, is a common occurrence or not.
Since I've started working for GeoServer I received quite a bit of
sample data from users, so I have a collection of around 350 .prj files
sitting on my disk. I made a quick app that opens all of them
and reports for each how much time it took and whether the lookup
was successful or not.
To make it quicker I made sure only wkt that were really different
were taken into consideration (but GeoServer won't have this luxury,
the results of a failed lookup are not cached, so if it takes 4
seconds to fail to lookup a code those 4 seconds will be paid over
and over... something I may try to fix in referencing btw).
The results were interesting. Out of 360 prj files I had in fact only 71
different WKT representations, out of which:
32 looked up to a EPSG code successfully
32 failed to lookup but did not throw an exception in the process
7 that did throw an exception in the process
Those that failed to lookup took usually around 3-4 seconds to
report the null epsg code (that is, lookup failure). This is
better than the 6 above because the server VM is hot after
a few lookups.
So if the data set sitting on my disk is any indication, the
SRSConverter against real world data will be forced to return
the WKT representation in more than 50% of the cases.
I also attached the application and my set of .prj files so
that you can double check on your machine, and eventually
try with whatever other set of prjs you have handy
("find . -name "*.prj" -exec cp {} /tmp/prj \;"
is your friend if you want to collect all prjs on your fs
in a single folder)
Also, from what I can see looking in the generated featureType.xml,
the WKT definition is used anyways for the native CRS, what is
forcing the lookup is possibly the native BBOX (did not fully
investigate).
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
import org.geotools.referencing.CRS;
import org.opengis.referencing.crs.CoordinateReferenceSystem;
public class WKTDecoder {
public static void main(String[] args) throws Exception {
String WKT = "PROJCS[\"BC_Albers\",GEOGCS[\"GCS_North_American_1983\",DATUM[\"North_American_Datum_1983\",SPHEROID[\"GRS_1980\",6378137,298.257222101]],PRIMEM[\"Greenwich\",0],UNIT[\"Degree\",0.017453292519943295]],PROJECTION[\"Albers\"],PARAMETER[\"False_Easting\",1000000],PARAMETER[\"False_Northing\",0],PARAMETER[\"Central_Meridian\",-126],PARAMETER[\"Standard_Parallel_1\",50],PARAMETER[\"Standard_Parallel_2\",58.5],PARAMETER[\"Latitude_Of_Origin\",45],UNIT[\"Meter\",1]]";
CoordinateReferenceSystem crs = CRS.parseWKT(WKT);
long start = System.currentTimeMillis();
try {
System.out.println(CRS.lookupEpsgCode(crs, true));
} finally {
long end = System.currentTimeMillis();
System.out.println((end - start) / 1000.0);
}
}
}
prj.tar.bz2
Description: application/bzip
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.util.HashSet;
import java.util.Set;
import org.geotools.referencing.CRS;
import org.opengis.referencing.crs.CoordinateReferenceSystem;
public class FullLookups {
public static void main(String[] args) throws Exception {
// don't want to see the ton of warnings coming out of referencing logging
System.err.close();
// the wkt we already encountered
Set<String> wkts = new HashSet<String>();
int skipped = 0;
int success = 0;
int failed = 0;
int error = 0;
File root = new File("/tmp/prj");
for (File f : root.listFiles()) {
String wkt = readFile(f);
if(wkts.contains(wkt)) {
skipped++;
continue;
}
wkts.add(wkt);
long start = System.currentTimeMillis();
try {
CoordinateReferenceSystem crs = CRS.parseWKT(wkt);
Integer code = CRS.lookupEpsgCode(crs, true);
if(code == null) {
failed++;
} else {
success++;
}
System.out.println(f.getName() + " -> " + code);
} catch(Exception e) {
System.out.println(f.getName() + " error: " + e.getMessage());
error++;
} finally {
long end = System.currentTimeMillis();
System.out.println(f.getName() + " lookup took: " + (end - start) / 1000.0);
}
}
System.out.println(skipped + "," + success + ", " + failed + ", " + error);
}
private static String readFile(File f) throws Exception {
BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(f)));
String line;
StringBuffer sb = new StringBuffer();
while((line = reader.readLine()) != null)
sb.append(line).append("\n");
reader.close();
return sb.toString();
}
}
<featureType>
<id>FeatureTypeInfoImpl-7d5a744c:121ea60e9dc:-7ff5</id>
<name>bc_elections_1996</name>
<nativeName>bc_elections_1996</nativeName>
<namespace>
<id>NamespaceInfoImpl-7d5a744c:121ea60e9dc:-7fff</id>
</namespace>
<nativeCRS class="projected">PROJCS["BC_Albers",
GEOGCS["GCS_North_American_1983",
DATUM["North_American_Datum_1983",
SPHEROID["GRS_1980", 6378137.0, 298.257222101]],
PRIMEM["Greenwich", 0.0],
UNIT["degree", 0.017453292519943295],
AXIS["Longitude", EAST],
AXIS["Latitude", NORTH]],
PROJECTION["Albers_Conic_Equal_Area"],
PARAMETER["central_meridian", -126.0],
PARAMETER["latitude_of_origin", 45.0],
PARAMETER["standard_parallel_1", 50.0],
PARAMETER["false_easting", 1000000.0],
PARAMETER["false_northing", 0.0],
PARAMETER["standard_parallel_2", 58.5],
UNIT["m", 1.0],
AXIS["x", EAST],
AXIS["y", NORTH]]</nativeCRS>
<srs>EPSG:42102</srs>
<nativeBoundingBox>
<minx>276602.7918284227</minx>
<maxx>1870653.718316566</maxx>
<miny>367856.6358828312</miny>
<maxy>1735173.8343551145</maxy>
<crs class="projected">EPSG:42102</crs>
</nativeBoundingBox>
<latLonBoundingBox>
<minx>-139.00265518089313</minx>
<maxx>-110.4282042271735</maxx>
<miny>47.68107592853211</miny>
<maxy>60.593809484854205</maxy>
<crs>EPSG:4326</crs>
</latLonBoundingBox>
<projectionPolicy>FORCE_DECLARED</projectionPolicy>
<enabled>true</enabled>
<store class="dataStore">
<id>DataStoreInfoImpl-7d5a744c:121ea60e9dc:-7ffe</id>
</store>
<attributes>
<attribute>
<name>the_geom</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>ID</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>NAME</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>MLA</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>PARTY</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>NDP</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>LIBERAL</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>REFORM</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>PDA</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>GREEN</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>SOCRED</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>VTOTAL</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>VREJECT</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>VREGIST</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
<attribute>
<name>DATE</name>
<minOccurs>0</minOccurs>
<maxOccurs>1</maxOccurs>
<nillable>true</nillable>
</attribute>
</attributes>
<maxFeatures>0</maxFeatures>
<numDecimals>0</numDecimals>
</featureType>------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
