Jeff,
For tiger data I use the counties as that is the way the data is
organized so I have 3300 counties in the US and each county is a
directory with all the layers in that directory.
For Navteq data, I use shp2tile with the quad tree option and limit the
maximum number of entities in a tile to 8000. This seems to work
reasonably well, but I haven't actual tested variations on that to see
what is better.
You also should build mapserver with the debug option in ./configure and
then add DEBUG ON in all you layers and look at the apache error log for
the timing info on the layers. One of the big killers is the labelcache
processing time.
<soapbox>
I have discussed a new algorithm on the dev list regarding the
labelcache processing that should change the it from O(n*(n-1)) to O(n)
complexity problem. I'm hoping I can get someone to champion the
development because when there are a lot of labels the current
labelcache processing can take nearly 50% to the map rendering time.
</soapbox>
If the labelcache is you problem, try to thin the number of objects that
could potentially be labeled at the zoom scale in question. You might
try splitting the shapefile based on road class into major and minor
roads and only labeling the major roads until you are zoomed in even more.
-Steve
Jeff Dege wrote:
In my particular case, I'm only using this layer when zoomed in tightly.
The shapefile in question contains street data. There are millions of
features in the original shapefile. But I have very much smaller
shapefiles containing only the major roads, only the highways, and only
the interestates, that I use at wider zoom levels. It's only at the
tight zooms that I will draw the individual streets, so trying to read
large numbers of tiles is not an issue.
What's surprised me in this case is that it's the first time where only
drawing detail on close-in zooms, and adding spatial indexes to the
shapefiles with shptree wasn't enough to get reasonable performance.
I'm currently running shp2tile with different arguments, trying to see
how speed vs. number of files trade off against one another. But with a
half-gig shapefile, each trial is taking a while.
My current fastest attempt cut the time to draw to 1/4 of what I
originally had. Which isn't fast enough.
-----Original Message-----
From: UMN MapServer Users List
[mailto:[EMAIL PROTECTED] On Behalf Of Ed McNierney
Sent: Monday, April 23, 2007 1:05 PM
To: [email protected]
Subject: Re: [UMN_MAPSERVER-USERS] Attributes in tile indexes?
Jeff -
In addition to Steve's suggestion, please keep in mind that
what you are
trying to do is reduce the amount of data MapServer needs to
read in order
to retrieve the data you need. Think about your data
carefully and see
whether you're achieving that goal.
First, splitting the shapefile into multiple files and
indexing them with a
TILEINDEX helps MapServer select a subset of that data to be
used for a
given map draw. If MapServer has to open most or all of
those shapefiles
anyway, then the TILEINDEX isn't going to do a thing!
Since your method of producing shapefile tiles will minimize
overlap between
files, you should also check the relationship of your map's
spatial extent
with those tiles. Ideally, each tile should be (roughly)
somewhat larger
than the extent of the map view. With rectangularly-tiled
data, you cannot
avoid situations where the map view is centered on a corner
and you need to
open four different input shapefiles. If each rectangular
tile is, however,
slightly larger than the map image size, then you will never
get a situation
that is WORSE than four input files. If you render a zoomed-out view
covering dozens of input files, your TILEINDEX isn't going to
help much.
Think about how your TILEINDEX and split shapefiles are
behaving. Have you
substantially eliminated most of your input data from
consideration in a
map-drawing request? If not, then you need to go back and
think about how
to restructure your tiling and indexing, since that's the
whole goal of
those techniques.
- Ed
--
Ed McNierney
President and Chief Mapmaker
Maps a la carte, Inc. / TopoZone.com
73 Princeton Street, Suite 305
North Chelmsford, MA 01863
Phone: (978) 251-4242
Fax: (978) 251-1396
[EMAIL PROTECTED]
From: Stephen Woodbridge <[EMAIL PROTECTED]>
Reply-To: Stephen Woodbridge <[EMAIL PROTECTED]>
Date: Mon, 23 Apr 2007 11:53:56 -0400
To: <[email protected]>
Subject: Re: [UMN_MAPSERVER-USERS] Attributes in tile indexes?
Jeff,
Make SURE all your tiles have a spatial index *.qix and the
tileindex
also has a spatial index. These can be built with
shptree file.shp
-Steve W.
Jeff Dege wrote:
OK - I knew I had to be doming something stupid.
To tell a layer to draw using a tileindex shapefile, you
include the
commands:
TILEINDEX "tileindex"
TILEITEM "LOCATION"
Using:
DATA "tileindex"
won't work.
Now, after all of that, drawing with the split shapefiles
is slower than
with the original. So there's still some playing to do.
-----Original Message-----
From: UMN MapServer Users List
[mailto:[EMAIL PROTECTED] On Behalf Of Jeff Dege
Sent: Monday, April 23, 2007 9:53 AM
To: [email protected]
Subject: Re: [UMN_MAPSERVER-USERS] Attributes in tile indexes?
Well, I've been looking at this in more detail, and
finding that the
tileindex layer isn't drawing at all. With all other layers
turned off,
when I use shp2img with an extent that causes the tile layer
to draw, I
get a blank image. When I change the layer to use the original
shapefile, I get the data displayed. (And the layer takes 40
seconds to
draw, even after I've run shptree on the shapefile, which
is why I've
been playing around with tiling.)
I identified the individual tile shapefile that contains
the area I'm
trying to draw and used it directly, and it works fine. So the
splitting apart seems to be working. But the tileindex
shapefile seems
to be broken.
-----Original Message-----
From: UMN MapServer Users List
[mailto:[EMAIL PROTECTED] On Behalf Of Stephen
Woodbridge
Sent: Saturday, April 21, 2007 7:15 AM
To: [email protected]
Subject: Re: [UMN_MAPSERVER-USERS] Attributes in tile indexes?
Jeff,
This should just work. There is nothing special you should
have to do. I
do it all the time with my mapfiles. I use shp2tile to break
mine up but
that should not make a difference.
First I would try changing your layer to use just one of
the smaller
shape files without the tileindex and see if that works and
that you can
display the labels.
You might also want to post info about what OS, version of
mapserver,
and the LAYER block in question.
-Steve W
Jeff Dege wrote:
I have a shape file that's too big to draw maps with -
takes far too
long.
In an attempt to speed things up, I split it into 16
smaller shapefiles,
using:
ogr2ogr -f "ESRI Shapefile" -spat xmin ymin xmax
ymax newXX.shp
old.shp
Then did a shptree on each of these new files, then a
shptinidex to
create a single tile index shapefile on them, and tried to
build a map.
It works fine, so long as I don't try to display labels.
But if I do, I
get errors:
msDBFGetItemIndex(): Item 'NAME' not found.
Where "NAME" is the attribute I'm trying to use for the
label. It looks
like the tileindex shapefile doesn't expose the
attributes of the
underlying shape tiles. I've done an ogrinfo on the
shapefiles that I'd
generated, and the attributes are there.
Can I expose the attributes through the tileindex?
How?
