Re: [UMN_MAPSERVER-USERS] floating point grass rasters and mapserver - a resume
Also, is this work posted somewhere on the web as suggested? I did not
find it in the GRASS wiki, but maybe I have overlooked it. Thanks again.
Neil Best wrote:
Can anyone comment on whether this information is still current? Are there
any plans to accommodate GRASS rasters in Mapserver? Thanks for any info or
tips.
Neil
On Sun, 16 Oct 2005 12:43:44 +0200, Jachym Cepicky
<[EMAIL PROTECTED]> wrote:
very well!
I did it by hand till now (well I do not have so many raster in GRASS
and ... how often do one need to change the color table?)
could you post this script on GRASS-Addons site?
(http://grass.gdf-hannover.de/twiki/bin/view/GRASS/GrassAddOns)
Jáchym
On Sun, Oct 16, 2005 at 12:35:14PM +0200, Andrea Antonello wrote:
Some time ago I was dealing with mapserver getting maps from a GRASS
location.
The maps are all floating point maps and as you (whoever interested) probably
followed in a few mails, the result of most of my maps simply resulted in a
single color layer (in my case it was yellow).
Frank Warmerdam helped me to solve what was solvable and workaround what was
"workaroundable".
The following is an extract and resume of the mails exchange.
Any reply and correction is appreciated.
Alright, the problem was not only one, but a bunch concurring:
1) GRASS novalues are not supported and give problems
Running gdalinfo -mm on the map, we find that part of the problem is that
nodata values are being treated as "nan" (not a number). GDAL and mapserver
do not know how to excluse "nan" nodata values from the min/max calculation
so the autoscaling gets all screwed up.
Here we can see it in the gdalinfo output:
Band 1 Block=385x1 Type=Float64, ColorInterp=Palette
Min=0.000 Max=229172.410 Computed Min/Max=nan,nan
NoData Value=nan
Therefore the min and max have to ge defined and supplied to the map file
with
the following:
PROCESSING SCALE=min max
2) There is a problem with the creation of color ramps in the case of a big
number of not homogeneous distributed values:
Color Table (RGB with 11 entries)
0: 0,0,0,0
1: 255,255,0,255
2: 255,255,0,255
3: 255,255,0,255
...
This is why I had a yellow map. The first 256 values of the color table were
used by MapServer for the after-scaling values. Effectively the first 256
entries of the color table are all shades of yellow.
3) MapServer does not support the GRASS color rules metadata
Metadata:
COLOR_TABLE_RULES_COUNT=5
COLOR_TABLE_RULE_RGB_0=1.00e+00 4.583448e+04 255 255 0 0 255 0
COLOR_TABLE_RULE_RGB_1=4.583448e+04 9.166896e+04 0 255 0 0 255 255
COLOR_TABLE_RULE_RGB_2=9.166896e+04 1.375034e+05 0 255 255 0 0 255
COLOR_TABLE_RULE_RGB_3=1.375034e+05 1.833379e+05 0 0 255 255 0 255
COLOR_TABLE_RULE_RGB_4=1.833379e+05 2.291724e+05 255 0 255 255 0 0
The metadata contains the coloring rules that should be applied. However,
mapserver does not currently know how to use this sort of color rule
metadata. For now the only solution is to recreate classes that are value
similar. One problem though is that MapServer classes don't allow applying a
range of color to a range of values like GRASS (or QGIS). Applying own
CLASSes in the LAYER makes mapserver ignore the color table.
The CLASSes option has two main drawbacks:
a) The CLASS does not allow color ranges. There is no big solution (only
workarounds), if not to wait on the new range based class coloring that Bill
Binko has been working on. An implementation of that is available in 4.6.x
but it is under revision.
b) If we decide to create a set of CLASS rules to get the map drawn "as
if" it
was interpolated, you will need to create a lot of rules, which gives big big
performance problems in drawing the map. This can however be solved with a
PROCESSING command. Here the explanation:
The problem is that the class lookup is done once for each "bucket" in the
lookup table and that by default there are 65536 buckets. It helps a lot to
add the following line in the LAYER definition:
PROCESSING "SCALE_BUCKETS=200"
This basically alters the resolution of the lookup table but substantially
accelerates things.
After some testing on what would be the best to do for now, I'm now letting
the following bash script snippet for automated creation of the raster part
mapfile. It contains the solutions and workarounds reported in this document:
# the path to the grass cellheader file
absolutepath=$mapsetpath/cellhd/$i
echo "LAYER" > rastertmpfile
echo " NAME $i" >> rastertmpfile
echo " TYPE RASTER" >> rastertmpfile
echo " STATUS ON" >> rastertmpfile
echo " DATA \"$absolutepath\"" >> rastertmpfile
# extract the min and max
minmax=`gdalinfo -mm $absolutepath | grep Min | awk -F "=" '{print $2,
$3}' | awk '{print $1, $3}'`
# extract the gdal computed min and max
gdalrange=`gdalinfo -mm $absolutepath | grep "Computed Min" |awk -F
"Computed Min/Max=" '{print $2}' | awk -F "," '{print $1}'`
# if gdal has pro
Re: [UMN_MAPSERVER-USERS] floating point grass rasters and mapserver - a resume
Can anyone comment on whether this information is still current? Are there
any plans to accommodate GRASS rasters in Mapserver? Thanks for any info or
tips.
Neil
On Sun, 16 Oct 2005 12:43:44 +0200, Jachym Cepicky
<[EMAIL PROTECTED]> wrote:
>very well!
>
>I did it by hand till now (well I do not have so many raster in GRASS
>and ... how often do one need to change the color table?)
>
>could you post this script on GRASS-Addons site?
>(http://grass.gdf-hannover.de/twiki/bin/view/GRASS/GrassAddOns)
>
>Jáchym
>
>On Sun, Oct 16, 2005 at 12:35:14PM +0200, Andrea Antonello wrote:
>> Some time ago I was dealing with mapserver getting maps from a GRASS
location.
>> The maps are all floating point maps and as you (whoever interested) probably
>> followed in a few mails, the result of most of my maps simply resulted in a
>> single color layer (in my case it was yellow).
>> Frank Warmerdam helped me to solve what was solvable and workaround what was
>> "workaroundable".
>> The following is an extract and resume of the mails exchange.
>> Any reply and correction is appreciated.
>>
>> Alright, the problem was not only one, but a bunch concurring:
>> 1) GRASS novalues are not supported and give problems
>>
>> Running gdalinfo -mm on the map, we find that part of the problem is that
>> nodata values are being treated as "nan" (not a number). GDAL and mapserver
>> do not know how to excluse "nan" nodata values from the min/max calculation
>> so the autoscaling gets all screwed up.
>>
>> Here we can see it in the gdalinfo output:
>>
>> Band 1 Block=385x1 Type=Float64, ColorInterp=Palette
>> Min=0.000 Max=229172.410 Computed Min/Max=nan,nan
>> NoData Value=nan
>>
>>
>> Therefore the min and max have to ge defined and supplied to the map file
with
>> the following:
>> PROCESSING SCALE=min max
>>
>> 2) There is a problem with the creation of color ramps in the case of a big
>> number of not homogeneous distributed values:
>>
>>
>> Color Table (RGB with 11 entries)
>> 0: 0,0,0,0
>> 1: 255,255,0,255
>> 2: 255,255,0,255
>> 3: 255,255,0,255
>> ...
>>
>>
>> This is why I had a yellow map. The first 256 values of the color table were
>> used by MapServer for the after-scaling values. Effectively the first 256
>> entries of the color table are all shades of yellow.
>>
>> 3) MapServer does not support the GRASS color rules metadata
>>
>>
>> Metadata:
>> COLOR_TABLE_RULES_COUNT=5
>> COLOR_TABLE_RULE_RGB_0=1.00e+00 4.583448e+04 255 255 0 0 255 0
>> COLOR_TABLE_RULE_RGB_1=4.583448e+04 9.166896e+04 0 255 0 0 255 255
>> COLOR_TABLE_RULE_RGB_2=9.166896e+04 1.375034e+05 0 255 255 0 0 255
>> COLOR_TABLE_RULE_RGB_3=1.375034e+05 1.833379e+05 0 0 255 255 0 255
>> COLOR_TABLE_RULE_RGB_4=1.833379e+05 2.291724e+05 255 0 255 255 0 0
>>
>>
>> The metadata contains the coloring rules that should be applied. However,
>> mapserver does not currently know how to use this sort of color rule
>> metadata. For now the only solution is to recreate classes that are value
>> similar. One problem though is that MapServer classes don't allow applying a
>> range of color to a range of values like GRASS (or QGIS). Applying own
>> CLASSes in the LAYER makes mapserver ignore the color table.
>>
>> The CLASSes option has two main drawbacks:
>>
>> a) The CLASS does not allow color ranges. There is no big solution (only
>> workarounds), if not to wait on the new range based class coloring that Bill
>> Binko has been working on. An implementation of that is available in 4.6.x
>> but it is under revision.
>>
>> b) If we decide to create a set of CLASS rules to get the map drawn "as
if" it
>> was interpolated, you will need to create a lot of rules, which gives big big
>> performance problems in drawing the map. This can however be solved with a
>> PROCESSING command. Here the explanation:
>> The problem is that the class lookup is done once for each "bucket" in the
>> lookup table and that by default there are 65536 buckets. It helps a lot to
>> add the following line in the LAYER definition:
>>
>> PROCESSING "SCALE_BUCKETS=200"
>>
>> This basically alters the resolution of the lookup table but substantially
>> accelerates things.
>>
>>
>>
>> After some testing on what would be the best to do for now, I'm now letting
>> the following bash script snippet for automated creation of the raster part
>> mapfile. It contains the solutions and workarounds reported in this document:
>>
>> # the path to the grass cellheader file
>> absolutepath=$mapsetpath/cellhd/$i
>>
>> echo "LAYER" > rastertmpfile
>> echo " NAME $i" >> rastertmpfile
>> echo " TYPE RASTER" >> rastertmpfile
>> echo " STATUS ON" >> rastertmpfile
>> echo " DATA \"$absolutepath\"" >> rastertmpfile
>>
>> # extract the min and max
>> minmax=`gdalinfo -mm $absolutepath | grep Min | awk -F "=" '{print $2,
>> $3}' | awk '{print $1, $3}'`
>> # extract the gdal computed min and max
>> gdalrange=`gdalinfo -mm $ab
Re: [UMN_MAPSERVER-USERS] floating point grass rasters and mapserver - a resume
very well!
I did it by hand till now (well I do not have so many raster in GRASS
and ... how often do one need to change the color table?)
could you post this script on GRASS-Addons site?
(http://grass.gdf-hannover.de/twiki/bin/view/GRASS/GrassAddOns)
Jáchym
On Sun, Oct 16, 2005 at 12:35:14PM +0200, Andrea Antonello wrote:
> Some time ago I was dealing with mapserver getting maps from a GRASS
> location.
> The maps are all floating point maps and as you (whoever interested) probably
> followed in a few mails, the result of most of my maps simply resulted in a
> single color layer (in my case it was yellow).
> Frank Warmerdam helped me to solve what was solvable and workaround what was
> "workaroundable".
> The following is an extract and resume of the mails exchange.
> Any reply and correction is appreciated.
>
> Alright, the problem was not only one, but a bunch concurring:
> 1) GRASS novalues are not supported and give problems
>
> Running gdalinfo -mm on the map, we find that part of the problem is that
> nodata values are being treated as "nan" (not a number). GDAL and mapserver
> do not know how to excluse "nan" nodata values from the min/max calculation
> so the autoscaling gets all screwed up.
>
> Here we can see it in the gdalinfo output:
>
> Band 1 Block=385x1 Type=Float64, ColorInterp=Palette
> Min=0.000 Max=229172.410 Computed Min/Max=nan,nan
> NoData Value=nan
>
>
> Therefore the min and max have to ge defined and supplied to the map file
> with
> the following:
> PROCESSING SCALE=min max
>
> 2) There is a problem with the creation of color ramps in the case of a big
> number of not homogeneous distributed values:
>
>
> Color Table (RGB with 11 entries)
> 0: 0,0,0,0
> 1: 255,255,0,255
> 2: 255,255,0,255
> 3: 255,255,0,255
> ...
>
>
> This is why I had a yellow map. The first 256 values of the color table were
> used by MapServer for the after-scaling values. Effectively the first 256
> entries of the color table are all shades of yellow.
>
> 3) MapServer does not support the GRASS color rules metadata
>
>
> Metadata:
> COLOR_TABLE_RULES_COUNT=5
> COLOR_TABLE_RULE_RGB_0=1.00e+00 4.583448e+04 255 255 0 0 255 0
> COLOR_TABLE_RULE_RGB_1=4.583448e+04 9.166896e+04 0 255 0 0 255 255
> COLOR_TABLE_RULE_RGB_2=9.166896e+04 1.375034e+05 0 255 255 0 0 255
> COLOR_TABLE_RULE_RGB_3=1.375034e+05 1.833379e+05 0 0 255 255 0 255
> COLOR_TABLE_RULE_RGB_4=1.833379e+05 2.291724e+05 255 0 255 255 0 0
>
>
> The metadata contains the coloring rules that should be applied. However,
> mapserver does not currently know how to use this sort of color rule
> metadata. For now the only solution is to recreate classes that are value
> similar. One problem though is that MapServer classes don't allow applying a
> range of color to a range of values like GRASS (or QGIS). Applying own
> CLASSes in the LAYER makes mapserver ignore the color table.
>
> The CLASSes option has two main drawbacks:
>
> a) The CLASS does not allow color ranges. There is no big solution (only
> workarounds), if not to wait on the new range based class coloring that Bill
> Binko has been working on. An implementation of that is available in 4.6.x
> but it is under revision.
>
> b) If we decide to create a set of CLASS rules to get the map drawn "as if"
> it
> was interpolated, you will need to create a lot of rules, which gives big big
> performance problems in drawing the map. This can however be solved with a
> PROCESSING command. Here the explanation:
> The problem is that the class lookup is done once for each "bucket" in the
> lookup table and that by default there are 65536 buckets. It helps a lot to
> add the following line in the LAYER definition:
>
> PROCESSING "SCALE_BUCKETS=200"
>
> This basically alters the resolution of the lookup table but substantially
> accelerates things.
>
>
>
> After some testing on what would be the best to do for now, I'm now letting
> the following bash script snippet for automated creation of the raster part
> mapfile. It contains the solutions and workarounds reported in this document:
>
> # the path to the grass cellheader file
> absolutepath=$mapsetpath/cellhd/$i
>
> echo "LAYER" > rastertmpfile
> echo " NAME $i" >> rastertmpfile
> echo " TYPE RASTER" >> rastertmpfile
> echo " STATUS ON" >> rastertmpfile
> echo " DATA \"$absolutepath\"" >> rastertmpfile
>
> # extract the min and max
> minmax=`gdalinfo -mm $absolutepath | grep Min | awk -F "=" '{print $2,
> $3}' | awk '{print $1, $3}'`
> # extract the gdal computed min and max
> gdalrange=`gdalinfo -mm $absolutepath | grep "Computed Min" |awk -F
> "Computed Min/Max=" '{print $2}' | awk -F "," '{print $1}'`
>
>
> # if gdal has problems with colortables and novalues, it is not able to
> compute the range,
> # therefore we have to supply it and create a set of ad hoc colorrules
> if [ "$gd
[UMN_MAPSERVER-USERS] floating point grass rasters and mapserver - a resume
Some time ago I was dealing with mapserver getting maps from a GRASS location.
The maps are all floating point maps and as you (whoever interested) probably
followed in a few mails, the result of most of my maps simply resulted in a
single color layer (in my case it was yellow).
Frank Warmerdam helped me to solve what was solvable and workaround what was
"workaroundable".
The following is an extract and resume of the mails exchange.
Any reply and correction is appreciated.
Alright, the problem was not only one, but a bunch concurring:
1) GRASS novalues are not supported and give problems
Running gdalinfo -mm on the map, we find that part of the problem is that
nodata values are being treated as "nan" (not a number). GDAL and mapserver
do not know how to excluse "nan" nodata values from the min/max calculation
so the autoscaling gets all screwed up.
Here we can see it in the gdalinfo output:
Band 1 Block=385x1 Type=Float64, ColorInterp=Palette
Min=0.000 Max=229172.410 Computed Min/Max=nan,nan
NoData Value=nan
Therefore the min and max have to ge defined and supplied to the map file with
the following:
PROCESSING SCALE=min max
2) There is a problem with the creation of color ramps in the case of a big
number of not homogeneous distributed values:
Color Table (RGB with 11 entries)
0: 0,0,0,0
1: 255,255,0,255
2: 255,255,0,255
3: 255,255,0,255
...
This is why I had a yellow map. The first 256 values of the color table were
used by MapServer for the after-scaling values. Effectively the first 256
entries of the color table are all shades of yellow.
3) MapServer does not support the GRASS color rules metadata
Metadata:
COLOR_TABLE_RULES_COUNT=5
COLOR_TABLE_RULE_RGB_0=1.00e+00 4.583448e+04 255 255 0 0 255 0
COLOR_TABLE_RULE_RGB_1=4.583448e+04 9.166896e+04 0 255 0 0 255 255
COLOR_TABLE_RULE_RGB_2=9.166896e+04 1.375034e+05 0 255 255 0 0 255
COLOR_TABLE_RULE_RGB_3=1.375034e+05 1.833379e+05 0 0 255 255 0 255
COLOR_TABLE_RULE_RGB_4=1.833379e+05 2.291724e+05 255 0 255 255 0 0
The metadata contains the coloring rules that should be applied. However,
mapserver does not currently know how to use this sort of color rule
metadata. For now the only solution is to recreate classes that are value
similar. One problem though is that MapServer classes don't allow applying a
range of color to a range of values like GRASS (or QGIS). Applying own
CLASSes in the LAYER makes mapserver ignore the color table.
The CLASSes option has two main drawbacks:
a) The CLASS does not allow color ranges. There is no big solution (only
workarounds), if not to wait on the new range based class coloring that Bill
Binko has been working on. An implementation of that is available in 4.6.x
but it is under revision.
b) If we decide to create a set of CLASS rules to get the map drawn "as if" it
was interpolated, you will need to create a lot of rules, which gives big big
performance problems in drawing the map. This can however be solved with a
PROCESSING command. Here the explanation:
The problem is that the class lookup is done once for each "bucket" in the
lookup table and that by default there are 65536 buckets. It helps a lot to
add the following line in the LAYER definition:
PROCESSING "SCALE_BUCKETS=200"
This basically alters the resolution of the lookup table but substantially
accelerates things.
After some testing on what would be the best to do for now, I'm now letting
the following bash script snippet for automated creation of the raster part
mapfile. It contains the solutions and workarounds reported in this document:
# the path to the grass cellheader file
absolutepath=$mapsetpath/cellhd/$i
echo "LAYER" > rastertmpfile
echo " NAME $i" >> rastertmpfile
echo " TYPE RASTER" >> rastertmpfile
echo " STATUS ON" >> rastertmpfile
echo " DATA \"$absolutepath\"" >> rastertmpfile
# extract the min and max
minmax=`gdalinfo -mm $absolutepath | grep Min | awk -F "=" '{print $2,
$3}' | awk '{print $1, $3}'`
# extract the gdal computed min and max
gdalrange=`gdalinfo -mm $absolutepath | grep "Computed Min" |awk -F
"Computed Min/Max=" '{print $2}' | awk -F "," '{print $1}'`
# if gdal has problems with colortables and novalues, it is not able to
compute the range,
# therefore we have to supply it and create a set of ad hoc colorrules
if [ "$gdalrange" == "nan" ]
then
# minmax has the format "123 234", so just extract them
min=`echo $minmax | awk '{print $1}' | awk -F "." '{print $1}'`
max=`echo $minmax | awk '{print $2}' | awk -F "." '{print $1}'`
# define a default colortable, rainbow of grass, but with "bin + 1"
number of rules,
# to get more colors and give the sensation to be interpolated
bins=19
delta=$((($max-$min)/$bins))
if [ $delta -ne 0 ]
then
colors[1]="255 255 0"
