Re: [mapserver-users] dynamic ows_serviceurl
Paul via MapServer-users: > Thanx for the suggestions Kaido, Seth, Steve. Updating the mapfile at > deployment indeed is a better option then the querystring approach, let me > verify what scripting options (sed?) i have on our docker image. If you omit setting the ows_onlineresource (just try to commend it out) in the .map-file, mapserver will dynamically use the path (e.g. domain) it was reached by. Thus you can have the same mapfile on dev, test, qa or production and they will have individual domains. If you reach the server-instance, it will use the server-name (and port) rather than the proxied domain, but then it will be easier to problem shoot individual server instances using plain gis-tools that utilize metadata such as ows_onlineresource. We tested this some years ago, and have the following expereience using a reverse proxy in front of a mapserver cluster: http -> http ows_onlineresource will use http://whatever.domain.reached.by.frontend.com/path/to/mapservice https -> https will use https://whatever.domain.reached.by.frontend.com/path/to/mapservice https -> http (ssl-offloading) will unfortunately use http://blablabla, even if we set forward proto and forwarded port on the proxy-end. It would be nice if mapserver could use an ssl-offloading proxy, be it haproxy, nginx or something else. So, the ows_onlineresource-problem was solved for us by having a fail-over frontend holding the domains, and different mapserver-clusters for load balancing and fail-over - we just had to live with configuring certificates also for the backend servers for the different environments (domains). -- Kind regards Bjørn Ove Grøtan ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] dynamic ows_serviceurl
This is exactly why PHP MapScript has been so popular for so long
("wrappers" of OGC services) and why so many examples of this exists in
the MapServer documentation. Honestly though, someday I also hope for a
more direct way to handle these dynamic changes, but until then,
#mapscriptrocks :)
-jeff
--
Jeff McKenna
GatewayGeo: Developers of MS4W, MapServer Consulting and Training
co-founder of FOSS4G
http://gatewaygeo.com/
On 2022-04-13 5:37 p.m., Paul via MapServer-users wrote:
Thanx for the suggestions Kaido, Seth, Steve. Updating the mapfile at
deployment indeed is a better option then the querystring approach, let
me verify what scripting options (sed?) i have on our docker image.
___
MapServer-users mailing list
MapServer-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] dynamic ows_serviceurl
Thanx for the suggestions Kaido, Seth, Steve. Updating the mapfile at deployment indeed is a better option then the querystring approach, let me verify what scripting options (sed?) i have on our docker image. ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] dynamic ows_serviceurl
Hi Paul,
We are using basically the same process as Seth and Steve mentioned.
We are building map files per environment and per node using Chef.
Basically we parameterized the .map file and fill in parameters for every
node when we deploy.
And because we have multiple map files for different topics we are actually
building multiple map files per node per environment. (buildig map file
from templates - header_template + layers + styles + end_template)
But instead of using Chef it could be done using scripts and replacing
specific keywords with environment parameters.
This way web server won't have to do extra steps per request to change
values
Example parts of our "template" header file:
NAME "sim_<%= @mapname %>_<%=node.chef_environment%>"
STATUS ON
EXTENT <%= node['mapserver']['extent'] %>
FONTSET"<%= node['mapserver']['deploy_to']
%>/fonts/current/fonts.list"
IMAGETYPE "png"
CONFIG "MS_ERRORFILE" "<%=node['mapserver']['log_dir']%>/<%= @mapname
%>.log"
.
WEB
METADATA
"wfs_title" "SiM <%= @mapname %>
<%=node.chef_environment%>"
"wfs_enable_request""*"
"wfs_encoding" "UTF-8"
"wms_title" "SiM <%= @mapname %>
<%=node.chef_environment%>"
"wms_encoding" "UTF-8"
"wms_onlineresource"
"https://<%=node['mapserver']['proxy_url']%>/mapserv?map=/gis/<%=
@mapname %>.map&"
"wfs_onlineresource"
"https://<%=node['mapserver']['proxy_url']%>/mapserv?map=/gis/<%=
@mapname %>.map&"
Kaido
Kontakt Seth G () kirjutas kuupäeval K, 13. aprill
2022 kell 11:21:
> Hi Paul,
>
> The runtime substitution and setting a querystring on the server is a good
> approach - have you found any drawbacks to this? Most web servers can
> set/add querystrings without any need for further software.
>
> As Steve mentioned the alternative would be to generate Mapfiles for
> specific environments from a template.
> I use mappyfile for this, with a Python script that can be run as part of
> continuous deployment:
>
> import mappyfile
> mapfile = mappyfile.open("original.map")
> # update the web metadata settings
> mapfile["web"]["metadata"]["ows_onlineresource"] = "
> https://example.com/cgi-bin/mapserv?map={}.map".format("test")
> mappyfile.save(mapfile, "new.map")
>
> Typical other settings I'd change are connection strings, DEBUG level, and
> EXTENTs for different Mapfiles.
>
> Seth
>
> --
> web:https://geographika.net
> twitter: @geographika
>
>
> On Wed, Apr 13, 2022, at 3:52 AM, Steve Lime wrote:
>
> One other thing. One challenge with environment variables is knowing
> what's consistently available within a runtime environment - CGI is
> different from FastCGI and there are differences across web servers. That's
> one of the reasons pre-processing is kind of attractive IMHO. I suppose the
> v.8 config file could play a role but I'm unsure what that would be beyond
> a place to set values consistently.
>
> On Tue, Apr 12, 2022 at 8:04 PM Steve Lime wrote:
>
> Hi Paul: The idea of referencing environment variables has come up before.
> Another example would be to leverage settings injected into cloud
> deployments as environment variables (I think there's an old ticket that
> was recently re-opened on the topic). I'm not sure what the best way to
> handle this would be. One possibility would be using a mechanism similar to
> runtime subs where you'd reference the variables within the mapfile using
> ${variable} or whatever. Another idea would be to use a pre-processor of
> some sort to "compile" a mapfile from the environment. The latter would
> result in better performance since you'd generate the resulting file once
> and could also do things like inlining included snippets. This
> functionality would be helpful for the cloud deployments, simplifying
> deployments in multiple (dev, test, prod) environments and for keeping
> secrets out of repos altogether.
>
> We're putzing around with the pre-processing idea a bit but haven't gotten
> anything fully baked. Others?
>
> -Steve
>
> On Tue, Apr 12, 2022 at 4:56 AM Paul via MapServer-users <
> mapserver-users@lists.osgeo.org> wrote:
>
> hi list, this has probably been discussed before, but maybe there are
> updates (on v8?)
>
> we would like to extract the ows_onlineresource from the environment,
> instead of having an administrator to configure it on the mapfile
>
> currently we implement a hack where the webserver layer sends an
> additional query parameter `ows_url` to mapserver which contains the
> current url
>
> in the mapfile this is implemented as
>
> ```
>
> WEBVALIDATION "ows_url"
> "(\b(https?|ftp|file)://)?[-A-Za-z0-9+&@#/%?=~_|!:,.;]+[-A-Za-z0-9+&@#/%=~_|]"
> ENDMETADATA "ows_onlineresource" "%ows_url%"
> ENDEND
>
> ```
>
> I wonder if others run into similar challenges, or if there are much more
> simple best practices for this
Re: [mapserver-users] dynamic ows_serviceurl
Hi Paul,
The runtime substitution and setting a querystring on the server is a good
approach - have you found any drawbacks to this? Most web servers can set/add
querystrings without any need for further software.
As Steve mentioned the alternative would be to generate Mapfiles for specific
environments from a template.
I use mappyfile for this, with a Python script that can be run as part of
continuous deployment:
import mappyfile
mapfile = mappyfile.open("original.map")
# update the web metadata settings
mapfile["web"]["metadata"]["ows_onlineresource"] =
"https://example.com/cgi-bin/mapserv?map={}.map".format("test")
mappyfile.save(mapfile, "new.map")
Typical other settings I'd change are connection strings, DEBUG level, and
EXTENTs for different Mapfiles.
Seth
--
web:https://geographika.net
twitter: @geographika
On Wed, Apr 13, 2022, at 3:52 AM, Steve Lime wrote:
> One other thing. One challenge with environment variables is knowing what's
> consistently available within a runtime environment - CGI is different from
> FastCGI and there are differences across web servers. That's one of the
> reasons pre-processing is kind of attractive IMHO. I suppose the v.8 config
> file could play a role but I'm unsure what that would be beyond a place to
> set values consistently.
>
> On Tue, Apr 12, 2022 at 8:04 PM Steve Lime wrote:
>> Hi Paul: The idea of referencing environment variables has come up before.
>> Another example would be to leverage settings injected into cloud
>> deployments as environment variables (I think there's an old ticket that was
>> recently re-opened on the topic). I'm not sure what the best way to handle
>> this would be. One possibility would be using a mechanism similar to runtime
>> subs where you'd reference the variables within the mapfile using
>> ${variable} or whatever. Another idea would be to use a pre-processor of
>> some sort to "compile" a mapfile from the environment. The latter would
>> result in better performance since you'd generate the resulting file once
>> and could also do things like inlining included snippets. This functionality
>> would be helpful for the cloud deployments, simplifying deployments in
>> multiple (dev, test, prod) environments and for keeping secrets out of repos
>> altogether.
>>
>> We're putzing around with the pre-processing idea a bit but haven't gotten
>> anything fully baked. Others?
>>
>> -Steve
>>
>> On Tue, Apr 12, 2022 at 4:56 AM Paul via MapServer-users
>> wrote:
>>> hi list, this has probably been discussed before, but maybe there are
>>> updates (on v8?)
>>>
>>> we would like to extract the `ows_onlineresource from the environment,
>>> instead of having an administrator to configure it on the mapfile`
>>>
>>> currently we implement a hack where the webserver layer sends an additional
>>> query parameter `ows_url` to mapserver which contains the current url
>>>
>>> in the mapfile this is implemented as
>>>
>>> ```
>>> WEB
>>> VALIDATION
>>> "ows_url"
>>> "(\b(https?|ftp|file)://)?[-A-Za-z0-9+&@#/%?=~_|!:,.;]+[-A-Za-z0-9+&@#/%=~_|]"
>>> END
>>>
>>> METADATA
>>> "ows_onlineresource" "%ows_url%"
>>> END
>>> END
>>> ```
>>>
>>> I wonder if others run into similar challenges, or if there are much more
>>> simple best practices for this case.
>>>
>>> I can imagine allowing a environment parameter 'proxy-url' like geoserver
>>> would make sense, but it should allow some kind of template, eg:
>>>
>>> ```
>>> PROXY-URL https://example.com/cgi-bin/mapserv?map={map}.map
>>>
>>> ```
>>>
>>> which in our case would be
>>>
>>> ```
>>> PROXY-URL https://example.com/cgi-bin/{map}
>>>
>>> ```
>>>
>>> Thank you, Paul.
>>> ___
>>> MapServer-users mailing list
>>> MapServer-users@lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/mapserver-users
> ___
> MapServer-users mailing list
> MapServer-users@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-users
>
___
MapServer-users mailing list
MapServer-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] dynamic ows_serviceurl
One other thing. One challenge with environment variables is knowing what's
consistently available within a runtime environment - CGI is different from
FastCGI and there are differences across web servers. That's one of the
reasons pre-processing is kind of attractive IMHO. I suppose the v.8 config
file could play a role but I'm unsure what that would be beyond a place to
set values consistently.
On Tue, Apr 12, 2022 at 8:04 PM Steve Lime wrote:
> Hi Paul: The idea of referencing environment variables has come up before.
> Another example would be to leverage settings injected into cloud
> deployments as environment variables (I think there's an old ticket that
> was recently re-opened on the topic). I'm not sure what the best way to
> handle this would be. One possibility would be using a mechanism similar to
> runtime subs where you'd reference the variables within the mapfile using
> ${variable} or whatever. Another idea would be to use a pre-processor of
> some sort to "compile" a mapfile from the environment. The latter would
> result in better performance since you'd generate the resulting file once
> and could also do things like inlining included snippets. This
> functionality would be helpful for the cloud deployments, simplifying
> deployments in multiple (dev, test, prod) environments and for keeping
> secrets out of repos altogether.
>
> We're putzing around with the pre-processing idea a bit but haven't gotten
> anything fully baked. Others?
>
> -Steve
>
> On Tue, Apr 12, 2022 at 4:56 AM Paul via MapServer-users <
> mapserver-users@lists.osgeo.org> wrote:
>
>> hi list, this has probably been discussed before, but maybe there are
>> updates (on v8?)
>>
>> we would like to extract the ows_onlineresource from the environment,
>> instead of having an administrator to configure it on the mapfile
>>
>> currently we implement a hack where the webserver layer sends an
>> additional query parameter `ows_url` to mapserver which contains the
>> current url
>>
>> in the mapfile this is implemented as
>>
>> ```
>>
>> WEBVALIDATION "ows_url"
>> "(\b(https?|ftp|file)://)?[-A-Za-z0-9+&@#/%?=~_|!:,.;]+[-A-Za-z0-9+&@#/%=~_|]"
>> ENDMETADATA "ows_onlineresource" "%ows_url%"
>> ENDEND
>>
>> ```
>>
>> I wonder if others run into similar challenges, or if there are much more
>> simple best practices for this case.
>>
>> I can imagine allowing a environment parameter 'proxy-url' like geoserver
>> would make sense, but it should allow some kind of template, eg:
>>
>> ```
>> PROXY-URL https://example.com/cgi-bin/mapserv?map={map}.map
>> ```
>>
>> which in our case would be
>>
>> ```
>> PROXY-URL https://example.com/cgi-bin/{map}
>> ```
>>
>> Thank you, Paul.
>> ___
>> MapServer-users mailing list
>> MapServer-users@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapserver-users
>>
>
___
MapServer-users mailing list
MapServer-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] dynamic ows_serviceurl
Hi Paul: The idea of referencing environment variables has come up before.
Another example would be to leverage settings injected into cloud
deployments as environment variables (I think there's an old ticket that
was recently re-opened on the topic). I'm not sure what the best way to
handle this would be. One possibility would be using a mechanism similar to
runtime subs where you'd reference the variables within the mapfile using
${variable} or whatever. Another idea would be to use a pre-processor of
some sort to "compile" a mapfile from the environment. The latter would
result in better performance since you'd generate the resulting file once
and could also do things like inlining included snippets. This
functionality would be helpful for the cloud deployments, simplifying
deployments in multiple (dev, test, prod) environments and for keeping
secrets out of repos altogether.
We're putzing around with the pre-processing idea a bit but haven't gotten
anything fully baked. Others?
-Steve
On Tue, Apr 12, 2022 at 4:56 AM Paul via MapServer-users <
mapserver-users@lists.osgeo.org> wrote:
> hi list, this has probably been discussed before, but maybe there are
> updates (on v8?)
>
> we would like to extract the ows_onlineresource from the environment,
> instead of having an administrator to configure it on the mapfile
>
> currently we implement a hack where the webserver layer sends an
> additional query parameter `ows_url` to mapserver which contains the
> current url
>
> in the mapfile this is implemented as
>
> ```
>
> WEBVALIDATION "ows_url"
> "(\b(https?|ftp|file)://)?[-A-Za-z0-9+&@#/%?=~_|!:,.;]+[-A-Za-z0-9+&@#/%=~_|]"
> ENDMETADATA "ows_onlineresource" "%ows_url%"
> ENDEND
>
> ```
>
> I wonder if others run into similar challenges, or if there are much more
> simple best practices for this case.
>
> I can imagine allowing a environment parameter 'proxy-url' like geoserver
> would make sense, but it should allow some kind of template, eg:
>
> ```
> PROXY-URL https://example.com/cgi-bin/mapserv?map={map}.map
> ```
>
> which in our case would be
>
> ```
> PROXY-URL https://example.com/cgi-bin/{map}
> ```
>
> Thank you, Paul.
> ___
> MapServer-users mailing list
> MapServer-users@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-users
>
___
MapServer-users mailing list
MapServer-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] dynamic ows_serviceurl
hi list, this has probably been discussed before, but maybe there are updates
(on v8?)
we would like to extract the ows_onlineresource from the environment, instead
of having an administrator to configure it on the mapfile
currently we implement a hack where the webserver layer sends an additional
query parameter `ows_url` to mapserver which contains the current url
in the mapfile this is implemented as
```WEB
VALIDATION
"ows_url"
"(\b(https?|ftp|file)://)?[-A-Za-z0-9+&@#/%?=~_|!:,.;]+[-A-Za-z0-9+&@#/%=~_|]"
END
METADATA
"ows_onlineresource" "%ows_url%"
END
END```
I wonder if others run into similar challenges, or if there are much more
simple best practices for this case.
I can imagine allowing a environment parameter 'proxy-url' like geoserver would
make sense, but it should allow some kind of template, eg:
```PROXY-URL https://example.com/cgi-bin/mapserv?map={map}.map```
which in our case would be
```PROXY-URL https://example.com/cgi-bin/{map}```
Thank you, Paul.___
MapServer-users mailing list
MapServer-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users
