Hi guys ... as far as I know we shouldn't use pyramids with WCS at all...
WCS must return original data unless the user requires a specific resolution
or interpolation.
For the WCS limits stuff +1 on my side.
My 2C.
Alessio
-------------------------------------------------------
===
Notice that our office phone number has recently changed!
Please, update your records!
===
Ing. Alessio Fabiani
Founder / CTO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy
phone: (+39) 0584 96.23.13
fax: (+39) 0584 96.23.13
mobile:(+39) 349 82.27.000
http://www.geo-solutions.it
http://geo-solutions.blogspot.com
http://www.linkedin.com/in/alessiofabiani
http://twitter.com/simogeo
-------------------------------------------------------
On Tue, Sep 21, 2010 at 9:20 PM, Gabriel Roldán <[email protected]> wrote:
> On Tue, 2010-09-21 at 13:09 -0400, Freeman, Aleda (EEA) wrote:
> > In MassGIS' case we have rasters loaded in SDE, with pyramids and
> > statistics.
> > I don't think that counts as tiled right?
> every pyramid level in SDE raster is tiled, so yeah, I think we'll be on
> the well performing side of the fence when it comes to data origin.
>
> Cheers,
> Gabriel
> > Hoping that this analysis will cover SDE situation as well?
> >
> >
> > ______________________________________________________________________
> > From: [email protected]
> > [mailto:[email protected]] On Behalf Of
> > Justin Deoliveira
> > Sent: Tuesday, September 21, 2010 12:31 PM
> > To: Andrea Aime
> > Cc: [email protected]
> > Subject: Re: [Geoserver-devel] Implementing WCS limits
> >
> >
> >
> > Obviously i am a novice when it comes to raster stuff but this sounds
> > reasonable. It sounds like the approach is to play it safe by imposing
> > some reasonable limits based on heuristics, but at the same time
> > leaving the admin the ability to change or forgo them in cases where
> > they have ensured the data is properly set up for streaming/tiling.
> >
> > On Tue, Sep 21, 2010 at 8:04 AM, Andrea Aime <[email protected]>
> > wrote:
> > Hi,
> > this week I'm going to try and impleemnt some WCS limits
> > for people that want to serve big amounts of raster data
> > without the headache of a user hitting the server hard with
> > a very large request.
> >
> > Generally speaking we want to avoid that:
> > - the request ends up reading too much data (e.g., we don't
> > want a user to make a request that will make the server
> > read 10TB of data)
> > - the request ends up using too much memory
> > - the request ends up generating a too large response (e.g.,
> > again, we don't want the server to generate a 10GB response)
> >
> > Providing a general solution to the problem can be very
> > complex:
> > * a tiled data source may allow to streaming read by tiles,
> > allowing for small memory usage while still reading a
> > truckload of data
> > * a tiled output format can ensure nowhere in the chain
> > the whole image is composed in memory
> > * however, a non tiled input or a non tiled output will at
> > some point make the image be built fully in memory
> > * where and when is difficult to say, e.g., we might read
> > a small amount of data from the input and then build a
> > huge raster in memory because the user is supersampling
> > (asking a higher than native resolution) and the output
> > format is not tile enabled
> > * the MB read during input and output are again difficult
> > to control because of format and compression differences
> > * the WCS right now does not use overviews, but it might in
> > the future
> > Long story short, trying to control the actual amount of data
> > read, kept in memory and generated in output is beyond our
> > reach.
> >
> > What I'm going to propose is a simplified compromise based
> > on a worst case scenario.
> > We allow the administrator to setup a maximum of MB to be
> > read, and a maximum MB to be generated in output.
> > The measure in MB is computed as an equivalent single tile,
> > uncompressed situation (assuming everything has to be
> > read or generated in one shot):
> > width * height * bands * band_size
> >
> > This simplifying assumption ensures we are not going to
> > ever have more than the limits be read, kept in memory
> > or generated: normally (hopefully) we'll actually have
> > less.
> >
> > The distinction between input and output is there because
> > normally WCS request perform some resampling and generate
> > inputs at a resolution different than the outputs, and
> > in some setups the admin can play on the difference to
> > relax at least the input limits.
> > In particular, if the admin can ensure all rasterv sources
> > are tiled, it's possible to relax the input limits as
> > the tiled sources will never load the full data in memory,
> > and the WCS processing chain ensures that at worst the
> > data is recomposed in a single tile if the output format
> > cannot deal with inner tiling.
> > Even in a setup where all the sources are tiled and the output
> > format list has been somehow modified to allow only tiling
> > formats
> > (atm, only geotiff) it would still be good to setup some
> > (large) limits to avoid disk or network flooding for long
> > amounts
> > of time.
> >
> > Let's make an example. If we set a 200MB of limit as input,
> > and 20MB of limit as output, then following will be considered
> > valid:
> > * a request that makes GS read a 14481x14481 portion of a
> > 8bit single band raster data
> > * a request that makes GS read a 7240x7240 portion of a RGBA
> > (or other 4 band) image
> > * a request that makes GS generate a 4579*4579 8bit raster in
> > output
> > * a request that makes GS generate a 2290*2290 4byte raster in
> > output (RGBA, or a single band, floating point, double
> > precision
> > one).
> >
> > This should give the administrator some control and safety
> > without
> > forcing us to consider all the possibilities of formats,
> > compressions,
> > and tiling arrangements. Using equivalent MB also summarizes
> > well the
> > many possible setups of width, height, bands and band sizes in
> > a single
> > number that can be setup WCS wide and provide peace of mind to
> > the
> > administrator (and stability to the server)
> >
> > Opinions?
> >
> > Cheers
> > Andrea
> >
> > --
> > Andrea Aime
> > OpenGeo - http://opengeo.org
> > Expert service straight from the developers.
> >
> >
> ------------------------------------------------------------------------------
> > Start uncovering the many advantages of virtual appliances
> > and start using them to simplify application deployment and
> > accelerate your shift to cloud computing.
> > http://p.sf.net/sfu/novell-sfdev2dev
> > _______________________________________________
> > Geoserver-devel mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/geoserver-devel
> >
> >
> >
> > --
> > Justin Deoliveira
> > OpenGeo - http://opengeo.org
> > Enterprise support for open source geospatial.
> >
> >
> >
> ------------------------------------------------------------------------------
> > Start uncovering the many advantages of virtual appliances
> > and start using them to simplify application deployment and
> > accelerate your shift to cloud computing.
> > http://p.sf.net/sfu/novell-sfdev2dev
> > _______________________________________________ Geoserver-devel mailing
> list [email protected]
> https://lists.sourceforge.net/lists/listinfo/geoserver-devel
>
> --
> Gabriel Roldan
> [email protected]
> Expert service straight from the developers
>
>
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> Geoserver-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/geoserver-devel
>
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Geoserver-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
