Re: [Qgis-developer] Raster resampling suggestion
Hi Nick Resampling on data level does not work in case of colormap rasters, therefore the resampling filter currently comes after assigning rgba. One possibility would be to provide a user choice if resampling should be done on color values or on data. I think it is best if you post your code (as a pull request on github or via mailing list), then others can give it a try and provide feedback. I'm not really a raster power user myself (mostly working with rasters as background images for maps). Regards, Marco On 18.02.2014 05:41, Nick Dolezal wrote: Hey, I've been using qgis and have been pretty impressed with it so far. I would however like to offer a suggestion for improvement of the raster resampling. Right now, it appears the resampling is done on the final RGBA output, rather than on the data values. Further, there are issues with both negative kernel lobe artifacts from the cubic convolution kernel and NODATA interpolation artifacts from the bilinear resampling. My suggestion on how to deal with this is: 1. Resample the data values, not the RGBA output 2. To remove the NODATA artifacts, create two rasters while resampling: 2a. First, create a nearest-neighbor only resampled image from the original. 2b. Second, before interpolating, use a neighborhood mean that only fills NODATA cells, edge extend, or combination of the two to prevent NODATA interpolation artifacts. This must be done to a minimum of (kernelExtent - 1.0) cells. Then, interpolate normally. 3. Use the NN raster as a mask to gate data values from the resampled image. ie: dest[i] = nnMask[i] != NODATA ? cubicResampledSrc[i] : NODATA; Should be 2-4 cycles per pixel SIMD depending on dual issue. If the NODATA value is 0, you can just clip everything to either 0 or 1 in the NN raster and do lots of nice memory-aligned vmuls. 3B. If using cubic (or Lanczos), you'll need to integrate or do another clipping pass to make sure the evil negative kernel lobe fairy didn't turn any data into NODATA. 4. If you're maintaining the alpha channel at this point, I would suggest always using NN to resize it. I have implemented this myself, and it works quite well, for both bilinear and Lanczos interpolation. Not terribly creative, it's derived from how the alpha channel in PowerVR textures work. I did not find the neighborhood mean to significantly distort anything, compared to a very strict bilinear implementation that never interpolated NODATA but rather fell back on NN. Net results are: 1. Higher quality interpolation, if the source raster uses a data type with more width than uint8_t. You can actually zoom in crazy amounts on even bilinear interpolated rasters as long as you're interpolating something like 32-bit floats. 2. No interpolation into NODATA values. 3. With bicubic, elimination of negative kernel lobe black ringing artifacts in NODATA areas. (ringing artifacts will be present to some degree still. This can mitigated by transforming to the logarithmic domain and back but that's relatively expensive) 4. Possibly faster performance if you were interpolating 4 channels of RGBA interleaved data which could be done with 2x planar interpolations instead. If it would help, I can provide source code of a working implementation and release it without copyright, although as this stuff is simple but implementation-dependent, I'm not sure it'd be more useful than the above. Just pad any NODATA around the data values, interpolate and gate / clip with the 2nd NN raster. Thanks! ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Dr. Marco Hugentobler Sourcepole - Linux Open Source Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland marco.hugentob...@sourcepole.ch http://www.sourcepole.ch Technical Advisor QGIS Project Steering Committee ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Metadata fields in QGIS
I posted this to the user list a couple of weeks ago but no interest. Maybe the dev list has some thoughts? Hi List, I'm wondering if anyone uses the metadata fields for data in QGIS (Layer Properties - Metadata)? I've been told that this is supposed to be filled in manually. I ask because I originally thought it was supposed to be filled in automatically - http://hub.qgis.org/issues/8870 - And for OGC services (WMS, WFS etc) this seems logical as all of the data is in the GetCapabilities. Does anyone have a view on this? Should the metadata be filled in automatically or should the user enter it manually? My issue with the later is that I guess the metadata is only saved in a .qgs file, and not the source itself, so no-one else is going to be able to see manually entered data. Thoughts? Comments? Cheers, Jonathan -- This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] iterating over fields an values API 2.0
Yes!
That's it
Thanks
PS: Don't you think it should be clearly documented in the pyqgis cookbook?
-Message d'origine-
De : Martin Dobias [mailto:wonder...@gmail.com]
Envoyé : lundi 17 février 2014 18:43
À : PIERRE Sylvain
Cc : qgis-developer@lists.osgeo.org
Objet : Re: [Qgis-developer] iterating over fields an values API 2.0
On Mon, Feb 17, 2014 at 10:36 PM, PIERRE Sylvain sylvain.pie...@cg67.fr wrote:
When iterating over vector features like this:
fields = layer.pendingFields()
field_names = [field.name() for field in fields]
for elem in layer.getFeatures():
atr = dict(zip(field_names, elem.attributes()))
returns this:
{PyQt4.QtCore.QString(u'off_set'): PyQt4.QtCore.QVariant object at
0x05115C70, PyQt4.QtCore.QString(u'code'): PyQt4.QtCore.QVariant
0x05115C70object at
0x05115BC8, PyQt4.QtCore.QString(u'pk'): PyQt4.QtCore.QVariant
0x05115BC8object at
0x05115C00, PyQt4.QtCore.QString(u'id'): PyQt4.QtCore.QVariant
0x05115C00object at
0x05115B90, PyQt4.QtCore.QString(u'annee'): PyQt4.QtCore.QVariant
0x05115B90object
at 0x05115CA8, PyQt4.QtCore.QString(u'id_anom'):
PyQt4.QtCore.QVariant object at 0x05115B58,
PyQt4.QtCore.QString(u'fk_position'):
PyQt4.QtCore.QVariant object at 0x05115C38,
PyQt4.QtCore.QString(u'resolu'): PyQt4.QtCore.QVariant object at
0x05115CE0}
Instead of tupple like u'off_set': 1 ...
You have not set SIP api to version 2 to QString, QVariant and other Qt
classes. QGIS python package tries to do that automatically for you, but if you
imported PyQt4 python modules before QGIS python modules, that will not work.
You have two options:
1. move import of QGIS module before PyQt4 2. keep the order of imports as is,
but put these lines before import of PyQt4:
import sip
sip.setapi(QString, 2)
sip.setapi(QVariant, 2)
This is only necessary in standalone applications - inside QGIS application
this initialization is done before any plugins are loaded, so this issue does
not come up. Probably in qgis __init__.py there should be some code checking if
there are not Qt classes imported with API v1 - in such case we could throw an
exception.
Regards
Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] iterating over fields an values API 2.0
On Tue, Feb 18, 2014 at 9:53 PM, PIERRE Sylvain sylvain.pie...@cg67.fr wrote: Yes! That's it Thanks PS: Don't you think it should be clearly documented in the pyqgis cookbook? Agreed. Would you mind preparing a pull request to fix that? Thanks Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Spatialize attribute columns XY oor WKT, design choice
Hi Régis, Le 17/02/2014 12:08, Régis Haubourg a écrit : Hi, Users here ask us a good reliable way to spatialize data from XY or WKT columns and I'm wondering what could be the reference architecture to have it in core. I am aware of Xytools plugins, delimited text import and RTQspider plugin, and also of VRT GDAL format. Questions here are for 2.3 branch or later. Here are the functionnal requirements: [snip] VRT is IMHO no adaptated since it is not dynamic with csv data, GDAL lacks many options to open correctly csv (text delimiters, decimal delimiters...) . This is why delimited text exists. It also has no spatial index. It still remains extremly powerfull for UNION, spatial queries thanks to SQL dialect... I think you need something very close to GDAL's VRT, but supported natively by QGIS. It could be (just thinking loud) something like a special QgsVectorProvider that wraps another one (or many other ones) with options like pointsFromColumns, WKT field, etc. or with the possibility to have computed columns based on QgsExpression (geometryFromWKT already exists, we may want to add MakePoint, MakeLine, etc.) This provider would not allow edition of computed geometries and would be responsible of indexes and so on. -- Hugo Mercier Oslandia ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Metadata fields in QGIS
Hi Jonathan, Il 18/02/2014 13:18, Jonathan Moules ha scritto: I posted this to the user list a couple of weeks ago but no interest. Maybe the dev list has some thoughts? I think metadata should be properly handled by QGIS. We are using (and improving) the metadata plugin by Alex, but the longer term solution would be to integrate all the metadata in the main properties, as you suggest. All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Spatialize attribute columns XY oor WKT, design choice
Hi Hugo, I like the idea! Sounds clear to me. if core commiters agree, that would be nice to start such a work for 2.4. any objection? -- View this message in context: http://osgeo-org.1560.x6.nabble.com/Spatialize-attribute-columns-XY-oor-WKT-design-choice-tp5104209p5104521.html Sent from the Quantum GIS - Developer mailing list archive at Nabble.com. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Numeric scale error when print (ticket #9079)
Hi all, has anyone seen this? http://hub.qgis.org/issues/9079 if confirmed (it seems so to me) it would be pretty bad. It is not a regression as at least also qgis 1.8 is affected. Cheers! ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] blockers list, almost done!
Hi all, thanks to your effort (sponsors, donors and developers) the blockers list in down to 4 tickets: one is probably already fixed (9526), another it seems it is being worked on by Alvaro (9581) and a third one (9060) it is pretty serious because it causes to have areas (and distances) computed wrong when otfr is on. I would like to know is someone is going to have a look to #9060 before the new release because it is a pretty serious issue in qgis 2.0.1 Thanks in advance! ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Plugin [464] MetaSearch Catalogue Client approval notification.
Plugin MetaSearch Catalogue Client approval by rduivenvoorde. The plugin version [464] MetaSearch Catalogue Client 0.1.0 is now approved Link: http://plugins.qgis.org/plugins/MetaSearch/ ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] QGIS MetaSearch Catalogue Client 0.1.0 released
The MetaSearch team announces the release of MetaSearch 0.1.0. The 0.1.0 release marks the initial offering of a CSW client for QGIS 2, with significant features to enable plug and play interoperability with OGC CSW services. - find and bind functionality allowing users to discover and add layers from WMS, WFS, WCS, or WMTS services dynamically to the map - template-based service and record metadata view, allowing for custom templating of responses - visual footprint of CSW results - display of all metadata access links, allowing users to click/download data to add to QGIS - spatial and aspatial querying of CSW services Version 0.1.0 represents a significant engineering effort to bring CSW support for QGIS 2, make the codebase sustainable/extensible and easy for developers and documentation teams to make contributions. Longer term plans include: - supporting other catalog APIs (CKAN, etc.) - making MetaSearch part of QGIS core - transifex localization - display thumbnail/browse images We are looking for feedback and testers for the plugin. Comments, bugs, features/enhancements are welcome (IRC, GitHub issue tracker, etc.). The full list of enhancements and bug fixes is available at https://github.com/geopython/MetaSearch/issues?milestone=1page=1state=closed The plugin is available from the QGIS Plugins Repository at http://plugins.qgis.org/plugins/MetaSearch Testers and developers are welcome. The MetaSearch team. http://geopython.github.io/MetaSearch ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Plugin [464] MetaSearch Catalogue Client unapproval notification.
Plugin MetaSearch Catalogue Client unapproval by rduivenvoorde. The plugin version [464] MetaSearch Catalogue Client 0.1.0 is now unapproved Link: http://plugins.qgis.org/plugins/MetaSearch/ ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Plugin [464] MetaSearch Catalogue Client approval notification.
Plugin MetaSearch Catalogue Client approval by NathanW. The plugin version [464] MetaSearch Catalogue Client 0.1.1 is now approved Link: http://plugins.qgis.org/plugins/MetaSearch/ ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Raster resampling suggestion
Hey Marco,
Here's the code I have for a bilinear-based implementation (attached),
though I've omitted the actual NN and bilinear resampling code itself for
brevity. This does have a number of implementation-specific assumptions
which would need to be reworked for the general case. (the general case in
GIS is hard and makes my brain bleed.) So it's not really close to the
point of being ready for a pull request.
It could be adapted to work on RGBA output, though with some modifications
mostly involving using the alpha channel with a NN resample in place of the
NODATA values.
It should be all ANSI C with no external dependencies. (except for one
irrelevant Accelerate framework call)
Hope this is useful -- let me know if anything else is needed.
On Tue, Feb 18, 2014 at 1:14 AM, Marco Hugentobler
marco.hugentob...@sourcepole.ch wrote:
Hi Nick
Resampling on data level does not work in case of colormap rasters,
therefore the resampling filter currently comes after assigning rgba. One
possibility would be to provide a user choice if resampling should be done
on color values or on data.
I think it is best if you post your code (as a pull request on github or
via mailing list), then others can give it a try and provide feedback. I'm
not really a raster power user myself (mostly working with rasters as
background images for maps).
Regards,
Marco
//
// GetZoomedTile_BilinearPunch:
//
//
// Wrapper around bilinear resize for a raster that preserves NODATA values
// without artifacts.
//
// Evenly resizes (zooming in only) a planar 32-bit float tile of varying resolution,
//normally 256x256 - 384x384.
//
// It will work on other widths and heights, but some of the logic expecting evenly
//divisible srcWH and resampleBufferWH may need to change, or need a ceilf.
//
// Preserves NODATA values vaguely (but not really) derived from PVR textures;
//a punch through technique is used based upon NN resampling.
//
// To avoid interpolating NODATAs, a neighborhood raster stats mean function is used.
//
// Currently assumes NODATA = 0.0F and that interpolated values will not be transformed
//into NODATA.
//
void gbImage_GetZoomedTile_BilinearPunch_v2(float* src,
const size_t srcWidth,
const size_t srcHeight,
const size_t srcWidthUnpadded,
const size_t srcHeightUnpadded,
const size_t srcDataCount,
float* dest,
const size_t destWidth,
const size_t destHeight,
float* resampleBuffer,
const size_t resampleBufferWidth,
const size_t resampleBufferHeight,
float* tempBuffer,
const size_t tempBufferBytes,
const intinterpolationTypeId,
const bool doNotTile,
const float NODATA)
{
const bool useBitmask = srcDataCount srcWidth * srcHeight;
const bool cleanOutput = destWidth == resampleBufferWidth destHeight == resampleBufferHeight;
// first, get NN resampled mask to filter resampled output with later
if (useBitmask interpolationTypeId == kGB_TileEngine_Interp_Bilinear)
{
// returns NN resampled raster tile of 0 or 1, which is somewhat more efficient than using conditionals later, but will *not* work in the general case.
// for the general case, do a normal NN resize here, and use a conditional in the final filtering step instead of vmul
gbImage_GetZoomedTile_NN_FromCrop_Bitmask(src, srcWidthUnpadded, srcHeightUnpadded, srcWidth, dest, destWidth, destHeight, NODATA);
_gbImage_FillNODATA(src, srcWidth, srcHeight, srcWidthUnpadded, srcHeightUnpadded, srcDataCount, tempBuffer, tempBufferBytes, interpolationTypeId, NODATA);
}//if
// always do NoNODATA if possible, other version is branchy and slower due to inline NODATA conditionals
if (interpolationTypeId == kGB_TileEngine_Interp_Bilinear || !useBitmask)
{
gbImage_BilinearVectorized_NoNODATA(src,
srcWidth,
srcHeight,
cleanOutput ? dest : resampleBuffer,
resampleBufferWidth,
resampleBufferHeight,
true);
}//if
else
{
Re: [Qgis-developer] blockers list, almost done!
Il 18/02/2014 20:09, Giovanni Manghi ha scritto: one is probably already fixed (9526), another it seems it is being worked on by Alvaro (9581) and a third one (9060) it is pretty serious because it causes to have areas (and distances) computed wrong when otfr is on. I would like to know is someone is going to have a look to #9060 before the new release because it is a pretty serious issue in qgis 2.0.1 Good news! There is also http://hub.qgis.org/issues/9324 but it does not seem to me serious enough to really block us. Any hope to have all 4 fixed within 1.5 days? All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Spatialize attribute columns XY oor WKT, design choice
Hi, Recently there was a discussion on IRC between NathanW and EvenR concerning SQLite's virtual table [1] I think this would be a (mighty) generic approach which could also serve this use case. I didn't give it much deeper thoughts yet, but I must admit it looks promising. Basically you just register a new table with SQLite and implement the backend for it. This gives you all the SQLite power (where we even have a driver for already) and therefore database goodies tailor suited to runtime-generated data. I also had the idea of creating such tables on the fly for iterators spanning multiple tables (for JOIN support in queries. Providing a provider-independent highlevel query syntax like expressions). If we do this I would like to make sure that we have the possibility to get access to the parsed query before it is executed. This is necessary to be able to optimize the query for databases for which we have our own drivers and which support such operations natively (Think of a join of two possibly huge postgres tables: you don't want to join them locally). I haven't yet verified if this is possible or not, and can therefore not say if this approach fulfils the requirements. There are python bindings available for virtual tables, so it may be possible to implement it this way [2] I think it would be good to gather some experience in this direction before implementing it in core. I am not even sure how it plays with SpatiaLite. Best Matthias [1] https://www.sqlite.org/vtab.html [2] http://apidoc.apsw.googlecode.com/hg/vtable.html On Tue 18 Feb 2014 05:59:29 PM CET, Régis Haubourg wrote: Hi Hugo, I like the idea! Sounds clear to me. if core commiters agree, that would be nice to start such a work for 2.4. any objection? -- View this message in context: http://osgeo-org.1560.x6.nabble.com/Spatialize-attribute-columns-XY-oor-WKT-design-choice-tp5104209p5104521.html Sent from the Quantum GIS - Developer mailing list archive at Nabble.com. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] blockers list, almost done!
For me this is also a regression from 2.0 but it is down as high priority for future release. This was working fine in 2.0. http://hub.qgis.org/issues/9357 I know it will stop me upgrading to 2.2 for this one script I have. 4 months isn't so long to wait for 2.4 if it can't be fixed for 2.2 I guess :) James -Opprinnelig melding- Fra: qgis-developer-boun...@lists.osgeo.org [mailto:qgis-developer-boun...@lists.osgeo.org] På vegne av Paolo Cavallini Sendt: 19. februar 2014 07:41 Til: qgis-developer@lists.osgeo.org Emne: Re: [Qgis-developer] blockers list, almost done! Il 18/02/2014 20:09, Giovanni Manghi ha scritto: one is probably already fixed (9526), another it seems it is being worked on by Alvaro (9581) and a third one (9060) it is pretty serious because it causes to have areas (and distances) computed wrong when otfr is on. I would like to know is someone is going to have a look to #9060 before the new release because it is a pretty serious issue in qgis 2.0.1 Good news! There is also http://hub.qgis.org/issues/9324 but it does not seem to me serious enough to really block us. Any hope to have all 4 fixed within 1.5 days? All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Plugin approval
Hello, I submitted a plugin three weeks ago, but it remains unapproved and I didn't receive the notification by e-mail, neither. The plugin is called Viewshed Analysis - currently the last one in the list of unapproved plugins (http://plugins.qgis.org/plugins/unapproved). In the meanwhile, I created a GitHub repository, set up a QGIS project and uploaded the plugin for the second time (in order to ease the process) - still no news... (Yes, I should have done it at the outset ...) Is it possible to get an information on problems in the plugin functionality, so that I might get down to fix them ? Thanks, Zoran ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Plugin approval
On 02/18/2014 11:32 PM, Zoran Čučković wrote: Hello, I submitted a plugin three weeks ago, but it remains unapproved and I didn't receive the notification by e-mail, neither. The plugin is called Viewshed Analysis - currently the last one in the list of unapproved plugins (http://plugins.qgis.org/plugins/unapproved). In the meanwhile, I created a GitHub repository, set up a QGIS project and uploaded the plugin for the second time (in order to ease the process) - still no news... (Yes, I should have done it at the outset ...) Is it possible to get an information on problems in the plugin functionality, so that I might get down to fix them ? Thanks, Zoran It's most likely just a backlog of plugins to review. See this thread for discussion. http://osgeo-org.1560.x6.nabble.com/one-email-address-for-plugin-approvers-td5103254.html Thanks, Alex ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] About Point Samplig Tool plugin
On 02/14/2014 01:43 AM, Geo DrinX wrote: Hello All, perhaps there is a problem with PointSamplingTool plugin, using layers with different SRS. In fact, I found all the resulting fields of the output SHP with NULL . Somebody test it, recently ? Could you help me ? Thank you Roberto There is a good chance that is true. I have always recommended for analysis one must reproject all data into the same projection. There are many other tools that won't work in QGIS and other software. Projection on the fly is really best reserved for exploratory purposes and cartography (even then sometimes it's better to reproject). Thanks, Alex ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
