Re: [QGIS-Developer] Slow downloads for LTR MSI

[Tim Sutton via QGIS-Developer]

Hi

IIRC, Richard's explanation as to why we turned off the CDN is that it was
trying to grab all the download archives each time it did a cache request
and it was blowing our server out of the water. So the plan is to split the
latest downloads and the archives to separate hosts / domains / sites such
that the CDN conly caches the latest builds

Regards

Tim

On Mon, Aug 22, 2022 at 1:49 AM Nathan Woodrow via QGIS-Developer <
qgis-developer@lists.osgeo.org> wrote:

> Hey,
>
> Well I am downloading from Australia but not sure what mirror it is using
> as I'm just grabbing the LTR off the site, even the osgeo4w installer is
> very slow.  But everytime I do the download speeds are very very poor.  At
> times it is down to 14kb/s.  I don't remember having this issue in the past
> but others have said it has been bad for a long while now.
>
> I thought we had a CDN in front of this in order to help distribution?
>
> It seems to be quite a problem here.  We have had multi people report it
> as a issue and it doesn't look good.
>
> I'm happy to have a look at whatever we need to do in order fo speed it
> up.   If we need a CDN I can talk to work about using our AWS setup with
> CloudFront to get much better speeds and see if we will cover it.
>
> I don't know how much out going data it would be but CloudFront is cheap
> if we put the downloads into S3 and serve it from there.
>
> Regards
> Nathan
>
> On Wed, 20 Jul 2022, 6:19 pm Jürgen E. Fischer via QGIS-Developer, <
> qgis-developer@lists.osgeo.org> wrote:
>
>> Hi,
>>
>> On Wed, 20. Jul 2022 at 09:44:26 +1000, Nathan Woodrow via QGIS-Developer
>> wrote:
>> > I seem to be getting really slow downloads of the MSI packages (haven't
>> > tested any other packages at the moment) from the main download page.
>>
>> It's redirected to several hosts - independant of the origin address.  So
>> you
>> might get better results just by retrying.  Where did you actually
>> download
>> from?  US (download.osgeo.org, ftp.osuosl.org; Oregon State, US) or
>> Europe
>> (qgis.org, norbit.de; both at Hetzner in Germany)?
>>
>>
>> Jürgen
>>
>> --
>> Jürgen E. Fischer   norBIT GmbH Tel.
>> +49-4931-918175-31
>> Dipl.-Inf. (FH) Rheinstraße 13  Fax.
>> +49-4931-918175-50
>> Software Engineer   D-26506 Norden
>> https://www.norbit.de
>> QGIS release manager (PSC)  Germany IRC: jef on
>> Libera|OFTC
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>


-- 
--
​

Tim Sutton
Visit http://kartoza.com to find out about open source:
 * Desktop GIS programming services
 * Geospatial web development
* GIS Training
* Consulting Services
Tim is a member of the QGIS Project Steering Committee
---
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Model for Temporal Measurement (vector) Datasets ?

[Tim Sutton via QGIS-Developer]

Hi Richard

Thanks for clarifying your ideas. Could we not implement something similar
to how WMS-T works in that we have a PostgreSQL-T provider extension for
example that passes a time filter to the underlying data base request. In
another system we are build for a client we have some logic to fetch
min/max/mean/sum/etc from a time slice so we could have something similar
such that each point has only one value for the current time slice.

Not sure if I make myself clear or not...


Regards

Tim

On Sun, Sep 4, 2022 at 4:22 PM Anita Graser via QGIS-Developer <
qgis-developer@lists.osgeo.org> wrote:

> Hi Richard,
>
> On 02.09.2022 13:07, Richard Duivenvoorde wrote:
> > Anybody is aware of a model in which data owns a location, but THAT
> > location has a (growing) set of [timestamp, value] observation data to
> > it
>
> MobilityDB has temporal temporal bools, ints, floats, texts, and
> geometries: https://docs.mobilitydb.com/MobilityDB/develop/ch03.html
>
> So that would cover the storage question but the visualization issue
> remains until we implement support for these types in Temporal Controller.
>
> Regards,
>
> Anita
>
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>


-- 
--
​

Tim Sutton
Visit http://kartoza.com to find out about open source:
 * Desktop GIS programming services
 * Geospatial web development
* GIS Training
* Consulting Services
Tim is a member of the QGIS Project Steering Committee
---
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Model for Temporal Measurement (vector) Datasets ?

[Anita Graser via QGIS-Developer]

Hi Richard,

On 02.09.2022 13:07, Richard Duivenvoorde wrote:

Anybody is aware of a model in which data owns a location, but THAT
location has a (growing) set of [timestamp, value] observation data to
it


MobilityDB has temporal temporal bools, ints, floats, texts, and
geometries: https://docs.mobilitydb.com/MobilityDB/develop/ch03.html

So that would cover the storage question but the visualization issue
remains until we implement support for these types in Temporal Controller.

Regards,

Anita


___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Model for Temporal Measurement (vector) Datasets ?

[Richard Duivenvoorde via QGIS-Developer]

On 9/4/22 15:27, Matthias Kuhn wrote:

Hi Richard,

It seems to me this should be possible using:

1. Two tables:
- Points
- Measurements (Timestamp, Measured attribute)

2.  A view (Points LEFT JOIN Measurements)



What would be the actual result in your case (eg in the case of 2 points with 
100 measurements each)?

That would be 2 records/features isn't it? With..., I dont know, the last join?
OR 200 records (which is the actual way QGIS/GIS is handling this?

Note that I'm not talking about storage here.
I'm looking in a way, the temporal controller (or something else) can 
(separately from the geometries) request the timestamp/value data.
For example: FIRST requests all Locations of Sensors,
Then (based on the status of some (temporal) controller) request a subset of 
the values/timestamps.

I'm looking into this, because I'm searching for a way to handle massive 
datasets, in which it is such overkill to again and again sent over the 
geometries (be it points or polygons)...
Or another use case: I want points on the map, and timestamps/values flying in 
realtime as they are measured...
Or: you have a giant km grid covering the whole of europe, and data (per hour) 
for 7 days of data, and you can request (remotely!) the value for every grid 
cell per 10 minutes (without having to request the geometry every time...)

Regards & all thanks for the input,

Regards,

Richard



___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Model for Temporal Measurement (vector) Datasets ?

[Matthias Kuhn via QGIS-Developer]

Hi Richard,

It seems to me this should be possible using:

1. Two tables:
- Points
- Measurements (Timestamp, Measured attribute)

2.  A view (Points LEFT JOIN Measurements)

Timescale as proposed by Tim can be added to the mix to optimize things if
you have additional requirements for time-specific aggregates or optimized
storage.

Matthias

On Sun, Sep 4, 2022 at 3:18 PM Richard Duivenvoorde via QGIS-Developer <
qgis-developer@lists.osgeo.org> wrote:

> On 9/4/22 14:35, Tim Sutton wrote:
> > Hi Richard
> >
> > Don’t people normally use things like https://www.timescale.com/ <
> https://www.timescale.com/?utm_source=timescaledb-paid_medium=google-search_campaign=brand-2022_content=homepage_term=timescale-postgres=EAIaIQobChMIspXn6JL7-QIVOpBoCR3QLQf8EAAYASAAEgL09PD_BwE
> >
> >
> > For this? In any case some of our clients that are doing things like
> vehicle tracking are…
>
> Yes to *store* it, but my point is: given 2 points with 100 timesteps,
> QGIS will create 200 features for it to be able to display it...
> So to 'view' that data in (Q)GIS, you have to flatten it to a 'table' with
> 200 features...
>
> While the actual interesting part is the value and the time, NOT those 2
> xy fields... (the case of moving vehicles is slightly different, there the
> xy values also change)...
> Maybe I'm in the wrong crowd here to tell the geoms are not so much of
> interest in my case ;-)
>
> I'm looking for a way/model (non tabular) that QGIS creates 2
> geometries/features, and then will style/paint it based on the timestamps
> and values
>
> The use case here: having 5000 sensors in Europe, having 500.000.000
> measurements in total, I don not want to create 500.000.000 features.
>
> I'd preferred to create those 5000 features, and then get the
> geom-id/value/timestamp for a given filter range and repaint my canvas.
>
> Something like 'dynamic' joins: the geometries will stay the same, but the
> 'joined' columns/table will change based on the 'temporal filter' requested.
>
> Sorry if I can not make myself more clear (or making it more complex that
> it maybe is...) :-)
>
> Regards,
>
> Richard Duivenvoorde
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>

-- 
 

QFIELD 2.0 IS HERE! - Hold the power of QGIS in 
your hand - learn more 
 - get it now 


___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Model for Temporal Measurement (vector) Datasets ?

[Richard Duivenvoorde via QGIS-Developer]

On 9/4/22 14:35, Tim Sutton wrote:

Hi Richard

Don’t people normally use things like https://www.timescale.com/ 


For this? In any case some of our clients that are doing things like vehicle 
tracking are…


Yes to *store* it, but my point is: given 2 points with 100 timesteps, QGIS 
will create 200 features for it to be able to display it...
So to 'view' that data in (Q)GIS, you have to flatten it to a 'table' with 200 
features...

While the actual interesting part is the value and the time, NOT those 2 xy 
fields... (the case of moving vehicles is slightly different, there the xy 
values also change)...
Maybe I'm in the wrong crowd here to tell the geoms are not so much of interest 
in my case ;-)

I'm looking for a way/model (non tabular) that QGIS creates 2 
geometries/features, and then will style/paint it based on the timestamps and 
values

The use case here: having 5000 sensors in Europe, having 500.000.000 
measurements in total, I don not want to create 500.000.000 features.

I'd preferred to create those 5000 features, and then get the 
geom-id/value/timestamp for a given filter range and repaint my canvas.

Something like 'dynamic' joins: the geometries will stay the same, but the 
'joined' columns/table will change based on the 'temporal filter' requested.

Sorry if I can not make myself more clear (or making it more complex that it 
maybe is...) :-)

Regards,

Richard Duivenvoorde
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Model for Temporal Measurement (vector) Datasets ?

[Tim Sutton via QGIS-Developer]

Hi Richard

Don’t people normally use things like https://www.timescale.com/

For this? In any case some of our clients that are doing things like vehicle 
tracking are…

Regards 

Tim

Sent from my iPhone

> On 2 Sep 2022, at 13:07, Richard Duivenvoorde via QGIS-Developer 
>  wrote:
> 
> Hi All,
> 
> Anybody is aware of a model in which data owns a location, but THAT location 
> has a (growing) set of [timestamp, value] observation data to it
> 
> In GIS/QGIS everything is tabular (talking about vector here, I know meshes 
> can have more shapes).
> 
> But in the IOT world the 'location' is 'just an attribute', and the 'other' 
> data is actually more of interest (and growing in time).
> 
> Currently to view measurements, 'we' often replicate the location for every 
> value/measurement: we create a 'Feature' for every time step (eg for example 
> via a WFS)
> 
> The OGC has a SensorThingsApi (STA) standard, in which for given location you 
> can request all values/observations (of filter a certain 
> sensortype/parameter). So one geometry has a full table of data to it.
> 
> BUT to 'work' with that data in QGIS, you always have to 'flatten' it, one 
> way or another, (I think), for every time step: create a feature with: 
> location, time, value...
> 
> My Question:
> 
> - isn't there some model (or can we come up with one) in which a location can 
> actually have a range of time/value data (like some table join like), and 
> when you use the Temporal Controller, you style your layer by RE-using the 
> one location, but getting the value for current Timestamp (in the 
> TimeController). Some sort of indirect filtering?
> 
> - or is this the mesh model (I know netcdf's in which you have often 
> predefined meshes/location and the data/time vector is growing)?
> And would a solution be to have some kind of in memory vector -> mesh loader 
> or so???
> 
> I hope this makes sense to others. Any input appreciated.
> 
> Regards,
> 
> Richard Duivenvoorde
> 
> PS, if you want to play with SensorThingsApi:
> - install: https://github.com/AirBreak-UIA/SensorThingsAPI_QGIS-plugin
> - connect it to: 
> https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Locations
> (around 5000 location and 500-million 'observations')
> - load all locations in one layer, and click on a location
> My point: the data is then viewable/selectable by table and graph, but NOT 
> loadable in a sense way (yet) in QGIS...
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer