Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[C Hamilton]

Thanks Jeroen,

Here is a link to the timezone geopackage.

https://drive.google.com/file/d/1IS_bExc5M2IhadK9hzE012oorvV3IbBp/view?usp=sharing

You could also use the shapefile from

https://github.com/evansiroky/timezone-boundary-builder/releases/tag/2020d

I am using the version with oceans. I suspect that the shapefile may be
slightly faster than the gpkg, but I haven't tested it yet.

Here is the gpkg of 10,000 random points.

https://drive.google.com/file/d/1Vz0ho8nQ9rxG6YUG_JebyVygBkjMFsGr/view?usp=sharing

Here is a short script that you can run in the QGIS python console to time
how long it takes to search the timezone for each of the 10,000 points.

https://drive.google.com/file/d/1VMBs6AdvdvWOI_S9XvEaDm3oVyr_vHNv/view?usp=sharing

You will either need to pip install the timezonefinder library or install
the datetimetools-3.0.1.zip release which will make the QGIS python console
aware of the library.

https://github.com/NationalSecurityAgency/qgis-datetimetools-plugin/releases/tag/3.0.1

The mention of ArcGIS is simply that we have an ESRI centric environment
and it is difficult to get QGIS accepted so I try to do everything I can to
make QGIS look appealing.

Thanks once again,

Calvin


On Tue, Feb 2, 2021 at 5:23 PM Groene Bij  wrote:

> Hi Calvin,
>
>
>
> Could you somehow send me a copy of the 10.000 random point layer and the
> gpkg with the time zones? I would like to try some things, and see if I can
> come up with some suggestions. I could send you an upload link where you
> can upload those files to me, if needed.
>
>
>
> And I am also trying to understand the issue at stake here. You keep
> referring to ArcGIS. Does ArcGIS has a similar tool incorporated In its
> software, including the data?
>
>
>
> And is it necessary to use this tool offline? If so, a database / data
> file is needed anyway. Than the question is, should it be part of the tool,
> or should you be able to download the data when needed (or better,
> beforehand). Benefit of downloading the data separately could be more up to
> date time zone information.
>
>
>
> Kind regards,
>
> Jeroen Hovens
>
>
>
>
>
> *Van:* Qgis-user  *Namens *C Hamilton
> *Verzonden:* dinsdag 2 februari 2021 19:40
> *Aan:* Nyall Dawson 
> *CC:* QGIS Users ; Tim Sutton ;
> qgis-developer 
> *Onderwerp:* Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools
> Plugin
>
>
>
> I created a layer of 10,000 random points. Using timezonefinder to look up
> each point took 44 seconds. I tried using *Vector->Geoprocessing
> Tools->Intersection...* to find the time zones for each of the points and
> it took over 4 minutes.
>
>
>
> Is there a better way to locate which polygon a point is in? 4 minutes is
> not good. Perhaps the intersection algorithm is not the way to go, but I am
> not sure what is. I can't imagine that iterating through each point and
> then iterating through each polygon until an intersection match is found is
> going to be overlay efficient, but I will try it. Is there a better way?
>
>
>
> Thanks,
>
>
>
> Calvin
>
>
>
>
>
> > I wonder what the difference in time would be to use a gpkg and QGIS
> point in polygon look up vs timezonefinder lookup. Will it be faster or
> slower? I'm not sure without testing. What are your thoughts? I would
> choose the faster of the two.
>
> I'd certainly hope a dedicated GIS application can do this faster than
> a random Python library :D
>
> > I just converted the timezone data to a gpkg and its size is 102Mb. The
> data size for timezonefinder using the same data set is 49Mb. The
> optimization in timezonefinder can produce a maximum error of 1cm at the
> equator as they use 32 bit ints for the data. It might actually be better
> to use the gpkg file, but it is double in size. I will investigate using
> the gpkg data.
>
> You may want to try a shapefile too, just in case...!
>
> Nyall
>
>
> >
> > It seems to me that if a plugin requires a certain dataset to function
> then that data should be included with it and not require an additional
> download, but I understand the issues that this could cause so I am not
> sure of the best way forward. For our use it is definitely better to
> include the data with the plugin.
> >
> > I guess what I would like you to take from this conversation is that
> some users have a completely different environment and face different
> challenges then what you do when it comes to dealing with software. I wish
> that it was easier for me to get QGIS acceptance in our workforce but
> ArcGIS still rules.
> >
> > Thanks for your ideas and all of the hard work that you do to keep QGIS
> moving forward.
> >
> > I wish you all t

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Groene Bij]

Hi Calvin,

 

Could you somehow send me a copy of the 10.000 random point layer and the gpkg 
with the time zones? I would like to try some things, and see if I can come up 
with some suggestions. I could send you an upload link where you can upload 
those files to me, if needed.

 

And I am also trying to understand the issue at stake here. You keep referring 
to ArcGIS. Does ArcGIS has a similar tool incorporated In its software, 
including the data?

 

And is it necessary to use this tool offline? If so, a database / data file is 
needed anyway. Than the question is, should it be part of the tool, or should 
you be able to download the data when needed (or better, beforehand). Benefit 
of downloading the data separately could be more up to date time zone 
information.

 

Kind regards,

Jeroen Hovens

 

 

Van: Qgis-user  Namens C Hamilton
Verzonden: dinsdag 2 februari 2021 19:40
Aan: Nyall Dawson 
CC: QGIS Users ; Tim Sutton ; 
qgis-developer 
Onderwerp: Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

 

I created a layer of 10,000 random points. Using timezonefinder to look up each 
point took 44 seconds. I tried using Vector->Geoprocessing 
Tools->Intersection... to find the time zones for each of the points and it 
took over 4 minutes. 

 

Is there a better way to locate which polygon a point is in? 4 minutes is not 
good. Perhaps the intersection algorithm is not the way to go, but I am not 
sure what is. I can't imagine that iterating through each point and then 
iterating through each polygon until an intersection match is found is going to 
be overlay efficient, but I will try it. Is there a better way?

 

Thanks,

 

Calvin

 



> I wonder what the difference in time would be to use a gpkg and QGIS point in 
> polygon look up vs timezonefinder lookup. Will it be faster or slower? I'm 
> not sure without testing. What are your thoughts? I would choose the faster 
> of the two.

I'd certainly hope a dedicated GIS application can do this faster than
a random Python library :D

> I just converted the timezone data to a gpkg and its size is 102Mb. The data 
> size for timezonefinder using the same data set is 49Mb. The optimization in 
> timezonefinder can produce a maximum error of 1cm at the equator as they use 
> 32 bit ints for the data. It might actually be better to use the gpkg file, 
> but it is double in size. I will investigate using the gpkg data.

You may want to try a shapefile too, just in case...!

Nyall


>
> It seems to me that if a plugin requires a certain dataset to function then 
> that data should be included with it and not require an additional download, 
> but I understand the issues that this could cause so I am not sure of the 
> best way forward. For our use it is definitely better to include the data 
> with the plugin.
>
> I guess what I would like you to take from this conversation is that some 
> users have a completely different environment and face different challenges 
> then what you do when it comes to dealing with software. I wish that it was 
> easier for me to get QGIS acceptance in our workforce but ArcGIS still rules.
>
> Thanks for your ideas and all of the hard work that you do to keep QGIS 
> moving forward.
>
> I wish you all the very best!
>
> Calvin
>
>> Given that the size of the python library is almost entirely the size
>> of the timezone boundaries themselves, have you considered:
>> - avoiding the library entirely, and insteading using a standard
>> shapefile/gpkg/... of the boundaries and using QGIS vector layer
>> methods to determine the timezone for a point
>> - deferring the download of the boundary spatial data, so that it's
>> not supplied with the plugin but instead the plugin automatically
>> downloads it on first launch?
>>
>> This would avoid the need for the large size plugin, allowing it to be
>> supplied via the standard QGIS repo while still providing its full
>> functionality...
>>
>> Nyall
>>
>>

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[C Hamilton]

I created a layer of 10,000 random points. Using timezonefinder to look up
each point took 44 seconds. I tried using *Vector->Geoprocessing
Tools->Intersection...* to find the time zones for each of the points and
it took over 4 minutes.

Is there a better way to locate which polygon a point is in? 4 minutes is
not good. Perhaps the intersection algorithm is not the way to go, but I am
not sure what is. I can't imagine that iterating through each point and
then iterating through each polygon until an intersection match is found is
going to be overlay efficient, but I will try it. Is there a better way?

Thanks,

Calvin


>
> > I wonder what the difference in time would be to use a gpkg and QGIS
> point in polygon look up vs timezonefinder lookup. Will it be faster or
> slower? I'm not sure without testing. What are your thoughts? I would
> choose the faster of the two.
>
> I'd certainly hope a dedicated GIS application can do this faster than
> a random Python library :D
>
> > I just converted the timezone data to a gpkg and its size is 102Mb. The
> data size for timezonefinder using the same data set is 49Mb. The
> optimization in timezonefinder can produce a maximum error of 1cm at the
> equator as they use 32 bit ints for the data. It might actually be better
> to use the gpkg file, but it is double in size. I will investigate using
> the gpkg data.
>
> You may want to try a shapefile too, just in case...!
>
> Nyall
>
>
> >
> > It seems to me that if a plugin requires a certain dataset to function
> then that data should be included with it and not require an additional
> download, but I understand the issues that this could cause so I am not
> sure of the best way forward. For our use it is definitely better to
> include the data with the plugin.
> >
> > I guess what I would like you to take from this conversation is that
> some users have a completely different environment and face different
> challenges then what you do when it comes to dealing with software. I wish
> that it was easier for me to get QGIS acceptance in our workforce but
> ArcGIS still rules.
> >
> > Thanks for your ideas and all of the hard work that you do to keep QGIS
> moving forward.
> >
> > I wish you all the very best!
> >
> > Calvin
> >
> >> Given that the size of the python library is almost entirely the size
> >> of the timezone boundaries themselves, have you considered:
> >> - avoiding the library entirely, and insteading using a standard
> >> shapefile/gpkg/... of the boundaries and using QGIS vector layer
> >> methods to determine the timezone for a point
> >> - deferring the download of the boundary spatial data, so that it's
> >> not supplied with the plugin but instead the plugin automatically
> >> downloads it on first launch?
> >>
> >> This would avoid the need for the large size plugin, allowing it to be
> >> supplied via the standard QGIS repo while still providing its full
> >> functionality...
> >>
> >> Nyall
> >>
> >>
>
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Nyall Dawson]

On Fri, 29 Jan 2021 at 01:13, C Hamilton  wrote:
>
> Nyall,
>
> What you mention has merit, but I was not kidding about trying to keep a 
> plugin self contained for government use. To set up automatic downloading on 
> our networks would require PKI authentication and that opens up another can 
> of worms. I know how to deal with PKI because I already maintain plugins that 
> use authentication. It would be easier for our users to have them download a 
> data pack and have some manual installation process, but it is even easier if 
> the data is already a part of the plugin. The other option would be to 
> produce two different versions of the plugin. One for government use and the 
> other for community use, but my sponsor would probably frown on that and not 
> want me to use my time to maintain the community version.
>
> The author of timezonefinder has done a number of optimizations to provide 
> fast lookups. See
>
> https://timezonefinder.readthedocs.io/en/latest/3_about.html
>
> I wonder what the difference in time would be to use a gpkg and QGIS point in 
> polygon look up vs timezonefinder lookup. Will it be faster or slower? I'm 
> not sure without testing. What are your thoughts? I would choose the faster 
> of the two.

I'd certainly hope a dedicated GIS application can do this faster than
a random Python library :D

> I just converted the timezone data to a gpkg and its size is 102Mb. The data 
> size for timezonefinder using the same data set is 49Mb. The optimization in 
> timezonefinder can produce a maximum error of 1cm at the equator as they use 
> 32 bit ints for the data. It might actually be better to use the gpkg file, 
> but it is double in size. I will investigate using the gpkg data.

You may want to try a shapefile too, just in case...!

Nyall


>
> It seems to me that if a plugin requires a certain dataset to function then 
> that data should be included with it and not require an additional download, 
> but I understand the issues that this could cause so I am not sure of the 
> best way forward. For our use it is definitely better to include the data 
> with the plugin.
>
> I guess what I would like you to take from this conversation is that some 
> users have a completely different environment and face different challenges 
> then what you do when it comes to dealing with software. I wish that it was 
> easier for me to get QGIS acceptance in our workforce but ArcGIS still rules.
>
> Thanks for your ideas and all of the hard work that you do to keep QGIS 
> moving forward.
>
> I wish you all the very best!
>
> Calvin
>
>> Given that the size of the python library is almost entirely the size
>> of the timezone boundaries themselves, have you considered:
>> - avoiding the library entirely, and insteading using a standard
>> shapefile/gpkg/... of the boundaries and using QGIS vector layer
>> methods to determine the timezone for a point
>> - deferring the download of the boundary spatial data, so that it's
>> not supplied with the plugin but instead the plugin automatically
>> downloads it on first launch?
>>
>> This would avoid the need for the large size plugin, allowing it to be
>> supplied via the standard QGIS repo while still providing its full
>> functionality...
>>
>> Nyall
>>
>>
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[David Strip]

  
  
On 1/28/2021 11:10 AM, C Hamilton
  wrote:

In
  thinking about it, if the data set is too large to include with
  QGIS, it might be worth having a simplified geometry data set
  included and if the user wants more precise data then they can
  download it and use it instead.
I was wondering this same question when this arrived in my mailbox.
It seems to me that one could achieve a very substantial reduction
in size (>90%?) while creating only a very small increase in the
error rate, as it seems these will only occur in areas where
boundaries contain very small scale details. I'll admit that have
not attempted an empirical test of this conjecture. 

  

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Richard Duivenvoorde]

On 1/28/21 7:10 PM, C Hamilton wrote:
> In thinking about it, if the data set is too large to include with QGIS, it 
> might be worth having a simplified geometry data set included and if the user 
> wants more precise data then they can download it and use it instead.

I downloaded the shape-with-oceans (ah and THAT one has those rectangles I 
talked about earlier :-) ): 
101Mb after exporting to geopackage, 68Mb after zipping that... 
For users sake: either add the 68Mb (Juergen? Others? Opinion) or (easier) let 
Tim put your plugin (including this data/module) on plugins.qgis.org

Regards,

Richard Duivenvoorde

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[C Hamilton]

In thinking about it, if the data set is too large to include with QGIS, it
might be worth having a simplified geometry data set included and if the
user wants more precise data then they can download it and use it instead.

On Thu, Jan 28, 2021 at 10:22 AM Richard Duivenvoorde 
wrote:

> On 1/28/21 4:13 PM, C Hamilton wrote:
> > What you mention has merit, but I was not kidding about trying to keep a
> plugin self contained for government use. To set up automatic downloading
> on our networks would require PKI authentication and that opens up another
> can of worms. I know how to deal with PKI because I already
> maintain plugins that use authentication. It would be easier for our users
> to have them download a data pack and have some manual
> installation process, but it is even easier if the data is already a part
> of the plugin. The o
>
> Hi Calvin,
>
> What I 'think' Nyall meant was to add such data to QGIS itself? We already
> have a gpkg with the world countries on board (everybody know the easter
> egg: 'type world in the coordinate input' yes?).
> But... adding 102Mb to the QGIS installer is maybe too much :-) Or can you
> shrink that down to some smaller size/format?
> Also, not sure in which version this will show up then (and on what
> ancient QGIS version your government client is).
>
> If all fails, I'd accept Tim's proposal to get it uploaded on
> plugins.qgis.org via the backdoor/Tim...
>
> Regards,
>
> Richard Duivenvoorde
>
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[C Hamilton]

Richard,

If the map is simplified, then you lose precision, but it would be
wonderful if a gpkg of the time zones could be added to QGIS. Here is the
time zone map data.

https://github.com/evansiroky/timezone-boundary-builder

Calvin






On Thu, Jan 28, 2021 at 10:22 AM Richard Duivenvoorde 
wrote:

> On 1/28/21 4:13 PM, C Hamilton wrote:
> > What you mention has merit, but I was not kidding about trying to keep a
> plugin self contained for government use. To set up automatic downloading
> on our networks would require PKI authentication and that opens up another
> can of worms. I know how to deal with PKI because I already
> maintain plugins that use authentication. It would be easier for our users
> to have them download a data pack and have some manual
> installation process, but it is even easier if the data is already a part
> of the plugin. The o
>
> Hi Calvin,
>
> What I 'think' Nyall meant was to add such data to QGIS itself? We already
> have a gpkg with the world countries on board (everybody know the easter
> egg: 'type world in the coordinate input' yes?).
> But... adding 102Mb to the QGIS installer is maybe too much :-) Or can you
> shrink that down to some smaller size/format?
> Also, not sure in which version this will show up then (and on what
> ancient QGIS version your government client is).
>
> If all fails, I'd accept Tim's proposal to get it uploaded on
> plugins.qgis.org via the backdoor/Tim...
>
> Regards,
>
> Richard Duivenvoorde
>
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Richard Duivenvoorde]

On 1/28/21 4:13 PM, C Hamilton wrote:
> What you mention has merit, but I was not kidding about trying to keep a 
> plugin self contained for government use. To set up automatic downloading on 
> our networks would require PKI authentication and that opens up another can 
> of worms. I know how to deal with PKI because I already maintain plugins that 
> use authentication. It would be easier for our users to have them download a 
> data pack and have some manual installation process, but it is even easier if 
> the data is already a part of the plugin. The o

Hi Calvin, 

What I 'think' Nyall meant was to add such data to QGIS itself? We already have 
a gpkg with the world countries on board (everybody know the easter egg: 'type 
world in the coordinate input' yes?).
But... adding 102Mb to the QGIS installer is maybe too much :-) Or can you 
shrink that down to some smaller size/format?
Also, not sure in which version this will show up then (and on what ancient 
QGIS version your government client is).

If all fails, I'd accept Tim's proposal to get it uploaded on plugins.qgis.org 
via the backdoor/Tim...

Regards,

Richard Duivenvoorde
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[C Hamilton]

Nyall,

What you mention has merit, but I was not kidding about trying to keep a
plugin self contained for government use. To set up automatic downloading
on our networks would require PKI authentication and that opens up another
can of worms. I know how to deal with PKI because I already
maintain plugins that use authentication. It would be easier for our users
to have them download a data pack and have some manual
installation process, but it is even easier if the data is already a part
of the plugin. The other option would be to produce two different versions
of the plugin. One for government use and the other for community use, but
my sponsor would probably frown on that and not want me to use my time to
maintain the community version.

The author of timezonefinder has done a number of optimizations to provide
fast lookups. See

https://timezonefinder.readthedocs.io/en/latest/3_about.html

I wonder what the difference in time would be to use a gpkg and QGIS point
in polygon look up vs timezonefinder lookup. Will it be faster or slower?
I'm not sure without testing. What are your thoughts? I would choose the
faster of the two.

I just converted the timezone data to a gpkg and its size is 102Mb. The
data size for timezonefinder using the same data set is 49Mb. The
optimization in timezonefinder can produce a maximum error of 1cm at the
equator as they use 32 bit ints for the data. It might actually be better
to use the gpkg file, but it is double in size. I will investigate using
the gpkg data.

It seems to me that if a plugin requires a certain dataset to function then
that data should be included with it and not require an
additional download, but I understand the issues that this could cause so I
am not sure of the best way forward. For our use it is definitely better to
include the data with the plugin.

I guess what I would like you to take from this conversation is that some
users have a completely different environment and face different challenges
then what you do when it comes to dealing with software. I wish that it was
easier for me to get QGIS acceptance in our workforce but ArcGIS still
rules.

Thanks for your ideas and all of the hard work that you do to keep QGIS
moving forward.

I wish you all the very best!

Calvin

Given that the size of the python library is almost entirely the size
> of the timezone boundaries themselves, have you considered:
> - avoiding the library entirely, and insteading using a standard
> shapefile/gpkg/... of the boundaries and using QGIS vector layer
> methods to determine the timezone for a point
> - deferring the download of the boundary spatial data, so that it's
> not supplied with the plugin but instead the plugin automatically
> downloads it on first launch?
>
> This would avoid the need for the large size plugin, allowing it to be
> supplied via the standard QGIS repo while still providing its full
> functionality...
>
> Nyall
>
>
>
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Tim Sutton]

Hi

On Wed, Jan 27, 2021 at 11:38 PM Nyall Dawson 
wrote:

> On Thu, 28 Jan 2021 at 00:32, C Hamilton  wrote:
> >
> > Tim,
> >
> > I understand the position you are in and I really struggled in even
> wanting to put in the time to develop this plugin knowing that there would
> probably be a problem getting it accepted by the QGIS developers because of
> its size, but our users have a very real need for this capability which is
> why I consented to develop it at all.
> >
> > In order to deal with locating time zones based on a coordinate there is
> simply no way to get around the size of the python library required if
> accuracy of the results is desired. That feature alone requires 42Mb. I
> struggle to convince our GIS analysts to use QGIS over ArcGIS and so I try
> to provide solutions that work better than those in ArcGIS, but if there is
> any sort of complication that makes it more difficult for one of the
> analysts to get the resources they need for QGIS then I have lost the
> battle. This includes having them pip install additional python libraries
> because most of them do not have system admin privileges.
>
> Given that the size of the python library is almost entirely the size
> of the timezone boundaries themselves, have you considered:
> - avoiding the library entirely, and insteading using a standard
> shapefile/gpkg/... of the boundaries and using QGIS vector layer
> methods to determine the timezone for a point
> - deferring the download of the boundary spatial data, so that it's
> not supplied with the plugin but instead the plugin automatically
> downloads it on first launch?
>
> This would avoid the need for the large size plugin, allowing it to be
> supplied via the standard QGIS repo while still providing its full
> functionality...
>

So Chris, try Nyall's approach and if that does not work for you, pop a
note here and we will see if we can make a local exception without
generally increasing the size limit for plugins.

Regards

Tim



>
> Nyall
>
>
> >
> > I can easily set up the ability for users to point to a separate repo
> that houses this plugin, but once again that is an additional complication.
> Every once in a while I find some QGIS repo of plugins that are not a part
> of the official QGIS repo and I think that it is too bad that they are
> probably not used much outside of the organization who hosts them. Unless a
> plugin exists on the official QGIS repo it will not be discoverable and
> will not likely be used by many.
> >
> > I think the plans I have with this plugin will make it very useful to
> the QGIS community, but if it cannot be a part of the QGIS repo it will not
> be of much use to the community. Our users will make use of it because we
> will provide it to them. I would like to swap out the astral library for
> the Skyfield library with the JPL ephemeris data to provide more precise
> sun and moon calculations, but that adds an additional ~20Mb.
> >
> > I would love it if you would provide timezonefinder and Skyfield
> libraries by default with the QGIS distribution, but that increases the
> size of the QGIS download. When you are dealing with time zones and
> astronomical calculations, there is no way in getting around the plugin
> size to get the best accuracy available. I personally am not interested in
> providing tools with less accuracy.
> >
> > I don't know how important it will be for the QIGS community to have
> these tools available, but I know that there will be a number of users who
> have been looking for these capabilities in QGIS and will be valuable to
> them. Right now I have coded only one of a number of tools that I have
> planned, but part of my plans will be determined by the QGIS acceptance of
> a larger plugin. I think my other QGIS plugins I have provided including
> "Lat Lon Tools", "Shape Tools" & "KML Tools" shows how useful my plugins
> have been.
> >
> > If there is something I am missing or some other way around this let me
> know, but keep in mind I hold to two fundamental principals with my
> plugins. 1) They are as accurate at possible. 2) They are self contained
> meaning that OUR users don't need to install any software to run them. The
> functionality in Date/Time tools is simply going to take up space and there
> is no way around it that I can see. I think your "unwritten expectation" of
> small plugins prevents some capabilities from being introduced, but I also
> agree that in most instances plugins should be kept small. What I am trying
> to accomplish simply does not come small.
> >
> > Let me know if you want me to just provide the capabilities for our
> users alone or to expand it for use by the QGIS community.
> >
> > I await the QGIS developers decision on this.
> >
> > Thanks!
> >
> >>> Personally I would be more on the -1 on upping the limit for plugins -
> currently at 10mb I think - since large plugins will consume more space on
> our plugins server and our user's bandwidth. There is an unwritten
> expectation 

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Nyall Dawson]

On Thu, 28 Jan 2021 at 00:32, C Hamilton  wrote:
>
> Tim,
>
> I understand the position you are in and I really struggled in even wanting 
> to put in the time to develop this plugin knowing that there would probably 
> be a problem getting it accepted by the QGIS developers because of its size, 
> but our users have a very real need for this capability which is why I 
> consented to develop it at all.
>
> In order to deal with locating time zones based on a coordinate there is 
> simply no way to get around the size of the python library required if 
> accuracy of the results is desired. That feature alone requires 42Mb. I 
> struggle to convince our GIS analysts to use QGIS over ArcGIS and so I try to 
> provide solutions that work better than those in ArcGIS, but if there is any 
> sort of complication that makes it more difficult for one of the analysts to 
> get the resources they need for QGIS then I have lost the battle. This 
> includes having them pip install additional python libraries because most of 
> them do not have system admin privileges.

Given that the size of the python library is almost entirely the size
of the timezone boundaries themselves, have you considered:
- avoiding the library entirely, and insteading using a standard
shapefile/gpkg/... of the boundaries and using QGIS vector layer
methods to determine the timezone for a point
- deferring the download of the boundary spatial data, so that it's
not supplied with the plugin but instead the plugin automatically
downloads it on first launch?

This would avoid the need for the large size plugin, allowing it to be
supplied via the standard QGIS repo while still providing its full
functionality...

Nyall


>
> I can easily set up the ability for users to point to a separate repo that 
> houses this plugin, but once again that is an additional complication. Every 
> once in a while I find some QGIS repo of plugins that are not a part of the 
> official QGIS repo and I think that it is too bad that they are probably not 
> used much outside of the organization who hosts them. Unless a plugin exists 
> on the official QGIS repo it will not be discoverable and will not likely be 
> used by many.
>
> I think the plans I have with this plugin will make it very useful to the 
> QGIS community, but if it cannot be a part of the QGIS repo it will not be of 
> much use to the community. Our users will make use of it because we will 
> provide it to them. I would like to swap out the astral library for the 
> Skyfield library with the JPL ephemeris data to provide more precise sun and 
> moon calculations, but that adds an additional ~20Mb.
>
> I would love it if you would provide timezonefinder and Skyfield libraries by 
> default with the QGIS distribution, but that increases the size of the QGIS 
> download. When you are dealing with time zones and astronomical calculations, 
> there is no way in getting around the plugin size to get the best accuracy 
> available. I personally am not interested in providing tools with less 
> accuracy.
>
> I don't know how important it will be for the QIGS community to have these 
> tools available, but I know that there will be a number of users who have 
> been looking for these capabilities in QGIS and will be valuable to them. 
> Right now I have coded only one of a number of tools that I have planned, but 
> part of my plans will be determined by the QGIS acceptance of a larger 
> plugin. I think my other QGIS plugins I have provided including "Lat Lon 
> Tools", "Shape Tools" & "KML Tools" shows how useful my plugins have been.
>
> If there is something I am missing or some other way around this let me know, 
> but keep in mind I hold to two fundamental principals with my plugins. 1) 
> They are as accurate at possible. 2) They are self contained meaning that OUR 
> users don't need to install any software to run them. The functionality in 
> Date/Time tools is simply going to take up space and there is no way around 
> it that I can see. I think your "unwritten expectation" of small plugins 
> prevents some capabilities from being introduced, but I also agree that in 
> most instances plugins should be kept small. What I am trying to accomplish 
> simply does not come small.
>
> Let me know if you want me to just provide the capabilities for our users 
> alone or to expand it for use by the QGIS community.
>
> I await the QGIS developers decision on this.
>
> Thanks!
>
>>> Personally I would be more on the -1 on upping the limit for plugins - 
>>> currently at 10mb I think - since large plugins will consume more space on 
>>> our plugins server and our user's bandwidth. There is an unwritten 
>>> expectation that plugins should be small and quick to fetch. If others feel 
>>> strongly with a contrary opinion, contact me offlist and I will help you 
>>> get it published without generally changing the limit.
>>
>>
>> Regard
>>
>> Tim
>>
>> ___
>> Qgis-user 

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[C Hamilton]

Tim,

I understand the position you are in and I really struggled in even wanting
to put in the time to develop this plugin knowing that there would probably
be a problem getting it accepted by the QGIS developers because of its
size, but our users have a very real need for this capability which is why
I consented to develop it at all.

In order to deal with locating time zones based on a coordinate there is
simply no way to get around the size of the python library required if
accuracy of the results is desired. That feature alone requires 42Mb. I
struggle to convince our GIS analysts to use QGIS over ArcGIS and so I try
to provide solutions that work better than those in ArcGIS, but if there is
any sort of complication that makes it more difficult for one of the
analysts to get the resources they need for QGIS then I have lost the
battle. This includes having them pip install additional python libraries
because most of them do not have system admin privileges.

I can easily set up the ability for users to point to a separate repo that
houses this plugin, but once again that is an additional complication.
Every once in a while I find some QGIS repo of plugins that are not a part
of the official QGIS repo and I think that it is too bad that they are
probably not used much outside of the organization who hosts them. Unless a
plugin exists on the official QGIS repo it will not be discoverable and
will not likely be used by many.

I think the plans I have with this plugin will make it very useful to the
QGIS community, but if it cannot be a part of the QGIS repo it will not be
of much use to the community. Our users will make use of it because we will
provide it to them. I would like to swap out the astral library for the
Skyfield library with the JPL ephemeris data to provide more precise sun
and moon calculations, but that adds an additional ~20Mb.

I would love it if you would provide timezonefinder and Skyfield libraries
by default with the QGIS distribution, but that increases the size of the
QGIS download. When you are dealing with time zones and astronomical
calculations, there is no way in getting around the plugin size to get the
best accuracy available. I personally am not interested in providing tools
with less accuracy.

I don't know how important it will be for the QIGS community to have these
tools available, but I know that there will be a number of users who have
been looking for these capabilities in QGIS and will be valuable to them.
Right now I have coded only one of a number of tools that I have planned,
but part of my plans will be determined by the QGIS acceptance of a larger
plugin. I think my other QGIS plugins I have provided including "Lat Lon
Tools", "Shape Tools" & "KML Tools" shows how useful my plugins have been.

If there is something I am missing or some other way around this let me
know, but keep in mind I hold to two fundamental principals with my
plugins. 1) They are as accurate at possible. 2) They are self contained
meaning that OUR users don't need to install any software to run them. The
functionality in Date/Time tools is simply going to take up space and there
is no way around it that I can see. I think your "unwritten expectation" of
small plugins prevents some capabilities from being introduced, but I also
agree that in most instances plugins should be kept small. What I am trying
to accomplish simply does not come small.

Let me know if you want me to just provide the capabilities for our users
alone or to expand it for use by the QGIS community.

I await the QGIS developers decision on this.

Thanks!

Personally I would be more on the -1 on upping the limit for plugins -
>> currently at 10mb I think - since large plugins will consume more space on
>> our plugins server and our user's bandwidth. There is an unwritten
>> expectation that plugins should be small and quick to fetch. If others feel
>> strongly with a contrary opinion, contact me offlist and I will help you
>> get it published without generally changing the limit.
>>
>
> Regard
>
> Tim
>
>> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Tim Sutton]

Hi




On Fri, 22 Jan 2021, 09:39 Richard Duivenvoorde, 
wrote:

> On 1/21/21 4:22 PM, C Hamilton wrote:
>

> I would enjoy any suggestions or feedback. Also for the QGIS team, can a
> plugin this large be included into the plugin repo?
>
> If you promise not to release once a week, personally I would be ok with
> it :-)
>

Personally I would be more on the -1 on upping the limit for plugins -
currently at 10mb I think - since large plugins will consume more space on
our plugins server and our user's bandwidth. There is an unwritten
expectation that plugins should be small and quick to fetch. If others feel
strongly with a contrary opinion, contact me offlist and I will help you
get it published without generally changing the limit.

Regard

Tim

>
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Richard Duivenvoorde]

On 1/22/21 3:54 PM, C Hamilton wrote:

> And upon restart I do not get the message anymore... So apparently 
> 'Plugin Dependencies Manager' is not working ok the first time (I started a 
> fresh profiles-path AND thus profile)
> 
> 
> I am not sure I correctly set up the plugin_dependencies tag in metadata.txt. 
> I couldn't find any examples. Here is what I have in metadata.txt. 

I had a look into the code, and concluded that is meant to be used for 
inter-plugin dependencies.
Then I googled:
https://github.com/qgis/QGIS-Enhancement-Proposals/issues/132
so I think you can remove it from your metadata.txt (unless Alessandro maybe 
can reveal a new trick from his sleeve about Python module dependencies :-) )

> One remark: we (as The Netherlands, are currently UTC+1 (CET), but If I 
> click just outside the north of NL (Waddenzee) outside of the xx-miles zone, 
> I get +0
> As if the timeline is very east of NL there? Scandinavia is also +1 and 
> is on the same degree.. more or less
> 
> Time zone boundaries are not in bands except in the ocean and even there they 
> may shift one way or the other around Islands. They are based on political 
> and geographical boundaries. Take a look at: 
> https://github.com/evansiroky/timezone-boundary-builder 
>   

Ah, clear (I think). I loaded that shapefile and see that the data is more or 
less a buffer around the countries/islands.
I thought more that is was a combinaties of areas shaped as long lines, like 
the colored image attached.

When you click in the North sea just above The Netherlands, you end up with 
UTC. but that is probably ok as maybe a timezone is/should be undefined if you 
do not click on land?
 
> Is it possible to visualize the time-lines?
> 
> 
> What do you mean by visualize the time-lines? I am planning on showing the 
> timezone bounding box when you click on the map.

As said: I thought there were lines crossing the oceans... but live is never 
that easy :-)

Regards,

Richard Duivenvoorde
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[C Hamilton]

>
>
> > For those who use the OSGeo4W installer you may need to include the
> scipy library. The standalone installers already have it by default.
>
> I'm on Debian, and installed the plugin.
>
> I'm shown the 'Plugin Dependencies Manager' dialog (which I've never seen
> before, is it in master or in the plugin?) See screenshot. But I cannot
> click 'Fix manually' or so...
>
> But I have scipy  installed:
>
> python3-scipy/testing,testing,now 1.5.4-1+b1 amd64 [installed,automatic]
>   scientific tools for Python 3
>
> and in QGIS I can even run the scipy-get started:
>
> import numpy as np
> print("I like ", np.pi)
>
> Ah, and after the installation, the plugin is working fine (I think)
>
> And upon restart I do not get the message anymore... So apparently 'Plugin
> Dependencies Manager' is not working ok the first time (I started a fresh
> profiles-path AND thus profile)
>

I am not sure I correctly set up the plugin_dependencies tag in
metadata.txt. I couldn't find any examples. Here is what I have in
metadata.txt.

plugin_dependencies=scipy

The Windows standalone versions have numpy and scipy already installed and
my OSGeo4W has numpy installed by default but not scipy.


>
> One remark: we (as The Netherlands, are currently UTC+1 (CET), but If I
> click just outside the north of NL (Waddenzee) outside of the xx-miles
> zone, I get +0
> As if the timeline is very east of NL there? Scandinavia is also +1 and is
> on the same degree.. more or less


Time zone boundaries are not in bands except in the ocean and even there
they may shift one way or the other around Islands. They are based on
political and geographical boundaries. Take a look at:
https://github.com/evansiroky/timezone-boundary-builder


>

Is it possible to visualize the time-lines?
>

What do you mean by visualize the time-lines? I am planning on showing the
timezone bounding box when you click on the map.


> > I would enjoy any suggestions or feedback. Also for the QGIS team, can a
> plugin this large be included into the plugin repo?
>
> If you promise not to release once a week, personally I would be ok with
> it :-)
>

Noted. Once I get more feedback back then I will create a new release and
make it available and not update it every week.
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] New QGIS Date/Time Tools Plugin

[Richard Duivenvoorde]

On 1/21/21 4:22 PM, C Hamilton wrote:
> For those who use the OSGeo4W installer you may need to include the scipy 
> library. The standalone installers already have it by default.

I'm on Debian, and installed the plugin.

I'm shown the 'Plugin Dependencies Manager' dialog (which I've never seen 
before, is it in master or in the plugin?) See screenshot. But I cannot click 
'Fix manually' or so...

But I have scipy  installed:

python3-scipy/testing,testing,now 1.5.4-1+b1 amd64 [installed,automatic]
  scientific tools for Python 3

and in QGIS I can even run the scipy-get started:

import numpy as np
print("I like ", np.pi)

Ah, and after the installation, the plugin is working fine (I think)

And upon restart I do not get the message anymore... So apparently 'Plugin 
Dependencies Manager' is not working ok the first time (I started a fresh 
profiles-path AND thus profile)

One remark: we (as The Netherlands, are currently UTC+1 (CET), but If I click 
just outside the north of NL (Waddenzee) outside of the xx-miles zone, I get +0 
As if the timeline is very east of NL there? Scandinavia is also +1 and is on 
the same degree.. more or less
Is it possible to visualize the time-lines?

> I would enjoy any suggestions or feedback. Also for the QGIS team, can a 
> plugin this large be included into the plugin repo?

If you promise not to release once a week, personally I would be ok with it :-)

Regards,

Richard Duivenvoorde
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user