Re: units issues
On 2017-05-24 20:07, Doug Henderson wrote:
> , On 24 May 2017 at 18:16, Brian Inglis wrote:
>> I would appreciate it if anyone has any experience with, or can point me
>> to, any python3 code in any Cygwin package which works Windows proxy
>> authentication?
> I had a glance at the source. I recall adapting it to load SQL tables
> with units data. I recall writing code very similar to that in the
> units_cur{2|3} python sources to keep the table current.
> units_cur2 is the python2 version, units_cur3 is the python3 version
> of the updater.
> It is using the urllib and ElementTree packages which are parts of the
> standard python release.
> A possibly more featureful replacement for urllib is the requests
> package, available in the cygwin packages python2-requests and
> python3-requests. See http://docs.python-requests.org/en/master/ for
> the documentation. It claims "HTTP(S) Proxy Support" however I have
> not used that feature.
> There are two similar lines that use urllib:
> ... urllib.request.urlopen('URL1>') ...
> ... urllib.request.urlopen('?f=xml') ...
> Using requests, they become:
> ... requests.get('').text ...
> ... requests.get('', params={'f':'xml'}).text ...
> Some additional code is required to support proxies, but I can't help
> much there.
> See http://docs.python-requests.org/en/master/user/advanced/#proxies for info.
Hi Doug,
Thanks for the pointers which document things better than urllib.
It looks like it should be possible to support proxies by setting
environment variables:
{HTTP,HTTPS}_PROXY={http,https,socks5}://user:password@host:port/
I will check this out further to see whether that requires urllib2,
urllib3, or requests, which are packaged for cygwin, and follow up with
the OP on the issue, to see what works.
--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
Re: units issues
, On 24 May 2017 at 18:16, Brian Inglis wrote:
> I would appreciate it if anyone has any experience with, or can point me
> to, any python3 code in any Cygwin package which works Windows proxy
> authentication?
I had a glance at the source. I recall adapting it to load SQL tables
with units data. I recall writing code very similar to that in the
units_cur{2|3} python sources to keep the table current.
units_cur2 is the python2 version, units_cur3 is the python3 version
of the updater.
It is using the urllib and ElementTree packages which are parts of the
standard python release.
A possibly more featureful replacement for urllib is the requests
package, available in the cygwin packages python2-requests and
python3-requests. See http://docs.python-requests.org/en/master/ for
the documentation. It claims "HTTP(S) Proxy Support" however I have
not used that feature.
There are two similar lines that use urllib:
... urllib.request.urlopen('URL1>') ...
... urllib.request.urlopen('?f=xml') ...
Using requests, they become:
... requests.get('').text ...
... requests.get('', params={'f':'xml'}).text ...
Some additional code is required to support proxies, but I can't help
much there.
See http://docs.python-requests.org/en/master/user/advanced/#proxies for info.
HTH
Doug
--
Doug Henderson, Calgary, Alberta, Canada - from gmail.com
Re: units issues
On 2017-05-24 12:13, Achim Gratz wrote: > Brian Inglis writes: >> Updating the currencies only when setup is run seems to me to be >> insufficient if users want to use current currency conversions. > > Well yes. But that means that each user should be able to update, not > just the administrator. > >> Would the best approach be to punt on running the update script at all, >> install a null /usr/share/units/currency.units file, announce and >> document that if currency conversions are desirable, the user should >> arrange to run the update script, either from the command line, profile, >> cron job, or Task Scheduler? > > In that case the file should not be packaged at all and it should live > in /usr/var somewplace, I think. But if you think a bit further you'll > see that this wouldn't be sufficient when the user cannot write there. > >> If we kept the postinstall script, we could change it to run only if the >> currency.units file is non-null and drop the time check: >> [ -s /usr/share/units/currency.units ] && /usr/bin/units_cur >> or keep the time check, using find -mtime +0, or date and stat. > > Again, if you package that updater, it'd make sense to create a > sub-package just for that and provide any customization as to which > update style is preferred (the things you mentioned are all sensible in > some situation, just not together). The longer I think about it the > less convincing the idea that a postinstalll script should do the actual > updating, although it might be one of the options you offer. The updater is part of the installed package. Running it in postinstall was added by the original cygwin packager. I am now leaning heavily towards a /usr/share/doc/Cygwin/units.README documenting the commands to automate currency updates properly, the possible approaches to implementing the commands, leaving the mechanism to the user to implement, and conspicuously referring to that doc in future upgrade notices. I will try again to contact upstream via bug-un...@gnu.org this time, and work on fixing the issues with the updater site cert expiry, by retrying with cert checking disabled, and see if I can come up with another fix to work with Windows proxy authentication. I would appreciate it if anyone has any experience with, or can point me to, any python3 code in any Cygwin package which works Windows proxy authentication? -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
Re: units issues
Brian Inglis writes: > Updating the currencies only when setup is run seems to me to be > insufficient if users want to use current currency conversions. Well yes. But that means that each user should be able to update, not just the administrator. > Would the best approach be to punt on running the update script at all, > install a null /usr/share/units/currency.units file, announce and > document that if currency conversions are desirable, the user should > arrange to run the update script, either from the command line, profile, > cron job, or Task Scheduler? In that case the file should not be packaged at all and it should live in /usr/var somewplace, I think. But if you think a bit further you'll see that this wouldn't be sufficient when the user cannot write there. > If we kept the postinstall script, we could change it to run only if the > currency.units file is non-null and drop the time check: > [ -s /usr/share/units/currency.units ] && /usr/bin/units_cur > or keep the time check, using find -mtime +0, or date and stat. Again, if you package that updater, it'd make sense to create a sub-package just for that and provide any customization as to which update style is preferred (the things you mentioned are all sensible in some situation, just not together). The longer I think about it the less convincing the idea that a postinstalll script should do the actual updating, although it might be one of the options you offer. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: units issues
Buchbinder, Barry (NIH/NIAID) [E] writes: > I would prefer that by default updates happen automatically and those > who do not want automatic updates do something to stop them from > happening. That there should be updates was never debated, only the machanism of doing so is discussed here. > For instance, someone who does not want updates makes the > definitions file read-only and have the script check for write > permission and exit or skip updating if the file cannot be written. A > zero length read-only file works if one is worried about someone using > stale conversion factors. An environmental variable whose existence > marks no-update might be another possibility. No again. In general, it is a no-no to alter or remove files that came with the package (besides, the user might not even be able to, depending on how Cygwin gets installed). Cygwin isn't very fussy about such errors at the moment, but that shouldn't be an excuse to become sloppy. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada
Re: units issues
On 2017-05-24 10:26, Buchbinder, Barry (NIH/NIAID) [E] wrote: > Brian Inglis sent the following at Tuesday, May 23, 2017 10:37 PM >> On 2017-05-23 17:55, Doug Henderson wrote: >>> On 23 May 2017 at 15:49, Brian Inglis wrote: Updating the currencies only when setup is run seems to me to be insufficient if users want to use current currency conversions. >> It's a command line utility from GNU with currency conversion factors >> in a separate definition file included from the main file, updated by >> a Python script, which downloads an RSS XML file of current (Euro) >> rates from a free source with a permissive licence, and converts it to >> definitions acceptable to the utility, overwriting the existing file. >> >> The main issues are that, as currently implemented, currency rates are >> updated automatically by a postinstall script only when setup is run; >> setup may be running in an environment without external access, so the >> postinstall script will generate an error; users may not want or care >> about currency updates; and the postinstall script uses find to avoid >> updating if there is no currency file, or it has been updated recently. >> >> One option to deal with this is update the package to install a >> zero length currency definitions file, so currency conversions are >> not defined, but the program has no issues, and drop the permanent >> postinstall script to perform updates. Then announce and document that >> users who want updated currency conversion rates need to run the update >> script from the command line, a profile script, cron job, or Windows >> Scheduled Task, as is desirable if they use currency conversions. > I would prefer that by default updates happen automatically and those > who do not want automatic updates do something to stop them from > happening. For instance, someone who does not want updates makes the > definitions file read-only and have the script check for write > permission and exit or skip updating if the file cannot be written. A > zero length read-only file works if one is worried about someone using > stale conversion factors. An environmental variable whose existence > marks no-update might be another possibility. I agree with Achim that currency updates should not be done at postinstall, as the user could be doing an automated install with no external access, and we don't know that. With no guaranteed automatic execution facility, we don't even have a mechanism, how should updates should be done automatically and how can we provide this? Alternative suggestions for automated updates are welcome and why I am asking here. For other services, we leave it to users to figure out how to start them up and shut them down before setup is run. We could install an /etc/profile.d script which asks the user first time thru if they want daily currency updates during login over the network, and then either null or update currencies, and after do or do not update currencies. We could document how updates could be run: run the command from the shell, or add the necessary code to: some profile or profile.d script; command alias or function wrapper in some profile script; script which uses units currencies, cron job, or Windows Scheduled Task. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
RE: units issues
Brian Inglis sent the following at Tuesday, May 23, 2017 10:37 PM > >On 2017-05-23 17:55, Doug Henderson wrote: > >> On 23 May 2017 at 15:49, Brian Inglis wrote: >> >>> Updating the currencies only when setup is run seems to me to be >>> insufficient if users want to use current currency conversions. >> >> Currencies needs to be split to a different package from non-currency >> units. Non-currency units is very static, whereas most currencies >> changes daily. In the absence of updates, non-currency units are still >> useful; but currencies are incorrect in most cases, and increasingly >> so as time pass. >> >> It is difficult to predict user requirements based on current package >> dependencies as non-currency units and currencies are likely used by >> almost disjoint sets of packages. Users of non-currency units should >> not be burdened by the complexities of currencies. But if they do need >> currencies, they should be up to date. >> >> A possible solution is to check the currency of the currency data on >> library initialization or each use by comparing the current date with >> the data's date of update. When the data is stale, automatically >> update it if possible, or notify the user of the problem. > >It's a command line utility from GNU with currency conversion factors >in a separate definition file included from the main file, updated by >a Python script, which downloads an RSS XML file of current (Euro) >rates from a free source with a permissive licence, and converts it to >definitions acceptable to the utility, overwriting the existing file. > >The main issues are that, as currently implemented, currency rates are >updated automatically by a postinstall script only when setup is run; >setup may be running in an environment without external access, so the >postinstall script will generate an error; users may not want or care >about currency updates; and the postinstall script uses find to avoid >updating if there is no currency file, or it has been updated recently. > >One option to deal with this is update the package to install a >zero length currency definitions file, so currency conversions are >not defined, but the program has no issues, and drop the permanent >postinstall script to perform updates. Then announce and document that >users who want updated currency conversion rates need to run the update >script from the command line, a profile script, cron job, or Windows >Scheduled Task, as is desirable if they use currency conversions. > >There are also issues with the Python update script, as the currency >source site cert expired recently, causing the update and postinstall >scripts to fail, with no workaround other than a replacement or patch; >and a poster has problems using the update script with Windows proxies, >which I have addressed to the upstream maintainer for discussion about >approaches. If there is no response, I will create a Cygwin update patch >and submit it upstream, but there has been no visible response to issues >raised on the GNU site. I would prefer that by default updates happen automatically and those who do not want automatic updates do something to stop them from happening. For instance, someone who does not want updates makes the definitions file read-only and have the script check for write permission and exit or skip updating if the file cannot be written. A zero length read-only file works if one is worried about someone using stale conversion factors. An environmental variable whose existence marks no-update might be another possibility. Best wishes, - Barry Disclaimer: Statements made herein are not made on behalf of NIAID.
