Bug#935887: python3-mapproxy: Change default template_dir and fix capabilities for Python >= 3.6.7

2019-08-27 Thread Marek Lukács 
Hi Bas,

at the moment I do not have good sandbox for playing with packaging. But I
have checked the source package of mapproxy_1.11.0-3 (Debian Buster apt).

As I have not tested it and I just read it and checked on my filesystem I
get feeling, that something is going wrong during packaging. Looks like the
debain/rules do some tweaks, move files and create symlinks and uses
../python3.7/... names. Than probably something (or task of dh) renames
.../python3.7/... to .../python3/...

I am not good in package maintenance and do not know the correct sequence
it executes during packaging.

I have the possible explanation, what happens:

For me it looks like, it moves /templates/ from
/usr/lib/python3.7/dist-packages into /usr/share/python3-mapproxy and
creates broken link in /usr/lib/python3.7/dist-packages pointing to
(system) /usr/lib/python3.7/dist-packages (I expect that the file is
missing on building machine, so it is broken link). Later dh (or whatever
it is) task which corrects it, it ignores broken links so it will not fix
it, and those broken links remains and for example on my system I see those
links really in /usr/lib/python3.7/dist-packages/mapproxy/...

But as I noticed, I have not tested it, and I do not fully understand to
packaging sequences and tools, so it is only my feeling.

Why I get this feeling:

Check mapproxy-1.11.0/debian/rules, block override_dh_auto_install and
inside this block to following part:

for V in `py3versions -v -s`; do \
for F in DejaVuSans.ttf DejaVuSansMono.ttf; do \
$(RM)
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/image/fonts/$$F
; \
ln -s /usr/share/fonts/truetype/dejavu/$$F \

debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/image/fonts/$$F
; \
done ; \
if [ ! -e
debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/service/templates
]; then \
mkdir -p
debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/service ; \
mv
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/service/templates
\

 debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/service/
; \
else \
$(RM) -r
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/service/templates
; \
fi ; \
ln -s /usr/share/python3-$(PYBUILD_NAME)/service/templates \

debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/service/templates
; \
if [ ! -e
debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/test/schemas
]; then \
mkdir -p
debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/test ; \
mv
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/test/schemas
\

 debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/test/ ; \
else \
$(RM) -r
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/test/schemas
; \
fi ; \
ln -s /usr/share/python3-$(PYBUILD_NAME)/test/schemas \

debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/test/schemas
; \
if [ ! -e
debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/test/system/fixture/cache_data
]; then \
mkdir -p
debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/test/system/fixture
; \
mv
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/test/system/fixture/cache_data
\

 
debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/test/system/fixture/
; \
else \
$(RM) -r
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/test/system/fixture/cache_data
; \
fi ; \
ln -s
/usr/share/python3-$(PYBUILD_NAME)/test/system/fixture/cache_data \

debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/test/system/fixture/cache_data
; \
done

For me it looks like it move templates from
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/service/templates
into
debian/python3-$(PYBUILD_NAME)/usr/share/python3-$(PYBUILD_NAME)/service/templates,
but it creates symlink
debian/python3-$(PYBUILD_NAME)/usr/lib/python$$V/dist-packages/mapproxy/service/templates
pointing to system path
/usr/share/python3-$(PYBUILD_NAME)/service/templates instead expected
debian/usr/share/python3-$(PYBUILD_NAME)/service/templates, so debian is
missing at the beginning ($(PYBUILD_NAME)=mapproxy, $$V=3.7 (result of
py3versions -v -s)).

During build time it creates broken symlink, as system path
/usr/share/python3-$(PYBUILD_NAME)/service/templates does not exists at the
build

Bug#935887: python3-mapproxy: Change default template_dir and fix capabilities for Python >= 3.6.7

2019-08-27 Thread Marek Lukács 
Attached is my sample configuration file I created as subset of my
production configuration. So probably it looks, like I do coordinate system
translation from JTSK to JTSK, but in fact, I use many sources in different
systems (most of them are in GLOBAL_WEBMERCATOR), and all sources I crop to
Czechia and Slovakia area only and translate it to JTSK, so I can use it in
other tools using JTSK as coordinate system.

Back to template dir issue. I have tested it, if I create symlink (which is
missing in Debian Buster package), it works without the need to modify
defaults.py, so I revert the change in defaults.py and created the missing
symlink on my system.

The capability issue: I see, the patch from upstream I post you, is not
even in Sid package, so probably it is not working even there. But the
symlink is there, so I take the patch of first issue back and ask to create
the symlink instead.

How to reproduce issues:

# apt-get install python3-mapproxy

download attachment to some testing directory and continue

$ /usr/lib/python3-mapproxy/mapproxy-util serve-develop -b 0.0.0.0:8080
mapproxy.yaml

(1) open webbrowser and point it to url http://developserver:8080/demo/

If you do not have correct templates symlink, it will fail with internal
error, after you create symlink it works

(2) next point webbrowser to
http://developserver:8080/service?REQUEST=GetCapabilities

it will end with internal error as well. If you patch those template files,
it will work.

About the package maintenance. Yes I use it daily, but for me is more
important to have it working, then new with new features. Actually I am not
so interested in testing many new features, but prefer stable and usable
package. If you do not require to test any new commit in upstream, I can
help you, but if you expect a guy who is downloading any new code and is
testing all new features, I feel I am not the right one. Times I used new
and cool things have already gone. :-)

Best regards,

Marek

On Tue, Aug 27, 2019 at 12:57 PM Bas Couwenberg  wrote:

> On 2019-08-27 12:39, Marek Lukács wrote:
> > Do you need to prepare sample configuration for you? I can do simple
> > copy
> > of my configuration with for example single layer/service.
>
> A (minimal) sample config to get a service working to reproduce the
> issues would be very helpful. Thanks in advance.
>
> I don't use mapproxy myself, I just maintain the package because it's
> also in OSGeoLive and no one who uses the package in Debian is
> contributing to the package maintenance.
>
> Perhaps you can get involved with package maintenance since you actually
> use the package.
>
> Kind Regards,
>
> Bas
>


mapproxy.yaml
Description: Binary data

Bug#935887: python3-mapproxy: Change default template_dir and fix capabilities for Python >= 3.6.7

2019-08-27 Thread Marek Lukács 
On Tue, Aug 27, 2019 at 12:23 PM Bas Couwenberg  wrote:

> Control: tags -1 + moreinfo
> Control: tags -1 - d-i
>
> On 2019-08-27 11:52, Marek Lukács wrote:
> > MapProxy does not work on Python 3.7 installs. I figured out two
> > issues,
> > one related to Debian installation (directory structure) and other to
> > upstream known issue:
> >
> > 1. Directory structure
> >
> > Debian moves templates into /usr/share/python3-mapproxy/, but default
> > template_dir variable is not changed, so Python looks for templates
> > instead in /usr/lib/python3/dist-packages/mapproxy/.
> >
> > How to fix:
> > change default directory in
> > /usr/lib/python3/dist-packages/mapproxy/config/defaults.py
>
> What is the actual issue you experience?
>

Not working, failing with error:

fatal error in demo for /demo/?
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/mapproxy/wsgiapp.py", line 193, in
__call__
resp = self.handlers[handler_name].handle(req)
  File "/usr/lib/python3/dist-packages/mapproxy/service/demo.py", line 121,
in handle
demo = self._render_template('demo/demo.html')
  File "/usr/lib/python3/dist-packages/mapproxy/service/demo.py", line 161,
in _render_template
template = get_template(template, default_inherit="demo/static.html")
  File "/usr/lib/python3/dist-packages/mapproxy/template.py", line 36, in
__call__
default_inherit=default_inherit, get_template=self)
  File
"/usr/lib/python3/dist-packages/mapproxy/util/ext/tempita/__init__.py",
line 135, in from_filename
f = open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory:
'/usr/lib/python3/dist-packages/mapproxy/service/templates/demo/demo.html'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/mapproxy/wsgiapp.py", line 193, in
__call__
resp = self.handlers[handler_name].handle(req)
  File "/usr/lib/python3/dist-packages/mapproxy/service/demo.py", line 121,
in handle
demo = self._render_template('demo/demo.html')
  File "/usr/lib/python3/dist-packages/mapproxy/service/demo.py", line 161,
in _render_template
template = get_template(template, default_inherit="demo/static.html")
  File "/usr/lib/python3/dist-packages/mapproxy/template.py", line 36, in
__call__
default_inherit=default_inherit, get_template=self)
  File
"/usr/lib/python3/dist-packages/mapproxy/util/ext/tempita/__init__.py",
line 135, in from_filename
f = open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory:
'/usr/lib/python3/dist-packages/mapproxy/service/templates/demo/demo.html'


> Do you have custom templates?
>

No


> The directories moved to /usr/share/python3-mapproxy are symlinked fro
> /usr/lib/python3/dist-packages/mapproxy:
>
>   lrwxrwxrwx root/root 0 2019-08-06 13:00
> ./usr/lib/python3/dist-packages/mapproxy/service/templates ->
> ../../../../../share/python3-mapproxy/service/templates
>   lrwxrwxrwx root/root 0 2019-08-06 13:00
> ./usr/lib/python3/dist-packages/mapproxy/test/schemas ->
> ../../../../../share/python3-mapproxy/test/schemas
>   lrwxrwxrwx root/root 0 2019-08-06 13:00
> ./usr/lib/python3/dist-packages/mapproxy/test/system/fixture/cache_data
> ->
> ../../../../../../../share/python3-mapproxy/test/system/fixture/cache_data
>
> As long as custom templates are installed in
> /usr/share/python3-mapproxy/service/templates they should work.
>

Do not have symlink in my case. Symlink can fix the issue as well.


> How can I reproduce your issue with with templates?
>

On Debian Buster install package python3-mapproxy, configure services and
start it. It will not work, as on Debian Buster (at least in my case) there
is no symlink to templates in dist-packages.


> > 2. Upstream code is not compatible with Python 3.7
> >
> > Tempia does not correctly parses function signature with recent Python
> > versions. Likely introduced with changes in Python tokenizer package
> > with 3.6.7. Use keyword args as workaround.
> >
> > This is already known bug in upstream:
> >
> https://github.com/mapproxy/mapproxy/commit/81c84d47bbc77da324c36ca718fbca7dde1c1c66
> >
> > How to fix:
> > Patch according to commit
> >
> https://github.com/mapproxy/mapproxy/commit/81c84d47bbc77da324c36ca718fbca7dde1c1c66
>
> We can cherry-pick this commit and include it as a patch, as a new
> mapproxy releases will likely take some time.
>
> How can I reproduce the issue you experience with Python 3.7?
>

Just install python3-mapproxy package, configure mapproxy and request for
XML capabilities. It will result in error:

TypeError: Extra position arguments: True at

Bug#935887: python3-mapproxy: Change default template_dir and fix capabilities for Python >= 3.6.7

2019-08-27 Thread Marek Lukács 
Package: python3-mapproxy
Version: 1.11.0-3
Severity: grave
Tags: patch d-i upstream
Justification: renders package unusable

MapProxy does not work on Python 3.7 installs. I figured out two issues,
one related to Debian installation (directory structure) and other to
upstream known issue:

1. Directory structure

Debian moves templates into /usr/share/python3-mapproxy/, but default
template_dir variable is not changed, so Python looks for templates
instead in /usr/lib/python3/dist-packages/mapproxy/.

How to fix:
change default directory in
/usr/lib/python3/dist-packages/mapproxy/config/defaults.py

2. Upstream code is not compatible with Python 3.7

Tempia does not correctly parses function signature with recent Python
versions. Likely introduced with changes in Python tokenizer package
with 3.6.7. Use keyword args as workaround.

This is already known bug in upstream:
https://github.com/mapproxy/mapproxy/commit/81c84d47bbc77da324c36ca718fbca7dde1c1c66

How to fix:
Patch according to commit
https://github.com/mapproxy/mapproxy/commit/81c84d47bbc77da324c36ca718fbca7dde1c1c66

Both issues can be fixed by following patch. The issue (1) has to be
checked on Python 2.7 release, as the word 'python3' is in the path and
I wonder, if in Python 2.7 release it is not only 'python'.

-- PATCH BEGIN --
diff -ur orig/usr/lib/python3/dist-packages/mapproxy/config/defaults.py 
pack/usr/lib/python3/dist-packages/mapproxy/config/defaults.py
--- orig/usr/lib/python3/dist-packages/mapproxy/config/defaults.py  
2017-11-20 13:25:56.0 +0100
+++ pack/usr/lib/python3/dist-packages/mapproxy/config/defaults.py  
2019-08-27 11:08:33.876870762 +0200
@@ -52,7 +52,7 @@
 log_conf = 'log.ini'
 
 # directory with mapproxy/service/templates/* files
-template_dir = None
+template_dir = '/usr/share/python3-mapproxy/service/templates'
 
 cache = dict(
 base_dir = './cache_data',
diff '--color=auto' -ur 
orig/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms100capabilities.xml
 
pack/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms100capabilities.xml
--- 
orig/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms100capabilities.xml
  2017-11-20 13:25:56.0 +0100
+++ 
pack/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms100capabilities.xml
  2019-08-27 10:51:31.792753187 +0200
@@ -66,7 +66,7 @@
 
   
 
-{{def layer_capabilities(layer, with_srs)}}
+{{def layer_capabilities(layer, with_srs=False)}}
   
 {{if layer.name}}
 {{ layer.name }}
diff '--color=auto' -ur 
orig/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms110capabilities.xml
 
pack/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms110capabilities.xml
--- 
orig/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms110capabilities.xml
  2017-11-20 13:25:56.0 +0100
+++ 
pack/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms110capabilities.xml
  2019-08-27 10:51:59.520757176 +0200
@@ -101,7 +101,7 @@
   
 {{endif}}
 
-{{def layer_capabilities(layer, with_srs)}}
+{{def layer_capabilities(layer, with_srs=False)}}
   
 {{if layer.name}}
 {{ layer.name }}
diff '--color=auto' -ur 
orig/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms111capabilities.xml
 
pack/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms111capabilities.xml
--- 
orig/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms111capabilities.xml
  2017-11-20 13:25:56.0 +0100
+++ 
pack/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms111capabilities.xml
  2019-08-27 10:52:23.824760688 +0200
@@ -113,7 +113,7 @@
   
 {{endif}}
 
-{{def layer_capabilities(layer, with_srs)}}
+{{def layer_capabilities(layer, with_srs=False)}}
   
 {{if layer.name}}
 {{ layer.name }}
diff '--color=auto' -ur 
orig/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms130capabilities.xml
 
pack/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms130capabilities.xml
--- 
orig/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms130capabilities.xml
  2017-11-20 13:25:56.0 +0100
+++ 
pack/usr/lib/python3.6/dist-packages/mapproxy/service/templates/wms130capabilities.xml
  2019-08-27 10:53:27.812770013 +0200
@@ -196,7 +196,7 @@
   {{endif}}
   
 {{endif}}
-{{def layer_capabilities(layer, with_srs)}}
+{{def layer_capabilities(layer, with_srs=False)}}
   {{py: md = bunch(default='', **layer.md)}}
 
   
diff '--color=auto' -ur 
orig/usr/lib/python3.7/dist-packages/mapproxy/service/templates/wms100capabilities.xml
 
pack/usr/lib/python3.7/dist-packages/mapproxy/service/templates/wms100capabilities.xml
--- 
orig/usr/lib/python3.7/dist-packages/mapproxy/service/templates/wms100capabilities.xml
  2017-11-20 13:25:56.0 +0100
+++ 
pack/usr/lib/python3.7/dist-packages/mapproxy/service/templates/wms100capabilities.xml
  2019-08-27 10:51:31.792753187 +0200
@@ -66,7 +66,7 @@

Bug#896952: freeradius: NT/LM password check fails, if Calling-Station-Id per user check activated

2018-04-26 Thread Marek Lukács 
Package: freeradius
Version: 3.0.12+dfsg-5+deb
Architecture: amd64

I want to have per user MAC address checking and per user VLAN
assignment. It is possible if:



1/ Requests attributes are copied into inner tunnel by adding

copy_request_to_tunnel = yes

into

eap { peap { } }

in file /etc/freeradius/3.0/mods-enabled/eap



2/ Send inner tunnel attributes to outside by adding

use_tunneled_reply = yes

into

eap { peap { } }

in file /etc/freeradius/3.0/mods-enabled/eap



3/ users are defined like:

username Cleartext-Password := "password" , Calling-Station-ID ==
"00-DE-AD-BE-EF-00"
Tunnel-Type = VLAN,
Tunnel-Medium-Type = IEEE-802,
Tunnel-Private-Group-ID = 100,
Fall-Through = Yes

in file /etc/freeradius/3.0/users



This works fine and MAC address is checked for Android devices. But if
using Windows 10 device, authentication fails with:

(7) eap_mschapv2: # Executing group from file
/etc/freeradius/3.0/sites-enabled/inner-tunnel
(7) eap_mschapv2:   authenticate {
(7) mschap: WARNING: No Cleartext-Password configured.  Cannot create
NT-Password
(7) mschap: WARNING: No Cleartext-Password configured.  Cannot create
LM-Password
(7) mschap: Creating challenge hash with username: czpzlpwd0006
(7) mschap: Client is using MS-CHAPv2
(7) mschap: ERROR: FAILED: No NT/LM-Password.  Cannot perform authentication
(7) mschap: ERROR: MS-CHAP2-Response is incorrect
(7) [mschap] = reject
(7)   } # authenticate = reject




But it happens only if Calling-Station-ID is next to
Cleartext-Password in users file. If that line does not have
Calling-Station-ID and user is defined like:

username Cleartext-Password := "password"
Tunnel-Type = VLAN,
Tunnel-Medium-Type = IEEE-802,
Tunnel-Private-Group-ID = 100,
Fall-Through = Yes

Authentication works and Windows 10 device is authenticated, but no
MAC address is checked.



My other modifications to my configurations:

1/ enabled ntdomain realms in
/etc/freeradius/3.0/sites-enabled/default and
/etc/freeradius/3.0/sites-enabled/inner-tunnel

2/ configured local DOMAIN in /etc/freeradius/3.0/proxy.conf

realm DOMAIN {
}



It looks, like mschap is using NTLM password checking if MS Windows
device is authenticating, but another method (MD5?) if it is Android
device.

It results that users with Android device can be configured including
Calling-Station-ID, but Windows devices must be configured without
Calling-Station-ID, so no MAC address checking for Windows devices.
For me it looks, like mschap NT/LM auth is not parsing correctly the
line, if there is Calling-Station-ID.

I expect, that I can use per user MAC address checking independently
on used end device, so Android and Windows users can be configured
with Calling-Station-ID.



I am using Debian GNU/Linux 9.4, kernel 4.9.0-6-amd64 #1 SMP Debian
4.9.82-1+deb9u3 (2018-03-02) x86_64 GNU/Linux and libc6
2.24-11+deb9u3.

Bug#764328: ipset: Entry timeouts are incorrectly set if used huge value

2014-10-07 Thread Marek Lukács 
Package: ipset
Version: 6.12.1-1
Severity: normal
Tags: upstream

Dear Maintainer,

If set has huge default timeout value, or you try to add entry with huge 
timeout value, ipset uses 4294967 instead on x86_64 arch.

How to reproduce:

1. Create testing set
ipset create testset hash:ip timeout 2419200

2. Add new netry and check timeout value
ipset add testset 1.2.3.4 ; ipset list testset | grep 1.2.3.4

Thank you for taking care about this.

Regards,

Marek Lukács


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org