Re: xml.etree and namespaces -- why?

[Robert Latest via Python-list]

Jon Ribbens wrote:
> That's because you *always* need to know the URI of the namespace,
> because that's its only meaningful identifier. If you assume that a
> particular namespace always uses the same prefix then your code will be
> completely broken. The following two pieces of XML should be understood
> identically:
>
> http://www.inkscape.org/namespaces/inkscape;>
>   
>
> and:
>
> http://www.inkscape.org/namespaces/inkscape;>
>   
>
> So you can see why e.get('inkscape:label') cannot possibly work, and why
> e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.

I get it. It does.

> The xml.etree author obviously knew that this was cumbersome, and
> hence you can do something like:
>
> namespaces = {'inkspace': 'http://www.inkscape.org/namespaces/inkscape'}
> element = root.find('inkspace:foo', namespaces)
>
> which will work for both of the above pieces of XML.

Makes sense. It forces me to make up my own prefixes which I can then safely
use in my code rather than relying on the xml's generator to not change their
prefixes.

BTW, I only now thought to look at what actually is at Inkscape's namespace
URI, and it turns out to be quite a nice explanation of what a namespace is and
why it looks like a URL.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: xml.etree and namespaces -- why?

[Axy via Python-list]

I have no idea why, I used to remove namespaces, following the advice 
from stackoverflow:


https://stackoverflow.com/questions/4255277/lxml-etree-xmlparser-remove-unwanted-namespace

_ns_removal_xslt_transform = etree.XSLT(etree.fromstring('''
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform;>

    

    
    
  
    
    

    
    
  
    
    

    
    
  
    
    
    
'''))

xml_doc = _ns_removal_xslt_transform(

    etree.fromstring(my_xml_data)

)


Later on, when I worked with SVG, I used BeautifulSoup.

Axy.
--
https://mail.python.org/mailman/listinfo/python-list

Re: xml.etree and namespaces -- why?

[Jon Ribbens via Python-list]

On 2022-10-19, Robert Latest  wrote:
> If the XML input has namespaces, tags and attributes with prefixes
> in the form prefix:sometag get expanded to {uri}sometag where the
> prefix is replaced by the full URI.
>
> Which means that given an Element e, I cannot directly access its attributes
> using e.get() because in order to do that I need to know the URI of the
> namespace.

That's because you *always* need to know the URI of the namespace,
because that's its only meaningful identifier. If you assume that a
particular namespace always uses the same prefix then your code will be
completely broken. The following two pieces of XML should be understood
identically:

http://www.inkscape.org/namespaces/inkscape;>
  

and:

http://www.inkscape.org/namespaces/inkscape;>
  

So you can see why e.get('inkscape:label') cannot possibly work, and why
e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.

The xml.etree author obviously knew that this was cumbersome, and
hence you can do something like:

namespaces = {'inkspace': 'http://www.inkscape.org/namespaces/inkscape'}
element = root.find('inkspace:foo', namespaces)

which will work for both of the above pieces of XML.

But unfortunately as far as I can see nobody's thought about doing the
same for attributes rather than tags.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: xml.etree and namespaces -- why?

[Axy via Python-list]

I mean, it's worth to look at BeautifulSoup source how do they do that.
With BS I work with attributes exactly as you want, and I explicitly
tell BS to use lxml parser.

Axy.

On 19/10/2022 14:25, Robert Latest via Python-list wrote:

Hi all,

For the impatient: Below the longish text is a fully self-contained Python
example that illustrates my problem.

I'm struggling to understand xml.etree's handling of namespaces. I'm trying to
parse an Inkscape document which uses several namespaces. From etree's
documentation:

 If the XML input has namespaces, tags and attributes with prefixes in the
 form prefix:sometag get expanded to {uri}sometag where the prefix is
 replaced by the full URI.

Which means that given an Element e, I cannot directly access its attributes
using e.get() because in order to do that I need to know the URI of the
namespace. So rather than doing this (see example below):

 label = e.get('inkscape:label')

I need to do this:

 label = e.get('{' + uri_inkscape_namespace + '}label')

...which is the method mentioned in etree's docs:

 One way to search and explore this XML example is to manually add the URI
 to every tag or attribute in the xpath of a find() or findall().
 [...]
 A better way to search the namespaced XML example is to create a
 dictionary with your own prefixes and use those in the search functions.

Good idea! Better yet, that dictionary or rather, its reverse, already exists,
because etree has used it to unnecessarily mangle the namespaces in the first
place. The documentation doesn't mention where it can be found, but we can
just use the 'xmlns:' attributes of the  root element to rebuild it. Or
so I thought, until I found out that etree deletes exactly these attributes
before handing the  element to the user.

I'm really stumped here. Apart from the fact that I think XML is bloated shit
anyway and has no place outside HTML, I just don't get the purpose of etree's
way of working:

1) Evaluate 'xmlns:' attributes of the  element
2) Use that info to replace the existing prefixes by {uri}
3) Realizing that using {uri} prefixes is cumbersome, suggest to
the user to build their own prefix -> uri dictionary
to undo the effort of doing 1) and 2)
4) ...but witholding exactly the information that existed in the original
document by deleting the 'xmlns:' attributes from the  tag

Why didn't they leave the whole damn thing alone? Keep  intact and keep
the attribute 'prefix:key' literally as they are. For anyone wanting to use
the {uri} prefixes (why would they) they could have thrown in a helper
function for the prefix->URI translation.

I'm assuming that etree's designers knew what they were doing in order to make
my life easier when dealing with XML. Maybe I'm missing the forest for the
trees. Can anybody enlighten me? Thanks!


 self-contained example
import xml.etree.ElementTree as ET

def test_svg(xml):
 root = ET.fromstring(xml)
 for e in root.iter():
 print(e.tag) # tags are shown prefixed with {URI}
 if e.tag.endswith('svg'):
# Since namespaces are defined inside the  tag, let's use the info
# from the 'xmlns:' attributes to undo etree's URI prefixing
 print('Element :')
 for k, v in e.items():
 print('  %s: %s' % (k, v))
# ...but alas: the 'xmlns:' attributes have been deleted by the parser

xml = '''


http://www.inkscape.org/namespaces/inkscape;
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd;
xmlns="http://www.w3.org/2000/svg;
xmlns:svg="http://www.w3.org/2000/svg;>
   
   
   
 
   

'''

if __name__ == '__main__':
 test_svg(xml)

--
https://mail.python.org/mailman/listinfo/python-list

xml.etree and namespaces -- why?

[Robert Latest via Python-list]

Hi all,

For the impatient: Below the longish text is a fully self-contained Python
example that illustrates my problem.

I'm struggling to understand xml.etree's handling of namespaces. I'm trying to
parse an Inkscape document which uses several namespaces. From etree's
documentation:

If the XML input has namespaces, tags and attributes with prefixes in the
form prefix:sometag get expanded to {uri}sometag where the prefix is
replaced by the full URI.

Which means that given an Element e, I cannot directly access its attributes
using e.get() because in order to do that I need to know the URI of the
namespace. So rather than doing this (see example below):

label = e.get('inkscape:label')

I need to do this:

label = e.get('{' + uri_inkscape_namespace + '}label')

...which is the method mentioned in etree's docs:

One way to search and explore this XML example is to manually add the URI
to every tag or attribute in the xpath of a find() or findall().
[...]
A better way to search the namespaced XML example is to create a
dictionary with your own prefixes and use those in the search functions.

Good idea! Better yet, that dictionary or rather, its reverse, already exists,
because etree has used it to unnecessarily mangle the namespaces in the first
place. The documentation doesn't mention where it can be found, but we can
just use the 'xmlns:' attributes of the  root element to rebuild it. Or
so I thought, until I found out that etree deletes exactly these attributes
before handing the  element to the user.

I'm really stumped here. Apart from the fact that I think XML is bloated shit
anyway and has no place outside HTML, I just don't get the purpose of etree's
way of working:

1) Evaluate 'xmlns:' attributes of the  element
2) Use that info to replace the existing prefixes by {uri}
3) Realizing that using {uri} prefixes is cumbersome, suggest to
   the user to build their own prefix -> uri dictionary
   to undo the effort of doing 1) and 2)
4) ...but witholding exactly the information that existed in the original
   document by deleting the 'xmlns:' attributes from the  tag

Why didn't they leave the whole damn thing alone? Keep  intact and keep
the attribute 'prefix:key' literally as they are. For anyone wanting to use
the {uri} prefixes (why would they) they could have thrown in a helper
function for the prefix->URI translation.

I'm assuming that etree's designers knew what they were doing in order to make
my life easier when dealing with XML. Maybe I'm missing the forest for the
trees. Can anybody enlighten me? Thanks!


 self-contained example
import xml.etree.ElementTree as ET

def test_svg(xml):
root = ET.fromstring(xml)
for e in root.iter():
print(e.tag) # tags are shown prefixed with {URI}
if e.tag.endswith('svg'):
# Since namespaces are defined inside the  tag, let's use the info
# from the 'xmlns:' attributes to undo etree's URI prefixing
print('Element :')
for k, v in e.items():
print('  %s: %s' % (k, v))
# ...but alas: the 'xmlns:' attributes have been deleted by the parser

xml = '''


http://www.inkscape.org/namespaces/inkscape;
   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd;
   xmlns="http://www.w3.org/2000/svg;
   xmlns:svg="http://www.w3.org/2000/svg;>
  
  
  

  

'''

if __name__ == '__main__':
test_svg(xml)
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45675] pkgutil.get_data() doesn't add subpackages to namespaces when importing

[Matt Wozniski]

New submission from Matt Wozniski :

If a module hasn't yet been imported, `pkgutil.get_data(pkg_name, data_file)` 
will import it, but when it does, it doesn't add the submodule to its parent 
package when the parent package is a PEP 420 implicit namespace package.

```
$ mkdir -p namespace/package
$ touch namespace/package/__init__.py
$ echo data >namespace/package/data_file
$ python3.10 -c 'import namespace.package, pkgutil; 
print(pkgutil.get_data("namespace.package", "data_file")); import namespace; 
print(namespace.package)'
b'data\n'

$ python3.10 -c 'import pkgutil; print(pkgutil.get_data("namespace.package", 
"data_file")); import namespace.package; import namespace; 
print(namespace.package)'
b'data\n'
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: module 'namespace' has no attribute 'package'
$
```

In this reproducer, we've got an implicit namespace package called "namespace" 
and a regular package inside it called "namespace.package". The regular package 
has a data file called "data_file".

If we import the regular package and then call pkgutil.get_data() to access the 
data file, it successfully retrieves the data file, and the module object for 
the namespace package winds up with an attribute referencing the module object 
for the regular package.

If we call pkgutil.get_data() to access the data file before importing the 
regular package, it successfully retrieves the data file, but the module object 
for the namespace package doesn't have an attribute referencing the module 
object for the regular package, even if we later do a normal import for the 
regular package.

It looks like pkgutil is importing the module when it hasn't already been 
imported (which I would expect) and adding it and any parent packages to 
sys.modules (which I would also expect), but then not adding submodules as 
attributes to their parent modules like `import` would (which seems like a bug).

--
components: Library (Lib)
messages: 405331
nosy: godlygeek, pablogsal
priority: normal
severity: normal
status: open
title: pkgutil.get_data() doesn't add subpackages to namespaces when importing
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue45675>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Stefan Behnel]

Stefan Behnel  added the comment:

The obvious work-around is to not use a default namespace. The result is just a 
visual difference, not a semantic one.

If someone wants to continue with the existing PR, I'll try to free some time 
to review any improvements.

--
versions: +Python 3.10, Python 3.11 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Daan Luttik]

Daan Luttik  added the comment:

Is there any workaround for this? this bug still seems to be present in python 
3.9.6.

--
nosy: +dtluttik
versions: +Python 3.9 -Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 
3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
type: security -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all

[Ali mazloum]

Change by Ali mazloum :


--
type:  -> security

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:


New changeset a045991f60b51636a784623dda7ad84b5b2c6b73 by Batuhan Taskaya in 
branch 'main':
bpo-42355: symtable.get_namespace() now checks whether there are multiple or 
any namespaces found (GH-23278)
https://github.com/python/cpython/commit/a045991f60b51636a784623dda7ad84b5b2c6b73


--

___
Python tracker 
<https://bugs.python.org/issue42355>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Carl Schaefer]

Change by Carl Schaefer :


--
nosy: +carlschaefer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18182] xml.dom.createElement() does not take implicit namespaces into account

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
keywords: +patch
pull_requests: +22172
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23278

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42355] symtable: get_namespace doesn't check whether if there are multiple namespaces or no namespaces at all

[Batuhan Taskaya]

New submission from Batuhan Taskaya :

>>> table = symtable.symtable("A = 1", "", "exec")
>>> table.lookup("A")

>>> table.lookup("A").get_namespace()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/lib/python3.10/symtable.py", line 312, in get_namespace
    raise ValueError("name is bound to multiple namespaces")
ValueError: name is bound to multiple namespaces
>>> table.lookup("A").get_namespaces()
()

--
assignee: BTaskaya
components: Library (Lib)
messages: 380969
nosy: BTaskaya, benjamin.peterson
priority: normal
severity: normal
status: open
title: symtable: get_namespace doesn't check whether if there are multiple 
namespaces or no namespaces at all
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42355>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> unittest discovery doesn't detect namespace packages when given 
no parameters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Friday Finking: Imports, Namespaces, Poisoning, Readability

[Rhodri James]

AFAIKT the "three" are readability, naming conflicts and source location.

On 05/06/2020 01:15, DL Neil via Python-list wrote:

- how do you like to balance these three (and any other criteria)?


Readability is king.  Or queen, if you prefer.  Anything that damages 
readability drops dramatically in its desirability.  One consequence of 
that is that I rarely use "import ... as", since as you mentioned, 
renamed libraries are a rich source of WTF moments.  I would use "import 
numpy as np" (if I ever used numpy) because it seems to be standard, but 
I wouldn't type "import pygame as pg" despite having used PyGame 
extensively in the past.


I tend to use "from xxx import yy_yy, " when I'm not importing many 
things from a library and their names are sufficiently obvious.  The 
definitions of "not many" and "sufficiently obvious" are pretty 
flexible.  In particular, I'll list more items as the qualified names 
get longer, as you noted.  Source location is a non-issue; it's trivial 
to jump to the imports at the top of the file, look it up and jump back 
again.  Or have two buffers viewing the same file; that's a technique I 
use quite a lot anyway.  If I need to know the signature, that's what 
documentation is for.


I would never rename an object I've imported using "as".  It's just not 
worth the inevitable heartache.  If that means I have namespace 
collisions, that's either a vote to use the qualified name or to change 
the name of whatever I wrote that clashes with it.


- is your preference (or selection) influenced by the facilities offered 
by your favorite editor/IDE?


Not really.  I'm an EMACS user, so any of the "fancy" IDE handiwork is 
immediately out.  The only reason you need an IDE is if your support 
tools suck (sorry, Windows users).


- does your decision differ according to whether the 'target module' is 
one of yours, from the PSL, from some third party, corporate, ...?


I'm more likely to change my own module to fit :-)

- do you prefer PSL's importlib over Python's native import-s, for one 
of these (or any other) reason?


Um, why would I?

--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Finking: Imports, Namespaces, Poisoning, Readability

[Tim Chase]

On 2020-06-05 12:15, DL Neil via Python-list wrote:
> Finking/discussion:
> 
> - how do you like to balance these three (and any other criteria)?

For most of what I do, I only ever have one such module so I'm not
trying keep multiple short-names in my head concurrently.  For me,
it's usually tkinter which I import as either "tk" or just "t".  It's
usually cases where the whole content would dump a lot of junk in my
module's namespace (like tkinter or pygame), so I like to keep the
namespacing; but I also don't want to clutter my code with long
references to it.

> - is your preference (or selection) influenced by the facilities
> offered by your favorite editor/IDE?

Not really.  I'm a vi/vim/ed guy.  While vim has some nice "go to
the definition of the thing under the cursor" (gd/gD/tags/etc),
regardless of my $EDITOR, it's not overly taxing to see "tk.thing"
and if I had any question about it, jump to the top of the file where
all the import-statements are, and see that tkinter is being imported
as tk.  It's there explicitly, unlike those abominable "from tkinter
import *" tutorials.

> - does your decision differ according to whether the 'target
> module' is one of yours, from the PSL, from some third party,
> corporate, ...?

I've not given it much thought, but I tend to use the above method
with PSL/3rd-party libraries where there's a lot of things in that
namespace; but for my own code, they're usually pretty svelte, so I
stick to "from mymodule import a, b, c" if there are lots or just an
"import mymodule" without aliasing it shorter.  If the imported names
were to grow to an unweildy size, I'd go the same route as above
("import myutils as u").

My largely irrelevant $0.02 USD, adjusted for tax and inflation,

-tkc






-- 
https://mail.python.org/mailman/listinfo/python-list

Friday Finking: Imports, Namespaces, Poisoning, Readability

[DL Neil via Python-list]

How do you prefer to balance all of the above when import-ing?


Python offers a number of options for importing modules, eg

import module, ...
from importable import object, ...

most of which can be augmented with the "as preferred_name" syntax.
(ignoring the much-reviled "*" (import everything, not private) variation)


The basic application of the as-clause, is a means of avoiding conflict 
within the namespace. Thus, if the name "vector" is already used within 
the module, we can:


from importable import vector as direction
...
go( direction )

Another common use is abbreviation - despite modern editors offering 
look-ahead guesses and superceding the typing-saver it once was; eg:


import numpy as np
from importable import vector as v

The "np" is not as good an example (for this discussion) because it is 
very commonly-used and therefore well-understood (at least amongst the 
Stats and DataSc communities). However, the "v" suffers the same fate as 
abbreviated object-names everywhere - after a while, one starts to 
forget exactly what the opaque name actually means.



In between the intense COVID-19 busy-ness, I have been attempting a 
contEd course (and perversely choosing to use Python instead of their 
chosen language) and thus am 'discovering' (PSL's) pygame library. 
Accordingly:


import pygame as pg

So, after employing that, every time I encountered the abbreviation I 
had to stop and think: 'what is this "pg" thing?'. Agreed, that's part 
of the "learning process" - and in-time, even my sluggish grey-matter 
managed to reach the point of readability (but can I assume the same of 
my reader/colleagues/course-tutor?).


However, as I started building helper functions and my own objects 
'around' the pygame library, without much thought, I followed the pattern:


import pygame as pg   # this is from PSL
import pygame_utilities as pgu# this is from my own work
...
if pgu.has_alpha( ink_color_tuple ): ...

It didn't take long before I was being stopped, to think: "where did 
"pgu" come-from?" (remember, being a good, little, boy*, all of my 
import statements are parked way-up at the top of the code-listing).



The opposite approach - specifically avoiding abbreviations and the 
readability issues they may carry, and bravely facing the challenge of 
avoiding name-clashes, might involve something like:


from pygame_plane_geometry import ( CartesianError,
GeometricNtuple,
CartesianCoordinate,
Vector,
Polygon,
...
)
...
position = CartesianCoordinate( 1, 2 )

Now, we will enjoy much-improved readability - but will have to rely 
upon the IDE's 'pop-up', to see an object's init/signature (for 
example). Thus, we face an extra concern of 'where do I find this 
module?' to be able to clarify or examine any detail.



The final 'crunch' then - to enjoy full readability AND avoid naming 
conflict AND offer full source-location information:


import pygame_plane_geometry
...
position = pygame_plane_geometry.CartesianCoordinate( 1, 2 )

Lovely stuff! Except that we're either 'banging on the door' of 
column-79, or our eyes are tiring from 'all that reading'. Maybe?



Finking/discussion:

- how do you like to balance these three (and any other criteria)?

- is your preference (or selection) influenced by the facilities offered 
by your favorite editor/IDE?


- does your decision differ according to whether the 'target module' is 
one of yours, from the PSL, from some third party, corporate, ...?


- do you prefer PSL's importlib over Python's native import-s, for one 
of these (or any other) reason?


- AOB?


WebRefs:
https://docs.python.org/3/tutorial/modules.html
https://docs.python.org/3/reference/import.html


* if you believe that, I have a bridge from Brooklyn available at an 
absolute bargain price...

(https://nycwalks.com/blog/the-brooklyn-bridge-if-you-believe-that-i-have-a-bridge-in-brooklyn-to-sell-to-you/)
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Márton Marczell]

Márton Marczell  added the comment:

Can he please get a review of the pull request?

--
nosy: +marczellm

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40104] ElementTree does not find elements in a default namespace with namespaces

[Myron Walker]

Myron Walker  added the comment:

Looks like this is fixed in the latest source code.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40104] ElementTree does not find elements in a default namespace with namespaces

[Myron Walker]

New submission from Myron Walker :

When you have an xml document like the one with a default namespace below.  
When you try to lookup nodes in the document they are not found.

```
docTree.find("specVersion")
None
```

If you add a namespaces map with the '' key and the default namespaces like:

{ '': 'urn:schemas-upnp-org:device-1-0' }

then the nodes are still not found.  The issue is that there is a case
missing from xpath_tokenizer that does not yield a pair with the default 
namespace when one is specified.  Here is a fix.

https://github.com/myronww/cpython/commit/0fc65daca239624139f2a018a83f0b0ec04a8068




```
from xml.etree.ElementTree import fromstring as parse_xml_fromstring
from xml.etree.ElementTree import ElementTree

SAMPLEXML = """

10

urn:schemas-wifialliance-org:device:WFADevice:1
R7400 (Wireless AP)



rootNode = parse_xml_fromstring(SAMPLEXML)
# Create a namespaces map like { '': 'urn:schemas-upnp-org:device-1-0' }
defaultns = {"": docNode.tag.split("}")[0][1:]}

specVerNode = docNode.find("specVersion", namespaces=defaultns)
```
Results should look like this

```
docNode.find("specVersion", namespaces=defaultns)

```

--
components: Library (Lib)
messages: 365273
nosy: Myron Walker
priority: normal
severity: normal
status: open
title: ElementTree does not find elements in a default namespace with namespaces
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40104>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset ace018ca47c03ca699603341b12781b5329d2eaa by Victor Stinner in 
branch 'master':
bpo-40050: Rephrase NEWS entry (GH-19148)
https://github.com/python/cpython/commit/ace018ca47c03ca699603341b12781b5329d2eaa


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18509
pull_request: https://github.com/python/cpython/pull/19148

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:

Since this reference leak is fixed, I reapplied Hai Shi's change for _weakref 
in bpo-1635741:

New changeset 93460d097f50db0870161a63911d61ce3c5f4583 by Victor Stinner in 
branch 'master':
bpo-1635741: Port _weakref extension module to multiphase initialization (PEP 
489) (GH-19140)
https://github.com/python/cpython/commit/93460d097f50db0870161a63911d61ce3c5f4583

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:

I close the issue.

I pushed commit 83d46e0622d2efdf5f3bf8bf8904d0dcb55fc322 which should not be 
controversial. In short, the fix is to remove two unused imports :-D

The fix doesn't remove module.__spec__ nor avoid usage of _weakref, it only fix 
this issue (reference leak) by copying code from _bootstrap.py to 
_bootstrap_external.py. In fact, modules like _io were already correctly 
imported by _bootstrap_external._setup(). Only _thread, _weakref and winreg 
imports caused this bug.

I don't think that it's worth it to backport the change to stable branches 3.7 
and 3.8, since stable branches don't use multiphase initialization for 
_weakref. The two unused imports don't cause any harm in 
importlib._bootstrap_external.


> Can I ask why you suddenly want to throw __spec__ objects out due to a leak 
> tied back to _weakref switching to multi-phase initialization?

I was thinking just aloud to try to find a solution for this reference leak.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[Brett Cannon]

Brett Cannon  added the comment:

>From Victor:
* Maybe the test_importlib should before save/restore the the "Python state" 
rather than modifying modules

You will have to be more specific than that as there is an import_state() 
context manager to control import state via the sys module.


* Maybe module.__spec__ should leak less importlib internals: explicitly clear 
namespaces? Use static methods? I'm not sure how to do that.

Are you saying change everything on __spec__ objects to be static methods? That 
won't work because the whole point of __spec__ objects is to essentially be 
data classes to store details of a module.

* Remove module.__spec__? ... that would mean rejecting PEP 451 implemented in 
Python 3.4, that sounds like a major regression :-(

No. You would break the world and undo years of work to clean up import 
semantics with no good way to go back to the old way.

Can I ask why you suddenly want to throw __spec__ objects out due to a leak 
tied back to _weakref switching to multi-phase initialization? Also note that 
the use of weakrefs by importlib isn't tied to __spec__ objects but to 
per-module import locks in importlib._bootstrap.

--

___
Python tracker 
<https://bugs.python.org/issue40050>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 83d46e0622d2efdf5f3bf8bf8904d0dcb55fc322 by Victor Stinner in 
branch 'master':
bpo-40050: Fix importlib._bootstrap_external (GH-19135)
https://github.com/python/cpython/commit/83d46e0622d2efdf5f3bf8bf8904d0dcb55fc322


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18496
pull_request: https://github.com/python/cpython/pull/19135

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:

> Before _weakref is converted to multiphase initialization:
> (...)
> => same module

That's because modules which don't use the multiphase initialization are cached 
in _PyImport_FixupExtensionObject(): see msg364914 for details. Next calls to 
_imp.create_builtin("_weakref") simply returned the cached _weakref module. 
It's loaded exactly once.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:

Quick & dirty workaround:

diff --git a/Lib/importlib/_bootstrap_external.py 
b/Lib/importlib/_bootstrap_external.py
index 7353bf9a78..d988552f2d 100644
--- a/Lib/importlib/_bootstrap_external.py
+++ b/Lib/importlib/_bootstrap_external.py
@@ -1620,7 +1620,10 @@ def _setup(_bootstrap_module):
 setattr(self_module, '_thread', thread_module)
 
 # Directly load the _weakref module (needed during bootstrap).
-weakref_module = _bootstrap._builtin_from_name('_weakref')
+if '_weakref' not in sys.modules:
+weakref_module = _bootstrap._builtin_from_name('_weakref')
+else:
+weakref_module = sys.modules['_weakref']
 setattr(self_module, '_weakref', weakref_module)
 
 # Directly load the winreg module (needed during bootstrap).


But I think that the issue is larger than just _weakref.

* Maybe the test_importlib should before save/restore the the "Python state" 
rather than modifying modules
* Maybe module.__spec__ should leak less importlib internals: explicitly clear 
namespaces? Use static methods? I'm not sure how to do that.
* Remove module.__spec__? ... that would mean rejecting PEP 451 implemented in 
Python 3.4, that sounds like a major regression :-(

--
nosy: +brett.cannon, eric.snow, ncoghlan
title: test_importlib leaked [6303, 6299, 6303] references -> importlib: 
module.__spec__ leaks  importlib namespaces (test_importlib leaked xxx 
references)

___
Python tracker 
<https://bugs.python.org/issue40050>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Phil Connell]

Change by Phil Connell :


--
nosy: +pconnell

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Maarten ter Huurne]

Maarten ter Huurne  added the comment:

Can I please get a review of the pull request?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18182] xml.dom.createElement() does not take implicit namespaces into account

[karl]

karl  added the comment:

The current specification as of today documents
https://dom.spec.whatwg.org/#dom-document-createelementns


If you run this in the browser console, 

var nsdoc = 'http://foo.bar/zoo';
var xmldoc = document.implementation.createDocument(nsdoc, 'Zoo', null);
var cpd = document.createElementNS(nsdoc, 'Compound');
var chimp = document.createElementNS(nsdoc, 'Chimp');
cpd.appendChild(chimp)
xmldoc.documentElement.appendChild(cpd);

/* serializing */
var docserializer = new XMLSerializer();
var flatxml = docserializer.serializeToString(xmldoc);
flatxml


you get:

http://foo.bar/zoo;>
  

  



but if you run this in the browser console,

var nsdoc = 'http://foo.bar/zoo';
var xmldoc = document.implementation.createDocument(nsdoc, 'Zoo', null);
var cpd = document.createElement('Compound');
var chimp = document.createElement('Chimp');
cpd.appendChild(chimp)
xmldoc.documentElement.appendChild(cpd);

/* serializing */
var docserializer = new XMLSerializer();
var flatxml = docserializer.serializeToString(xmldoc);
flatxml


you get:


http://foo.bar/zoo;>
  http://www.w3.org/1999/xhtml;>

  



which is a complete different beast.


I don't think there is an issue here. And we can close this bug safely.

--
nosy: +karlcow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Roger Gammans]

Change by Roger Gammans :


--
nosy: +rgammans

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13378] ET: add custom namespaces to serialization methods

[STINNER Victor]

STINNER Victor  added the comment:

This issue is 8 years old and has already 3 patches attached, it is not 
newcomer friendly: I remove the "easy" keyword.

--
keywords:  -easy
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Namespaces: memory vs 'pollution'

[Ethan Furman]

On 07/22/2019 08:12 PM, DL Neil wrote:

On 22/07/19 9:40 PM, Thomas Jollans wrote:



Just FYI, in the scientific Python community certain short abbreviations
are the norm. Many modules have a ‘standard’ abbreviation that most
people use, minimizing confusion.

import numpy as np
import matplotlib as mpl
from matplotlib import pyplot as plt
import pandas as pd
import xarray as xr



The use of meaningful names is a minimum professional standard (IMHO), and in 
my experience, a virtue that pays for itself. I am ever-so grateful that most 
modern text-editors will guess/read-my-mind and save the typing!


It's not just typing, but reading.  Long names, when used over and over, become 
noise.

--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Ethan Furman]

On 07/22/2019 07:27 PM, DL Neil wrote:


NameError conveys nothing to the user.
PythonVersionError is more communicative - and speaks volumes to 'us'.

The mainline code is something like:

 p = PythonEnvironment()
 try:
     p.compatibility( ...spec... )    # eg must be Py3 not 2.n
 except PythonVersionError:
     print( more illuminating errmsg )


This is why tests should check for the exact error raised, not that any error 
was raised.  (That testing bug bit me a couple times before I learned that 
lesson.)

--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Thomas Jollans]

On 23/07/2019 04.27, DL Neil wrote:
> On 23/07/19 11:00 AM, Ethan Furman wrote:
>> On 07/20/2019 05:02 PM, DL Neil wrote:
>>
>>> Upon closer inspection, I realised it didn't just fail; it failed
>>> badly! Some silly, little, boy had imported the PythonEnvironment
>>> class but failed to ALSO import PythonVersionError. So, the reported
>>> error was not the expected exception!
>>
>> I don't understand the significance of not importing PythonVersionError:
>>
>> - PythonEnvironment does not need it to be imported
>>
>> - when PythonEnvironment raises PythonVersionError you still get
>> PythonVersionError
>>
>> - if your code says `except PythonVersionError` and you haven't
>> imported it you'll get a NameError
>>
Actually, if your code says ‘except PythonVersionError’ this will only
raise a NameError if there is an actual exception.

(this was not what I expected - I learned something new today!)

If no exception is being handled, the except statements are never
examined. This makes this harder to spot than you might think!

% cat test.py   
try:
    print('ok')
except FakeError:
    print('ah shit')

% python3 test.py
ok



That makes me think...


def get_exc(exc):
    print("we're checking for exception", exc)
    return exc


try:
    print("raising no. 1")
    raise ValueError
except get_exc(TypeError):
    print("t'was a TypeError")
except get_exc(ValueError):
    print("t'was a ValueError")

try:
    print("raising no. 2")
    raise TypeError
except get_exc(TypeError):
    print("t'was a TypeError")
except get_exc(ValueError):
    print("t'was a ValueError")


>> So, what's the issue?
>
>
> Have I correctly understood the question?
>
> NameError conveys nothing to the user.
> PythonVersionError is more communicative - and speaks volumes to 'us'.
>
> The mainline code is something like:
>
> p = PythonEnvironment()
> try:
>     p.compatibility( ...spec... )    # eg must be Py3 not 2.n
> except PythonVersionError:
>     print( more illuminating errmsg )
>
> If I am 'the user' I'd be quite happy without the try...except; but
> mere mortals need/deserve something more. Accordingly, the
> PythonVersionError custom exception/class.
>
> Yes, we could trap NameError, but that might also catch other
> name-errors (unlikely in this example, but 'just sayin'). Thus the
> greater specificity of the custom class.
>
>
> NB please see alternative 'solution' proposed (for critique) as
> "Nesting Custom Errors in Classes" discussion topic.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Rhodri James]

On 23/07/2019 03:27, DL Neil wrote:

On 23/07/19 11:00 AM, Ethan Furman wrote:

On 07/20/2019 05:02 PM, DL Neil wrote:

Upon closer inspection, I realised it didn't just fail; it failed 
badly! Some silly, little, boy had imported the PythonEnvironment 
class but failed to ALSO import PythonVersionError. So, the reported 
error was not the expected exception!


I don't understand the significance of not importing PythonVersionError:

- PythonEnvironment does not need it to be imported

- when PythonEnvironment raises PythonVersionError you still get 
PythonVersionError


- if your code says `except PythonVersionError` and you haven't 
imported it you'll get a NameError


So, what's the issue?



Have I correctly understood the question?

NameError conveys nothing to the user.
PythonVersionError is more communicative - and speaks volumes to 'us'.

The mainline code is something like:

 p = PythonEnvironment()
 try:
     p.compatibility( ...spec... )    # eg must be Py3 not 2.n
 except PythonVersionError:
     print( more illuminating errmsg )

If I am 'the user' I'd be quite happy without the try...except; but mere 
mortals need/deserve something more. Accordingly, the PythonVersionError 
custom exception/class.


Ah, so you *used* PythonVersionError without importing it.  Sorry, but 
that's on you.


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[DL Neil]

On 22/07/19 9:40 PM, Thomas Jollans wrote:

On 22/07/2019 07.06, DL Neil wrote:


Current thoughts:

 import environment_module as em

- so, even more of an abbreviation than suggested!?
- I rarely need to write a long list of import statements, so there
won't be many.
- not normally using such abbreviations in my code, they will stand-out.
- considered using upper-case, eg "EM" - it is a form of constant
after-all



Just FYI, in the scientific Python community certain short abbreviations
are the norm. Many modules have a ‘standard’ abbreviation that most
people use, minimizing confusion.

import numpy as np
import matplotlib as mpl
from matplotlib import pyplot as plt
import pandas as pd
import xarray as xr

and so on.

As long as you're consistent and use the same abbreviation across your
entire codebase, and you put the imports at the top where people can
find them, I think using 2–4 letter abbreviations, even without any
decoration, is a fine approach.



+1
Thanks for this.

Wow, but my little/personal utilities are far from such 'exalted' company!

The important provision is that 'everyone' (affected) understands. As 
long as we all come from the same subject-domain, eg statisticians, then 
there's no problem with 'breaking the rules' because such abbreviations 
are easily-understood 'symbols', by definition.


After all, PEP-8 does say: "names ... [should] reflect usage"!

However, I'd never concede that to a group of 'commerce' trainees, for 
example!


The use of meaningful names is a minimum professional standard (IMHO), 
and in my experience, a virtue that pays for itself. I am ever-so 
grateful that most modern text-editors will guess/read-my-mind and save 
the typing!


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[DL Neil]

On 23/07/19 11:00 AM, Ethan Furman wrote:

On 07/20/2019 05:02 PM, DL Neil wrote:

Upon closer inspection, I realised it didn't just fail; it failed 
badly! Some silly, little, boy had imported the PythonEnvironment 
class but failed to ALSO import PythonVersionError. So, the reported 
error was not the expected exception!


I don't understand the significance of not importing PythonVersionError:

- PythonEnvironment does not need it to be imported

- when PythonEnvironment raises PythonVersionError you still get 
PythonVersionError


- if your code says `except PythonVersionError` and you haven't imported 
it you'll get a NameError


So, what's the issue?



Have I correctly understood the question?

NameError conveys nothing to the user.
PythonVersionError is more communicative - and speaks volumes to 'us'.

The mainline code is something like:

p = PythonEnvironment()
try:
p.compatibility( ...spec... )   # eg must be Py3 not 2.n
except PythonVersionError:
print( more illuminating errmsg )

If I am 'the user' I'd be quite happy without the try...except; but mere 
mortals need/deserve something more. Accordingly, the PythonVersionError 
custom exception/class.


Yes, we could trap NameError, but that might also catch other 
name-errors (unlikely in this example, but 'just sayin'). Thus the 
greater specificity of the custom class.



NB please see alternative 'solution' proposed (for critique) as "Nesting 
Custom Errors in Classes" discussion topic.

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Thomas Jollans]

On 22/07/2019 07.06, DL Neil wrote:
>
> Current thoughts:
>
> import environment_module as em
>
> - so, even more of an abbreviation than suggested!?
> - I rarely need to write a long list of import statements, so there
> won't be many.
> - not normally using such abbreviations in my code, they will stand-out.
> - considered using upper-case, eg "EM" - it is a form of constant
> after-all


Just FYI, in the scientific Python community certain short abbreviations
are the norm. Many modules have a ‘standard’ abbreviation that most
people use, minimizing confusion.

import numpy as np
import matplotlib as mpl
from matplotlib import pyplot as plt
import pandas as pd
import xarray as xr

and so on.

As long as you're consistent and use the same abbreviation across your
entire codebase, and you put the imports at the top where people can
find them, I think using 2–4 letter abbreviations, even without any
decoration, is a fine approach.

-- Thomas


> - considered adding a single under(-line) suffix, eg "em_" (on the
> basis of "made you think"). No, don't make me think (too much)!
> - so, perhaps a two-letter abbreviation with a single under(-line), eg
> "e_m", won't be confused with other naming conventions and yet
> stands-out. (sadly, that is "stands-out" to me, but 'everyone else'
> won't know its significance...)
>
> The try...except construct is a brilliant idea because it does exactly
> what I asked - requires both the class (what I wanted to include) AND
> the custom-exception class (what it needs included).
>
> If the class does not have any related error classes, then the
> try...except can simply check for 'exists?'...
>
>
> It's a habit I'm about to adopt. Many thanks! 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Ethan Furman]

On 07/20/2019 05:02 PM, DL Neil wrote:


Upon closer inspection, I realised it didn't just fail; it failed badly! Some 
silly, little, boy had imported the PythonEnvironment class but failed to ALSO 
import PythonVersionError. So, the reported error was not the expected 
exception!


I don't understand the significance of not importing PythonVersionError:

- PythonEnvironment does not need it to be imported

- when PythonEnvironment raises PythonVersionError you still get 
PythonVersionError

- if your code says `except PythonVersionError` and you haven't imported it 
you'll get a NameError

So, what's the issue?

--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[DL Neil]

On 22/07/19 5:30 AM, Roel Schroeven wrote:

DL Neil schreef op 21/07/2019 om 2:02:

How do you remember to from-import- 'everything' that is needed?
... > Upon closer inspection, I realised it didn't just fail; it 
failed badly!

Some silly, little, boy had imported the PythonEnvironment class but
failed to ALSO import PythonVersionError. So, the reported error was not
the expected exception!
...
Is there some 'easy way' to make sure that one doesn't just import the
desired class, but also remembers to import 'everything else' that might
be necessary. In this case, it'd be rather nice to:

from environment_module import Python*

NB this is a syntax error, and won't import both the PythonEnvironment
and PythonVersionError classes.

 > ...

What do you do to (respecting purism) ensure 'everything' (necessary) is
imported (and nothing more), preferably without relying upon (faulty, in
my case) human-memory or reading through volumes of code/documentation?


This is one of the advantages of using import instead of from-import.

import environment_module

...
try:
     
     # Just an example, since I don't know PythonEnvironment
     env = environment_module.PythonEnvironment()
     ...
except environment_module.PythonVersionError:
     # Handle the exception

In this case you have a pretty long module name (by the way, you could 
probably shorten it by removing _module; there's normally no need to 
repeat it in the name of a module that it's a module), making repeating 
it everywhere somewhat awkward. You can import the module using another 
name to work around that:


import environment_module as envmod

...
try:
     
     # Just an example, since I don't know PythonEnvironment
     env = envmod.PythonEnvironment()
     ...
except envmod.PythonVersionError:
     # Handle the exception



Greetings to Belgians, and thanks!

Yes, I like this (interestingly, I recall posing a question some months 
back, asking how many of us bother to check for import exceptions).



Long names: agreed, although I don't worry about it too much because 
competent editors 'pop-up' suggestions as one types. (incidentally, you 
are correct - I wouldn't really use that naming system, but inserted the 
word "module" in a bid to make the example illustrative)


The opposite is even worse (and sometimes working with statisticians I'm 
often given 'algebra' with one-letter 'variable names') - in the general 
case, I criticise non-obvious abbreviations in code-review. Yet...



Yesterday, I went 'full-circle' around the options for import statements 
(and some importlib facilities), looking at the effects on 'namespace 
pollution' and trying to find some sort of "wild-card" method. In the 
end, my conclusions were close-to, but not as well-developed as the above.



Current thoughts:

import environment_module as em

- so, even more of an abbreviation than suggested!?
- I rarely need to write a long list of import statements, so there 
won't be many.

- not normally using such abbreviations in my code, they will stand-out.
- considered using upper-case, eg "EM" - it is a form of constant after-all
- considered adding a single under(-line) suffix, eg "em_" (on the basis 
of "made you think"). No, don't make me think (too much)!
- so, perhaps a two-letter abbreviation with a single under(-line), eg 
"e_m", won't be confused with other naming conventions and yet 
stands-out. (sadly, that is "stands-out" to me, but 'everyone else' 
won't know its significance...)


The try...except construct is a brilliant idea because it does exactly 
what I asked - requires both the class (what I wanted to include) AND 
the custom-exception class (what it needs included).


If the class does not have any related error classes, then the 
try...except can simply check for 'exists?'...



It's a habit I'm about to adopt. Many thanks!
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Roel Schroeven]

DL Neil schreef op 21/07/2019 om 2:02:

How do you remember to from-import- 'everything' that is needed?
... > Upon closer inspection, I realised it didn't just fail; it failed badly!
Some silly, little, boy had imported the PythonEnvironment class but
failed to ALSO import PythonVersionError. So, the reported error was not
the expected exception!
...
Is there some 'easy way' to make sure that one doesn't just import the
desired class, but also remembers to import 'everything else' that might
be necessary. In this case, it'd be rather nice to:

from environment_module import Python*

NB this is a syntax error, and won't import both the PythonEnvironment
and PythonVersionError classes.

> ...

What do you do to (respecting purism) ensure 'everything' (necessary) is
imported (and nothing more), preferably without relying upon (faulty, in
my case) human-memory or reading through volumes of code/documentation?


This is one of the advantages of using import instead of from-import.

import environment_module

...
try:

# Just an example, since I don't know PythonEnvironment
env = environment_module.PythonEnvironment()
...
except environment_module.PythonVersionError:
# Handle the exception

In this case you have a pretty long module name (by the way, you could 
probably shorten it by removing _module; there's normally no need to 
repeat it in the name of a module that it's a module), making repeating 
it everywhere somewhat awkward. You can import the module using another 
name to work around that:


import environment_module as envmod

...
try:

# Just an example, since I don't know PythonEnvironment
env = envmod.PythonEnvironment()
...
except envmod.PythonVersionError:
# Handle the exception


--
"Honest criticism is hard to take, particularly from a relative, a
friend, an acquaintance, or a stranger."
-- Franklin P. Jones

Roel Schroeven

--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Peter J. Holzer]

On 2019-07-21 12:02:27 +1200, DL Neil wrote:
> What do you do to (respecting purism) ensure 'everything' (necessary) is
> imported (and nothing more), preferably without relying upon (faulty, in my
> case) human-memory or reading through volumes of code/documentation?

I write tests (not as consistently as I would like).

I don't think there is much more you can do. The problem is that Python
doesn't have variable declarations, so there is no reliable way to
discover an undefined symbol except by executing the line where it is
used.

An IDE might help. Even though the problem is in general not solvable,
most uses of symbols fall into a few common categories which can be
discovered by static analysis. So an IDE (or even a syntax-highlighting
editor) could flag all symbols where it can't find the definition.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Namespaces: memory vs 'pollution'

[DL Neil]

How do you remember to from-import- 'everything' that is needed?


I have a 'utility module' which contains a bunch of classes which 
examine/check/log aspects of the execution environment. One of which is 
PythonEnvironment, another relates to the HostSystem (as examples). They 
are most-frequently used to ensure that an application runs within an 
appropriate environment, eg don't use Posix code on MS-Windows and don't 
call for v3.7 features from Python3.5.


Today, I 'grabbed' the module, imported the PythonEnvironment, and 
PyTested the application code's Python version - as much for my 
amusement as anything else I asked for an impossible versionNR. Of 
course, the test failed.


Upon closer inspection, I realised it didn't just fail; it failed badly! 
Some silly, little, boy had imported the PythonEnvironment class but 
failed to ALSO import PythonVersionError. So, the reported error was not 
the expected exception!


Yes, the two classes appear in the module one-above-the-other, and yes, 
in the module's own pytest import-ing both classes is 'right there' in 
front of my nose. So, not my finest hour!



Is there some 'easy way' to make sure that one doesn't just import the 
desired class, but also remembers to import 'everything else' that might 
be necessary. In this case, it'd be rather nice to:


from environment_module import Python*

NB this is a syntax error, and won't import both the PythonEnvironment 
and PythonVersionError classes.


NBB I could import the module, fish-around in its __dict__, and do 
something like update-ing locals() with every key containing "Python" - 
but that is surely a truly, ugly, hack.



The 'lazy' answer is (purists, look away now!):

from environment_module import *

but that import-s "everything, including the kitchen sink"!

After you've recovered from the shock-horror of thinking about such 
blatant "namespace pollution", if the module only contains a bunch of 
classes, is it really such an evil act?
(and if the module's import-as-abbreviations are moved 'inside' the 
relevant class, those 'disappear' from the global namespace reducing the 
risk of name-clashes down to the names of the custom-classes and the PSL 
imports, eg os and platform)



What do you do to (respecting purism) ensure 'everything' (necessary) is 
imported (and nothing more), preferably without relying upon (faulty, in 
my case) human-memory or reading through volumes of code/documentation?


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

[issue35617] unittest discover does not work with implicit namespaces

[Terry J. Reedy]

Terry J. Reedy  added the comment:

If this is truly a duplicate, this should be closed and the issue number in the 
pr title changed to 23882.  Any reviewer should look at both proposed fixes, as 
they are not the same.  The other patch has what looks like the beginning of a 
test.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Emmanuel Arias]

Emmanuel Arias  added the comment:

Hi!

IMO its a good propose. You will have to modify some tests.

--
nosy: +eamanu
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Simon Fagerholm]

Simon Fagerholm  added the comment:

Martin: Yeah, they same to be same! Can't believe I didn't find it

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Martin Panter]

Martin Panter  added the comment:

Is this related to Issue 23882? That one is about making discovery work when 
“__init__.py” files are removed.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Eric V. Smith]

Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Simon Fagerholm]

Simon Fagerholm  added the comment:

Issue originally from SO: 
https://stackoverflow.com/questions/46976256/recursive-unittest-discovery-with-python3-and-without-init-py-files

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch, patch, patch
pull_requests: +10691, 10692, 10693
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +10691
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch, patch, patch, patch
pull_requests: +10691, 10692, 10693, 10694
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch, patch
pull_requests: +10691, 10692
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35617] unittest discover does not work with implicit namespaces

[Simon Fagerholm]

New submission from Simon Fagerholm :

When "python -m unittest discover" is run in a folder that is an implicit 
namespace package with the structure as below, no tests are discovered.
The condition that the tests must be importable from the top level directory is 
fulfilled and has been tested by importing the tests from the top level.

I did some investigating and have a PR underway that seems to fix it

Example project structure is:
.
├── requirements.txt
├── main.py
├── tests
├── unit
│   └── test_thing1.py
└── integration.py
└── test_integration_thing1.py

--
components: Library (Lib)
messages: 332748
nosy: Simon Fagerholm
priority: normal
severity: normal
status: open
title: unittest discover does not work with implicit namespaces
type: behavior
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue35617>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Maarten ter Huurne]

Maarten ter Huurne  added the comment:

I think I have a good solution now, see the pull request for details. It does 
touch a lot of code, but I split all the changes into small consistent units, 
so it should be easier to verify whether they are correct.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Maarten ter Huurne]

Change by Maarten ter Huurne :


--
pull_requests: +10286
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Maarten ter Huurne]

Maarten ter Huurne  added the comment:

I was working on what I thought would be an elegant solution to this problem: 
for non-qualified attributes, add the element's namespace before accessing the 
cache and strip the namespace prefix after accessing the cache if it's equal to 
the element's prefix.

However, this approach doesn't work: even though non-qualified attributes will 
be processed like they are the element's namespace, they are considered to have 
no namespace. This means  is considered valid XML, even 
though it effectively defines the same attribute twice.

https://www.w3.org/TR/REC-xml-names/#uniqAttrs

In my opinion the spec made a silly choice here, but that's probably not 
something that can fixed anymore.

I haven't decided yet whether I'll make another attempt at fixing this issue. 
In any case, I hope this tale of caution benefits someone.

--
nosy: +mthuurne

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Роман Донченко]

Change by Роман Донченко :


--
nosy: +SpecLad

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17088] ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used

[Rafael Ascensao]

Rafael Ascensao  added the comment:

what's the status on this?

--
nosy: +Rafael Ascensao

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How do native namespaces work?

[Peter Otten]

Thomas Nyberg wrote:

> Hello,
> 
> I'm trying to understand native namespaces. I'm currently using python
> 3.5 as packaged in debian 9. I've been following the instructions here:
> 
> https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages
> 
> Those instructions link to the following example:
> 
> https://github.com/pypa/sample-namespace-packages/tree/master/native
> 
> I presume I'm doing something silly, but I can't get that to work. I've
> tried creating a virtual environment as:
> 
> --
> $ python3 -m venv venv
> $ source venv/bin/activate
> --
> 
> And then I've gone into those repo folders and run their setup.py files
> (e.g. for namespace a):
> 
> --
> $ cd sample-namespace-packages/native/pkg_a/
> $ python3 setup.py install
> --
> 
> Then if I try to run their sample verification file, things fail:
> 
> --
> $ cat sample-namespace-packages/verify_packages.py
> from example_pkg import a
> from example_pkg import b
> 
> print(a.name)
> print(a.__path__)
> print(b.name)
> print(b.__path__)
> $ python3 sample-namespace-packages/verify_packages.py
> Traceback (most recent call last):
>   File "sample-namespace-packages/verify_packages.py", line 1, in 
> from example_pkg import a
> ImportError: No module named 'example_pkg'
> --
> 
> Things seem to be installing:
> 
> --
> $ ls venv/lib/python3.5/site-packages/
> easy-install.pth   pkg_resources
> easy_install.pypkg_resources-0.0.0.dist-info
> example_pkg_a-1-py3.5.egg  __pycache__
> example_pkg_b-1-py3.5.egg  setuptools
> pipsetuptools-32.3.1.dist-info
> pip-9.0.1.dist-info
> --
> 
> Am I missing something totally obvious here? Does anyone here see what
> I'm doing wrong? Thanks for any help!

For me it's failing in the same way. I then tried 

pip install path/to/pkg_a

which works. Comparing the site-packages directories gave

added example_pkg/a/__init__.py
added example_pkg/a/__pycache__/__init__.cpython-34.pyc
added example_pkg_a-1.egg-info/PKG-INFO
added example_pkg_a-1.egg-info/SOURCES.txt
added example_pkg_a-1.egg-info/dependency_links.txt
added example_pkg_a-1.egg-info/installed-files.txt
added example_pkg_a-1.egg-info/top_level.txt
removed easy-install.pth
removed example_pkg_a-1-py3.4.egg

i. e. it looks like the modules have to be extracted from the egg and cannot 
be imported directly even though that is what I thought the pth file was 
supposed to achieve. This is all a bit of black magic to me -- I've no idea 
if this is some kind of misconfiguration or a version problem or whatever.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How do native namespaces work?

[Thomas Nyberg]

On 10/05/2017 04:07 PM, Peter Otten wrote:
> Are you sure you are using the correct interpreter? When I activate a 
> virtual environment it changes the prompt like so:

Sorry I just cut out the extra cruft from my prompt for clarity. (In
hindsight, I should probably have left the `(venv)` prompt in my cleaned
up prompt...) The venv was definitely activated (and in fact I tried again).

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How do native namespaces work?

[Peter Otten]

Thomas Nyberg wrote:

> Hello,
> 
> I'm trying to understand native namespaces. I'm currently using python
> 3.5 as packaged in debian 9. I've been following the instructions here:
> 
> https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages
> 
> Those instructions link to the following example:
> 
> https://github.com/pypa/sample-namespace-packages/tree/master/native
> 
> I presume I'm doing something silly, but I can't get that to work. I've
> tried creating a virtual environment as:
> 
> --
> $ python3 -m venv venv
> $ source venv/bin/activate
> --
> 
> And then I've gone into those repo folders and run their setup.py files
> (e.g. for namespace a):
> 
> --
> $ cd sample-namespace-packages/native/pkg_a/
> $ python3 setup.py install

Are you sure you are using the correct interpreter? When I activate a 
virtual environment it changes the prompt like so:

$ python3 -m venv venv
$ . venv/bin/activate
(venv) $ 
(venv) $ which python3
/home/peter/venv/bin/python3
(venv) $ deactivate 
$ 
$ which python3
/usr/bin/python3

> --
> 
> Then if I try to run their sample verification file, things fail:
> 
> --
> $ cat sample-namespace-packages/verify_packages.py
> from example_pkg import a
> from example_pkg import b
> 
> print(a.name)
> print(a.__path__)
> print(b.name)
> print(b.__path__)
> $ python3 sample-namespace-packages/verify_packages.py
> Traceback (most recent call last):
>   File "sample-namespace-packages/verify_packages.py", line 1, in 
> from example_pkg import a
> ImportError: No module named 'example_pkg'
> --
> 
> Things seem to be installing:
> 
> --
> $ ls venv/lib/python3.5/site-packages/
> easy-install.pth   pkg_resources
> easy_install.pypkg_resources-0.0.0.dist-info
> example_pkg_a-1-py3.5.egg  __pycache__
> example_pkg_b-1-py3.5.egg  setuptools
> pipsetuptools-32.3.1.dist-info
> pip-9.0.1.dist-info
> --
> 
> Am I missing something totally obvious here? Does anyone here see what
> I'm doing wrong? Thanks for any help!
> 
> Cheers,
> Thomas


-- 
https://mail.python.org/mailman/listinfo/python-list

How do native namespaces work?

[Thomas Nyberg]

Hello,

I'm trying to understand native namespaces. I'm currently using python
3.5 as packaged in debian 9. I've been following the instructions here:


https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages

Those instructions link to the following example:

https://github.com/pypa/sample-namespace-packages/tree/master/native

I presume I'm doing something silly, but I can't get that to work. I've
tried creating a virtual environment as:

--
$ python3 -m venv venv
$ source venv/bin/activate
--

And then I've gone into those repo folders and run their setup.py files
(e.g. for namespace a):

--
$ cd sample-namespace-packages/native/pkg_a/
$ python3 setup.py install
--

Then if I try to run their sample verification file, things fail:

--
$ cat sample-namespace-packages/verify_packages.py
from example_pkg import a
from example_pkg import b

print(a.name)
print(a.__path__)
print(b.name)
print(b.__path__)
$ python3 sample-namespace-packages/verify_packages.py
Traceback (most recent call last):
  File "sample-namespace-packages/verify_packages.py", line 1, in 
from example_pkg import a
ImportError: No module named 'example_pkg'
--

Things seem to be installing:

--
$ ls venv/lib/python3.5/site-packages/
easy-install.pth   pkg_resources
easy_install.pypkg_resources-0.0.0.dist-info
example_pkg_a-1-py3.5.egg  __pycache__
example_pkg_b-1-py3.5.egg  setuptools
pipsetuptools-32.3.1.dist-info
pip-9.0.1.dist-info
--

Am I missing something totally obvious here? Does anyone here see what
I'm doing wrong? Thanks for any help!

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[carlosjosepita]

Hi all,

although it doesn't fit the bill 100%, I sometimes use this extremely simple 
function as a decorator:

def new(call):
return call()


For example:

@new
class MySingleton:
x = 2
y = 2
def sum(self, x, y):
return x + y


@new
def my_obj():
x = 2
y = 2
def sum(x, y):
return x + y
return Bundle(locals())


where Bundle is a simple subclass of dict implementing __xxxattr__ dunder 
methods.

Cheers
--
Carlos


 
-- 
https://mail.python.org/mailman/listinfo/python-list

A nestedmodule decorator (Re: Namespaces are one honking great idea)

[Gregory Ewing]

Steven D'Aprano wrote:

There's only so far I can go without support from the compiler.


It turns out one can go surprisingly far. Here's something I
cooked up that seems to meet almost all the requirements.
The only shortcoming I can think of is that a nestedmodule
inside another nestedmodule won't be able to see the names
in the outer nestedmodule directly (much like nested classes).

% python3 test_nestedmodule.py
0.7071067811865475

#--
#
#   test_nestedmodule.py
#
#--

from math import pi, sin
from nestedmodule import nestedmodule

def f(x):
return x**2

@nestedmodule
def test():

def g(x):
return f(x) * pi

def h(x):
return sin(g(x))

y = test.h(0.5)
print(y)

#--
#
#   nestedmodule.py
#
#--

from types import CodeType, ModuleType

def hack_code(f):
"""Hack 'return locals()' onto the end of the bytecode of f."""
code1 = f.__code__
bytes1 = code1.co_code
names1 = code1.co_names
n = len(names1)
names2 = names1 + ('locals',)
bytes2 = bytes1[:-4] + bytes([116, n, 0, 131, 0, 0, 83])
code2 = CodeType(code1.co_argcount, code1.co_kwonlyargcount, 
code1.co_nlocals,
code1.co_stacksize, code1.co_flags, bytes2, code1.co_consts, names2,
code1.co_varnames, code1.co_filename, code1.co_name, 
code1.co_firstlineno,
code1.co_lnotab, code1.co_freevars, code1.co_cellvars)
return code2

def nestedmodule(f):
c = hack_code(f)
l = eval(c, f.__globals__)
m = ModuleType(f.__name__)
m.__dict__.update(l)
return m
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Steven D'Aprano]

On Tuesday 05 July 2016 13:47, Chris Angelico wrote:

> On Tue, Jul 5, 2016 at 1:35 PM, Steven D'Aprano <st...@pearwood.info> wrote:
>>> If you push your code into a separate .py file, you can reference the
>>> original module by importing it. Is that also the normal way to use
>>> "outer" functions etc from inside a namespace?
>>
>> Good question!
>>
>> With the current implementation, importing should work, but it's not
>> necessary. The surrounding module (the real .py module) is inserted into
>> the name resolution path of functions:
>>
>> py> x = 999
>> py> @namespace.Namespace
>> ... class Test:
>> ... def test():
>> ... print(x)
>> ...
>> py> Test.test()
>> 999
> 
> Ah, fascinating. This does break the "just unindent and move to a new
> file if you want to break it out" equivalency, but it does make sense
> - it's a *nested* namespace, which modules (even in a package) are
> not. So you have the outer namespace acting pretty much the way
> builtins do. (Do nested namespaces work?)

I haven't got that far, but I expect that nested namespaces will be nested in 
name only, like nested functions in Python 1.5.

There's only so far I can go without support from the compiler. If you nest a 
class inside another class, the inner class doesn't see variables in the outer 
class even during construction. So I'm pretty sure my namespace metaclass will 
inherit the same limitation.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Chris Angelico]

On Tue, Jul 5, 2016 at 1:35 PM, Steven D'Aprano <st...@pearwood.info> wrote:
>> If you push your code into a separate .py file, you can reference the
>> original module by importing it. Is that also the normal way to use
>> "outer" functions etc from inside a namespace?
>
> Good question!
>
> With the current implementation, importing should work, but it's not
> necessary. The surrounding module (the real .py module) is inserted into
> the name resolution path of functions:
>
> py> x = 999
> py> @namespace.Namespace
> ... class Test:
> ... def test():
> ... print(x)
> ...
> py> Test.test()
> 999

Ah, fascinating. This does break the "just unindent and move to a new
file if you want to break it out" equivalency, but it does make sense
- it's a *nested* namespace, which modules (even in a package) are
not. So you have the outer namespace acting pretty much the way
builtins do. (Do nested namespaces work?)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Steven D'Aprano]

On Tue, 5 Jul 2016 12:58 pm, Chris Angelico wrote:

> On Tue, Jul 5, 2016 at 12:34 PM, Steven D'Aprano 
> wrote:
>> *** IF *** you are willing to push the code out into its own separate .py
>> file, you can use a module and write your code in a more natural form:
>>
>>
>> # module example.py
>> var = 999
>>
>> def spam(arg):
>> return eggs(arg) + var
>>
>> def eggs(arg):
>> return arg*2
>>
>>
>> What I'm calling a "namespace" is just a module object that lives inside
>> another module, without requiring a separate .py file. It only uses
>> the "class" statement for pragmatic reasons: there's no other statement
>> available that will do the job.
> 
> If you push your code into a separate .py file, you can reference the
> original module by importing it. Is that also the normal way to use
> "outer" functions etc from inside a namespace?

Good question! 

With the current implementation, importing should work, but it's not
necessary. The surrounding module (the real .py module) is inserted into
the name resolution path of functions:

py> x = 999
py> @namespace.Namespace
... class Test:
... def test():
... print(x)
...
py> Test.test()
999


Of course, the module-global x will be shadowed by any x in the Test
namespace (which is the intent), and you cannot assign to them (also a
feature). A bare `x = 1` inside the function will make x a local, unless
you declare it global first, in which case it should assign to the Test
namespace scope instead.




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Chris Angelico]

On Tue, Jul 5, 2016 at 12:34 PM, Steven D'Aprano  wrote:
> *** IF *** you are willing to push the code out into its own separate .py
> file, you can use a module and write your code in a more natural form:
>
>
> # module example.py
> var = 999
>
> def spam(arg):
> return eggs(arg) + var
>
> def eggs(arg):
> return arg*2
>
>
> What I'm calling a "namespace" is just a module object that lives inside
> another module, without requiring a separate .py file. It only uses
> the "class" statement for pragmatic reasons: there's no other statement
> available that will do the job.

If you push your code into a separate .py file, you can reference the
original module by importing it. Is that also the normal way to use
"outer" functions etc from inside a namespace?

# demo.py
pi = 3.14
def stupidfib(x):
if x < 2: return x
return stupidfib(x-1) + stupidfib(x-2)


Namespace asdf: # (or class, however it's done)
def foo(x):
return stupidfib(x * pi) / pi

How should foo reference those "even more global" names? "from .
import pi, stupidfib" would work if you converted the module into a
package ("mv demo.py demo/__init__.py"), and "from demo import pi,
stupidfib" would work if you converted the namespace into a peer
module. Either could make sense.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Steven D'Aprano]

On Mon, 4 Jul 2016 09:23 pm, jmp wrote:

> On 07/01/2016 04:13 PM, Steven D'Aprano wrote:
>> But classes are not like the others: they must be instantiated before
>> they can be used, and they are more than just a mere namespace grouping
>> related entities. Classes support inheritance. Classes should be used for
>> "is-a" relationships, not "has-a" relationships. Although classes (and
>> instances) are namespaces, they provide fundamentally different kind of
>> behaviour than modules and packages.
> 
> A namespace would not hurt but I really don't get why you don't consider
> classes a valid and rather helpful namespace.

I never said that.

This is where the term "namespace" can be ambiguous. "Namespace" can refer
to any of:

- an abstract mapping of symbols (names) to values;

- specific kinds of namespaces:

  * the concrete C++/C#/PHP data type called "namespace";
  * Python packages and modules;
  * classes;
  * instances of a class;

- the implementation (the __dict__ attribute of modules, classes);

etc. Now clearly a class is not the same thing as the class __dict__, and a
module is not the same as a class, and neither modules nor classes are the
same as a C++ namespace. Doesn't mean that classes aren't valid namespaces,
just that their semantics, use-cases and behaviour are different to those
of modules.



> 1/ classes do not have to be instantiated.

That's a bit of a sore point to me.

Some years ago I wrote here to ask what name is given to a class that is not
instantiated before being used. Singleton classes get instantiated once,
allowing a single instance. What if you had a class that didn't need
instantiating at all, so that the class itself was *effectively* the
singleton? What should that be called?


Instead of this:

class SingletonClass:
...

singleton = SingletonClass()
singleton.method()


what if we had this instead?

class singleton:
...

singleton.method()


I was roundly told that this was crazy talk, that the whole point of classes
was that they must be instantiated to use them, that code like the second
example would be confusing and very possibly bring about the fall of
Western Civilisation. The idea that this might be a legitimate alternative
to the singleton design pattern was dismissed.

(The Python community is terribly conservative when it comes to code.)

And, in a sense, they were right: there are two common ways to get
singleton-like behaviour in general, and in Python specifically:

- use class that allows only a single instance;

- use a module.

Using the class itself is unusual and surprising (especially to Java
programmers, where classes aren't even first-class values), and more so,
it's *inconvenient*.

To write a class which is used without instantiation, you should raise an
error on instantiation, decorate every method using classmethod or
staticmethod, and have methods have to call each other using the dotted
name:

class Example:
var = 999

def __init__(self):
raise TypeError('do not instantiate this class')

@classmethod
def spam(cls, arg):
return cls.eggs(arg) + cls.var

@classmethod
def eggs(cls, arg):
return arg*2


*** IF *** you are willing to push the code out into its own separate .py
file, you can use a module and write your code in a more natural form:


# module example.py
var = 999

def spam(arg):
return eggs(arg) + var

def eggs(arg):
return arg*2


What I'm calling a "namespace" is just a module object that lives inside
another module, without requiring a separate .py file. It only uses
the "class" statement for pragmatic reasons: there's no other statement
available that will do the job.


> 2/ the fact that classes are more than a namespace is not an argument.
> Almost any object in python is able to do more than what you are
> actually using.

There's one aspect of classes that is a deliberate design feature, but goes
against what I'm after: the fact that the class namespace itself is NOT
part of the method name resolution rules (except during class
construction). Try this:

x = 999

class Example:
x = 1
print(x)  # called during class construction
@classmethod
def test(cls):
print(x)


Example.test()



This will print 1, then 999. We quite often get people complaining about
this. I'm not one of them. I want classes to keep the current rules. But it
shows that a class is not the right answer for a module-inside-a-module
object.


> 3/ Classes are used as much as 'is-a' than 'has-a', class instances
> *have* a state usually described by attributes

Instances have state, of course, but the relationship I'm talking about is
instance to class.

class Dog:
...

lassie = Dog()

Lassie is a dog, not "Lassie has a dog".

Lassie has a tail, not "Lassie is a tail".

Re: Namespaces are one honking great idea

[Lawrence D’Oliveiro]

On Tuesday, July 5, 2016 at 10:10:04 AM UTC+12, I wrote:
>
> On Monday, July 4, 2016 at 11:37:44 PM UTC+12, Chris Angelico wrote:
> 
> > Functions within the namespace can't call other functions within the
> > same namespace using unqualified names. This was a stated requirement.
> 
> Doesn’t my @namespace decorator provide that?

No it doesn’t.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Lawrence D’Oliveiro]

On Monday, July 4, 2016 at 11:37:44 PM UTC+12, Chris Angelico wrote:

> Functions within the namespace can't call other functions within the
> same namespace using unqualified names. This was a stated requirement.

Doesn’t my @namespace decorator provide that?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[jmp]

On 07/04/2016 01:37 PM, Chris Angelico wrote:

On Mon, Jul 4, 2016 at 9:23 PM, jmp <jeanmic...@sequans.com> wrote:

On 07/01/2016 04:13 PM, Steven D'Aprano wrote:


But classes are not like the others: they must be instantiated before they
can be used, and they are more than just a mere namespace grouping related
entities. Classes support inheritance. Classes should be used for "is-a"
relationships, not "has-a" relationships. Although classes (and instances)
are namespaces, they provide fundamentally different kind of behaviour
than
modules and packages.



A namespace would not hurt but I really don't get why you don't consider
classes a valid and rather helpful namespace.

1/ classes do not have to be instantiated.
2/ the fact that classes are more than a namespace is not an argument.
Almost any object in python is able to do more than what you are actually
using.
3/ Classes are used as much as 'is-a' than 'has-a', class instances *have* a
state usually described by attributes
4/ "Although classes (and instances) are namespaces, ". You seem to
contradict yourself. It was probably a rhetorical construct but it's rather
confusing.


Functions within the namespace can't call other functions within the
same namespace using unqualified names. This was a stated requirement.

ChrisA



Ho, I missed that one.

But if it's the only missing requirement, wouldn't be like stating that 
python instances are not instances because methods cannot call other 
methods without "self."ed qualified name ? We like explicit qualified 
stuff in python right ? ("explicit is better than implicit")


jm



--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Chris Angelico]

On Mon, Jul 4, 2016 at 9:23 PM, jmp <jeanmic...@sequans.com> wrote:
> On 07/01/2016 04:13 PM, Steven D'Aprano wrote:
>>
>> But classes are not like the others: they must be instantiated before they
>> can be used, and they are more than just a mere namespace grouping related
>> entities. Classes support inheritance. Classes should be used for "is-a"
>> relationships, not "has-a" relationships. Although classes (and instances)
>> are namespaces, they provide fundamentally different kind of behaviour
>> than
>> modules and packages.
>
>
> A namespace would not hurt but I really don't get why you don't consider
> classes a valid and rather helpful namespace.
>
> 1/ classes do not have to be instantiated.
> 2/ the fact that classes are more than a namespace is not an argument.
> Almost any object in python is able to do more than what you are actually
> using.
> 3/ Classes are used as much as 'is-a' than 'has-a', class instances *have* a
> state usually described by attributes
> 4/ "Although classes (and instances) are namespaces, ". You seem to
> contradict yourself. It was probably a rhetorical construct but it's rather
> confusing.

Functions within the namespace can't call other functions within the
same namespace using unqualified names. This was a stated requirement.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[jmp]

On 07/01/2016 04:13 PM, Steven D'Aprano wrote:

But classes are not like the others: they must be instantiated before they
can be used, and they are more than just a mere namespace grouping related
entities. Classes support inheritance. Classes should be used for "is-a"
relationships, not "has-a" relationships. Although classes (and instances)
are namespaces, they provide fundamentally different kind of behaviour than
modules and packages.


A namespace would not hurt but I really don't get why you don't consider 
classes a valid and rather helpful namespace.


1/ classes do not have to be instantiated.
2/ the fact that classes are more than a namespace is not an argument. 
Almost any object in python is able to do more than what you are 
actually using.
3/ Classes are used as much as 'is-a' than 'has-a', class instances 
*have* a state usually described by attributes
4/ "Although classes (and instances) are namespaces, ". You seem to 
contradict yourself. It was probably a rhetorical construct but it's 
rather confusing.


jm

--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Ethan Furman]

On 07/03/2016 03:02 PM, Kevin Conway wrote:
>At some point earlier Ethan Furman declared:


It's not a language change.


Perhaps. My argument is that anything that introduces a new class-like
construct and set of lexical scoping rules is a language change. For
example, if this change went into 2.7.13 would Jython suddenly be broken
because it hasn't implemented the new scoping rules?


It's not a language change*.  There is nothing for Jython, IronPython, 
Brython, etc., to implement.  No scoping rule changes, nothing.  The 
magic in Steven's name space is implemented by the metaclass by 
(presumably) rebuilding all the functions -- and that is how he manages 
the effective scoping rules.


--
~Ethan~


*Okay, it is not a language change the same way the addition of Enum was 
not a language change.  On the other hand, asyncio did have some 
language changes (await, async, etc.).

--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[BartC]

On 04/07/2016 00:21, Lawrence D’Oliveiro wrote:

On Monday, July 4, 2016 at 10:02:59 AM UTC+12, Kevin Conway wrote:

If the problem with using classes to satisfy the namespace need is
that it's unwieldy to use dot qualified paths then isn't that quite similar
to saying namespaces are unwieldy?


Python has a simple solution to that:

a = mod.a
b = mod.b


Except that then

 a = 0

doesn't do the same thing as:

 mod.a = 0

But it'll probably work for things that are not assigned to.

--
Bartc

--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Lawrence D’Oliveiro]

On Monday, July 4, 2016 at 10:02:59 AM UTC+12, Kevin Conway wrote:
> If the problem with using classes to satisfy the namespace need is
> that it's unwieldy to use dot qualified paths then isn't that quite similar
> to saying namespaces are unwieldy?

Python has a simple solution to that:

a = mod.a
b = mod.b

etc
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Kevin Conway]

>> Regardless, all use cases you've listed are already satisfied by use of
>> the static and class method decorators. Methods decorated with these do
>> not require an instance initialization to use.

> And are significantly less easy to use, as the functions MUST refer to
each
> other by their dotted names.

My response to this may come off as a bit glib, but it isn't intended that
way. If the problem with using classes to satisfy the namespace need is
that it's unwieldy to use dot qualified paths then isn't that quite similar
to saying namespaces are unwieldy? Leveraging classes as a nested module
creates a de-facto internal namespace of "cls" for self reference and I'm
unsure of why that is unwanted but "module.namespace.blah" is wanted.

I suppose my issue is not so much that namespace objects are a bad idea as
that the proposal does little to express why the existing tools are
deficient enough to require a new concept.

> For the proponents of namespace, what is deficient in the above example
> that necessitates a language change?

>> It's not a language change.

Perhaps. My argument is that anything that introduces a new class-like
construct and set of lexical scoping rules is a language change. For
example, if this change went into 2.7.13 would Jython suddenly be broken
because it hasn't implemented the new scoping rules?

>> IIRC, classes came about as a "module in a module".

> I'm pretty sure they did not. Object oriented programming (and hence
> classes) came about from simulating real world objects, hence the
> name:

I realize my statement was ambiguous. I didn't mean to suggest that
classes, as an idea and OOP tool, were derived from the concept of a
namespace. I meant to say that the Python implementation of classes is
quite similar to the implementation of modules in the cPython code. The
commit messages from the earlier days (ref: ab5db02) don't carry much
detail so I'm mostly basing my belief of the implementation similarities
and anecdotes from the community. Right or wrong, I don't believe it has
much weight in the namespace discussion and I shouldn't have brought it up.

On Sun, Jul 3, 2016, 00:17 Ethan Furman <et...@stoneleaf.us> wrote:

> On 07/02/2016 08:44 PM, Steven D'Aprano wrote:
>
> > Try getting this behaviour from within a class:
> >
> >
> > class Food(metaclass=Namespace):
> >
> >  # (1) no special decorators required
> >  def spam(n):
> >  return ' '.join(['spam']*n)
> >
> >  # (2) can call functions from inside the namespace
> >  breakfast = spam(5)
> >
> >  # (3) no "cls" or "self" argument
> >  def lunch():
> >  # (4) can access variables using their undotted name
> >  return breakfast + ' and eggs'
> >
> >  def supper():
> >  # (5) likewise functions (a special case of #4)
> >  return lunch() + ' and a fried slice of spam'
> >
> >  def mutate(n):
> >  # global inside the namespace refers to the namespace,
> >  # not the surrounding module
> >  global breakfast
> >  breakfast = spam(5)
>
> > Can you do all of that with an ordinary class?
>
> You can get #2 already, but not the rest (without your spiffy code ;) :
>
> Python 3.5.1+ (3.5:f840608f79da, Apr 14 2016, 12:29:06)
> [GCC 4.8.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> class Huh:
> ...   def blah(text):
> ... print('blah blah %s blah blah blah' % text)
> ...   blah('whatever')
> ...
> blah blah whatever blah blah blah
>
> --
> ~Ethan~
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Ethan Furman]

On 07/02/2016 08:44 PM, Steven D'Aprano wrote:


Try getting this behaviour from within a class:


class Food(metaclass=Namespace):

 # (1) no special decorators required
 def spam(n):
 return ' '.join(['spam']*n)

 # (2) can call functions from inside the namespace
 breakfast = spam(5)

 # (3) no "cls" or "self" argument
 def lunch():
 # (4) can access variables using their undotted name
 return breakfast + ' and eggs'

 def supper():
 # (5) likewise functions (a special case of #4)
 return lunch() + ' and a fried slice of spam'

 def mutate(n):
 # global inside the namespace refers to the namespace,
 # not the surrounding module
 global breakfast
 breakfast = spam(5)



Can you do all of that with an ordinary class?


You can get #2 already, but not the rest (without your spiffy code ;) :

Python 3.5.1+ (3.5:f840608f79da, Apr 14 2016, 12:29:06)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Huh:
...   def blah(text):
... print('blah blah %s blah blah blah' % text)
...   blah('whatever')
...
blah blah whatever blah blah blah

--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Steven D'Aprano]

On Sun, 3 Jul 2016 01:34 am, Kevin Conway wrote:

> staticmethod isn't technically required to use a method through the class
> (or subclasses), it simply provides the appropriate magic to allow it to
> be called through instances.
> 
> For example, the following code covers all described use cases of the
> proposed namespace. 

It really doesn't. Here's your class:

> Methods are invoked without creating instances and 
> state is managed on the class object directly.
> 
> class CustomNs:
> 
> stateful_data = 1
> 
> @staticmethod
> def echo(text):
> print(text)
> 
> @classmethod
> def mutate(cls):
> cls.stateful_data += 1
> print(cls.stateful_data)


The class scope is not "first class" (pun intended), as you cannot refer to
class-level variables without prefixing them with the class, even from
within the class.

Although the class *may* be used without instantiating, this violates the
user's expectations and may be confusing.

I acknowledge that my own namespace has a similar problem, as it uses the
class keyword, but the use of an explicit Namespace decorator or metaclass
gives a hint that something is different. And if you try calling the
Namespace, as if to instantiate it, you get a TypeError.


[...]
> For the proponents of namespace, what is deficient in the above example
> that necessitates a language change?

It's not a language change. Although I'd like a new keyword, to avoid
the "but it looks like a class" problem, its not necessary.


Try getting this behaviour from within a class:


class Food(metaclass=Namespace):

# (1) no special decorators required
def spam(n):
return ' '.join(['spam']*n)

# (2) can call functions from inside the namespace
breakfast = spam(5)

# (3) no "cls" or "self" argument
def lunch():
# (4) can access variables using their undotted name
return breakfast + ' and eggs'

def supper():
# (5) likewise functions (a special case of #4)
return lunch() + ' and a fried slice of spam'

def mutate(n):
# global inside the namespace refers to the namespace,
# not the surrounding module
global breakfast
breakfast = spam(5)



Everything written inside the namespace object could have started as top
level module code. I select the code, hit my editor's "Indent" command, and
insert a single line at the top to turn it into a namespace.

If I decide to move the code into a separate file, I just copy the block,
excluding the "class ... metaclass" header line, past into a new file,
select all, and Dedent. Done.

Can you do all of that with an ordinary class?





-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Steven D'Aprano]

On Sat, 2 Jul 2016 11:50 am, Kevin Conway wrote:

> I believe the namespace object you are referring to is exactly a class.

Yes, a namespace is exactly like a class, minus inheritance, instantiation,
the implied "is-a" relationship, and the whole Java "utility class"
anti-pattern.

In other words, exactly not like a class *wink*

Classes and modules are both namespaces: "an abstract container or
environment created to hold a logical grouping of unique identifiers or
symbols (i.e. names)", to quote Wikipedia:

https://en.wikipedia.org/wiki/Namespace

But classes provide much more functionality, over and above mere namespace:
inheritance, instantiation, etc. I'm not anti-classes or opposed to OOP,
these features are great when they are wanted. But when they aren't wanted,
they're a nuisance.

Modules also provide more than just an abstract namespace: they provide
scope, but that's exactly what I want for my Namespace entity: it should be
a namespace, and it should be a scope for functions. In other words, it
should be a module. I just don't want to have to place the source code in a
separate file, because sometimes that's annoying and overkill.

Modules are good for information hiding and encapsulation, just like
classes, but without inheritance etc. But because you cannot put more than
one module inside a single file, the developer has to make a choice between
information hiding and encapsulation:

(1) You can keep your functions encapsulated inside a single module/file,
but you cannot hide some of them from parts of the module that don't care
about them.

(2) You can hide those functions from the parts of the module that don't
care about them, but only by breaking encapsulation, moving them to another
file, and exposing another name in the file system namespace.


The "namespace" feature offers an alternative.

Namespaces in C#, C++ and PHP:

https://msdn.microsoft.com/en-us/library/z2kcy19k.aspx
https://msdn.microsoft.com/en-us/library/5cb46ksf.aspx
http://php.net/manual/en/language.namespaces.php



> IIRC, classes came about as a "module in a module".

I'm pretty sure they did not. Object oriented programming (and hence
classes) came about from simulating real world objects, hence the name:

http://www.exforsys.com/tutorials/oops-concepts/the-history-of-object-oriented-programming.html

In languages like Pascal, you can nest functions and procedures inside
others, providing something like a namespace functionality, except that
Pascal strictly enforces information hiding. There's no way for code to
peer inside a Pascal function and see nested functions.


> Regardless, all use cases you've listed are already satisfied by use of
> the static and class method decorators. Methods decorated with these do
> not require an instance initialization to use.

And are significantly less easy to use, as the functions MUST refer to each
other by their dotted names.




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Ethan Furman]

On 07/02/2016 08:34 AM, Kevin Conway wrote:


For the proponents of namespace, what is deficient in the above example
that necessitates a language change?


Adding a new widget is not changing the language.

--
~Ethan~

--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Kevin Conway]

> staticmethod isn't technically required to use a method through the class
(or subclasses), it simply provides the appropriate magic to allow it to be
called through instances.

For example, the following code covers all described use cases of the
proposed namespace. Methods are invoked without creating instances and
state is managed on the class object directly.

class CustomNs:

stateful_data = 1

@staticmethod
def echo(text):
print(text)

@classmethod
def mutate(cls):
cls.stateful_data += 1
print(cls.stateful_data)

CustomNs.echo("test")
print(CustomNs.stateful_data)
CustomNs.mutate()
print(CustomNs.stateful_data)

For the proponents of namespace, what is deficient in the above example
that necessitates a language change?

On Sat, Jul 2, 2016, 00:02 Random832  wrote:

> On Fri, Jul 1, 2016, at 21:50, Kevin Conway wrote:
> > I believe the namespace object you are referring to is exactly a
> > class. IIRC, classes came about as a "module in a module".
>
> No, because classes have instances. And conceptually they seem like they
> *should* have instances. Just using the term "class" carries
> expectations.
>
> More to the point, what modules do that classes do not is provide a
> global namespace for functions defined within them, so that variables
> within them can be used (well, read - writing them requires a
> declaration) by the functions without extra qualification.
>
> > Regardless, all use cases you've listed are already satisfied by use
> > of the static and class method decorators. Methods decorated with
> > these do not require an instance initialization to use.
>
> staticmethod isn't technically required to use a method through the
> class (or subclasses), it simply provides the appropriate magic to allow
> it to be called through instances.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Random832]

On Fri, Jul 1, 2016, at 21:50, Kevin Conway wrote:
> I believe the namespace object you are referring to is exactly a
> class. IIRC, classes came about as a "module in a module".

No, because classes have instances. And conceptually they seem like they
*should* have instances. Just using the term "class" carries
expectations.

More to the point, what modules do that classes do not is provide a
global namespace for functions defined within them, so that variables
within them can be used (well, read - writing them requires a
declaration) by the functions without extra qualification.

> Regardless, all use cases you've listed are already satisfied by use
> of the static and class method decorators. Methods decorated with
> these do not require an instance initialization to use.

staticmethod isn't technically required to use a method through the
class (or subclasses), it simply provides the appropriate magic to allow
it to be called through instances.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Lawrence D’Oliveiro]

On Saturday, July 2, 2016 at 1:50:56 PM UTC+12, Kevin Conway wrote:
> Regardless, all use cases you've listed are already satisfied by use of the
> static and class method decorators.

Except for the need to decorate every such function inside the class. How about:

import types

def namespace(inclass) :
"decorator which turns a class into a module."
outclass = types.ModuleType \
  (
name = inclass.__name__,
doc = inclass.__doc__,
  )
for attr in dir(inclass) :
if not attr.startswith("__") or attr in ("__package__",) :
setattr(outclass, attr, getattr(inclass, attr))
#end if
#end for
return \
outclass
#end namespace

Example use:

@namespace
class try_it :
"try it!"

val = "some text"

def func() :
print("func got called")
#end func

#end try_it
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Rustom Mody]

On Friday, July 1, 2016 at 8:19:36 PM UTC+5:30, BartC wrote:
> On 01/07/2016 15:13, Steven D'Aprano wrote:
> 
> > Sometimes we have a group of related functions and variables that belong
> > together, but are not sufficiently unrelated to the rest of the module that
> > we want to split them out into another file.
> 
> > Here's a proof of concept. I use a class with a custom metaclass like this:
> >
> >
> > # Python 3 version
> > class ExampleNS(metaclass=Namespace):
> > x = 1
> > y = []
> >
> > def spam(n):
> > return 'spam'*n
> 
> > py> Example.spam(5)
> > 'spamspamspamspamspam'
> 
> 
> Why not just extend the capabilities of a class? I actually thought this 
> would work until I tried it and it didn't:

Well I also thought similarly -- sure a normal (instance) method cannot be
used without an instance but sure this is what class/static methods are for?
ie Steven's option 4.

Then I remembered that python's LEGB rule is bizarre:
Outer scopes are added outside inner scopes ... except for classes
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Kevin Conway]

I believe the namespace object you are referring to is exactly a class.
IIRC, classes came about as a "module in a module".

Regardless, all use cases you've listed are already satisfied by use of the
static and class method decorators. Methods decorated with these do not
require an instance initialization to use.

On Fri, Jul 1, 2016, 20:17 Steven D'Aprano  wrote:

> On Sat, 2 Jul 2016 05:29 am, Ethan Furman wrote:
>
> > On 07/01/2016 10:10 AM, Steven D'Aprano wrote:
> >> On Sat, 2 Jul 2016 02:00 am, Ethan Furman wrote:
> >
> >>> Did you mean for this to go to -Ideas?
> >>
> >> Not yet. I wanted some initial feedback to see if anyone else liked the
> >> idea before taking it to Bikeshedding Central :-)
> >>
> >> Besides, I expect Python-Ideas will say it needs to be a package on PpPI
> >> first. Although I am kinda considering sneaking this into the std lib as
> >> an undocumented internal feature, like simplegeneric in pkgutil. (Don't
> >> tell anyone I said that *wink* )
> >
> > Are there good use-cases for this in the stdlib?
>
> I have at least one.
>
>
>
> --
> Steven
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Steven D'Aprano]

On Sat, 2 Jul 2016 05:29 am, Ethan Furman wrote:

> On 07/01/2016 10:10 AM, Steven D'Aprano wrote:
>> On Sat, 2 Jul 2016 02:00 am, Ethan Furman wrote:
> 
>>> Did you mean for this to go to -Ideas?
>>
>> Not yet. I wanted some initial feedback to see if anyone else liked the
>> idea before taking it to Bikeshedding Central :-)
>>
>> Besides, I expect Python-Ideas will say it needs to be a package on PpPI
>> first. Although I am kinda considering sneaking this into the std lib as
>> an undocumented internal feature, like simplegeneric in pkgutil. (Don't
>> tell anyone I said that *wink* )
> 
> Are there good use-cases for this in the stdlib?

I have at least one.



-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Ethan Furman]

On 07/01/2016 10:10 AM, Steven D'Aprano wrote:

On Sat, 2 Jul 2016 02:00 am, Ethan Furman wrote:



Did you mean for this to go to -Ideas?


Not yet. I wanted some initial feedback to see if anyone else liked the idea
before taking it to Bikeshedding Central :-)

Besides, I expect Python-Ideas will say it needs to be a package on PpPI
first. Although I am kinda considering sneaking this into the std lib as an
undocumented internal feature, like simplegeneric in pkgutil. (Don't tell
anyone I said that *wink* )


Are there good use-cases for this in the stdlib?

--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Steven D'Aprano]

On Sat, 2 Jul 2016 12:49 am, BartC wrote:

> On 01/07/2016 15:13, Steven D'Aprano wrote:
> 
>> Sometimes we have a group of related functions and variables that belong
>> together, but are not sufficiently unrelated to the rest of the module
>> that we want to split them out into another file.
> 
>> Here's a proof of concept. I use a class with a custom metaclass like
>> this:
>>
>>
>> # Python 3 version
>> class ExampleNS(metaclass=Namespace):
>> x = 1
>> y = []
>>
>> def spam(n):
>> return 'spam'*n
> 
>> py> Example.spam(5)
>> 'spamspamspamspamspam'
> 
> 
> Why not just extend the capabilities of a class? 

The usual ways to change the behaviour of classes from Python code is via
decorator, which lets you modify the class after it is created, a
metaclass, which lets you modify it before it is created, or by using
descriptors to customize attribute access.

I'm using a metaclass. When you write:

class K: ...

that is syntactic sugar for calling the default metaclass (`type`) with some
arguments, and `type` then returns the new class K. But if you use a
metaclass (usually, but not necessarily a subclass of `type`) you can
customize the creation of the new class, or even return a completely
different object altogether. That's what I'm doing.

Why am I returning a module instead of a class? Because (1) conceptually a
namespace (in the C++ sense) is like a module; (2) classes have a whole lot
of expectations that aren't relevant to namespaces (like inheritance,
instantiation, "is-a", etc); and (3) it's easy to use a module, which
already provides most of the behaviour I want.


> I actually thought this 
> would work until I tried it and it didn't:
> 
> class C():
>  def fn():
>  print ("Hi!")
> 
> C.fn()

That actually will work in Python 3. (I had forgotten that.) In Python 2,
you have to inherit C from object, AND decorate fn with staticmethod:

class C(object):
@staticmethod
def fn():
print ("Hi!")


But for my use, that's not enough. I want this to work:

class Foo:
def f():
return g().upper()
def g():
return "hi!"


That is, Foo should behave as if it were a module imported from a file,
except without actually being imported from a file.




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces are one honking great idea

[Steven D'Aprano]

On Sat, 2 Jul 2016 02:00 am, Ethan Furman wrote:

> On 07/01/2016 07:13 AM, Steven D'Aprano wrote:
> 
> I like the idea, but I have a couple questions about the design choices.

Thanks!

>   Comments below.


[...]
>> Despite the "class" statement (a limitation of Python's lack of dedicated
>> syntax for namespaces), the Example namespace behaves like (in fact,
>> *is*) a module embedded inside a module.
> 
> So the idea is to have several "mini-modules" inside a single file?

That would certainly be possible.


> Can a mini-module function access a different mini-module's function?

Yes, via the dotted attribute name, just as you might say "math.sin".

The only difference is that you don't use import to get access to the
(mini-)module. It's just there, defined in your own file.


> Can a mini-module function access/interact with the module's global scope?

Yes. I'm still experimenting with different implementations, but I have a
proof of concept working.


>> Especially note that inside the namespace, the global statement makes a
>> name refer to the namespace scope.
> 
> Interesting.  :)
> 
> 
>> Unresolved questions
>> =
>>
>> Would a decorator be better than a metaclass?
> 
> I think a decorator may provide more of a clue that something
> interesting is happening -- whether a decorator is feasible depends on
> whether you need the __prepare__ magic of metaclasses.

I don't think __prepare__ is needed.



>>  @namespace
>>  class Example:
>>  ...
> 
> I believe Python already has a NameSpace type, so a different name is
> probably better.  How about minimodule? ;)

I think you are thinking of "SimpleNamespace".

In any case, so long as my namespace lives in a different, er, namespace as
the other namespace, there's no conflict :-)

In the long term, I think my namespace (if accepted into the stdlib) would
belong in functools, collections or types. Perhaps even its own module. I
don't think this needs to be a built-in.



>> Some limitations
>> =
>>
>> The simplified implementation shown doesn't allow functions inside the
>> namespace to access names outside of the namespace easily.
> 
> How not-easy is it?

Not that hard now. I have an implementation working. It actually turned out
to be easy. (Thanks Raymond for ChainMap!)



>> In practice, the
>> user might want the name resolution order inside the namespace to be:
>>
>>   local variables
>>   nonlocal variables
>>   namespace globals
>>   module globals
>>   builtins
> 
> That would be ideal, I think.  How close can we get to that?

I now have that working, for some definition of working. Its not extensively
tested, so there might be some bugs or issues I haven't thought of, but the
basic concept is working: inside the namespace/mini-module object,
functions see namespace globals ahead of module globals, which are ahead of
builtins.


[...]
> Did you mean for this to go to -Ideas?


Not yet. I wanted some initial feedback to see if anyone else liked the idea
before taking it to Bikeshedding Central :-)

Besides, I expect Python-Ideas will say it needs to be a package on PpPI
first. Although I am kinda considering sneaking this into the std lib as an
undocumented internal feature, like simplegeneric in pkgutil. (Don't tell
anyone I said that *wink* )





-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list