[issue46455] Deprecate / remove os.name=java
Xavier Morel added the comment: PS: "platform.system()" also documents `Java` as a value which doesn't seem to make that much sense, however it's an open set so probably less of an issue / source of confusion. -- ___ Python tracker <https://bugs.python.org/issue46455> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46455] Deprecate / remove os.name=java
New submission from Xavier Morel : os.name is defined as: > The following names have currently been registered: 'posix', 'nt', 'java'. In my understanding, the value `'java'` is for the benefit of jython, which is rather poorly. Other third-party implementations which may or may not have a "full os module" (e.g. ironpython) do not -- as far as I can tell -- get to be registered against this value, and Python 3.3's addition of `sys.implementation` seems like a more reliable (and better supported) way to perform implementation-specific checks. Therefore I feel `os.name == 'java'` only exists to confuse readers of the documentation, but doesn't really provide any value, and should be removed. -- components: Library (Lib) messages: 411123 nosy: xmorel priority: normal severity: normal status: open title: Deprecate / remove os.name=java versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46455> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45368] ~(True) and ~(False) gives incorrect result
Xavier Morel added the comment: > True is a boolean so ~True should return False according to me. That's be a BC break for no reason: if you want to invert a boolean you can just `not` it. > True is not the same as 1 For historical reasons, in Python it is: >>> bool.mro() [, , ] >>> True == 1 True >>> False == 0 True So when you call ~True, you're calling `int.__invert__(True)`, which behaves as what it is: the bitwise inverse of a two's-complement signed integer. -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue45368> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45365] concurrent.futures.Future should be suitable for use outside of executors
New submission from Xavier Morel : concurrent.futures.Future currently has the note: > Future instances are created by Executor.submit() and should not be created > directly except for testing. That seems like a shame as futures are useful concurrency construct and having to rebuild them "by hand" seems like a waste. What are the issues which prevent safely using futures outside of executors, and is there a way they could be fixed / lifted? -- components: Library (Lib) messages: 403181 nosy: xmorel priority: normal severity: normal status: open title: concurrent.futures.Future should be suitable for use outside of executors type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue45365> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: > If working Python 3 program suddenly became emitting BytesWarning it will > confuse users. Oh yeah no I meant making it a normal warning, without needing the `-b` flag, not enabling it by default. Because controlling / configuring warnings can be done programmatically and dynamically from inside the program, but in my understanding -b is either set or unset when calling it, if you're not setting it you're SOL. Or so I understand, I didn't find any switch accessible from inside the Python program. > I think it would work. But it is significant amount of work (add > corresponding C API, parsing and copying code, documentation, tests) Wouldn't it just be removing the check on `-b` in the tp_str slot of bytes and bytearray? > and for small benefit. Would not be better to filter out warnings by message? Well yes and no, my issue remains the one from the beginning: it's quite easy to unexpectedly stringify bytes, which usually isn't what's desired. Ensuring that doesn't happen requires making sure every developer and environment (non-production at least) does run python with -b to enable the warning, which is not as easy as the software being able to set that internally (which is doable for various deprecation warnings and friends). -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: And though I did not check, I expect the `-b` flag exists mostly because of the performance impact of the warning any time bytes are checked for equality, but surely that impact would be limited and probably not very relevant for the stringification of bytes and that could (eventually) be moved out of the `-b` flag and to the regular warnings system? I recall Inada-san tends to be quite involved in BytesWarning, is adding people to the nosy list something that's done or is it in bad taste? -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: > I am not against documenting the behavior of -b and BytesWarning clearly. I > don't think that anyone would be against. Just somebody have to provide a PR. Right but what about the ability to enable warning on stringification without enabling the warning on comparison? There are packages / situations where comparing strings and bytes makes a lot of sense (e.g. use string and byte versions of data as keys in the same dict), but I think stringifying bytes rarely if ever does. > Explicitly repr-ing a bytes instance does not produce a warning, and never > produced, so I don't understand what do you mean. Well that's absolutely ideal as I did not check if it did, just wanted to be clear that I wasn't looking for a warning in that case. -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: Serhiy an other question (because I've encountered it *again*), do you think it'd be possible to split out the specific warning of stringifying (but *not* explicitely repr-ing) a bytes instance from everything else? There are use-cases for it, but I think it's rather error prone when a `bytes` unexpectedly finds itself mixed into either string or non-string data which stringify "normally" and generates what is often garbage output. -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44178] Add an interpreter-level critical section construct
New submission from Xavier Morel : Python code uses a fair amount of globals, and sometimes there's no good choice but to do dodgy things and temporarily update global state for one reason or another e.g. redirect a standard fd or stream (cf redirect_stdout), monkey-patch a builtin to intercept some behaviour, etc... One issue with this is, if the program is multithreaded there is no real way to prevent the interpreter from suspending the current thread *while doing the dodgy thing*. The only interpreter-level global lock would be the GIL, and while native extension can just not release the GIL this is not an option for Python code (as far as I know). `sys` does provide getswitchinterval()/setswitchinterval() (and under the old GIL had getcheckinterval()/setcheckinterval()), but the semantics of these functions is a bit obscure (e.g. does `sys.setcheckinterval` update the *current* time slice os is the current thread doomed? can it fail? what happens if it does?) and they feel extremely uncomfortable. As such, having a context manager which would explicitly but only prevent thread switching while it's held would be valuable. Assuming `setswitchinginterval` operates on the "current" timeslice, this would not provide any more misbehaving powers as any code can already call `sys.setswitchinterval(sys.maxsize)` (unless there are already mitigation systems in-place), and the contextmanager could just do that internally if that is the "correct" way to prevent thread switching under the current scheme. The one thing which is not great is that this scheme might not really work under GILectomy, but then neither does switchinterval I think. -- components: Interpreter Core messages: 393951 nosy: xmorel priority: normal severity: normal status: open title: Add an interpreter-level critical section construct type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue44178> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12056] "…" (HORIZONTAL ELLIPSIS) should be an alternative syntax for "..." (FULL STOP FULL STOP FULL STOP)
Xavier Morel added the comment:
> But if we allow for ellipsis, then would we not also have to start allowing
> characters like ≥ and ≤ in Python?
No, they're not defined as canonically equivalent to >= and <= by the Unicode
specification:
>>> unicodedata.normalize('NFKD', '…')
...
>>> unicodedata.normalize('NFKD', '≤')
≤
--
___
Python tracker
<https://bugs.python.org/issue12056>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44157] redirect_* should also redirect C-level streams
New submission from Xavier Morel : In 3.4 (resp. 3.5), `redirect_stdout` and `redirect_stderr` were added to provide easy and reentrant stream redirection. Although that is documented, it seems like a waste that they only redirect the high-level `sys.std*` streams: those are already pretty easy to redirect by simply setting the corresponding attribute, which is essentially what the contextmanager does. However, * that does not work if the writer has imported the original stream object (`from sys import stderr`) * that does not work if the writer bypasses or is not aware of the Python layer e.g. a native library In that case, the user still has to deal with dup/dup2 dances in order to redirect the underlying fds, which is significantly more error-prone (and verbose) than intercepting the high-level `sys.std*` ever was. On the other hand, it's also less expressive as it requires an actual fd, rather than a Python-level file-like object. So I can understand not doing it by default. But still, I think I feel like it'd be very useful to either expand the existing context managers to perform fd redirection, or have a separate context manager able to redirect arbitrary fds. -- components: Library (Lib) messages: 393810 nosy: xmorel priority: normal severity: normal status: open title: redirect_* should also redirect C-level streams type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue44157> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: > In normal circumstances you should never deal with BytesWarning. The -b > option is only used for testing your program for some possible bugs caused by > migration from Python 2. If your program always worked only with Python 3, > the -b option has no use for you. That's technically true for perfect programs, but practically if the program is not tested enough it's possible for it to contain unexpected instances of stringifying bytes which pass by mostly unnoticed, just with garbage bits of output which may or may not get reported. Noticing such cases in an existing codebase is exactly what prompted me to try and enable BytesWarning (and then try and filter them to just conversion warnings, as the category is quite noisy with somewhat uninteresting warnings), and then spend a while trying to understand why programmatically enabling BytesWarning didn't do anything. As noted in the last paragraph it's fine if `-b` is required to enable the feature, I'd just like to know if it could be documented clearly as it's somewhat frustrating to pore over the documentation of `warnings`, `BytesWarning`, and `-b`, diff the state of the filters configuration, etc… and end up finding out from a bpo comment whether the behaviour you're observing is intentional or not. -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43527] Support full stack trace extraction in warnings.
New submission from Xavier Morel : When triggering warnings, it's possible to pass in a `stacklevel` in order to point to a more informative cause than the `warnings.warn` call. For instance `stacklevel=2` is a common one for DeprecationWarning in order to mark the call itself as deprecated in the caller's codebase. The problem with this is that it's not transitive, so when a dependency triggers a warning it can be hard to know where that comes from in the codebase (at least without `-Werror` which can prevent reaching the interesting warning entirely), and whether this is an issue in the codebase (e.g. passing bytes where the library really works in terms of strings) or whether it would be possible to work around the warning by using some other API. In that case, the ability to show a full stack trace from the `stacklevel` down is very useful to diagnose such issues. Not quite sure how it would be managed though: I'd think this should be part of the warnings filter information, but the `stacklevel` currently isn't stored there, and it might be risky to extend the warnings filter with a 6th field). -- components: Library (Lib) messages: 388914 nosy: xmorel priority: normal severity: normal status: open title: Support full stack trace extraction in warnings. type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43527> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: Addendum: is there a way to force `-b` from within the running Python program? -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
New submission from Xavier Morel :
When setting `BytesWarning` programmatically (via the warnings API), though the
`warnings.filters` value matches what's obtained via `python -b` and an
explicit `warnings.warn` trigger will trigger, "native" triggers of the warning
fail to trigger properly:
import warnings
warnings.simplefilter('default', category=BytesWarning)
str(b'')
warnings.warn('test', category=BytesWarning)
If run using `python`, this will print:
test.py:4: BytesWarning: test
warnings.warn('test', category=BytesWarning)
There is no warning for the string-ification of the bytes instance.
If run using `python -b`, the behaviour is as one would expect:
test.py:3: BytesWarning: str() on a bytes instance
str(b'')
test.py:4: BytesWarning: test
warnings.warn('test', category=BytesWarning)
Inspecting `warnings.filters` shows now difference in their contents, in both
cases it is:
[('default', None, , None, 0), ('default', None,
, '__main__', 0), ('ignore', None, , None, 0), ('ignore', None, , None, 0), ('ignore', None, , None, 0), ('ignore', None, , None,
0)]
(in Python 3.9).
The warning module's own suggestion:
import sys
if not sys.warnoptions:
import warnings
warnings.simplefilter("default") # Change the filter in this process
also fails to enable BytesWarning.
If this is intended behaviour, which seems to be the case according to
ncoghlan's comment https://bugs.python.org/issue32230#msg307721, it should be
clearly documented, as it's rather frustrating.
--
components: Library (Lib)
messages: 388912
nosy: xmorel
priority: normal
severity: normal
status: open
title: Programmatic management of BytesWarning doesn't work for native triggers.
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue43526>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43036] TOS-behaviour documentation is inconsistent
Change by Xavier Morel : -- title: TOS-behaviour documentation is -> TOS-behaviour documentation is inconsistent ___ Python tracker <https://bugs.python.org/issue43036> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43036] TOS-behaviour documentation is
New submission from Xavier Morel :
I was looking at the disassembly of a fairly straightforward listcomp:
[e for e in s if e[0]]
1 0 BUILD_LIST 0
2 LOAD_FAST0 (.0)
>>4 FOR_ITER16 (to 22)
6 STORE_FAST 1 (e)
8 LOAD_FAST1 (e)
10 LOAD_CONST 0 (0)
12 BINARY_SUBSCR
14 POP_JUMP_IF_FALSE4
16 LOAD_FAST1 (e)
18 LIST_APPEND 2
20 JUMP_ABSOLUTE4
>> 22 RETURN_VALUE
6, 8 bothered me because STORE_FAST is documented as
> Stores TOS into the local co_varnames[var_num].
So it seems like it leaves TOS and thus the LOAD is unnecessary, However
looking at ceval.c:
case TARGET(STORE_FAST): {
PREDICTED(STORE_FAST);
PyObject *value = POP();
SETLOCAL(oparg, value);
FAST_DISPATCH();
}
so STORE_FAST does pop the TOS and the LOAD_FAST is necessary. This is
confusing because there are instructions which literally have POP in their name
whose stack behaviour is documented explicitly.
Should all bytecode instructions have their stack behaviour explicitly
documented, or only those with an *odd* stack behaviour (e.g.
JUMP_IF_FALSE_OR_POP) and the rest maybe covered by a note saying that they
will pop their parameters and push back their result or somesuch?
--
Furthermore, maybe optimising `STORE_LOCAL a; LOAD_LOCAL a` to `DUP_TOP;
STORE_LOCAL a` would be useful? It obviously would have no effect on bytecode
size since wordcode, and `fastlocals[i]` would be in cache and the conditional
check likely predicted, but it seems like skipping them entirely would still be
more reliable? This idea is somewhat supported by expression assignments
already generating the latter:
>>> @dis.dis
... def foo():
... if a := thing():
... do(a)
...
3 0 LOAD_GLOBAL 0 (thing)
2 CALL_FUNCTION0
4 DUP_TOP
6 STORE_FAST 0 (a)
8 POP_JUMP_IF_FALSE 18
4 10 LOAD_GLOBAL 1 (do)
12 LOAD_FAST0 (a)
14 CALL_FUNCTION1
16 POP_TOP
>> 18 LOAD_CONST 0 (None)
20 RETURN_VALUE
This optimisation would also trigger for e.g.
[x[i] for x in xs]
or
a = foo()
if a:
# do thing
making the latter generate bytecode identical to walrus assignments at least
for the trivial case: currently the only difference (aside from line numbers)
is that the normal assignment generates STORE_FAST;LOAD_FAST while expression
assignments generate DUP_TOP;STORE_FAST.
--
assignee: docs@python
components: Documentation
messages: 385762
nosy: docs@python, xmorel
priority: normal
severity: normal
status: open
title: TOS-behaviour documentation is
type: enhancement
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue43036>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42644] logging.disable('WARN') breaks AsyncIO
Xavier Morel added the comment: Oh I now see you've created a PR to do essentially that, nm. -- ___ Python tracker <https://bugs.python.org/issue42644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42644] logging.disable('WARN') breaks AsyncIO
Xavier Morel added the comment: > I think that patching logging.disable to raise a type error immediately would > be welcome FWIW `logging` has a built-in checker / converter[0] which is already used in a bunch of places (e.g. the aforementioned setLevel), it could just be added inside `disable` in the same way it's used in setLevel[1] and would uniformly convert "level names" to proper levels or raise an error. That seems like a really easy patch / contribution. [0] https://github.com/python/cpython/blob/3.9/Lib/logging/__init__.py#L189 [1] https://github.com/python/cpython/blob/3.9/Lib/logging/__init__.py#L906 -- ___ Python tracker <https://bugs.python.org/issue42644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42644] logging.disable('WARN') breaks AsyncIO
Xavier Morel added the comment: The problem seems to be in the user code? As you were told by "Carreau", loggin.disable takes a logging level (an integer), since you're giving it a string which it dutifully stores, it blows up at the next logging call which happens to be in asyncio. This is not an asyncio bug, nor is it a crash anymore than passing a broken key function to `sorted` is a *Python* crash. At most it's an enancement request: `logging.disable` could raise a TypeError immediately or convert the value to a loglevel the way `setLevel` does instead of storing the broken value as-is (all it does currently is store the value it receives on the root logger without any check). -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue42644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42572] Better path handling with argparse
Xavier Morel added the comment: > What exactly do you do with a path argument? Because they mention "convert[ing] the string to a path", I would expect an output of `pathlib.Path`, optionally checked for existence / non-existence and / or kind (file, directory, symlink, ...). Obviously it is rather easy to bring your own, but OP's expectation is that paths input is common enough (especially in smaller standalone scripts I would expect) that having a helper in a standard library would be... helpful. -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue42572> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
Xavier Morel added the comment: Tried patterning the PR after the one which originally added the warning. Wasn't too sure how the news item was supposed to be generated, and grepping the repository didn't reveal any clear script doing that, so I made up a date and copied an existing random bit (which I expect is just a deduplicator in case multiple NEWS items are created at the same instant?) -- ___ Python tracker <https://bugs.python.org/issue42470> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
Change by Xavier Morel : -- pull_requests: +22532 pull_request: https://github.com/python/cpython/pull/23665 ___ Python tracker <https://bugs.python.org/issue42470> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
Xavier Morel added the comment: I was preparing to open the PR but now I'm doubting: should I open the PR against master and miss islington will backport it, or should I open the PR against 3.9? -- ___ Python tracker <https://bugs.python.org/issue42470> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
Xavier Morel added the comment: > Do you want to submit a PR for this? Sure. Do you think the code I proposed would be suitable? > * The current logic matches the logic before the warning was added. > * The proposed logic matches what the code will do after the > deprecation period (it will only check for non-sequences). Yes, that was my basis for it as it seemed sensible, but you're right that it's a bit of a behavioural change as you note: > * There is some value in the warning in that it lets you know an > inefficient code path is being used (i.e. the conversion to a tuple). > * The proposed logic doesn't just change the warning, it changes > what actually happens to the data. IMO the change is for the > better, but it is a behavior change and could potentially cause > a failure in someone's code. Aye, and also I guess the "sequence" implementation of the input collection might be less efficient than one-shot converting to a set and sampling from the set. > * The case of an object that is both a sequence and a set is > likely very rare. Chances are you're right, but it's what got me to stumble upon it ($dayjob makes significant use of a "unique list / ordered set" smart-ish collection, that collection was actually registered against Set and Sequence specifically because Python 3's random.sample typechecks those, we registered against both as the collection technically implements both interfaces so that seemed like a good idea at the time). -- ___ Python tracker <https://bugs.python.org/issue42470> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
New submission from Xavier Morel :
In 3.9, using `random.sample` on sets triggers
DeprecationWarning: Sampling from a set deprecated
since Python 3.9 and will be removed in a subsequent version.
*However* it also triggers on types which implement *both* Sequence and Set,
despite Sequence on its own being fine.
The issue is that it first checks for Set and triggers a warning, and only then
checks that the input is a sequence:
if isinstance(population, _Set):
_warn('Sampling from a set deprecated\n'
'since Python 3.9 and will be removed in a subsequent
version.',
DeprecationWarning, 2)
population = tuple(population)
if not isinstance(population, _Sequence):
raise TypeError("Population must be a sequence. For dicts or sets,
use sorted(d).")
the check should rather be:
if not isinstance(population, _Sequence):
if isinstance(population, _Set):
_warn('Sampling from a set deprecated\n'
'since Python 3.9 and will be removed in a subsequent
version.',
DeprecationWarning, 2)
population = tuple(population)
else:
raise TypeError("Population must be a sequence. For dicts or
sets, use sorted(d).")
this also only incurs a single instance check for `_Sequence` types instead of
two.
--
messages: 381885
nosy: xmorel
priority: normal
severity: normal
status: open
title: DeprecationWarning triggers for sequences which happen to be sets as well
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue42470>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42106] docs.python.org prioritises search horribly
Xavier Morel added the comment: Apparently it's at least a possibility on DDG's side (https://duckduckgo.com/search_box), don't know how easy it'd be to integrate in sphinx, or whether a hard dependency on an external search engine is acceptable / desirable. -- ___ Python tracker <https://bugs.python.org/issue42106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42106] docs.python.org prioritises search horribly
New submission from Xavier Morel : I expect it simply uses sphinx and I don't know if sphinx's search is easily customisable but the experience is really terrible when looking for the doc of a specific thing, *especially* when that thing is or is related to a builtin, which I'd expect to be extremely common. Though I expect part of the issue might also be that builtin or ABC methods are not documented *as such*, they're just written as code in tables. For instance if you search "append": https://docs.python.org/3/search.html?q=append On my end, neither list nor MutableSequence appear anywhere on this page, even scrolling down. Searching for "list": https://docs.python.org/3/search.html?q=list The documentation for the builtin "list" object also doesn't appear on the page. "Data Structures"[0] and "built-in types"[1] appear below the fold and the former is genuinely useful but also very easy to miss (I had not actually noticed it before going back in order to get the various links and try to extensively describe the issue). Neither actually links to the `list` builtin type though. Above the fold we find various "list" methods and classes from the stdlib as well as the PDB `list` comment, none of which seems like the best match for the query. And even Google doesn't help much there, most of the hits are for third-party documentation, and the one docs.python.org link is to the Data Structures page *of Python 2.7*. [0] https://docs.python.org/3/tutorial/datastructures.html?highlight=list [1] https://docs.python.org/3/library/stdtypes.html?highlight=list -- assignee: docs@python components: Documentation messages: 379203 nosy: docs@python, xmorel priority: normal severity: normal status: open title: docs.python.org prioritises search horribly type: enhancement versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33387] Simplify bytecodes for try-finally, try-except and with blocks.
Xavier Morel added the comment: The 3.9 changelog mentions WITH_EXCEPT_FINISH but that seems to have ultimately been called WITH_EXCEPT_START, maybe it shoudl be updated also? -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue33387> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24828] Segfault when using store-context AST node in a load context
Xavier Morel added the comment: Should I close this since I believe 2.7 is not supported anymore? -- ___ Python tracker <https://bugs.python.org/issue24828> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40325] Random.seed does not affect string hash randomization leading to non-intuitive results
Xavier Morel added the comment: @rhettinger checking software against 3.9 there's a little issue with the way the check is done: if passed something which is *both* a sequence and a set (e.g. an ordered set), `random.sample` will trigger a warning, which I don't think is correct. Should I open a new issue for that? Fix seems simple: just move the check for _Set inside the check for _Sequence, and raise if that doesn't pass either. -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue40325> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42033] Seemingly unnecessary complexification of foo(**kw)
Xavier Morel added the comment: I have not noticed anything, I was just looking at the bytecode changes and stumbled upon this oddity. Though I would expect a small slowdown as every fn(**kw) would now incur an extra dict copy, unless there’s something in call_function_ex which copies the input dict iff its ref count is not one? For whatever that’s worth, the 3.8 bytecode has been there since call_function_ex was added in 3.6 and before that call_function_kw looks identical (load_global foo, load_local var, call_function_kw) -- ___ Python tracker <https://bugs.python.org/issue42033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42033] Seemingly unnecessary complexification of foo(**kw)
New submission from Xavier Morel : Following bpo-39320 the highly specialised bytecode for vararg calls were replaced by simpler ones, but there seems to be at least one area where the generated bytecode regressed for possibly no reason? In Python 3.8, foo(**var) compiles to: 0 LOAD_GLOBAL 0 (foo) 2 BUILD_TUPLE 0 4 LOAD_FAST2 (var) 6 CALL_FUNCTION_EX 1 In Python 3.9, it compiles to: 0 LOAD_GLOBAL 0 (foo) 2 BUILD_TUPLE 0 4 BUILD_MAP0 6 LOAD_FAST2 (var) 8 DICT_MERGE 1 0 CALL_FUNCTION_EX 1 The PR 18141 does not seem to change the implementation of CALL_FUNCTION_EX so I would expect that if it was fine with taking the `var` arbitrary mapping before it stil is now, and the extra two opcodes (and creation of a dict) is unnecessary? -- messages: 378613 nosy: Mark.Shannon, xmorel priority: normal severity: normal status: open title: Seemingly unnecessary complexification of foo(**kw) versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue42033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41641] Add a "message" action to warnings, to trigger for every *unique* message
New submission from Xavier Morel :
Warning actions allow deduplicating warning triggers based on category
("once"), category + file ("module") and category + exact location ("default").
One thing which is missing is support for a single location generating warnings
*on behalf* of other pieces of code.
`warnings.warn` provides some support via the "stacklevel" parameter, but it is
difficult to compute if the warning is generated from somewhat nested generic
code (or even impossible if the warning is generated from completely generic
code checking over e.g. data files or the like, or if the invoker could be
invoked from multiple places with different depth difference from the code
they're working for).
But in that case, the warning messages will often be unique per functional
unit, so it could be a useful base for deduplication, even though the warning
could dynamically be invoked multiple times (per functional unit) because e.g.
code is reloaded or whatever.
--
components: Library (Lib)
messages: 375924
nosy: xmorel
priority: normal
severity: normal
status: open
title: Add a "message" action to warnings, to trigger for every *unique* message
type: enhancement
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue41641>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30951] Documentation error in inspect module
Xavier Morel added the comment: Maybe something along the lines of "names other than arguments and function locals", or "names of the symbols used in the code object, other than arguments and function locals"? This is still slightly confusing because in the case of an import the name is present in both mappings, but less so than the outright lie of the current version. Serhiy, is there a way to access the class's code object from Python? Also does that mean varnames is not used by class code objects despite it stating unambiguously that it stores locals? -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue30951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: > Fair enough. So we can just add a sentence informing readers that `meta_path`, by default, holds entries to handle the standard kinds of modules (.py files, extension modules…). Yeah that's basically what I meant, talking about a "warning" in python 3 may have been the wrong wording or given an incorrect impression, sorry about that. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: > A warning is probably too strong. Also, it's easy to check sys.meta_path at > the interpreter prompt, so I'm not sure it's worth mentioning at all. It's easy if you think of it and did not miss a small bit of the Python 3.3 release note indicating that the *documented guarantees* of Python 2.x, 3.0, 3.1 and 3.2 had been dropped, even as they were incorrectly still present in the official documentation itself. I spent a few hours trying to understand why our import hooks did not work correctly anymore in 3.5, and I never even considered printing sys.meta_path in an open interpreter, I realised the behavioural change because I ended up printing meta_path after the addition of our custom hooks to make sure they were actually there. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: > I'm fine with adding a note to the Python 2 docs, but putting it in Python 3 > seems to be going in the wrong direction as having sys.meta_path not be empty > is how Python will be going forward. I don't think putting a note is any hint that the Python 3 behaviour (which I appreciate) would change, it would simply tell the reader that the list is not empty by default and they ought decide whether they want their finders to take precedence over (and insert at head) or be fallback to (and append) the builtin finders. It could also link to the standard/builtin finders. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29920] Document cgitb.text and cgitb.html
Xavier Morel added the comment: Should I close this since the PR was merged? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: And it turns out the change was noted in the Python 3.3 release notes[0] though not all the consequences were spelled out (and the meta_path and path_hooks documentations were not changed) [0] https://docs.python.org/3/whatsnew/3.3.html#visible-changes -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: Addendum: the warning was present (in the documentation) until Python 3.5, even though Python 3.3 is apparently where the "default finders" were moved to meta_path: > python3.2 -c 'import sys;print(sys.meta_path)' [] > python3.3 -c 'import sys;print(sys.meta_path)' [, , ] -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
New submission from Xavier Morel: Encountered this issue porting Python 2 code manipulating meta_path to Python 3. In Python 2, meta_path is empty by default and its documentation specifically notes that: > sys.meta_path is searched before any implicit default finders or sys.path. As a result, sys.meta_path.append() works perfectly well and is overwhelmingly common[0]. In Python 3, this note was removed but the documentation does not specifically state default finders are present in meta_path, and the old code using sys.meta_path.append may now break as the default finders will silently take precedence on the custom ones if they believe they can do the job. I'd like to add a warning to the Python 3 documentation (3.5+?) noting the presence of existing default loaders, and to the Python 2 documentation suggesting `sys.meta_path.insert(0, finder)` for future-proofing, would there be objections or issues? [0] https://github.com/search?q=meta_path.append=Code=✓ -- assignee: docs@python components: Documentation messages: 294919 nosy: docs@python, xmorel priority: normal severity: normal status: open title: Warn that meta_path is not empty type: behavior versions: Python 2.7, Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29920] Document cgitb.text and cgitb.html
Changes by Xavier Morel <xavier.mo...@masklinn.net>: -- pull_requests: -747 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29920] Document cgitb.text and cgitb.html
Xavier Morel added the comment: PR targetted to master rather than 2.7 -- pull_requests: +748 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29920] Document cgitb.text and cgitb.html
New submission from Xavier Morel: Currently, cgitb documents the hook (enable) and somewhat unclearly the ability to dump the HTML traceback to stdout, but despite that being technically available it does not document the ability to dump the traceback to a string as either text or html. Possible further improvement: make ``cgitb.html`` and ``cgitb.text`` implicitly call `sys.exc_info()` if not given a parameter (much like `cgitb.handler` does). -- assignee: docs@python components: Documentation messages: 290608 nosy: docs@python, xmorel priority: normal pull_requests: 747 severity: normal status: open title: Document cgitb.text and cgitb.html type: enhancement versions: Python 2.7, Python 3.6, Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24828] Segfault when using store-context AST node in a load context
New submission from Xavier Morel: It looks to be fixed in 3.3 and up, but in Python 2.7 import ast m = ast.Module(body=[ ast.Expr(value=ast.Name(id='foo', ctx=ast.Store())) ]) ast.fix_missing_locations(m) code = compile(m, '', mode='exec') eval(code) will segfault on eval. So will a similarly incorrect ast.Attribute node. Version tested: Python 2.7.10 (default, May 28 2015, 12:02:55) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin -- components: Library (Lib) messages: 248269 nosy: xmorel priority: normal severity: normal status: open title: Segfault when using store-context AST node in a load context type: crash versions: Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24828 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Changes by Xavier Morel xavier.mo...@masklinn.net: -- nosy: +xmorel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Changes by Xavier Morel xavier.mo...@masklinn.net: -- nosy: +xmorel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14776 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21590] Systemtap and DTrace support
Changes by Xavier Morel xavier.mo...@masklinn.net: -- nosy: +xmorel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21590 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15639] csv.Error description is incorrectly broad
Xavier Morel added the comment: Correction: csv also seems to raise csv.Error if the file contains NUL bytes: Error: line contains NULL byte -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15639 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15639] csv.Error description is incorrectly broad
New submission from Xavier Morel: In both Python 2.7 and Python 3.x, csv.Error is documented as: Raised by any of the functions when an error is detected. As far as I can tell from using the module and looking at the code, this is completely incorrect. There is actually a single instance of csv.Error being used: the instantiation of csv.Dialect (which converts TypeError raised from _csv._Dialect() into csv.Error, a comment notes that this is for compatibility with py 2.3). And the only way to hit that code paths seems to be subclassing `Dialect` and putting incorrect values in the various attributes (providing them to `csv.reader` raises a TypeError). I believe the documentation to csv.Error should be changed to: 1. Mark it as effectively deprecated 2. Indicate that the only situation in which it it may be raised is when initializing a subclass of csv.Dialect -- assignee: docs@python components: Documentation messages: 168096 nosy: docs@python, xmorel priority: normal severity: normal status: open title: csv.Error description is incorrectly broad versions: Python 2.7, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15639 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: Had some time to play with this today, here's a draft matrix of actions and add_argument parameters which is pretty readable, but: * It's incredibly not helpful for people who don't know argparse * I tried adding effects descriptions in the cells instead of mere tick marks, the table becomes completely unreadable. I added a note directive below the table but it only lists a few really important/weird things, and it really won't scale beyond the current 3 items (which might already be too much) * I completely removed the ``help`` action from the table as it's unlikely anyone will want to override it (and its row was completely blank) * Hyperlinking and cross-linking (to the params, the actions, footnotes) would probably be a good idea, although it would definitely make the raw text (in-rst) I also tried my hand at formatting ``nargs``, but I don't see it as much clearer than http://docs.python.org/library/argparse.html#nargs without the examples, it's still just a mapping from a value to a behavior. I think the result would be just as good if the current ``nargs`` description was made into a definition list (in effect, it already is one emulated through an unordered list) and the examples were moved or removed. -- Added file: http://bugs.python.org/file25018/argparse-actions-matrix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: completion for list item 4: although it would definitely make the raw text (in-rst) much harder to read compared to the current table (which can be used from the rst source without compiling) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: Creating the tables should not be too hard, especially using e.g. org-mode, but: 1. Where should those tables live? The argparse documentation is pretty big and there's no completely obvious place. I would guess table 1. could just replace the list of arguments in http://docs.python.org/py3k/library/argparse.html#the-add-argument-method but things are harder for `action` (as many actions have examples as well, so the listing can't just be replaced) and for `nargs` 2. If the current lists of `argument: role` (in ArgumentParser and add_argument) are not sufficient, why would a table somehow be considering it would *add* visual clutter (table borders, for instance)? Maybe a good alternative would be to build a table style for info fields lists and to use :param: wherever that belongs? Or the doc could be split into a quickstart with just a listing of arguments and a *very simple* example or two, and then the exhaustive documentation, which could even be a separate document? -- nosy: +xmorel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: My specific suggestion is to have a dedicated Quick Reference section before the first example. OK, that looks like a good plan. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: The Parameters column would span multiple lines, with one parameter and a brief description of the parameter on each line. I started looking into that, and it turns out that's more annoying than expected: a bunch of parameters are shared by many (to all) actions, leading to lots of duplications in the table. And the full matrix of actions to parameters is not really explained in the doc. In fact, I'm coming around to thinking a matrix of the interaction between actions and arguments could be better and clearer than a table of actions with parameters bunched together at the end. In any case, it would certainly be more maintainable... except for rST not really having support for attributes on data tables, and (as far as I can tell) can't handle horizontal headers. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12056] … (HORIZONTAL ELLIPSIS) should be an alternative syntax for ... (FULL STOP FULL STOP FULL STOP)
New submission from Xavier Morel xavier.mo...@masklinn.net: In Python 3, ... became useable as a normal expression, and translates into an ellipsis instance. Unicode defines an ellipsis character … (U+2026 HORIZONTAL ELLIPSIS) which is canonically equivalent to a 3-sequence of FULL STOP [U+002E U+002E U+002E] I think it would be nice if Python supported … as an alternative to ... -- components: Interpreter Core, Unicode messages: 135771 nosy: xmorel priority: normal severity: normal status: open title: … (HORIZONTAL ELLIPSIS) should be an alternative syntax for ... (FULL STOP FULL STOP FULL STOP) type: feature request versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12056 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11979] Minor improvements to the Sockets readme: typos, wording and sphinx features usage
New submission from Xavier Morel xavier.mo...@masklinn.net: First patch fixes a typo (the reads a reply - then reads a reply) and — I believe — improves the wording of a pair of sentences. Second patch makes use of Sphinx's ``:abbr:`` role, and removes some period which I think are redundant with the use of ``::`` -- assignee: docs@python components: Documentation files: wording messages: 134970 nosy: docs@python, xmorel priority: normal severity: normal status: open title: Minor improvements to the Sockets readme: typos, wording and sphinx features usage versions: Python 3.3 Added file: http://bugs.python.org/file21853/wording ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11979] Minor improvements to the Sockets readme: typos, wording and sphinx features usage
Changes by Xavier Morel xavier.mo...@masklinn.net: Added file: http://bugs.python.org/file21854/sphinx-features ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: Do tests currently exist for smtpd run as a script? I have to confess I didn't think to check. If not, our experience with converting compileall to argparse indicates a thorough test suite is needed (and even so we missed some things we hadn't thought to test). OK, so if there is no test suite currently I should create one, and if there is one I should ensure it's complete? I guess I should use compileall as an example of how to test modules-as-scripts if the former? Overall, smtpd-as-a-script is really pretty simple, it totals 28 lines apart from the argument parsing itself (which is a bit under 60 lines ignoring the help text/pattern and gets a bit under 50 including it post-patch), and as I mentioned the only part which actually needed changing is the arguments parsing, which was very well factored out. In other words, if the current code works, is updating it a sufficient reason to change it, considering the chances of introducing new bugs? I'm not sure, but one of the ways I see standard libraries is not just as ready to use code, it's also as a repository of how to do things. And to that end, if argparse is now considered the canonical way to parse command-line arguments (which I would expect), there are very few examples of how to do things in the stdlib itself (though there are examples outside of it, due to the life argparse had pre-stdlib). It also rose to the occasion as I was wondering about the numerous standard library modules-as-scripts which are either undocumented or under-documented: because it had a good command-line documentation and a clean separation between the configuration (options parsing) and the actual execution, but no module documentation (in Doc/) it seemed like a good starting point: if it's not feasible to correctly convert best cases (and smtpd is probably one, short of modules using optparse probably) then the whole idea is stillborn: I do not see how it would be possible to fare better on some of the fully undocumented modules using manual options parsing, yet it would have to be expected. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: any of the undocumented command-line interfaces are intentionally undocumented -- they were there for the convenience of the developer for exercising the module as it was being developed and are not part of the official API. I can understand that, but it's not clear from the source code which is which, and for several of them third-parties have taken up the convenience. Maybe a more formal process of defining which tools are to be considered official and which are easter-eggs is needed? In any case, even for easter eggs surely a good command-line documentation is a good idea (even if the module documentation side is not added due to the easter egg status) isn't it? The standard library's primary purpose is to be a library for use in user programs, not to serve as a suite of applications. Sure, and I don't think most of the module-as-scripts have what it takes to be seen as applications, but many are useful and/or interesting helpers/shortcuts for day to day tasks. http.server and smtpd are good examples of useful modules-as-scripts (http.server being the exact opposite of smtpd in that it has a module-as-script documentation but no documentation whatsoever on the command-line) Real apps need more features than we usually offer and they need much faster development cycles than supported by our 18-24 month release cycle. I think my suggestion has been misunderstood: I don't want to turn these into real apps, they're fine as basic scripts/helpers to spend 5mn on a task instead of half an hour. I think they deserve better than to be documented and known through StackOverflow or Reddit what are the -m stdlib features lists. To the extent the standard library serves as a set of examples, that is just side benefit. There are other ways to make code specifically for examples (we include tons of examples in the documentation; we had a Demos directory; there is the wiki; and there is the ASPN Cookbook site; etc) Sure, but I think it's still an important driver of how things are done in that they're *concrete* examples, not abstract (of course the Cookbook is generally concrete as well). I'm not discounting the importance or quality of the rest of the documentation at all, or at least that was not my intention. All that being said, there are some exceptions and it make may sense to document the interface in some where we really do want a command-line app. I'll look at any patches you want to submit, but try to not go wild turning the library into a suite of applications. For the most part, that is not what the standard library is about. As I said, my only intention here is be to document (and argparsify/formalize) what is already there. I considered doing more (e.g. for the specific case of smtpd-as-a-script making ports optional even when hosts are specified, that kind of things) but decided against this distraction, I really just want to make existing module-as-script features simpler to discover and use. That said, do you have guidelines of which areas this idea would be most interestingly/efficiently applied? Maybe a list of modules-as-scripts you know are used regularly and could benefit from some improvements in their interface or documentation? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: One more note I forgot previously: the conversion of as much scripts as possible to argparse would be for three reasons: * Make behavior consistent across the board (e.g. -h/--help) * Make CLI documentation format consistent across the board either so users know what to expect and when * Provide easy to reach (for users) examples of using argparse -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: Barry, do I correctly understand your comment to mean I should write end-to-end tests of the CLI (until reaching the already tested meat of smtpd), not just the CLI options parsing? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: Only document and formalize the parts that are well thought out. I don't believe I have the knowledge, right or ability to make that call for any module or package but a pair of extremely obvious ones (http.server seems a pretty good candidate as it's documented in the module doc, and is just missing a CLI help). Should I create a bug and nosy the people who have been involved in the module to get their opinion for each one? (as with this one, but maybe with a different wording/initial proposal) For this one, David expresses concerns on the stability on the interface (and the point of the idea), you express even bigger concerns with the idea itself and more generally modules-as-scripts in the stdlib (not their existence so much as their official support, which full documentation would imply, if I didn't misunderstand you), and Barry seems to okay the idea as long as an extensive test suite is created beforehand to ensure there is no regression, is that sufficient to go ahead with more work and defer the final decision for when that will be done? Also, if this is to become an actual project if the smtpd stuff ever reaches fruition (though not necessarily a big or impactful one), should there be a meta-bug? (I know they're used in many bugzilla projects and I see the tracker handles dependencies) There's no harm in adding --help or a usage example, but please avoid making behavior guarantees that we really don't want to have to live with. I'm not sure what that results in: for quick hacks which are not to be officially supported, does this mean fixing the CLI (adding a help for tools missing one) is OK but no more? Is an argparse switch still acceptable? Tests for the CLI tool? I'm guessing documentation in the module would definitely be off limits for those, is my interpretation correct? And again, who would be allowed to make the call on which modules-as-scripts may be considered supported, and can't be? With this one, I created separate patches for the documentation and the CLI parsing alterations, which would allow merging the CLI without adding it to the official documentation (which would I think imply a lack of official support), would that be OK for future works, if this one pans out? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
New submission from Xavier Morel xavier.mo...@masklinn.net: argparse has been merged to the standard library in 3.2, and (tell me if I'm wrong) would therefore be the best-practices way to parse command-line arguments. Numerous stdlib modules can be used as scripts, but they tend to have ad-hoc documentation (if they are at all documented) and arguments parsing (using any of the 4 available methods in Python: straight from sys.argv, getopt, optparse and argparse). I picked smtpd as a first shot since it does something useful (SMTP proxy) and has a pretty good (if ad-hoc) command-line documentation. smtpd is currently using getopt for its options parsing and the argument parsing is very cleanly factored: the port only had to replace the implementation of the `parseargs` function (and add and remove some helpers). * The port keeps the existing arguments semantics (including the mandatory host:port syntax for the local and remote specs if overridden) * The port tries to maintain the old error messages, but due to the way argparse works (or the way I used it for the specs) the parity is not perfect when providing incorrect specs * The CLI help uses argparse's formatting, and the documentation for the local and remote specs was set on these arguments rather than as an epilog. The version string was also removed from the help screen * Because they are set by argparse's arguments validation, the status codes on incorrect arguments are slightly different: - running smtpd.py as a regular user without the `--nosetuid` flag still exits with status 1 - providing incorrect spec formats (missing or non-int port) or providing too many positional arguments (3 or more) now exits with status 2 (formerly 1) -- assignee: docs@python components: Documentation, Library (Lib) files: smtpd-to-argparse.diff keywords: patch messages: 128917 nosy: barry, docs@python, xmorel priority: normal severity: normal status: open title: smtpd-as-a-script feature should be documented and should use argparse versions: Python 3.3 Added file: http://bugs.python.org/file20812/smtpd-to-argparse.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: Second patch: documenting smtpd-as-a-script in the module's rst -- Added file: http://bugs.python.org/file20813/smtpd-as-script-doc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Hashtables in pyhton ...
Konrad Mühler wrote:
Hi,
are there predefinded chances to use hashtables in python? How can I use
Hashtable in python? Or do I have to implement this on my own?
Thanks
A Java Hashtable/Hashmap is equivalent to a Python dictionary, which is
a builtin objects (and not a second-class citizen in some distant, dark,
cold library full of Evil Things ©). You can access it either via the
regular constructor `dict()` or via the syntaxical shorcut
{ key1: val1, key2: val2, key3: val3}
--
http://mail.python.org/mailman/listinfo/python-list
Re: a question about zip...
KraftDiner wrote: I had a structure that looked like this ((0,1), (2, 3), (4, 5), (6,7) I changed my code slightly and now I do this: odd = (1,3,5,7) even = (0,2,4,6) all = zip(even, odd) however the zip produces: [(0, 1), (2, 3), (4, 5), (6, 7)] Which is a list of tuples.. I wanted a tuple of tuples... tuple(zip(even, odd)) and if you fear for memory efficiency tuple(iterator.izip(even, odd)) -- http://mail.python.org/mailman/listinfo/python-list
Re: python debugging question
[EMAIL PROTECTED] wrote: I am a python newbie. I have writen some 500 lines of code. There are 4 classes and in all 5 files. Now, I am trying to run the program. I am getting wrong values for the simulation results. Is there any debugging facilities in python which would let me go step by step and check the values of the variables at every step. I have done something like this in MS Visual Stdio 6.0 sometime back. For python, I am using SPE. what is a good way of debugging such large and iterative programs ? Any tips. Every help is appreciated. Thanks Check the PDB standard package, reading Debugging in Python (http://www.ferg.org/papers/debugging_in_python.html) may also help. And I think SPE provides WinPDB, check the user manual. I'd suggest trying to work out why it doesn't work at a higher level instead of relying on the debugger already, 500 lines is not much code to go through, a few well placed prints or sys.stderr.write() should be more than enough. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why I chose Python over Ruby
Bil Kleb wrote: Xavier Morel wrote: 2) Ruby does not have true first-class functions living in the same namespace as other variables while Python does : In Ruby you need extra syntax that ruins the first-class-ness : The extra syntax is a side-effect of the parensless call of method, it doesn't mean that methods are not first-class objects. The parensless calls also allow one to write beautiful DSLs with Ruby. -- Bil http://fun3d.larc.nasa.gov Yes, they're a tradeoff, they have both advantages and inconvenients. Matz decided that the advantages were more than worth the inconvenients, and it is often true indeed for Matz built the language with this very feature (and some others) in mind. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why I chose Python over Ruby
Torsten Bronger wrote: Yes, however, this is also true for Python in my opinion. Ruby's ability to generate DSLs is an order of magnitude better than Python's at least. I only know of the Lisp dialects that get better at DSLs. Check Rails' validation methods (in the models), or if you don't want to dwelve into rails (which I do understand), go check Why's Dwemthy's Array (http://poignantguide.net/dwemthy/) (note: you may need to read Why's Poignant Guide to Ruby first, the creation of the Array itself is fairly tough stuff with metaprogramming black magic and all). -- http://mail.python.org/mailman/listinfo/python-list
Re: how do you move to a new line in your text editor?
John Salerno wrote: So I'm wondering, how do you all handle moving around in your code in cases like this? Is there some sort of consistency to these things that you can write rules for your text editor to know when to outdent? It doesn't seem like you can do this reliably, though. Under windows, I'm using SciTE which is an extremely lightweight editor, but it handlers smart unindent: pressing backspace at the beginning of a line unindents one level, whether you're indenting with tabs (and need to remove a tab) or space (and need to remove 2, 4, 8 spaces) doesn't matter. And since SciTE also has Visual Studio's smart home key (home brings you first at the beginning of the text == current indent, then at the beginning of the line itself == indent level 0) SciTE also features somewhat smart indent from time to time: it indents one level after a :. This is good for if/else/while/..., but it also indents one level after : in dicts, which is way bad. Oh, and it automatically unindents one level after a return statement. Other than that, SciTE doesn't really understand python, if you want a really pythonic editor you need to check Stani's Python Editor, WingsIDE or Komodo. -- http://mail.python.org/mailman/listinfo/python-list
Re: Use python from command line
trixie wrote: Using WinXP and Python24 on generic desktop. Being used to linux and command line operations I cannot make Windows accept the 'python myprog.py' command. Any help appreciated. Bob I think you could've given us the error message that this command yields. Issue is probably that the Python directory isn't on your path -- http://mail.python.org/mailman/listinfo/python-list
Re: Why I chose Python over Ruby
I'll just play the devil's advocate here Francois wrote: 1) In Ruby there is a risk of Variable/Method Ambiguity when calling a method with no parameters without using () : Yes, but that's in my opinion a programmer error, not necessarily a language error. 2) Ruby does not have true first-class functions living in the same namespace as other variables while Python does : In Ruby you need extra syntax that ruins the first-class-ness : The extra syntax is a side-effect of the parensless call of method, it doesn't mean that methods are not first-class objects. And Ruby solved this issue with blocks/procs (that and closures are the only reasons I found for blocks to exist in ruby). In python you pass functions around, Ruby's equivalent of unbound functions are blocks/procs, what your code created here is a ruby method, equivalent to a bound method in Python, the semantics are really different (and in Python using this bound method would also require extra work). -- http://mail.python.org/mailman/listinfo/python-list
Re: How to do an 'inner join' with dictionaries
[EMAIL PROTECTED] wrote:
Let's say I have two dictionaries:
dict1 is 1:23, 2:76, 4:56
dict2 is 23:A, 76:B, 56:C
How do I get a dictionary that is
1:A, 2:B, 4:C
dict1 = {1:23,2:76,4:56}
dict2 = {23:'A',76:'B',56:'C'}
dict((k, dict2[v]) for k, v in dict1.items())
{1: 'A', 2: 'B', 4: 'C'}
?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pure python implementation of string-like class
Ross Ridge wrote: Xavier Morel wrote: Not if you're still within Unicode / Universal Character Set code space. Akihiro Kayama in his original post made it clear that he wanted to use a character set larger than entire Unicode code space. Ross Ridge He implies that, but in later messages he 1. Implies that he wants to use the Unicode private spaces, which are in the Unicode code space 2. Says explicitly that his needs concern Kanji encoding, which do fit in the existing Unicode code space, even if you take the largest estimates of the number of existing Kanjis (~8), and which are (I think) already partially represented by the CJK Unified Ideograms and CJK Unified Ideograms extension A sets of regular Unicode. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pure python implementation of string-like class
Akihiro KAYAMA wrote: Sorry for my terrible English. I am living in Japan, and we have a large number of characters called Kanji. UTF-16(U+...U+10) is enough for practical use in this country also, but for academic purpose, I need a large codespace over 20-bits. I wish I could use unicode's private space (U+6000...U+7FFF) in Python. -- kayama I think the Kanji are part of the Han script as far as Unicode is concerned, you should check it (CJK unified ideograms and CJK unified ideograms extension A), they may not all be there, but the 27502 characters from these two tables should be enough for most uses. Oh, by the way, the Unicode code space only goes up to 10, while UCS-4's encoding allows code values up to and including 7FFF the upper Unicode private space is Plane Sixteen (10–10), the other private spaces being a part of the Basic Multilingual Plane (U+E000–U+F8FF) and Plane Fifteen (U+F–U+F) and even UTF-32 doesn't go beyond 10. Since the Dai Kan-Wa jiten only lists about 50,000 kanji (even though it probably isn't perfectly complete) it fits with ease in both plane fifteen and sixteen (65535 code points each). -- http://mail.python.org/mailman/listinfo/python-list
Re: Pure python implementation of string-like class
Ross Ridge wrote: Steve Holden wrote: Wider than UTF-16 doesn't make sense. It makes perfect sense. Ross Ridge Not if you're still within Unicode / Universal Character Set code space. While UCS-4 technically goes beyond any Unicode Transformation Format (UTF-7, 8, 16 and 32 stop at 10) it also goes beyond the range of the UCS itself (0-10). UTF-32 is the limitation of UCS-4 to the Unicode standard. While it could be argued that Unicode/UCS limit of 10 was chosen _because_ of the limitations of UTF-16, It's probably irrelevant to the discussion. -- http://mail.python.org/mailman/listinfo/python-list
Re: Processing text using python
nuttydevil wrote: Hey everyone! I'm hoping someone will be able to help me, cause I haven't had success searching on the web so far... I have large chunks of text ( all in a long string) that are currently all in separate notebook files. I want to use python to read these strings of text, THREE CHARACTERS AT A TIME. (I'm studying the genetic code you see, so I need to read and analyse each sequence one codon at a time effectively.) Does anyone have any idea of how to do this using python? I'm going to be optimistic and thank you for your help in advance! Samantha. Since you're reading from files, the read operation of file-like objects takes an argument specifying the number of characters to read from the stream e.g. f = file(stuff.txt) f.read(3) 'car' f.read(3) 'act' f.read() 'erization' Would that be enough for what you need? -- http://mail.python.org/mailman/listinfo/python-list
Re: Processing text using python
Fredrik Lundh wrote: did you read the string chapter in the tutorial ? http://docs.python.org/tut/node5.html#SECTION00512 around the middle of that chapter, there's a section on slicing: substrings can be specified with the slice notation: two indices separated by a colon Fredrik, how would you use slices to split a string by groups of 3 characters? -- http://mail.python.org/mailman/listinfo/python-list
Re: define loop statement?
Rene Pijlman wrote: David Isaac: I would like to be able to define a loop statement (nevermind why) so that I can write something like loop 10: do_something instead of for i in range(10): do_something Possible? If so, how? Yes. By implementing a compiler or an interpreter for your programming language. Or a preprocessor that converts your language to Python, or some other suitable intermediate language. Or a programmer, that converts your pseudocode and some coffee to the desired algorithm :-) Or by hacking through the Python source and creating his own somehow pythonish but absolutely not python language -- http://mail.python.org/mailman/listinfo/python-list
Re: define loop statement?
Rene Pijlman wrote: David Isaac: I would like to be able to define a loop statement (nevermind why) so that I can write something like loop 10: do_something instead of for i in range(10): do_something Possible? If so, how? Yes. By implementing a compiler or an interpreter for your programming language. Or a preprocessor that converts your language to Python, or some other suitable intermediate language. Or a programmer, that converts your pseudocode and some coffee to the desired algorithm :-) Or by hacking through the Python source and creating his own somehow pythonish but absolutely not python language -- http://mail.python.org/mailman/listinfo/python-list
Re: Python, Forms, Databases
Tempo wrote: Larry I do see your point. There does seem to be a lot more support for PHP and MySQL together than there is Python and ASP. But I want to first try to accomplish my goal by using Python first before I give up and revert back to PHP. So if I was going to parse HTML forms and place the data into a MySQL database, what should I use? CGI module? Zope? Webware? Thanks for any and all help. If you're talking about a pair of page and nothing more, the CGI module and manually handling your stuff (with a DBAPI2 MySQL module for the DB link) is more than enough. If you want to create something more complex (a full database driven website), it would probably be a good idea to check some of Python's web frameworks, Django for example. -- http://mail.python.org/mailman/listinfo/python-list
Re: appending to a list via properties
Alex Martelli wrote: Carl Banks [EMAIL PROTECTED] wrote: ... class better_list (list): tail = property(None, list.append) This is an impressive, spiffy little class. Yes, nice use of property. Alex I don't know, I usually see people considering that properties are cool as long as they don't have side effects, as long as they're virtual members. The tail property doesn't behave like member data at all, the semantics are strange, counter-intuitive (tail would usually be the end of the list, either [-1] or [1:], in this case it's some magic position at the end of the list). And it has one hell of a side effect. -- http://mail.python.org/mailman/listinfo/python-list
Re: Create dict from two lists
Diez B. Roggisch wrote: py wrote: I have two lists which I want to use to create a dictionary. List x would be the keys, and list y is the values. x = [1,2,3,4,5] y = ['a','b','c','d','e'] Any suggestions? looking for an efficent simple way to do this...maybe i am just having a brain fart...i feel like this is quit simple. dict(zip(x,y)) Diez I'd even suggest using izip (from itertools), it's often much faster than zip (end result is the same). Demo: from itertools import izip l1 = range(5000) l2 = range(5000, 1) from timeit import Timer t1 = Timer(dict(zip(l1, l2)), from __main__ import l1, l2) t2 = Timer(dict(izip(l1, l2)), from __main__ import l1, l2, izip) min(t1.repeat(5, 1)) 17.989041903370406 min(t2.repeat(5, 1)) 10.381146486494799 42% speed gain from using izip instead of zip (memory was not an issue during the test, machine had 1.3Gb of available ram) -- http://mail.python.org/mailman/listinfo/python-list
Re: creat a DOM from an html document
Mark Harrison wrote: I thought I saw a package that would create a DOM from html, with allowances that it would do a best effort job to parse non-perfectly formed html. Now I can't seem to find this... does anybody have a recommendation as to a good package to look at? Many TIA! Mark While it doesn't generate a W3C DOM, BeautifulSoup is probably your best bet for parsing less-than-perfect HTML and get something useable out of it. Once you have your (parsed) document, you can either use it as is or try to convert it to a valid W3C DOM though. -- http://mail.python.org/mailman/listinfo/python-list
Re: A __getattr__ for class methods?
Dylan Moreland wrote:
I'm trying to implement a bunch of class methods in an ORM object in
order to provide functionality similar to Rails' ActiveRecord. This
means that if I have an SQL table mapped to the class Person with
columns name, city, and email, I can have class methods such as:
Person.find_by_name
Person.find_by_city_and_name
Person.find_by_name_and_city_and_email
I have a metaclass generating basic properties such as .name and .city,
but I don't want to generate a class method for every permutation of
the attributes. I'd like to have something much like __getattr__ for
instance attributes, so that if a method like
Person.find_by_city_and_email cannot be found, I can construct a call
to the basic find method that hides the SQL. Is there any way of doing
this, or am I trying to mirror a functionality that Python simply does
not have?
I'm not sure that the way you tackled this is the good approach: while
it's quite flexible as far as the search criteria go, it'll require less
than obvious code to match keywords (field names) and values, will lead
to somewhat verbose syntax (especially with many criteria), and the
syntax itself is unforgiving (every new search field requires at least 5
additional characters on top of the field name itself), brittle (you'll
have to do an extensive validation of your method name and fields unless
you want everything to break, and Python doesn't support your syntax)
and not really pythonic.
Since you seem to know ActiveRecord, you're probably familiar with the
way AR does this task: with a hash of column_name, value pairs. The
syntax is quite straightforward especially due to the fact that any key
= value comma-separated pairs sequence generates a hash, without
requiring explicit hash boundaries ( { and } ). The syntax is extremely
straightforward since it relies on the language's syntax itself, and the
correctness of the grammar is checked by the compiler/interpreter itself
since no custom syntax is built. This construct is therefore quite
solid, on top of being obvious to a Rubyist.
Now, you're in luck, because Python has even better than that: **kwargs,
the optional keyword arguments.
As you probably know, Python has a good support for keyword args,
allowing you to fill your arguments out of order (and leave the 3th
argument at it's default value while specifying the value of the 5th
argument). But **kwargs goes beyond the regular explicit keyword
arguments: when specified, **kwargs is a dict populated with the
implicit keyword arguments (keyword:value pairs), the ones you haven't
specified in the argument tuple of your method.
This means that if I define a function as
def foo(bar=0, **kwargs):
print kwargs
Then calling foo() will print an empty dict
As will calling foo(3) or foo(bar=3), because the explicit bar keyword
argument is used.
Now if I call foo(something=5, somethingelse=woohoo), kwargs will
evaluate to
{somethingelse:woohoo,something:5}
This means that all you have to do is replace your method definition with
def find_by(cls, **kwargs): pass
and in the method itself iterate over the key:value pairs of kwargs to
automagically get both the field names and the values upon which your
search shall be performed.
Calling the method would then look something like:
Person.find_by( name=thenameyoulookfor, city=somecity)
the syntax is fairly obvious and pythonic, has a low verbosity, and
Python itself will do the parsing and the grammatical validation of your
method calls for you.
--
http://mail.python.org/mailman/listinfo/python-list
Re: A __getattr__ for class methods?
Oh, and I wondered too: is your goal to build an ORM, or do you just need an ORM? Cause if it's the latter then Python does already have some fairly good ORMs such as SQLAlchemy or PyDO2, you don't *need* to create yours. -- http://mail.python.org/mailman/listinfo/python-list
Re: Ternary Operator Now?
Ben Wilson wrote: I read somewhere else that Python was getting a ternary operator (e.g. x = (true/false) ? y : z). I read the PEP about it and that the PEP had been approved this past Fall. Has this been released into the wild yet? IIRC, the operator is like: x = y if C : else z PEP 308 Conditional Expressions has been accepted for Python 2.5, I'm pretty sure implementation hasn't even started yet. -- http://mail.python.org/mailman/listinfo/python-list
Re: Dual Core outlook
malv wrote: Of course, multiprocessing has been used for many years but this always involved a much higher level of sophistication on the part of the designers. This point seems to be largely hidden from the public, ignorant and semi-ignorant, by the chip manufacturers. Will new languages see the light rendering the spreading of applications over many processors quasi transparent? What is the outlook for Python? Would Ironpython with .net do better? What about talk by the Java lobby that Java would be very much suited for taking advantage of dual-core? Is there any thruth to this? Python has support for OS threads, but it also has the GIL (a global lock from the interpreter). This means that Python behaves the same way Java or C# do as far as threads are concerned but for one field: multiprocessor (or multicore) situations. While C# or Java can execute two threads of the same process on two separate cores/processors Python can't. This means that you can't choke a multiprocessor/multicores machine with Python threads, and that if you want to scale in a multiprocessor/multicore environment (e.g. make use of all the available cores/processors) you have to use processes not threads (e.g. you have to launch multiple instances of the Python interpreter, each one having the ability to execute on a different core/processor). Now the question is: do you want to do it? Do you have several extremely demanding computational threads? If yes, then Python probably isn't what you need (unless you implement your computations in specific C modules and get rid of the GIL issue). If not (a single computational thread and several low-resources GUI/network/whatever threads), it's a non-issue. -- http://mail.python.org/mailman/listinfo/python-list
Re: Dual Core outlook
malv wrote: Maybe this is too simplistic, but given two programs, one in Python the other in Java or C#. Would this mean that running the latter on a dual core processor would significantly increase execution speed, whereas the Python program would be running in one processor only without any speed up? This is extremely dependent on the way the program is built. To get significant increases in execution speed, you need multiple *computational* threads, that means that you need 2+ threads that do the heavy work (number crunching, compression, ...), if you have one thread with heavy computations and the others handling low-computation parts of the program (UI) you may get a more _responsive_ program, but it won't be _faster_ (at least not noticeably), because the bottleneck of the application (the heavy computations) will still run on a single core/processor. Multicore/multiprocessor is not magic, and it's not trivial at all, it's hard to create correct multithreaded programs, and even harder to parallelize the heavy computation part between various threads. Is a Java program capable of this out of the box or does this require specific additional code? Java and C# both use unlocked OS threads any time you use their threads. -- http://mail.python.org/mailman/listinfo/python-list
Re: tricky regular expressions
Ernesto wrote: I'm not sure if I should use RE's or some other mechanism. Thanks I think a line-based state machine parser could be a better idea. Much simpler to build and debug if not faster to execute. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is Python good for web crawlers?
Paul Rubin wrote: Generally I use urllib.read() to get the whole html page as a string, then process it from there. I just look for the substrings I'm interested in, making no attempt to actually parse the html into a DOM or anything like that. BeautifulSoup works *really* well when you want to parse the source (e.g. when you don't want to use string matching, or when the structures you're looking for are a bit too complicated for simple string matching/substring search) The API of the package is extremely simple, straightforward and... obvious. -- http://mail.python.org/mailman/listinfo/python-list
Re: tricky regular expressions
Ernesto wrote: Xavier Morel wrote: Ernesto wrote: I'm not sure if I should use RE's or some other mechanism. Thanks I think a line-based state machine parser could be a better idea. Much simpler to build and debug if not faster to execute. What is a line-based state machine ? Parse your file line-by-line (since it seems that it's the way your data is organized). Keep state informations somewhere. Change your state based on the current state and the data being fed to your parser. For example, here you basically have 3 states: No Title, which is the initial state of the machine (it has not encountered any title yet, and you do stuff based on titles) Title loaded, when you've met a title. Title loaded loops on itself: if you meet a Title: whatever line, you change the title currently stored but you stay in the Title loaded state (you change the current state of the machine from title loaded to title loaded). Request loaded, which can be reached only when you're in the Title loaded, and then encounter a line starting with Request: . When you reach that stage, do your processing (you have a title loaded, which is the latest title you encountered, and you have a request loaded, which is the request that immediately follows the loaded title), then you go back to the No Title state, since you've processed (and therefore unloaded) the current title. So, the state diagram could kind of look like that: (it's supposed to be a single state diagram, but i suck at ascii diagrams so i'll create one mini-diagram for each state) NoTitle =0 TitleLoaded =0 Event: on encountering a line starting with Title: Action: save the title (to whatever variable you see fit) Change state to: TitleLoaded TitleLoaded =1 TitleLoaded || 2 \/ Request =1 Event: on encountering a line starting with Title: Action: save the title (replace the current value of your title variable) Change state to: TitleLoaded =2 Event: on encountering a line starting with Request: Action: save the request?; immediately process the Request state Change state to: Request Request =3 NoTitle || 4 \/ TitleLoaded =3 Event: the Request state is reached, the request is either Play or Next Action: Do whatever you want to do; nuke the content of the title variable Change state to: NoTitle =4 Event: the Request state is reached, the request is neither Play nor Next Action: Nuke the content of the request variable (if you saved it), do nothing else Change state to: TitleLoaded As a final note, i'd recommend reading Text Processing in Python, even though it puts a quite big emphasis on functional programming (which you may or may not appreciate), it's an extremely good initiation to text-files handling, parsing and processing. -- http://mail.python.org/mailman/listinfo/python-list
Re: Hi reliability files, writing,reading and maintaining
Terry Reedy wrote: John Pote [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] I would wish to secure this data gathering against crashes of the OS, I have read about people running *nix servers a year or more without stopping. He'd probably want to check the various block-journaling filesystems to boot (such as Reiser4 or ZFS). Even though they don't reach DB-level of data integrity they've reached an interresting and certainly useful level of recovery. To transparently write to duplicate disks, lookup RAID (but not level 0 which I believe does no duplication). Indeed, Raid0 stores data across several physical drives (striping), Raid1 fully duplicates the data over several physical HDs (mirror raid), Raid5 uses parity checks (which puts it between Raid0 and Raid1) and requires at least 3 physical drives (Raid0 and Raid1 require 2 or more). You can also nest Raid arrays, the most common nesting are Raid 01 (creating Raid1 arrays of Raid0 arrays), Raid 10 (creating Raid0 arrays of Raid1 arrays), Raid 50 (Raid0 array of Raid5 arrays), and the Raids for Paranoids, Raid 15 and Raid 51 arrays (creatung a Raid5 array of Raid1 arrays, or a Raid1 array of Raid5 arrays, both basically means that you're wasting most of your storage space for redundancy informations, but that the probability of losing any data is extremely low). -- http://mail.python.org/mailman/listinfo/python-list
Re: Importing a class, please help...
anon wrote: Would somebody please drop me a hint, please? Yeah, the definition of JAR is Java ARchive, why the hell would a Python script be able to read a JAR in the first place (truth is it is, a JAR file is nothing but a renamed ZIP, therefore the zipfile module allows you to read it's content) and -- more importantly -- import Java classes. Notice the difference between *Python* script and *Java* class? That's because they're two different languages. If you want to use Java classes and modules in a Pythonic context, you want Jython (http://www.jython.org/) not Python (http://www.python.org/) -- http://mail.python.org/mailman/listinfo/python-list
Re: Learning Python
Byte wrote: x = input(raw_input(Please enter your name: )) if x==myself: print 'OK' It kinda works - I can get to the please enter your name bit but then it simply reprints your input as output. Someone please HELP! -- C:\python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. help(input) Help on built-in function input in module __builtin__: input(...) input([prompt]) - value Equivalent to eval(raw_input(prompt)). help(raw_input) Help on built-in function raw_input in module __builtin__: raw_input(...) raw_input([prompt]) - string Read a string from standard input. The trailing newline is stripped. If the user hits EOF (Unix: Ctl-D, Windows: Ctl-Z+Return), raise EOFError. On Unix, GNU readline is used if enabled. The prompt string, if given, is printed without a trailing newline before reading. -- Where the hell did you get the idea of stacking input on a raw_input in the first place? -- http://mail.python.org/mailman/listinfo/python-list
Re: Learning Python
Byte wrote: p.s. Xavier Morel, you seem to be using Windows, not Linux I don't see how this may even remotely be relevant, Python is cross platform and (mostly) works the same regardless of the OS I got the idea of stacking input on a raw_input from the official Python Tutorial. Reference please, I never saw that anywhere in the tutorial -- http://mail.python.org/mailman/listinfo/python-list
Re: Learning Python
Byte wrote: http://docs.python.org/tut/node6.html#SECTION00610 -- x = int(raw_input(Please enter an integer: )) -- Unless my eyes fail me, it's written int, not input, the goal of this line is to convert the return value of raw_input (a string) into an integer. -- http://mail.python.org/mailman/listinfo/python-list
Re: Learning Python
Byte wrote: Assumption. Im also new to programing, so could do something stupid like think a Windows path is a command/input/etc. (really, ive done things like that before.) Don't assume anything when you have no reason to, and especially don't assume that a cross-platform programming language behaves differently from a platform to another, especially on built-in basic functions Now, im running this on a terminal, but am acctually writing my scripts in a text editor. How can I get a script to do a sum with the editor? e.g. it asks for a sum, then does it etc. parse the expression, extract the operands and the operation, apply the operation to the operands (are you trying to code a calculator?) -- http://mail.python.org/mailman/listinfo/python-list
