[Python-ideas] Re: Context manager for csv module 'reader' and 'DictReader'?
On Mon, 6 Sept 2021 at 01:13, Greg Ewing wrote: > > On 6/09/21 3:07 am, C. Titus Brown via Python-ideas wrote: > > with csv.DictReader.open(filename) as r: > > for row in r: > >… > > You can do this now: > > from contextlib import closing > with closing(csv.DictReader.open(filename)) as r: > ... What version of Python are you using? >>> import csv >>> csv.DictReader.open Traceback (most recent call last): File "", line 1, in AttributeError: type object 'DictReader' has no attribute 'open' > IMO this is preferable than going around adding context manager > methods to everything that has open-like functionality. I disagree. It would be better if resource acquisition (e.g. opening a file) always took place in an __enter__ method so that it could always be under control of a with statement. Having closing as a separate function negates that because there has to be a separate function that acquires the resource before the closing function is called and hence before __enter__ is called. -- Oscar ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AR6IHZ2HY4TKZXKANUGA7HTPDQMBCLRC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Context manager for csv module 'reader' and 'DictReader'?
On 6/09/21 3:07 am, C. Titus Brown via Python-ideas wrote: with csv.DictReader.open(filename) as r: for row in r: … You can do this now: from contextlib import closing with closing(csv.DictReader.open(filename)) as r: ... IMO this is preferable than going around adding context manager methods to everything that has open-like functionality. -- Greg ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RGCQJZSVMKBU36YK27XGXJI3OUQFBL2L/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Context manager for csv module 'reader' and 'DictReader'?
On Sun, Sep 5, 2021 at 10:32 AM David Mertz, Ph.D. wrote: > Most Pandas read methods take either a path-like argument or a file-like > argument, and figure out which it is by introspection when called. > Actually, most of them even accept a URL-like argument as well > > I don't think this is a terrible approach. It doesn't make things quite as > explicit as the standard library generally does. > The folks in favor of adding this to json are split between overloading load() (my preference) and adding a new, explicit method: loadf() or something like that. Either way, it's useful. I honestly don't get the objection. MY takle is that mosto f the core devs are doing "systsms programming" rather than "scripting" -- which means that they a: Want to be more explicit and robust Need more flexibiilty around various file-like objects Don't mind a bit more expertise required. (e.g. specifying the encoding correctly) Don't mind a couple extra lines of code. So why add a function to make it simple and easy to read a file fro a path-like? I really wish the core devs would remember how useful Python is as a scripting language, and how many people use it that way. Pandas is a good example, I lot of folks use it in a scripting context, so it provided features that make it easy to do so. -CHB On Sun, Sep 5, 2021, 1:19 PM Christopher Barker wrote: > >> >> This would only be helpful when the CSV is on the disk but csv.reader() >>> takes a file object so that it can used with anything like a socket for >>> example. json.load() does the same thing. >>> >> >> There has been discussion about adding loading from a “path like” to the >> JSON lib. See this list about ten months ago and a recent thread on discuss. >> >> There resistance, but it ended with the idea that maybe there should be a >> PEP for a common interface for all “file” readers — eg JSON, CSV, etc.. And >> that interface could be supported by third party libs. That interface >> *maybe* would include a single step load from a path-like functionality. >> >> It seems to me that it is better to keep a generic interface that are >>> composable. >>> >> >> No one is suggesting removing the load-from-a-file-like interface. I have >> no idea why the fact that some people sometimes need a more flexible >> interface (maybe most people, even) somehow means that we shouldn’t make >> things easy and obvious for a common use case. >> >> -CHB >> >> >> >> >>> Cheers, >>> Rémi >>> >>> >>> >>> ? And something similar for ‘csv.reader’? I’m not wedded to the details >>> here. >>> >>> The two main reasons I think this might be a positive addition are - >>> >>> * you wouldn’t have to know or remember the right way to open a CSV file >>> (newline=“”). >>> * it elides very common code. >>> >>> but perhaps there are things I’m missing here? >>> >>> As a side note, I think ‘csv.reader’ could usefully be renamed to >>> something else (maybe just Reader?), since it’s kind of out of sync with >>> the CamelCase used in ‘DictReader’. But maybe that’s just an attempt at >>> foolish consistency :). >>> >>> best, >>> —titus >>> >>> ___ >>> Python-ideas mailing list -- python-ideas@python.org >>> To unsubscribe send an email to python-ideas-le...@python.org >>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-ideas@python.org/message/EKHYCTYMXZG3VI4JYFA3Y3LD3ZNMI3IX/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >>> >>> ___ >>> Python-ideas mailing list -- python-ideas@python.org >>> To unsubscribe send an email to python-ideas-le...@python.org >>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-ideas@python.org/message/ZJNJQFC2LKQ76GTEBQNKTB3WO2POTKEN/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> -- >> Christopher Barker, PhD (Chris) >> >> Python Language Consulting >> - Teaching >> - Scientific Software Development >> - Desktop GUI and Web Development >> - wxPython, numpy, scipy, Cython >> ___ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/GAPBNBGLOWD27FCTJJV7FEPUZRKUE6FL/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org
[Python-ideas] Re: Improve traceback for common `with` statement mistake
On Mon, Sep 6, 2021 at 9:37 AM Finn Mason wrote:
>
> Hello all,
>
> In Python 3.10 and 3.11, exception tracebacks are being greatly improved. I
> noticed that there's nothing related to a fairly common (in my personal
> experience) cryptic traceback relating to the `with` statement:
>
> >>> with ContextManager as ctx:
> ... # do something with `ctx`
> ...
> Traceback (most recent call last):
> File "", line 1, in
> AttributeError: __enter__
I'm seeing a different message, so it looks like something HAS been improved:
Python 3.11.0a0 (heads/main:ed524b4569, Aug 14 2021, 11:29:01) [GCC
8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import contextlib
>>> with contextlib.ExitStack: ...
...
Traceback (most recent call last):
File "", line 1, in
TypeError: 'ABCMeta' object does not support the context manager protocol
> This occurs when one forgets to use a instance of a context manager class and
> uses the class itself. It's obviously not a very helpful traceback. ("Is it
> not a context manager?" "Is it the wrong class?") Something like the
> following would be better.
>
> >>> with ContextManager as ctx:
> ... # do something with `ctx`
> ...
> Traceback (most recent call last):
> File "", line 1, in
> AttributeError: is not a context manager. Did you mean
> "with ContextManager()..."?
>
> The actual traceback message should probably be more specific than " Context manager> is not a context manager".
> Thoughts?
>
The "did you mean" part depends on or assumes that it can figure out
that calling it would have given a viable context manager. This could
be done by probing whether thing.__enter__ exists, but I'm not sure
that'd be entirely safe. Also, it won't catch those made from
generators:
>>> @contextlib.contextmanager
... def foo(): yield
...
>>> with foo: ...
...
Traceback (most recent call last):
File "", line 1, in
TypeError: 'function' object does not support the context manager protocol
>>> with foo(): ...
...
Ellipsis
>>> foo
In any case, the biggest advantage (IMO) comes from naming the type,
which will make it easy to distinguish the object from its type.
ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/NGP5LQSRS2XPNHIRBN3O76UGMEB623TW/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Improve traceback for common `with` statement mistake
Hello all,
In Python 3.10 and 3.11, exception tracebacks are being greatly improved. I
noticed that there's nothing related to a fairly common (in my personal
experience) cryptic traceback relating to the `with` statement:
>>> with ContextManager as ctx:
... # do something with `ctx`
...
Traceback (most recent call last):
File "", line 1, in
AttributeError: __enter__
This occurs when one forgets to use a instance of a context manager class
and uses the class itself. It's obviously not a very helpful traceback.
("Is it not a context manager?" "Is it the wrong class?") Something like
the following would be better.
>>> with ContextManager as ctx:
... # do something with `ctx`
...
Traceback (most recent call last):
File "", line 1, in
AttributeError: is not a context manager. Did you
mean "with ContextManager()..."?
The actual traceback message should probably be more specific than " is not a context manager".
Thoughts?
--
Finn
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/5NHD4LS4NGTXNM5RDKCCSBXQPURBTC4Q/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Context manager for csv module 'reader' and 'DictReader'?
Most Pandas read methods take either a path-like argument or a file-like argument, and figure out which it is by introspection when called. Actually, most of them even accept a URL-like argument as well I don't think this is a terrible approach. It doesn't make things quite as explicit as the standard library generally does. But it's convenient, and there's no real ambiguity. On Sun, Sep 5, 2021, 1:19 PM Christopher Barker wrote: > > This would only be helpful when the CSV is on the disk but csv.reader() >> takes a file object so that it can used with anything like a socket for >> example. json.load() does the same thing. >> > > There has been discussion about adding loading from a “path like” to the > JSON lib. See this list about ten months ago and a recent thread on discuss. > > There resistance, but it ended with the idea that maybe there should be a > PEP for a common interface for all “file” readers — eg JSON, CSV, etc.. And > that interface could be supported by third party libs. That interface > *maybe* would include a single step load from a path-like functionality. > > It seems to me that it is better to keep a generic interface that are >> composable. >> > > No one is suggesting removing the load-from-a-file-like interface. I have > no idea why the fact that some people sometimes need a more flexible > interface (maybe most people, even) somehow means that we shouldn’t make > things easy and obvious for a common use case. > > -CHB > > > > >> Cheers, >> Rémi >> >> >> >> ? And something similar for ‘csv.reader’? I’m not wedded to the details >> here. >> >> The two main reasons I think this might be a positive addition are - >> >> * you wouldn’t have to know or remember the right way to open a CSV file >> (newline=“”). >> * it elides very common code. >> >> but perhaps there are things I’m missing here? >> >> As a side note, I think ‘csv.reader’ could usefully be renamed to >> something else (maybe just Reader?), since it’s kind of out of sync with >> the CamelCase used in ‘DictReader’. But maybe that’s just an attempt at >> foolish consistency :). >> >> best, >> —titus >> >> ___ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/EKHYCTYMXZG3VI4JYFA3Y3LD3ZNMI3IX/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> ___ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/ZJNJQFC2LKQ76GTEBQNKTB3WO2POTKEN/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -- > Christopher Barker, PhD (Chris) > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/GAPBNBGLOWD27FCTJJV7FEPUZRKUE6FL/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IXWTRZ6RINR5OZECMXHAP7B7MUUY2RH5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Context manager for csv module 'reader' and 'DictReader'?
> This would only be helpful when the CSV is on the disk but csv.reader() > takes a file object so that it can used with anything like a socket for > example. json.load() does the same thing. > There has been discussion about adding loading from a “path like” to the JSON lib. See this list about ten months ago and a recent thread on discuss. There resistance, but it ended with the idea that maybe there should be a PEP for a common interface for all “file” readers — eg JSON, CSV, etc.. And that interface could be supported by third party libs. That interface *maybe* would include a single step load from a path-like functionality. It seems to me that it is better to keep a generic interface that are > composable. > No one is suggesting removing the load-from-a-file-like interface. I have no idea why the fact that some people sometimes need a more flexible interface (maybe most people, even) somehow means that we shouldn’t make things easy and obvious for a common use case. -CHB > Cheers, > Rémi > > > > ? And something similar for ‘csv.reader’? I’m not wedded to the details > here. > > The two main reasons I think this might be a positive addition are - > > * you wouldn’t have to know or remember the right way to open a CSV file > (newline=“”). > * it elides very common code. > > but perhaps there are things I’m missing here? > > As a side note, I think ‘csv.reader’ could usefully be renamed to > something else (maybe just Reader?), since it’s kind of out of sync with > the CamelCase used in ‘DictReader’. But maybe that’s just an attempt at > foolish consistency :). > > best, > —titus > > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/EKHYCTYMXZG3VI4JYFA3Y3LD3ZNMI3IX/ > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/ZJNJQFC2LKQ76GTEBQNKTB3WO2POTKEN/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GAPBNBGLOWD27FCTJJV7FEPUZRKUE6FL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Context manager for csv module 'reader' and 'DictReader'?
> On 5 Sep 2021, at 17:07, C. Titus Brown via Python-ideas > wrote: > > Hi all, > > the product of Sunday morning idle curiosity... > > I’ve been using the csv module a lot, and I’m wondering if there would be > value in adding a standard mechanism for opening a CSV file (correctly) using > a context manager? > > So, instead of > > with open(filename, newline=“”) as fp: > r = csv.DictReader(fp) > for row in r: > … > > support something like > > with csv.DictReader.open(filename) as r: > for row in r: > … This would only be helpful when the CSV is on the disk but csv.reader() takes a file object so that it can used with anything like a socket for example. json.load() does the same thing. It seems to me that it is better to keep a generic interface that are composable. Cheers, Rémi > > ? And something similar for ‘csv.reader’? I’m not wedded to the details here. > > The two main reasons I think this might be a positive addition are - > > * you wouldn’t have to know or remember the right way to open a CSV file > (newline=“”). > * it elides very common code. > > but perhaps there are things I’m missing here? > > As a side note, I think ‘csv.reader’ could usefully be renamed to something > else (maybe just Reader?), since it’s kind of out of sync with the CamelCase > used in ‘DictReader’. But maybe that’s just an attempt at foolish consistency > :). > > best, > —titus > > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/EKHYCTYMXZG3VI4JYFA3Y3LD3ZNMI3IX/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZJNJQFC2LKQ76GTEBQNKTB3WO2POTKEN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Context manager for csv module 'reader' and 'DictReader'?
Seems nice, tarfile has a similar shortcut too. I do tend to reach for pandas now whenever I can for csv processing On Sun, 5 Sep 2021, 16:10 C. Titus Brown via Python-ideas, < python-ideas@python.org> wrote: > Hi all, > > the product of Sunday morning idle curiosity... > > I’ve been using the csv module a lot, and I’m wondering if there would be > value in adding a standard mechanism for opening a CSV file (correctly) > using a context manager? > > So, instead of > > with open(filename, newline=“”) as fp: >r = csv.DictReader(fp) >for row in r: > … > > support something like > > with csv.DictReader.open(filename) as r: >for row in r: > … > > ? And something similar for ‘csv.reader’? I’m not wedded to the details > here. > > The two main reasons I think this might be a positive addition are - > > * you wouldn’t have to know or remember the right way to open a CSV file > (newline=“”). > * it elides very common code. > > but perhaps there are things I’m missing here? > > As a side note, I think ‘csv.reader’ could usefully be renamed to > something else (maybe just Reader?), since it’s kind of out of sync with > the CamelCase used in ‘DictReader’. But maybe that’s just an attempt at > foolish consistency :). > > best, > —titus > > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/EKHYCTYMXZG3VI4JYFA3Y3LD3ZNMI3IX/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RA2RH37O3XHZ4XGQ36OLOZPWU5LOEXMM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Context manager for csv module 'reader' and 'DictReader'?
Hi all, the product of Sunday morning idle curiosity... I’ve been using the csv module a lot, and I’m wondering if there would be value in adding a standard mechanism for opening a CSV file (correctly) using a context manager? So, instead of with open(filename, newline=“”) as fp: r = csv.DictReader(fp) for row in r: … support something like with csv.DictReader.open(filename) as r: for row in r: … ? And something similar for ‘csv.reader’? I’m not wedded to the details here. The two main reasons I think this might be a positive addition are - * you wouldn’t have to know or remember the right way to open a CSV file (newline=“”). * it elides very common code. but perhaps there are things I’m missing here? As a side note, I think ‘csv.reader’ could usefully be renamed to something else (maybe just Reader?), since it’s kind of out of sync with the CamelCase used in ‘DictReader’. But maybe that’s just an attempt at foolish consistency :). best, —titus ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EKHYCTYMXZG3VI4JYFA3Y3LD3ZNMI3IX/ Code of Conduct: http://python.org/psf/codeofconduct/
