Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-07 Thread Adam Funk
On 2015-12-04, Oscar Benjamin wrote: > Or you can use fileinput which is designed to be exactly this kind of > context manager and to be used in this way. Although fileinput is slightly > awkward in defaulting to reading stdin. That default is what I specifically like about fileinput --- it's a n

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-04 Thread Oscar Benjamin
On 4 Dec 2015 08:36, "Serhiy Storchaka" wrote: > > On 04.12.15 00:26, Oscar Benjamin wrote: >> >> On 3 Dec 2015 16:50, "Terry Reedy" wrote: >>> >>> fileinput is an ancient module that predates iterators (and generators) >> >> and context managers. Since by 2.7 open files are both context managers

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-04 Thread Serhiy Storchaka
On 04.12.15 00:26, Oscar Benjamin wrote: On 3 Dec 2015 16:50, "Terry Reedy" wrote: fileinput is an ancient module that predates iterators (and generators) and context managers. Since by 2.7 open files are both context managers and line iterators, you can easily write your own multi-file line i

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Oscar Benjamin
On 3 Dec 2015 16:50, "Terry Reedy" wrote: > > On 12/3/2015 10:18 AM, Adam Funk wrote: >> >> On 2015-12-03, Adam Funk wrote: >> >>> I'm having trouble with some input files that are almost all proper >>> UTF-8 but with a couple of troublesome characters mixed in, which I'd >>> like to ignore instea

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Laura Creighton
In a message of Thu, 03 Dec 2015 19:17:51 +, Adam Funk writes: >On 2015-12-03, Laura Creighton wrote: > >> In a message of Thu, 03 Dec 2015 15:12:15 +, Adam Funk writes: >>>I'm having trouble with some input files that are almost all proper >>>UTF-8 but with a couple of troublesome characte

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Adam Funk
On 2015-12-03, Laura Creighton wrote: > In a message of Thu, 03 Dec 2015 15:12:15 +, Adam Funk writes: >>I'm having trouble with some input files that are almost all proper >>UTF-8 but with a couple of troublesome characters mixed in, which I'd >>like to ignore instead of throwing ValueError.

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Adam Funk
On 2015-12-03, Terry Reedy wrote: > fileinput is an ancient module that predates iterators (and generators) > and context managers. Since by 2.7 open files are both context managers > and line iterators, you can easily write your own multi-file line > iteration that does exactly what you want.

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Adam Funk
On 2015-12-03, Peter Otten wrote: > def my_hook_encoded(encoding, errors=None): > import io > def openhook(filename, mode): > mode = mode.replace('U', '').replace('b', '') or 'r' > return io.open( > filename, mode, > encoding=encoding, newline='',

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Terry Reedy
On 12/3/2015 10:18 AM, Adam Funk wrote: On 2015-12-03, Adam Funk wrote: I'm having trouble with some input files that are almost all proper UTF-8 but with a couple of troublesome characters mixed in, which I'd like to ignore instead of throwing ValueError. I've found the openhook for the encod

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Laura Creighton
In a message of Thu, 03 Dec 2015 15:12:15 +, Adam Funk writes: >I'm having trouble with some input files that are almost all proper >UTF-8 but with a couple of troublesome characters mixed in, which I'd >like to ignore instead of throwing ValueError. I've found the >openhook for the encoding >

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread MRAB
On 2015-12-03 15:12, Adam Funk wrote: I'm having trouble with some input files that are almost all proper UTF-8 but with a couple of troublesome characters mixed in, which I'd like to ignore instead of throwing ValueError. I've found the openhook for the encoding for line in fileinput.input(opt

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Peter Otten
Adam Funk wrote: > On 2015-12-03, Adam Funk wrote: > >> I'm having trouble with some input files that are almost all proper >> UTF-8 but with a couple of troublesome characters mixed in, which I'd >> like to ignore instead of throwing ValueError. I've found the >> openhook for the encoding >> >>

Re: getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Adam Funk
On 2015-12-03, Adam Funk wrote: > I'm having trouble with some input files that are almost all proper > UTF-8 but with a couple of troublesome characters mixed in, which I'd > like to ignore instead of throwing ValueError. I've found the > openhook for the encoding > > for line in fileinput.input

getting fileinput to do errors='ignore' or 'replace'?

2015-12-03 Thread Adam Funk
I'm having trouble with some input files that are almost all proper UTF-8 but with a couple of troublesome characters mixed in, which I'd like to ignore instead of throwing ValueError. I've found the openhook for the encoding for line in fileinput.input(options.files, openhook=fileinput.hook_enc