Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Nathaniel Smith]

On Sun, Mar 18, 2018 at 4:58 AM, George Fischhof  wrote:
> Of course several details could be put into it, but I think it would better
> to let the developers decide the details, because they know the environment
> and the possibilities.

That's not how PEPs work :-). Someone has to do the work of collating
contradictory feedback and making opinionated design proposals, and
the person who does that is called the PEP author.

In this case, I'd also suggest framing the PEP as a list of specific
things that should be added to pathlib.Path, with justifications for
each. If your argument is "X should be in pathlib because it's in some
other module", then that's not very compelling -- by definition it
already means we have an X, so why do we need another? I think for a
number of these cases there actually is a good answer to that
question, but your PEP has to actually provide that answer :-).

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Jason Maldonis]

On Sun, Mar 18, 2018 at 4:16 PM, George Fischhof  wrote:

> Hi Jason,
>
> the status of os and shutil became this because of C functions in
> implementation (I got something similar answer before)
> ...
>
> What do you think, what would be a good way to solve this
> - add stuff from os to shutil
> - add stuff from os and shutil to pathlib
> - create a new module on top of os, shutil and pathlib (we could name it
> for example filelib
>

I'm pretty ignorant of the situation so my opinion probably isn't worth
very much, but I'll give it a try ;)

If shutil is functioning as a high-level library, it makes sense to me to
add the stuff from os to shutil. Then users can simply import shutil and
know that they have all the functionality they need to work with files.

Someone said that pathlib's Paths work with the os module but not with
shutil -- if os is the low-level module and shutil is the high-level, that
seems a bit strange but I'm sure it made sense to do that. It seems to me
that adding os functionality to shutil to make shutil "complete", and also
modifying shutil functions to work with Path objects makes the most sense.

Once that's done, shutil will be _the_ high-level library for using file
operations in python, with the option to use Path objects if you want (side
note: I'm a big fan of the Path class!).

After that, it may make sense to add some (maybe all) of the methods in
shutil to the pathlib module similar to how Path already implements some
file operations (e.g. Path.glob and Path.unlink).

All of that seems like a three step process to me:
1) Add the missing methods from os to shutil
2) Make the shutil methods work nicely with Path objects
3) Reevaluate and see what methods make sense to add to the pathlib module

As a user, I'd be thrilled with that setup.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Greg Ewing]

Jason Maldonis wrote:
In python, `os` and `shutil` are currently the low-level modules, and it 
stands to reason that we might consider combining these somehow 
(although I'm assuming that there was a good reason not to in the first 
place


The functions in os are thin wrappers around system calls,
whereas those in shutil require substantial amounts of
code (either python or C) to implement and are used
relatively rarely.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Ryan Gonzalez]

On Sun, Mar 18, 2018 at 4:16 PM, George Fischhof  wrote:

> Hi Jason,
>
> the status of os and shutil became this because of C functions in
> implementation (I got something similar answer before)
> ...
>
> What do you think, what would be a good way to solve this
> - add stuff from os to shutil
> - add stuff from os and shutil to pathlib
> - create a new module on top of os, shutil and pathlib (we could name it
> for example filelib
>
>
But...why? It's not clear to me as to how this would actually, you know,
*solve* anything.


> George
>
> 2018-03-18 19:43 GMT+01:00 Jason Maldonis :
>
>> Surely shutil is a *high* level modules.
>>> os is a low level module that shutil builds on.
>>> Adding the missing pieces to shutil would make it the place to go to do
>>> file operations.
>>> Pity its not called filelib.
>>>
>>
>> Gotcha, thank you! shutil being a high level library complicates
>> things... So we have two "high-level" libraries (pathlib and shutil) and
>> both of them provide different pieces of useful functionality. Maybe I am
>> starting to see why this is complicated. Thanks for reading my above reply
>> and taking the time to respond.
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>


-- 
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else
https://refi64.com/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[George Fischhof]

Hi Jason,

the status of os and shutil became this because of C functions in
implementation (I got something similar answer before)
...

What do you think, what would be a good way to solve this
- add stuff from os to shutil
- add stuff from os and shutil to pathlib
- create a new module on top of os, shutil and pathlib (we could name it
for example filelib

George

2018-03-18 19:43 GMT+01:00 Jason Maldonis :

> Surely shutil is a *high* level modules.
>> os is a low level module that shutil builds on.
>> Adding the missing pieces to shutil would make it the place to go to do
>> file operations.
>> Pity its not called filelib.
>>
>
> Gotcha, thank you! shutil being a high level library complicates things...
> So we have two "high-level" libraries (pathlib and shutil) and both of them
> provide different pieces of useful functionality. Maybe I am starting to
> see why this is complicated. Thanks for reading my above reply and taking
> the time to respond.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Jason Maldonis]

>
> Surely shutil is a *high* level modules.
> os is a low level module that shutil builds on.
> Adding the missing pieces to shutil would make it the place to go to do
> file operations.
> Pity its not called filelib.
>

Gotcha, thank you! shutil being a high level library complicates things...
So we have two "high-level" libraries (pathlib and shutil) and both of them
provide different pieces of useful functionality. Maybe I am starting to
see why this is complicated. Thanks for reading my above reply and taking
the time to respond.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Barry Scott]

> On 18 Mar 2018, at 17:50, Jason Maldonis  wrote:
> 
> Maybe this is obvious or I am missing something crucial, but I'm surprised 
> that this hasn't been discussed yet:
> 
> From a user perspective, imo the problem is that users currently need three 
> modules (pathlib, os, and shutil) to have a nice interface for working with, 
> copying, and removing files. In general, it seems to me that there should be 
> two modules for file-like operations: a fast, low-level module, and a 
> (slower) high-level module that provides a more natural interface than the 
> low-level module. It seems natural that the high-level module should simply 
> use the low-level module to do the file operators, and just provide a nice 
> (probably object-oriented) interface for those methods.
> 
> In python, `os` and `shutil` are currently the low-level modules, and it 
> stands to reason that we might consider combining these somehow (although I'm 
> assuming that there was a good reason not to in the first place, which is why 
> they both exist, but I haven't looked into it). And `pathlib` is currently 
> the "high-level" module.
> 
> I see two problems currently:
> 
> 1) the low-level module is split in half (for example, operations for copying 
> are contained in `shutil` and operations for removing are contained in `os`). 
> This is a bit annoying for the user, but it's not game-breaking. It does, 
> however, make python feel a bit unnatural in this context, and that's even 
> more unusual because normally python feels very natural. So this becomes sort 
> of a "huh this feels weird" situation.
> 
> 2) The `pathlib` modules only provides a high-level interface for working 
> with _single_ Path objects. There isn't really functionality to work with 
> multiple Path objects (as pointed out by someone previously if I am 
> understanding correctly).
> 
> I don't think the current PEP under consideration adequately solves either of 
> these problems. Currently, it seems like it's trying to make `pathlib` both a 
> high- and low-level module, which imo doesn't make sense.
> 
> But I do think we need, if not a single low-level module, at least a 
> high-level module that makes it unnecessary to use the low-level modules. 
> That means that `pathlib` needs more functionality added to it, which is 
> similar in spirit to the current PEP proposal

Surely shutil is a *high* level modules.
os is a low level module that shutil builds on.
Adding the missing pieces to shutil would make it the place to go to do file 
operations.
Pity its not called filelib.

Barry


> 
> - Jason, a reader
> 
> 
> 
> 
> On Sun, Mar 18, 2018 at 9:46 AM, Paul Moore  > wrote:
> On 18 March 2018 at 04:41, Nathaniel Smith  > wrote:
> > My understanding is that the point of Path is to be a convenient,
> > pleasant-to-use mechanism for accessing common filesystem operations.
> > And it does a pretty excellent job of that. But it seems obvious to me
> > that it's still missing a number of fairly basic operations that
> > people need all the time.
> 
> IMO, the pathlib module (just) defines Path. So I'm -1 on adding
> anything to pathlib that isn't a method of a Path object. Beyond that,
> I agree with you that Path should be a convenient interface for
> filesystem path objects. I haven't personally found that there's much
> missing that I've needed, but I agree that there are some gaps from a
> theoretical point of view, and adding methods to fill those gaps could
> be justifiable. OTOH, the fspath protocol was explicitly designed so
> that standalone functions (such as the ones in os and shutil) can work
> cleanly with Path objects - so there's a strong argument that "not
> everything needs to be a method" applies here. For example, while
> there isn't a Path.makedirs(), what's so bad about os.makedirs(Path)?
> (There's consistency and discoverability arguments, but they are not
> what I'd call compelling on their own).
> 
> > I don't think the PEP is there yet, and we
> > can quibble over the details -- just copying over all the historical
> > decisions in shutil isn't obviously the right move (maybe it should be
> > Path.mkdir(include_parents=True) and Path.unlink(recursive=True)
> > instead of Path.makedirs and Path.rmtree?), but there's definitely
> > room for improvement.
> 
> I agree that there are some potential candidates for "useful
> additional methods for Path objects", but I'd like to see these
> discussed on a case by case basis, much like you do here, rather than
> as a blanket "if it's in some other module and it works on paths, it
> should be in pathlib.
> 
> My biggest problem with the proposal as it stands is that it makes no
> attempt to justify the suggestions on a case by case basis (the first
> version wasn't even explicit in the functions it was proposing!) but
> argues from a pure "lump everything together" standpoint.
> 
> 

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations -- version 2

[Paul Moore]

On 18 March 2018 at 14:44, Barry Scott  wrote:
> It seems that the main quest to answer first is this.
>
> Should Path() have methods to access all file operations?

No, (Counterexample, having a Path operation to set Windows ACLs for a path).

> Maybe it was a mistake to add the ones that are already there.

Seems unlikely, as Path objects are pretty useful in their current
form. Even if it were, in some cases, backward compatibility dictates
keeping them so who cares?

> Especially in light of the fspath protocol that now exists.

Again, *maybe* the arguments for some of the existing methods were
weakened by the existence of the fspath protocol, but not enough to
override backward compatibility, so who cares?

> If yes proceed to details the methods and their semantics.

Not applicable, see above.

> If no then concentrate on making shutil etc easy to use this Path() args.

Wrong conclusion - just because it shouldn't have methods for *all*
doesn't imply that *nothing* new warrants being a Path method. Just
that cases should be considered on their individual merits.

> Maybe do this anyway.

Certainly this is a reasonable thing to propose, Probably doesn't even
need much discussion, an issue on bpo should be sufficient.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Barry Scott]

> On 18 Mar 2018, at 11:58, George Fischhof  wrote:
> 
> Hi Folks,
> 
> it seems for me that the welcoming of this proposal is rather positive than 
> not.

I think that is up for debate.

> 
> Of course several details could be put into it, but I think it would better 
> to let the developers decide the details, because they know the environment 
> and the possibilities.

You mean you have no intention of doing the implementation?
If not you who is willing to the go the, not inconsiderable, work.

> 
> The name of the functions and the method they solve the problem (for example 
> rmdir(tree=True9 instead of removedirs()) is all the same.

But it is not the same. os.removedirs only removes dirs, where as shutil.rmdir 
will remove files and dirs.

> 
> The (main) goal would be that file and directory operations reside in one 
> module.

As I said earlier the question is should that module be pathlib?
There are good arguments on both side.

Barry

> And currently the pathlib seems to be the best candidate.
> (we could put then into a very new module, but it would be just another 
> duplicataion)
> 
> 
> So what do You think, this proposal IS PEPable or should I do something with 
> this to achieve the PEPable state?
> 
> 
> BR,
> George
> 
> 2018-03-18 9:05 GMT+01:00 Nick Coghlan  >:
> On 16 March 2018 at 03:15, Chris Angelico  > wrote:
> On Fri, Mar 16, 2018 at 12:38 AM, George Fischhof  > wrote:
> >
> >
> > " if new file functions are added, they will go only in pathlib,
> >   which makes pathlib effectively mandatory;"
> > Yes but I think this part of the evolution: slowly everyone will shift to
> > pathlib,
> > and being mandatory is true for the current status as well: if you need a
> > function, you need the module.
> > Right now if you wan to execute some file operations, you need os plus
> > shutil, because the half of the
> > functions are in one of them, the other half is in the other module
> 
> The os module is cheap; pathlib has a definite cost. If every file
> operation goes through pathlib
> 
> Keep in mind that the `os` layer will never go away: `pathlib` still needs a 
> lower level API to call to *do the work* of actually interacting with the 
> underlying operating system APIs (e.g. this is why we added os.scandir).
> 
> A similar situation applies when it comes to glob, fnmatch, etc.
> 
> Even `shutil` will likely retain its place as a lower level procedural API 
> behind pathlib's object-oriented facade, since raw strings are still 
> frequently going to be easier to work with when mixing and matching Python 
> code and native operating system shell code.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncogh...@gmail.com    |   
> Brisbane, Australia
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org 
> https://mail.python.org/mailman/listinfo/python-ideas 
> 
> Code of Conduct: http://python.org/psf/codeofconduct/ 
> 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Jason Maldonis]

Maybe this is obvious or I am missing something crucial, but I'm surprised
that this hasn't been discussed yet:

>From a user perspective, imo the problem is that users currently need three
modules (pathlib, os, and shutil) to have a nice interface for working
with, copying, and removing files. In general, it seems to me that there
should be two modules for file-like operations: a fast, low-level module,
and a (slower) high-level module that provides a more natural interface
than the low-level module. It seems natural that the high-level module
should simply use the low-level module to do the file operators, and just
provide a nice (probably object-oriented) interface for those methods.

In python, `os` and `shutil` are currently the low-level modules, and it
stands to reason that we might consider combining these somehow (although
I'm assuming that there was a good reason not to in the first place, which
is why they both exist, but I haven't looked into it). And `pathlib` is
currently the "high-level" module.

I see two problems currently:

1) the low-level module is split in half (for example, operations for
copying are contained in `shutil` and operations for removing are contained
in `os`). This is a bit annoying for the user, but it's not game-breaking.
It does, however, make python feel a bit unnatural in this context, and
that's even more unusual because normally python feels very natural. So
this becomes sort of a "huh this feels weird" situation.

2) The `pathlib` modules only provides a high-level interface for working
with _single_ Path objects. There isn't really functionality to work with
multiple Path objects (as pointed out by someone previously if I am
understanding correctly).

I don't think the current PEP under consideration adequately solves either
of these problems. Currently, it seems like it's trying to make `pathlib`
both a high- and low-level module, which imo doesn't make sense.

But I do think we need, if not a single low-level module, at least a
high-level module that makes it unnecessary to use the low-level modules.
That means that `pathlib` needs more functionality added to it, which is
similar in spirit to the current PEP proposal.

- Jason, a reader




On Sun, Mar 18, 2018 at 9:46 AM, Paul Moore  wrote:

> On 18 March 2018 at 04:41, Nathaniel Smith  wrote:
> > My understanding is that the point of Path is to be a convenient,
> > pleasant-to-use mechanism for accessing common filesystem operations.
> > And it does a pretty excellent job of that. But it seems obvious to me
> > that it's still missing a number of fairly basic operations that
> > people need all the time.
>
> IMO, the pathlib module (just) defines Path. So I'm -1 on adding
> anything to pathlib that isn't a method of a Path object. Beyond that,
> I agree with you that Path should be a convenient interface for
> filesystem path objects. I haven't personally found that there's much
> missing that I've needed, but I agree that there are some gaps from a
> theoretical point of view, and adding methods to fill those gaps could
> be justifiable. OTOH, the fspath protocol was explicitly designed so
> that standalone functions (such as the ones in os and shutil) can work
> cleanly with Path objects - so there's a strong argument that "not
> everything needs to be a method" applies here. For example, while
> there isn't a Path.makedirs(), what's so bad about os.makedirs(Path)?
> (There's consistency and discoverability arguments, but they are not
> what I'd call compelling on their own).
>
> > I don't think the PEP is there yet, and we
> > can quibble over the details -- just copying over all the historical
> > decisions in shutil isn't obviously the right move (maybe it should be
> > Path.mkdir(include_parents=True) and Path.unlink(recursive=True)
> > instead of Path.makedirs and Path.rmtree?), but there's definitely
> > room for improvement.
>
> I agree that there are some potential candidates for "useful
> additional methods for Path objects", but I'd like to see these
> discussed on a case by case basis, much like you do here, rather than
> as a blanket "if it's in some other module and it works on paths, it
> should be in pathlib.
>
> My biggest problem with the proposal as it stands is that it makes no
> attempt to justify the suggestions on a case by case basis (the first
> version wasn't even explicit in the functions it was proposing!) but
> argues from a pure "lump everything together" standpoint.
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations -- version 2

[Barry Scott]

It seems that the main quest to answer first is this.

Should Path() have methods to access all file operations?

Maybe it was a mistake to add the ones that are already there.
Especially in light of the fspath protocol that now exists.

If yes proceed to details the methods and their semantics.

If no then concentrate on making shutil etc easy to use this Path() args.
Maybe do this anyway.

Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Paul Moore]

On 18 March 2018 at 04:41, Nathaniel Smith  wrote:
> My understanding is that the point of Path is to be a convenient,
> pleasant-to-use mechanism for accessing common filesystem operations.
> And it does a pretty excellent job of that. But it seems obvious to me
> that it's still missing a number of fairly basic operations that
> people need all the time.

IMO, the pathlib module (just) defines Path. So I'm -1 on adding
anything to pathlib that isn't a method of a Path object. Beyond that,
I agree with you that Path should be a convenient interface for
filesystem path objects. I haven't personally found that there's much
missing that I've needed, but I agree that there are some gaps from a
theoretical point of view, and adding methods to fill those gaps could
be justifiable. OTOH, the fspath protocol was explicitly designed so
that standalone functions (such as the ones in os and shutil) can work
cleanly with Path objects - so there's a strong argument that "not
everything needs to be a method" applies here. For example, while
there isn't a Path.makedirs(), what's so bad about os.makedirs(Path)?
(There's consistency and discoverability arguments, but they are not
what I'd call compelling on their own).

> I don't think the PEP is there yet, and we
> can quibble over the details -- just copying over all the historical
> decisions in shutil isn't obviously the right move (maybe it should be
> Path.mkdir(include_parents=True) and Path.unlink(recursive=True)
> instead of Path.makedirs and Path.rmtree?), but there's definitely
> room for improvement.

I agree that there are some potential candidates for "useful
additional methods for Path objects", but I'd like to see these
discussed on a case by case basis, much like you do here, rather than
as a blanket "if it's in some other module and it works on paths, it
should be in pathlib.

My biggest problem with the proposal as it stands is that it makes no
attempt to justify the suggestions on a case by case basis (the first
version wasn't even explicit in the functions it was proposing!) but
argues from a pure "lump everything together" standpoint.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations -- version 2

[Paul Moore]

On 17 March 2018 at 22:49, George Fischhof  wrote:
>>>
>>> * os.truncate
>>>
>>> * shutil.copyfileobj
>>>
>>>
>>> This function does not take paths as arguments. I guess it does not belong
>>> here.
>
> No path, it is true (right now), but it has sense to have this function
> among the others.

So you're proposing to include these how? As standalone functions in
pathlib? Strong -1 on that. I thought the intention here was to add
methods to path objects (which I still have reservations about). But
I'm definitely against adding standalone functions to pathlib - I see
pathlib as where Path objects are defined, not as "the place where all
path-related and other file operations should live").

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[George Fischhof]

Hi Folks,

it seems for me that the welcoming of this proposal is rather positive than
not.

Of course several details could be put into it, but I think it would better
to let the developers decide the details, because they know the environment
and the possibilities.

The name of the functions and the method they solve the problem (for
example rmdir(tree=True9 instead of removedirs()) is all the same.

The (main) goal would be that file and directory operations reside in one
module. And currently the pathlib seems to be the best candidate.
(we could put then into a very new module, but it would be just another
duplicataion)


So what do You think, this proposal IS PEPable or should I do something
with this to achieve the PEPable state?


BR,
George

2018-03-18 9:05 GMT+01:00 Nick Coghlan :

> On 16 March 2018 at 03:15, Chris Angelico  wrote:
>
>> On Fri, Mar 16, 2018 at 12:38 AM, George Fischhof 
>> wrote:
>> >
>> >
>> > " if new file functions are added, they will go only in pathlib,
>> >   which makes pathlib effectively mandatory;"
>> > Yes but I think this part of the evolution: slowly everyone will shift
>> to
>> > pathlib,
>> > and being mandatory is true for the current status as well: if you need
>> a
>> > function, you need the module.
>> > Right now if you wan to execute some file operations, you need os plus
>> > shutil, because the half of the
>> > functions are in one of them, the other half is in the other module
>>
>> The os module is cheap; pathlib has a definite cost. If every file
>> operation goes through pathlib
>
>
> Keep in mind that the `os` layer will never go away: `pathlib` still needs
> a lower level API to call to *do the work* of actually interacting with the
> underlying operating system APIs (e.g. this is why we added os.scandir).
>
> A similar situation applies when it comes to glob, fnmatch, etc.
>
> Even `shutil` will likely retain its place as a lower level procedural API
> behind pathlib's object-oriented facade, since raw strings are still
> frequently going to be easier to work with when mixing and matching Python
> code and native operating system shell code.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Serhiy Storchaka]

17.03.18 19:15, Stephen J. Turnbull пише:

As far as I can recall, pathlib is intended from the beginning to
(1) represent paths in hierarchical local filesystems as Paths,
(2) manipulate individual Paths in various ways consistent with the
 semantics of a hierarchal filesystem, and
(3) offer various ways to access the object denoted by a single Path.
Its functionality is very complete as far as that goes.

It does not contain methods to
(4) operate on directories as collections (with the exception of the
 iterdir, glob, and rglob methods, which expose directory contents as
 iterators of Paths),
(5) perform operations on several objects denoted by Paths at once
 (copy and its multiple operand variants),
(6) perform process control or access process characteristics,
(7) perform operations (eg, mounting partitions and flow control on
 TTYs) on devices (block or character), even if they can be
 accessed via paths in some filesystem as in POSIX, or
(8) deal with users and other specialized OS objects.

I conclude there never was any intention to overlap with os or shutil,
except to the extent that they provide for any kind of path
manipulation.  Rather, I suppose the intent was to provide a
substitute for os.path with a more convenient, complete,
object-oriented API and consistent semantics, based on more than a
decade of experience with os.path.


I concur with Stephen. Let's not move a half of the stdlib that have any 
relations with files to the pathlib module.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[George Fischhof]

2018-03-18 9:05 GMT+01:00 Nick Coghlan :

> On 16 March 2018 at 03:15, Chris Angelico  wrote:
>
>> On Fri, Mar 16, 2018 at 12:38 AM, George Fischhof 
>> wrote:
>> >
>> >
>> > " if new file functions are added, they will go only in pathlib,
>> >   which makes pathlib effectively mandatory;"
>> > Yes but I think this part of the evolution: slowly everyone will shift
>> to
>> > pathlib,
>> > and being mandatory is true for the current status as well: if you need
>> a
>> > function, you need the module.
>> > Right now if you wan to execute some file operations, you need os plus
>> > shutil, because the half of the
>> > functions are in one of them, the other half is in the other module
>>
>> The os module is cheap; pathlib has a definite cost. If every file
>> operation goes through pathlib
>
>
> Keep in mind that the `os` layer will never go away: `pathlib` still needs
> a lower level API to call to *do the work* of actually interacting with the
> underlying operating system APIs (e.g. this is why we added os.scandir).
>
> A similar situation applies when it comes to glob, fnmatch, etc.
>
> Even `shutil` will likely retain its place as a lower level procedural API
> behind pathlib's object-oriented facade, since raw strings are still
> frequently going to be easier to work with when mixing and matching Python
> code and native operating system shell code.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>

:-)
+ 1

George
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[George Fischhof]

2018-03-18 5:41 GMT+01:00 Nathaniel Smith :

> On Sat, Mar 17, 2018 at 10:15 AM, Stephen J. Turnbull
>  wrote:
> > (5) perform operations on several objects denoted by Paths at once
> > (copy and its multiple operand variants),
>
> Sure it does: Path.rename and Path.replace. I know why rename and copy
> have historically been in separate modules, but the distinction is
> pretty arcane and matters a lot more to implementers than it does to
> users.
>
> Similarly, it's hard to explain why we have Path.mkdir but not
> Path.makedirs -- and these have historically both lived in the 'os'
> module, so we can't blame it on Path being a mirror of os.path. It's
> also not obvious why we should have Path.rmdir, but not Path.rmtree.
>
> My understanding is that the point of Path is to be a convenient,
> pleasant-to-use mechanism for accessing common filesystem operations.
> And it does a pretty excellent job of that. But it seems obvious to me
> that it's still missing a number of fairly basic operations that
> people need all the time. I don't think the PEP is there yet, and we
> can quibble over the details -- just copying over all the historical
> decisions in shutil isn't obviously the right move (maybe it should be
> Path.mkdir(include_parents=True) and Path.unlink(recursive=True)
> instead of Path.makedirs and Path.rmtree?), but there's definitely
> room for improvement.
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>


:-)
+ 1

George
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations -- version 2

[Barry]

> On 17 Mar 2018, at 22:49, George Fischhof  wrote:
> 
> 
> 
> 2018. márc. 17. 21:34 ezt írta ("Barry" ):
> 
> 
>> On 17 Mar 2018, at 10:42, George Fischhof  wrote:
>> 
>> Hi folks,
>> 
>> I added the list of functions to the proposal, here is the new version.
>> 
>> George
>> 
>> 
>> 
>> 
>> PEP: 
>> Title: Pathlib Module Should Contain All File Operations
>> Author: George Fischhof 
>> Status: Draft
>> Type: Standards Track
>> Content-Type: text/x-rst
>> Created: 15-Mar-2018
>> Python-Version: 3.8
>> Post-History: 12-Mar-2018, 17-Mar-2018
>> 
>> 
>> Abstract
>> 
>> 
>> This PEP proposes pathlib module to be a centralized place for all
>> file-system related operations.
>> 
>> 
>> Rationale
>> =
>> 
>> Right now we have several modules that contain functions related
>> to file-system operations mainly the os, pathlib and shutil.
>> For beginners it is quite hard to remember where can he / she find
>> a function (copy resides in shutil, but the remove function can be
>> found in the os module.  (And sometimes developers with moderate
>> experience have to check the documentation as well.)
>> 
>> After the release of version 3.6 several methods became aware of
>> path-like object.  There are only a few ones which does not support
>> the path-like object.  After making these methods path-like object
>> aware, these functions could be added to pathlib.
>> 
>> With functions in pathlib the developers should not have to think
>> on which method (function) can be found in which module.
>> 
>> Makes the life easier.
>> 
>> 
>> Implementation
>> ==
>> 
>> For compatibility reasons the pathlib should contain wrappers to
>> the original functions.  The original functions should remain
>> at their original place.  (Or if pathlib contains the function, the
>> original modules should have a wrapper to it.)
>> 
>> 
>> List of mentioned functions
>> ===
>> 
>> * os.link => path.hardlink_to
>>   (should be named similarly to path.softlink_to)
>> 
>> * os.mkfifo
>> 
>> * os.readlink
>> 
>> * os.remove
>> 
>> * os.removedirs
>>   (shutil.rmtree has the same functionalaty)
> 
> I think you will find these to are a lot different.
> 
> 
> From high level, it is only question of implementation: the main goal is to 
> remove a chain of directories

This is a PEP you and proposing details matter and this is one of the details 
tha you need be clear on.

> 
>> 
>> * os.truncate
>> 
>> * shutil.copyfileobj
> 
> This function does not take paths as arguments. I guess it does not belong 
> here.
> 
> 
> No path, it is true (right now), but it has sense to have this function among 
> the others.

What do you mena right now?

> 
> 
>> 
>> * shutil.copyfile
>> 
>> * shutil.copymode
>> 
>> * shutil.copystat
>> 
>> * shutil.copy
>> 
>> * shutil.copy2
>> 
>> * shutil.copytree with shutil.ignore_patterns
>> 
>> * shutil.move
>> 
>> * shutil.disk_usage
>> 
>> * shutil.chown
> 
> Is yout plan to also expose all the constants need to use chmod etc from 
> pathlib?
> 
> Of course, because this PEP is about conveniece and easyness.

Can you say so then and not leave it in doubt.

> 
>> 
>> All functions from os module accept path-like objects,
>> and none of the shutil functions.
> 
> Should shutils be improved to accept path-like first?
> 
> Maybe. The PEP should show the goal. This is just a task among others to 
> achieve the goal. But it can be done simultanuosly.
> 
> George
> 
> Barry
> 
>> 
>> 
>> Copyright
>> =
>> 
>> This document has been placed in the public domain.
>> 
>> 
>> 
>> ..
>>Local Variables:
>>mode: indented-text
>>indent-tabs-mode: nil
>>sentence-end-double-space: t
>>fill-column: 70
>>coding: utf-8
>>End:
>> 
>> 
>> 
>> 
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP proposal -- Pathlib Module Should Contain All File Operations

[Nick Coghlan]

On 16 March 2018 at 03:15, Chris Angelico  wrote:

> On Fri, Mar 16, 2018 at 12:38 AM, George Fischhof 
> wrote:
> >
> >
> > " if new file functions are added, they will go only in pathlib,
> >   which makes pathlib effectively mandatory;"
> > Yes but I think this part of the evolution: slowly everyone will shift to
> > pathlib,
> > and being mandatory is true for the current status as well: if you need a
> > function, you need the module.
> > Right now if you wan to execute some file operations, you need os plus
> > shutil, because the half of the
> > functions are in one of them, the other half is in the other module
>
> The os module is cheap; pathlib has a definite cost. If every file
> operation goes through pathlib


Keep in mind that the `os` layer will never go away: `pathlib` still needs
a lower level API to call to *do the work* of actually interacting with the
underlying operating system APIs (e.g. this is why we added os.scandir).

A similar situation applies when it comes to glob, fnmatch, etc.

Even `shutil` will likely retain its place as a lower level procedural API
behind pathlib's object-oriented facade, since raw strings are still
frequently going to be easier to work with when mixing and matching Python
code and native operating system shell code.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/