[issue41678] File-level, optionally external sorting

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Thanks for the suggestion, but Pablo and I agree that this isn't within scope 
for the standard library.  Marking as closed.

If you want to discuss further, please post to the Python ideas list.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41678] File-level, optionally external sorting

[Platon workaccount]

Platon workaccount  added the comment:

Why is shutil.make_archive suitable for a standard library but the file sorter 
not?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41678] File-level, optionally external sorting

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I am of the same opinion as Raymond

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41678] File-level, optionally external sorting

[Raymond Hettinger]

Raymond Hettinger  added the comment:

This doesn't seem like something that should be in the standard library.  It is 
more of an application than a building block for writing code.  It is a good 
candidate for an external package on PyPI rather than the standard library.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41678] File-level, optionally external sorting

[Dennis Sweeney]

Dennis Sweeney  added the comment:

Attached is a proof of concept.

--
Added file: https://bugs.python.org/file49436/disksort.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41678] File-level, optionally external sorting

[Dennis Sweeney]

Dennis Sweeney  added the comment:

If we were to do this, I think a better API might be to accept an arbitrary 
iterable, then produce a sorted iterable:

def sorted_on_disk(iterable, key=None, reverse=False) -> Iterable:
...

It would sort chunks of the input and store them in files as sequences of 
pickles, merging them as they got bigger, and then return an iterator over the 
resulting single sorted file.

This would be more composable with other standard Python functions and would be 
a good way of separating concerns. sorted(...) and heapq.merge(...) already 
have the correct APIs to do it this way.

Potential implementation detail: For some small fixed n, always doing a 2^n-way 
heapq.merge instead of a bunch of 2-way merges would do fewer passes over the 
data and would allow the keys to be computed 1/n as many times, assuming we 
wouldn't decorate-sort-undecorate.

--
nosy: +Dennis Sweeney

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41678] File-level, optionally external sorting

[Platon workaccount]

Platon workaccount  added the comment:

I mean Python's analog of sort [-k x.y] table.txt from GNU Coreutils.

>> What do you refer when you say "sorting a file"?

Sorting a file with multi-line plain text. Optionally, text consisting of
several columns separated by a specific character.

>> What does "key" act upon? Strings representing the lines in the file?

This is a sort rule argument similar to that of the existing in-memory
sort()/sorted() method.

>> For allow_disk_use=False, what's the difference between opening the
file, reading the lines, using sort() and writing the contents?

If False, there is no difference.

вт, 1 сент. 2020 г. в 00:18, Pablo Galindo Salgado :

>
> Pablo Galindo Salgado  added the comment:
>
> What do you refer when you say "sorting a file"?
>
> What does "key" act upon? Strings representing the lines in the file?
>
> For allow_disk_use=False, what's the difference between opening the file,
> reading the lines, using sort() and writing the contents?
>
> --
> nosy: +pablogsal
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41678] File-level, optionally external sorting

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

What do you refer when you say "sorting a file"?

What does "key" act upon? Strings representing the lines in the file?

For allow_disk_use=False, what's the difference between opening the file, 
reading the lines, using sort() and writing the contents?

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41678] File-level, optionally external sorting

[Platon workaccount]

New submission from Platon workaccount :

Feature request: a convenient sorter of whole files with the possibility of 
disk usage. Here's the syntax in my mind:

shutil.sort(src, dst, headers=0, key=None, reverse=False, allow_disk_use=False)

--
messages: 376157
nosy: platon.work
priority: normal
severity: normal
status: open
title: File-level, optionally external sorting
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com