[issue494066] Access to readline history elements

[admin]

Change by admin :


--
github: None -> 35765

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue474831] Command history doesn't work on Mandrake

[admin]

Change by admin :


--
github: None -> 35402

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400738] add history file functions for Modules/readline.c

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400742] readline history read/write

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400739] add history file functions for Modules/readline.c

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400739] add history file functions for Modules/readline.c

[admin]

Change by admin :


--
github: None -> 32521

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400742] readline history read/write

[admin]

Change by admin :


--
github: None -> 32524

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400738] add history file functions for Modules/readline.c

[admin]

Change by admin :


--
github: None -> 32520

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Julien Palard]

Julien Palard  added the comment:

> dev docs direct to `/license.html` which redirects to `/3/license.html`

All docs are redirecting to `/license.html`, this allow it to work also out of 
docs.python.org.

> 3.9 docs have the same; wouldn’t it be better to have `/3.9/license.html`?

It's maybe a slight better yes (the page should be the same so I don't know if 
it make real difference), but doing so means a bit of complexity, which I don't 
think is worth it.

I tried to keep it simple for this fix (the previous state was "links to the 
current page" which was worse).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Éric Araujo]

Éric Araujo  added the comment:

dev docs direct to `/license.html` which redirects to `/3/license.html`

3.9 docs have the same; wouldn’t it be better to have `/3.9/license.html`?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Julien Palard]

Julien Palard  added the comment:

I checked on docs.python.org and it's fixed.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[miss-islington]

Change by miss-islington :


--
pull_requests: +28745
pull_request: https://github.com/python/cpython/pull/30541

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +28744
pull_request: https://github.com/python/cpython/pull/30540

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Julien Palard]

Change by Julien Palard :


--
keywords: +patch
pull_requests: +28728
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30527

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Julien Palard]

Julien Palard  added the comment:

This should be fixed in python-docs-theme==2022.1.

I'll close the issue when I actually see the fix applied on docs.python.org.

--
nosy: +mdk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Alex Waygood]

Change by Alex Waygood :


--
nosy:  -AlexWaygood

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Alex Waygood]

Alex Waygood  added the comment:

@Éric: I personally found it difficult to immediately understand what the issue 
was about when reading only the title on the BPO homepage, and thought the 
change in title would help clarify, having seen other documentation issues 
marked similarly on the tracker.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Éric Araujo]

Éric Araujo  added the comment:

Tracked to https://github.com/python/python-docs-theme/issues/89

AlexWaygood: can I ask why add pseudo-tags to the title field when we do have 
structured data (components set to doc) on this tracker?  Is it a new practice?

--
nosy: +AlexWaygood

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] [doc] "history and license" link has wrong target

[Alex Waygood]

Change by Alex Waygood :


--
title: "history and license" link has wrong target -> [doc] "history and 
license" link has wrong target

___
Python tracker 
<https://bugs.python.org/issue45729>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] "history and license" link has wrong target

[Éric Araujo]

Éric Araujo  added the comment:

It happens on all pages for all versions, because the link is empty.
3.8 doesn’t have that line so doesn’t show the bug.  I am trying to find where 
that is defined.  Strangely, I don’t find results looking for `please donate` 
which is the line below…

--
nosy: +eric.araujo
type:  -> behavior
versions: +Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45729] "history and license" link has wrong target

[Jacob Lifshay]

New submission from Jacob Lifshay :

https://docs.python.org/3.10/library/csv.html
at the bottom the "history and license" link just links back to csv.html, 
rather than the correct target.

--
assignee: docs@python
components: Documentation
messages: 405807
nosy: docs@python, programmerjake
priority: normal
severity: normal
status: open
title: "history and license" link has wrong target
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue45729>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45632] Strange readline save history behaviour in site.py

[doraeric]

New submission from doraeric :

I noticed that in site.py, it saves history to a hard-coded location if the 
current history length is 0. The history is considered as not loaded if the 
length is 0, but it may be actually loaded from a empty file. In this case, the 
history is save to hard-coded ~/.python_history, and maybe another user-defined 
location in PYTHONSTARTUP file.

I think a better solution is to export some config or functions from site.py, 
so users can explicitly disable the atexit callback from site.py.

---
[site.py] 
https://github.com/python/cpython/blob/b9cdd0fb9c463c2503a4d854bb6529a9db58fe1b/Lib/site.py#L470-L491

--
components: Library (Lib)
messages: 405114
nosy: doraeric
priority: normal
severity: normal
status: open
title: Strange readline save history behaviour in site.py
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue45632>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42626] readline history, vi-editingmode and ANSI color codes bug

[Joakim Nilsson]

Joakim Nilsson  added the comment:

Yes, I have noticed that too, and I agree, not as bad as if it was garbled,
but still an annoyance.

On Fri, Jul 30, 2021 at 1:34 AM Andrei Kulakov 
wrote:

>
> Andrei Kulakov  added the comment:
>
> Joakim: by the way, what I was able to reproduce is just a visual bug.
> IOW, the text is still the same and all there in the buffer, but it shows
> up only after a combination of rightward movement and 'a', and a copy of it
> shows up on the left side. So it's not as serious as if the text was
> garbled or impossible to enter or change.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42626] readline history, vi-editingmode and ANSI color codes bug

[Andrei Kulakov]

Andrei Kulakov  added the comment:

Joakim: by the way, what I was able to reproduce is just a visual bug. IOW, the 
text is still the same and all there in the buffer, but it shows up only after 
a combination of rightward movement and 'a', and a copy of it shows up on the 
left side. So it's not as serious as if the text was garbled or impossible to 
enter or change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42626] readline history, vi-editingmode and ANSI color codes bug

[Andrei Kulakov]

Andrei Kulakov  added the comment:

Also this bug happens on both iterm2 and terminal on MacOS, but note that on 
'terminal', it's impossible to ctrl-c or ctrl-z out of the test script, so if 
you test it, you will need to close the window or the tab, and lose shell 
history and background jobs, so beware.

--

___
Python tracker 
<https://bugs.python.org/issue42626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42626] readline history, vi-editingmode and ANSI color codes bug

[Andrei Kulakov]

Andrei Kulakov  added the comment:

I was able to reproduce something very similar, and I believe essentially the 
same issue perhaps varying due to config. I think I've actually seen similar 
glitches with python command line readline handling of k.

So, entering 1234 always works fine. The issue starts with 5 chars:

1234
12345
12345   12345

As you can see, the line is duplicated and going up to it shifts cursor to the 
right, at first the duplication on the right side is blank but if I move the 
cursor to the right, the '1234' are revealed. '5' is still hidden. Then I have 
to press 'a' to see the '5' revealed and go into insert mode.

There's no way to go to the left of the duplicated string.

I hope someone more experienced in readline or more motivated can look more 
into this.

I duplicated it in 3.11 built from source, on MacOS Big Sur (on M1 cpu, but 
that shouldn't matter).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42626] readline history, vi-editingmode and ANSI color codes bug

[Joakim Nilsson]

Joakim Nilsson  added the comment:

Sorry, my mistake. If you remove the last line "input()" from readline.py, it 
should be reproducible. Tested on Python 3.9.2.

Link to video demonstration of the bug: 
http://nijoakim.com/readline-example.mp4.

--
Added file: https://bugs.python.org/file50190/readline-example.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42626] readline history, vi-editingmode and ANSI color codes bug

[Andrei Kulakov]

Andrei Kulakov  added the comment:

I can't reproduce on MacOS in both Py 3.9.1 and in dev version. Works fine, I 
can erase everything after '2' and before it.

--
nosy: +andrei.avk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42626] readline history, vi-editingmode and ANSI color codes bug

[Joakim Nilsson]

New submission from Joakim Nilsson :

Tested on Debian Bullseye with Python 3.9.

If 'set editing-mode vi' is used in .inputrc and the attached program is run, a 
bug occurs when navigating the readline history. It seems only to occur when 
ANSI color escape characters are input to the 'input()' function.

To reproduce the bug:
  Run 'readline-example.py'
  Enter '0123456789' in the prompt without quotes.
  Press enter.
  Press escape and then 'k' to go back in history in vi normal mode.
  The cursor is now placed between '2' and '3' and it is impossible to erase 
anything after the '2'. (To enter vi insert mode, press i and start editing the 
text normally.)

This bug does not occur for shorter strings. If for example '012345678' is 
input, the program behaves normally. If the escape characters are not used in 
the 'input()' function, program behaves normally.

--
components: Library (Lib)
files: readline-example.py
messages: 382917
nosy: nijoakim
priority: normal
severity: normal
status: open
title: readline history, vi-editingmode and ANSI color codes bug
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49673/readline-example.py

___
Python tracker 
<https://bugs.python.org/issue42626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41075] IDLE: Better support history navigation

[E. Paine]

E. Paine  added the comment:

> but not with binding to modifier-up/down.

I cannot reproduce. I have tested on both Windows and Linux and found I could 
successfully change to (and use) the modifier-up/down bindings both through the 
Keys page of the configdialog and by editing the keys def. There was no 
traceback on either platform. @wyz23x2, can you reproduce this problem?

My personal preference for the new bindings, when tested, was alt-up/down as it 
is most similar to the existing bindings. Also, control-up/down is used in 
other apps for moving the cursor multiple lines at a time and shift-up/down (at 
least in my mind) seems like a weird/random choice. Terry, I don't know what 
you have in mind, but I think these bindings should be in addition to the 
existing ones rather than instead of (I also tested this and found it to work 
correctly).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41075] IDLE: Better support history navigation

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy: +epaine

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41075] IDLE: Better support history navigation

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Yes, history works fine with the current bindings to alt-p and alt-n, or with 
binding to up or down, but not with binding to modifier-up/down.

--

___
Python tracker 
<https://bugs.python.org/issue41075>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41075] IDLE: Better support history navigation

[Terry J. Reedy]

Terry J. Reedy  added the comment:

History and up/down is apart of #2704.  It has patches that include the up/down 
change, but I don't expect to use any of them as they are.

I tried Shift/Control/Alt - Up/Down and none worked.  Rebinding just Up/Down 
did (in Shell only, leaving up/down unchanged in Editor).  But not having 
up/down work to move between lines in the Shell multiline statement entry area 
is not acceptable to me.  I don't know if modifiers can never work with up/down 
or if there is something that would make it work.

--
nosy:  -epaine
stage: resolved -> 
status: closed -> open
title: Make support of navigating through prev. commands in IDLE more 
conspicuous -> IDLE: Better support history navigation
versions:  -Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue41075>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40441] Plural typo in Design and History FAQ

[Ammar Askar]

Change by Ammar Askar :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40441] Plural typo in Design and History FAQ

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

After merging the associated PR, I believe this is resolved. Thanks Alex for 
reporting and solving this

--
nosy: +nanjekyejoannah
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40441] Plural typo in Design and History FAQ

[alexpovel]

New submission from alexpovel :

The documentation under "Design and History FAQ" has a typo in its "Why doesn’t 
Python have a “with” statement for attribute assignments?" section:

https://docs.python.org/3/faq/design.html#why-doesn-t-python-have-a-with-statement-for-attribute-assignments

where it says:

> Some language have a construct that looks like this:

which should of course read:

> Some languages have a construct that looks like this:

Attached is a git diff patch file.

--
assignee: docs@python
components: Documentation
files: fix_plural_typo_in_documentation.patch
keywords: patch
messages: 367696
nosy: alexpovel, docs@python
priority: normal
pull_requests: 19120
severity: normal
status: open
title: Plural typo in Design and History FAQ
versions: Python 3.8
Added file: 
https://bugs.python.org/file49100/fix_plural_typo_in_documentation.patch

___
Python tracker 
<https://bugs.python.org/issue40441>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[DL Neil via Python-list]

On 2/02/20 1:00 AM, R.Wieser wrote:

As sent to the OP. I appreciate these discussions, in the expectation of
learning something-new. (and with rust-removal paints at the ready!)


Indeed.   Even if its just a different POV which makes you rethink the
reasons of your own one.



+1
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

DL,

>> While I agree with you there, I've been searching for other ways to 
>> detect a
>> keypress (in a console-based script) and have found none.   
>
> Color me disappointed!

I was disapponted too, but realized that being able to just capture any 
keypress (how does the 'puter know the script has focus, as there is nothing 
to give it to) is an easy way to userland keylogging ...

> but wouldn't that only work if there was an input() loop?

Just being able to check for such a keypress at the end of a "do the next 
download" (in the OPs case) loop would be enough for me.

> Understood. However, your understanding of RDBMS interrupt 'security' 
> appears either aged, or biased to a particular implementation (which (in 
> my imagination) wouldn't match the high-standards I expect you hold).

:-) You might well be correct there.   Mea culpa.  As for the high standards 
?While I am quite sure that currently any self-respecting DB system will 
be crammed full with methods to forgo corruption, it always sticks in my 
mind that its possible.In other words, its possible I'm simply a bit too 
cautious.

Than again, switching an unresponsive 'puter off by pressing the powerbutton 
for a couple of seconds is a wide/well-known practice ... :-(

> Good stuff! Yet at the same time, that very lack of clarity is good reason 
> to allow that 'another' solution might be 'right' - even if it is not 
> 'better'.
> (for any definitions of "right" or "better", or ...)

"The very lack of clarity" ?

But yes, keeping the possibility open that some other method (than the ones 
presented) could be even more apropriate is part of why I leave the choice 
with the asker.   Its all too easy to (unwittingly) automatically think in a 
certain direction, and than overlook other solutions ("for someone with a 
hammer" and all that).

> As sent to the OP. I appreciate these discussions, in the expectation of 
> learning something-new. (and with rust-removal paints at the ready!)

Indeed.   Even if its just a different POV which makes you rethink the 
reasons of your own one.

Regards,
Rudy Wieser


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Peter J. Holzer]

On 2020-01-30 07:56:30 +1100, Chris Angelico wrote:
> On Thu, Jan 30, 2020 at 7:49 AM MRAB  wrote:
> > On 2020-01-29 20:00, jkn wrote:
> > > I could have a file with all the URLs listed and work through each line 
> > > in turn.
> > > But then I would have to rewrite the file (say, with the 
> > > previously-successful
> > > lines commented out) as I go.
> > >
> > Why comment out the lines yourself when the download manager could do it
> > for you?
> >
> > Load the list from disk.
> >
> > For each uncommented line:
> >
> >  Download the file.
> >
> >  Comment out the line.
> >
> >  Write the list back to disk.
> 
> Isn't that exactly what the OP was talking about? It involves
> rewriting the file at every step, with the consequent risks of
> trampling on other changes, corruption on error, etc, etc, etc.

If you do it right, the risk is small:

1) read file,
2) do the download
3) write temporary file with changes
4) rename temporary file to file.

Remaining risks:

Someone might start the download manager twice. This can be
prevented with a lock file.

The computer might crash between 3 and 4 (or shortly after 4). In
this case you might lose the contents of file. This can be prevented
with proper use of fsync, but at this point it gets complicated and
filesystem dependent (there were a number of papers and talks about
this topic over the last few years), so personally I would live with
the risk or use a database.

Subrisk: Your disk might lie to your computer about having
stored the data. In this case there is nothing you can do except
buying a better disk.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Sat, Feb 1, 2020 at 4:17 PM DL Neil via Python-list
 wrote:
>
> On 31/01/20 9:53 PM, R.Wieser wrote:
> >> Using ctrl+c is a VERY BAD idea.
> >
> > To have it just exit the program ?  Yes, indeed.
> >
> > Though you /could/ keep track of what needs to be finished and have the
> > ctrl-c handler do that for you (barf).
> >
> > Another posibility is to capture the ctrl-c and set a flag, which than
> > instructs the program to terminate loops wherever possible - and thus have
> > the program finish as if there was no more to do.
> >
> >> (see also 'sledgehammer to crack a nut')
> >
> > While I agree with you there, I've been searching for other ways to detect a
> > keypress (in a console-based script) and have found none.   IOW, you do not
> > (seem to) have another option.
>
> Color me disappointed! I was hoping to be enlightened as to how one
> might code exactly that. (so I went looking...)
>
> My first thought was a try...except looking for a keyboard error, but
> wouldn't that only work if there was an input() loop? The idea of
> putting, effectively the entire code-base, into a try...except block
> seemed crude and likely to cause side-effects.

No, it should be fine - you'll get KeyboardInterrupt even without
input(). At least, you will on Unix systems; can someone confirm that
this is also the case on Windows?

Having a try/except around your main function that catches one
specific exception is usually fine. I often do this in the main of
something that's designed to run a server - for development, I'll
invoke the script directly and then Ctrl-C to stop it, for deployment,
it'll be run through some service manager.

> Sigh, if I had a dollar for every time someone looked at a simple
> solution (MS-Excel I'm looking at you!) and uttered 'suggestions' such
> as "we could run the entire company off one of these spreadsheet things"
> or "if we captured all the company's transactions, couldn't we run it
> off one Python pgm?"...

... ugh, I feel your pain... yep, heard that sort of thing way too many times.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[DL Neil via Python-list]

On 31/01/20 9:53 PM, R.Wieser wrote:

Using ctrl+c is a VERY BAD idea.


To have it just exit the program ?  Yes, indeed.

Though you /could/ keep track of what needs to be finished and have the
ctrl-c handler do that for you (barf).

Another posibility is to capture the ctrl-c and set a flag, which than
instructs the program to terminate loops wherever possible - and thus have
the program finish as if there was no more to do.


(see also 'sledgehammer to crack a nut')


While I agree with you there, I've been searching for other ways to detect a
keypress (in a console-based script) and have found none.   IOW, you do not
(seem to) have another option.


Color me disappointed! I was hoping to be enlightened as to how one 
might code exactly that. (so I went looking...)


My first thought was a try...except looking for a keyboard error, but 
wouldn't that only work if there was an input() loop? The idea of 
putting, effectively the entire code-base, into a try...except block 
seemed crude and likely to cause side-effects.


https://stackoverflow.com/questions/1112343/how-do-i-capture-sigint-in-python 
put me on a good path. So, I opened a door in my memory, shut my ears to 
the intense screeching noise, and used a yard-broom to sweep away all 
the rust that flaked off the hinges...


PSL's signal — Set handlers for asynchronous events 
(https://docs.python.org/3/library/signal.html) appears to handle 
keyboard interrupts, *at least in MS-Windows*.


Rusty? I think, years ago, I used signal(s) to establish time-limits on 
systems that had a possibility of 'run-away' or over-refining a solution 
which would only ever be asymptotic. I've not used the library in this 
manner.


That said, I'm using one of Python's asynchronous features, and Internet 
activity is riddled with delay-periods which make such retrievals ripe 
for async (code) solutions...




Why do you need to abandon the process mid-way?


Take your pick.Mostly because of "I need to leave *now*".   Or perhaps
because of an "you have X seconds before the device shuts down because of
the battery being low" situation.


I (personally) cannot make use of the above. However, even if my 
portable's battery threatens complete disaster, the system "hibernates", 
and thus averts major disaster, eg file system (or RDBMS) corruption. 
This seems enough for 99% of 'average uses'. Of course it is regarded as 
trivially-unacceptable in the non-stop world, eg banking. Would our OP 
classify this project as anything other than the first of those 
categories? Thus:



What is the OP's definition of "unlikely" or "acceptable risk"?


Good question, but one I had no wish for to try to ascertain.   I just gave
some "worst case" secenario based replies (aka, the "did you think of 
situation" kind).


Understood. However, your understanding of RDBMS interrupt 'security' 
appears either aged, or biased to a particular implementation (which (in 
my imagination) wouldn't match the high-standards I expect you hold).


Have already admitted that I (personally) may 'over use' RDBMS because 
it is 'easy' for me. When we do feel the justification to examine the 
details of "worst case scenarios"/"did you think of this?" RDBMS 
stand-out as a 'solution' simply because of the incredible amounts of 
time and depths of thought, highly-specialised minds (far superior to 
any input I could offer) have invested in providing highly refined 
services/servers. (which I dare-say informed @Chris' comments)


Neither MSFT nor the relevant GNU/Linux projects assure common file 
systems to such a degree (and particularly not in the mentioned 
circumstances). That's why there are FS 'rescue' procedures (Linux runs 
fsck automatically when 'trouble' is noted during system-start and every 
so-many reboots, regardless).


I'm sufficiently intrigued to wonder if anyone will dispute/correct such 
- but not sufficiently motivated to research any position other than the 
one currently held - so call me "Curious", but not "George" then...




There are so many reasons why such won't work first-time, when they should
every time; that it may be quite difficult to detect 'corruption'


:-) That was not even considered.  Just the "what do I still need to
download" datafile.


Yes, we left the OP's (actual) spec, way-back somewhere!



Accordingly, there is no non-atomic transaction in the proposal


The problem there is that you are second-guessing to what the OP will be
writing, and that it will be good.   I didn't and I don't assume that.  I've
seen too much "it works, so its good" noobie code. :-)


Absolutely! One of the 'joys' of working text-only is that even when one 
knows (much) more about the background to a posted-question, it is still 
so easy to 'answer-wrong'.


Even in my courses, where the Discussion Lists (are supposedly) related 
to each week/chapter of a course, you'd think we'd have adequate 'hints' 
as to the level of the trainee and where his/her question is 
'coming-from'. Not 

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[DL Neil via Python-list]

On 31/01/20 9:30 AM, jkn wrote:

Err, well, thanks for that discussion gents...

As it happens I do know how to use a database, but I regard it as overkill for
what I am trying to do here. I think a combination of hashing the URL,
and using a suffix to indicate the result of previous downloaded attempts, will
work adequately for my needs.

 Thanks again
 Jon N



Don't let the, um, robust discussion concern you. Certainly you bear no 
responsibility.


As elsewhere, your understanding of 'the problem' is more complete than 
any of us can claim. Accordingly, your 'answer' is the correct one! 
Should you 'fail fast, and fail often', then you have other courses that 
may be worth considering...



Speaking personally, I find a lot to be learned from reading what 
different folk have to say, and taking benefit from their experiences 
and different ways of looking at the same thing. It's worth remembering 
that not everyone comes across in email as a mild-mannered, 
absent-minded, over-stereotyped, British boffin.



There is/was a London PUG Discussion List, just as there likely are for 
other British groups/localities. I don't recall a recent posting - or 
maybe they've ex-communicated me for requiring them to shout very (very) 
loudly to reach me...



Thanks for the challenge!
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

jkn,

> I'm happy to consider the risk and choose (eg.) the hash function 
> accordingly, thanks.

No problem, just wanted you to be aware and (thus) able to choose.

Regards,
Rudy Wieser


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[jkn]

On Friday, January 31, 2020 at 9:41:32 AM UTC, R.Wieser wrote:
> jkn,
> 
> > I think a combination of hashing the URL,
> 
> I hope you're not thinking of saving the hash (into the "done" list) instead 
> if the URL itself.   While hash collisions do not happen often (especially 
> not in a small list), you cannot rule them out.  And when that happens that 
> would mean you would be skipping downloads ...
> 
> Regards,
> Rudy Wieser

I'm happy to consider the risk and choose (eg.) the hash function accordingly, 
thanks.

Jon N

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[jkn]

On Friday, January 31, 2020 at 9:41:32 AM UTC, R.Wieser wrote:
> jkn,
> 
> > I think a combination of hashing the URL,
> 
> I hope you're not thinking of saving the hash (into the "done" list) instead 
> if the URL itself.   While hash collisions do not happen often (especially 
> not in a small list), you cannot rule them out.  And when that happens that 
> would mean you would be skipping downloads ...
> 
> Regards,
> Rudy Wieser

Hi Rudy

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

jkn,

> I think a combination of hashing the URL,

I hope you're not thinking of saving the hash (into the "done" list) instead 
if the URL itself.   While hash collisions do not happen often (especially 
not in a small list), you cannot rule them out.  And when that happens that 
would mean you would be skipping downloads ...

Regards,
Rudy Wieser


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Fri, Jan 31, 2020 at 7:56 PM R.Wieser  wrote:
>
> Dennis,
>
> > A full client/server RDBM should never be affected by an abort
> > of a client program.
>
> What you describe is on the single query level.   What I was thinking of was
> having several queries that /should/ work as a single unit, but could get
> interrupted (because of the OPs ctrl-c).
>
> Yes, Chris was probably right about using a transaction to fix that, but
> that has to be learned - knowledge I do not take for granted with a
> hobbyist.
>

Do you also permit people to drive cars without knowing about the
handbrake? When you start using a database, you have to learn how to
speak its language (usually SQL). Transactions are part of that. What
is so hard about this? They're not some esoteric thing that only a few
people use. EVERY request to a PostgreSQL database (and many other
DBMSes) is part of a transaction, whether implicit or explicit.

I Googled "postgres python tutorial". The first hit was
https://www.postgresqltutorial.com/postgresql-python/ and it mentions
transactions in the very first section, on connecting to the database.

Second hit: https://pynative.com/python-postgresql-tutorial/ - the
word "transaction" comes up on the landing page, albeit part way down;
but the first example that does any mutation does use an explicit
commit.

I didn't check for SQLAlchemy but it's likely the same.

Use transactions. Learn them.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

DL,

>> Nothing that can't be countered by keeping copies of the last X number of
>> to-be-dowloaded-URLs files.
>
> That's a good idea, but how would the automated system 'know' to give-up
> on the current file and utilise generation n-1? Unable to open the file or
> ???

Well, that would be one reason for it.   But there was also talk about
integrity checking, both on the file and URL-entry level.   Failing any of
that could trigger it.

But to be honest, I would not even automate it.   Such a situation should
not occur often, if at all.  So having it is cause for concern, and should
be investigated.   Instead I would give the user an error message, and offer
him the list of older files to choose one from to continue with (or have him
specify the file on the commandline).

> Using ctrl+c is a VERY BAD idea.

To have it just exit the program ?  Yes, indeed.

Though you /could/ keep track of what needs to be finished and have the
ctrl-c handler do that for you (barf).

Another posibility is to capture the ctrl-c and set a flag, which than
instructs the program to terminate loops wherever possible - and thus have
the program finish as if there was no more to do.

> (see also 'sledgehammer to crack a nut')

While I agree with you there, I've been searching for other ways to detect a
keypress (in a console-based script) and have found none.   IOW, you do not
(seem to) have another option.

> Why do you need to abandon the process mid-way?

Take your pick.Mostly because of "I need to leave *now*".   Or perhaps
because of an "you have X seconds before the device shuts down because of
the battery being low" situation.

> What is the OP's definition of "unlikely" or "acceptable risk"?

Good question, but one I had no wish for to try to ascertain.   I just gave
some "worst case" secenario based replies (aka, the "did you think of 
situation" kind).

> There are so many reasons why such won't work first-time, when they should
> every time; that it may be quite difficult to detect 'corruption'

:-) That was not even considered.  Just the "what do I still need to
download" datafile.

> Accordingly, there is no non-atomic transaction in the proposal

The problem there is that you are second-guessing to what the OP will be
writing, and that it will be good.   I didn't and I don't assume that.  I've
seen too much "it works, so its good" noobie code. :-)

> Do I want to deal with the complexities of managing files and corruptions,
> in that arena?
...
> Do you?

:-)   Its was-and-is not my choice to make.   I gave the OP some stuff to
think about, and left making the choice upto him.All three of the
presented options are viable.

> Be aware that formation rules for URLs are not congruent with OS FS rules!

Yup.   Though I seldom see that happen.  Though I guess I should have
mentioned that ... :-|

Regards,
Rudy Wieser



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

Dennis,

> A full client/server RDBM should never be affected by an abort
> of a client program.

What you describe is on the single query level.   What I was thinking of was 
having several queries that /should/ work as a single unit, but could get 
interrupted (because of the OPs ctrl-c).

Yes, Chris was probably right about using a transaction to fix that, but 
that has to be learned - knowledge I do not take for granted with a 
hobbyist.

> So... (And I know many despise it, but it IS easy to set up) that
> leaves running ...some ... server process.

I think that would make the matter even worse.  It could be silently doing 
some housekeeping work, and a hard shutdown of the 'puter (for whatever 
reason) could easily trash the whole database.

And yes, I'm playing the advocate of the devil there.   :-)

Regards,
Rudy Wieser


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[DL Neil via Python-list]

On 30/01/20 9:35 PM, R.Wieser wrote:

MRAB's scheme does have the disadvantages to me that Chris has pointed
out.

Nothing that can't be countered by keeping copies of the last X number of
to-be-dowloaded-URLs files.


That's a good idea, but how would the automated system 'know' to give-up 
on the current file and utilise generation n-1? Unable to open the file 
or ???




As for rewriting every time, you will /have/ to write something for every
action (and flush the file!), if you think you should be able to ctrl-c (or
worse) out of the program.


Which is the nub of the problem!

Using ctrl+c is a VERY BAD idea. Depending upon the sophistication of 
the solution/existing code, surely there is another way...


Even closing/pulling-out the networking connection to cause an exception 
within Python, would enable management of a more 'clean' and 'data safe' 
shutdown!

(see also 'sledgehammer to crack a nut')

Why do you need to abandon the process mid-way?



But, you could opt to write this sessions successfully downloaded URLs to a
seperate file, and only merge that with the origional one program start.
That together with an integrity check of the seperate file (eventually on a
line-by-line (URL) basis) should make the origional files corruption rather
unlikely.


What is the OP's definition of "unlikely" or "acceptable risk"?
If RDBMS == "unnecessary complexity", then (presumably) 'concern' will 
be commensurately low, and much of the discussion to-date, moot?


I've not worked on 'downloads' (which I take to mean data files, eg 
forms from the tax office - guess what task I'm procrastinating over?) 
but have automated the downloading of web page content/headers. There 
are so many reasons why such won't work first-time, when they should 
every time; that it may be quite difficult to detect 'corruption' (as 
distinct from so many of these other issues that may arise)...




A database /sounds/ good, but what happens when you ctrl-c outof a
non-atomic operation ?   How do you fix that ?IOW: Databases can be
corrupted for pretty-much the same reason as for a simple datafile (but with
much worse consequences).


[apologies for personal comment]
I, (with my skill-set, tool-set, collection of utilities, ... - see 
earlier mention of "bias") reach for an RDBMS more quickly than many*. 
Mea culpa or 'more power to [my] right arm'?



The DB suggestion (posted earlier) involved only a single table, to 
which fields would be added/populated during processing as a record of 
progress/status. Thus, replacing the single file that the OP 
(originally) outlined as fitting his/her needs, with a single DB-table.


Accordingly, there is no non-atomic transaction in the proposal - UPDATE 
is atomic in most (competent) RDBMS.
(again, in my ignorance of that project, please don't (anyone) think I'm 
including/excluding SQLite)



Contrarily, if the 'single table idea' is hardly a "database" by most 
definitions, why bother? The answer lies in the very mechanisms to 
combat corruptions and interruptions being discussed! As a 
fundamentally-lazy person, I'd rather leave the RDBMS-coders to wrestle 
with such complexities 'for me'. Then, I can 'stand on the shoulders' of 
such 'giants', by driving their (competently working) 'black box'...

(YMMV!)


Now, it transpires, the OP possesses DB skills. So, (s)he is in a 
position to make the go/no decision which suits the actual spec. Yahoo! 
(not TM)




Also think of the old adagio: "I had a problem, and than I thought I could
use X.  Now I have two problems..." - with X traditionally being "regular
expressions".   In other words: do KISS (keep it )


Good point! (I'm not a great fan of RegEx-es either)
- reduce/avoid complexity, "simple is better than complex"! (Python: 
import this)



Surely though, it is only appropriate to dive into the concerns and 
complexities of DB accuracy and "consistency", if we do likewise with 
file systems?


The rationale of my 'laziness' argument 'for' using an RDBMS, also 
applies to plain-vanilla file systems. Do I want to deal with the 
complexities of managing files and corruptions, in that arena?

(you could easily guess the answer to that!)

Do you?
(the answer may be quite different - but no matter, I'm not going to say 
you are "wrong", as long as in making such a decision (files?DB) we 
compare 'like with like' - in fact, before that: as long as the client's 
spec says that we need to be worrying about such detail!

(otherwise YAGNI applies!)



By the way: The "just write the URLs in a folder" method is not at all a bad
one.   /Very/ easy to maintain, resilent (especially when you consider the
self-repairing capabilities of some filesystems) and the polar opposite of a
"customer lock-in". :-)


+1
Be aware that formation rules for URLs are not congruent with OS FS rules!
(such concerns don't apply if the URLs are data within a file/table)



* was astonished to discover (a show-of-hands poll at some conference or 
other) that 

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[jkn]

Err, well, thanks for that discussion gents...

As it happens I do know how to use a database, but I regard it as overkill for
what I am trying to do here. I think a combination of hashing the URL,
and using a suffix to indicate the result of previous downloaded attempts, will
work adequately for my needs.

Thanks again
Jon N

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Fri, Jan 31, 2020 at 4:11 AM R.Wieser  wrote:
> But, do you remember what the OP said ?
> [quote]
> want to download these as a 'background task'. ... you can CTRL-C out,
> [/quote]
>
> Why now do I think that, when such a backgroud process is forgotten and the
> 'puter switched off, the file, database or otherwise, could easily get
> damaged ?  And that a ctrl-c at the wrong moment could cause the same if the
> cleanup (of the database, which could easily still be, in its own thread,
> busy housekeeping) isn't done correctly.
>
> Also, have you /never/ encountered a corrupted file, database or otherwise ?
> You must not have done much with 'puters at all ...
>

On the contrary, I grew up with IBM DB2 and then PostgreSQL, both of
which are actually resilient against power failures. I've also
recovered data from crashed hard drives. So, yeah, I think I know what
I'm talking about. Sorry.

Have a nice day. Enjoy dealing with corruption in places where
ordinary best-practices and basic tutorials can prevent it from ever
happening.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

Chris,

> Yes, and then you backpedalled furiously when I showed that
> proper transactions prevent this.

You're a fool, out for a fight.

/You/ might know exactly how to handle a database to make sure its 
/transactions/ will not leave the database in a corrupt state, but as I 
mentioned a few posts back:

[quote]
 I think that a database is /definitily/ overcomplicating stuff, especially 
when looking at the OP who's supposed to write and maintain it.
[/quote]

Mind the part after the comma.

Also, you happily ignored the part in my second post where I mentioned that 
corruption can be of a different kind - one that you have absolutily /no/ 
control over:

[quote]
An unreadable file is often described as being corrupt
[/quote]

> I lost track of what you were actually trying to say.

Well, all that I wanted to say - to the OP - is in my first reply.  You 
could always try to re-read it.   The other posts where just responses to 
your "that can't be!" challenges to me.

> So either justify this position, showing that a database can
> become corrupted the same way a flat file can,

:-)   In the /same/ way ?Thats rather multi-interpretable, don't you 
think ?   Are you trying to set me up ?

My point was that the /effect/ of a same corruption is different for both: 
A flat file is /much/ easier to rescue.

But, do you remember what the OP said ?
[quote]
want to download these as a 'background task'. ... you can CTRL-C out,
[/quote]

Why now do I think that, when such a backgroud process is forgotten and the 
'puter switched off, the file, database or otherwise, could easily get 
damaged ?  And that a ctrl-c at the wrong moment could cause the same if the 
cleanup (of the database, which could easily still be, in its own thread, 
busy housekeeping) isn't done correctly.

Also, have you /never/ encountered a corrupted file, database or otherwise ? 
You must not have done much with 'puters at all ...

Though I think I'm going to end our conversation here, as you're way more 
agressive than the subject calls for.

Goodbye chris.   Have a good life.

Regards,
Rudy Wieser


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Fri, Jan 31, 2020 at 2:01 AM R.Wieser  wrote:
>
> Chris,
>
> >> I think that a database is /definitily/ overcomplicating stuff,
> >
> > Okay, sure... but you didn't say that.
>
> I'm sorry ?   In my first reply I described a file-based approach and
> mentioned that the folder approach is a rather good one.  What do you think
> I ment there ?
>
> > You said that a database could become corrupted.
>
> Yes, I did.  Here:
>
> [Quote]
> IOW: Databases can be corrupted for pretty-much the same reason as for a
> simple datafile (but with much worse consequences).
> [/quote]
>
> Please do notice the part between the brackets.

Yes, and then you backpedalled furiously when I showed that proper
transactions prevent this. After which... I lost track of what you
were actually trying to say.

So either justify this position, showing that a database can become
corrupted the same way a flat file can, in normal use; or explain to
me what you're actually arguing here.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

Chris,

>> I think that a database is /definitily/ overcomplicating stuff,
>
> Okay, sure... but you didn't say that.

I'm sorry ?   In my first reply I described a file-based approach and 
mentioned that the folder approach is a rather good one.  What do you think 
I ment there ?

> You said that a database could become corrupted.

Yes, I did.  Here:

[Quote]
IOW: Databases can be corrupted for pretty-much the same reason as for a 
simple datafile (but with much worse consequences).
[/quote]

Please do notice the part between the brackets.

> So what IS your line of argument?

To be frank, the fact that you think you should still ask baffles me.  I 
hope that the above clears it up though.

Regards,
Rudy Wieser


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Dan Sommers]

On Thu, 30 Jan 2020 23:34:59 +1100
Chris Angelico  wrote:

> ... I wasn't advocating for the use of a database; my first and
> strongest recommendation was, and still is, a stateless system wherein
> the files themselves are the entire indication of which documents have
> been downloaded.

Yes, I like stateless systems, too, but that system isn't stateless.  As
I understand the problem of a "crudely persistem download manager,"
there's a collection of to-be-downloaded URLs (which may be empty) and
some data that's been downloaded (which may also be empty).  You can
certainly encode a lot of that directly in the file system, but it's
still state.

Using a database instead solves a lot of the tricky bits that the bare
file system doesn't (which is what ChrisA said in what I snipped).  It's
just Greenspun's Tenth Rule with the words C, Fortran, and Common Lisp
crossed out and Python and ACID Database written in crayon.

Dan

-- 
“Atoms are not things.” – Werner Heisenberg
Dan Sommers, http://www.tombstonezero.net/dan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Thu, Jan 30, 2020 at 11:11 PM R.Wieser  wrote:
>
> Chris,
>
> > That's what transactions are for.
>
> Again,
>
> >> I guess that that went right over your head. :-)/You/ might know
> >> exactly
> >> what should and shouldn't be done, what makes you think the OP currently
> >> does ?
>
> > I don't understand why you're denigrating databases,
>
> Am I denigrating a nut-and-bolt when glue would be an easier solution ?
>
> The problem with most people (programmers included) that they often
> over-think (and thus over-complicate) stuff.I think that a database is
> /definitily/ overcomplicating stuff, especially when looking at the OP who's
> supposed to write and maintain it.  Y personal MMV.
>

Okay, sure... but you didn't say that. You said that a database could
become corrupted.

If you read back in this thread, you'll see that the first
recommendations were NOT about databases, and then databases were
offered as an alternative. You then said that they could become
corrupted just as files can. This is outright false, and is also not
helpful to the discussion. So I responded. But I wasn't advocating for
the use of a database; my first and strongest recommendation was, and
still is, a stateless system wherein the files themselves are the
entire indication of which documents have been downloaded.

So what IS your line of argument?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

Chris,

> That's what transactions are for.

Again,

>> I guess that that went right over your head. :-)/You/ might know 
>> exactly
>> what should and shouldn't be done, what makes you think the OP currently
>> does ?

> I don't understand why you're denigrating databases,

Am I denigrating a nut-and-bolt when glue would be an easier solution ?

The problem with most people (programmers included) that they often 
over-think (and thus over-complicate) stuff.I think that a database is 
/definitily/ overcomplicating stuff, especially when looking at the OP who's 
supposed to write and maintain it.  Y personal MMV.

Regards,
Rudy Wieser


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Thu, Jan 30, 2020 at 8:36 PM R.Wieser  wrote:
>
> Chris,
>
> > Uhh
> >
> > Proper databases don't HAVE non-atomic operations. That's kinda their job.
>
> Uhh...  yes, /singular/ operations are considered to be atomic.  A series of
> operations /ment/ to be executed as a single one on the other hand aren't.

That's what transactions are for. If you have a series of operations
meant to be executed as an atomic operation, you begin a transaction,
do the operations, and then commit. Again, that is the *job* of the
database.

> >> Also think of the old adagio: "I had a problem,
>
> I guess that that went right over your head. :-)/You/ might know exactly
> what should and shouldn't be done, what makes you think the OP currently
> does ?
>
> > But that's still not corrupting the database.
>
> Depending on your definition of corruption.  An unreadable file is often
> described as being corrupt, though the same can be said of a database in an
> inconsistent state.
>

If the OP doesn't know how to use a database, that doesn't change my
recommendations regarding the use of a database. I don't understand
why you're denigrating databases, when basically the only way to mess
up transactional integrity is to fail to use them properly, which is
something easily learned. The downside of a database is that it might
be overkill, but it's safe against corruption.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

Chris,

> Uhh
>
> Proper databases don't HAVE non-atomic operations. That's kinda their job.

Uhh...  yes, /singular/ operations are considered to be atomic.  A series of 
operations /ment/ to be executed as a single one on the other hand aren't.

> Unless you mean that there's a non-atomic operation that consists of
[snip]

>> Also think of the old adagio: "I had a problem,

I guess that that went right over your head. :-)/You/ might know exactly 
what should and shouldn't be done, what makes you think the OP currently 
does ?

> But that's still not corrupting the database.

Depending on your definition of corruption.  An unreadable file is often 
described as being corrupt, though the same can be said of a database in an 
inconsistent state.

Regards,
Rudy Wieser

>> Also think of the old adagio:
>
> BTW, the word you want here is "adage", unless you mean that
> it's a piece of music being played slowly :)

Thanks.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Thu, Jan 30, 2020 at 7:41 PM R.Wieser  wrote:
> Also think of the old adagio:

BTW, the word you want here is "adage", unless you mean that it's a
piece of music being played slowly :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Thu, Jan 30, 2020 at 7:41 PM R.Wieser  wrote:
> A database /sounds/ good, but what happens when you ctrl-c outof a
> non-atomic operation ?   How do you fix that ?IOW: Databases can be
> corrupted for pretty-much the same reason as for a simple datafile (but with
> much worse consequences).

Uhh

Proper databases don't HAVE non-atomic operations. That's kinda their job.

Unless you mean that there's a non-atomic operation that consists of
placing some kind of "claim" on a URL and then going and downloading
it, and finally reporting completion, but that's easily enough handled
by simply ensuring that there are no downloader processes running, and
then clear any incomplete claims. But that's still not corrupting the
database.

Maybe you've only ever worked with half-baked apologies for database systems?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[R.Wieser]

jkn,

> MRAB's scheme does have the disadvantages to me that Chris has pointed 
> out.

Nothing that can't be countered by keeping copies of the last X number of 
to-be-dowloaded-URLs files.

As for rewriting every time, you will /have/ to write something for every 
action (and flush the file!), if you think you should be able to ctrl-c (or 
worse) out of the program.

But, you could opt to write this sessions successfully downloaded URLs to a 
seperate file, and only merge that with the origional one program start. 
That together with an integrity check of the seperate file (eventually on a 
line-by-line (URL) basis) should make the origional files corruption rather 
unlikely.


A database /sounds/ good, but what happens when you ctrl-c outof a 
non-atomic operation ?   How do you fix that ?IOW: Databases can be 
corrupted for pretty-much the same reason as for a simple datafile (but with 
much worse consequences).

Also think of the old adagio: "I had a problem, and than I thought I could 
use X.  Now I have two problems..." - with X traditionally being "regular 
expressions".   In other words: do KISS (keep it )


By the way: The "just write the URLs in a folder" method is not at all a bad 
one.   /Very/ easy to maintain, resilent (especially when you consider the 
self-repairing capabilities of some filesystems) and the polar opposite of a 
"customer lock-in". :-)

Regards,
Rudy Wieser


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Thu, Jan 30, 2020 at 8:59 AM DL Neil via Python-list
 wrote:
> * NB I don't use SQLite (in favor of going 'full-fat') and thus cannot
> vouch for its behavior under load/queuing mechanism/concurrent
> accesses... but I'm biased and probably think/write SQL more readily
> than Python - oops!

I don't use SQLite either, and I always have a PostgreSQL database
around that I can use. That said, though, I believe SQLite is fine in
terms of reliability; the reason it's a bad choice for concurrency is
that it uses large-scale locks to ensure safety, which means that
multiple writers will block against each other. But that's fine for
this use-case.

So my recommendations would be:
1) Something stateless, or where the state is intrinsic to the downloaded files
2) Or failing that, use a database rather than a flat file for your state.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[DL Neil via Python-list]

On 30/01/20 10:38 AM, jkn wrote:

On Wednesday, January 29, 2020 at 8:27:03 PM UTC, Chris Angelico wrote:

On Thu, Jan 30, 2020 at 7:06 AM jkn  wrote:

I want to be a able to use a simple 'download manager' which I was going to 
write
(in Python), but then wondered if there was something suitable already out 
there.
I haven't found it, but thought people here might have some ideas for existing 
work, or approaches.

The situation is this - I have a long list of file URLs and want to download 
these
as a 'background task'. I want this to process to be 'crudely persistent' - you
can CTRL-C out, and next time you run things it will pick up where it left off.


A decent project. I've done this before but in restricted ways.


The download part is not difficult. Is is the persistence bit I am thinking 
about.
It is not easy to tell the name of the downloaded file from the URL.

I could have a file with all the URLs listed and work through each line in turn.
But then I would have to rewrite the file (say, with the previously-successful
lines commented out) as I go.

...


 Thanks for the idea. I should perhaps have said more clearly that it is not
easy (though perhaps not impossible) to infer the name of the downloaded data
from the URL - it is not a 'simple' file URL, more of a tag.

However I guess your scheme would work if I just hashed the URL and created
a marker file - "a39321604c.downloaded" once downloaded. The downloaded content
would be separately (and somewhat opaquely) named, but that doesn't matter.

MRAB's scheme does have the disadvantages to me that Chris has pointed out.


Accordingly, +1 to @Dan's suggestion of a database*:

- it can be structured to act as a queue, for URLs yet to be downloaded
- when downloading starts, the pertinent row can be updated to include 
the fileNM in use (a separate field from the URL)
- when the download is complete, further update the row with a suitable 
'flag'
- as long as each write/update is commit-ed, the system will be 
interrupt-able (^c).


Upon resumption, query the DB looking for entries without 
completion-flags, and re-start/resume the download process.


If a downloaded file is (later) found to be corrupt, either add the 
details to the queue again, or remove the 'flag' from the original entry.


This method could also be extended/complicated to work if you (are smart 
about) implement multiple retrieval threads...



* NB I don't use SQLite (in favor of going 'full-fat') and thus cannot 
vouch for its behavior under load/queuing mechanism/concurrent 
accesses... but I'm biased and probably think/write SQL more readily 
than Python - oops!

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[jkn]

On Wednesday, January 29, 2020 at 8:27:03 PM UTC, Chris Angelico wrote:
> On Thu, Jan 30, 2020 at 7:06 AM jkn  wrote:
> >
> > Hi all
> > I'm almost embarrassed to ask this as it's "so simple", but thought I'd 
> > give
> > it a go...
> 
> Hey, nothing wrong with that!
> 
> > I want to be a able to use a simple 'download manager' which I was going to 
> > write
> > (in Python), but then wondered if there was something suitable already out 
> > there.
> > I haven't found it, but thought people here might have some ideas for 
> > existing work, or approaches.
> >
> > The situation is this - I have a long list of file URLs and want to 
> > download these
> > as a 'background task'. I want this to process to be 'crudely persistent' - 
> > you
> > can CTRL-C out, and next time you run things it will pick up where it left 
> > off.
> 
> A decent project. I've done this before but in restricted ways.
> 
> > The download part is not difficult. Is is the persistence bit I am thinking 
> > about.
> > It is not easy to tell the name of the downloaded file from the URL.
> >
> > I could have a file with all the URLs listed and work through each line in 
> > turn.
> > But then I would have to rewrite the file (say, with the 
> > previously-successful
> > lines commented out) as I go.
> >
> 
> Hmm. The easiest way would be to have something from the URL in the
> file name. For instance, you could hash the URL and put the first few
> digits of the hash in the file name, so
> http://some.domain.example/some/path/filename.html might get saved
> into "a39321604c - filename.html". That way, if you want to know if
> it's been downloaded already, you just hash the URL and see if any
> file begins with those digits.
> 
> Would that kind of idea work?
> 
> ChrisA

Hi Chris
Thanks for the idea. I should perhaps have said more clearly that it is not
easy (though perhaps not impossible) to infer the name of the downloaded data
from the URL - it is not a 'simple' file URL, more of a tag.

However I guess your scheme would work if I just hashed the URL and created
a marker file - "a39321604c.downloaded" once downloaded. The downloaded content
would be separately (and somewhat opaquely) named, but that doesn't matter.

MRAB's scheme does have the disadvantages to me that Chris has pointed out.

Jon N
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Dan Sommers]

On Thu, 30 Jan 2020 07:26:36 +1100
Chris Angelico  wrote:

> On Thu, Jan 30, 2020 at 7:06 AM jkn  wrote:

> > The situation is this - I have a long list of file URLs and want to
> > download these as a 'background task'. I want this to process to be
> > 'crudely persistent' - you can CTRL-C out, and next time you run
> > things it will pick up where it left off.

> A decent project. I've done this before but in restricted ways.

> > The download part is not difficult. Is is the persistence bit I am
> > thinking about.  It is not easy to tell the name of the downloaded
> > file from the URL.

Where do the names of the downloaded files come from now, and why can't
that same algorithm be used later to determine the existence of the
file?  How much control do you have over this algorithm (which leads to
what ChrisA suggested)?

> > I could have a file with all the URLs listed and work through each
> > line in turn.  But then I would have to rewrite the file (say, with
> > the previously-successful lines commented out) as I go.

Files have that problem.  Other solutions, e.g., a sqlite3 database,
don't.  Also, a database might give you a place to store other
information about the URL, such as the name of the associated file.

> Hmm. The easiest way would be to have something from the URL in the
> file name. For instance, you could hash the URL and put the first few
> digits of the hash in the file name, so
> http://some.domain.example/some/path/filename.html might get saved
> into "a39321604c - filename.html". That way, if you want to know if
> it's been downloaded already, you just hash the URL and see if any
> file begins with those digits.

> Would that kind of idea work?

Dan

-- 
“Atoms are not things.” – Werner Heisenberg
Dan Sommers, http://www.tombstonezero.net/dan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Thu, Jan 30, 2020 at 7:49 AM MRAB  wrote:
>
> On 2020-01-29 20:00, jkn wrote:
> > I could have a file with all the URLs listed and work through each line in 
> > turn.
> > But then I would have to rewrite the file (say, with the 
> > previously-successful
> > lines commented out) as I go.
> >
> Why comment out the lines yourself when the download manager could do it
> for you?
>
> Load the list from disk.
>
> For each uncommented line:
>
>  Download the file.
>
>  Comment out the line.
>
>  Write the list back to disk.
>

Isn't that exactly what the OP was talking about? It involves
rewriting the file at every step, with the consequent risks of
trampling on other changes, corruption on error, etc, etc, etc.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[MRAB]

On 2020-01-29 20:00, jkn wrote:

Hi all
 I'm almost embarrassed to ask this as it's "so simple", but thought I'd 
give
it a go...

I want to be a able to use a simple 'download manager' which I was going to 
write
(in Python), but then wondered if there was something suitable already out 
there.
I haven't found it, but thought people here might have some ideas for existing 
work, or approaches.

The situation is this - I have a long list of file URLs and want to download 
these
as a 'background task'. I want this to process to be 'crudely persistent' - you
can CTRL-C out, and next time you run things it will pick up where it left off.

The download part is not difficult. Is is the persistence bit I am thinking 
about.
It is not easy to tell the name of the downloaded file from the URL.

I could have a file with all the URLs listed and work through each line in turn.
But then I would have to rewrite the file (say, with the previously-successful
lines commented out) as I go.

Why comment out the lines yourself when the download manager could do it 
for you?


Load the list from disk.

For each uncommented line:

Download the file.

Comment out the line.

Write the list back to disk.


I also thought of having the actual URLs as filenames (of zero length) in a
'source' directory. The process would then look at each filename in turn, and
download the appropriate URL. Then the 'filename file' would either be moved to
a 'done' directory, or perhaps renamed to something that the process wouldn't
subsequently pick up.

But I would have thought that some utility to do this kind of this exists 
already. Any pointers? Or any comments on the above suggested methods?


--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestions on mechanism or existing code - maintain persistence of file download history

[Chris Angelico]

On Thu, Jan 30, 2020 at 7:06 AM jkn  wrote:
>
> Hi all
> I'm almost embarrassed to ask this as it's "so simple", but thought I'd 
> give
> it a go...

Hey, nothing wrong with that!

> I want to be a able to use a simple 'download manager' which I was going to 
> write
> (in Python), but then wondered if there was something suitable already out 
> there.
> I haven't found it, but thought people here might have some ideas for 
> existing work, or approaches.
>
> The situation is this - I have a long list of file URLs and want to download 
> these
> as a 'background task'. I want this to process to be 'crudely persistent' - 
> you
> can CTRL-C out, and next time you run things it will pick up where it left 
> off.

A decent project. I've done this before but in restricted ways.

> The download part is not difficult. Is is the persistence bit I am thinking 
> about.
> It is not easy to tell the name of the downloaded file from the URL.
>
> I could have a file with all the URLs listed and work through each line in 
> turn.
> But then I would have to rewrite the file (say, with the previously-successful
> lines commented out) as I go.
>

Hmm. The easiest way would be to have something from the URL in the
file name. For instance, you could hash the URL and put the first few
digits of the hash in the file name, so
http://some.domain.example/some/path/filename.html might get saved
into "a39321604c - filename.html". That way, if you want to know if
it's been downloaded already, you just hash the URL and see if any
file begins with those digits.

Would that kind of idea work?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Suggestions on mechanism or existing code - maintain persistence of file download history

[jkn]

Hi all
I'm almost embarrassed to ask this as it's "so simple", but thought I'd give
it a go...

I want to be a able to use a simple 'download manager' which I was going to 
write
(in Python), but then wondered if there was something suitable already out 
there.
I haven't found it, but thought people here might have some ideas for existing 
work, or approaches.

The situation is this - I have a long list of file URLs and want to download 
these
as a 'background task'. I want this to process to be 'crudely persistent' - you
can CTRL-C out, and next time you run things it will pick up where it left off.

The download part is not difficult. Is is the persistence bit I am thinking 
about.
It is not easy to tell the name of the downloaded file from the URL.

I could have a file with all the URLs listed and work through each line in turn.
But then I would have to rewrite the file (say, with the previously-successful
lines commented out) as I go.

I also thought of having the actual URLs as filenames (of zero length) in a
'source' directory. The process would then look at each filename in turn, and
download the appropriate URL. Then the 'filename file' would either be moved to
a 'done' directory, or perhaps renamed to something that the process wouldn't
subsequently pick up.

But I would have thought that some utility to do this kind of this exists 
already. Any pointers? Or any comments on the above suggested methods?

Thanks
J^n

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue38791] readline history file is hard-coded

[Jonathan Conder]

Jonathan Conder  added the comment:

I agree. Did a cursory search before posting but missed it somehow

--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38791] readline history file is hard-coded

[Zackery Spytz]

Zackery Spytz  added the comment:

This issue seems like a duplicate of bpo-29779 (which has a pull request).

--
nosy: +ZackerySpytz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38791] readline history file is hard-coded

[Jonathan Conder]

Change by Jonathan Conder :


--
keywords: +patch
pull_requests: +16658
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17149

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38791] readline history file is hard-coded

[pmp-p]

Change by pmp-p :


--
nosy: +twouters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38791] readline history file is hard-coded

[pmp-p]

Change by pmp-p :


--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38791] readline history file is hard-coded

[Jonathan Conder]

New submission from Jonathan Conder :

Other tools such as bash and less allow their history file to be customised 
with an environment variable. Will add a patch for this in a bit.

This could also be customised using PYTHONSTARTUP, but then the user has to 
duplicate a bunch of code which is already part of the site module.

--
components: Library (Lib)
messages: 356573
nosy: jconder
priority: normal
severity: normal
status: open
title: readline history file is hard-coded
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue38791>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35217] REPL history is broken when python is invoked with cmd /c

[Eryk Sun]

Eryk Sun  added the comment:

The Windows console has a fixed number of history buffers. In Windows 7 the 
default maximum is four history buffers. In this case, if we run a script via 
cmd.exe -> py.exe -> python.exe, then only one history buffer remains for a 
child process. As you've observed, no history buffer is available for the fifth 
process if we run cmd.exe -> python.exe. 

You can increase the default maximum in the console's Alt+Space+D defaults 
dialog, under "Command History" -> "Number of Buffers". This corresponds to the 
"NumberOfHistoryBuffers" value in the registry key "HKCU\Console". 

You can increase the maximum for the current window in the console's 
Alt+Space+P properties dialog. If the current console was allocated by an 
application that was launched from a shortcut (i.e. a .LNK file), this property 
gets persisted in the shortcut itself. Otherwise it gets persisted in a 
registry key under "HKCU\Console" that's named for the startup window title. If 
no title was specified, the default title is the path to the executable, with 
backslashes replaced by underscore and the Windows directory as %SystemRoot%. 
For example, if you run cmd.exe from the Win+R run dialog, its console 
properties are stored in the registry key 
"HKCU\Console\%SystemRoot%_system32_cmd.exe".

--
nosy: +eryksun
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue35217>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35280] Interactive shell overwrites history

[dingens]

New submission from dingens :

When opening two shells at the same time, history from the one that is closed 
first gets lost.

Steps to reproduce:
- let's call the contents of the history before this experiment `z`
- open two interactive shells called A and B
- execute commands in them, say `a` and `b`, respectively
- close shell A
- close shell B

Actual behavior:
- history contains `z` then `b`

Expected behavior:
- history contains `z` then `a` then `b`

possibly related: issue22940

--
messages: 330103
nosy: dingens
priority: normal
severity: normal
status: open
title: Interactive shell overwrites history
type: behavior
versions: Python 3.6

___
Python tracker 
<https://bugs.python.org/issue35280>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35217] REPL history is broken when python is invoked with cmd /c

[零欸特]

New submission from 零欸特 :

Windows 7 x64
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit 
(AMD64)]

Steps to reproduce:

1. Create a script:
```
from subprocess import run
run(["cmd.exe", "/c", "python"])
run(["python"])
run("python", shell=True)
```
2. Run the script.

Actual result:

The script will invoke Python REPL 3 times. The first and the third REPL don't 
save the command history. Pressing up/down arrows would clear the entire line. 
Pressing F7 has no effect.

The second REPL works fine.

Expected result:

Command history should work in all instances.

--
components: Windows
messages: 329729
nosy: paul.moore, steve.dower, tim.golden, zach.ware, 零欸特
priority: normal
severity: normal
status: open
title: REPL history is broken when python is invoked with cmd /c
type: behavior
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue35217>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[miss-islington]

miss-islington  added the comment:


New changeset a845b7ab3e8ba1c20ef4c3ee23ebf50df7e7c4c6 by Miss Islington (bot) 
in branch '3.6':
bpo-33887: Add TOC to Design and History FAQ(GH-7766)
https://github.com/python/cpython/commit/a845b7ab3e8ba1c20ef4c3ee23ebf50df7e7c4c6


--

___
Python tracker 
<https://bugs.python.org/issue33887>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[miss-islington]

miss-islington  added the comment:


New changeset 070c91e46579429f7af7599af6d9e67a8dc5be50 by Miss Islington (bot) 
in branch '3.7':
bpo-33887: Add TOC to Design and History FAQ(GH-7766)
https://github.com/python/cpython/commit/070c91e46579429f7af7599af6d9e67a8dc5be50


--

___
Python tracker 
<https://bugs.python.org/issue33887>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[miss-islington]

miss-islington  added the comment:


New changeset 3e3157bd55f197ab36b280b26aea8dcd04e37fcf by Miss Islington (bot) 
in branch '2.7':
bpo-33887: Add TOC to Design and History FAQ(GH-7766)
https://github.com/python/cpython/commit/3e3157bd55f197ab36b280b26aea8dcd04e37fcf


--
nosy: +miss-islington

___
Python tracker 
<https://bugs.python.org/issue33887>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[Mariatta Wijaya]

Mariatta Wijaya  added the comment:

Thanks for the PR!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[miss-islington]

Change by miss-islington :


--
pull_requests: +7488

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[miss-islington]

Change by miss-islington :


--
pull_requests: +7487

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[miss-islington]

Change by miss-islington :


--
pull_requests: +7486

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[Mariatta Wijaya]

Mariatta Wijaya  added the comment:


New changeset 38cf49bf695903ac7a8516bca6bbb6b32d935bb5 by Mariatta (Andrés 
Delfino) in branch 'master':
bpo-33887: Add TOC to Design and History FAQ(GH-7766)
https://github.com/python/cpython/commit/38cf49bf695903ac7a8516bca6bbb6b32d935bb5


--
nosy: +Mariatta

___
Python tracker 
<https://bugs.python.org/issue33887>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[Andrés Delfino]

Andrés Delfino  added the comment:

Sorry, Raymond, you are right.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Please don't fill-up the tracker with dozens of micro-PRs for tiny 
documentation changes of dubious benefit or occasional detriment.  On the 
balance, it isn't helpful to have a newcomer second-guess every little writing 
decision that has been made.  Ideally, we should focus on issues that are known 
to actually be confusing real users.  Also, it would be ideal to spend time 
learning the existing documentation norms.  

The little micro-edits each take time to discuss, review and apply or reject.  
This eats into the small bits of spare time that the reviewers have and takes 
them away from other issues.

Perhaps you could channel your enthusiasm into reviewing other doc patches or 
open bugs.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[Andrés Delfino]

Change by Andrés Delfino :


--
keywords: +patch
pull_requests: +7374
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33887] doc Add TOC in Design and History FAQ

[Andrés Delfino]

New submission from Andrés Delfino :

IHMO, having all questions put together is really useful.

--
assignee: docs@python
components: Documentation
messages: 319824
nosy: adelfino, docs@python
priority: normal
severity: normal
status: open
title: doc Add TOC in Design and History FAQ
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue33887>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [OT] multicore/cpu history

[Grant Edwards]

On 2018-03-25, Steven D'Aprano  wrote:

> Not really. With multiple CPUs, you have the option of running two 
> distinct OSes in isolation, not merely virtual machines but actual 
> distinct machines in the same box.

Not on any of the multi-CPU motherboards I ever worked with.  The CPUs
shared SDRAM and used the same physical address space.  They both saw
the same PCI/ISA buses and all other peripherals. There was no way you
could run two different OSes without some sort of hypervisor -- there
was no practical difference between them and a modern multi-core CPU.

-- 
Grant Edwards   grant.b.edwardsYow! All of life is a blur
  at   of Republicans and meat!
  gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] multicore/cpu history

[Peter J. Holzer]

On 2018-03-25 22:52:59 +, Steven D'Aprano wrote:
> On Sun, 25 Mar 2018 23:29:07 +0200, Peter J. Holzer wrote:
> >> >> By the way, multiple CPU machines are different from CPUs with
> >> >> multiple cores:
> >> >>
> >> >> http://smallbusiness.chron.com/multiple-cpu-vs-multicore-33195.html
> >> > 
> >> > Yeah, it was always "multiple CPUs", not "multiple cores" when I was
> >> > growing up.
> > 
> > Yes, but the difference is only an implementation detail.
> 
> Not really. With multiple CPUs, you have the option of running two 
> distinct OSes in isolation, not merely virtual machines but actual 
> distinct machines in the same box.

Not in general, no. There may be hardware architectures which allow this
(if I remember correctly, hardware partitioning on HP and IBM unix
machines in the early noughties worked like this), but on a typical PC
motherboard this wouldn't work: There is a lot of shared hardware
outside of the CPUs, and two OSes running on different processors would
have to be aware of each other to avoid stepping on each other's toes.
And if they can do that, they can also do it on two cores of the same
CPU.

In a normal SMP system, there is no real difference between having 2
8-core processors and 1 16-core processor from the OS's point of view.
The scheduler cares about it because caches and NUMA may make migrating
a process from one core to another more expensive depending on where
that other core is physically, but otherwise a core is processor.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A bit of Python history

[Abdur-Rahmaan Janhangeer]

amazing !

Abdur-Rahmaan Janhangeer
https://github.com/Abdur-rahmaanJ

On Fri, 6 Apr 2018, 15:52 Steven D'Aprano, <
steve+comp.lang.pyt...@pearwood.info> wrote:

> I stumbled across this post from the Timbot, Tim Peters, back in 2000,
> where he correctly predicted that Python would get generators and
> iterators in Python 2:
>
> https://mail.python.org/pipermail/python-list/2000-January/033955.html
>
> as well as list comprehensions.
>
>
> --
> Steve
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

A bit of Python history

[Steven D'Aprano]

I stumbled across this post from the Timbot, Tim Peters, back in 2000, 
where he correctly predicted that Python would get generators and 
iterators in Python 2:

https://mail.python.org/pipermail/python-list/2000-January/033955.html

as well as list comprehensions.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] multicore/cpu history

[Gene Heskett]

On Monday 26 March 2018 12:12:46 Dennis Lee Bieber wrote:

> On Mon, 26 Mar 2018 11:30:54 -0400, Gene Heskett
> 
>
> declaimed the following:
> >On Monday 26 March 2018 10:06:36 Dennis Lee Bieber wrote:
>
>   
>
> >>As I recall, the bootloader on the Raspberry Pi runs on the
> >> graphics processor, and it sets up the memory image for Linux
> >> before passing control to the ARM processor.
> >
> >Does that come with docs on how to change kernels in case the one you
> > are running has to be rebooted 2-10 times to get the keyboard/mouse
> > ducks all in a row so it doesn't randomly throw away events?
>
>   If Broadcom hasn't changed things, the bootloader is a black-box blob
> provided by Broadcom to the R-Pi foundation. The Linux kernel likely
> falls under regular Linux image management (the bootloader doesn't
> contain the Linux image, just runs on the graphics processor to load
> Linux). I don't think R-Pi uses U-Boot (whereas the BeagleBone Black
> is shoving everything into U-Boot -- making most of the BBB text books
> out-of-date as the kernel no longer loads device tree overlays,
> they've been pre-loaded by U-Boot)

Which to me, isn't a lot of help. But, thats broadcom...

Too bad I can't put a bounty on them. It seems to me that any outfit with 
more lawyers than engineers ought to fall over from top heavy and foot 
damage eventually, but they seem to be the exception to that rule.

OTOH, its even harder to get any usable info out of Pine to facilitate 
using an rtai kernel on a product of theirs called a rock64.  Which can 
build that kernel in under an hour. And it does use u-boot, or claims 
to.  Here we have another credit card sized boy wonder computer thats 
probably 10x or more faster than a top of the line pi, with a usb3 port, 
and the info as to how to make it work seems locked in a safe behind 
non-disses they can't even admit to.

Hell of a way to run a train.  Thanks Dennis.

> --
>   Wulfraed Dennis Lee Bieber AF6VN
>   wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] multicore/cpu history

[Gene Heskett]

On Monday 26 March 2018 10:06:36 Dennis Lee Bieber wrote:

> On Mon, 26 Mar 2018 10:02:15 + (UTC), Steven D'Aprano
>
>  declaimed the following:
> >Hardware people can probably tell you what it is that CPUs do that
> > FPUs and GPUs don't do. Or specialised Bitcoin mining chips.
> > Whatever it is that they don't do, but CPUs do, is probably a good
> > dividing line between "CPU" and "auxiliary chip".
>
>   And then... to confuse matters...
>
>   As I recall, the bootloader on the Raspberry Pi runs on the graphics
> processor, and it sets up the memory image for Linux before passing
> control to the ARM processor.

Does that come with docs on how to change kernels in case the one you are 
running has to be rebooted 2-10 times to get the keyboard/mouse ducks 
all in a row so it doesn't randomly throw away events?
>
> --
>   Wulfraed Dennis Lee Bieber AF6VN
>   wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/



-- 
Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] multicore/cpu history

[Steven D'Aprano]

On Mon, 26 Mar 2018 10:03:43 +1100, Chris Angelico wrote:

> At what point does it change from being two CPUs to being one CPU and
> one auxiliary processing unit?

When someone writes an OS that will run on the "auxiliary processing 
unit" alone, then it's probably time to start calling it a CPU :-)


> Back in the 80s and early 90s, the
> auxiliary was most likely to be a floating-point unit; today, it'd be a
> graphics chip. But it could just as easily be a Lisp chip.

Or a Forth chip.

Hardware people can probably tell you what it is that CPUs do that FPUs 
and GPUs don't do. Or specialised Bitcoin mining chips. Whatever it is 
that they don't do, but CPUs do, is probably a good dividing line between 
"CPU" and "auxiliary chip".


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] multicore/cpu history

[Chris Angelico]

On Mon, Mar 26, 2018 at 9:52 AM, Steven D'Aprano
 wrote:
> On Sun, 25 Mar 2018 23:29:07 +0200, Peter J. Holzer wrote:
>
> [...]
>>> >> By the way, multiple CPU machines are different from CPUs with
>>> >> multiple cores:
>>> >>
>>> >> http://smallbusiness.chron.com/multiple-cpu-vs-multicore-33195.html
>>> >
>>> > Yeah, it was always "multiple CPUs", not "multiple cores" when I was
>>> > growing up.
>>
>> Yes, but the difference is only an implementation detail.
>
> Not really. With multiple CPUs, you have the option of running two
> distinct OSes in isolation, not merely virtual machines but actual
> distinct machines in the same box. And the CPUs don't necessarily need to
> be the same type, see for example the hybrid Apple Mac/Lisp Machine
> released in the late 1980s or early 90s.

At what point does it change from being two CPUs to being one CPU and
one auxiliary processing unit? Back in the 80s and early 90s, the
auxiliary was most likely to be a floating-point unit; today, it'd be
a graphics chip. But it could just as easily be a Lisp chip.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list