Re: DoesNotExist at /admin/blog/blog/add/: blog matching query does not exist

[mailinglists]

Sorry, the wrong list.

Op 2022-09-12T20:35:46+ schreef mailingli...@vanwingerde.nl
 in bericht
, inzake:
 het volgende.

> Suddenly I can no longer add blogs to Django. Django says 'blog
> matching query does not exist'. That seems strange to me because I
> want to add something to the database and not request a blog. What
> could be going on here?
> 
> admin.py:
> class blogadmin(admin.ModelAdmin):
> def save_model(self, request, obj, form, change):
> if not change:
> obj.added_by_user = request.user.username
> obj.ip_added = request.META['REMOTE_ADDR']
> if obj.ip_added == '127.0.0.1':
> obj.ip_added = request.META['HTTP_X_FORWARDED_FOR']
> obj.publish = False
> del obj.publish_date
> else:
> obj.changed_by_user = request.user.username
> obj.ip_changed = request.META['REMOTE_ADDR']
> if obj.ip_changed == '127.0.0.1':
> obj.ip_changed = request.META['HTTP_X_FORWARDED_FOR']
> if obj.publish == True:
> if not obj.publish_date:
> obj.publish_date = obj.datetime_changed
> super(blogadmin,self).save_model(request, obj, form, change)
> 
> I can add text, photographs etc., but no blogs.
> 
> Notifications:
> DoesNotExist at /admin/blog/blog/add/
> 
> blog matching query does not exist.
> 
> Request Method:   POST
> Request URL:  http://127.0.0.1:8001/admin/blog/blog/add/
> Django Version:   4.1.1
> Exception Type:   DoesNotExist
> Exception Value:  
> 
> blog matching query does not exist.
> 
> Traceback
> Environment:
> Request Method: POST
> Request URL: http://127.0.0.1:8001/admin/blog/blog/add/
> 
> Django Version: 4.1.1
> Python Version: 3.10.5
> Installed Applications:
> ['grappelli',
>  'django.contrib.admin',
>  'django.contrib.admindocs',
>  'django.contrib.auth',
>  'django.contrib.contenttypes',
>  'django.contrib.sessions',
>  'django.contrib.messages',
>  'django.contrib.staticfiles',
>  'blog.apps.BlogConfig']
> Installed Middleware:
> ['django.middleware.security.SecurityMiddleware',
>  'django.contrib.sessions.middleware.SessionMiddleware',
>  'django.middleware.common.CommonMiddleware',
>  'django.middleware.csrf.CsrfViewMiddleware',
>  'django.contrib.auth.middleware.AuthenticationMiddleware',
>  'django.contrib.messages.middleware.MessageMiddleware',
>  'django.middleware.clickjacking.XFrameOptionsMiddleware',
>  'django.contrib.admindocs.middleware.XViewMiddleware']
> 
> Traceback (most recent call last):
>   File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/exception.py",
> line 55, in inner response = get_response(request) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/base.py",
> line 197, in _get_response response = wrapped_callback(request,
> *callback_args, **callback_kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
> line 686, in wrapper return self.admin_site.admin_view(view)(*args,
> **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
> line 133, in _wrapped_view response = view_func(request, *args,
> **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/views/decorators/cache.py",
> line 62, in _wrapped_view_func response = view_func(request, *args,
> **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/sites.py",
> line 242, in inner return view(request, *args, **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
> line 1890, in add_view return self.changeform_view(request, None,
> form_url, extra_context) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
> line 46, in _wrapper return bound_method(*args, **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
> line 133, in _wrapped_view response = view_func(request, *args,
> **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
> line 1750, in changeform_view return self._changeform_view(request,
> object_id, form_url, extra_context) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
> line 1802, in _changeform_view self.save_model(request, new_object,
> form, 

Re: DoesNotExist at /admin/blog/blog/add/: blog matching query does not exist

[Enerel Amgalan via Python-list]

unsubscribe 

-Bat

> On Sep 12, 2022, at 14:32, mailingli...@vanwingerde.nl wrote:
> 
> Suddenly I can no longer add blogs to Django. Django says 'blog
> matching query does not exist'. That seems strange to me because I want
> to add something to the database and not request a blog. What could be
> going on here?
> 
> admin.py:
> class blogadmin(admin.ModelAdmin):
>def save_model(self, request, obj, form, change):
>if not change:
>obj.added_by_user = request.user.username
>obj.ip_added = request.META['REMOTE_ADDR']
>if obj.ip_added == '127.0.0.1':
>obj.ip_added = request.META['HTTP_X_FORWARDED_FOR']
>obj.publish = False
>del obj.publish_date
>else:
>obj.changed_by_user = request.user.username
>obj.ip_changed = request.META['REMOTE_ADDR']
>if obj.ip_changed == '127.0.0.1':
>obj.ip_changed = request.META['HTTP_X_FORWARDED_FOR']
>if obj.publish == True:
>if not obj.publish_date:
>obj.publish_date = obj.datetime_changed
>super(blogadmin,self).save_model(request, obj, form, change)
> 
> I can add text, photographs etc., but no blogs.
> 
> Notifications:
> DoesNotExist at /admin/blog/blog/add/
> 
> blog matching query does not exist.
> 
> Request Method:POST
> Request URL:http://127.0.0.1:8001/admin/blog/blog/add/
> Django Version:4.1.1
> Exception Type:DoesNotExist
> Exception Value:
> 
> blog matching query does not exist.
> 
> Traceback
> Environment:
> Request Method: POST
> Request URL: http://127.0.0.1:8001/admin/blog/blog/add/
> 
> Django Version: 4.1.1
> Python Version: 3.10.5
> Installed Applications:
> ['grappelli',
> 'django.contrib.admin',
> 'django.contrib.admindocs',
> 'django.contrib.auth',
> 'django.contrib.contenttypes',
> 'django.contrib.sessions',
> 'django.contrib.messages',
> 'django.contrib.staticfiles',
> 'blog.apps.BlogConfig']
> Installed Middleware:
> ['django.middleware.security.SecurityMiddleware',
> 'django.contrib.sessions.middleware.SessionMiddleware',
> 'django.middleware.common.CommonMiddleware',
> 'django.middleware.csrf.CsrfViewMiddleware',
> 'django.contrib.auth.middleware.AuthenticationMiddleware',
> 'django.contrib.messages.middleware.MessageMiddleware',
> 'django.middleware.clickjacking.XFrameOptionsMiddleware',
> 'django.contrib.admindocs.middleware.XViewMiddleware']
> 
> Traceback (most recent call last):
>  File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/exception.py",
> line 55, in inner response = get_response(request) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/base.py",
> line 197, in _get_response response = wrapped_callback(request,
> *callback_args, **callback_kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
> line 686, in wrapper return self.admin_site.admin_view(view)(*args,
> **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
> line 133, in _wrapped_view response = view_func(request, *args,
> **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/views/decorators/cache.py",
> line 62, in _wrapped_view_func response = view_func(request, *args,
> **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/sites.py",
> line 242, in inner return view(request, *args, **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
> line 1890, in add_view return self.changeform_view(request, None,
> form_url, extra_context) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
> line 46, in _wrapper return bound_method(*args, **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
> line 133, in _wrapped_view response = view_func(request, *args,
> **kwargs) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
> line 1750, in changeform_view return self._changeform_view(request,
> object_id, form_url, extra_context) File
> "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
> line 1802, in _changeform_view self.save_model(request, new_object,
> form, not add) File "/var/django/liakoster.nl/blog-1/blog/admin.py",
> l

DoesNotExist at /admin/blog/blog/add/: blog matching query does not exist

[mailinglists]

Suddenly I can no longer add blogs to Django. Django says 'blog
matching query does not exist'. That seems strange to me because I want
to add something to the database and not request a blog. What could be
going on here?

admin.py:
class blogadmin(admin.ModelAdmin):
def save_model(self, request, obj, form, change):
if not change:
obj.added_by_user = request.user.username
obj.ip_added = request.META['REMOTE_ADDR']
if obj.ip_added == '127.0.0.1':
obj.ip_added = request.META['HTTP_X_FORWARDED_FOR']
obj.publish = False
del obj.publish_date
else:
obj.changed_by_user = request.user.username
obj.ip_changed = request.META['REMOTE_ADDR']
if obj.ip_changed == '127.0.0.1':
obj.ip_changed = request.META['HTTP_X_FORWARDED_FOR']
if obj.publish == True:
if not obj.publish_date:
obj.publish_date = obj.datetime_changed
super(blogadmin,self).save_model(request, obj, form, change)

I can add text, photographs etc., but no blogs.

Notifications:
DoesNotExist at /admin/blog/blog/add/

blog matching query does not exist.

Request Method: POST
Request URL:http://127.0.0.1:8001/admin/blog/blog/add/
Django Version: 4.1.1
Exception Type: DoesNotExist
Exception Value:

blog matching query does not exist.

Traceback
Environment:
Request Method: POST
Request URL: http://127.0.0.1:8001/admin/blog/blog/add/

Django Version: 4.1.1
Python Version: 3.10.5
Installed Applications:
['grappelli',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'blog.apps.BlogConfig']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.contrib.admindocs.middleware.XViewMiddleware']

Traceback (most recent call last):
  File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/exception.py",
line 55, in inner response = get_response(request) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/base.py",
line 197, in _get_response response = wrapped_callback(request,
*callback_args, **callback_kwargs) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
line 686, in wrapper return self.admin_site.admin_view(view)(*args,
**kwargs) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
line 133, in _wrapped_view response = view_func(request, *args,
**kwargs) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/views/decorators/cache.py",
line 62, in _wrapped_view_func response = view_func(request, *args,
**kwargs) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/sites.py",
line 242, in inner return view(request, *args, **kwargs) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
line 1890, in add_view return self.changeform_view(request, None,
form_url, extra_context) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
line 46, in _wrapper return bound_method(*args, **kwargs) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py",
line 133, in _wrapped_view response = view_func(request, *args,
**kwargs) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
line 1750, in changeform_view return self._changeform_view(request,
object_id, form_url, extra_context) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
line 1802, in _changeform_view self.save_model(request, new_object,
form, not add) File "/var/django/liakoster.nl/blog-1/blog/admin.py",
line 26, in save_model super(blogadmin,self).save_model(request, obj,
form, change) File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py",
line 1225, in save_model obj.save() File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py",
line 831, in save self.save_base( File
"/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py",
line 882, in save_base updated = self._save_table( File
"/va

Zato blog post: A successful Python 3 migration story

[Terry Reedy]

The migration was from 2.7 to 2.7 and 3.x, rather than 3.x only.
I think it worth reading for anyone interested in the subject.
https://zato.io/blog/posts/python-3-migration-success-story.html

60,000 lines of Python and Cython, 130 external dependencies (but only 
10 not already 3.x ready) took 2 people 80 hours total.


Their head start was to write the 2.7 modules, from the beginning, with 
the following at the top.
from __future__ import absolute_import, division, print_function, 
unicode_literals



--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Blog for basic python programming

[manishti2004]

Python for Engineers - Solve Problems by Coding Solutions

https://pythonforengineers.blogspot.in
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[justin walters]

On Fri, Sep 29, 2017 at 12:14 PM, Bill  wrote:

>
> I'll write for the possible benefit of any beginners who may be reading.
> I guess by definition, if one still has a "bug" it's because one doesn't
> quite understand what the code is doing. And I would say you should lose
> your license if you "fix something", and don't understand why it works
> (within reason of course--some mystery's of library functions should
> probably remain so forever). So ADT (Any Damn Thing--I just made that up
> that acronym) you can do to understand your code better is fair game! : )
>   In fact, in my experience, the sooner you start getting a little bit
> angry, the sooner you'll get to the heart of matter.  Usually, what looks
> like a long route, isn't, in the end.  Don't be afraid to write *really
> descriptive* output statements, and do so even though you "don't need to".
> Besides for making you more productive, it will help soothe you : )
>  Beginners almost never need to...  I think that getting out of the
> beginner phase requires developing a certain amount of humility.  Just wait
> 5 or 10 years, any look back, and see if what I've written isn't more true
> than false.
>
> The only part I am unsure of is whether you are supposed to get a little
> big angry or not (YMMV).  I find 2 cups of coffee about right. That is, 2
> before and 2 after lunch. Of course, that does not include "meetings".
> --
> https://mail.python.org/mailman/listinfo/python-list
>

Reminds me of a bug I had to chase down recently.

I've been working on the front-end of this web application for a while.
It's an SPA built
with Vuejs. The bug arose in the login workflow. Basically, it went like
this:

client loads login screen -> user enters credentials into form and submits
-> client sends credentials to server ->
server verifies credentials -> server sends back auth token -> client
receives auth token and stores it ->
client redirects user to home screen -> home screen makes get request for
some data

Now, this worked perfectly fine everywhere except for Safari 9.1 on OSX.

A user could login just fine on Safari 9.1, but after that, no requests
would complete. Safari's dev tools
were no help because they were not showing any errors or any failed
requests. I checked the server logs
and found that no requests were even sent.

It took me 2 days to figure out this bug. I tracked it down to the function
that injected the authorization
header into all requests if the user was logged in. Based on
troubleshooting, I knew it couldn't be anything else.
That said, I was still confused because this worked on literally every
other browser(even IE 9).

After searching for people with similar problems and coming up with nothing
I got to thinking about the
asynchronous nature of JS. So, out of sheer frustration I moved the line of
code that stored the auth token
from one function to another, booted up my testing environment, and it
worked.

So, the bug was basically because Safari was waiting for a specific
function call to complete before
it committed the token to local storage even though the line of code that
did so was within said function.

So, two days worth of work to move a single line of code from one function
to another. You can only imagine
the tirade of curse words directed at apple during the above calamity.

Had I simply written a console log for every function down the chain, I may
have been able to find the
cause of the bug more quickly.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Gregory Ewing]

Bill wrote:
Don't be afraid to write *really descriptive* output statements, and do 
so even though you "don't need to".


Yeah, often when I'm writing something tricky I'll proactively
put in some code to print intermediate state to reassure myself
that things are on track. Being more verbose with them than I
think necessary can save a few trips around the debug cycle.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Gregory Ewing]

Steve D'Aprano wrote:

(1) I know there's a bug in a specific chunk of code, but I'm having trouble
working out where. When everything else fails, if I perturb the code a bit
(reorder lines, calculate things in a different order, rename variables, etc)
it may change the nature of the bug enough for me to understand what's
happening.


> Its an experiment, but not really "carefully designed".

I think it's more carefully designed than you give it credit for.
You still need to understand quite a lot about the program to know
what changes are likely to yield useful information, and how to
interpret the results.


Its more like "what
happens if we hit this bit with a hammer?"


In biology it's called a "shotgun experiment". "If we blast this
bit of DNA with radiation, what part of the organism does it
mess up?"


(2) I hate off by one errors, and similar finicky errors that mean your code is
*almost* right. I especially hate them when I'm not sure which direction I'm
off by one. If you have unit tests that are failing, sometimes its quicker and
easier to randomly perturb the specific piece of code until you get the right
answer, rather than trying to analyse it.


With off-by-one errors it's still pretty specific -- start
the loop at 1 instead of 0, etc.

But in cases like that I prefer to rewrite the code so that
it's obvious where it should start and finish.


The complexity of code increases faster than our ability to manage that
complexity.


And then there's "If you write the code as cleverly as you can,
you won't be smart enough to debug it!"

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Chris Angelico]

On Sat, Sep 30, 2017 at 5:14 AM, Bill  wrote:
> I'll write for the possible benefit of any beginners who may be reading.  I
> guess by definition, if one still has a "bug" it's because one doesn't quite
> understand what the code is doing. And I would say you should lose your
> license if you "fix something", and don't understand why it works (within
> reason of course--some mystery's of library functions should probably remain
> so forever).

My programmer's license comes from MIT and it can't be lost.

https://opensource.org/licenses/MIT

Kappa

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Chris Angelico]

On Sat, Sep 30, 2017 at 2:42 AM, Steve D'Aprano
 wrote:
> Oh, and I'd like to make a (moderate) defense of a kind of "bug fixing by 
> random
> perturbation". Obviously making unrelated, arbitrary changes to code is bad.
> But making non-arbitrary but not fully understood changes to relevant code
> sections can be useful in (at least) two scenarios.
>
> (1) I know there's a bug in a specific chunk of code, but I'm having trouble
> working out where. When everything else fails, if I perturb the code a bit
> (reorder lines, calculate things in a different order, rename variables, etc)
> it may change the nature of the bug enough for me to understand what's
> happening.
>
> That's not *random* or *arbitrary* changes, but they are changes not directed 
> at
> any specific outcome other than "make the code a bit different, and see if the
> error changes". I'd like to say it is the debugging technique of last resort,
> except its perhaps not quite as *last* resort as I'd like, especially in code
> I'm not familiar with.
>
> Its an experiment, but not really "carefully designed". Its more like "what
> happens if we hit this bit with a hammer?" except that programmers, unlike
> engineers, have the luxury of an Undo switch :-)

Sometimes, when I'm debugging something with one of my students, I'll
say something like "Let's do something stupid". That prefaces a
suggested change that is, perhaps:
* logging something that, by all sane logic, cannot possibly be wrong;
* altering the form of a piece of code in a way that shouldn't affect anything;
* changing something that logically should break the code worse, not fix it;
* or worse.

They're still not "random" changes, but when you exhaust all the
logical and sane things to try, sometimes you do something stupid and
it reveals the bug. I wouldn't EVER tell someone to assume that
they've hit a language or library bug - but if you make a meaningless
change and now it works, maybe that's what you've hit. It does happen.

"Why does my program crash when it talks to THIS server, but it's fine
talking to THAT server?"

... half an hour of debugging later ...

"Okay, so THIS server supports elliptic curve cryptography, but THAT
one doesn't. Great. That still doesn't explain the crash."

... two hours of debugging later ...

"Huh. Maybe this library has a bug with ECC?"

That's pretty close to what happened to me today, with the exception
that I spent less time on it (because I had the library's source code
handy). But in any case, it's a good reason to occasionally try
something stupid.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Steve D'Aprano]

On Fri, 29 Sep 2017 08:34 pm, D'Arcy Cain wrote:

> On 09/29/2017 03:15 AM, Steve D'Aprano wrote:
>> "Carefully-designed experiments" -- yeah, that is so totally how the coders
>> I've worked with operate *wink*
>> 
>> I think that's an awfully optimistic description of how the average
>> programmer works :-)
> 
> Better not hire average programmers then.

Okay.

Can I consider the top 10% of programmers, or must I only consider only those in
the top 1%?

If I'm on a budget and the code isn't that critical, can I consider those in the
top 20% for junior roles?


> I do "Carefully-designed 
> experiments" to find non-obvious bugs so I guess I am not average.

Okay.

By the way, *in context* (before you deleted the original text) there was no
mention about "non-obvious bugs". Greg Ewing and Chris Angelico were talking
about the general difference between the process used by novices and that used
by experts and how beginners often attempt to fix bugs by making random
changes, while experts don't.

I've certainly seen beginners make arbitrary changes to unrelated parts of their
code trying to fix a bug. Often many different changes all at once. So that's
another difference between beginners and experts:

- experts have the self-control to make only one change at a time, 
  when it matters; beginners don't know when it matters.


Oh, and I'd like to make a (moderate) defense of a kind of "bug fixing by random
perturbation". Obviously making unrelated, arbitrary changes to code is bad.
But making non-arbitrary but not fully understood changes to relevant code
sections can be useful in (at least) two scenarios.

(1) I know there's a bug in a specific chunk of code, but I'm having trouble
working out where. When everything else fails, if I perturb the code a bit
(reorder lines, calculate things in a different order, rename variables, etc)
it may change the nature of the bug enough for me to understand what's
happening.

That's not *random* or *arbitrary* changes, but they are changes not directed at
any specific outcome other than "make the code a bit different, and see if the
error changes". I'd like to say it is the debugging technique of last resort,
except its perhaps not quite as *last* resort as I'd like, especially in code
I'm not familiar with.

Its an experiment, but not really "carefully designed". Its more like "what
happens if we hit this bit with a hammer?" except that programmers, unlike
engineers, have the luxury of an Undo switch :-)


(2) I hate off by one errors, and similar finicky errors that mean your code is
*almost* right. I especially hate them when I'm not sure which direction I'm
off by one. If you have unit tests that are failing, sometimes its quicker and
easier to randomly perturb the specific piece of code until you get the right
answer, rather than trying to analyse it.

"Should I add one here? Maybe subtract one? Start at zero or one? Ah bugger it,
I'll try them all and see which one works."

This is only effective when you have exhaustive tests that exercise all the
relevant cases and can tell you when you've hit the right solution.

On the other hand, sometimes the bug isn't as clear cut as you thought, and you
really do need to analyse the situation carefully.


> I get the impression that many people here are above average too.
> 
> Personally I think you are being pessimistic about "average"
> programmers.  Perhaps you just know the sloppy kind.

One needs to only look at the quality of software, whether open source or
commercial closed source, to feel pessimistic about the ability of even
excellent programmers to write good, secure, bug-free code.

The complexity of code increases faster than our ability to manage that
complexity.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[leam hall]

On Fri, Sep 29, 2017 at 10:52 AM, justin walters  wrote:

>
> I got through writing all of the above without realizing that you meant you
> wanted to build a
> desktop application and not a web application. Though, I think the advice
> is still helpful.
>
>
Yes and no. Seriously thanks!

I am at first targeting a desktop app just to be simpler and to push me to
learn Tkinter. However, it's more likely to end up a simple web app once I
learn enough Bottle/Flask to make it work. Or I may just skip Tkinter for
the nonce and see if I can do it with web forms.

Leam
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[justin walters]

On Fri, Sep 29, 2017 at 2:57 AM, Leam Hall  wrote:

> On 09/27/2017 10:33 PM, Stefan Ram wrote:
>
>Some areas of knowledge follow, a programmer should not be
>>ignorant in all of them:
>>
>
> ---
>
> Stefan, this is list AWESOME!
>
> I have started mapping skills I have to the list and ways to build skills
> I don't have. Last night I started working on a project that has been on my
> mind for over a year; taking a CSV list of game characters and putting them
> into a MongoDB datastore. Now I need to figure out how to build an
> interface for CRUD operations using Python, pymongo, and maybe Tk.
>
> I appreciate the structure your list provides. Thank you!
>
> Leam
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

Python web backends just happen to be my specialty. I do mostly Django, but
it doesn't
mesh well with MongoDB. Well, it does if you still use an RDBMS forsome
things.

I can recommend the following:

- Flask: http://flask.pocoo.org/
Small and light But not tiny. Not really fast, not really slow. Not a
ton of batteries included.
Tons of third-party extensions.

- ApiStar: https://github.com/encode/apistar
New kid on the block. Specializes in APIS. No template integrations.
Best for serving
JSON through a RESTful interface. Fairly quick, but not blazing fast.
Has the upside that any
web API can be exposed as a CLI API. Not a ton of third party
extensions available. Would
be a good choice if you don't want to build a desktop application
instead of a web application
as it will help design the API that something like Tkinter will sit on
top of.

- Sanic: https://github.com/channelcat/sanic
Another new kid. Python 3.5+ only. Uses the new async capabilities
quite heavilly.
Based on falcon. Blazing fast. No batteries included. Small number of
fairly high
quality third-party extensions.

- Django: https://www.djangoproject.com/
The old workhorse. Mature and proven. Best choice for reliability. Not
fast, not slow.
Huge collection of third party extensions ranging in quality. Though,
it is pretty heavilly
integrated with it's relational Db backends. If you decide on this, you
would need to
use postgres/sqlite/mysql to store all of Django's built in model
classes(tables).

I got through writing all of the above without realizing that you meant you
wanted to build a
desktop application and not a web application. Though, I think the advice
is still helpful.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Chris Angelico]

On Fri, Sep 29, 2017 at 8:34 PM, D'Arcy Cain  wrote:
> On 09/29/2017 03:15 AM, Steve D'Aprano wrote:
>>
>> "Carefully-designed experiments" -- yeah, that is so totally how the
>> coders I've
>> worked with operate *wink*
>>
>> I think that's an awfully optimistic description of how the average
>> programmer
>> works :-)
>
>
> Better not hire average programmers then.  I do "Carefully-designed
> experiments" to find non-obvious bugs so I guess I am not average.  I get
> the impression that many people here are above average too.
>
> Personally I think you are being pessimistic about "average" programmers.
> Perhaps you just know the sloppy kind.

Based on any mathematical definition of "average", yes, I am pretty
pessimistic about the average programmer :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[D'Arcy Cain]

On 09/29/2017 03:15 AM, Steve D'Aprano wrote:

"Carefully-designed experiments" -- yeah, that is so totally how the coders I've
worked with operate *wink*

I think that's an awfully optimistic description of how the average programmer
works :-)


Better not hire average programmers then.  I do "Carefully-designed 
experiments" to find non-obvious bugs so I guess I am not average.  I 
get the impression that many people here are above average too.


Personally I think you are being pessimistic about "average" 
programmers.  Perhaps you just know the sloppy kind.


--
D'Arcy J.M. Cain
Vybe Networks Inc.
http://www.VybeNetworks.com/
IM:da...@vex.net VoIP: sip:da...@vybenetworks.com
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Bill]

Steve D'Aprano wrote:


(say). Reading error messages is a skill that must be learned, even in Python.
Let alone (say) gcc error messages, which are baroque to an extreme. The other
day I was getting an error like:

/tmp/ccchKJVU.o: In function `__static_initialization_and_destruction_0(int,
int)':
foo.cpp:(.text+0x7c): undefined reference to `std::ios_base::Init::Init()'
foo.cpp:(.text+0x91): undefined reference to `std::ios_base::Init::~Init()'
collect2: error: ld returned 1 exit status




Yes, that's the sort of character-building that I was referring to (that 
a beginner needs to learn!)They have to learn that if it "breaks", 
then there must be a simpler way to break it!  Hopefully one which will 
satisfy Log_2 (n).: )


--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Steve D'Aprano]

On Fri, 29 Sep 2017 03:28 pm, Gregory Ewing wrote:

> Chris Angelico wrote:
>> finding the bug is basically searching
>> through a problem space of all things that could potentially cause
>> this symptom. A novice could accidentally stumble onto the right
>> solution to a tricky bug, or an expert could search a thousand other
>> things and only get to the true cause after a long time.
> 
> What I think is more important than how *long* it takes is
> *how* they go about finding the bug.
> 
> A novice will make wild guesses about what might be wrong
> and make random changes to the program in the hope of
> making it work. An experienced programmer will conduct
> a series of carefully-designed experiments to narrow
> down the cause.

"Carefully-designed experiments" -- yeah, that is so totally how the coders I've
worked with operate *wink*

I think that's an awfully optimistic description of how the average programmer
works :-)


> A result of this is that a true expert will never have
> to try a thousand possibilities. He will be able to
> search a space of N possible causes in O(log N) operations.

I don't think that, *in general*, possible causes of a bug can be neatly sorted
in such a way that we can do a binary search over the space of all possible
causes.

I also think it is unlikely that even the best programmer enumerates all the
possible causes before working on them. If you don't enumerate all the causes
first, how to you sort them?

In my experience, even good coders are likely to say "there's only three
possible things that could be causing this bug", and later on report it was
actually the fifth thing.

More likely, the experienced programmer is better at eliminating irrelevancies
and narrowing in on the space of likely candidates, or at least narrowing down
on the region of code that is relevant, while less experienced programmers
waste more time looking at things which couldn't possibly be the cause[1].

More likely, the experienced programmer makes better use of his or her tools.
While the novice is still messing about trying to guess the problem by pure
logical reasoning, the expert is using a few useful print calls, or a debugger,
to narrow down to where the problem actually is.

And experts are likely to be better at extrapolating "well, since that's not the
problem... maybe its this?". And you learn that not by pure logic, but by being
bitten by nasty bugs:

"I can't see any possibly way that this could be involved, but I came across a
similar situation once before and this was the solution... [confirms hypothesis
and fixes bug] oh of course, that's how the bug occurs! Its obvious in
hindsight."

More experienced programmers are confident enough to know when to walk away from
a problem and let their subconscious work on it. Or when to go and explain it
to the cleaning lady[2]. Or when to admit defeat and hand over to a fresh pair
of eyes who won't be stuck in the same mindset.

Even just getting to the point of being able to reason *where to start* requires
a fair amount of experience. A true beginner to programming altogether won't
even know how to read a stack trace to identify the line where the bug occurs,
let alone understand why it happened. I wish I had a dollar for every time some
beginner says something like:

"I get a syntax error. What's wrong with my code?"

and eventually after much arm-twisting is convinced to post the last line of the
traceback, which turns out not to be a syntax error at all:

"TypeError: object of type 'int' has no len()"

(say). Reading error messages is a skill that must be learned, even in Python.
Let alone (say) gcc error messages, which are baroque to an extreme. The other
day I was getting an error like:

/tmp/ccchKJVU.o: In function `__static_initialization_and_destruction_0(int,
int)':
foo.cpp:(.text+0x7c): undefined reference to `std::ios_base::Init::Init()'
foo.cpp:(.text+0x91): undefined reference to `std::ios_base::Init::~Init()'
collect2: error: ld returned 1 exit status


Of course, its obvious that the problem here is that I needed to install g++ as
well as gcc, right? :-)


> This doesn't necessarily translate into time, because
> in some situations the experiments can be very time-
> consuming to perform. But other things being equal,
> the expert's bug-finding algorithm is faster than the
> novice's.





[1] Although, the *really* experienced programmer knows that in sufficiently
baroque and highly-coupled code, a bug could be caused by *anything*
*anywhere*. (This is one reason why global variables are bad.)

I still haven't gotten over hearing about a bug in the Internet Explorer
routines for handling WMF files, which lead to being unable to copy and paste
plain text in any application.


[2] The place I worked had a cuddly penguin toy called Mr Snuggles, and the
programmers would go and explain the problem to him. It never[3] failed.

[3] Well, hardly ever.


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, 

Re: Beginners and experts (Batchelder blog post)

[Chris Angelico]

On Fri, Sep 29, 2017 at 3:28 PM, Gregory Ewing
 wrote:
> Chris Angelico wrote:
>>
>> finding the bug is basically searching
>> through a problem space of all things that could potentially cause
>> this symptom. A novice could accidentally stumble onto the right
>> solution to a tricky bug, or an expert could search a thousand other
>> things and only get to the true cause after a long time.
>
>
> What I think is more important than how *long* it takes is
> *how* they go about finding the bug.
>
> A novice will make wild guesses about what might be wrong
> and make random changes to the program in the hope of
> making it work. An experienced programmer will conduct
> a series of carefully-designed experiments to narrow
> down the cause.
>
> A result of this is that a true expert will never have
> to try a thousand possibilities. He will be able to
> search a space of N possible causes in O(log N) operations.
>
> This doesn't necessarily translate into time, because
> in some situations the experiments can be very time-
> consuming to perform. But other things being equal,
> the expert's bug-finding algorithm is faster than the
> novice's.

This is very true, which is part of why I said that in programming, it
takes one to know one - to observe a candidate and determine his/her
experimental technique, you basically need to yourself be a
programmer.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Gregory Ewing]

Chris Angelico wrote:

finding the bug is basically searching
through a problem space of all things that could potentially cause
this symptom. A novice could accidentally stumble onto the right
solution to a tricky bug, or an expert could search a thousand other
things and only get to the true cause after a long time.


What I think is more important than how *long* it takes is
*how* they go about finding the bug.

A novice will make wild guesses about what might be wrong
and make random changes to the program in the hope of
making it work. An experienced programmer will conduct
a series of carefully-designed experiments to narrow
down the cause.

A result of this is that a true expert will never have
to try a thousand possibilities. He will be able to
search a space of N possible causes in O(log N) operations.

This doesn't necessarily translate into time, because
in some situations the experiments can be very time-
consuming to perform. But other things being equal,
the expert's bug-finding algorithm is faster than the
novice's.
--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Larry Martell]

On Thu, Sep 28, 2017 at 5:08 PM, Chris Angelico  wrote:
> Yep. Pick anyone on this list that you believe is an expert, and ask
> him/her for a story of a long debug session that ended up finding a
> tiny problem. I can pretty much guarantee that every expert programmer
> will have multiple such experiences, and it's just a matter of
> remembering one with enough detail to share the story.

The software development process can be summed up thusly:

I can’t fix this
Crisis of confidence
Questions career
Questions life
Oh it was a typo, cool
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Chris Angelico]

On Fri, Sep 29, 2017 at 7:47 AM, Bill  wrote:
> I won't claim to be any sort of "expert".  But one memorable problem, for
> me, was ultimately accounted for by the "inherent problem" of the floating
> point variables x0 and xo coexisting in the same module.  It's sort of funny
> if you think about it just right. FWIW, my job was to fix the problem, I
> didn't create it!

Today I helped one of my students debug an issue that was exacerbated
by a flawed shuffle function that, while capable of returning any
permutation of the input, had a bit of a tendency to leave things
early if they started early - it was about 8% more likely to pick the
first element than the last. Doesn't sound like much, but it increased
the chances of a collision pretty significantly. Now THAT was fun to
debug.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Bill]

Chris Angelico wrote:

On Fri, Sep 29, 2017 at 6:59 AM, Bill  wrote:

Chris Angelico wrote:

Be careful with this one. For anything other than trivial errors (and
even for some trivial errors), finding the bug is basically searching
through a problem space of all things that could potentially cause
this symptom. A novice could accidentally stumble onto the right
solution to a tricky bug, or an expert could search a thousand other
things and only get to the true cause after a long time.

  some "expert"!   ; )


Yep. Pick anyone on this list that you believe is an expert, and ask
him/her for a story of a long debug session that ended up finding a
tiny problem. I can pretty much guarantee that every expert programmer
will have multiple such experiences, and it's just a matter of
remembering one with enough detail to share the story.

ChrisA
I won't claim to be any sort of "expert".  But one memorable problem, 
for me, was ultimately accounted for by the "inherent problem" of the 
floating point variables x0 and xo coexisting in the same module.  It's 
sort of funny if you think about it just right. FWIW, my job was to fix 
the problem, I didn't create it!


Bill

--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Chris Angelico]

On Fri, Sep 29, 2017 at 6:59 AM, Bill  wrote:
> Chris Angelico wrote:
>>
>> Be careful with this one. For anything other than trivial errors (and
>> even for some trivial errors), finding the bug is basically searching
>> through a problem space of all things that could potentially cause
>> this symptom. A novice could accidentally stumble onto the right
>> solution to a tricky bug, or an expert could search a thousand other
>> things and only get to the true cause after a long time.
>
>  some "expert"!   ; )
>

Yep. Pick anyone on this list that you believe is an expert, and ask
him/her for a story of a long debug session that ended up finding a
tiny problem. I can pretty much guarantee that every expert programmer
will have multiple such experiences, and it's just a matter of
remembering one with enough detail to share the story.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Bill]

Chris Angelico wrote:

On Fri, Sep 29, 2017 at 5:45 AM, Bill  wrote:

Paul Moore wrote:

On 27 September 2017 at 17:41, leam hall  wrote:

Hehe...I've been trying to figure out how to phrase a question. Knowing
I'm
not the only one who gets frustrated really helps.

I'm trying to learn to be a programmer. I can look at a book and read
basic
code in a few languages but it would be unfair to hire myself out as a
programmer. I'm just not yet worth what it costs to pay my bills.

You're already ahead of the game in wanting to be useful, rather than
just knowing the jargon :-) However, I've always found that the
biggest asset a programmer can have is the simple willingness to
learn.


I basically agree with what has been posted. I just wanted to mention a
couple things that separates beginners and non-beginners. One is "how long
it takes to identify and fix an error"--even a syntax error. And that is a
skill which is acquired with some practice, maybe more "some" than anyone
likes.

Be careful with this one. For anything other than trivial errors (and
even for some trivial errors), finding the bug is basically searching
through a problem space of all things that could potentially cause
this symptom. A novice could accidentally stumble onto the right
solution to a tricky bug, or an expert could search a thousand other
things and only get to the true cause after a long time.


 some "expert"!   ; )




So while
you're partly correct in saying "how long", you can't just put someone
on the clock and say "if you find the bug in less than five minutes,
you're hired". Ultimately, the only person who can truly evaluate a
programmer's skill is another programmer, usually by watching the
candidate go through this sort of debugging work. But yeah, broadly
speaking, an experienced programmer can usually debug something more
quickly than a novice can. On average.

ChrisA


--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Chris Angelico]

On Fri, Sep 29, 2017 at 5:45 AM, Bill  wrote:
> Paul Moore wrote:
>>
>> On 27 September 2017 at 17:41, leam hall  wrote:
>>>
>>> Hehe...I've been trying to figure out how to phrase a question. Knowing
>>> I'm
>>> not the only one who gets frustrated really helps.
>>>
>>> I'm trying to learn to be a programmer. I can look at a book and read
>>> basic
>>> code in a few languages but it would be unfair to hire myself out as a
>>> programmer. I'm just not yet worth what it costs to pay my bills.
>>
>> You're already ahead of the game in wanting to be useful, rather than
>> just knowing the jargon :-) However, I've always found that the
>> biggest asset a programmer can have is the simple willingness to
>> learn.
>
>
> I basically agree with what has been posted. I just wanted to mention a
> couple things that separates beginners and non-beginners. One is "how long
> it takes to identify and fix an error"--even a syntax error. And that is a
> skill which is acquired with some practice, maybe more "some" than anyone
> likes.

Be careful with this one. For anything other than trivial errors (and
even for some trivial errors), finding the bug is basically searching
through a problem space of all things that could potentially cause
this symptom. A novice could accidentally stumble onto the right
solution to a tricky bug, or an expert could search a thousand other
things and only get to the true cause after a long time. So while
you're partly correct in saying "how long", you can't just put someone
on the clock and say "if you find the bug in less than five minutes,
you're hired". Ultimately, the only person who can truly evaluate a
programmer's skill is another programmer, usually by watching the
candidate go through this sort of debugging work. But yeah, broadly
speaking, an experienced programmer can usually debug something more
quickly than a novice can. On average.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Bill]

Paul Moore wrote:

On 27 September 2017 at 17:41, leam hall  wrote:

Hehe...I've been trying to figure out how to phrase a question. Knowing I'm
not the only one who gets frustrated really helps.

I'm trying to learn to be a programmer. I can look at a book and read basic
code in a few languages but it would be unfair to hire myself out as a
programmer. I'm just not yet worth what it costs to pay my bills.

You're already ahead of the game in wanting to be useful, rather than
just knowing the jargon :-) However, I've always found that the
biggest asset a programmer can have is the simple willingness to
learn.


I basically agree with what has been posted. I just wanted to mention a 
couple things that separates beginners and non-beginners. One is "how 
long it takes to identify and fix an error"--even a syntax error. And 
that is a skill which is acquired with some practice, maybe more "some" 
than anyone likes. Another critical skill is the ability to write good 
documentation--from program requirements, on down. Another is to know 
what is means to "test". Another is to have some familiarity with the 
UML.  Skills in 3 of these 4 area might be assisted by reading about 
software engineering.  So after you have those skills, then, perhaps, 
you can think about "interviewing"--of course a degree will help.  As 
always, your mileage may vary...  It IS True that you don't have to wait 
until you "know everything"--most of use will never get there.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Leam Hall]

On 09/28/2017 04:15 AM, Paul Moore wrote:

With Python, I'd say that an appreciation of the available libraries
is key - both what's in the stdlib, and what's available from PyPI.
That's not to say you should memorise the standard library, but rather
cultivate an approach of "hmm, I'm pretty sure I remember there being
a library for that" and going to look. The best way of getting this is
to actually work with code - you can start with doing coding projects
of your own (it's *always* a good exercise to have a problem that
interests you, and work on coding it - no matter what it is, you'll
learn more about understanding requirements, testing, bug fixing, and
practical programming by working on a project you care about than
you'll ever get reading books) and/or you can look at existing open
source projects that you're interested in, and offer help (there's
always a bug tracker, and typically some simpler items - and you'll
learn a lot from interacting with a larger project).



When I first started in Unix/Linux there was a group called SAGE. They 
had a list of tasks a system admin was expected to be able to do and 
they sorted the list by "Junior", "Senior", or somesuch. I started at 
the bottom of the list and worked my way up.


One useful thing was to make a sorted list of commands in /usr/bin, 
/bin, /usr/sbin, and /sbin, and then read the first bit of the man page 
that showed what the command did. Fun stuff.


Leam
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Leam Hall]

On 09/28/2017 07:35 AM, Stefan Ram wrote:


   But remember that paid programmers usually do not "code",
   in the sense of "write a program from scratch". Most of the
   work is maintenance programming, where an important part of
   the job is to read and understand a piece of code.
   Coding from scratch also happens, it just less common.

   (So that would be a reasonable interview test: Being able
   to understand a piece of given code and do some requested
   modification to it.)


Another Perl story. I used to love Perl and then got to the point where 
trying to code in it made me physically nauseous. Not sure why.


Guy had written a perl based time tracker for our contractor team. We'd 
enter tasks done and it would give a text based output to send to mgmt. 
Of course the guy hadn't planned on leaving after a few months and his 
program stored data by date but didn't separate by year. So I had to go 
figure out what it was doing since he was using a perl specific data 
archiver. Eventually just wound up blowing away the data store so each 
year was new. Told others how to handle it as I didn't want to do more 
perl and wasn't good enough at anything to replicate it all myself.


Leam
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Leam Hall]

My question has received several helpful responses, thanks!

On 09/28/2017 01:01 PM, Dennis Lee Bieber wrote:

On Wed, 27 Sep 2017 12:41:24 -0400, leam hall 
declaimed the following:
"Programmer"... or "Software Engineer"?

I haven't kept up on "job titles" but for my history, "programmer" is
an entry level position, just a few steps up from "data entry operator"
(aka "keypunch operator" -- to show my age)


"Person who automates routine tasks".

I used to get asked for MAC addresses. I was playing with TCL at the 
time and it had a built in webserver sort of thing. The boxes were 
Solaris. Made a cron job to run Explorer on the servers and another to 
collate them to a node with the TCL webserver. Gave the Network team the 
URL.


I'll show my age; 5 bit ASCII punched tape and actual ferrite core 
memory.   :P



As a "programmer" (in my archaic world): be fluent in the language and
core of the runtime (though perhaps not a master -- I still don't get
Python's decorators and meta-class concepts; my uses haven't needed them).
Be able to read language agnostic requirement/design documentation and
translate to the language in question. At this level, knowledge of the
problem domain is probably not needed. At the higher levels, the language
begins to be irrelevant, but more knowledge of the problem domain becomes
important -- the difference between designing/coding a web-based
store-front (HTTP/HTML, database, security) vs number-crunching image
streams from space probes...

Afraid I've likely just tossed it back to you -- what really is your
goal? 


As an introvert with a speech impediment I live by "Don't call me, I 
won't call you."


Well, okay, yes. I did to Toastmasters and can shine at an interview. 
Still, day to day I prefer to create solutions that solve problems and 
answer questions before they are asked so no one asks me. I know a 
little Networking, Database, Systems Engineering, Project Management, 
Security, large datacenter, and other cool buzzwords to easily find a 
job doing Linux system admin. What I want to move away from is doing now 
what I was doing 10-15 years ago.


A couple years ago I was back into C. A RHEL bug came up and management 
needed to understand the severity of the issue. I was able to read the 
reports, dig through the kernel code, and explain the issues and risks 
to MBA and PM types. I'm not about to represent myself as a C programmer 
but I can follow #include files.


One place brought on Unix people and your first day was split between 
the Eng team lead and the Ops team lead. They would decide which you 
were more suited for. The Eng team lead wrote Perl and asked me to 
explain some of their code. I did and also pointed out a bug. Seems I 
was a better fit for the Ops team.  :P


My short term goals are to use Python to get better at OOP coding and to 
automate in Python stuff that might work in shell/awk but are more fun 
in python. To that end I'm reading Booch, just ordered an old copy of 
the Python Cookbook, and am coding a game/fiction tool to help me keep 
track of characters.


It is often said to learn a language you grab the basics and then join a 
project. I'm happy to contribute to open source projects but the 
learning curve to "useful" has always been steep for me. There's gap 
between reading "Learning {language}" and contributing code.


Python is very useful because all my RHEL boxes have it installed. If I 
build a tool I know it will be able to run. While I enjoy Ruby more, 
it's not on the servers and it ain't going on the servers. I need to be 
useful to keep getting paid. Due to developer count the ability to 
instigate a python project is easier than a non-rails ruby project so I 
can build my "software engineering team" skills as well.


I appreciate your guidance and feedback; keep it coming!

Leam




--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Neil Cerutti]

On 2017-09-28, bartc  wrote:
> On 28/09/2017 12:31, Steve D'Aprano wrote:
>> Until now, I thought that people who wrote crappy code did so
>> because they didn't know any better. This is the first time
>> I've seen somebody state publicly that they have no interest
>> in writing clean code.
>
> I meant I have no interest in reading books about it or someone
> else's opinion. I have my own ideas of what is clean code and
> what isn't.

The world contains many programmers with more experience than
one's-self, and some of them are good at explaining what they
know in a comprehensible and entertaining way. I believe you will
benefit from and even enjoy some of the literature. Here's a
recent favorite: "The Pragmatic Programmer", Andrew Hunt and
David Thomas. ISBN-13: 978-0201616224

-- 
Neil Cerutti

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[bartc]

On 28/09/2017 12:31, Steve D'Aprano wrote:

On Thu, 28 Sep 2017 09:12 pm, bartc wrote:


And I have little interest in most of this lot (my eyes glaze over just
reading some of these):

  >  - how to use operating systems


You've never used a system call? Written to a file? Moved the mouse?


Wasn't that more this option:

 - how to program operating systems via system calls

Which was in my first group. Using an OS, I just do the minimum 
necessary and don't get involved in much else.


(In my first phase as a programmer, there were personnel whose job it 
was to do that. In the next phase, with microprocessors, there /was/ no 
operating system! Bliss. That phase didn't last long, but fortunately 
those OSes (MSDOS and the like) didn't do much so didn't get in the way 
either.)





  >  - how to use an editor well (e.g., vim or emacs)


You have no interest in using your editor well?


I use my own editor as much as possible. That doesn't have any elaborate 
features that it is necessary to 'learn'.





  >  - style (e.g. "Clean Code" by Robert Martin, pep 8, ...)


Until now, I thought that people who wrote crappy code did so because they
didn't know any better. This is the first time I've seen somebody state
publicly that they have no interest in writing clean code.


I meant I have no interest in reading books about it or someone else's 
opinion. I have my own ideas of what is clean code and what isn't.





  >  - test (e.g., unit test), TDD


You don't test your code?


I assume this meant formal methods of testing.


I suppose that makes it a lot easier to program. Just mash down on the keyboard
with both hands, and say that the code is done and working correctly, and move
on to the next project.

*wink*


Actually I used to like using random methods (Monte Carlo) to solve 
problems. That doesn't scale well however, at some point you have to 
properly think through a solution.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[alister via Python-list]

On Wed, 27 Sep 2017 18:18:10 -0700, Larry Hudson wrote:

> On 09/27/2017 09:41 AM, leam hall wrote:
>> On Sat, Sep 23, 2017 at 5:26 PM, Ned Batchelder 
>> wrote:
> [snip]
>> 
>> The question is, what should a person "know" when hiring out as a
>> programmer? What is 'know" and what should be "known"? Specifically
>> with Python.
>> 
>> 
> Hopefully NOT like this person...
> (Source:  http://rinkworks.com/stupid/cs_misc.shtml There is no direct
> link to this item, it's about 2/3 the way down in a long web page...)
> 
> 
> Since I teach nights at a local community college, I get a lot of
> professional programmers in my classes upgrading their education. One
> student, who was one such person, attended every lecture and smiled and
> nodded and took notes. But he only turned in his first assignment. The
> results of his first test were horrid. Out of curiosity, I asked my
> wife, who barely knew how to turn a computer on much less program one,
> to take the test (which was mostly true/false and multiple choice
> questions). My wife scored higher than this guy.
> 
> The semester's end came, and he flubbed his final, too. A few weeks
> later, I got a call from him complaining about his 'F'. I pointed out he
> hadn't turned in any of his assignments, and those counted 75% of the
> grade.
> 
> "Did you hear me say something besides what the other students heard?" I
> asked.
> 
> "Well, I thought my test grades would carry me," he replied.
> 
> It had turned out his company had paid for him to take the course. Since
> he failed, it suddenly came to the attention of his employer that he
> didn't know how to program, and now his job was in jeopardy. As I hung
> up the phone, I mused that his company shouldn't fire him. It was a
> perfect match: a programmer who couldn't program and a company that
> couldn't figure out sooner that he couldn't.
> 

the whole page seems to be full of 
"look how dumb this user is because they do no automatically know things 
that I had to learn"



-- 
  After they got rid of capital punishment, they had to hang twice
  as many people as before.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Steve D'Aprano]

On Thu, 28 Sep 2017 09:12 pm, bartc wrote:

> And I have little interest in most of this lot (my eyes glaze over just
> reading some of these):
> 
>  >  - how to use operating systems

You've never used a system call? Written to a file? Moved the mouse?


>  >  - how to use an editor well (e.g., vim or emacs)

You have no interest in using your editor well?


>  >  - style (e.g. "Clean Code" by Robert Martin, pep 8, ...)

Until now, I thought that people who wrote crappy code did so because they
didn't know any better. This is the first time I've seen somebody state
publicly that they have no interest in writing clean code.


>  >  - test (e.g., unit test), TDD

You don't test your code?

I suppose that makes it a lot easier to program. Just mash down on the keyboard
with both hands, and say that the code is done and working correctly, and move
on to the next project.

*wink*



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[bartc]

On 28/09/2017 03:33, Stefan Ram wrote:

Larry Hudson  writes:

Hopefully NOT like this person...

Since I teach nights at a local community college



a programmer who couldn't program


   It is not clear what »this person« refers to:

   Do you hope one is not like that teacher who
   publicly is shaming one of his students, though
   without actually giving the name of the student.

   Or do you hope one is not like that student who
   did not turn in the assignments?

   The fact that programmers can't program is
   known since the invention of the "FizzBuzz"
   programmer test. But in the case of the student,
   one actually can't know for sure whether he only
   had problems with the /upgrade/ of his education,
   but still can program in his everyday job.

   So, what was the question? Quoted from earlier in
   the same thread:

|The question is, what should a person "know" when hiring out
|as a programmer? What is 'know" and what should be "known"?
|Specifically with Python.

   Some areas of knowledge follow, a programmer should not be
   ignorant in all of them:


I can probably manage the following, even if I hate some of it or some 
might be advanced:



 - writing FizzBuzz
 - writing GUI software [My own]


(Writing a GUI system or using one? I've done both and try and avoid GUI 
completely if possible.)



 - writing software to analyze text files
 - writing software to generate images from data
 - writing software to analyze images
 - how to program operating systems via system calls
 - algorithms and datastructures
 - numerical mathematics
 - how to write a recursive descent parser
 - maths (how to transform to polar coordinates or
   what the use of a fourier transformation is)


And I have little interest in most of this lot (my eyes glaze over just 
reading some of these):


>  - writing GUI software [Other people's]
>  - writing software to analyze data bases
>  - writing user interfaces for data bases
>  - how to use operating systems
>  - how to administer a computer
>  - how to use the command languages of operating systems
>  - how to use an editor well (e.g., vim or emacs)
>  - how to use UN*X tools (grep, uniq, sed, ...)
>  - regular expressions
>  - a source management tool (like git)
>  - design patterns
>  - design by contract
>  - OOA/OOD
>  - the most important libraries for Python (standard and other)
>  - data base design / normalization
>  - style (e.g. "Clean Code" by Robert Martin, pep 8, ...)
>  - refactors
>  - software engineering
>  - being able to read and write EBNF
>  - software-project managemet (e.g. Agile, "Scrum")
>  - computer science (complexity, NP, grammars, ...)
>  - test (e.g., unit test), TDD
>  - programming interviews (there are many books about this topic!)
>  - Using a real Newsreader (not Google Groups)
>  - common algorithms/heuristics for global optimization
>  - common types of statistical analyses and neural networks

It seems the first group is more pure coding (and fun, a lot of it), and 
the second is the usual lot of tools and technologies that programmers 
seems to have to know about these days (and not so much fun).


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Paul Moore]

On 27 September 2017 at 17:41, leam hall  wrote:
> Hehe...I've been trying to figure out how to phrase a question. Knowing I'm
> not the only one who gets frustrated really helps.
>
> I'm trying to learn to be a programmer. I can look at a book and read basic
> code in a few languages but it would be unfair to hire myself out as a
> programmer. I'm just not yet worth what it costs to pay my bills.

You're already ahead of the game in wanting to be useful, rather than
just knowing the jargon :-) However, I've always found that the
biggest asset a programmer can have is the simple willingness to
learn. "Programming" is far too broad a subject for anyone to know all
about it, so being able (and willing!) to find things out, to look for
and follow good practices, and to keep learning, is far more important
than knowing the specifics of how to code a class definition.

Most programmers work in teams, so you will likely be working with an
existing code base for reference (even if you're not doing actual
maintenance coding), so you'll have examples to work from anyway.

> To move forward takes a plan and time bound goals. At least for us old
> folks; we only have so much time left. I want to avoid retirement and just
> work well until I keel over.
>
> I don't come from a CS background but as a Linux sysadmin. My current push
> is OOP. Grady Booch's book on Analysis and Design is great and I've got the
> GoF for right after that. I've been doing more testing but need to write
> more tests. Writing code and starting to work with others on that code as
> well.

I haven't read Booch, but I've heard good things about it. The GoF is
good, but a lot of the problem's it's addressing aren't really issues
in Python. So be prepared to find that the solutions look a bit
over-engineered from a Python perspective. The ideas are really
useful, though.

Keep in mind that in Python, OOP is just one option of many - it's a
very useful approach for many problems, but it's not as all-embracing
as people with a Java or C# background imply. In particular, Python
uses a lot less subclassing than those languages (because duck typing
is often more flexible).

> The question is, what should a person "know" when hiring out as a
> programmer? What is 'know" and what should be "known"? Specifically with
> Python.

With Python, I'd say that an appreciation of the available libraries
is key - both what's in the stdlib, and what's available from PyPI.
That's not to say you should memorise the standard library, but rather
cultivate an approach of "hmm, I'm pretty sure I remember there being
a library for that" and going to look. The best way of getting this is
to actually work with code - you can start with doing coding projects
of your own (it's *always* a good exercise to have a problem that
interests you, and work on coding it - no matter what it is, you'll
learn more about understanding requirements, testing, bug fixing, and
practical programming by working on a project you care about than
you'll ever get reading books) and/or you can look at existing open
source projects that you're interested in, and offer help (there's
always a bug tracker, and typically some simpler items - and you'll
learn a lot from interacting with a larger project).

Hope this helps,
Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Dan Sommers]

On Wed, 27 Sep 2017 12:41:24 -0400, leam hall wrote:

> The question is, what should a person "know" when hiring out as a
> programmer? What is 'know" and what should be "known"? Specifically
> with Python.

The longer I claim to be a programmer, the more I discover how wide a
net that is.  Web sites, embedded systems, user interfaces (text and
graphical), databases, communications protocols, applications
programming, systems programming, real time systems, text processing,
scientific computing, simulations, security, etc., etc., etc.  And I
stopped there only because I hope I've made my point and not because
that's an exhaustive list.  Some of it you'll know up front; it's a
pretty boring job on which you learn nothing new.

What must be known is how to produce a program that does what the
customer says they want (note that they likely don't know what they
need, only what they want, but that's a whole other ball of wax).
You'll also have to know enough about the problem domain to converse
with the customer to turn what will be a vague request into something
tangible.  I'm sure you already do this when it comes to automating your
own tasks.

If I'm hiring myself out as a plumber, I should know how to unclog
drains; and install, repair, replace toilets, water heaters, and other
plumbing fixtures (or whatever else a plumber might be called on to do).
Ignore the question of licensing; it doesn't apply to programmers.

It's the same whether you use Python, something else, or some
combination.

Wow, that's a lot more than I intended to write.  I don't mean to be
discouraging, only enlightening.  We all started somewhere, and your
background as a sysadmin puts you way ahead of a lot of future
programmers.

HTH,
Dan

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Chris Angelico]

On Thu, Sep 28, 2017 at 11:18 AM, Larry Hudson via Python-list
 wrote:
> 
> It had turned out his company had paid for him to take the course. Since he
> failed, it suddenly came to the attention of his employer that he didn't
> know how to program, and now his job was in jeopardy. As I hung up the
> phone, I mused that his company shouldn't fire him. It was a perfect match:
> a programmer who couldn't program and a company that couldn't figure out
> sooner that he couldn't.
> 

I had a coworker like that at my previous job. The boss basically was
paying him to learn to code, and yet (for reasons which to this day I
cannot fathom) let him make a lot of decisions about technology. Thus
we used PHP and MySQL (okay, it could have been worse), with a
multi-threaded daemon in addition to the actual web server (I take
that back, it WAS worse). I had to fight uphill to convince my boss of
the value of git ("why bother, I have daily backups for a week and
then weekly backups for two years"), and even then, this coworker
didn't commit or push until, well, pushed. Eventually he quit the
company (rumour has it he was hoping we'd beg him to stay, since we
were that short-handed), and I had to take over his code... and found
it full of The Daily WTF level abominations. Heard of the "For-Case
paradigm"? Check out this [1] old article if you're not familiar with
it. Well, he gave me this thrilling variant (reconstructed from
memory):

$i=1;
while ($i<3) {
switch($i)
{case 1:
... snip one set of validations
$i=3;
break;
case 3:
... some more validation work
$i=2;
break;
case 2:
... snip another set of validations
$i=4;
break;
}}

I don't remember the exact details, but it was something like this. It
looked like the code just stepped straight through the switch block.
So I stripped out the junk and just did the validations sequentially.

And the code stopped working.

Since I *had* been committing to git frequently, I checked out the
previous version. It worked. I redid the simplification. It broke
again. I stuck in some console output, and found that one of the
blocks of code was actually getting skipped... and due to the
compounding of two or three other bugs, valid input would get rejected
by the validation that wasn't happening.

I have no idea whether he intentionally removed part of the
validation, or if he just never noticed that it wasn't running. It was
a truly impressive piece of work.

ChrisA

[1] https://thedailywtf.com/articles/The_FOR-CASE_paradigm
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Larry Hudson via Python-list]

On 09/27/2017 09:41 AM, leam hall wrote:

On Sat, Sep 23, 2017 at 5:26 PM, Ned Batchelder 
wrote:

[snip]


The question is, what should a person "know" when hiring out as a
programmer? What is 'know" and what should be "known"? Specifically with
Python.



Hopefully NOT like this person...
(Source:  http://rinkworks.com/stupid/cs_misc.shtml
There is no direct link to this item, it's about 2/3 the way down in a long web 
page...)


Since I teach nights at a local community college, I get a lot of professional programmers in my 
classes upgrading their education. One student, who was one such person, attended every lecture 
and smiled and nodded and took notes. But he only turned in his first assignment. The results of 
his first test were horrid. Out of curiosity, I asked my wife, who barely knew how to turn a 
computer on much less program one, to take the test (which was mostly true/false and multiple 
choice questions). My wife scored higher than this guy.


The semester's end came, and he flubbed his final, too. A few weeks later, I got a call from him 
complaining about his 'F'. I pointed out he hadn't turned in any of his assignments, and those 
counted 75% of the grade.


"Did you hear me say something besides what the other students heard?" I asked.

"Well, I thought my test grades would carry me," he replied.

It had turned out his company had paid for him to take the course. Since he failed, it suddenly 
came to the attention of his employer that he didn't know how to program, and now his job was in 
jeopardy. As I hung up the phone, I mused that his company shouldn't fire him. It was a perfect 
match: a programmer who couldn't program and a company that couldn't figure out sooner that he 
couldn't.



--
 -=- Larry -=-
--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Larry Martell]

On Wed, Sep 27, 2017 at 12:41 PM, leam hall  wrote:
> The question is, what should a person "know" when hiring out as a
> programmer? What is 'know" and what should be "known"? Specifically with
> Python.

Fake it till you make it!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[leam hall]

On Sat, Sep 23, 2017 at 5:26 PM, Ned Batchelder <n...@nedbatchelder.com>
wrote:

> On 9/23/17 2:52 PM, Leam Hall wrote:
>
>> On 09/23/2017 02:40 PM, Terry Reedy wrote:
>>
>>> https://nedbatchelder.com//blog/201709/beginners_and_experts.html
>>>
>>> Great post.
>>>
>>
>> Yup. Thanks for the link. I often have that "I bet > Fred> doesn't get frustrated." thing going. Nice to know Ned bangs his head
>> now and again.  :P
>>
>>
> "Ow!" --me


Hehe...I've been trying to figure out how to phrase a question. Knowing I'm
not the only one who gets frustrated really helps.

I'm trying to learn to be a programmer. I can look at a book and read basic
code in a few languages but it would be unfair to hire myself out as a
programmer. I'm just not yet worth what it costs to pay my bills.

To move forward takes a plan and time bound goals. At least for us old
folks; we only have so much time left. I want to avoid retirement and just
work well until I keel over.

I don't come from a CS background but as a Linux sysadmin. My current push
is OOP. Grady Booch's book on Analysis and Design is great and I've got the
GoF for right after that. I've been doing more testing but need to write
more tests. Writing code and starting to work with others on that code as
well.

The question is, what should a person "know" when hiring out as a
programmer? What is 'know" and what should be "known"? Specifically with
Python.

Thanks!

Leam
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Ned Batchelder]

On 9/23/17 2:52 PM, Leam Hall wrote:

On 09/23/2017 02:40 PM, Terry Reedy wrote:

https://nedbatchelder.com//blog/201709/beginners_and_experts.html

Great post.


Yup. Thanks for the link. I often have that "I bet Fred> doesn't get frustrated." thing going. Nice to know Ned bangs his 
head now and again.  :P




"Ow!" --me


--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Terry Reedy]

On 9/23/2017 2:52 PM, Leam Hall wrote:

On 09/23/2017 02:40 PM, Terry Reedy wrote:

https://nedbatchelder.com//blog/201709/beginners_and_experts.html

Great post.


Yup. Thanks for the link. I often have that "I bet Fred> doesn't get frustrated." thing going. Nice to know Ned bangs his 
head now and again.  :P


As do I ;-).


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Beginners and experts (Batchelder blog post)

[Leam Hall]

On 09/23/2017 02:40 PM, Terry Reedy wrote:

https://nedbatchelder.com//blog/201709/beginners_and_experts.html

Great post.


Yup. Thanks for the link. I often have that "I bet Fred> doesn't get frustrated." thing going. Nice to know Ned bangs his 
head now and again.  :P


Leam


--
https://mail.python.org/mailman/listinfo/python-list

Beginners and experts (Batchelder blog post)

[Terry Reedy]

https://nedbatchelder.com//blog/201709/beginners_and_experts.html

Great post.
--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Planet Scipy blog

[Mark Lawrence]

On 08/10/2015 14:39, beliavsky--- via Python-list wrote:

There used to be a blog about SciPy at https://planet.scipy.org/ , discussing the 
applications of Python to scientific computing. Now there is a static page about 
"MPI for Python". What happened?



Presumably http://www.scipy.org/

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Planet Scipy blog

[Laura Creighton]

In a message of Thu, 08 Oct 2015 15:10:21 +0100, Mark Lawrence writes:
>On 08/10/2015 14:39, beliavsky--- via Python-list wrote:
>> There used to be a blog about SciPy at https://planet.scipy.org/ , 
>> discussing the applications of Python to scientific computing. Now there is 
>> a static page about "MPI for Python". What happened?
>>
>
>Presumably http://www.scipy.org/
>
>-- 
>My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence

No, all the blog links there go to the MPI for Python link, too.

Laura

-- 
https://mail.python.org/mailman/listinfo/python-list

Planet Scipy blog

[beliavsky--- via Python-list]

There used to be a blog about SciPy at https://planet.scipy.org/ , discussing 
the applications of Python to scientific computing. Now there is a static page 
about "MPI for Python". What happened?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Planet Scipy blog

[Oscar Benjamin]

On Thu, 8 Oct 2015 15:29 Laura Creighton <l...@openend.se> wrote:

In a message of Thu, 08 Oct 2015 15:10:21 +0100, Mark Lawrence writes:
>On 08/10/2015 14:39, beliavsky--- via Python-list wrote:
>> There used to be a blog about SciPy at https <https://planet.scipy.org/>
:// <https://planet.scipy.org/>planet.scipy.org/ <https://planet.scipy.org/>
, discussing the applications of Python to scientific computing. Now there
is a static page about "MPI for Python". What happened?
>>
>
>Presumably http:// <http://www.scipy.org/>www.scipy.org/
<http://www.scipy.org/>
>
>--
>My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence

No, all the blog links there go to the MPI for Python link, too.



Numpy etc have been having some hosting problems recently. This might be a
temporary glitch.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

EuroPython has a new blog

[M.-A. Lemburg]

The EuroPython Society has setup a new blog for EuroPython in its
efforts to provide more conference facilities for the EuroPython
organization and to enhance the EuroPython attendee experience.

http://blog.europython.eu/

There’s an RSS feed in case you want to subscribe to it:

http://blog.europython.eu/rss

The blog itself is hosted on Tumblr, so you can also follow the blog
using your Tumblr account: log in, visit the blog and click “Follow in
the upper right corner:

http://www.tumblr.com/follow/europython

If you’re looking for older blog entries, please check the
EuroPython 2013 blog, which lists the entries for 2011-2013:

https://ep2013.europython.eu/blog/

Enjoy,
-- 
Marc-Andre Lemburg
Director
EuroPython Society
http://www.europython-society.org/
-- 
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/

EuroPython has a new blog

[M.-A. Lemburg]

The EuroPython Society has setup a new blog for EuroPython in its
efforts to provide more conference facilities for the EuroPython
organization and to enhance the EuroPython attendee experience.

http://blog.europython.eu/

There’s an RSS feed in case you want to subscribe to it:

http://blog.europython.eu/rss

The blog itself is hosted on Tumblr, so you can also follow the blog
using your Tumblr account: log in, visit the blog and click “Follow in
the upper right corner:

http://www.tumblr.com/follow/europython

If you’re looking for older blog entries, please check the
EuroPython 2013 blog, which lists the entries for 2011-2013:

https://ep2013.europython.eu/blog/

Enjoy,
-- 
Marc-Andre Lemburg
Director
EuroPython Society
http://www.europython-society.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: EuroPython has a new blog

[Stéphane Wirtel]

Hi Marc-André,

Cool for EuroPython,

Good idea, I think we will use tumblr for a small blog for Python-FOSDEM.

Stef



On 17 Jan 2014, at 13:37, M.-A. Lemburg wrote:

 The EuroPython Society has setup a new blog for EuroPython in its
 efforts to provide more conference facilities for the EuroPython
 organization and to enhance the EuroPython attendee experience.

   http://blog.europython.eu/

 There’s an RSS feed in case you want to subscribe to it:

   http://blog.europython.eu/rss

 The blog itself is hosted on Tumblr, so you can also follow the blog
 using your Tumblr account: log in, visit the blog and click “Follow in
 the upper right corner:

   http://www.tumblr.com/follow/europython

 If you’re looking for older blog entries, please check the
 EuroPython 2013 blog, which lists the entries for 2011-2013:

   https://ep2013.europython.eu/blog/

 Enjoy,
 --
 Marc-Andre Lemburg
 Director
 EuroPython Society
 http://www.europython-society.org/
 --
 https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[wxjmfauth]

Le mercredi 8 janvier 2014 01:02:22 UTC+1, Terry Reedy a écrit :
 On 1/7/2014 9:54 AM, Terry Reedy wrote:
 
  On 1/7/2014 8:34 AM, wxjmfa...@gmail.com wrote:
 
  Le dimanche 5 janvier 2014 23:14:07 UTC+1, Terry Reedy a écrit :
 
 
 
  Memory: Point 2. A *design goal* of FSR was to save memory relative  to
 
  UTF-32, which is what you apparently prefer. Your examples show that FSF
 
  successfully met its design goal. But you call that success, saving
 
  memory, 'wrong'. On what basis?
 
 
 
  Point 2: This Flexible String Representation does no
 
  effectuate any memory optimization. It only succeeds
 
  to do the opposite of what a corrrect usage of utf*
 
  do.
 
 
 
  Since the FSF *was* successful in saving memory, and indeed shrank the
 
  Python binary by about a megabyte, I have no idea what you mean.
 
 
 
 Tim Delaney apparently did, and answered on the basis of his 
 
 understanding. Note that I said that the design goal was 'save memory 
 
 RELATIVE TO UTF-32', not 'optimize memory'. UTF-8 was not considered an 
 
 option. Nor was any form of arithmetic coding
 
 https://en.wikipedia.org/wiki/Arithmetic_coding
 
 to truly 'optimize memory'.
 
 


The FSR acts more as an coding scheme selector than
as a code point optimizer.

Claiming that it saves memory is some kind of illusion;
a little bit as saying Py2.7 uses relatively less memory than
Py3.2 (UCS-2).

 sys.getsizeof('a' * 1 + 'z')
10026
 sys.getsizeof('a' * 1 + '€')
20040
 sys.getsizeof('a' * 1 + '\U0001')
40044
 sys.getsizeof('€' * 1 + '€')
20040
 sys.getsizeof('€' * 1 + '\U0001')
40044
 sys.getsizeof('\U0001' * 1 + '\U0001')
40044

jmf

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Terry Reedy]

On 1/8/2014 4:59 AM, wxjmfa...@gmail.com wrote:
[responding to me]

The FSR acts more as an coding scheme selector


That is what PEP 393 describes and what I and many others have said. The 
FSR saves memory by selecting from three choices the most compact coding 
scheme for each string.


I ask again, have you read PEP 393? If you are going to critique the 
FSR, you should read its basic document.


 than as a code point optimizer.

I do not know what you mean by 'code point optimizer'.


Claiming that it saves memory is some kind of illusion;


Do you really think that the mathematical fact 10026  20040  40044 
(from your example below) is some kind of illusion? If so, please take 
your claim to a metaphysics list. If not, please stop trolling.



a little bit as saying Py2.7 uses relatively less memory than
Py3.2 (UCS-2).


This is inane as 2.7 and 3.2 both use the same two coding schemes. 
Saying '1  2' is different from saying '2  2'.


On 3.3+

sys.getsizeof('a' * 1 + 'z')

10026

sys.getsizeof('a' * 1 + '€')

20040

sys.getsizeof('a' * 1 + '\U0001')

40044


3.2- wide (UCS-4) builds use about 40050 bytes for all three unicode 
strings. One again, you have posted examples that show how FSR saves 
memory, thus negating your denial of the saving.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Mark Lawrence]

On 07/01/2014 13:34, wxjmfa...@gmail.com wrote:

Le dimanche 5 janvier 2014 23:14:07 UTC+1, Terry Reedy a écrit :

Ned : this has already been explained and illustrated.

jmf



This has never been explained and illustrated.  Roughly 30 minutes ago 
Terry Reedy once again completely shot your argument about memory usage 
to pieces.  You did not bother to respond to the comments from Tim 
Delaney made almost one day ago.  Please give up.


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[wxjmfauth]

Le dimanche 5 janvier 2014 23:14:07 UTC+1, Terry Reedy a écrit :
 On 1/5/2014 9:23 AM, wxjmfa...@gmail.com wrote:
 
  Le samedi 4 janvier 2014 23:46:49 UTC+1, Terry Reedy a écrit :
 
  On 1/4/2014 2:10 PM, wxjmfa...@gmail.com wrote:
 
  And I could add, I *never* saw once one soul, who is
 
  explaining what I'm doing wrong in the gazillion
 
  of examples I gave on this list.
 
 
 
  If this is true, it is because you have ignored and not read my
 
  numerous, relatively polite posts. To repeat very briefly:
 
  1. Cherry picking (presenting the most extreme case as representative).
 
  2. Calling space saving a problem (repeatedly).
 
   3. Ignoring bug fixes.
 
 ...
 
 
 
  My examples are ONLY ILLUSTRATING, this FSR
 
  is wrong by design, can be on the side of
 
  memory, performance, linguistic or even
 
  typography.
 
 
 
 Let me expand on 3 of my points. First, performance == time:
 
 
 
 Point 3. You correctly identified a time regression in finding a 
 
 character in a string. I saw that the slowdown was *not* inherent in the 
 
 FSR but had to be a glitch in the code, and reported it on pydev with 
 
 the hope that someone would fix it even if it were not too important in 
 
 real use cases. Someone did.
 
 
 
 Point 1. You incorrectly generalized that extreme case. I reported (a 
 
 year ago last September) that the overall stringbench results were about 
 
 the same. I also pointed out that there is an equally non-representative 
 
 extreme case in the opposite direction, and that it would equally be 
 
 wrong of me to use that to claim that FSR is faster. (It turns out that 
 
 this FSR speed advantage *is* inherent in the design.)
 
 
 
 Memory: Point 2. A *design goal* of FSR was to save memory relative  to 
 
 UTF-32, which is what you apparently prefer. Your examples show that FSF 
 
 successfully met its design goal. But you call that success, saving 
 
 memory, 'wrong'. On what basis?
 
 
 
 You *claim* the FSR is 'wrong by design', but your examples only show 
 
 that is was temporarily wrong in implementation as far as speed and 
 
 correct by design as far as memory goes.
 
 

Point 3: You are right. I'm very happy to agree.

Point 2: This Flexible String Representation does no
effectuate any memory optimization. It only succeeds
to do the opposite of what a corrrect usage of utf*
do.

Ned : this has already been explained and illustrated.

jmf

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Terry Reedy]

On 1/7/2014 8:34 AM, wxjmfa...@gmail.com wrote:

Le dimanche 5 janvier 2014 23:14:07 UTC+1, Terry Reedy a écrit :



Memory: Point 2. A *design goal* of FSR was to save memory relative  to
UTF-32, which is what you apparently prefer. Your examples show that FSF
successfully met its design goal. But you call that success, saving
memory, 'wrong'. On what basis?



Point 2: This Flexible String Representation does no
effectuate any memory optimization. It only succeeds
to do the opposite of what a corrrect usage of utf*
do.


Since the FSF *was* successful in saving memory, and indeed shrank the 
Python binary by about a megabyte, I have no idea what you mean.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Tim Delaney]

On 8 January 2014 00:34, wxjmfa...@gmail.com wrote:


 Point 2: This Flexible String Representation does no
 effectuate any memory optimization. It only succeeds
 to do the opposite of what a corrrect usage of utf*
 do.


UTF-8 is a variable-width encoding that uses less memory to encode code
points with lower numerical values, on a per-character basis e.g. if a code
point = U+007F it will use a single byte to encode; if = U+07FF two bytes
will be used; ... up to a maximum of 6 bytes for code points = U+400.

FSR is a variable-width memory structure that uses the width of the code
point with the highest numerical value in the string e.g. if all code
points in the string are = U+00FF a single byte will be used per
character; if all code points are = U+ two bytes will be used per
character; and in all other cases 4 bytes will be used per character.

In terms of memory usage the difference is that UTF-8 varies its width
per-character, whereas the FSR varies its width per-string. For any
particular string, UTF-8 may well result in using less memory than the FSR,
but in other (quite common) cases the FSR will use less memory than UTF-8
e.g. if the string contains only contains code points = U+00FF, but some
are between U+0080 and U+00FF (inclusive).

In most cases the FSR uses the same or less memory than earlier versions of
Python 3 and correctly handles all code points (just like UTF-8). In the
cases where the FSR uses more memory than previously, the previous
behaviour was incorrect.

No matter which representation is used, there will be a certain amount of
overhead (which is the majority of what most of your examples have shown).
Here are examples which demonstrate cases where UTF-8 uses less memory,
cases where the FSR uses less memory, and cases where they use the same
amount of memory (accounting for the minimum amount of overhead required
for each).

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64
bit (AMD64)] on win32
Type help, copyright, credits or license for more information.
 import sys

 fsr = u
 utf8 = fsr.encode(utf-8)
 min_fsr_overhead = sys.getsizeof(fsr)
 min_utf8_overhead = sys.getsizeof(utf8)
 min_fsr_overhead
49
 min_utf8_overhead
33

 fsr = u\u0001 * 1000
 utf8 = fsr.encode(utf-8)
 sys.getsizeof(fsr) - min_fsr_overhead
1000
 sys.getsizeof(utf8) - min_utf8_overhead
1000

 fsr = u\u0081 * 1000
 utf8 = fsr.encode(utf-8)
 sys.getsizeof(fsr) - min_fsr_overhead
1024
 sys.getsizeof(utf8) - min_utf8_overhead
2000

 fsr = u\u0001\u0081 * 1000
 utf8 = fsr.encode(utf-8)
 sys.getsizeof(fsr) - min_fsr_overhead
2024
 sys.getsizeof(utf8) - min_utf8_overhead
3000

 fsr = u\u0101 * 1000
 utf8 = fsr.encode(utf-8)
 sys.getsizeof(fsr) - min_fsr_overhead
2025
 sys.getsizeof(utf8) - min_utf8_overhead
2000

 fsr = u\u0101\u0081 * 1000
 utf8 = fsr.encode(utf-8)
 sys.getsizeof(fsr) - min_fsr_overhead
4025
 sys.getsizeof(utf8) - min_utf8_overhead
4000

Indexing a character in UTF-8 is O(N) - you have to traverse the the string
up to the character being indexed. Indexing a character in the FSR is O(1).
In all cases the FSR has better performance characteristics for indexing
and slicing than UTF-8.

There are tradeoffs with both UTF-8 and the FSR. The Python developers
decided the priorities for Unicode handling in Python were:

1. Correctness
  a. all code points must be handled correctly;
  b.  it must not be possible to obtain part of a code point (e.g. the
first byte only of a multi-byte code point);

2. No change in the Big O characteristics of string operations e.g.
indexing must remain O(1);

3. Reduced memory use in most cases.

It is impossible for UTF-8 to meet both criteria 1b and 2 without
additional auxiliary data (which uses more memory and increases complexity
of the implementation). The FSR meets all 3 criteria.

Tim Delaney
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Terry Reedy]

On 1/7/2014 9:54 AM, Terry Reedy wrote:

On 1/7/2014 8:34 AM, wxjmfa...@gmail.com wrote:

Le dimanche 5 janvier 2014 23:14:07 UTC+1, Terry Reedy a écrit :



Memory: Point 2. A *design goal* of FSR was to save memory relative  to
UTF-32, which is what you apparently prefer. Your examples show that FSF
successfully met its design goal. But you call that success, saving
memory, 'wrong'. On what basis?



Point 2: This Flexible String Representation does no
effectuate any memory optimization. It only succeeds
to do the opposite of what a corrrect usage of utf*
do.


Since the FSF *was* successful in saving memory, and indeed shrank the
Python binary by about a megabyte, I have no idea what you mean.


Tim Delaney apparently did, and answered on the basis of his 
understanding. Note that I said that the design goal was 'save memory 
RELATIVE TO UTF-32', not 'optimize memory'. UTF-8 was not considered an 
option. Nor was any form of arithmetic coding

https://en.wikipedia.org/wiki/Arithmetic_coding
to truly 'optimize memory'.

--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Devin Jeanpierre]

On Sat, Jan 4, 2014 at 6:27 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 Fast is never more important than correct. It's just that sometimes you
 might compromise a little (or a lot) on what counts as correct in order for
 some speed.

Is this statement even falsifiable? Can you conceive of a circumstance
where someone has traded correctness for speed, but where one couldn't
describe it that latter way? I can't. I think by definition you can
always describe it that way, you just make what counts as
correctness be what the customer wants given the resources
available. The conventional definition, however, is what the
customer wants, imagining that you have infinite resources. With just
a little redefinition that seems reasonable, you can be made never to
be wrong!


I avoid making unfalsifiable arguments that aren't explicitly labeled
as such. I try to reword them as, I prefer to look at it as ... --
it's less aggressive, which means people are more likely to really
listen to what you have to say. It also doesn't pretend to be an
argument when it isn't.

-- Devin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[wxjmfauth]

Le dimanche 5 janvier 2014 03:54:29 UTC+1, Chris Angelico a écrit :
 On Sun, Jan 5, 2014 at 1:41 PM, Steven D'Aprano
 
 steve+comp.lang.pyt...@pearwood.info wrote:
 
  wxjmfa...@gmail.com wrote:
 
 
 
  The very interesting aspect in the way you are holding
 
  unicodes (strings). By comparing Python 2 with Python 3.3,
 
  you are comparing utf-8 with the the internal representation
 
  of Python 3.3 (the flexible string represenation).
 
 
 
  This is incorrect. Python 2 has never used UTF-8 internally for Unicode
 
  strings. In narrow builds, it uses UTF-16, but makes no allowance for
 
  surrogate pairs in strings. In wide builds, it uses UTF-32.
 
 
 
 That's for Python's unicode type. What Robin said was that they were
 
 using either a byte string (str) with UTF-8 data, or a Unicode
 
 string (unicode) with character data. So jmf was right, except that
 
 it's not specifically to do with Py2 vs Py3.3.
 
 

Yes, the key point is the preparation of the unicode
text for the PDF producer.

This is at this level the different flavours of Python
may be relevant.

I see four possibilites, I do not know what
the PDF producer API is expecting.

- Py2 with utf-8 byte string (ev. utf-16, utf-32)
- Py2 with its internal unicode
- Py3.2 with its internal unicode
- Py3.3 with its internal unicode

jmf
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Johannes Bauer]

On 31.12.2013 10:53, Steven D'Aprano wrote:
 Mark Lawrence wrote:
 
 http://blog.startifact.com/posts/alex-gaynor-on-python-3.html.
 
 I quote:
 
 ...perhaps a brave group of volunteers will stand up and fork Python 2, and
 take the incremental steps forward. This will have to remain just an idle
 suggestion, as I'm not volunteering myself.
 
 I expect that as excuses for not migrating get fewer, and the deadline for
 Python 2.7 end-of-life starts to loom closer, more and more haters^W
 Concerned People will whine about the lack of version 2.8 and ask for
 *somebody else* to fork Python.
 
 I find it, hmmm, interesting, that so many of these Concerned People who say
 that they're worried about splitting the Python community[1] end up
 suggesting that we *split the community* into those who have moved forward
 to Python 3 and those who won't.

Exactly. I don't know what exactly their problem is. I've pushed the
migration of *large* projects at work to Python3 when support was pretty
early and it really wasn't a huge deal.

Specifically because I love pretty much every single aspect that Python3
introduced. The codec support is so good that I've never seen anything
like it in any other programming language and then there's the tons of
beautiful changes (div/intdiv, functools.lru_cache, print(),
datetime.timedelta.total_seconds(), int.bit_length(), bytes/bytearray).

Regards,
Joe

-- 
 Wo hattest Du das Beben nochmal GENAU vorhergesagt?
 Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa hidbv3$om2$1...@speranza.aioe.org
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Steven D'Aprano]

Devin Jeanpierre wrote:

 On Sat, Jan 4, 2014 at 6:27 PM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
 Fast is never more important than correct. It's just that sometimes you
 might compromise a little (or a lot) on what counts as correct in order
 for some speed.
 
 Is this statement even falsifiable? Can you conceive of a circumstance
 where someone has traded correctness for speed, but where one couldn't
 describe it that latter way? I can't. 

Every time some programmer optimises a piece of code (or, more often,
*thinks* they have optimised it) which introduces bugs into the software,
that's a case where somebody has traded correctness for speed where my
statement doesn't apply. Sometimes the response to the subsequent bug
report is will not fix, and a retroactive change in the software
requirements. (Oh, did we say that indexing a string would return a
character? We meant it would return a character, so long as the string only
includes no Unicode characters in the astral planes.) Sometimes it is to
revert the optimisation or otherwise fix the bug.

I accept that there is sometimes a fine line here. I'm assuming that
software applications have their requirements fully documented, which in
the real world is hardly ever the case. Although, even if the requirements
aren't always written down, often they are implicitly understood. (Although
it gets very interesting when the users' understanding and the developers'
understanding is different.)

Take as an example this torture test for a mathematical sum function,
where the built-in sum() gets the wrong answer but math.fsum() gets it
right:

py from math import fsum
py values = [1e12, 0.0001, -1e12, 0.0001]*1
py fsum(values)
2.0
py sum(values)
2.4413841796875


Here's another example of the same thing, just to prove it's not a fluke:

py values = [1e17, 1, 1, -1e17]
py fsum(values)
2.0
py sum(values)
0.0


The reason for the different results is that fsum() tries hard to account
for intermediate rounding errors and sum() does not. If you benchmark the
two functions, you'll find that sum() is significantly faster than fsum. So
the question to be asked is, does sum() promise to calculate floating point
sums accurately? If so, then this is a bug, probably introduced by the
desire for speed. But in fact, sum() does not promise to calculate floating
point sums accurately. What it promises to do is to calculate the
equivalent of a + b + c + ... for as many values as given, and that's
exactly what it does. Conveniently, that's faster than fsum(), and usually
accurate enough for most uses.

Is sum() buggy? No, of course not. It does what it promises, it's just that
what it promises to do falls short of calculate floating point summations
to high accuracy.

Now, here's something which *would* be a bug, if sum() did it:

class MyInt(int):
def __add__(self, other):
return MyInt(super(MyInt, self).__add__(other))
def __radd__(self, other):
return MyInt(super(MyInt, self).__radd__(other))
def __repr__(self):
return MyInt(%d) % self


Adding a zero MyInt to an int gives a MyInt:

py MyInt(0) + 23
MyInt(23)

so sum() should do the same thing. If it didn't, if it optimised away the
actual addition because adding zero to a number can't change anything, it
would be buggy. But in fact, sum() does the right thing:

py sum([MyInt(0), 23])
MyInt(23)


 I think by definition you can 
 always describe it that way, you just make what counts as
 correctness be what the customer wants given the resources
 available. 

Not quite. Correct means does what the customer wants. Or if there is no
customer, it's does what you say it will do.

How do we tell when software is buggy? We compare what it actually does to
the promised behaviour, or expected behaviour, and if there is a
discrepancy, we call it a bug. We don't compare it to some ideal that
cannot be met. A bug report that math.pi does not have infinite number of
decimal places would be closed as Will Not Fix.

Likewise, if your customer pays you to solve the Travelling Salesman Problem
exactly, even if it takes a week to calculate, then nothing short of a
program that solves the Travelling Salesman Problem exactly will satisfy
their requirements. It's no good telling the customer that you can
calculate a non-optimal answer twenty times faster if they want the actual
optimal answer.

(Of course, you may try to persuade them that they don't really need the
optimal solution, or that they cannot afford it, or that you cannot deliver
and they need to compromise.)


 The conventional definition, however, is what the 
 customer wants, imagining that you have infinite resources.

I don't think the resources really come into it. At least, certainly not
*infinite* resources. fsum() doesn't require infinite resources to
calculate floating point summations to high accuracy. An even more accurate
(but even slower) version would convert each float into a Fraction, then
add the Fractions.


 

Re: Blog about python 3

[Chris Angelico]

On Sun, Jan 5, 2014 at 11:28 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 - The Unix 'locate' command doesn't do a live search of the file
   system because that would be too slow, it uses a snapshot of the
   state of the file system.


 Is locate buggy because it tells you what files existed the last time the
 updatedb command ran, instead of what files exist right now? No, of course
 not. locate does exactly what it promises to do.

Even more strongly: We say colloquially that Google, DuckDuckGo, etc,
etc, are tools for searching the web. But they're not. They're tools
for *indexing* the World Wide Web, and then searching that index. It's
plausible to actually search your file system (and there are times
when you want that), but completely implausible to search the (F or
otherwise) web. We accept the delayed appearance of a page in the
search results because we want immediate results, no waiting a month
to find anything! So the difference between what's technically
promised and what's colloquially described may be more than just
concealing bugs - it may be the vital difference between uselessness
and usefulness. And yet we like the handwave.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Mark Lawrence]

On 31/12/2013 09:53, Steven D'Aprano wrote:

Mark Lawrence wrote:


http://blog.startifact.com/posts/alex-gaynor-on-python-3.html.


I quote:

...perhaps a brave group of volunteers will stand up and fork Python 2, and
take the incremental steps forward. This will have to remain just an idle
suggestion, as I'm not volunteering myself.

I expect that as excuses for not migrating get fewer, and the deadline for
Python 2.7 end-of-life starts to loom closer, more and more haters^W
Concerned People will whine about the lack of version 2.8 and ask for
*somebody else* to fork Python.



Should the somebody else fork Python, in ten (ish) years time the 
Concerned People will be complaining that they can't port their code to 
Python 4 and will somebody else please produce version 2.9.


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Stefan Behnel]

Johannes Bauer, 05.01.2014 13:14:
 I've pushed the
 migration of *large* projects at work to Python3 when support was pretty
 early and it really wasn't a huge deal.

I think there are two sides to consider. Those who can switch their code
base to Py3 and be happy (as you did, apparently), and those who cannot
make the switch but have to keep supporting Py2 until 'everyone' else has
switched, too. The latter is a bit more work generally and applies mostly
to Python packages on PyPI, i.e. application dependencies.

There are two ways to approach that problem. One is to try convincing
people that Py3 has failed, let's stop migrating more code before I have
to start migrating mine, and the other is to say let's finish the
migration and get it done, so that we can finally drop Py2 support in our
new releases and clean up our code again.

As long as we stick in the middle and keep the status quo, we keep the
worst of both worlds. And, IMHO, pushing loudly for a Py2.8 release
provides a very good excuse for others to not finish their part of the
migration, thus prolonging the maintenance burden for those who already did
their share.

Maybe a couple of major projects should start dropping their Py2 support,
just to make their own life easier and to help others in taking their
decision, too.

(And that's me saying that, who maintains two major projects that still
have legacy support for Py2.4 ...)

Stefan


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[wxjmfauth]

Le samedi 4 janvier 2014 23:46:49 UTC+1, Terry Reedy a écrit :
 On 1/4/2014 2:10 PM, wxjmfa...@gmail.com wrote:
 
  Le samedi 4 janvier 2014 15:17:40 UTC+1, Chris Angelico a écrit :
 
 
 
  any, and Python has only one, idiot like jmf who completely
 
 
 
 Chris, I appreciate the many contributions you make to this list, but 
 
 that does not exempt you from out standard of conduct.
 
 
 
  misunderstands what's going on and uses microbenchmarks to prove
 
  obscure points... and then uses nonsense to try to prove... uhh...
 
 
 
 Troll baiting is a form of trolling. I think you are intelligent enough 
 
 to know this.  Please stop.
 
 
 
  I do not mind to be considered as an idiot, but
 
  I'm definitively not blind.
 
 
 
  And I could add, I *never* saw once one soul, who is
 
  explaining what I'm doing wrong in the gazillion
 
  of examples I gave on this list.
 
 
 
 If this is true, it is because you have ignored and not read my 
 
 numerous, relatively polite posts. To repeat very briefly:
 
 
 
 1. Cherry picking (presenting the most extreme case as representative).
 
 
 
 2. Calling space saving a problem (repeatedly).
 
 
 
 3. Ignoring bug fixes.
 
 
 
 4. Repetition (of the 'gazillion example' without new content).
 
 
 
 Have you ever acknowledged, let alone thank people for, the fix for the 
 
 one bad regression you did find. The FSR is still a work in progress. 
 
 Just today, Serhiy pushed a patch speeding up the UTF-32 encoder, after 
 
 previously speeding up the UTF-32 decoder.
 
 
 
 -- 

My examples are ONLY ILLUSTRATING, this FSR
is wrong by design, can be on the side of
memory, performance, linguistic or even
typography.

I will not refrain you to waste your time
in adjusting bytes, if the problem is not
on that side.

jmf

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Ned Batchelder]

On 1/5/14 9:23 AM, wxjmfa...@gmail.com wrote:

Le samedi 4 janvier 2014 23:46:49 UTC+1, Terry Reedy a écrit :

On 1/4/2014 2:10 PM, wxjmfa...@gmail.com wrote:

I do not mind to be considered as an idiot, but
I'm definitively not blind.

And I could add, I *never* saw once one soul, who is
explaining what I'm doing wrong in the gazillion
of examples I gave on this list.


If this is true, it is because you have ignored and not read my
numerous, relatively polite posts. To repeat very briefly:

1. Cherry picking (presenting the most extreme case as representative).

2. Calling space saving a problem (repeatedly).

3. Ignoring bug fixes.

4. Repetition (of the 'gazillion example' without new content).

Have you ever acknowledged, let alone thank people for, the fix for the
one bad regression you did find. The FSR is still a work in progress.
Just today, Serhiy pushed a patch speeding up the UTF-32 encoder, after
previously speeding up the UTF-32 decoder.

--


My examples are ONLY ILLUSTRATING, this FSR
is wrong by design, can be on the side of
memory, performance, linguistic or even
typography.


JMF: this has been pointed out to you time and again: the flexible 
string representation is not wrong.  To show that it is wrong, you would 
have to demonstrate some semantic of Unicode that is violated.  You have 
never done this.  You've picked pathological cases and shown 
micro-timing output, and memory usage.  The Unicode standard doesn't 
promise anything about timing or memory use.


The FSR makes a trade-off of time and space.  Everyone but you considers 
it a good trade-off.  I don't think you are showing real use cases, but 
if they are, I'm sorry that your use-case suffers.  That doesn't make 
the FSR wrong. The most accurate statement is that you don't like the 
FSR.  That's fine, you're entitled to your opinion.


You say the FSR is wrong linguistically.  This can't be true, since an 
FSR Unicode string is indistinguishable from an internally-UTF-32 
Unicode string, and no, memory use or timings are irrelevant when 
discussing the linguistic performance of a Unicode string.


You've also said that the internal representation of the FSR is 
incorrect because of encodings somehow.  Encodings have nothing to do 
with the internal representation of a Unicode string, they are for 
interchanging data.  You seem to know a lot about Unicode, but when you 
make this fundamental mistake, you call all of your expertise into question.


To re-iterate what you are doing wrong:

1) You continue to claim things that are not true, and that you have 
never substantiated.


2) You paste code samples without accompanying text that explain what 
you are trying to demonstrate.


3) You ignore refutations that disprove your points.

These are all the behaviors of a troll.  Please stop.

If you want to discuss the details of Unicode implementations, I'd 
welcome an offlist discussion, but only if you will approach it honestly 
enough to leave open the possibility that you are wrong.  I know I would 
be glad to learn details of Unicode that I have missed, but so far you 
haven't provided any.


--Ned.



I will not refrain you to waste your time
in adjusting bytes, if the problem is not
on that side.

jmf




--
Ned Batchelder, http://nedbatchelder.com

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Roy Smith]

In article 52c94fec$0$29973$c3e8da3$54964...@news.astraweb.com,
 Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:

 How do we tell when software is buggy? We compare what it actually does to
 the promised behaviour, or expected behaviour, and if there is a
 discrepancy, we call it a bug. We don't compare it to some ideal that
 cannot be met. A bug report that math.pi does not have infinite number of
 decimal places would be closed as Will Not Fix.

That's because it is inherently impossible to fix that.  But lots of 
bug reports legitimately get closed with Will Not Fix simply because 
the added value from fixing it doesn't justify the cost (whether in 
terms of development effort, or run-time resource consumption).

Go back to the package sorting example I gave.  If the sorting software 
mis-reads the address and sends my package to Newark instead of New York 
by mistake, that's clearly a bug.

Presumably, it's an error which could be eliminated (or, at least, the 
rate of occurrence reduced) by using a more sophisticated OCR algorithm.  
But, if those algorithms take longer to run, the overall expected value 
of implementing the bug fix software may well be negative.

In the real world, nobody cares if software is buggy.  They care that it 
provides value.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Roy Smith]

In article mailman.4930.1388908293.18130.python-l...@python.org,
 Chris Angelico ros...@gmail.com wrote:

 On Sun, Jan 5, 2014 at 2:20 PM, Roy Smith r...@panix.com wrote:
  I've got a new sorting algorithm which is guaranteed to cut 10 seconds
  off the sorting time (i.e. $0.10 per package).  The problem is, it makes
  a mistake 1% of the time.
 
 That's a valid line of argument in big business, these days, because
 we've been conditioned to accept low quality. But there are places
 where quality trumps all, and we're happy to pay for that. Allow me to
 expound two examples.
 
 1) Amazon
 
 http://www.amazon.com/exec/obidos/ASIN/1782010165/evertype-20
 
 I bought this book a while ago. It's about the size of a typical
 paperback. It arrived in a box too large for it on every dimension,
 with absolutely no packaging. I complained. Clearly their algorithm
 was: Most stuff will get there in good enough shape, so people can't
 be bothered complaining. And when they do complain, it's cheaper to
 ship them another for free than to debate with them on chat.

You're missing my point.

Amazon's (short-term) goal is to increase their market share by 
undercutting everybody on price.  They have implemented a box-packing 
algorithm which clearly has a bug in it.  You are complaining that they 
failed to deliver your purchase in good condition, and apparently don't 
care.  You're right, they don't.  The cost to them to manually correct 
this situation exceeds the value.  This is one shipment.  It doesn't 
matter.  You are one customer, you don't matter either.  Seriously.  
This may be annoying to you, but it's good business for Amazon.  For 
them, fast and cheap is absolutely better than correct.

I'm not saying this is always the case.  Clearly, there are companies 
which have been very successful at producing a premium product (Apple, 
for example).  I'm not saying that fast is always better than correct.  
I'm just saying that correct is not always better than fast.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Chris Angelico]

On Mon, Jan 6, 2014 at 3:34 AM, Roy Smith r...@panix.com wrote:
 Amazon's (short-term) goal is to increase their market share by
 undercutting everybody on price.  They have implemented a box-packing
 algorithm which clearly has a bug in it.  You are complaining that they
 failed to deliver your purchase in good condition, and apparently don't
 care.  You're right, they don't.  The cost to them to manually correct
 this situation exceeds the value.  This is one shipment.  It doesn't
 matter.

If it stopped there, it would be mildly annoying (1% of our shipments
will need to be replaced, that's a 1% cost for free replacements).
The trouble is that they don't care about the replacement either, so
it's really that 100% (or some fairly large proportion) of their
shipments will arrive with some measure of damage, and they're hoping
that their customers' threshold for complaining is often higher than
the damage sustained. Which it probably is, a lot of the time.

 You are one customer, you don't matter either.  Seriously.
 This may be annoying to you, but it's good business for Amazon.  For
 them, fast and cheap is absolutely better than correct.

But this is the real problem, business-wise. Can you really run a
business by not caring about your customers? (I also think it's pretty
disappointing that a business like Amazon can't just toss in some
bubbles, or packing peanuts (what we call trucks for hysterical
raisins), or something. It's not that hard to have a machine just blow
in some sealed air before the box gets closed... surely?) Do they have
that much of a monopoly, or that solid a customer base, that they're
happy to leave *everyone* dissatisfied? We're not talking about 1%
here. From the way the cust svc guy was talking, I get the impression
that they do this with all parcels.

And yet I can't disagree with your final conclusion. Empirical
evidence goes against my incredulous declaration that surely this is
a bad idea - according to XKCD 1165, they're kicking out nearly a
cubic meter a *SECOND* of packages. That's fairly good evidence that
they're doing something that, whether it be right or wrong, does fit
with the world's economy. Sigh.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Roy Smith]

Chris Angelico ros...@gmail.com wrote:

 Can you really run a business by not caring about your customers?

http://snltranscripts.jt.org/76/76aphonecompany.phtml
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Terry Reedy]

On 1/5/2014 9:23 AM, wxjmfa...@gmail.com wrote:

Le samedi 4 janvier 2014 23:46:49 UTC+1, Terry Reedy a écrit :

On 1/4/2014 2:10 PM, wxjmfa...@gmail.com wrote:

And I could add, I *never* saw once one soul, who is
explaining what I'm doing wrong in the gazillion
of examples I gave on this list.



If this is true, it is because you have ignored and not read my
numerous, relatively polite posts. To repeat very briefly:
1. Cherry picking (presenting the most extreme case as representative).
2. Calling space saving a problem (repeatedly).

 3. Ignoring bug fixes.
...


My examples are ONLY ILLUSTRATING, this FSR
is wrong by design, can be on the side of
memory, performance, linguistic or even
typography.


Let me expand on 3 of my points. First, performance == time:

Point 3. You correctly identified a time regression in finding a 
character in a string. I saw that the slowdown was *not* inherent in the 
FSR but had to be a glitch in the code, and reported it on pydev with 
the hope that someone would fix it even if it were not too important in 
real use cases. Someone did.


Point 1. You incorrectly generalized that extreme case. I reported (a 
year ago last September) that the overall stringbench results were about 
the same. I also pointed out that there is an equally non-representative 
extreme case in the opposite direction, and that it would equally be 
wrong of me to use that to claim that FSR is faster. (It turns out that 
this FSR speed advantage *is* inherent in the design.)


Memory: Point 2. A *design goal* of FSR was to save memory relative  to 
UTF-32, which is what you apparently prefer. Your examples show that FSF 
successfully met its design goal. But you call that success, saving 
memory, 'wrong'. On what basis?


You *claim* the FSR is 'wrong by design', but your examples only show 
that is was temporarily wrong in implementation as far as speed and 
correct by design as far as memory goes.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Terry Reedy]

On 1/5/2014 9:23 AM, wxjmfa...@gmail.com wrote:


My examples are ONLY ILLUSTRATING, this FSR
is wrong by design,


Let me answer you a different way. If FSR is 'wrong by design', so are 
the alternatives. Hence, the claim is, in itself, useless as a guide to 
choosing. The choices:


* Keep the previous complicated system of buggy narrow builds on some 
systems and space-wasting wide builds on other systems, with Python code 
potentially acting differently on the different builds. I am sure that 
you agree that this is a bad design.


* Improved the dual-build system by de-bugging narrow builds. I proposed 
to do this (and gave Python code proving the idea) by adding the 
complication of an auxiliary array of indexes of astral chars in a 
UTF-16 string. I suspect you would call this design 'wrong' also.


* Use the memory-wasting UTF-32 (wide) build on all systems. I know you 
do not consider this 'wrong', but come on. From an information theoretic 
and coding viewpoint, it clearly is. The top (4th) byte is *never* used. 
The 3rd byte is *almost never* used. The 2nd byte usage ranges from 
common to almost never for different users.


Memory waste is also time waste, as moving information-free 0 bytes 
takes the same time as moving informative bytes.


Here is the beginning of the rationale for the FSR (from 
http://www.python.org/dev/peps/pep-0393/ -- have you ever read it?).


There are two classes of complaints about the current implementation of 
the unicode type: on systems only supporting UTF-16, users complain that 
non-BMP characters are not properly supported. On systems using UCS-4 
internally (and also sometimes on systems using UCS-2), there is a 
complaint that Unicode strings take up too much memory - especially 
compared to Python 2.x, where the same code would often use ASCII 
strings


The memory waste was a reason to stick with 2.7. It could break code 
that worked in 2.x. By removing the waste, the FSR makes switching to 
Python 3 more feasible for some people. It was a response to real 
problems encountered by real people using Python. It fixed both classes 
of complaint about the previous system.


* Switch to the time-wasting UTF-8 for text storage, as some have done. 
This is different from using UTF-8 for text transmission, which I hope 
becomes the norm soon.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Terry Reedy]

On 1/5/2014 11:51 AM, Chris Angelico wrote:

On Mon, Jan 6, 2014 at 3:34 AM, Roy Smith r...@panix.com wrote:

Amazon's (short-term) goal is to increase their market share by
undercutting everybody on price.  They have implemented a box-packing
algorithm which clearly has a bug in it.  You are complaining that they
failed to deliver your purchase in good condition, and apparently don't
care.  You're right, they don't.  The cost to them to manually correct
this situation exceeds the value.  This is one shipment.  It doesn't
matter.


If it stopped there, it would be mildly annoying (1% of our shipments
will need to be replaced, that's a 1% cost for free replacements).
The trouble is that they don't care about the replacement either, so
it's really that 100% (or some fairly large proportion) of their
shipments will arrive with some measure of damage, and they're hoping
that their customers' threshold for complaining is often higher than
the damage sustained. Which it probably is, a lot of the time.


My wife has gotten several books from Amazon and partners and we have 
never gotten one loose enough in a big enough box to be damaged. Either 
the box is tight or has bubble packing. Leaving aside partners, maybe 
distribution centers have different rules.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Chris Angelico]

On Mon, Jan 6, 2014 at 9:56 AM, Terry Reedy tjre...@udel.edu wrote:
 On 1/5/2014 11:51 AM, Chris Angelico wrote:

 On Mon, Jan 6, 2014 at 3:34 AM, Roy Smith r...@panix.com wrote:

 Amazon's (short-term) goal is to increase their market share by
 undercutting everybody on price.  They have implemented a box-packing
 algorithm which clearly has a bug in it.  You are complaining that they
 failed to deliver your purchase in good condition, and apparently don't
 care.  You're right, they don't.  The cost to them to manually correct
 this situation exceeds the value.  This is one shipment.  It doesn't
 matter.


 If it stopped there, it would be mildly annoying (1% of our shipments
 will need to be replaced, that's a 1% cost for free replacements).
 The trouble is that they don't care about the replacement either, so
 it's really that 100% (or some fairly large proportion) of their
 shipments will arrive with some measure of damage, and they're hoping
 that their customers' threshold for complaining is often higher than
 the damage sustained. Which it probably is, a lot of the time.


 My wife has gotten several books from Amazon and partners and we have never
 gotten one loose enough in a big enough box to be damaged. Either the box is
 tight or has bubble packing. Leaving aside partners, maybe distribution
 centers have different rules.

Or possibly (my personal theory) the CS rep I was talking to just
couldn't be bothered solving the problem. Way way too much work to
make the customer happy, much easier and cheaper to give a 30% refund
and hope that shuts him up.

But they managed to ship two books (the original and the replacement)
with insufficient packaging. Firstly, that requires the square of the
probability of failure; and secondly, if you care even a little bit
about making your customers happy, put a little note on the second
order instructing people to be particularly careful of this one! Get
someone to check it before it's sent out. Make sure it's right this
time. I know that's what we used to do in the family business whenever
anything got mucked up.

(BTW, I had separately confirmed that the problem was with Amazon, and
not - as has happened to me with other shipments - caused by
Australian customs officials opening the box, looking through it, and
then packing it back in without its protection. No, it was shipped
that way.)

Anyway, this is veering so far off topic that we're at no risk of
meeting any Python Alliance ships - as Mal said, we're at the corner
of No and Where. But maybe someone can find an on-topic analogy to put
some tentative link back into this thread...

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Steven D'Aprano]

Chris Angelico wrote about Amazon:

 And yet I can't disagree with your final conclusion. Empirical
 evidence goes against my incredulous declaration that surely this is
 a bad idea - according to XKCD 1165, they're kicking out nearly a
 cubic meter a SECOND of packages.

Yes, but judging by what you described as their packing algorithm that's
probably only a tenth of a cubic metre of *books*, the rest being empty box
for the book to rattle around in and get damaged.

-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Steven D'Aprano]

Roy Smith wrote:

 In article mailman.4930.1388908293.18130.python-l...@python.org,
  Chris Angelico ros...@gmail.com wrote:
 
 On Sun, Jan 5, 2014 at 2:20 PM, Roy Smith r...@panix.com wrote:
  I've got a new sorting algorithm which is guaranteed to cut 10 seconds
  off the sorting time (i.e. $0.10 per package).  The problem is, it
  makes a mistake 1% of the time.
 
 That's a valid line of argument in big business, these days, because
 we've been conditioned to accept low quality. But there are places
 where quality trumps all, and we're happy to pay for that. Allow me to
 expound two examples.
 
 1) Amazon
 
 http://www.amazon.com/exec/obidos/ASIN/1782010165/evertype-20
 
 I bought this book a while ago. It's about the size of a typical
 paperback. It arrived in a box too large for it on every dimension,
 with absolutely no packaging. I complained. Clearly their algorithm
 was: Most stuff will get there in good enough shape, so people can't
 be bothered complaining. And when they do complain, it's cheaper to
 ship them another for free than to debate with them on chat.
 
 You're missing my point.
 
 Amazon's (short-term) goal is to increase their market share by
 undercutting everybody on price.  They have implemented a box-packing
 algorithm which clearly has a bug in it.  You are complaining that they
 failed to deliver your purchase in good condition, and apparently don't
 care.  You're right, they don't.  The cost to them to manually correct
 this situation exceeds the value.  This is one shipment.  It doesn't
 matter.  You are one customer, you don't matter either.  Seriously.
 This may be annoying to you, but it's good business for Amazon.  For
 them, fast and cheap is absolutely better than correct.

One, you're missing my point that to Amazon, fast and cheap *is* correct.
They would not agree with you that their box-packing algorithm is buggy, so
long as their customers don't punish them for it. It meets their
requirements: ship parcels as quickly as possible, and push as many of the
costs (damaged books) onto the customer as they can get away with. If they
thought it was buggy, they would be trying to fix it.

Two, nobody is arguing against the concept that different parties have
different concepts of what's correct. To JMF, the flexible string
representation is buggy, because he's detected a trivially small slowdown
in some artificial benchmarks. To everyone else, it is not buggy, because
it does what it sets out to do: save memory while still complying with the
Unicode standard. A small slowdown on certain operations is a cost worth
paying.

Normally, the definition of correct that matters is that belonging to the
paying customer, or failing that, the programmer who is giving his labour
away for free. (Extend this out to more stakeholders if you wish, but the
more stakeholders you include, the harder it is to get consensus on what's
correct and what isn't.) From the perspective of Amazon's customers,
presumably so long as the cost of damaged and lost books isn't too high,
they too are willing to accept Amazon's definition of correct in order to
get cheap books, or else they would buy from someone else.

(However, to the extent that Amazon has gained monopoly power over the book
market, that reasoning may not apply. Amazon is not *technically* a
monopoly, but they are clearly well on the way to becoming one, at which
point the customer has no effective choice and the market is no longer
free.)

The Amazon example is an interesting example of market failure, in the sense
that the free market provides a *suboptimal solution* to a problem. We'd
all like reasonably-priced books AND reliable delivery, but maybe we can't
have both. Personally, I'm not so sure about that. Maybe Jeff Bezos could
make do with only five solid gold Mercedes instead of ten[1], for the sake
of improved delivery? But apparently not.

But I digress... ultimately, you are trying to argue that there is a single
absolute source of truth for what counts as correct. I don't believe
there is. We can agree that some things are clearly not correct -- Amazon
takes your money and sets the book on fire, or hires an armed military
escort costing $20 million a day to deliver your book of funny cat
pictures. We might even agree on what we'd all like in a perfect world:
cheap books, reliable delivery, and a pony. But in practice we have to
choose some features over others, and compromise on requirements, and
ultimately we have to make a *pragmatic* choice on what counts as correct
based on the functional requirements, not on a wishlist of things we'd like
with infinite time and money.

Sticking to the Amazon example, what percentage of books damaged in delivery
ceases to be a bug in the packing algorithm and becomes just one of those
things? One in ten? One in ten thousand? One in a hundred billion billion?
I do not accept that book gets damaged in transit counts as a bug. More
than x% of books get damaged, that's a bug. Average cost to 

Re: Blog about python 3

[Chris Angelico]

On Mon, Jan 6, 2014 at 12:23 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 (However, to the extent that Amazon has gained monopoly power over the book
 market, that reasoning may not apply. Amazon is not *technically* a
 monopoly, but they are clearly well on the way to becoming one, at which
 point the customer has no effective choice and the market is no longer
 free.)

They don't need a monopoly on the whole book market, just on specific
books - which they did have, in the cited case. I actually asked the
author (translator, really - it's a translation of Alice in
Wonderland) how he would prefer me to buy, as there are some who sell
on Amazon and somewhere else. There was no alternative to Amazon, ergo
no choice and the market was not free. Like so many things, one choice
(I want to buy Ailice's Anters in Ferlielann) mandates another
(Must buy through Amazon).

I don't know what it cost Amazon to ship me two copies of a book, but
still probably less than they got out of me, so they're still ahead.
Even if they lost money on this particular deal, they're still way
ahead because of all the people who decide it's not worth their time
to spend an hour or so trying to get a replacement. So yep, this
policy is serving Amazon fairly well.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Mark Lawrence]

On 06/01/2014 01:54, Chris Angelico wrote:

On Mon, Jan 6, 2014 at 12:23 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:

(However, to the extent that Amazon has gained monopoly power over the book
market, that reasoning may not apply. Amazon is not *technically* a
monopoly, but they are clearly well on the way to becoming one, at which
point the customer has no effective choice and the market is no longer
free.)


They don't need a monopoly on the whole book market, just on specific
books - which they did have, in the cited case. I actually asked the
author (translator, really - it's a translation of Alice in
Wonderland) how he would prefer me to buy, as there are some who sell
on Amazon and somewhere else. There was no alternative to Amazon, ergo
no choice and the market was not free. Like so many things, one choice
(I want to buy Ailice's Anters in Ferlielann) mandates another
(Must buy through Amazon).

I don't know what it cost Amazon to ship me two copies of a book, but
still probably less than they got out of me, so they're still ahead.
Even if they lost money on this particular deal, they're still way
ahead because of all the people who decide it's not worth their time
to spend an hour or so trying to get a replacement. So yep, this
policy is serving Amazon fairly well.

ChrisA



So much for my You never know, we might even end up with a thread 
whereby the discussion is Python, the whole Python and nothing but the 
Python. :)


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[wxjmfauth]

Le vendredi 3 janvier 2014 12:14:41 UTC+1, Robin Becker a écrit :
 On 02/01/2014 18:37, Terry Reedy wrote:
 
  On 1/2/2014 12:36 PM, Robin Becker wrote:
 
 
 
  I just spent a large amount of effort porting reportlab to a version
 
  which works with both python2.7 and python3.3. I have a large number of
 
  functions etc which handle the conversions that differ between the two
 
  pythons.
 
 
 
  I am imagine that this was not fun.
 
 
 
 indeed :)
 
 
 
  For fairly sensible reasons we changed the internal default to use
 
  unicode rather than bytes.
 
 
 
  Do you mean 'from __future__ import unicode_literals'?
 
 
 
 No, previously we had default of utf8 encoded strings in the lower levels of 
 the 
 
 code and we accepted either unicode or utf8 string literals as inputs to text 
 
 functions. As part of the port process we made the decision to change from 
 
 default utf8 str (bytes) to default unicode.
 
 
 
  Am I correct in thinking that this change increases the capabilities of
 
  reportlab? For instance, easily producing an article with abstracts in 
  English,
 
  Arabic, Russian, and Chinese?
 
 
 
 It's made no real difference to what we are able to produce or accept since 
 utf8 
 
 or unicode can encode anything in the input and what can be produced depends 
 on 
 
 fonts mainly.
 
 
 
After doing all that and making the tests
 
 ...
 
  I know some of these tests are fairly variable, but even for simple
 
  things like paragraph parsing 3.3 seems to be slower. Since both use
 
  unicode internally it can't be that can it, or is python 2.7's unicode
 
  faster?
 
 
 
  The new unicode implementation in 3.3 is faster for some operations and 
  slower
 
  for others. It is definitely more space efficient, especially compared to a 
  wide
 
  build system. It is definitely less buggy, especially compared to a narrow 
  build
 
  system.
 
 
 
  Do your tests use any astral (non-BMP) chars? If so, do they pass on narrow 
  2.7
 
  builds (like on Windows)?
 
 
 
 I'm not sure if we have any non-bmp characters in the tests. Simple CJK etc 
 etc 
 
 for the most part. I'm fairly certain we don't have any ability to handle 
 
 composed glyphs (multi-codepoint) etc etc
 
 
 
 
 
 
 
 
 
  For one thing, indexing and slicing just works on all machines for all 
  unicode
 
  strings. Code for 2.7 and 3.3 either a) does not index or slice, b) does not
 
  work for all text on 2.7 narrow builds, or c) has extra conditional code 
  only
 
  for 2.7.



To Robin Becker

I know nothing about ReportLab except its existence.
Your story is very interesting. As I pointed, I know
nothing about the internal of ReportLab, the technical
aspects: the Python part, the used api for the PDF creation).
I have however some experience with the unicode TeX engine,
XeTeX, understand I'm understanding a little bit what's
happening behind the scene.

The very interesting aspect in the way you are holding
unicodes (strings). By comparing Python 2 with Python 3.3,
you are comparing utf-8 with the the internal representation
of Python 3.3 (the flexible string represenation).
In one sense, more than comparing Py2 with Py3.

It will be much more interesting to compare utf-8/Python
internals at the light of Python 3.2 and Python 3.3. Python
3.2 has a decent unicode handling, Python 3.3 has an absurd
(in mathematical sense) unicode handling. This is really
shining with utf-8, where this flexible string representation
is just doing the opposite of what a correct unicode
implementation does!

On the memory side, it is obvious to see it.

 sys.getsizeof('a'*1 + 'z')
10026
 sys.getsizeof('a'*1 + '€')
20040
 sys.getsizeof(('a'*1 + 'z').encode('utf-8'))
10018
 sys.getsizeof(('a'*1 + '€').encode('utf-8'))
10020

On the performance side, it is much more complexe,
but qualitatively, you may expect the same results.


The funny aspect is that by working with utf-8 in that
case, you are (or one has) forcing Python to work
properly, but one pays on the side of the performance.
And if one wishes to save memory, one has to pay on the
side of performance.

In othe words, attempting to do what Python is
not able to do natively is just impossible!


I'm skipping the very interesting composed glyphs subject
(unicode normalization, ...),  but I wish to point that
with the flexible string representation, one reaches
the top level of surrealism. For a tool which is supposed
to handle these very specific unicode tasks...

jmf
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Roy Smith]

In article mailman.4882.1388808283.18130.python-l...@python.org,
 Mark Lawrence breamore...@yahoo.co.uk wrote:

 Surely everybody prefers fast but incorrect code in 
 preference to something that is correct but slow?

I realize I'm taking this statement out of context, but yes, sometimes 
fast is more important than correct.  Sometimes the other way around.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Chris Angelico]

On Sun, Jan 5, 2014 at 12:55 AM, Roy Smith r...@panix.com wrote:
 In article mailman.4882.1388808283.18130.python-l...@python.org,
  Mark Lawrence breamore...@yahoo.co.uk wrote:

 Surely everybody prefers fast but incorrect code in
 preference to something that is correct but slow?

 I realize I'm taking this statement out of context, but yes, sometimes
 fast is more important than correct.  Sometimes the other way around.

More usually, it's sometimes better to be really fast and mostly
correct than really really slow and entirely correct. That's why we
use IEEE floating point instead of Decimal most of the time. Though
I'm glad that Python 3 now deems the default int type to be capable of
representing arbitrary integers (instead of dropping out to a separate
long type as Py2 did), I think it's possibly worth optimizing small
integers to machine words - but mainly, the int type focuses on
correctness above performance, because the cost is low compared to the
benefit. With float, the cost of arbitrary precision is extremely
high, and the benefit much lower.

With Unicode, the cost of perfect support is normally seen to be a
doubling of internal memory usage (UTF-16 vs UCS-4). Pike and Python
decided that the cost could, instead, be a tiny measure of complexity
and actually *less* memory usage (compared to UTF-16, when lots of
identifiers are ASCII). It's a system that works only when strings are
immutable, but works beautifully there. Fortunately Pike doesn't have
any, and Python has only one, idiot like jmf who completely
misunderstands what's going on and uses microbenchmarks to prove
obscure points... and then uses nonsense to try to prove... uhh...
actually I'm not even sure what, sometimes. I wouldn't dare try to
read his posts except that my mind's already in a rather broken state,
as a combination of programming and Alice in Wonderland.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Ned Batchelder]

On 1/4/14 9:17 AM, Chris Angelico wrote:

On Sun, Jan 5, 2014 at 12:55 AM, Roy Smith r...@panix.com wrote:

In article mailman.4882.1388808283.18130.python-l...@python.org,
  Mark Lawrence breamore...@yahoo.co.uk wrote:


Surely everybody prefers fast but incorrect code in
preference to something that is correct but slow?


I realize I'm taking this statement out of context, but yes, sometimes
fast is more important than correct.  Sometimes the other way around.


More usually, it's sometimes better to be really fast and mostly
correct than really really slow and entirely correct. That's why we
use IEEE floating point instead of Decimal most of the time. Though
I'm glad that Python 3 now deems the default int type to be capable of
representing arbitrary integers (instead of dropping out to a separate
long type as Py2 did), I think it's possibly worth optimizing small
integers to machine words - but mainly, the int type focuses on
correctness above performance, because the cost is low compared to the
benefit. With float, the cost of arbitrary precision is extremely
high, and the benefit much lower.

With Unicode, the cost of perfect support is normally seen to be a
doubling of internal memory usage (UTF-16 vs UCS-4). Pike and Python
decided that the cost could, instead, be a tiny measure of complexity
and actually *less* memory usage (compared to UTF-16, when lots of
identifiers are ASCII). It's a system that works only when strings are
immutable, but works beautifully there. Fortunately Pike doesn't have
any, and Python has only one, idiot like jmf who completely
misunderstands what's going on and uses microbenchmarks to prove
obscure points... and then uses nonsense to try to prove... uhh...
actually I'm not even sure what, sometimes. I wouldn't dare try to
read his posts except that my mind's already in a rather broken state,
as a combination of programming and Alice in Wonderland.

ChrisA



I really wish we could discuss these things without baiting trolls.

--
Ned Batchelder, http://nedbatchelder.com

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[wxjmfauth]

Le samedi 4 janvier 2014 15:17:40 UTC+1, Chris Angelico a écrit :
 On Sun, Jan 5, 2014 at 12:55 AM, Roy Smith r...@panix.com wrote:
 
  In article mailman.4882.1388808283.18130.python-l...@python.org,
 
   Mark Lawrence breamore...@yahoo.co.uk wrote:
 
 
 
  Surely everybody prefers fast but incorrect code in
 
  preference to something that is correct but slow?
 
 
 
  I realize I'm taking this statement out of context, but yes, sometimes
 
  fast is more important than correct.  Sometimes the other way around.
 
 
 
 More usually, it's sometimes better to be really fast and mostly
 
 correct than really really slow and entirely correct. That's why we
 
 use IEEE floating point instead of Decimal most of the time. Though
 
 I'm glad that Python 3 now deems the default int type to be capable of
 
 representing arbitrary integers (instead of dropping out to a separate
 
 long type as Py2 did), I think it's possibly worth optimizing small
 
 integers to machine words - but mainly, the int type focuses on
 
 correctness above performance, because the cost is low compared to the
 
 benefit. With float, the cost of arbitrary precision is extremely
 
 high, and the benefit much lower.
 
 
 
 With Unicode, the cost of perfect support is normally seen to be a
 
 doubling of internal memory usage (UTF-16 vs UCS-4). Pike and Python
 
 decided that the cost could, instead, be a tiny measure of complexity
 
 and actually *less* memory usage (compared to UTF-16, when lots of
 
 identifiers are ASCII). It's a system that works only when strings are
 
 immutable, but works beautifully there. Fortunately Pike doesn't have
 
 any, and Python has only one, idiot like jmf who completely
 
 misunderstands what's going on and uses microbenchmarks to prove
 
 obscure points... and then uses nonsense to try to prove... uhh...
 
 actually I'm not even sure what, sometimes. I wouldn't dare try to
 
 read his posts except that my mind's already in a rather broken state,
 
 as a combination of programming and Alice in Wonderland.
 


I do not mind to be considered as an idiot, but
I'm definitively not blind.

And I could add, I *never* saw once one soul, who is
explaining what I'm doing wrong in the gazillion
of examples I gave on this list.

---

Back to ReportLab. Technically I would be really
interested to see what could happen at the light
of my previous post.

jmf

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Terry Reedy]

On 1/4/2014 2:10 PM, wxjmfa...@gmail.com wrote:

Le samedi 4 janvier 2014 15:17:40 UTC+1, Chris Angelico a écrit :



any, and Python has only one, idiot like jmf who completely


Chris, I appreciate the many contributions you make to this list, but 
that does not exempt you from out standard of conduct.



misunderstands what's going on and uses microbenchmarks to prove
obscure points... and then uses nonsense to try to prove... uhh...


Troll baiting is a form of trolling. I think you are intelligent enough 
to know this.  Please stop.



I do not mind to be considered as an idiot, but
I'm definitively not blind.

And I could add, I *never* saw once one soul, who is
explaining what I'm doing wrong in the gazillion
of examples I gave on this list.


If this is true, it is because you have ignored and not read my 
numerous, relatively polite posts. To repeat very briefly:


1. Cherry picking (presenting the most extreme case as representative).

2. Calling space saving a problem (repeatedly).

3. Ignoring bug fixes.

4. Repetition (of the 'gazillion example' without new content).

Have you ever acknowledged, let alone thank people for, the fix for the 
one bad regression you did find. The FSR is still a work in progress. 
Just today, Serhiy pushed a patch speeding up the UTF-32 encoder, after 
previously speeding up the UTF-32 decoder.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Chris Angelico]

On Sun, Jan 5, 2014 at 9:46 AM, Terry Reedy tjre...@udel.edu wrote:
 On 1/4/2014 2:10 PM, wxjmfa...@gmail.com wrote:

 Le samedi 4 janvier 2014 15:17:40 UTC+1, Chris Angelico a écrit :


 any, and Python has only one, idiot like jmf who completely


 Chris, I appreciate the many contributions you make to this list, but that
 does not exempt you from out standard of conduct.


 misunderstands what's going on and uses microbenchmarks to prove
 obscure points... and then uses nonsense to try to prove... uhh...


 Troll baiting is a form of trolling. I think you are intelligent enough to
 know this.  Please stop.

My apologies. I withdraw the aforequoted post. You and Ned are
correct, those comments were inappropriate. Sorry.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Steven D'Aprano]

Roy Smith wrote:

 In article mailman.4882.1388808283.18130.python-l...@python.org,
  Mark Lawrence breamore...@yahoo.co.uk wrote:
 
 Surely everybody prefers fast but incorrect code in
 preference to something that is correct but slow?
 
 I realize I'm taking this statement out of context, but yes, sometimes
 fast is more important than correct.

I know somebody who was once touring in the States, and ended up travelling
cross-country by road with the roadies rather than flying. She tells me of
the time someone pointed out that they were travelling in the wrong
direction, away from their destination. The roadie driving replied Who
cares? We're making fantastic time!

(Ah, the seventies. So many drugs...)

Fast is never more important than correct. It's just that sometimes you
might compromise a little (or a lot) on what counts as correct in order for
some speed.

To give an example, say you want to solve the Travelling Salesman Problem,
and find the shortest path through a whole lot of cities A, B, C, ..., Z.
That's a Hard Problem, expensive to solve correctly.

But if you loosen the requirements so that a correct solution no longer has
to be the absolutely shortest path, and instead accept solutions which are
nearly always close to the shortest (but without any guarantee of how
close), then you can make the problem considerably easier to solve.

But regardless of how fast your path-finder algorithm might become, you're
unlikely to be satisfied with a solution that travels around in a circle
from A to B a million times then shoots off straight to Z without passing
through any of the other cities.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Chris Angelico]

On Sun, Jan 5, 2014 at 1:27 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 But regardless of how fast your path-finder algorithm might become, you're
 unlikely to be satisfied with a solution that travels around in a circle
 from A to B a million times then shoots off straight to Z without passing
 through any of the other cities.

On the flip side, that might be the best salesman your company has
ever known, if those three cities have the most customers!

ChrisA
wondering why nobody cares about the customers in TSP discussions
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[MRAB]

On 2014-01-05 02:32, Chris Angelico wrote:

On Sun, Jan 5, 2014 at 1:27 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:

But regardless of how fast your path-finder algorithm might become, you're
unlikely to be satisfied with a solution that travels around in a circle
from A to B a million times then shoots off straight to Z without passing
through any of the other cities.


On the flip side, that might be the best salesman your company has
ever known, if those three cities have the most customers!

ChrisA
wondering why nobody cares about the customers in TSP discussions


Or, for that matter, ISP customers who don't live in an urban area. :-)

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Steven D'Aprano]

wxjmfa...@gmail.com wrote:

 The very interesting aspect in the way you are holding
 unicodes (strings). By comparing Python 2 with Python 3.3,
 you are comparing utf-8 with the the internal representation
 of Python 3.3 (the flexible string represenation).

This is incorrect. Python 2 has never used UTF-8 internally for Unicode
strings. In narrow builds, it uses UTF-16, but makes no allowance for
surrogate pairs in strings. In wide builds, it uses UTF-32.

Other implementations, such as Jython or IronPython, may do something else.


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Chris Angelico]

On Sun, Jan 5, 2014 at 1:41 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 wxjmfa...@gmail.com wrote:

 The very interesting aspect in the way you are holding
 unicodes (strings). By comparing Python 2 with Python 3.3,
 you are comparing utf-8 with the the internal representation
 of Python 3.3 (the flexible string represenation).

 This is incorrect. Python 2 has never used UTF-8 internally for Unicode
 strings. In narrow builds, it uses UTF-16, but makes no allowance for
 surrogate pairs in strings. In wide builds, it uses UTF-32.

That's for Python's unicode type. What Robin said was that they were
using either a byte string (str) with UTF-8 data, or a Unicode
string (unicode) with character data. So jmf was right, except that
it's not specifically to do with Py2 vs Py3.3.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Roy Smith]

I wrote: 
  I realize I'm taking this statement out of context, but yes, sometimes
  fast is more important than correct.

In article 52c8c301$0$29998$c3e8da3$54964...@news.astraweb.com,
 Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
 Fast is never more important than correct.

Sure it is.

Let's imagine you're building a system which sorts packages for 
delivery.  You sort 1 million packages every night and put them on 
trucks going out for final delivery.

Some assumptions:

Every second I can cut from the sort time saves me $0.01.

If I mis-sort a package, it goes out on the wrong truck, doesn't get 
discovered until the end of the day, and ends up costing me $5 
(including not just the direct cost of redelivering it, but also 
factoring in ill will and having to make the occasional refund for not 
meeting the promised delivery time).

I've got a new sorting algorithm which is guaranteed to cut 10 seconds 
off the sorting time (i.e. $0.10 per package).  The problem is, it makes 
a mistake 1% of the time.

Let's see:

1 million packages x $0.10 = $100,000 saved per day because I sort them 
faster.  10,000 of them will go to the wrong place, and that will cost 
me $50,000 per day.  By going fast and making mistakes once in a while, 
I increase my profit by $50,000 per day.

The numbers above are fabricated, but I'm sure UPS, FexEx, and all the 
other package delivery companies are doing these sorts of analyses every 
day.  I watch the UPS guy come to my house.  He gets out of his truck, 
walks to my front door, rings the bell, waits approximately 5 
microseconds, leaves the package on the porch, and goes back to his 
truck.  I'm sure UPS has figured out that the amortized cost of the 
occasional stolen or lost package is less than the cost for the delivery 
guy to wait for me to come to the front door and sign for the delivery.

Looking at another problem domain, let's say you're a contestant on 
Jeopardy.  If you listen to the entire clue and spend 3 seconds making 
sure you know the correct answer before hitting the buzzer, it doesn't 
matter if you're right or wrong.  Somebody else beat you to the buzzer, 
2.5 seconds ago.

Or, let's take an example from sports.  I'm standing at home plate 
holding a bat.  60 feet away from me, the pitcher is about to throw a 
baseball towards me at darn close to 100 MPH (insert words like bowl 
and wicket as geographically appropriate).  400 ms later, the ball is 
going to be in the catcher's glove if you don't hit it.  If you have an 
absolutely perfect algorithm to determining if it's a ball or a strike, 
which takes 500 ms to run, you're going back to the minor leagues.  If 
you have a 300 ms algorithm which is right 75% of the time, you're 
heading to the hall of fame.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Rustom Mody]

On Sun, Jan 5, 2014 at 8:50 AM, Roy Smith r...@panix.com wrote:
 I wrote:
  I realize I'm taking this statement out of context, but yes, sometimes
  fast is more important than correct.

 In article 52c8c301$0$29998$c3e8da3$54964...@news.astraweb.com,
  Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
 Fast is never more important than correct.

 Sure it is.

 Let's imagine you're building a system which sorts packages for
 delivery.  You sort 1 million packages every night and put them on
 trucks going out for final delivery.

 Some assumptions:

 Every second I can cut from the sort time saves me $0.01.

 If I mis-sort a package, it goes out on the wrong truck, doesn't get
 discovered until the end of the day, and ends up costing me $5
 (including not just the direct cost of redelivering it, but also
 factoring in ill will and having to make the occasional refund for not
 meeting the promised delivery time).

 I've got a new sorting algorithm which is guaranteed to cut 10 seconds
 off the sorting time (i.e. $0.10 per package).  The problem is, it makes
 a mistake 1% of the time.

 Let's see:

 1 million packages x $0.10 = $100,000 saved per day because I sort them
 faster.  10,000 of them will go to the wrong place, and that will cost
 me $50,000 per day.  By going fast and making mistakes once in a while,
 I increase my profit by $50,000 per day.

 The numbers above are fabricated, but I'm sure UPS, FexEx, and all the
 other package delivery companies are doing these sorts of analyses every
 day.  I watch the UPS guy come to my house.  He gets out of his truck,
 walks to my front door, rings the bell, waits approximately 5
 microseconds, leaves the package on the porch, and goes back to his
 truck.  I'm sure UPS has figured out that the amortized cost of the
 occasional stolen or lost package is less than the cost for the delivery
 guy to wait for me to come to the front door and sign for the delivery.

 Looking at another problem domain, let's say you're a contestant on
 Jeopardy.  If you listen to the entire clue and spend 3 seconds making
 sure you know the correct answer before hitting the buzzer, it doesn't
 matter if you're right or wrong.  Somebody else beat you to the buzzer,
 2.5 seconds ago.

 Or, let's take an example from sports.  I'm standing at home plate
 holding a bat.  60 feet away from me, the pitcher is about to throw a
 baseball towards me at darn close to 100 MPH (insert words like bowl
 and wicket as geographically appropriate).  400 ms later, the ball is
 going to be in the catcher's glove if you don't hit it.  If you have an
 absolutely perfect algorithm to determining if it's a ball or a strike,
 which takes 500 ms to run, you're going back to the minor leagues.  If
 you have a 300 ms algorithm which is right 75% of the time, you're
 heading to the hall of fame.


Neat examples -- thanks
Only minor quibble isnt $5 cost of mis-sorting a gross underestimate?

I am reminded of a passage of Dijkstra in Discipline of Programming --
something to this effect

He laments the fact that hardware engineers were not including
overflow checks in machine ALUs.
He explained as follows:
If a test is moderately balanced (statistically speaking) a programmer
will not mind writing an if statement

If however the test is very skew -- say if 99% times, else 1% -- he
will tend to skimp on the test, producing 'buggy' code [EWD would
never use the bad b word or course]

The cost equation for hardware is very different -- once the
investment in the silicon is done with -- fixed cost albeit high --
there is no variable cost to executing that circuitry once or a
zillion times

Moral of Story: Intel should take up FSR
[Ducks and runs for cover]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Roy Smith]

In article mailman.4929.1388896998.18130.python-l...@python.org,
 Rustom Mody rustompm...@gmail.com wrote:

 On Sun, Jan 5, 2014 at 8:50 AM, Roy Smith r...@panix.com wrote:
  I wrote:
   I realize I'm taking this statement out of context, but yes, sometimes
   fast is more important than correct.
 
  In article 52c8c301$0$29998$c3e8da3$54964...@news.astraweb.com,
   Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
  Fast is never more important than correct.
 
  Sure it is.
 
  Let's imagine you're building a system which sorts packages for
  delivery.  You sort 1 million packages every night and put them on
  trucks going out for final delivery.
 
  Some assumptions:
 
  Every second I can cut from the sort time saves me $0.01.
 
  If I mis-sort a package, it goes out on the wrong truck, doesn't get
  discovered until the end of the day, and ends up costing me $5
  (including not just the direct cost of redelivering it, but also
  factoring in ill will and having to make the occasional refund for not
  meeting the promised delivery time).
 
  I've got a new sorting algorithm which is guaranteed to cut 10 seconds
  off the sorting time (i.e. $0.10 per package).  The problem is, it makes
  a mistake 1% of the time.
 
  Let's see:
 
  1 million packages x $0.10 = $100,000 saved per day because I sort them
  faster.  10,000 of them will go to the wrong place, and that will cost
  me $50,000 per day.  By going fast and making mistakes once in a while,
  I increase my profit by $50,000 per day.
 
  The numbers above are fabricated, but I'm sure UPS, FexEx, and all the
  other package delivery companies are doing these sorts of analyses every
  day.  I watch the UPS guy come to my house.  He gets out of his truck,
  walks to my front door, rings the bell, waits approximately 5
  microseconds, leaves the package on the porch, and goes back to his
  truck.  I'm sure UPS has figured out that the amortized cost of the
  occasional stolen or lost package is less than the cost for the delivery
  guy to wait for me to come to the front door and sign for the delivery.
 
  Looking at another problem domain, let's say you're a contestant on
  Jeopardy.  If you listen to the entire clue and spend 3 seconds making
  sure you know the correct answer before hitting the buzzer, it doesn't
  matter if you're right or wrong.  Somebody else beat you to the buzzer,
  2.5 seconds ago.
 
  Or, let's take an example from sports.  I'm standing at home plate
  holding a bat.  60 feet away from me, the pitcher is about to throw a
  baseball towards me at darn close to 100 MPH (insert words like bowl
  and wicket as geographically appropriate).  400 ms later, the ball is
  going to be in the catcher's glove if you don't hit it.  If you have an
  absolutely perfect algorithm to determining if it's a ball or a strike,
  which takes 500 ms to run, you're going back to the minor leagues.  If
  you have a 300 ms algorithm which is right 75% of the time, you're
  heading to the hall of fame.
 
 
 Neat examples -- thanks
 Only minor quibble isnt $5 cost of mis-sorting a gross underestimate?

I have no idea.  Like I said, the numbers are all fabricated.

I do have a friend who used to work for UPS.  He told me lots of UPS 
efficiency stories.  One of them had to do with mis-routed packages.  
IIRC, the process for dealing with a mis-routed package was to NOT waste 
any time trying to figure out why it was mis-routed.  It was just thrown 
back into the input hopper to go through the whole system again.  The 
sorting software kept track of how many times it had sorted a particular 
package.  Only after N attempts (where N was something like 3), was it 
kicked out of the automated process for human intervention.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Steven D'Aprano]

Roy Smith wrote:

 I wrote:
  I realize I'm taking this statement out of context, but yes, sometimes
  fast is more important than correct.
 
 In article 52c8c301$0$29998$c3e8da3$54964...@news.astraweb.com,
  Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote:
 Fast is never more important than correct.
 
 Sure it is.

Sure it isn't. I think you stopped reading my post too early.

None of your examples contradict what I am saying. They all involve exactly
the same sort of compromise regarding correctness that I'm talking about,
where you loosen what counts as correct for the purpose of getting extra
speed. So, for example:

 Let's imagine you're building a system which sorts packages for
 delivery.  You sort 1 million packages every night and put them on
 trucks going out for final delivery.

What's your requirement, i.e. what counts as correct for the delivery
algorithm being used? Is it that every parcel is delivered to the specified
delivery address the first time? No it is not. What counts as correct for
the delivery algorithm is something on the lines of No less than 95% of
parcels will be sorted correctly and delivered directly; no more than 5%
may be mis-sorted at most three times (or some similar requirement).

It may even been that the requirements are even looser, e.g.:

No more than 1% of parcels will be lost/damaged/stolen/destroyed

in which case they don't care unless a particular driver loses or destroys
more than 1% of his deliveries. But if it turns out that Fred is dumping
every single one of his parcels straight into the river, the fact that he
can make thirty deliveries in the time it takes other drivers to make one
will not save his job. But it's much faster to dump the parcels in the
river does not matter. What matters is that the deliveries are made within
the bounds of allowable time and loss.

Things get interesting when the people setting the requirements and the
people responsible for meeting those requirements aren't able to agree.
Then you have customers who complain that the software is buggy, and
developers who complain that the customer requirements are impossible to
provide. Sometimes they're both right.


 Looking at another problem domain, let's say you're a contestant on
 Jeopardy.  If you listen to the entire clue and spend 3 seconds making
 sure you know the correct answer before hitting the buzzer, it doesn't
 matter if you're right or wrong.  Somebody else beat you to the buzzer,
 2.5 seconds ago.

I've heard of Jeopardy, but never seen it. But I know about game shows, and
in this case, what you care about is *winning the game*, not answering the
questions correctly. Answering the questions correctly is only a means to
the end, which is Win. If the rules allow it, your best strategy might
even be to give wrong answers, every time!

(It's not quite a game show, but the British quiz show QI is almost like
that. The rules, if there are any, encourage *interesting* answers over
correct answers. Occasionally that leads to panelists telling what can best
be described as utter porkies[1].)

If Jeopardy does not penalise wrong answers, the best strategy might be to
jump in with an answer as quickly as possible, without caring too much
about whether it is the right answer. But if Jeopardy penalises mistakes,
then the best strategy might be to take as much time as you can to answer
the question, and hope for others to make mistakes. That's often the
strategy in Test cricket: play defensively, and wait for the opposition to
make a mistake.


 Or, let's take an example from sports.  I'm standing at home plate
 holding a bat.  60 feet away from me, the pitcher is about to throw a
 baseball towards me at darn close to 100 MPH (insert words like bowl
 and wicket as geographically appropriate).  400 ms later, the ball is
 going to be in the catcher's glove if you don't hit it.  If you have an
 absolutely perfect algorithm to determining if it's a ball or a strike,
 which takes 500 ms to run, you're going back to the minor leagues.  If
 you have a 300 ms algorithm which is right 75% of the time, you're
 heading to the hall of fame.

And if you catch the ball, stick it in your pocket and race through all the
bases, what's that? It's almost certainly faster than trying to play by the
rules. If speed is all that matters, that's what people would do. But it
isn't -- the correct strategy depends on many different factors, one of
which is that you have a de facto time limit on deciding whether to swing
or let the ball through.

Your baseball example is no different from the example I gave before. Find
the optimal path for the Travelling Salesman Problem in a week's time,
versus Find a close to optimal path in three minutes is conceptually the
same problem, with the same solution: an imperfect answer *now* can be
better than a perfect answer *later*.




[1] Porkies, or pork pies, from Cockney rhyming slang.

-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Chris Angelico]

On Sun, Jan 5, 2014 at 2:20 PM, Roy Smith r...@panix.com wrote:
 I've got a new sorting algorithm which is guaranteed to cut 10 seconds
 off the sorting time (i.e. $0.10 per package).  The problem is, it makes
 a mistake 1% of the time.

That's a valid line of argument in big business, these days, because
we've been conditioned to accept low quality. But there are places
where quality trumps all, and we're happy to pay for that. Allow me to
expound two examples.

1) Amazon

http://www.amazon.com/exec/obidos/ASIN/1782010165/evertype-20

I bought this book a while ago. It's about the size of a typical
paperback. It arrived in a box too large for it on every dimension,
with absolutely no packaging. I complained. Clearly their algorithm
was: Most stuff will get there in good enough shape, so people can't
be bothered complaining. And when they do complain, it's cheaper to
ship them another for free than to debate with them on chat. Because
that's what they did. Fortunately I bought the book for myself, not
for a gift, because the *replacement* arrived in another box of the
same size, with ... one little sausage for protection. That saved it
in one dimension out of three, so it arrived only slightly
used-looking instead of very used-looking. And this a brand new book.
When I complained the second time, I was basically told any
replacement we ship you will be exactly the same. Thanks.

2) Bad Monkey Productions

http://kck.st/1bgG8Pl

The cheapest the book itself will be is $60, and the limited edition
early ones are more (I'm getting the gold level book, $200 for one of
the first 25 books, with special sauce). The people producing this are
absolutely committed to quality, as are the nearly 800 backers. If
this project is delayed slightly in order to ensure that we get
something fully awesome, I don't think there will be complaints. This
promises to be a beautiful book that'll be treasured for generations,
so quality's far FAR more important than the exact delivery date.

I don't think we'll ever see type #2 become universal, for the same
reason that people buy cheap Chinese imports in the supermarket rather
than something that costs five times as much from a specialist. The
expensive one might be better, but why bother? When the cheap one
breaks, you just get another. The expensive one might fail too, so why
take that risk?

But it's always a tradeoff, and there'll always be a few companies
around who offer the more expensive product. (We have a really high
quality cheese slicer. It's still the best I've seen, after something
like 20 years of usage.) Fast or right? It'd have to be really
*really* fast to justify not being right, unless the lack of rightness
is less than measurable (like representing time in nanoseconds -
anything smaller than that is unlikely to be measurable on most
computers).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Terry Reedy]

On 1/2/2014 11:49 PM, Steven D'Aprano wrote:

Robin Becker wrote:


For fairly sensible reasons we changed the internal default to use unicode
rather than bytes. After doing all that and making the tests compatible
etc etc I have a version which runs in both and passes all its tests.
However, for whatever reason the python 3.3 version runs slower


For whatever reason is right, unfortunately there's no real way to tell
from the limited information you give what that might be.

Are you comparing a 2.7 wide or narrow build? Do your tests use any
so-called astral characters (characters in the Supplementary Multilingual
Planes, i.e. characters with ord()  0x)?

If I remember correctly, some early alpha(?) versions of Python 3.3
consistently ran Unicode operations a small but measurable amount slower
than 3.2 or 2.7. That especially effected Windows. But I understand that
this was sped up in the release version of 3.3.


There was more speedup in 3.3.2 and possibly even more in 3.3.3, so OP 
should run the latter.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[wxjmfauth]

It's time to understand the Character Encoding Models
and the math behind it.
Unicode does not differ from any other coding scheme.

How? With a sheet of paper and a pencil.

jmf
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Chris Angelico]

On Fri, Jan 3, 2014 at 9:10 PM,  wxjmfa...@gmail.com wrote:
 It's time to understand the Character Encoding Models
 and the math behind it.
 Unicode does not differ from any other coding scheme.

 How? With a sheet of paper and a pencil.


One plus one is two, therefore Python is better than Haskell.

Four times five is twelve, and four times six is thirteen, and four
times seven is enough to make Alice think she's Mabel, and London is
the capital of Paris, and the crocodile cheerfully grins. Therefore,
by obvious analogy, Unicode times new-style classes equals a 64-bit
process.

I worked that out with a sheet of paper and a pencil. The pencil was a
little help, but the paper was three sheets in the wind.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Blog about python 3

[Robin Becker]

On 02/01/2014 18:25, David Hutto wrote:

Just because it's 3.3 doesn't matter...the main interest is in
compatibility. Secondly, you used just one piece of code, which could be a
fluke, try others, and check the PEP. You need to realize that evebn the
older versions are benig worked on, and they have to be refined. So if you
have a problem, use the older and import from the future would be my
suggestion


Suggesting that I use another piece of code to test python3 against python2 is a 
bit silly. I'm sure I can find stuff which runs faster under python3, but 
reportlab is the code I'm porting and that is going the wrong way.

--
Robin Becker

--
https://mail.python.org/mailman/listinfo/python-list