Re: How to import all things defined the files in a module directory in __init__.py?

[Chris Angelico]

On Sat, Sep 24, 2016 at 9:32 PM, Brendan Abel <007bren...@gmail.com> wrote:
>> Splitting it up would make it slower to load.
>
> It's usually the opposite.  When packages are split up, you only have to
> load the specific portions you need.  Putting it all in a single module
> forces you to always load everything.

This can be true ONLY if they're sufficiently separate that most users
can pick and choose. If the bulk of users are going to need every
piece, the split will slow it down significantly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Brendan Abel]

> Splitting it up would make it slower to load.

It's usually the opposite.  When packages are split up, you only have to
load the specific portions you need.  Putting it all in a single module
forces you to always load everything.

On Fri, Sep 23, 2016 at 11:59 PM, Lawrence D’Oliveiro <
lawrenced...@gmail.com> wrote:

> On Saturday, September 24, 2016 at 2:11:09 PM UTC+12, Chris Angelico wrote:
> > It's a large and complex module, and about at the boundary of being
> > broken up a bit.
>
> Splitting it up would make it slower to load.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Steve D'Aprano]

On Sat, 24 Sep 2016 04:59 pm, Lawrence D’Oliveiro wrote:

> On Saturday, September 24, 2016 at 2:11:09 PM UTC+12, Chris Angelico
> wrote:
>> It's a large and complex module, and about at the boundary of being
>> broken up a bit.
> 
> Splitting it up would make it slower to load.

Would it? You've bench marked it and found that it makes a significant
difference?

In any case, you're missing the point. Size carries its own cost to the
human reader, never mind whether or not the interpreter can deal with it:

https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two

The decimal module is approaching a size where it is no longer comfortable
to read or edit:

- 6450 lines in the source file (include comments, blanks, etc);

- 27 top-level functions;

- 54 global variables/constants;

- 19 classes, which isn't too bad on its own, but:

- one of those classes has 83 methods;

- and another has 127 methods;

- the Decimal class itself is 3311 lines alone; excluding blanks, comments
and docstrings, it is 2013 SLOC.

This a partly a matter of taste, and to my taste, the decimal module is
about as big as I would like to see a module before I split it into
submodules purely on the basis of size.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Lawrence D’Oliveiro]

On Saturday, September 24, 2016 at 2:11:09 PM UTC+12, Chris Angelico wrote:
> It's a large and complex module, and about at the boundary of being
> broken up a bit.

Splitting it up would make it slower to load.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Chris Angelico]

On Sat, Sep 24, 2016 at 11:42 AM, Lawrence D’Oliveiro
 wrote:
> On Friday, September 23, 2016 at 4:25:21 AM UTC+12, Chris Angelico wrote:
>> For reference, the Decimal module (ignoring the C accelerator) is over six
>> thousand lines of code, as a single module. Now, that might be pushing the
>> boundaries a bit ...
>
> What “boundaries” do you think that might be pushing? 6000 lines doesn’t 
> sound unreasonable to me at all.

It's a large and complex module, and about at the boundary of being
broken up a bit. So it's likely to be the largest file in the stdlib
(not counting tests).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Lawrence D’Oliveiro]

On Friday, September 23, 2016 at 4:25:21 AM UTC+12, Chris Angelico wrote:
> For reference, the Decimal module (ignoring the C accelerator) is over six
> thousand lines of code, as a single module. Now, that might be pushing the
> boundaries a bit ...

What “boundaries” do you think that might be pushing? 6000 lines doesn’t sound 
unreasonable to me at all.
-- 
https://mail.python.org/mailman/listinfo/python-list

Automated refactoring tools (was: How to import all things defined the files in a module directory in __init__.py?)

[Ben Finney]

Peng Yu  writes:

> Is there such a good automated tool for python refactoring?

This sounds like a job for — Bicycle Repair Man!

Watch him extract jumbled code into well ordered classes.

Gasp, as he renames all occurrences of a method.

Thank You Bicycle Repair Man!



There is also Rope  which is a
library used in numerous programming environments, including Emacs
.

-- 
 \   “I just got out of the hospital; I was in a speed-reading |
  `\ accident. I hit a bookmark and flew across the room.” —Steven |
_o__)   Wright |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Peng Yu]

On Thu, Sep 22, 2016 at 8:35 PM, Ben Finney  wrote:
> Peng Yu  writes:
>
>> On Wed, Sep 21, 2016 at 11:14 PM, Ben Finney  
>> wrote:
>> > [Importing ‘*’ from a module] will also make the names in the code
>> > impossible to automatically match against where they came from.
>> > Explicit is better than implicit; you are proposing to make an
>> > unknown horde of names in the code implicit and untraceable.
>>
>> This will make refactoring easy. If everything is explicit, when one
>> do refactoring, at two places need to be changed which can be a
>> burden.
>
> That's completely backward: Importing ‘*’ from the module makes
> refactoring significantly *more* difficult.
>
> With explicit ‘import foo; foo.lorem()’, an automated tool can know that
> when ‘lorem’ changes to a different name, this module's use of
> ‘foo.lorem’ should also change.

Is there such a good automated tool for python refactoring?

> With non-explicit ‘from foo import *; lorem()’, then an automated too
> has *no way* of knowing that ‘lorem’ should change when you alter that
> name in the ‘foo’ module.
>
> So no, what you say above is the opposite of correct. Instead, using
> star import makes a rename *more* difficult to do correctly.
>
> --
>  \  “Faith is generally nothing more than the permission religious |
>   `\ people give to one another to believe things strongly without |
> _o__)  evidence.” —Sam Harris, _Letter to a Christian Nation_ 2006 |
> Ben Finney
>
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Regards,
Peng
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Ben Finney]

Peng Yu  writes:

> On Wed, Sep 21, 2016 at 11:14 PM, Ben Finney  
> wrote:
> > [Importing ‘*’ from a module] will also make the names in the code
> > impossible to automatically match against where they came from.
> > Explicit is better than implicit; you are proposing to make an
> > unknown horde of names in the code implicit and untraceable.
>
> This will make refactoring easy. If everything is explicit, when one
> do refactoring, at two places need to be changed which can be a
> burden.

That's completely backward: Importing ‘*’ from the module makes
refactoring significantly *more* difficult.

With explicit ‘import foo; foo.lorem()’, an automated tool can know that
when ‘lorem’ changes to a different name, this module's use of
‘foo.lorem’ should also change.

With non-explicit ‘from foo import *; lorem()’, then an automated too
has *no way* of knowing that ‘lorem’ should change when you alter that
name in the ‘foo’ module.

So no, what you say above is the opposite of correct. Instead, using
star import makes a rename *more* difficult to do correctly.

-- 
 \  “Faith is generally nothing more than the permission religious |
  `\ people give to one another to believe things strongly without |
_o__)  evidence.” —Sam Harris, _Letter to a Christian Nation_ 2006 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Chris Angelico]

On Fri, Sep 23, 2016 at 1:50 AM, Peng Yu  wrote:
>
> This will make refactoring easy. If everything is explicit, when one
> do refactoring, at two places need to be changed which can be a
> burden.

So you want it to be easy to move stuff around between files in a
package? Sounds like you don't actually have a package at all - you
have a single module that you're splitting across several files.

How big would this file be if you simply put it all into one .py file
and got rid of the package altogether? For reference, the Decimal
module (ignoring the C accelerator) is over six thousand lines of
code, as a single module. Now, that might be pushing the boundaries a
bit, but certainly there's no reason to split a file that's under a
thousand lines of code. Just keep it all in one file, and organize it
using marker comments - that way, there's no API change as you reorder
stuff.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Peng Yu]

On Wed, Sep 21, 2016 at 11:14 PM, Ben Finney  wrote:
> Peng Yu  writes:
>
>> I want to import all the thing (or the ones available in the
>> respective __all__) defined in each of the file by putting the
>> following lines in __init__.py
>>
>> from file1 import *
>> 
>> from filen import *
>>
>> However, I don't want to hardcode the file names in __init__.py. Is
>> there an automatic way of doing it?
>
> Why do you want to do that? It will defeat static analysis of the code,
> which is one of the more important tools to make your code reliable.
>
> It will also make the names in the code impossible to automatically
> match against where they came from. Explicit is better than implicit;
> you are proposing to make an unknown horde of names in the code implicit
> and untraceable.
>
> Why? What problem are you trying to solve that you believe needs this
> approach?

This will make refactoring easy. If everything is explicit, when one
do refactoring, at two places need to be changed which can be a
burden.

-- 
Regards,
Peng
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to import all things defined the files in a module directory in __init__.py?

[Ben Finney]

Peng Yu  writes:

> I want to import all the thing (or the ones available in the
> respective __all__) defined in each of the file by putting the
> following lines in __init__.py
>
> from file1 import *
> 
> from filen import *
>
> However, I don't want to hardcode the file names in __init__.py. Is
> there an automatic way of doing it?

Why do you want to do that? It will defeat static analysis of the code,
which is one of the more important tools to make your code reliable.

It will also make the names in the code impossible to automatically
match against where they came from. Explicit is better than implicit;
you are proposing to make an unknown horde of names in the code implicit
and untraceable.

Why? What problem are you trying to solve that you believe needs this
approach?

-- 
 \   “Anything that we scientists can do to weaken the hold of |
  `\religion should be done and may in the end be our greatest |
_o__)  contribution to civilization.” —Steven Weinberg |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

How to import all things defined the files in a module directory in __init__.py?

[Peng Yu]

Hi,

Suppose that I have file1.py, ..., filen.py in a module directory.

I want to import all the thing (or the ones available in the
respective __all__) defined in each of the file by putting the
following lines in __init__.py

from file1 import *

from filen import *

However, I don't want to hardcode the file names in __init__.py. Is
there an automatic way of doing it?

-- 
Regards,
Peng
-- 
https://mail.python.org/mailman/listinfo/python-list