Re: Exporter and subroutine circular dependencies between modules
On 3/13/22 13:13, David Christensen wrote:
> I have been wrestling with the Exporter module and subroutine circular
> dependencies between modules for a number of years.
https://www.mail-archive.com/module-authors@perl.org/msg10914.html
We revisited this topic at a San Francisco Perl Mongers meeting today:
https://mail.pm.org/pipermail/sanfrancisco-pm/2022-June/004851.html
I showed the following snippet of code that demonstrates the solution I
am currently using:
6 package Dpchrist::Lib5::Test;
7
8
9 use strict;
10 use warnings;
11 use threads;
12 use threads::shared;
13
14 our @EXPORT_OK;
15
16 BEGIN {
17 @EXPORT_OK =qw(
18 is_poly
19 );
20 }
21
22 use parent qw(
23 Exporter
24 Test::Builder::Module
25 );
The key points are:
* Put the Exporter-related statements (lines 14-25) near the top of the
module, before other code.
* Statement ordering is important:
* First -- declare @EXPORT_OK, but do not define/ initialize it (line
14).
* Next -- define/ initialize @EXPORT_OK in a BEGIN block (lines 16-20).
* Finally -- 'use parent' to inherit from Exporter (lines 22, 23, and
25).
* The above module also happens to inherit from Test::Builder::Module.
My other modules do not need or have line 24.
* As I develop code and introduce bugs, I frequently see warnings to the
effect "subroutine redefined" when there is a circular loop between
modules. Once I fix the bugs, those warnings go away.
Without understanding the "how" and "why" of perl(1), Exporter, "compile
time", "run time", "require", "use", "parent", "import", etc. -- of the
several solutions myself and others have tried over time, this one seems
to work the best for me.
David
Re: Exporter and subroutine circular dependencies between modules
On 3/13/22 16:12, Shawn H Corey wrote: Each module has its own runtime. That is, a module is loaded, compiled and run; all before the main program. In other words, when the module returns from its `use` statement, it has already run. AIUI that is the view from 20,000 ft.. But, there are details and interactions I do not perceive -- 'compile time' vs. 'run time', 'use' vs. 'require' and 'import', the symbol table, recursion, etc.. I need to dig into the Camel book again; I would rather avoid a code walk. David
Re: Exporter and subroutine circular dependencies between modules
On 3/13/22 15:44, Diab Jerius wrote: On Sun, Mar 13, 2022, 18:19 Shawn H Corey wrote: use Exporter qw( import ); our @EXPORT = qw( foo ); our @EXPORT_OK = qw( ); our %EXPORT_TAGS = ( all => [ @EXPORT, @EXPORT_OK ], ); This automatically creates a tag for `:all`. I prefer yet another fashion (everything is driven by what's in %EXPORT_TAGS), Yes, sometimes that makes more sense. David
Re: Exporter and subroutine circular dependencies between modules
On 3/13/22 15:18, Shawn H Corey wrote:
On 2022-03-13 18:08, Diab Jerius via module-authors wrote:
require Exporter;
our @ISA = qw( Exporter );
our @EXPORT = qw( foo )
I prefer this way:
# --
# Exports
use Exporter qw( import );
our @EXPORT = qw( foo );
our @EXPORT_OK = qw( );
our %EXPORT_TAGS = (
all => [ @EXPORT, @EXPORT_OK ],
);
This automatically creates a tag for `:all`.
Thank you for the reply. :-)
That approach does not solve the circular dependency issue:
2022-03-13 19:02:58 dpchrist@tinkywinky
~/samba/dpchrist/sandbox/perl/Exporter-circular-use
$ cat /etc/debian_version ; uname -a ; perl -v | head -n 2
9.13
Linux tinkywinky 4.9.0-17-amd64 #1 SMP Debian 4.9.290-1 (2021-12-12)
x86_64 GNU/Linux
This is perl 5, version 24, subversion 1 (v5.24.1) built for
x86_64-linux-gnu-thread-multi
2022-03-13 19:03:22 dpchrist@tinkywinky
~/samba/dpchrist/sandbox/perl/Exporter-circular-use
$ cvs diff
Index: Bar11.pm
===
RCS file:
/var/local/cvs/dpchrist/sandbox/perl/Exporter-circular-use/Bar11.pm,v
retrieving revision 1.1
diff -r1.1 Bar11.pm
9,11c9,14
< require Exporter;
< our @ISA = qw( Exporter );
< our @EXPORT= qw( bar );
---
> use Exporter qw( import );
> our @EXPORT = qw( bar );
> our @EXPORT_OK = qw();
> our %EXPORT_TAGS = (
> all => [ @EXPORT, @EXPORT_OK ],
> );
Index: Foo11.pm
===
RCS file:
/var/local/cvs/dpchrist/sandbox/perl/Exporter-circular-use/Foo11.pm,v
retrieving revision 1.1
diff -r1.1 Foo11.pm
9,11c9,15
< require Exporter;
< our @ISA = qw( Exporter );
< our @EXPORT= qw( foo );
---
> use Exporter qw( import );
> our @EXPORT = qw( foo );
> our @EXPORT_OK = qw();
> our %EXPORT_TAGS = (
> all => [ @EXPORT, @EXPORT_OK ],
> );
>
2022-03-13 19:03:24 dpchrist@tinkywinky
~/samba/dpchrist/sandbox/perl/Exporter-circular-use
$ ./foo11
main=7
foo=6
bar=5
Undefined subroutine ::foo called at Bar11.pm line 25.
And, I have doubts about use'ing Exporter and importing Exporter::import().
But, you have reminded me that it is better style to export nothing by
default (the demo script and modules used @EXPORT for brevity).
So, combining the ideas thus far:
BEGIN { our @EXPORT_OK = qw( foo ) }
use parent 'Exporter';
David
2022-03-13 19:06:46 dpchrist@tinkywinky
~/samba/dpchrist/sandbox/perl/Exporter-circular-use
$ ./Exporter-circular-use.t
ok 1 - foo00
ok 2 - foo01
ok 3 - foo10
Undefined subroutine ::foo called at Bar11.pm line 22.
not ok 4 - foo11
# Failed test 'foo11'
# at ./Exporter-circular-use.t line 24.
# got: 'main=7
# foo=6
# bar=5
# '
# expected: 'main=7
# foo=6
# bar=5
#foo=4
# bar=3
# foo=2
# bar=1
# '
Undefined subroutine ::foo called at Bar12.pm line 23.
not ok 5 - foo12
# Failed test 'foo12'
# at ./Exporter-circular-use.t line 24.
# got: 'main=7
# foo=6
# bar=5
# '
# expected: 'main=7
# foo=6
# bar=5
#foo=4
# bar=3
# foo=2
# bar=1
# '
ok 6 - foo21
ok 7 - foo22
ok 8 - foo33
ok 9 - foo44
1..9
# Looks like you failed 2 tests of 9.
Exporter-circular-use-20220313-185149.tar.gz
Description: application/gzip
Re: Exporter and subroutine circular dependencies between modules
On 3/13/22 15:08, Diab Jerius via module-authors wrote:
On 3/13/22 16:13, David Christensen wrote:
module-authors:
I have been wrestling with the Exporter module and subroutine circular
dependencies between modules for a number of years. I have yet to
find a satisfactory solution.
If you move import of modules with circular dependencies to runtime
rather than compile time:
$ diff Exporter-circular-use{.orig,.new}
diff Exporter-circular-use.orig/Bar11.pm
Exporter-circular-use.new/Bar11.pm
13c13,14
< use Foo11;
---
> require Foo11;
> Foo11->import;
diff Exporter-circular-use.orig/Foo11.pm
Exporter-circular-use.new/Foo11.pm
13c13,14
< use Bar11;
---
> require Bar11;
> Bar11->import;
$ perl Exporter-circular-use.t
ok 1 - foo00
ok 2 - foo01
ok 3 - foo10
ok 4 - foo11
1..4
Thank you for the reply. :-)
Yes, that works.
Or move the export completely into compile time:
$ diff Exporter-circular-use{.orig,.new}
diff Exporter-circular-use.orig/Bar11.pm
Exporter-circular-use.new/Bar11.pm
9,11c9,11
< require Exporter;
< our @ISA = qw( Exporter );
< our @EXPORT = qw( bar );
---
> use parent 'Exporter';
> our @EXPORT;
> BEGIN{ @EXPORT = qw( bar ); }
diff Exporter-circular-use.orig/Foo11.pm
Exporter-circular-use.new/Foo11.pm
9,11c9,11
< require Exporter;
< our @ISA = qw( Exporter );
< our @EXPORT = qw( foo );
---
> use parent 'Exporter';
> our @EXPORT;
> BEGIN { @EXPORT = qw( foo ); }
Yes, that also works. Putting the @EXPORT allocation and initialization
in a BEGIN block ahead of the 'use parent' looks even better:
2022-03-13 18:15:46 dpchrist@tinkywinky
~/samba/dpchrist/sandbox/perl/Exporter-circular-use
$ diff Foo11.pm Foo33.pm
1c1
< package Foo11;
---
> package Foo33;
9,11c9,10
< require Exporter;
< our @ISA = qw( Exporter );
< our @EXPORT= qw( foo );
---
> BEGIN { our @EXPORT = qw( foo ) }
> use parent 'Exporter';
13c12
< use Bar11;
---
> use Bar33;
David
2022-03-13 18:38:24 dpchrist@tinkywinky
~/samba/dpchrist/sandbox/perl/Exporter-circular-use
$ cat /etc/debian_version ; uname -a ; perl -v | head -n 2
9.13
Linux tinkywinky 4.9.0-17-amd64 #1 SMP Debian 4.9.290-1 (2021-12-12)
x86_64 GNU/Linux
This is perl 5, version 24, subversion 1 (v5.24.1) built for
x86_64-linux-gnu-thread-multi
2022-03-13 18:40:00 dpchrist@tinkywinky
~/samba/dpchrist/sandbox/perl/Exporter-circular-use
$ ./Exporter-circular-use.t
ok 1 - foo00
ok 2 - foo01
ok 3 - foo10
Undefined subroutine ::foo called at Bar11.pm line 22.
not ok 4 - foo11
# Failed test 'foo11'
# at ./Exporter-circular-use.t line 24.
# got: 'main=7
# foo=6
# bar=5
# '
# expected: 'main=7
# foo=6
# bar=5
#foo=4
# bar=3
# foo=2
# bar=1
# '
Undefined subroutine ::foo called at Bar12.pm line 23.
not ok 5 - foo12
# Failed test 'foo12'
# at ./Exporter-circular-use.t line 24.
# got: 'main=7
# foo=6
# bar=5
# '
# expected: 'main=7
# foo=6
# bar=5
#foo=4
# bar=3
# foo=2
# bar=1
# '
ok 6 - foo21
ok 7 - foo22
ok 8 - foo33
1..8
# Looks like you failed 2 tests of 8.
Exporter-circular-use-20220313-181541.tar.gz
Description: application/gzip
Re: Exporter and subroutine circular dependencies between modules
On Sun, Mar 13, 2022 at 1:14 PM David Christensen
wrote:
module-authors:
I have been wrestling with the Exporter module and subroutine circular
dependencies between modules for a number of years. I have yet to find
On 3/13/22 14:57, Karen Etheridge wrote:
> I haven't looked at your code, but I assume what you're doing is
exporting
> a sub from each module, and something in each of those modules is calling
> each of those subs in turn, using the exported symbol. This is indeed an
> unresolvable circular dependency because unqualified sub names must be
> resolved at compile (i.e. "use" or BEGIN{}) time.
>
> One solution is to not import all the symbols -- break the cycle by
simply
> 'use'ing a module without importing its subs, and call that sub by its
> fully-qualified name (e.g. Foo::foo() rather than foo()), which only
needs
> to be resolvable at runtime.
Thank you for the reply. :-)
I am looking for a solution that allows me to call module subroutines
via imported names, rather than via fully-qualified names.
David
Re: Exporter and subroutine circular dependencies between modules
On 3/13/22 19:12, Shawn H Corey wrote:
On 2022-03-13 18:44, Diab Jerius wrote:
I prefer yet another fashion (everything is driven by what's in
%EXPORT_TAGS), but back to the point of the OP's problem, this
doesn't initialize @EXPORT until runtime, so I think will have the
same issue (I'm away from keyboard at the moment).
Each module has its own runtime. That is, a module is loaded, compiled
and run; all before the main program. In other words, when the module
returns from its `use` statement, it has already run.
Except that in this case, the two modules are importing each other, so
that this linear model does not hold.
I'm back at my keyboard, so can implement your suggestion.
$ diff Exporter-circular-use{.orig,.new}
diff Exporter-circular-use.orig/Bar11.pm
Exporter-circular-use.new/Bar11.pm
9,10c9
< require Exporter;
< our @ISA = qw( Exporter );
---
> use Exporter 'import';
diff Exporter-circular-use.orig/Foo11.pm
Exporter-circular-use.new/Foo11.pm
9,10c9
< require Exporter;
< our @ISA = qw( Exporter );
---
> use Exporter 'import';
With the same unfortunate result:
$ perl foo11
main=7
foo=6
bar=5
Undefined subroutine ::foo called at Bar11.pm line 21.
Re: Exporter and subroutine circular dependencies between modules
On 2022-03-13 18:44, Diab Jerius wrote: I prefer yet another fashion (everything is driven by what's in %EXPORT_TAGS), but back to the point of the OP's problem, this doesn't initialize @EXPORT until runtime, so I think will have the same issue (I'm away from keyboard at the moment). Each module has its own runtime. That is, a module is loaded, compiled and run; all before the main program. In other words, when the module returns from its `use` statement, it has already run.
Re: Exporter and subroutine circular dependencies between modules
On Sun, Mar 13, 2022, 18:19 Shawn H Corey wrote: > On 2022-03-13 18:08, Diab Jerius via module-authors wrote: > > require Exporter; > our @ISA= qw( Exporter ); > our @EXPORT= qw( foo ) > > I prefer this way: > > # -- > # Exports > use Exporter qw( import ); > our @EXPORT = qw( foo ); > our @EXPORT_OK = qw( ); > our %EXPORT_TAGS = ( > all => [ @EXPORT, @EXPORT_OK ], > ); > > This automatically creates a tag for `:all`. > I prefer yet another fashion (everything is driven by what's in %EXPORT_TAGS), but back to the point of the OP's problem, this doesn't initialize @EXPORT until runtime, so I think will have the same issue (I'm away from keyboard at the moment).
Re: Exporter and subroutine circular dependencies between modules
On 2022-03-13 18:08, Diab Jerius via module-authors wrote: require Exporter; our @ISA = qw( Exporter ); our @EXPORT = qw( foo ) I prefer this way: # -- # Exports use Exporter qw( import ); our @EXPORT = qw( foo ); our @EXPORT_OK = qw( ); our %EXPORT_TAGS = ( all => [ @EXPORT, @EXPORT_OK ], ); This automatically creates a tag for `:all`.
Re: Exporter and subroutine circular dependencies between modules
On 3/13/22 16:13, David Christensen wrote:
module-authors:
I have been wrestling with the Exporter module and subroutine circular
dependencies between modules for a number of years. I have yet to
find a satisfactory solution.
[...]
What is the "proper" way to avoid or solve the problem of subroutine
circular dependencies between modules?
The issue in your case is in part[1] due to the fact that in Foo11.pm,
this part is run during the execution phase:
require Exporter;
our @ISA = qw( Exporter );
our @EXPORT = qw( foo )
while
use Bar11;
is actually run earlier, during the compilation phase, so Bar11 can't
get any symbols from Foo11 because it's not actually exporting them yet.
If you move import of modules with circular dependencies to runtime
rather than compile time:
$ diff Exporter-circular-use{.orig,.new}
diff Exporter-circular-use.orig/Bar11.pm
Exporter-circular-use.new/Bar11.pm
13c13,14
< use Foo11;
---
> require Foo11;
> Foo11->import;
diff Exporter-circular-use.orig/Foo11.pm
Exporter-circular-use.new/Foo11.pm
13c13,14
< use Bar11;
---
> require Bar11;
> Bar11->import;
$ perl Exporter-circular-use.t
ok 1 - foo00
ok 2 - foo01
ok 3 - foo10
ok 4 - foo11
1..4
Or move the export completely into compile time:
$ diff Exporter-circular-use{.orig,.new}
diff Exporter-circular-use.orig/Bar11.pm
Exporter-circular-use.new/Bar11.pm
9,11c9,11
< require Exporter;
< our @ISA = qw( Exporter );
< our @EXPORT = qw( bar );
---
> use parent 'Exporter';
> our @EXPORT;
> BEGIN{ @EXPORT = qw( bar ); }
diff Exporter-circular-use.orig/Foo11.pm
Exporter-circular-use.new/Foo11.pm
9,11c9,11
< require Exporter;
< our @ISA = qw( Exporter );
< our @EXPORT = qw( foo );
---
> use parent 'Exporter';
> our @EXPORT;
> BEGIN { @EXPORT = qw( foo ); }
it also will work;
[1] I'm not completely solid on symbol resolution time in Perl, so I'll
prevaricate.
Re: Exporter and subroutine circular dependencies between modules
I haven't looked at your code, but I assume what you're doing is exporting
a sub from each module, and something in each of those modules is calling
each of those subs in turn, using the exported symbol. This is indeed an
unresolvable circular dependency because unqualified sub names must be
resolved at compile (i.e. "use" or BEGIN{}) time.
One solution is to not import all the symbols -- break the cycle by simply
'use'ing a module without importing its subs, and call that sub by its
fully-qualified name (e.g. Foo::foo() rather than foo()), which only needs
to be resolvable at runtime.
On Sun, Mar 13, 2022 at 1:14 PM David Christensen
wrote:
> module-authors:
>
> I have been wrestling with the Exporter module and subroutine circular
> dependencies between modules for a number of years. I have yet to find
>
