Hi katja,
     Thanks for the response.  A few questions and responses:
* is knob's move from flatspace to flatgui a special case, or is it 
representative of some trend of object classes scurrying about from directory 
to directory?

If it is a special case, then either it is a bug: 
"I, user, do declare my intention to load flatspace/knob, a library so standard 
that it is installed in a path relative to Pd itself, yet so nonstandard that I 
cannot trust libdir/objectname"
or Pd is wildly misleading:
"I, user, do declare my intention to load library flatspace/knob as a 
STDLIB[see terms]"
[small print] terms: the standard library path is not responsible for items 
lost or stolen by the libraries residing in the standard path.

If moving objects around is general trend, well... I really hope it isn't.

* I don't path priority can solve the problem.  Pd has one big class called 
"objectmaker" which registers a new method for each object creator in Pd, 
including externals.  So when you type "some_object" in an object box, the 
first thing Pd is going to search are the methods of objectmaker.  And if it 
finds a method named "some_object" it's going to execute the function it finds 
in that method entry.  Thus, if "some_object" was already loaded anywhere in 
the running Pd instance, it's going to have a method entry in objectmaker and 
Pd isn't going to search any paths for a library to load.  Now you're back to 
gambling without prefixes.

Btw-- you can explore these methods in objectmaker from within Pd-l2ork with 
[nbx]--[classinfo objectmaker].  You can also see everything that's gone 
through class_new with [classlist(---[pdinfo].

On Thursday, October 16, 2014 9:44 AM, katja <katjavet...@gmail.com> wrote:
 


Thanks for bringing up this important topic. Over the years I flip-flopped 
between [library/objectname] and [declare] a few times, for reasons which I'll 
try to explain.

The first occasion where I needed [library/objectname] was when [pow~] in pd 
core had it's inlets swapped between pd versions, while [cyclone/pow~] retained 
the old inlet order and could be used for backward compatibility.

When Pd-extended stopped loading libraries by default I started using 
[library/objectname] for externals from all Pd-extended libraries, which was 
recommended practice.

Later, I came across a few cases where externals moved from one lib to another. 
For example, [flatspace/knob] became [flatgui/knob]. I feared further library 
reorganizations and abandoned the rigid [library/objectname] format. Now I 
started to appreciate the flexibility provided by [declare]. I used 
declarations like [declare -stdlib
 flatspace/knob] and [declare -stdlib flatgui/knob] in top level patches to 
make projects compatible across Pd-extended versions. Also, I use [declare] for 
binaries which are included in project directories: for my homebrew binaries, 
or selected Pd-extended binaries in projects which should run with vanilla Pd. 
It is much easier to change or add a few declarations in the top level patch 
than changing library names in all object instances in a large project.

But [declare] only adds search paths and it doesn't protect against name clash. 
I came across [bsaylor/svf~] which has different number of inlets and outlets 
than [cyclone/svf~]. Without library name, [svf~] instantiates the cyclone 
version. But when [bsaylor/svf~] was instantiated earlier in the Pd session, or 
with [import bsaylor] in the patch, the bsaylor version may be instantiated. 
Without namespace specification Pd seems to use the version that was loaded 
first within a Pd session. 

I realized that instantiating without library namespaces is a gambling. And 
this does not only apply to binary executables, but to abstractions as well. 
Imagine you share a Pd project including abstractions and binaries, and you 
publish a new version of the project, where bugs in externals were fixed or new 
features were added to abstractions. Users may want to compare versions, and 
load the old version first in a Pd session. Even when they close the old 
version of the project before loading the new version, the old executables and 
abstraction definitions are still in Pd's memory, and Pd will not load the new 
ones unless they were prefixed by a different namespace, or until Pd is 
restarted. So, the new project version may seem to still contain the old bugs!

The current solution for such issues is systematic use of lib names and 
relative paths in object or abstraction instances. For example, include 
abstractions in a subdirectory named 'abstractions', and instantiate as 
[abstractions/objectname]. Pd will then load the abstraction from the 
subdirectory in the project, even when many projects have an 'abstractions' 
subdirectory containing an abstraction with the same name. This tedious and 
inflexible approach will only work when directory structures don't change. A 
project or library structure must be well organized right from the start. If 
something changes in the structure, you may need to find all instances of a 
class in your patch(es) and redefine them. Rigid directory structures, and the 
prospect of minor changes causing heaps of maintenance work, can be an obstacle 
for future innovations in a project, or in Pd-extended as a whole.

In any case, I'm reluctant to use namespaces for each and every instance. It 
can make your patch today, and break it tomorrow. It would be super if 
[declare] would not only add a search path, but also locally enforce priority 
for it. Say I have my own implementation of [svf~] stored in the 'bin' 
subdirectory of a Pd project directory. The main patch contains [declare -path 
bin] and several instances of [svf~]. Pd should search the bin subdirectory and 
other locally declared paths for an [svf~] executable. If Pd finds the one in 
the bin subdirectory, it should interpret all instances of [svf~] as [bin/svf~] 
and instantiate them accordingly. If an object name appears in two locally 
declared paths, you would still have a name clash. But designing a single 
project directory without name clashes is not so difficult. 

How hard would it be to realize such path priority, I wonder? Currently, Pd can 
distinguish between different abstractions with the same name and relative path 
prefix, if they appear in different project directories. See attached test, 
where different versions [abstractions/namespace-testabstraction] are 
instantiated from patches in two different 'project folders'. Apparently Pd 
identifies these files with their absolute path, otherwise it could not load 
two different versions of [abstractions/namespace-testabstraction], right?

Katja



On Thu, Oct 16, 2014 at 6:21 AM, Jonathan Wilkes via Pd-list 
<pd-list@lists.iem.at> wrote:

Hi list,     Let's say I make a patch that uses the zexy and hcs libraries.  I 
want my patch to be portable to Pd-extended, Pd-l2ork, and whatever hand-rolled 
Pd-vanilla+zexy+hcs installations are sitting out there on users machines.
>
>If that is my goal, then I am going to use libdir/objectname syntax for all my 
>zexy and hcs objects in my patch.  I am going to do this because if a user 
>ever reports
 that there is a nameclash, it is very likely to be due to a bug somewhere, and 
that bug is very likely to be fixable.
>
>In fact, I will go so far as to say there is no other way to make a portable 
>patch that uses externals on the versions of Pd I described above...
>* both [import] and [declare] will happen _after_ the the default
 libs of Pd-extended/L2ork are loaded, so I can't use unprefixed object names 
with impunity (same with .pdrc).  And even if it does work, it's not 
future-proof, as someone may add an object that aliases one I'm using in my 
patch.
>* command lines flags don't ship with my patch, and they get parsed _after_ 
>the user or default prefs so again, I'm just hoping to get lucky.
>
>* even with Pd-extended's current "load-nothing" dogma, all possible object 
>names are stored globally (in what looks like both prefixed and unprefixed 
>forms, like "foo/bar" and "bar").  So if my patch is an abstraction and I 
>don't use libdir/objectname syntax for the object names within it, all I can 
>do is hope the user hasn't already loaded something that aliases one of my 
>objects.  (And again, not future-proof)
>
>
>
>I see only two possibilities, then: either the community changes the core 
>functionality so that there is a local object-name table for each 
>canvas-environment*, or libdir/objectname should be the canonical way to make 
>portable patches.  Since libdir/objectname exists and 
>"canvas-environment-local namespaces" don't, I suggest libdir/objectname as 
>the only workable approach.
>
>
>
>However, I'm only about 3/4 of the way through reading and comprehending the 
>library-loading code in Pd.  So if anyone has thoughts or suggestions I'm all 
>ears.
>
>
>-Jonathan
>
>
>* canvas-environment = all canvases which share the same $0-- basically, a 
>canvas and its [pd] subpatches (and graphs).
>
>_______________________________________________
>Pd-list@lists.iem.at mailing list
>UNSUBSCRIBE and account-management -> 
>http://lists.puredata.info/listinfo/pd-list
>
>




Maybe Hans can chime in here, because I know he wanted Pd-extended to load 
nothing by default (not even internal Vanilla objects).  In that case there 
must be some plan to do canvas-environment-local objectmakers, or some other 
mechanism to keep various abstractions and open patches from polluting each 
other's namespaces.  I don't understand how that would work, and at least on 
the face of it this seems difficult to do.  I also don't see any work done to 
that end, which is why I'm advocating for libdir/objectname as it already 
exists.

* namespace-testabstraction works because abstractions do not get registered as 
methods to the objectmaker.  So on the one hand it's a simpler system because 
you only need to control path priority (and in your test case I _think_ the 
patch's path is searched first).  But those abstractions only get loaded 
_after_ searching for external libs in any of the available paths.  So 
unfortunately you're back to gambling.  (Again, introspection objects like 
[classinfo] and [pdinfo] are extremely handy here.)

I'll post a guide to how things get loaded in the next few days.  It turns out 
to be a lot more complex than I realized.

Anyway, my idea is:
1) keep libs as they are for compatibility
2) use prefixes
3) develop new libs with an eye toward usability and consistency (easier than 
wading through old, unmaintained libs and picking and choosing stuff)

-Jonathan

On Thursday, October 16, 2014 9:44 AM, katja <katjavet...@gmail.com> wrote:
 


Thanks for bringing up this important topic. Over the years I flip-flopped 
between [library/objectname] and [declare] a few times, for reasons which I'll 
try to explain.

The first occasion where I needed [library/objectname] was when [pow~] in pd 
core had it's inlets swapped between pd versions, while [cyclone/pow~] retained 
the old inlet order and could be used for backward compatibility.

When Pd-extended stopped loading libraries by default I started using 
[library/objectname] for externals from all Pd-extended libraries, which was 
recommended practice.

Later, I came across a few cases where externals moved from one lib to another. 
For example, [flatspace/knob] became [flatgui/knob]. I feared further library 
reorganizations and abandoned the rigid [library/objectname] format. Now I 
started to appreciate the flexibility provided by [declare]. I used 
declarations like [declare -stdlib
 flatspace/knob] and [declare -stdlib flatgui/knob] in top level patches to 
make projects compatible across Pd-extended versions. Also, I use [declare] for 
binaries which are included in project directories: for my homebrew binaries, 
or selected Pd-extended binaries in projects which should run with vanilla Pd. 
It is much easier to change or add a few declarations in the top level patch 
than changing library names in all object instances in a large project.

But [declare] only adds search paths and it doesn't protect against name clash. 
I came across [bsaylor/svf~] which has different number of inlets and outlets 
than [cyclone/svf~]. Without library name, [svf~] instantiates the cyclone 
version. But when [bsaylor/svf~] was instantiated earlier in the Pd session, or 
with [import bsaylor] in the patch, the bsaylor version may be instantiated. 
Without namespace specification Pd seems to use the version that was loaded 
first within a Pd session. 

I realized that instantiating without library namespaces is a gambling. And 
this does not only apply to binary executables, but to abstractions as well. 
Imagine you share a Pd project including abstractions and binaries, and you 
publish a new version of the project, where bugs in externals were fixed or new 
features were added to abstractions. Users may want to compare versions, and 
load the old version first in a Pd session. Even when they close the old 
version of the project before loading the new version, the old executables and 
abstraction definitions are still in Pd's memory, and Pd will not load the new 
ones unless they were prefixed by a different namespace, or until Pd is 
restarted. So, the new project version may seem to still contain the old bugs!

The current solution for such issues is systematic use of lib names and 
relative paths in object or abstraction instances. For example, include 
abstractions in a subdirectory named 'abstractions', and instantiate as 
[abstractions/objectname]. Pd will then load the abstraction from the 
subdirectory in the project, even when many projects have an 'abstractions' 
subdirectory containing an abstraction with the same name. This tedious and 
inflexible approach will only work when directory structures don't change. A 
project or library structure must be well organized right from the start. If 
something changes in the structure, you may need to find all instances of a 
class in your patch(es) and redefine them. Rigid directory structures, and the 
prospect of minor changes causing heaps of maintenance work, can be an obstacle 
for future innovations in a project, or in Pd-extended as a whole.

In any case, I'm reluctant to use namespaces for each and every instance. It 
can make your patch today, and break it tomorrow. It would be super if 
[declare] would not only add a search path, but also locally enforce priority 
for it. Say I have my own implementation of [svf~] stored in the 'bin' 
subdirectory of a Pd project directory. The main patch contains [declare -path 
bin] and several instances of [svf~]. Pd should search the bin subdirectory and 
other locally declared paths for an [svf~] executable. If Pd finds the one in 
the bin subdirectory, it should interpret all instances of [svf~] as [bin/svf~] 
and instantiate them accordingly. If an object name appears in two locally 
declared paths, you would still have a name clash. But designing a single 
project directory without name clashes is not so difficult. 

How hard would it be to realize such path priority, I wonder? Currently, Pd can 
distinguish between different abstractions with the same name and relative path 
prefix, if they appear in different project directories. See attached test, 
where different versions [abstractions/namespace-testabstraction] are 
instantiated from patches in two different 'project folders'. Apparently Pd 
identifies these files with their absolute path, otherwise it could not load 
two different versions of [abstractions/namespace-testabstraction], right?

Katja



On Thu, Oct 16, 2014 at 6:21 AM, Jonathan Wilkes via Pd-list 
<pd-list@lists.iem.at> wrote:

Hi list,     Let's say I make a patch that uses the zexy and hcs libraries.  I 
want my patch to be portable to Pd-extended, Pd-l2ork, and whatever hand-rolled 
Pd-vanilla+zexy+hcs installations are sitting out there on users machines.
>
>If that is my goal, then I am going to use libdir/objectname syntax for all my 
>zexy and hcs objects in my patch.  I am going to do this because if a user 
>ever reports
 that there is a nameclash, it is very likely to be due to a bug somewhere, and 
that bug is very likely to be fixable.
>
>In fact, I will go so far as to say there is no other way to make a portable 
>patch that uses externals on the versions of Pd I described above...
>* both [import] and [declare] will happen _after_ the the default
 libs of Pd-extended/L2ork are loaded, so I can't use unprefixed object names 
with impunity (same with .pdrc).  And even if it does work, it's not 
future-proof, as someone may add an object that aliases one I'm using in my 
patch.
>* command lines flags don't ship with my patch, and they get parsed _after_ 
>the user or default prefs so again, I'm just hoping to get lucky.
>
>* even with Pd-extended's current "load-nothing" dogma, all possible object 
>names are stored globally (in what looks like both prefixed and unprefixed 
>forms, like "foo/bar" and "bar").  So if my patch is an abstraction and I 
>don't use libdir/objectname syntax for the object names within it, all I can 
>do is hope the user hasn't already loaded something that aliases one of my 
>objects.  (And again, not future-proof)
>
>
>
>I see only two possibilities, then: either the community changes the core 
>functionality so that there is a local object-name table for each 
>canvas-environment*, or libdir/objectname should be the canonical way to make 
>portable patches.  Since libdir/objectname exists and 
>"canvas-environment-local namespaces" don't, I suggest libdir/objectname as 
>the only workable approach.
>
>
>
>However, I'm only about 3/4 of the way through reading and comprehending the 
>library-loading code in Pd.  So if anyone has thoughts or suggestions I'm all 
>ears.
>
>
>-Jonathan
>
>
>* canvas-environment = all canvases which share the same $0-- basically, a 
>canvas and its [pd] subpatches (and graphs).
>
>_______________________________________________
>Pd-list@lists.iem.at mailing list
>UNSUBSCRIBE and account-management -> 
>http://lists.puredata.info/listinfo/pd-list
>
>
_______________________________________________
Pd-list@lists.iem.at mailing list
UNSUBSCRIBE and account-management -> 
http://lists.puredata.info/listinfo/pd-list

Reply via email to