> On Jul 9, 2024, at 1:15 PM, Larry Garfield <la...@garfieldtech.com> wrote:
> On Sat, Jul 6, 2024, at 1:12 AM, Mike Schinkel wrote:
>> WordPress differs from the apps many *(most?)* developers on PHP
>> Internals work with in the following way:
>>
>> WordPress = *User-managed app*
>> Most = *Developer-managed apps*
>>
>> In a* Developer-Managed app* developers choose which 3rd party
>> functionality will be incorporated into their sites whereas with a
>> *User-managed app* users choose which 3rd party functionality will be
>> incorporated into their site. And that is the KEY difference.
>>
>> So I am wondering if we can get people on this PHP Internals list who
>> dismiss the needs of WordPress developer BECAUSE it is WordPress to
>> recognize that User-Managed apps ARE a class of PHP applications have
>> needs that **deserve** to be addressed?
>> *
>> *
>> Two (2)* unmet needs of User-Managed apps *that *"standard" *PHP
>> currently does not address come to mind:
>>
>> User-managed apps needs to be able to handle both:
>> *
>> *
>> 1. *User-added add-ons* *("plugins" in WordPress, "modules" in Drupal)
>> *that have conflicting dependencies, and
>>
>> 2. *Add-on directory structures *that do not follow a PSR-4 directory
>> hierarchy.
>>
>> As for #2, even if those apps could rearchitect their existing
>> directory structure they cannot realistically be expected to do with
>> because of the huge BC issues their users would experience.
>>
>> And newly created User-managed apps may still find that a PSR-4
>> directory structure is not in the best interest of their project or
>> their users. To elaborate, PSR-4 generally assumes that ALL code goes
>> into ONE hierarchy and that any and all code that will be autoload gets
>> placed in that hierarchy.
>>
>> But with add-ons it makes a lot more sense to have the entire add-on
>> contained in its own add-on directory. This is exactly where PSR-4
>> breaks down with respect to User-managed apps.
>>
>> Sure, you can have multiple PSR-4 autoloader root directories, but that
>> does not scale well to websites with a large number of add-ons as many
>> WordPress sites I worked on used. Some had over 100 plugins. With a
>> hierarchy of autoloader maps that Michael Morris is proposing WordPress
>> could collect up all the maps and create one map every time a plugin is
>> added, updated or deleted.
>>
>> </epiphany>
>
> I am going to jump in here on this point specifically, because it seems to be
> a mix of genuinely insightful observation (though not unique) and uninformed
> FUD.
>
> Some context: I haven't seriously used Wordpress in, ever. However, I was a
> Drupal lead developer for many years, and wrote, among other things, Drupal's
> DBAL, Drupal's first autoloader, Drupal's PSR-3 implementation, was involved
> in Drupal's file organization guidelines for Drupal 8+ (when Drupal adopted a
> PSR-0/4 autoloader), and led the Drupal 8 "Modernize all the things" effort.
> So I do have some non-trivial experience in this area.
Yes. I remember that about you as well, Crell.
> First, you're correct that there is an architectural difference between
> "projects that assume the owner has CLI access" and those that do not. You
> are also correct that most of the Internals crowd comes from the former.
>
> However, I don't think it's fair to say that's why Internals folks "dismiss"
> Wordpress generally.
No, I said you dismiss the concerns of user-managed apps *because* you dismiss
WordPress, which we know because you just gave a list of why you dismiss
WordPress.
What I was hoping was to make the distinction that user-managed apps —
regardless of if they are WordPress, are some other user-managed app — have
needs that are unfair to demonize simply because you have contempt for the
poster-child of user-managed apps.
> And yes, I fully agree that any module/package/thing needs to take into
> account the needs of both types of projects. As Rowan has repeated, that
> means keeping the impact of any changes minimal, so that the Composer
> ecosystem and WP ecosystem can TYPO3 ecosystem can build their own tooling on
> top of it.
>
> You'll note I did not list Drupal there. That's because modern Drupal is
> composer-based, and has been for many years. I was the one that pushed hard
> for adopting Composer, its autoloader, and PSR-4 for Drupal 8 in the first
> place. While much of the transition happened after I left the project, the
> groundwork is over a decade old. Composer is the preferred way to use
> Drupal, and to install Drupal modules.
Interesting regarding Drupal 8 and beyond.
The ONE place where I will accept your claims that my comments have been
"uninformed" and/or "ignorant" has been WRT the details of using Drupal 8+ and
beyond. BTW, based on your comments about Drupal and my subsequent analysis it
turns out that Drupal is now much more of a developer-managed app than a
user-managed app.
I started my PHP career on Drupal in 2008 and worked with it for two years
until the work dried up and I had to switch to WordPress. I later found a
paying Drupal project but after two weeks I fired myself because working with
Drupal was such a pain compared to working with WordPress (long story, but
basically it came down to theming architectures — Drupal's was coupled vs.
WordPress' being decoupled — making Drupal much harder to theme when compared
to WordPress.)
Interesting that Drupal 8+ fully embraced PSR-4. I was aware of their embrace
of Symfony in Drupal 8 and even wrote a blog post[1] right at 10 years ago on
how that decision would result in a sizable decline in Drupal's user base over
time. I drew on my experience watching the precipitous decline of Visual Basic
when Microsoft transitioned from Visual Basic 6 to VB.NET — and history shows I
was right about Drupal, at least in predicting the sizable market decline.
Visual Basic was once the most widely-used programming language in the world.
But after moving to VB.NET it fell into almost complete obscurity.
In the past decade Drupal's marketshare of CMS has fallen significantly, from
around 7.2% of CMS in Jan 2013 to 1.4% in July 2024. Of course there are far
too many factors to say Drupal's decline was in-fact because of their embrace
of complexity but it certainly correlates. A more charitable analysis is that
Drupal chose to go upmarket to the enterprise which needed that complexity, but
that would also imply that the rest of the market moved on from Drupal likely
because the complexity no longer met their needs. #fwiw
> So the line is not as hard between those two models as you might think.
>
>> PSR-4 generally assumes that ALL code goes
>> into ONE hierarchy and that any and all code that will be autoload gets
>> placed in that hierarchy.
>
> This is flatly untrue, and belies a considerable ignorance about how PSR-4
> and Composer work.
No, It shows that you ignored my use of the word "generally."
And the only thing it belies is that you chose the *least* charitable
characterization of my knowledge of how PSR-4 and Composer work that you could
have, instead of charitably presuming that maybe I actually do have valuable
insight you do not have.
I do not presume the same of you — knowing you have knowledge I do no have in
various areas. I do however make exception for those areas where you both claim
a lack of experience and then demonstrate a lack of knowledge, as I reveal in
sections below.
Btw, if I revised my words to avoid being imprecise with "generally" I would
write that PHP encourages **too many** autoloader callbacks, and the way most
of the PHP community tries to manage that is to condense all those independent
groups of code each with their own autoloader callback, down into as few PSR-4
hierarchies that with as few autoloader callbacks as possible.
Just like procedural __get and __set magic methods seemed like a good idea at
the time but now declarative property hooks will supplant them so to I argue
that procedural autoloaders — which seemed like a good idea at the time — have
the same types of issues as procedural magic methods and that moving to a
declarative form for autoloading would improve PHP significantly.
> PHP supports multiple autoloader callbacks, and has for over 15 years. You
> absolutely can register multiple if you'd like, using whatever logic you
> like. PHP will call each one in turn until the class is loaded.
Of *course* PHP supports multiple autoloader callbacks. I know, I had to work
with many autoloaders over the years of developing WordPress sites and plugins.
But PHP supporting multiple autoloader callbacks is far from the benefit you
are insinuating when they proliferate.
Consider a user-managed WordPress site — and I worked on many like this after
the site had gotten so slow the user had to hire a developer — where the user
has 50+ active plugins on the site. And then consider that up to 51 autoloaders
had been registered, or on average 25 autoloaders run EVERY new symbol load
attempt which results in a performance and complexity nightmare.
So if there is ignorance in this thread, it is the blissful ignorance of the
issues that too many autoloader callbacks can cause. And that ignorance from
someone who directly claims "I haven't seriously used Wordpress in, ever."
So please let us both acknowledge what each of us *do* know vs. what we do not,
and then accept that the *other* likely has insight into the areas the opposite
person does not know.
> All PSR-4 does is specify a directory structure that makes a common
> autoloader stupidly simple to write. It's just a few lines long. But you
> can already do any logic you like for an autoloader. PHP doesn't care.
>
> However, nothing precludes you from registering multiple autoloaders, all
> using PSR-4, all using a different path root. That has been trivially simple
> to do since 2009. (OK, it was PSR-0 at the time, but the implications here
> are the same.) So your statement above about "all code goes into one
> hierarchy" is simply flat out false.
>
> Of course, as you note, registering lots of separate autoloaders has a
> performance impact. That is true. Which is why I don't think anyone
> actually does that.
No one, well, except everyone who writes a plugin for WordPress.
And in production, except everyone who runs multiple plugins, which is
practically everyone who runs a WordPress site.
Again, your admitted lack of ever using WordPress is showing your lack of
experience on this topic.
> Composer, for instance, registers a single autoloader only. That autoloader
> internally tracks many dozens of PSR-4 roots (one for each package, sometimes
> two per package), as well as files that will get force-loaded when the
> autoloader is registered, plus generated classmaps.
>
> Using class maps, you can put a hundred classes in one file and composer can
> handle that *today*. That has always been possible. That no one does so is
> a sign that there's little reason to do so in most cases.
What is your evidence of your claim that "no one does so?"
I just pulled up the source for one my most recently WordPress clients — the
first one I looked at since your email — and searched for
`spl_autoload_register(" in the `/wp-content/plugins/` directory. I found 43
different autoloaders being registered. Here is a screenshot of that:
https://postimg.cc/TyWjW6Px
So clearly yes, "people do so." And many of them do. True, they are not
inside the bubble where you reside, a bubble where no other residents do that
either. But outside that bubble the practice is widespread as there is not a
better way that is also realistic for the individual site builder to choose.
> In fact, if you use an optimized/dumped autoloader, then Composer simply
> builds an internal giant lookup table of what class maps to what file. PSR-4
> is then *completely irrelevant* at runtime. It's already one giant O(1)
> lookup map. That can be done *today*. That *is* done today.
Yes, it can be done today. It *is* done today. By. Developer. Managed. Apps.
> But what about systems like Drupal, that don't put code in `/vendor/`?
> Drupal ties directly into Composer via its API,and has done so for a decade.
> Drupal, a "user-managed application" as you describe it, has Composer baked
> in at a core level.
>
> It looks like the integration has evolved considerably since I was last
> involved, but have a look at:
>
> https://git.drupalcode.org/project/drupal/-/tree/11.x/composer
Like I previously said, if the team managing a project does not care about BC —
as Drupal apparently did not in its transition to Composer — then tying
Composer into the core is not a problem.
But if they DO care about BC — as WordPress very much cares about — then fully
integrating Composer into an existing user-managed app is a non-starter. And
as Drupal's decline in user base over the past decade shows, doing so also
which carries a high level of risk of shedding user base, too.
> As of when I last looked at it (around 2016 or so), Drupal registers its
> module code roots with Composer directly, and then Composer takes over from
> there and integrates Drupal's code into its own indexes. There is still only
> one single autoloader registered with PHP. This is entirely fine.
Yes, as the Drupal module Ludwig[3] explains. It can be managed by a Drupal
module, but even its own authors do not recommend doing so, stating
(parenthetical mine): "However, please note that Composer (on the CLI) is
highly recommended whenever possible!"
Which again, means current Drupal is now effectively a developer-managed app.
> I would encourage you to do your research before speaking
> pseudo-authoritatively on this topic, as you clearly are mis-stating both the
> problem and the tools involved today.
Back at you; see the above.
> What Drupal does not do is address the "different dependency version"
> question. And neither does Wordpress. Or TYPO3. Or any other project.
> Because that's a core PHP limitation.
That too.
> PHP works very very differently. PHP has a single global list of symbols.
> (Well, two, for classes and functions.) Namespaces are just syntax sugar
> over very-long-names, nothing more. There is no "local symbol table," so
> having different local symbol tables point to different code blocks using the
> same name is not even conceivable.
>
> If you want to change that, and give PHP multiple local symbol tables, then
> autoloading... is utterly irrelevant. The question there is "how can we
> introduce local symbol tables in the engine without requiring 10 million
> developers to rewrite the file header of 1 billion PHP files across the
> world?"
You have not shown how having a local symbol table "requires 10 million
developer to rewrite header files..."
As I envision it, no such thing would be required.
However, maybe I just do not understand enough about how PHP internals
currently works? How about explaining your assertion that a local symbol table
requires a rewrite of all PHP file's header?
> Honestly, I'm not convinced its even possible. Someone with more engine
> knowledge than I could be able to find away, maybe, but I am skeptical. If
> it's even possible, I suspect it would be an absurdly large amount of work
> and necessarily include many hard BC breaks.
>
> If you'd like to prove me wrong, go for it. But that's the problem to
> address. Debating file paths is about four steps down the line before it's
> even relevant. And even then... if you can't make a PSR-4-organized package
> (of which there are several hundred thousand) slot into that new model
> comfortably with zero effort on the part of the package author, it's doomed.
Now is a good time to summarize the request that you claim is "4 steps down the
line" that I (and maybe Michael Morris?) wants to get PHP to incorporate, at
least as I see it.
The request is to add class maps with a PHP-standardized format into PHP core
so that when a library of code needs to register classes to be autoloaded they
can contribute to a cascading of class maps where ONE internal function checks
the single union of all class maps for a mapped symbol before turning over
control if no such symbol is found in the map to the procedural autoloaders
that are currently available. That way *any* PHP code could register its own
class map without having to get the core app to rearchitect itself to allow
that to happen.
It is really as simple as that, and the duplicate symbol issue is orthogonal,
but relating them can see synergy in implementation.
> So please, spare us the ill-informed descriptions of how you think
> autoloaders work, when you have demonstrated you do not know how they work.
>
> Spare us the litany of complaints about PSR-4 when you have demonstrated you
> don't know what PSR-4 says.
No, spare me your bad-faith accusations of ignorance — I demonstrated my
knowledge above — and your claims that "nobody is doing" exactly what I
illustrated above that they are doing.
> Spare us the gnashing of teeth about how hard it is to use a Wordpress plugin
> that hasn't been updated in 10 years with a modern plugin because the former
> is still using a 10 year old abandoned version of some library, when that's
> not PHP's problem, that's a Wordpress maintenance problem.
Again, showing that your contempt of WordPress is allowing you to dismiss the
needs of all user-manage apps as being "a maintenance problem."
> If you want to move this effort forward, here's your todo list:
>
> 1. Do some research in the engine to determine if local symbol tables are
> even possible without rewriting the engine.
Will do.
> 2. Work through the highly complex logic of handling three layer overlapping
> transitive dependencies in a diamond pattern with conflicting version
> requirements.
Why? I do not see that as a problem that needs to be addressed here for PHP
core to solve as it is too time-consuming to be resolved on load. It either
needs to be handled by:
1.) A tool like Composer, which already throws an error when such conflicting
requirements occur,
2.) A tool like WordPress' plugin manager, which will just throw an error in a
sandbox and let the user or site administrator resolve the issue, or
3.) PHP itself which would just throw an error when such a conflict emerges and
lets the developer figure it out.
Feels like you are just nerd-sniping here.
> 3. Investigate the performance impact of maintaining multiple versions of the
> same code in memory at once, when the order they get loaded will vary by
> request.
More nerd-sniping.
By the same token I could say to you "Investigate the performance impact of
maintaining a large number of autoloaders in the same codebase" — which I
illustrated is happening — except you have the status-quo on your side.
Performance impact due to "maintaining multiple versions of the same code in
memory at once" is no different than the performance impact of maintaining
multiple versions of equivalent code in memory at once that have had their
namespaces rewritten. That is what those who have been forced to solve the
same-symbol problem have been doing in userland. Most others have just given up
on running two plugins when their dependencies conflict.
It is not up to PHP to address when people load too much code into memory other
than for PHP to fail to continue running. Yes, if people duplicate *exact*
same code that should be addressed — ideally by Composer or the app itself —
but if different versions cause performance problems its the site builder's
responsibility to address, not PHP.
> 4. Think through how you'd support *both* composer-based and "user managed"
> applications with such a model, especially projects that are already
> architecturally a decade out of date (like Wordpress).
Already done. I explained how an `spl_autoload_map()` would do exactly that
above.
> When you have a proven that it's even possible to have multiple local symbol
> tables, we can talk. Until then, please spare us.
My one useful takeaway from your email — except that I already knew that — was
the need to figure out how PHP can handle multiple symbol tables. Beyond that,
your take your own advice and spare us (me) from your contempt and
condescension as they are not good looks on anyone.
-Mike
[1] https://mikeschinkel.com/2014/the-decline-of-drupal-or-how-to-fix-drupal-8/
[2] https://w3techs.com/technologies/history_overview/content_management/ms/y
[3] https://www.drupal.org/project/ludwig
