[Issue 16423] ModuleInfo missing when linking to static lib with classes

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P1  |P4

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

Walter Bright  changed:

   What|Removed |Added

   Keywords||objc
 CC||bugzi...@digitalmars.com

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #15 from Ketmar Dark  ---
(just a speculation) maybe ld merges identical symbols from different libraries
by default or something, and optlink doesn't.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #14 from Ketmar Dark  ---
i suspect optlink. sadly, i can't check with ms linker.

(ketmar dreaming of getting rid of optlink and moving the defaults to mingw's
binutils)

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #13 from Jacob Carlborg  ---
(In reply to Ketmar Dark from comment #12)
> funny, it doesn't cause any problems on GNU/Linux, only windows builds are
> affected.

It seems to work on all platforms except Windows. Might be something with the
linker that is different.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #12 from Ketmar Dark  ---
funny, it doesn't cause any problems on GNU/Linux, only windows builds are
affected.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #11 from Jacob Carlborg  ---
(In reply to Ketmar Dark from comment #10)
> by the way. applying 6076 broke windows builds: phobos refused to build with
> "duplicate ModuleInfo for object.d" error.

Yeah, I noticed that four months ago when the PR did not pass the tests ;)

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #10 from Ketmar Dark  ---
by the way. applying 6076 broke windows builds: phobos refused to build with
"duplicate ModuleInfo for object.d" error.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #9 from Ketmar Dark  ---
(In reply to Steven Schveighoffer from comment #8)
> But there may be code out there which expects Object.factory to work, and in
> some cases, it will not.

and there is. some of my UI parsers *depends* on the fact that class list and
factory are there. i did that exactly to avoid manual registering of UI
classes, which is highly error-prone.

that is, some features are not removed 'cause there *may* be the code that
depends on those. now there *is* the code that depends on full module/class
infos and factory. this alone should rule "let's remove object.factory" out.
let's see if real code has more weight than imaginary code. ;-)

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #8 from Steven Schveighoffer  ---
Whether we need it or not, it's currently a feature of the runtime. Do we not
care about existing code anymore?

I'm totally in favor of deprecating Object.factory, and also in favor of your
update to std.encoding that makes this issue moot for phobos at least. But
there may be code out there which expects Object.factory to work, and in some
cases, it will not.

Note, that in the example I give, including the module info does little to
increase the binary size. The ClassInfo is already present in the binary, and
therefore pulls in everything else the class may use.

Not only that, but building with the -lib switch does not work, but building
without does. This kind of inconsistency is not good for D, regardless of the
utility of the Object.factory feature. It makes no sense that the compiler is
"stupider" about trimming fat when it has full code access than the linker.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #7 from Martin Nowak  ---
(In reply to Steven Schveighoffer from comment #5)
> This is penny-wise and pound-foolish. First we need to solve megabyte sized
> template symbol names. Then we need full shared library support. The few
> kilobytes that moduleinfo adds is nothing. Trim out unused classinfo? Then
> we need to remove Object.factory method.

- Yes we need to solve the long symbol names, but they don't bloat the code
segment or affect the performance
- shared libraries were just one workaround I named (and they can be fully used
on linux/FreeBSD atm.)
- It's not the ModuleInfo or the ClassInfo, it's the fact, that this drags in
the complete tree of symbols reachable by any referenced class, i.e. the whole
module in most cases. The static constructors and class references are one of
the main reasons for our huge binaries (code segment, not file size which is
just a distribution problem).
- Object.factory is slow by design, it's in the interest of everyone to avoid
that facility if possible, and there aren't many use-cases that really require
it

If we have enough prove that it's a necessary pattern, I'd be in favor of
adding a global, synchronized, namespaced dynamic class registration facility
in phobos or druntime. That could be used for certain patterns such as
registering deserializable subclasses, factory.register!("orange.deserializer",
MyOpenSubclass).
It could also make use of a hash table to speed up instantiating, and maybe
even support non-standard constructors.
The question is really, do we really need it, and who builds it.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #6 from Jacob Carlborg  ---
(In reply to Steven Schveighoffer from comment #5)

> In fact, I actually TRY your suggestion to solve the original problem and it
> doesn't work. See the PR:
> https://github.com/dlang/phobos/pull/4744#issuecomment-241876407

No module info is generated if -lib is passed unless coverage, static
constructor , destructor or unittest calls are used. This is regardless if
classes are referenced or not.
https://github.com/jacob-carlborg/dmd/blob/204b4b3b5a2ccbc697fb4e9ae422294ee21867d2/src/glue.c#L457

I think the simplest workaround is to not use -lib but archive the object files
manually using "ar".

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

Steven Schveighoffer  changed:

   What|Removed |Added

 Status|RESOLVED|REOPENED
 Resolution|WONTFIX |---

--- Comment #5 from Steven Schveighoffer  ---
(In reply to Martin Nowak from comment #3)
> It goes against the intention of static libraries to drag unused classes and
> module into a binary, and that also goes against our permanent fight against
> giant binaries.

This is penny-wise and pound-foolish. First we need to solve megabyte sized
template symbol names. Then we need full shared library support. The few
kilobytes that moduleinfo adds is nothing. Trim out unused classinfo? Then we
need to remove Object.factory method.

> In fact there is issue 14555 and a PR
> (https://github.com/dlang/dmd/pull/4638) in the adverse direction.
> 
> If you really want to drag in all classes from a static library, one
> approach is to reference all of them from a common module.

I referenced the class info from within the module, and called that function.
Why is the linker excluding it? This seems like a real bug.

In fact, I actually TRY your suggestion to solve the original problem and it
doesn't work. See the PR:
https://github.com/dlang/phobos/pull/4744#issuecomment-241876407

> Having an explicit list of all classes kind of contradicts the purpose of
> Object.factory, but I don't see how we can easily achieve both.

We can't have Object.factory and then trim out all classinfo that aren't used
directly. But in this case, I AM using it directly, and it's STILL being
trimmed out.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #4 from Martin Nowak  ---
Another rather obvious solution is to tell the linker to include the whole
archive, sensibly named --whole-archive/--no-whole-archive for ld.

Unfortunately dmd currently reorders linker flags (see issue 15574), so the
following doesn't work.

  -L--whole-archive mystaticlib.a -L--no-whole-archive

Instead you'll have to invoke cc for linking yourself atm.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

Martin Nowak  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||c...@dawg.eu
 Resolution|--- |WONTFIX

--- Comment #3 from Martin Nowak  ---
It goes against the intention of static libraries to drag unused classes and
module into a binary, and that also goes against our permanent fight against
giant binaries.
In fact there is issue 14555 and a PR (https://github.com/dlang/dmd/pull/4638)
in the adverse direction.

If you really want to drag in all classes from a static library, one approach
is to reference all of them from a common module.
Having an explicit list of all classes kind of contradicts the purpose of
Object.factory, but I don't see how we can easily achieve both.

Another approach is to generate object files for each module and explicitly
link against all of them (the linker won't discard object files).

Dub supports this as --build-mode=singleFile IIRC.

Yet another approach is to use shared libraries instead.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

--- Comment #2 from Jacob Carlborg  ---
https://github.com/dlang/dmd/pull/6076

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

Jacob Carlborg  changed:

   What|Removed |Added

 CC||d...@me.com

--- Comment #1 from Jacob Carlborg  ---
Hmm, I actually fixed this, but only for Objective-C module info. I'll see if I
can come up with a fix.

--

[Issue 16423] ModuleInfo missing when linking to static lib with classes

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16423

Ketmar Dark  changed:

   What|Removed |Added

 CC||ket...@ketmar.no-ip.org

--