Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:  .1
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor31-can
-+-

Comment (by LarryBitcoin):

 ack fb0e8966f00c4349602594f312dae60cad2ea11d

 tested both as a library and as a standalone binary and with and without
 lto with the dirauth module disabled and everything builds correctly on
 linux/windows/osx/freebsd/android with both gcc/clang where available.

 Great work! thank you!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:  .1
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor31-can
-+-

Comment (by LarryBitcoin):

 Will test now and get back - thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:  .1
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor31-can
-+-

Comment (by nickm):

 I've added another commit to the branch; does it fix the remaining
 problems?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:  .1
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor31-can
-+-

Comment (by LarryBitcoin):

 I am not sure if the library failures are lto related or not but i tried
 to build on osx with lto off and the dirauth module disabled



 {{{
 Undefined symbols for architecture x86_64:
   "_dirserv_add_descriptor", referenced from:
   _init_keys in libtor-app.a(router.o)
   "_dirserv_add_multiple_descriptors", referenced from:
   _directory_handle_command_post in libtor-app.a(dircache.o)
   "_dirserv_add_own_fingerprint", referenced from:
   _init_keys in libtor-app.a(router.o)
   "_dirserv_load_fingerprint_file", referenced from:
   _do_hup in libtor-app.a(main.o)
   _init_keys in libtor-app.a(router.o)
   "_dirserv_orconn_tls_done", referenced from:
   _connection_or_client_learned_peer_id in libtor-
 app.a(connection_or.o)
   "_dirserv_router_get_status", referenced from:
   _nodelist_set_routerinfo in libtor-app.a(nodelist.o)
   "_dirserv_set_node_flags_from_authoritative_status", referenced from:
   _nodelist_set_routerinfo in libtor-app.a(nodelist.o)
   "_keypin_load_journal", referenced from:
   _run_tor_main_loop in libtor-app.a(main.o)
   "_keypin_open_journal", referenced from:
   _run_tor_main_loop in libtor-app.a(main.o)
 ld: symbol(s) not found for architecture x86_64
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:  .1
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor31-can
-+-

Comment (by LarryBitcoin):

 Nice one!

 I tried your branch and it looks like it fixes the lto binary

 The lto tor in library mode seems still broken (but i wonder now if it's
 my build config or not)

 {{{
 tor/src/core/libtor-app.a(main.o):main.c:function run_tor_main_loop:
 error: undefined reference to 'keypin_load_journal'
 tor/src/core/libtor-app.a(main.o):main.c:function run_tor_main_loop:
 error: undefined reference to 'keypin_open_journal'
 tor/src/core/libtor-app.a(main.o):main.c:function do_hup: error: undefined
 reference to 'dirserv_load_fingerprint_file'
 tor/src/core/libtor-app.a(connection_or.o):connection_or.c:function
 connection_or_client_learned_peer_id: error: undefined reference to
 'dirserv_orconn_tls_done'
 tor/src/core/libtor-app.a(nodelist.o):nodelist.c:function
 nodelist_set_routerinfo: error: undefined reference to
 'dirserv_router_get_status'
 tor/src/core/libtor-app.a(nodelist.o):nodelist.c:function
 nodelist_set_routerinfo: error: undefined reference to
 'dirserv_set_node_flags_from_authoritative_status'
 tor/src/core/libtor-app.a(router.o):router.c:function init_keys: error:
 undefined reference to 'dirserv_add_own_fingerprint'
 tor/src/core/libtor-app.a(router.o):router.c:function init_keys: error:
 undefined reference to 'dirserv_add_descriptor'
 tor/src/core/libtor-app.a(router.o):router.c:function init_keys: error:
 undefined reference to 'dirserv_load_fingerprint_file'
 tor/src/core/libtor-app.a(dircache.o):dircache.c:function
 directory_handle_command_post: error: undefined reference to
 'dirserv_add_multiple_descriptors'
 clang: error: linker command failed with exit code 1 (use -v to see
 invocation)
 ninja: build stopped: subcommand failed.

 }}}


 I'll run more tests locally and get back

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:  .1
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor31-can
-+-

Comment (by nickm):

 Replying to [comment:9 teor]:
 > Should we run the --disable-module-dirauth CI build with -O0, to catch
 errors like this earlier?

 I just made #31560 for that. :)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:  .1
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor31-can
-+-

Comment (by teor):

 Should we run the --disable-module-dirauth CI build with -O0, to catch
 errors like this earlier?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:  .1
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor31-can
-+-
Changes (by nickm):

 * status:  accepted => needs_review
 * sponsor:   => Sponsor31-can
 * actualpoints:   => .1


Comment:

 Fix in my branch `ticket31552_041`; pull request at
 https://github.com/torproject/tor/pull/1273

 (This is 31-can because it is followup from a modularization ticket.)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  accepted
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by nickm):

 Bug confirmed.

 What does the compiler do when it sees:
 {{{
void function();
// ...
if (0) {
  function();
}
 }}}
 but function() is never actually defined?

 Usually, the compiler will remove the code in the if (0) block, so that by
 the time the linker sees it, the call to function() is no longer there.

 But there are at least two circumstances in which the call to function()
 might last long enough for the linker to see that function() doesn't
 exist.

 1. When building with LTO, the call to function() might not get removed
 until later on.
 2. When building with -O0, the call won't get removed at all.

 I've reproduced these warnings by building Tor with -O0.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  nickm
 Type:  defect   | Status:
 |  accepted
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by nickm):

 * owner:  (none) => nickm
 * status:  new => accepted


Comment:

 Ah, this is for android!

 I suspect that the reason I can't reproduce this issue locally is that it
 has something to do with the behavior of LTO on the android toolchain
 specifically.  This is confirmed by #31301, which might be the same issue.

 Looking at the code: It seems that the functions that the compiler is
 complaining about are only called within blocks that, after compilation
 with --disable-module-dirauth, become "if (0) { ... }".  So I can believe
 that they are totally removed before linking sometimes, but not all the
 time.  The fix should be simple enough if this is the case.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  (none)
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by LarryBitcoin):

 I can confirm that binary build with lto off and --disable-module-dirauth
 set builds fine.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  (none)
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by LarryBitcoin):

 I have two projects that failed with the latest version: one that uses tor
 as a library in process rather than as an independent executable/process,
 and one that uses tor as its own 'vanilla' independent binary/process.
 Both worked fine with 0.4.0.5 and lto on and disable-module-dirauth
 set and don't with 0.4.1.5


 I tried the library without lto and it still failed


 {{{
 tor/src/core/libtor-app.a(main.o):main.c:function run_tor_main_loop:
 error: undefined reference to 'keypin_load_journal'
 tor/src/core/libtor-app.a(main.o):main.c:function run_tor_main_loop:
 error: undefined reference to 'keypin_open_journal'
 tor/src/core/libtor-app.a(main.o):main.c:function do_hup: error: undefined
 reference to 'dirserv_load_fingerprint_file'
 tor/src/core/libtor-app.a(connection_or.o):connection_or.c:function
 connection_or_client_learned_peer_id: error: undefined reference to
 'dirserv_orconn_tls_done'
 tor/src/core/libtor-app.a(networkstatus.o):networkstatus.c:function
 routers_update_status_from_consensus_networkstatus: error: undefined
 reference to 'dirserv_should_launch_reachability_test'
 src/core/libtor-app.a(nodelist.o):nodelist.c:function
 nodelist_set_routerinfo: error: undefined reference to
 'dirserv_router_get_status'
 tor/src/core/libtor-app.a(nodelist.o):nodelist.c:function
 nodelist_set_routerinfo: error: undefined reference to
 'dirserv_set_node_flags_from_authoritative_status'
 tor/src/core/libtor-app.a(routerlist.o):routerlist.c:function
 router_add_to_routerlist: error: undefined reference to
 'authdir_wants_to_reject_router'
 tor/src/core/libtor-app.a(routerlist.o):routerlist.c:function
 update_consensus_router_descriptor_downloads: error: undefined reference
 to 'dirserv_would_reject_router'
 tor/src/core/libtor-app.a(router.o):router.c:function init_keys: error:
 undefined reference to 'dirserv_add_own_fingerprint'
 tor/src/core/libtor-app.a(router.o):router.c:function init_keys: error:
 undefined reference to 'dirserv_add_descriptor'
 tor/src/core/libtor-app.a(router.o):router.c:function init_keys: error:
 undefined reference to 'dirserv_load_fingerprint_file'
 tor/src/core/libtor-app.a(dircache.o):dircache.c:function
 directory_handle_command_post: error: undefined reference to
 'dirserv_add_multiple_descriptors'
 clang: error: linker command failed with exit code 1 (use -v to see
 invocation)
 ninja: build stopped: subcommand failed.
 }}}

 I used the following config (static openssl, static libevent, static
 zlib).


 {{{

 CONFIGURE_ARGS="--prefix=${BUILD_ROOT}/tor/build --disable-system-torrc
 --disable-asciidoc --enable-pic --enable-static-openssl \
 --enable-static-libevent --enable-static-zlib --with-
 openssl-dir=${BUILD_ROOT}/openssl/build \
 --with-libevent-dir=${BUILD_ROOT}/libevent/build --with-
 zlib-dir=${BUILD_ROOT}/zlib/build \
 --disable-system-torrc --disable-systemd --disable-zstd
 --disable-lzma --disable-largefile ac_cv_c_bigendian=no --disable-
 unittests --disable-tool-name-check --disable-rust --disable-module-
 dirauth"


 }}}

 without --disable-module-dirauth it works


 The other use of tor is in its' binary form and that definitively fails
 with lto on, see https://travis-
 ci.org/greenaddress/bitcoin_ndk/jobs/578034561

 I'm still doing some testing to confirm if it works without lto in binary
 form but as a library even without lto it breaks.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  (none)
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by teor):

 Sometimes, LTO errors happen because the function signatures are different
 in different modules.
 Maybe we should check these functions.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  (none)
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:  Tor:
 |  0.4.1.x-final
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features,  |  Actual Points:
  modules, regression|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by nickm):

 * keywords:  build, configure, features, modules => build, configure,
 features, modules, regression
 * milestone:   => Tor: 0.4.1.x-final


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #31552 [Core Tor]: --disable-module-dirauth broken (missing symbols)

[Tor Bug Tracker & Wiki]

#31552: --disable-module-dirauth broken (missing symbols)
-+-
 Reporter:  LarryBitcoin |  Owner:  (none)
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Core Tor |Version:  Tor:
 |  0.4.1.5
 Severity:  Normal   | Resolution:
 Keywords:  build, configure, features, modules  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by nickm):

 It works for me, but I se that you're maybe using link-time optimization.
 Can you say more about your build environment?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs