Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
--+
 Reporter:  asn   |  Owner:  (none)
 Type:  defect| Status:  closed
 Priority:  Very High |  Milestone:  Tor:
  |  0.3.2.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:  fixed
 Keywords:  prop224, tor-hs, review-group-23  |  Actual Points:
Parent ID:| Points:
 Reviewer:|Sponsor:
--+
Changes (by nickm):

 * status:  merge_ready => closed
 * resolution:   => fixed


Comment:

 merging!  Please close the other tickets that are fixed by this branch.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
--+
 Reporter:  asn   |  Owner:  (none)
 Type:  defect| Status:
  |  merge_ready
 Priority:  Very High |  Milestone:  Tor:
  |  0.3.2.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  prop224, tor-hs, review-group-23  |  Actual Points:
Parent ID:| Points:
 Reviewer:|Sponsor:
--+

Comment (by asn):

 OK. Pushed another commit which tests that upload_descriptor_to_all()
 is in synch with pick_hsdir_v3() for various scenarios. This should give
 some more confidence and also unittest `pick_hsdir_v3()`.

 I think this is good to merge now!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
--+
 Reporter:  asn   |  Owner:  (none)
 Type:  defect| Status:
  |  merge_ready
 Priority:  Very High |  Milestone:  Tor:
  |  0.3.2.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  prop224, tor-hs, review-group-23  |  Actual Points:
Parent ID:| Points:
 Reviewer:|Sponsor:
--+

Comment (by nickm):

 Looks good to me.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
--+
 Reporter:  asn   |  Owner:  (none)
 Type:  defect| Status:
  |  merge_ready
 Priority:  Very High |  Milestone:  Tor:
  |  0.3.2.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  prop224, tor-hs, review-group-23  |  Actual Points:
Parent ID:| Points:
 Reviewer:|Sponsor:
--+

Comment (by asn):

 Addressed Nick's review! Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  merge_ready
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+

Comment (by nickm):

 Spec branch merged.  Thanks for the extra diagrams, dgoulet!  Now on to
 the code...

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  merge_ready
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+

Comment (by asn):

 Gitlab MR can be fond here: https://oniongit.eu/asn/tor/merge_requests/6

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  merge_ready
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by asn):

 * status:  needs_review => merge_ready


Comment:

 Also pushed code branch at `bug23387` which is ready for Nick's eyes.
 Please use the spec branch above as a companion to this patch when
 reviewing. Commit messages + the spec branch should provide enough
 motivation for the code changes.

 Furthermore, the HS reachability test is pretty awesome and gives good
 guarantees of the reachability enhancements we designed.

 If you have any questions don't hesitate to ask them since this is a non-
 trivial change.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  needs_review
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+

Comment (by asn):

 Pushed spec patch at `bug23387`. This is needed to better understand the
 code patch that will soon be posted here.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  needs_review
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+

Comment (by asn):

 Yo David,

 Seems like after our discussion about comment:4 (c) yesterday, we
 discovered a
 major reachability issue a few days before the freeze. Let's see how we
 can fix
 this.

 First of all, we should agree on what's the problem is. Let's consider
 `bug23387_asn` from #23387 to be the base here, as I think the valid_after
 fixes
 are essential. So in `bug23387_asn`, the main unsolved problem is (c) from
 comment:4 in #23387, aka scenario 6 from your `bug23387_032_02^` branch
 (minus
 top commit). That's the scenario where the HS has a newer consensus than
 the
 client, and the HS just moved to the next TP but the client is still stuck
 on
 the old one, and the service is not in any sort of overlap mode so it
 doesn't
 cover the old TP anymore.

 {{{
   +--+
   |  |
   | 00:00  12:00   00:00   12:00   00:00   12:00 |
   | SRV#1  TP#1SRV#2   TP#2SRV#3   TP#3  |
   |  |
   |  $==|---$===|---$===||
   |^ ^   |
   |C S   |
   +--+
 }}}

 Am I right, that this is the *only* problem right now? I'm a bit confused
 because I see an XXX in scenario 2 of your `bug23387_032_02^`, but I think
 that
 should be OK, and the problem was created by the top commit of that
 branch? I
 also see you citing scenario 2 in your bug23387_032_03 as the problem
 point.

 If I understand things correctly, I have two suggestions here:

 a) Short-term client-side solution: Design the shitty fallback system we
 discussed yesterday, where if the client fails to fetch the descriptor
 with
 SRV#1/TP#1, it will do a fallback fetch with SRV#2/TP#2.

 b) Medium-term service-side solution: Implement a ''reverse'' overlap
 period,
 where the service will keep on publishing the old descriptor even tho it
 has
 exited the overlap period, so that it covers for clients with old
 consensuses (so the old descriptor will still be with TP#1/SRV#1 in the
 example above)

 You can think of the normal overlap period, as a way to cover for clients
 that have a consensus in the future (after the TP rotates), and the
 reverse
 overlap period as a way to cover for clients that are in the past (before
 the TP rotates).

 I think your bug23387_032_03 was not doing the reverse overlap period
 concept, and was just creating the 'next' descriptor earlier in time. Or
 am
 I wrong?

 We don't need to do everything right now, but we should make sure that any
 future changes won't cause desynch between different versions of
 client/service.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  needs_review
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by asn):

 * status:  needs_revision => needs_review


Comment:

 OK please check latest `bug23387_asn`. It also contains a unittest that
 should provide some more assurance of these weird edge cases.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  needs_revision
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+

Comment (by asn):

 OK based on the comment above, I present a less-radical branch at
 `bug23387_asn`. This branch does not change the way overlap periods are
 detected, but still only rotates descs when exiting overlap periods. It
 also uses valid_after time when computing hsdir indices, and time period
 nums. Seems to work fine in my testing so far.

 I'm now writing a unittest that will test various cases with timing on the
 client and service. It's pretty advanced so I'll probably finish it
 tomorrow or the day after.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  needs_revision
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by asn):

 * status:  needs_review => needs_revision


Comment:

 Dumping some comments from my review and testing here:

 a) There is no commit msg in `daf72124` of why we don't like the previous
 approach of figuring out whether overlap period just started/ended. I
 don't really like this new approach of checking `consensus->valid_after ==
 tp_start_time`, since there is no guarantee that Tor will get all the
 consensuses: it can skip a consensus, and if it skips the right one then
 we will never detect an overlap change.

 This logic also caused an assert in my testing since we do:
 {{{
 if (!overlap_mode_is_active && !overlap_mode_just_ended) {
   tor_assert(!service->desc_next);
 }}}
 and I accidentally triggered that because I suspended my laptop for 6
 hours, which means that Tor missed the overlap-changing consensus, so it
 never actually rotated descriptors. Not sure how to fix this issue. I
 think the old logic was more robust; not sure why we ditched it.

 b) Also seems like we still need a `if (service->desc_next)` guard before
 `rotate_service_descriptors()` since it seems that it can go there with
 `desc_next` being NULL and it will overwrite `desc_current` with NULL.

 c) Also should we change all places we get the current time period num to
 use the consensus valid after? To reduce desynch possibilities?

 c) This final comment is not related to this branch, but I think there is
 a case of HSDir desynch that ''we don't handle in the current prop224
 spec'': Consider an HS with 13:00 consensus, and a client with 11:00
 consensus. This means that the client is in time period #N and the HS is
 in time period #N+1. The HS thinks it's in non-overlap mode so it only
 uploads the current descriptor for time period #N+1. Meanwhile, the client
 always downloads the current descriptor so it will attempt to download the
 descriptor for time period #N which could have expired in theory.

 This happens because we have an overlap period covering TP #N+1 from TP
 #N, but we don't have one that covers TP #N from TP #N+1.

 I'm dumping these thoughts here. Depending on my schedule today I might or
 might not be able to fix these on time.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  needs_review
 Priority:  Very High|  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by dgoulet):

 * status:  new => needs_review
 * priority:  High => Very High


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  new
 Priority:  High |  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+

Comment (by dgoulet):

 See branch: `bug23387_032_01`

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #23387 [Core Tor/Tor]: prop224: HSdir index desynch between client and service (was: prop224: Time period desynch between client and service)

[Tor Bug Tracker & Wiki]

#23387: prop224: HSdir index desynch between client and service
-+
 Reporter:  asn  |  Owner:  (none)
 Type:  defect   | Status:  new
 Priority:  High |  Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  prop224, tor-hs  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Description changed by asn:

Old description:

> David found his client unable to connect to his service. Apparently, they
> compute different hsdir indices because the time period num is not
> synched:
> {{{
> service side: Sep 01 12:36:59.000 [info] hs_get_responsible_hsdirs():
> Finding responsible HSDirs for blinded key
> mCs1ObO+OmLpjYy36SWX3tv5rV9S2P6/BNo8rVjUy0g, time period number 17411 and
> for next period
> }}}
> {{{
> client side: Sep 01 08:23:34.000 [info] hs_get_responsible_hsdirs:
> Finding responsible HSDirs for blinded key
> 3vsekKmh3WYjr85reqpS6Ts2xqJxSSgZHxgX/Jp1FK0, time period number 17410 and
> for current period
> }}}
>
> Theory: We use `time(NULL)` as the time in `node_set_hsdir_index()`
> whereas we use the live consensus `valid-after` in
> `rotate_all_descriptors()`. This can cause desynch within the same tor
> instance. We should probably use the live consensus `valid-after` in all
> cases to have a common point of reference, and avoid problems with clock
> skews.

New description:

 David found his client unable to connect to his service. Apparently, they
 compute different hsdir indices, since it was 12:20UTC (non-overlap
 period) and the live consensus had valid-after at 11:00UTC (overlap
 period). Apparently something got confused.

 Theory: We use `time(NULL)` as the time in `node_set_hsdir_index()`
 whereas we use the live consensus `valid-after` in
 `rotate_all_descriptors()`. This can cause desynch within the same tor
 instance. We should probably use the live consensus `valid-after` in all
 cases to have a common point of reference, and avoid problems with clock
 skews.

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs