Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  closed
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:  fixed
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:  1
 Reviewer:  nickm  |Sponsor:
---+---
Changes (by juga):

 * status:  merge_ready => closed
 * resolution:   => fixed


Comment:

 Merged.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  merge_ready
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:  1
 Reviewer:  nickm  |Sponsor:
---+---

Comment (by juga):

 Replying to [comment:11 nickm]:
 > For my own curiosity: why is the time cutoff set to "the number of
 releys to measure multiplied by TIMEOUT_MEASUREMENTS (around 90mins)"?

 Well spotted. Actually when measuring 300 relays it takes around 90min. In
 the case a relay doesn't get measured in 3min, there's no need to wait
 that 300 relays take each of them 3min, it can just finish when there hasn
 not been any relay measured in that time.
 I commited a fixup to do that, tested the whole loop in the public network
 and works.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  merge_ready
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:  1
 Reviewer:  nickm  |Sponsor:
---+---
Changes (by nickm):

 * status:  needs_review => merge_ready


Comment:

 This code looks reasonable to me.  If you have tested it and works for
 you, then I'd say "merge it!" :)

 For my own curiosity: why is the time cutoff set to "the number of releys
 to measure multiplied by TIMEOUT_MEASUREMENTS (around 90mins)"?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  needs_review
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:  1
 Reviewer:  nickm  |Sponsor:
---+---
Changes (by nickm):

 * reviewer:   => nickm


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  needs_review
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:  1
 Reviewer: |Sponsor:
---+---
Changes (by juga):

 * status:  needs_revision => needs_review


Comment:

 Solving also #28865:
 https://github.com/torproject/sbws/pull/342
 Moving to 1.1 milestone since #28932 check progress and logs error.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  needs_revision
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:  1
 Reviewer: |Sponsor:
---+---
Changes (by juga):

 * status:  needs_review => needs_revision


Comment:

 Because of #28865, this needs to be re-thought, to don't make it yet more
 complicated.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  needs_review
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:  1
 Reviewer: |Sponsor:
---+---
Changes (by juga):

 * points:   => 1


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  needs_review
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:
 Reviewer: |Sponsor:
---+---
Changes (by juga):

 * status:  assigned => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  juga
 Type:  defect | Status:  assigned
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:
 Reviewer: |Sponsor:
---+---
Changes (by juga):

 * owner:  (none) => juga
 * status:  needs_review => assigned


Comment:

 reassigning so that the ticket has owner

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  (none)
 Type:  defect | Status:  needs_review
 Priority:  Medium |  Milestone:  sbws: 1.1.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:
 Reviewer: |Sponsor:
---+---
Changes (by juga):

 * cc: juga, teor (removed)
 * status:  new => needs_review
 * milestone:  sbws: 1.0.x-final => sbws: 1.1.x-final


Comment:

 Replying to [comment:3 juga]:
 > I looked more at this.
 > `wait` is the method of an `event`
 
(https://github.com/python/cpython/blob/master/Lib/multiprocessing/pool.py#L662),
 and it blocks
 
(https://docs.python.org/3/library/threading.html?highlight=threading#threading.Event.wait)

 a maximum of `timeout`.
 `get` with timeout uses `wait`, and that's what we want since pending
 results that didn't trigger either callback or callback error have not
 been gotten, and `get` give us either the exception or the value.

 After merging #28932, this is an improvement, can wait to 1.1 milestone.

 https://github.com/torproject/sbws/pull/341

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  (none)
 Type:  defect | Status:  new
 Priority:  Medium |  Milestone:  sbws: 1.0.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:
 Reviewer: |Sponsor:
---+---

Comment (by juga):

 I looked more at this.
 `wait` is the method of an `event`
 
(https://github.com/python/cpython/blob/master/Lib/multiprocessing/pool.py#L662),
 and it blocks
 
(https://docs.python.org/3/library/threading.html?highlight=threading#threading.Event.wait)
 `sleep` is not blocking because happen in the main thread.
 Probably the cause (or at least the main one) is #28897.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  (none)
 Type:  defect | Status:  new
 Priority:  Medium |  Milestone:  sbws: 1.0.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:
 Reviewer: |Sponsor:
---+---

Comment (by juga):

 Replying to [ticket:28864 teor]:
 > After sbws queues an `AsyncResult`, it will wait forever for the result
 to be ready:
 >
 
https://github.com/torproject/sbws/blob/ee64d76df54ceb3a3c9e1e2a797fd70d68bb0035/sbws/core/scanner.py#L359-L364
 >
 > If at least one result hangs, then sbws will hang, because
 `AsyncResult.ready()` does not have a timeout.

 In theory this won't happen, since both circuits and requests have a
 timeout. Also the sleep happen in the main thread, not the thread that is
 getting the results

 > Instead, sbws should call `AsyncResult.wait([timeout])` on each result,
 after calling `pool.apply_async()` on a large number of results.
 >
 > See
 
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.AsyncResult.wait

 I'll try this anyway

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  (none)
 Type:  defect | Status:  new
 Priority:  Medium |  Milestone:  sbws: 1.0.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID:  #28663 | Points:
 Reviewer: |Sponsor:
---+---

Comment (by teor):

 This ticket probably needs to be done at the same time as #28865.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #28864 [Core Tor/sbws]: sbws AsyncResults have no timeout

[Tor Bug Tracker & Wiki]

#28864: sbws AsyncResults have no timeout
---+---
 Reporter:  teor   |  Owner:  (none)
 Type:  defect | Status:  new
 Priority:  Medium |  Milestone:  sbws: 1.0.x-final
Component:  Core Tor/sbws  |Version:  sbws: 1.0.2
 Severity:  Normal |   Keywords:
Actual Points: |  Parent ID:  #28663
   Points: |   Reviewer:
  Sponsor: |
---+---
 After sbws queues an `AsyncResult`, it will wait forever for the result to
 be ready:
 
https://github.com/torproject/sbws/blob/ee64d76df54ceb3a3c9e1e2a797fd70d68bb0035/sbws/core/scanner.py#L359-L364

 If at least one result hangs, then sbws will hang, because
 `AsyncResult.ready()` does not have a timeout.

 Instead, sbws should call `AsyncResult.wait([timeout])` on each result,
 after calling `pool.apply_async()` on a large number of results.

 See
 
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.AsyncResult.wait

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs