Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  closed
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+
Changes (by karsten):

 * status:  merge_ready => closed
 * resolution:   => fixed


Comment:

 Great! Squashed and merged to master. Closing. Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  merge_ready
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+
Changes (by iwakeh):

 * status:  needs_review => merge_ready


Comment:

 Replying to [comment:10 karsten]:
 > FWIW, local test succeeded.

 Great, thanks!  This looks fine, and, will make introducing the
 configuration improvement easier.
 Merge ready.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+

Comment (by karsten):

 FWIW, local test succeeded.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+
Changes (by karsten):

 * status:  needs_revision => needs_review


Comment:

 Replying to [comment:8 karsten]:
 > I just made the first two changes, but while looking into the third
 change I discovered a potential bug: it might be that we're reading and
 parsing the entire set of recent descriptors 7 times and throwing away all
 but one descriptor type in each execution. I need to look closer at that,
 but I have to leave for lunch now.

 False alarm. I overlooked a line that prevents just that. Phew.

 Alright, please find
 
[https://gitweb.torproject.org/user/karsten/onionoo.git/commit/?h=task-25700=5c96f5c0ee2fb8d8fe85a4a16d9061608c421a0b
 commit 5c96f5c in my task-25700 branch]. I ''think'' that addresses all
 things you mentioned above. If not, please consider providing a fix, so
 that we can get this merged by end of today, if possible. I'm also
 currently running a local test.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+

Comment (by karsten):

 I just made the first two changes, but while looking into the third change
 I discovered a potential bug: it might be that we're reading and parsing
 the entire set of recent descriptors 7 times and throwing away all but one
 descriptor type in each execution. I need to look closer at that, but I
 have to leave for lunch now.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+

Comment (by iwakeh):

 Replying to [comment:5 karsten]:
 > Much lower. But I can't make a graph of that. I took the data for this
 graph from consensus tarballs by plotting time differences between
 consensus valid-after time and the file's last-modified time. We don't
 have a similar archive from collector2, AFAIK.

 Ah, thanks!  No, collector2 doesn't archive.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+
Changes (by iwakeh):

 * status:  needs_review => needs_revision


Comment:

 Queues get added twice
 
[https://gitweb.torproject.org/user/karsten/onionoo.git/tree/src/main/java/org/torproject/onionoo/updater/DescriptorSource.java?id=c7f62a33f665dc76528541124a3ceba583c50a4a#n57
 here].

 
[https://gitweb.torproject.org/user/karsten/onionoo.git/tree/src/main/java/org/torproject/onionoo/updater/DescriptorSource.java?id=c7f62a33f665dc76528541124a3ceba583c50a4a#n50
 When constructing recentDir] the code is not necessarily platform
 independent, maybe use Paths.get.

 As Onionooo should be made easier to configure soon and we decided on
 having one config file (in #24041) maybe the construction of directories
 and supplying the hosts could be put in one place?  This would also avoid
 repeating the path concatenation.  Something like a list of host URL and
 in directory?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+

Comment (by karsten):

 Much lower. But I can't make a graph of that. I took the data for this
 graph from consensus tarballs by plotting time differences between
 consensus valid-after time and the file's last-modified time. We don't
 have a similar archive from collector2, AFAIK.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+

Comment (by iwakeh):

 Just curious: what are the downtimes for collector2?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  iwakeh   |Sponsor:
-+
Changes (by iwakeh):

 * reviewer:   => iwakeh


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+

Comment (by karsten):

 Somewhat related, though mostly as a motivation why this ticket matters,
 here's a graph of temporary downtimes of the primary CollecTor host in the
 first quarter of 2018:

 [[Image(collector-downtimes.png, 700px)]]

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by karsten):

 * Attachment "collector-downtimes.png" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by karsten):

 * status:  assigned => needs_review


Comment:

 Please review
 
[https://gitweb.torproject.org/user/karsten/onionoo.git/commit/?h=task-25700=c7f62a33f665dc76528541124a3ceba583c50a4a
 commit c7f62a3 in my task-25700 branch].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #25700 [Metrics/Onionoo]: Fetch descriptors from both CollecTor instances

[Tor Bug Tracker & Wiki]

#25700: Fetch descriptors from both CollecTor instances
-+
 Reporter:  karsten  |  Owner:  karsten
 Type:  enhancement  | Status:  assigned
 Priority:  Medium   |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   |   Keywords:
Actual Points:   |  Parent ID:
   Points:   |   Reviewer:
  Sponsor:   |
-+
 As discussed at the last team meeting, we should attempt to make Onionoo
 more robust by fetching descriptors from both CollecTor instances.

 I started working on this by extending `DescriptorSource` and am currently
 trying it out. As soon as I have something to review, I'll post it here.

 This change won't require a protocol change and can go into the next
 release.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs