Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+
 Reporter:  karsten  |  Owner:  iwakeh
 Type:  defect   | Status:  closed
 Priority:  High |  Milestone:  Onionoo 1.12.0
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by iwakeh):

 * milestone:   => Onionoo 1.12.0


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+
 Reporter:  karsten  |  Owner:  iwakeh
 Type:  defect   | Status:  closed
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+

Comment (by cypherpunks):

 regression: #25740

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+
 Reporter:  karsten  |  Owner:  iwakeh
 Type:  defect   | Status:  closed
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by karsten):

 * status:  needs_review => closed
 * resolution:   => fixed


Comment:

 Looks good! Merged. Closing. Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  iwakeh
 Type:  defect   | Status:  needs_review
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by iwakeh):

 * status:  accepted => needs_review


Comment:

 Please find
 [https://gitweb.torproject.org/user/iwakeh/onionoo.git/log/?h=task-22594
 two more commits] for review.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  iwakeh
 Type:  defect   | Status:  accepted
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by iwakeh):

 * status:  needs_revision => accepted


Comment:

 Replying to [comment:14 karsten]:
 > Looks good! Two things:
 >  - Can you add a change log entry? Maybe something like this: ''Don't
 attempt to un-escape character sequences in contact lines (like "\uk")
 that only happen to start like escaped utf-8 characters (like "\u0055").''

 Yep, that was missing.

 >  - There's similar code where we replace `"u"` with `"\\u"` in
 `ResponseBuilder`. Does it make sense to use the new method there, too?

 Sure.  I was actually intending to add a new ticket about having all
 un/escape logic as well as the json-reading and writing in one place,
 which would make it easier to switch to some other JSON lib than gson in
 future and have the logic separate for re-processing files etc.
 But, it's fine to start with putting all un/escape logic in one place and
 add the remaining to a new ticket in case it would be too much for one
 ticket.

 >
 > Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+
 Reporter:  karsten  |  Owner:  iwakeh
 Type:  defect   | Status:  needs_revision
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+
Changes (by karsten):

 * status:  needs_review => needs_revision


Comment:

 Looks good! Two things:
  - Can you add a change log entry? Maybe something like this: ''Don't
 attempt to un-escape character sequences in contact lines (like "\uk")
 that only happen to start like escaped utf-8 characters (like "\u0055").''
  - There's similar code where we replace `"u"` with `"\\u"` in
 `ResponseBuilder`. Does it make sense to use the new method there, too?

 Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  iwakeh
 Type:  defect   | Status:  needs_review
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by iwakeh):

 * status:  accepted => needs_review


Comment:

 Please review
 [https://gitweb.torproject.org/user/iwakeh/onionoo.git/commit/?h=task-22594
 this patch], which avoid the above troubles, i.e., NFE, by only un-
 escaping valid utf.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  iwakeh
 Type:  defect   | Status:  accepted
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by iwakeh):

 * status:  new => accepted
 * owner:  metrics-team => iwakeh


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by karsten):

 * priority:  Medium => High


Comment:

 Raising priority to prevent bad operators from "hiding" their relays from
 Onionoo clients. See [https://lists.torproject.org/pipermail/metrics-
 team/2017-December/000610.html this thread] for more context.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Yep, sounds good.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 The hotfix was added to master and did go into the onionoo-4.0-1.3.0
 release.
 I didn't close this ticket b/c it doesn't seem there was a real fix yet.

 Having tag 'metrics-2018' the priority ought to be rather medium here.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Did we really fix the bug or just deploy a hotfix back then?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-2018 |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by iwakeh):

 * priority:  High => Medium


Comment:

 This was part of release onionoo-4.0-1.3.0 already.
 Setting priority to 'medium'.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 No specific reason, other than being in a rush to put out a hotfix.  Happy
 to undo that when we put in a real fix. :)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 Why two log statements?

  {{{
  catch (Throwable e) {
/* Handle below. */
 +  log.error("Problem while parsing JSON: " + documentString, e);
 -  log.error(documentString);
 -  log.error(e.getMessage(), e);
  }
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Replying to [comment:1 iwakeh]:
 > Regarding the hot-fix I would suggest to catch any Throwable in the
 above patch in order to also have possible future Gson exceptions logged.
 >
 > Maybe, a switch to a fully JSON compliant library would also help, but
 that and possible other solutions need more time, that's true.

 I just pushed the hotfix to master, which now catches `Throwable`.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by catalyst):

 * cc: catalyst (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

[Tor Bug Tracker & Wiki]

#22594: Escape characters in contact lines break hourly updater
-+--
 Reporter:  karsten  |  Owner:  metrics-team
 Type:  defect   | Status:  new
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 Regarding the hot-fix I would suggest to catch any Throwable in the above
 patch in order to also have possible future Gson exceptions logged.

 Maybe, a switch to a fully JSON compliant library would also help, but
 that and possible other solutions need more time, that's true.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs