[Pywikipedia-bugs] [Maniphest] [Created] T353855: Truncated JSON data in recent change event stream "/mediawiki/recentchange/1.0.0"

Wed, 20 Dec 2023 17:15:25 -0800 

Mmnormyle created this task.
Mmnormyle added projects: Pywikibot, Event-Platform, MediaWiki-Recent-changes.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.
Restricted Application added projects: Growth-Team, Data-Engineering.

TASK DESCRIPTION
  **Steps to replicate the issue** (include links if applicable):
  
  Using either pywikibot or SSEClient directly, I frequently see truncated JSON 
data in "recent change" wikipedia events.
  
    python
        site = pywikibot.Site('wikipedia:en')
        listener = site_rc_listener(site)
        for change in listener:
            ... 
  
  I frequently see warnings like below from pywikibot:
  
  Unterminated string starting at: line 1 column 868 (char 867)
  
  WARNING: Could not load json data from
  
{"$schema":"/mediawiki/recentchange/1.0.0","meta":{"uri":"https://es.wikipedia.org/wiki/Usuario_discusi%C3%B3n:ChandraHelsinky","request_id":"b834cdb2-ec40-4d7e-9c6d-b9c873579c14","id":"61170151-c676-43b6-9a67-b535f0f6b2a4","dt":"2023-12-21T00:54:09Z","domain":"es.wikipedia.org","stream":"mediawiki.recentchange","topic":"codfw.mediawiki.recentchange","partition":0,"offset":858052205},"id":301783823,"type":"edit","namespace":3,"title":"Usuario
 
discusión:ChandraHelsinky","title_url":"https://es.wikipedia.org/wiki/Usuario_discusi%C3%B3n:ChandraHelsinky","comment":"/*
 Info */ Respuesta","timestamp":1703120049,"user":"ChandraHe
  Unterminated string starting at: line 1 column 621 (char 620)
  WARNING: Could not load json data from
  
{"$schema":"/mediawiki/recentchange/1.0.0","meta":{"uri":"https://www.wikidata.org/wiki/Q44710541","request_id":"1a2351e5-52a1-4dea-a5c6-88c7e66c5bd7","id":"8ea7da32-494d-4169-8d7a-2c0e22586535","dt":"2023-12-21T01:01:43Z","domain":"www.wikidata.org","stream":"mediawiki.recentchange","topic":"codfw.mediawiki.recentchange","partition":0,"offset":858066105},"id":2095519196,"type":"edit","namespace":0,"title":"Q44710541","title_url":"https://www.wikidata.org/wiki/Q44710541","comment":"/*
 wbsetlabel-set:1|en */ The genetic architecture of freezing tolerance varies 
across the range of Arabidopsis 
thaliana","timestamp":1703120503,"user":"KrBot","bot":true,"notify_url":"https://www.wikidata.org/w/index.php?diff=2033406302&oldid=1636532311&rcid=20955191
  Unterminated string starting at: line 1 column 672 (char 671)
  WARNING: Could not load json data from
  
{"$schema":"/mediawiki/recentchange/1.0.0","meta":{"uri":"https://en.wikipedia.org/wiki/User:FranDoe16/Reese_White","request_id":"cab5b3b5-04a5-4afa-b132-fd9473ba7d79","id":"0f38238e-a94d-45ef-943d-a3bbe89102d0","dt":"2023-12-21T01:01:50Z","domain":"en.wikipedia.org","stream":"mediawiki.recentchange","topi
  Unterminated string starting at: line 1 column 303 (char 302)
  WARNING: Could not load json data from
  
{"$schema":"/mediawiki/recentchange/1.0.0","meta":{"uri":"https://commons.wikimedia.org/wiki/File:Valanga_irregularis_Grasshoppers_in_Hibiscus_sp_Pfingst_Rd_Chermside_P1070387.jpg","request_id":"d2fa4152-59ca-4dc8-8d32-f041d69aa162","id":"59c51b30-aafc-4d60-bcbf-92cfdbf61998","dt":"2023-12-21T01:02:04Z","domain":"commons.wikimedia.org","stream":"mediawiki.recentchange","topic":"codfw.mediawiki.recentchange","partition":0,"offset":858066708},"id":2384963855,"type":"edit","namespace":6,"title":"File:Valanga
 irregularis Grasshoppers in Hibiscus sp Pfingst Rd Chermside 
P1070387.jpg","title_url":"https://commons.wikimedia.org/wiki/File:Valanga_irregularis_Grasshoppers_in_Hibiscus_sp_Pfingst_Rd_Chermside_P1070387.jpg","comment":"/*
 wbsetlabel-add:1|en */ Valanga irregularis Grasshoppers in Hibiscus sp Pfingst 
Rd Chermside","timestamp":1703120524,"user":"John Robert 
McPherson","bot":false,"notify_url":"https://commons.wikimedia.org/w/index.php?diff=832769030&ol
  Unterminated string starting at: line 1 column 909 (char 908)
  
  **What happens?**:
  
  These truncated events are ignored.
  
  **Software version** (skip for WMF-hosted wikis like Wikipedia):
  
  pywikibot==8.6.0
  
  **Other information** (browser name/version, screenshots, etc.):
  
  I can also observe this issue when using sseclient directly.

TASK DETAIL
  https://phabricator.wikimedia.org/T353855

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Mmnormyle
Cc: Aklapper, pywikibot-bugs-list, Mmnormyle, mevo, Mohamed-Awnallah, 
lbowmaker, BTullis, PotsdamLamb, Ywats0ns, caldera, Jyoo1011, JohnsonLee01, 
SHEKH, Dijkstra, NavinRizwi, Khutuck, Zkhalido, holger.knust, Viztor, Wenyi, 
Jony, Nattes, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Altostratus, 
Taiwania_Justo, Avicennasis, mys_721tx, Xqt, GWicke, geraki, jayvdb, Masti, 
Alchimista

_______________________________________________
pywikibot-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]

Reply via email to