date:20200824

[Pywikipedia-bugs] [Maniphest] [Changed Subscribers] T261050: Frequent "Invalid CSRF token" errors on Wikimedia Commons using Pywikibot since August 2020

2020-08-24 Thread Xqt

Xqt added a subscriber: Tgr.
Xqt added a comment.


  Unfortunately I am not familiar with it. I haven't seen that problem for a 
long time. But we have no CI tests for commons as default site, I guess we 
should have some.

TASK DETAIL
  https://phabricator.wikimedia.org/T261050

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Xqt
Cc: Tgr, Dvorapa, Xqt, Aklapper, Reedy, pywikibot-bugs-list, Multichill, 
JohnsonLee01, SHEKH, Dijkstra, CBogen, Khutuck, Zkhalido, WDoranWMF, 
holger.knust, EvanProdromou, Viztor, DannyS712, Wenyi, Amorymeltzer, 
Ramsey-WMF, Tbscho, MayS, Sethakill, Mdupont, JJMC89, dg711, Poyekhali, 
Agabi10, Altostratus, Taiwania_Justo, Avicennasis, Pchelolo, Ixocactus, 
Wong128hk, mys_721tx, Bawolff, El_Grafo, Dinoguy1000, jayvdb, Anomie, Masti, 
Alchimista, Steinsplitter, Rxy, Keegan, Legoktm
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Declined] T261162: pywikibot keeps refcount even after clear_cache (osmium fails with "Node callback keeps reference to OSM object. This is not allowed.")

2020-08-24 Thread zhuyifei1999

zhuyifei1999 closed this task as "Declined".
zhuyifei1999 added a comment.


  This is an instance of cyclic reference. The frame refers to the entry, the 
entry has _getexception, the exception refers to the frame. The exception would 
also definitely reference the inner frame where the exception is instantiated 
and that frame would refer to the outer frame by `f_back`
  
  To prove this is a reference cycle, this works:
  
import osmium, pywikibot
import gc
en_wiki = pywikibot.Site("en", "wikipedia")

class TestHandler(osmium.SimpleHandler):
def node(self, n):
print(n.id)
entry = pywikibot.Page(en_wiki, 'non-existing article or even 
redirect')
entry.text
entry.clear_cache()  # Clear cache even works if there is no call 
to `.text`
del entry
gc.collect()

h = TestHandler()
# Download from 
http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf
h.apply_file('liechtenstein-latest.osm.pbf')
  
  `gc.collect` would kill unused reference cycles.
  
  osmium should not care about reference cycles. Ask them to perform a GC if 
they suspect values being used referred to outside the function.

TASK DETAIL
  https://phabricator.wikimedia.org/T261162

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: zhuyifei1999
Cc: zhuyifei1999, Aklapper, pywikibot-bugs-list, Stalker, JohnsonLee01, SHEKH, 
Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, 
Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T261162: pywikibot keeps refcount even after clear_cache (osmium fails with "Node callback keeps reference to OSM object. This is not allowed.")

2020-08-24 Thread zhuyifei1999

zhuyifei1999 added a comment.


  Using [[github.com/zhuyifei1999/guppy3|guppy]] to traverse the reference 
graph:
  
import osmium, pywikibot
import guppy
en_wiki = pywikibot.Site("en", "wikipedia")

thisframe = []
class TestHandler(osmium.SimpleHandler):
def node(self, n):
print(n.id)
entry = pywikibot.Page(en_wiki, 'non-existing article or even 
redirect')
entry.text
entry.clear_cache()  # Clear cache even works if there is no call 
to `.text`
thisframe.append(guppy.hpy().iso(n).referrers.theone)
# n = None

h = TestHandler()
# Download from 
http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf
try:
h.apply_file('liechtenstein-latest.osm.pbf')
except Exception:
pass

hp = guppy.hpy()
print(hp.iso(thisframe[0]).referrers)
print(hp.iso(thisframe[0]).referrers[0].referrers)
print(hp.iso(thisframe[0]).referrers[0].referrers[1].sp)
  
  
  
(venv) zhuyifei1999@zhuyifei1999-ThinkPad-T480 ~/mw-dev/pywikibot-core $ 
python T261162.py 
family and mylang are not set.
Defaulting to family='wikipedia' and mylang='test'.
26032956
Partition of a set of 2 objects. Total size = 568 bytes.
 Index  Count   % Size   % Cumulative  % Kind (class / dict of class)
 0  1  50  464  82   464  82 types.FrameType
 1  1  50  104  18   568 100 list
Partition of a set of 2 objects. Total size = 640 bytes.
 Index  Count   % Size   % Cumulative  % Kind (class / dict of class)
 0  1  50  568  89   568  89 types.FrameType
 1  1  50   72  11   640 100 types.TracebackType
 0: 
hp.Root.i0_modules['__main__'].__dict__['thisframe'][0].f_locals['entry'].__dict__['_getexception'].??

TASK DETAIL
  https://phabricator.wikimedia.org/T261162

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: zhuyifei1999
Cc: zhuyifei1999, Aklapper, pywikibot-bugs-list, Stalker, JohnsonLee01, SHEKH, 
Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, 
Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Created] T261162: pywikibot keeps refcount even after clear_cache (osmium fails with "Node callback keeps reference to OSM object. This is not allowed.")

2020-08-24 Thread Stalker

Stalker created this task.
Stalker added a project: Pywikibot.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.

TASK DESCRIPTION
  Steps to Reproduce: 
  Python3, pywikibot==4.1.1, osmium==3.0.1
  Download some .pbf file (openstreetmap binary format that should be parsed 
with osmium library). Liechtenstein is small (2.3MB) - 
http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf. Run following 
minimal example:
  
import osmium, pywikibot
en_wiki = pywikibot.Site("en", "wikipedia")

class TestHandler(osmium.SimpleHandler):
def node(self, n):
print(n.id)
entry = pywikibot.Page(en_wiki, 'non-existing article or even 
redirect')
entry.text
entry.clear_cache()  # Clear cache even works if there is no call 
to `.text`

h = TestHandler()
# Download from 
http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf
h.apply_file('liechtenstein-latest.osm.pbf')
  
  Actual Results:
  
  Script fails with bug mentioned even in osmium docs 
:
  
Traceback (most recent call last):
  File "main.py", line 15, in 
h.apply_file('liechtenstein-latest.osm.pbf')
RuntimeError: Node callback keeps reference to OSM object. This is not 
allowed.
CRITICAL: Exiting due to uncaught exception 
  
  Expected Results:
  I am expecting that pywikibot would clear all references and osmium (which 
checks if objects are released) would not complain.

TASK DETAIL
  https://phabricator.wikimedia.org/T261162

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Stalker
Cc: Aklapper, pywikibot-bugs-list, Stalker, JohnsonLee01, SHEKH, Dijkstra, 
Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, 
Altostratus, Avicennasis, mys_721tx, jayvdb, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T261050: Frequent "Invalid CSRF token" errors on Wikimedia Commons using Pywikibot since August 2020

2020-08-24 Thread Multichill

Multichill added a comment.


  @Xqt what is the correct way to force the bot to get a new token? Just 
site.get_tokens() or something else? I noticed some of my bots now stuck with 
100s of errors in a row.

TASK DETAIL
  https://phabricator.wikimedia.org/T261050

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Multichill
Cc: Dvorapa, Xqt, Aklapper, Reedy, pywikibot-bugs-list, Multichill, 
JohnsonLee01, SHEKH, Dijkstra, CBogen, Khutuck, Zkhalido, WDoranWMF, 
holger.knust, EvanProdromou, Viztor, DannyS712, Wenyi, Amorymeltzer, 
Ramsey-WMF, Tbscho, MayS, Sethakill, Mdupont, JJMC89, dg711, Poyekhali, 
Agabi10, Altostratus, Taiwania_Justo, Avicennasis, Pchelolo, Ixocactus, 
Wong128hk, mys_721tx, Bawolff, El_Grafo, Dinoguy1000, jayvdb, Anomie, Masti, 
Alchimista, Steinsplitter, Rxy, Keegan, Legoktm
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Changed Subscribers] T261050: Frequent "Invalid CSRF token" errors on Wikimedia Commons using Pywikibot since August 2020

[Pywikipedia-bugs] [Maniphest] [Declined] T261162: pywikibot keeps refcount even after clear_cache (osmium fails with "Node callback keeps reference to OSM object. This is not allowed.")

[Pywikipedia-bugs] [Maniphest] [Commented On] T261162: pywikibot keeps refcount even after clear_cache (osmium fails with "Node callback keeps reference to OSM object. This is not allowed.")

[Pywikipedia-bugs] [Maniphest] [Created] T261162: pywikibot keeps refcount even after clear_cache (osmium fails with "Node callback keeps reference to OSM object. This is not allowed.")

[Pywikipedia-bugs] [Maniphest] [Commented On] T261050: Frequent "Invalid CSRF token" errors on Wikimedia Commons using Pywikibot since August 2020

5 matches

Site Navigation

Mail list logo

Footer information