[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Dvorapa]

Dvorapa added a comment.


  Asking ws:it API for namespaces gives broken list when maxlag condition 
occurs. No maxlag/timeout error, no empty list like in other cases.

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Ladsgroup, Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, 
JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, WDoranWMF, holger.knust, 
EvanProdromou, Viztor, DannyS712, Wenyi, Amorymeltzer, Tbscho, MayS, Sethakill, 
Mdupont, JJMC89, dg711, Agabi10, Altostratus, Avicennasis, Pchelolo, mys_721tx, 
jayvdb, Ricordisamoa, Anomie, Masti, Alchimista, Rxy, Krenair, Legoktm
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Xqt]

Xqt added a comment.


  @Ladsgroup: Any idea who can help upstream solving this malfunction?

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Xqt
Cc: Ladsgroup, hashar, Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, 
Dvorapa, Dijkstra, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, 
Altostratus, Avicennasis, mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, 
Rxy, Krenair
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Xqt]

Xqt added a comment.


  Hm:
  
Validate page_from_repository on wikipedia:en ... Sleeping for 5.0 seconds, 
2020-03-09 16:11:44
856
Sleeping for 5.0 seconds, 2020-03-09 16:11:49
857
Sleeping for 5.0 seconds, 2020-03-09 16:11:54
858
Sleeping for 5.0 seconds, 2020-03-09 16:12:00
859
Sleeping for 7.1 seconds, 2020-03-09 16:12:05
860
skipped 'Maximum retries attempted due to maxlag without success.'
861
 27.878s test_page_from_repository_it_ws 
(tests.site_tests.TestCategoryFromWikibase)
  
  Do we just skip a subTest?

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Xqt
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Xqt]

Xqt added a comment.


  > Summary: Wikidata maxlag produces empty Wikidata namespace list
  
  I am wondering. I get a `TimeoutError` in that case. And `SkipTest` should 
skip the test then too.
  Where is the trick that the exception is not raisen? Probably there is no 
maxlag timeout from wikibase but just gives an empty or unsufficent 
`NamespacesDict` with `_build_namespaces()`
  
>>> import pwb, pywikibot as py
>>> s = py.Site()
>>> r = s.data_repository()
>>> ns = r.namespaces
Sleeping for 5.0 seconds, 2020-03-09 17:53:50
Sleeping for 5.5 seconds, 2020-03-09 17:53:55
Sleeping for 8.5 seconds, 2020-03-09 17:54:01
Sleeping for 11.3 seconds, 2020-03-09 17:54:10
Sleeping for 14.2 seconds, 2020-03-09 17:54:22
Traceback (most recent call last):
  File "", line 1, in 
ns = r.namespaces
  File "C:\pwb\GIT\core\pywikibot\site.py", line 1013, in namespaces
self._namespaces = NamespacesDict(self._build_namespaces())
  File "C:\pwb\GIT\core\pywikibot\site.py", line 2647, in _build_namespaces
for nsdata in self.siteinfo.get('namespaces', cache=False).values():
  File "C:\pwb\GIT\core\pywikibot\site.py", line 1683, in get
preloaded = self._get_general(key, expiry)
  File "C:\pwb\GIT\core\pywikibot\site.py", line 1629, in _get_general
default_info = self._get_siteinfo(props, expiry)
  File "C:\pwb\GIT\core\pywikibot\site.py", line 1552, in _get_siteinfo
data = request.submit()
  File "C:\pwb\GIT\core\pywikibot\data\api.py", line 2258, in submit
self._data = super(CachedRequest, self).submit()
  File "C:\pwb\GIT\core\pywikibot\data\api.py", line 2105, in submit
raise MaxlagTimeoutError(msg)
pywikibot.exceptions.MaxlagTimeoutError: Maximum retries attempted due to 
maxlag without success.

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Xqt
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Dvorapa]

Dvorapa added a comment.


  @Xqt Could we somehow check whether any of `self._entity_namespaces` or WD 
namespaces list is (in)complete and try to recreate that if it isn't?
  
  Perhaps we could check if WD namespaces list (cached) is empty or does not 
contain basic 0 to 15 namespaces?

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[matej_suchanek]

matej_suchanek added a comment.


  In T242083#5953475 , 
@Dvorapa wrote:
  
  > @matej_suchanek Do you know any solution for this issue?
  
  Failing hard in tests when `maxlag` is too high (no silent error and empty 
responses). Or not using (relaxing) `maxlag` on read requests.

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: matej_suchanek
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Dvorapa]

Dvorapa added a comment.


  @matej_suchanek Do you know any solution for this issue?
  
  Summary: Wikidata maxlag produces empty Wikidata namespace list. Therefore 
`self._entity_namespaces` (which depends on it) is empty/incomplete and makes 
tests fail if that method to create `self._entity_namespaces` fails due to 
maxlag

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Dvorapa]

Dvorapa added a comment.


  It seems so as it then works with empty/incomplete namespace list and 
produces the error message in the description

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Xqt]

Xqt added a comment.


  In T242083#5854100 , 
@Dvorapa wrote:
  
  > Okay, if DataSite.namespaces fails to get namespaces (due to timeout), this 
error is thrown, which is not a good behavior
  
  This means a general exception is passed silently I guess.

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Xqt
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Dvorapa]

Dvorapa added a comment.


  Basically, here (_build_namespaces): 
https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/pywikibot/site.py$2618
 should be a check that all of these (builtin_namespaces): 
https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/pywikibot/site.py$406
 are included, otherwise it should fail with API request error and **not return 
an empty list of namespaces** to continue!

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Dvorapa]

Dvorapa added a comment.


  So this is a duplicate of T242081 
, but the error message is not much 
useful

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Dvorapa]

Dvorapa added a comment.


  Okay, if DataSite.namespaces fails to get namespaces (due to timeout), this 
error is thrown, which is not a good behavior

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T242083: test_page_from_repository fails with EntityTypeUnknownException on multiple sites

[Dvorapa]

Dvorapa added a comment.


  But the code haven't changed much in the mentioned patch, which is weird

TASK DETAIL
  https://phabricator.wikimedia.org/T242083

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Xqt, matej_suchanek, Aklapper, pywikibot-bugs-list, Dvorapa, Zkhalido, 
Viztor, Wenyi, Tbscho, MayS, Mdupont, JJMC89, Altostratus, Avicennasis, 
mys_721tx, jayvdb, Ricordisamoa, Masti, Alchimista, Rxy
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs