[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[OlafJanssen]

OlafJanssen added a comment.


  One more thing: How should be go about implementing the updated line of code
  
sparql = SPARQLWrapper(endpoint_url, agent='Some uniquely identifyable user 
agent')
  
  in the outputs in the WDQS-interface, like so
  
  F30003068: afbeelding.png 

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: OlafJanssen
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[OlafJanssen]

OlafJanssen added a comment.


  OK, thanmk you, I'll use the approach you suggest, I've put it in my notebook 
by now

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: OlafJanssen
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[Chicocvenancio]

Chicocvenancio added a comment.


  I've tested with and without the UserAgent, it sometimes works when 
defaulting to the SPARQLWrapper agent, but without looking at server logs and 
configuration I don't have much clue on why this is happening. Perhaps someone 
can investigate and tell us.
  
  What I do know is that your original code is not changing the agent used to 
make the query, it is making a request to `http://www.kb.nl` with the `My User 
Agent 1.0` and then making the query to WQS with the SPARQLWrapper agent.  I 
suggest you use the `agent="SOME_STRING_THAT_FOLLOWS_POLICY` approach.

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Chicocvenancio
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[OlafJanssen]

OlafJanssen added a comment.


  @Chicocvenancio  Ah, I see... although my appraoch seems to work as well (by 
accident?).

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: OlafJanssen
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[Chicocvenancio]

Chicocvenancio added a comment.


  In T230135#5403134 , 
@OlafJanssen wrote:
  
  > @Chicocvenancio Thanks for elegant your solution. I looked at 
https://stackoverflow.com/questions/10606133/sending-user-agent-using-requests-library-in-python
 
  > I added this the notebook, is that a valid approach as well? (I'm not 
experienced with this sort of stuff yet)
  >
  >  
  >
  > import requests
  >  url = 'http://www.kb.nl'
  >  headers = {
  >
  >   'User-Agent': 'My User Agent 1.0',
  >   'From': 'olaf.jans...@kb.nl'  # This is another valid field
  >
  > }
  >  response = requests.get(url, headers=headers)
  
  That won't change the UserAgent used by SPARQLWrapper. SPARQLWrapper has an 
`agent` parameter you can use that will solve this.
  
...
def get_results(endpoint_url, query):
sparql = SPARQLWrapper(endpoint_url, agent='OlafJanssen from PAWS')
...
  
  Is one way to get this working.

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Chicocvenancio
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[OlafJanssen]

OlafJanssen added a comment.


  @Chicocvenancio Thanks for elegant your solution. I looked at 
https://stackoverflow.com/questions/10606133/sending-user-agent-using-requests-library-in-python
  
  I added this the notebook, is that a valid approach as well? (I'm not 
experienced with this sort of stuff yet)
  
  
  
  import requests
  
  url = 'http://www.kb.nl'
  
  headers = {
  
'User-Agent': 'My User Agent 1.0',
'From': 'olaf.jans...@kb.nl'  # This is another valid field
  
  }
  
  response = requests.get(url, headers=headers)

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: OlafJanssen
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[Chicocvenancio]

Chicocvenancio added a comment.


  This seems relevant https://github.com/RDFLib/sparqlwrapper/issues/139

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Chicocvenancio
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[Chicocvenancio]

Chicocvenancio added a comment.


  This seems relevant https://github.com/RDFLib/sparqlwrapper/issues/139

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Chicocvenancio
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[OlafJanssen]

OlafJanssen added a comment.


  @Chicocvenancio Thanks for that !pip tip, I was not aware of that
  
  @Lucas_Werkmeister_WMDE Thank you, I'll give that a try

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: OlafJanssen
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  You probably need to set a good User-Agent 
 – the default Python 
Requests one is blocked since about a month ago (announcement 
). 
(Though that doesn’t explain why it would still have worked yesterday…)

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, 
DannyS712, Nandana, Lahi, Gq86, R96340, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T230135: Wikidata queries in PAWS return HTTP Error 403: Forbidden

[Chicocvenancio]

Chicocvenancio added a comment.


  BTW you can use a ! to run commands directly from the notebooks. IE use `!pip 
install sparqlwrapper` instead of using the terminal every server start. see 
https://paws-public.wmflabs.org/paws-public/User:chicocvenancio/T230135.ipynb

TASK DETAIL
  https://phabricator.wikimedia.org/T230135

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Chicocvenancio
Cc: Chicocvenancio, OlafJanssen, darthmon_wmde, 94rain, DannyS712, Nandana, 
Lahi, Gq86, R96340, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Allthingsgo, 
QZanden, EBjune, merbst, LawExplorer, Framawiki, _jensen, rosalieper, Jonas, 
Xmlizer, Jane023, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, Xqt, aude, 
Tobias1984, yuvipanda, Capt_Swing, Manybubbles, zhuyifei1999, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs