Ryan10145 removed a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T182522EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ryan10145Cc: gerritbot, TerraCodes, Aklapper, Xqt, pywikibot-bugs-list, Asad_Ali_Palijo, Soteriaspace
Ryan10145 claimed this task.Ryan10145 updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION**imagecopy.py** does not support its `-cc` option because `-cc` is a global option to toggle cosmetic changes.
Also option descriptions are missing.TASK DETAILhttps
Ryan10145 removed a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T183664EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ryan10145Cc: gerritbot, rafidaslam, pywikibot-bugs-list, eflyjason, Aklapper, Xqt, zhuyifei1999, jayvdb
Ryan10145 removed a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T183675EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ryan10145Cc: gerritbot, Dalba, Ryan10145, Framawiki, Aklapper, Xqt, jayvdb, siebrand, Zoranzoki21, eflyjason
Ryan10145 added a comment.
Good point, it is probably best to avoid recursion and to instead use or. However, I feel that the while loop implementation is still somewhat difficult to understand, especially for a newer programmer. @Framawiki, what are your thoughts on which one to use?TASK
Ryan10145 added a comment.
@zhuyifei1999 I was thinking of something along the lines of
def download_dump(filepath, is_atomic):
try:
#Put download code here
except (OSError, IOError):
#Exception/remove stuff
if is_atomic:
# Print error, retrying
Ryan10145 added a comment.
@Framawiki I can work on it, but I'm not sure if it would be right to do it at this moment, since the patches for T183664 and T183667 would be affected. I'll try to get this done when there aren't any open patches affecting download_dump.pyTASK DETAILhttps
Ryan10145 added a comment.
I discovered the error, I was running the script incorrectly. I was using python scripts/maintenance/download_dump.py -filename instead of python pwb.py maintenance/download_dump.py
pywikibot.comms.http.fetch('https://dumps.wikimedia.org/idwiki/latest/idwiki-latest
Ryan10145 added a comment.
I was experimenting with the location of a test script, and I found something interesting. I used the following test script.
import sys
import pywikibot
sys.stdout.write('begin')
sys.stdout.flush()
response = pywikibot.comms.http.fetch('https://dumps.wikimedia.org
Ryan10145 added a comment.
I downloaded the patch that fixed the issue, but I am still running into issues with the streaming. First of all, when I test the functionality of pywikibot.comms.http.fetch(url, stream=True) from the pywikibot shell, it works as intended. I tested it using
def
Ryan10145 added a comment.
@eflyjason I tried it, and the none of the messages printed until the download completed.
For now, I will just write the download bar as if the pywikibot.comms.http.fetch(url, stream=True) behaves like requests.get(url, stream=True), and then submit the patch when
Ryan10145 added a comment.
I'm running into a problem with this task. From the line response = fetch(url, stream=True), I assumed that the file is not being downloaded on that line, and rather it will wait until the data is accessed to begin downloading it. Therefore, I thought that I could put
Ryan10145 claimed this task.
TASK DETAILhttps://phabricator.wikimedia.org/T183664EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ryan10145Cc: rafidaslam, pywikibot-bugs-list, eflyjason, Aklapper, Xqt, zhuyifei1999, jayvdb, siebrand, Zoranzoki21, Framawiki
Ryan10145 closed subtask T183675: download_dump.py: Make download process atomic as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T183663EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ryan10145Cc: pywikibot-bugs-list, eflyjason, Z
Ryan10145 closed this task as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T183675EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ryan10145Cc: gerritbot, Dalba, Ryan10145, Framawiki, Aklapper, Xqt, jayvdb, siebrand, Zoranzoki21,
Ryan10145 added a comment.
I just submitted a patch, but I'll try to make the change.TASK DETAILhttps://phabricator.wikimedia.org/T183675EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ryan10145Cc: gerritbot, Dalba, Ryan10145, Framawiki, Aklapper, Xqt, jayvdb
Ryan10145 added a comment.
Thank you for the help, I didn't see this before because my web browser would automatically correct the slashes when I clicked on it.TASK DETAILhttps://phabricator.wikimedia.org/T183675EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences
Ryan10145 added a comment.
I'm trying to run this script on my computer so I can do this task, but whenever I try to download the abstract.xml file, I get a Http reponse status 404 error.
This is my console output without -v:
python scripts/maintenance/download_dump.py -filename
abstract.xml
18 matches
Mail list logo
