[Glamtools] Welcoming new GWT users

2014-04-25 Thread
Could a list admin check the subscribers and compare with ? I'm thinking it might be useful to post a standard welcome notice for new users of the tool, including an invite to join this list to discuss their experie

Re: [Glamtools] Welcoming new GWT users

2014-04-25 Thread
k care of assigning GWToolset right on Commons? > > https://commons.wikimedia.org/wiki/Commons_talk:GLAMwiki_Toolset > > > We requested the rights almost three weeks ago :-) > > > Thanks > > Charles > > Le 25 avr. 2014 à 16:05, Fæ a écrit : > >> Cou

Re: [Glamtools] Welcoming new GWT users

2014-04-25 Thread
kimedia.org/wiki/Commons:Requests_for_rights#Translation_administrators_.26_GW_Toolset_users >> >> Maarten >> >>> >>> >>> Thanks >>> >>> Charles >>> >>> Le 25 avr. 2014 à 16:05, Fæ a écrit : >>> >>>> C

[Glamtools] GWToolset operational issues for very large TIFFs

2014-04-25 Thread
SUMMARY: This week I experienced an issue when uploading several hundred very high resolution maps as part the NYPL maps project.[1] Discussion has been going on in several places and this thread is an attempt to share a discussion in one place so all users can benefit. [Gilles, Could you join thi

Re: [Glamtools] Welcoming new GWT users

2014-04-25 Thread
Coming back to the original topic, I have started a simple table of users against projects on <https://commons.wikimedia.org/wiki/Commons:GWToolset_users>. Could a couple of folks add themselves there, I'm feeling a bit lonely. :-) Fae On 25 April 2014 15:05, Fæ wrote: > Coul

Re: [Glamtools] Welcoming new GWT users

2014-04-26 Thread
On 25 April 2014 20:12, Fæ wrote: > Coming back to the original topic, I have started a simple table of > users against projects on > <https://commons.wikimedia.org/wiki/Commons:GWToolset_users>. I have created a standard welcome notice that Bureaucrats or others can apply aft

Re: [Glamtools] Open GWT to all Commons users

2014-04-27 Thread
On 27/04/2014, Emmanuel Engelhart wrote: ... > Do you see any other risk? Yes. * If a user has 10 files to upload they should use the well designed standard wizard. The GWT is an amazingly cumbersome way to upload small numbers of files. * The GWT has a number of bugs and restricted features th

Re: [Glamtools] GWToolset beta: url from xml not recognized

2014-04-29 Thread
On 29/04/2014, Hans Muller wrote: > http://www.peacepalacelibrary.nl/images/wikimedia/Bor-Nederlantsche-Oorloghen_9139.tif"; *Possibly* because the whitelisted domain is "peacepalacelibrary.nl" and not "*.peacepalacelibrary.nl". Try trimming the "www" and see if that works. Fae -- fae...@gmail

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-04-29 Thread
From a user perspective, it would be neat if we could set something like "high / medium / low" priority at the time of creating the upload request. Some GWT users might be trying to process their image collection in time for a planned edit-a-thon or similar, so taking a week to upload might be a pr

[Glamtools] GWT housekeeping and bugzilla

2014-04-30 Thread
For those on this group who have not signed up for an account on bugzilla, I strongly encourage you to do so. The GWToolset is likely to have a number of changes logged on bugzilla and the more user feedback we can add to the changes the better. The most recent is Steinsplitter's

[Glamtools] Maximum filename length for GWT = 206?

2014-04-30 Thread
Hi, I had an odd problem with files not being created, which I think I can put down to how long filenames are handled by GWT. As an example, my xml specified (A) but GWT created (B): A. File:Index Map No.2 of a part of Suffolk County. South Side - Ocean Shore, Long Island. Part of Islip and Part

Re: [Glamtools] Maximum filename length for GWT = 206?

2014-04-30 Thread
Sorry, got my examples confused. The general point about filename truncation is still correct. Fae On 30/04/2014, Fæ wrote: > Hi, > > I had an odd problem with files not being created, which I think I can > put down to how long filenames are handled by GWT. > > As an example

[Glamtools] GWT best practice/rules for file name characters

2014-05-01 Thread
Does anyone have a working definition of characters allowed in the filenames that I could apply in my pre-processing of the xml files? See [1] and [2] for the technical standards that apply by default. In my NYPL uploads I have found that characters in the chosen filename like: * ö (o umlaut) * Æ

Re: [Glamtools] GWT best practice/rules for file name characters

2014-05-01 Thread
On 1 May 2014 13:21, Federico Leva (Nemo) wrote: > Sounds like a major bug, is it filed? Not on bugzilla yet. I'd like to know if others have a similar experience of GWT halting mid-run for these sorts of reasons. Unfortunately with no error reporting back from partial runs, diagnosis becomes qu

Re: [Glamtools] GWT best practice/rules for file name characters

2014-05-01 Thread
As an example, below is a failed record in my most recent 're-run', where some records are being skipped (I have no idea how the GWT chooses when to skip or fail). If someone can spot the problem I'd be very grateful ​​ . ​​ Islip Village and Vicinity Suffolk County, N.Y. NYPL1602998 L.E. Neum

Re: [Glamtools] GWT best practice/rules for file name characters

2014-05-01 Thread
On 01/05/2014, Fæ wrote: > As an example, below is a failed record in my most recent 're-run', where > some records are being skipped (I have no idea how the GWT chooses when to > skip or fail). If someone can spot the problem I'd be very grateful ... After downloading

Re: [Glamtools] GWT best practice/rules for file name characters

2014-05-01 Thread
eed someone to +2 the patch i > made so that we can test the new preview step on the beta cluster. > > > with kind regards, > dan > > > On May 1, 2014, at 18:35 , Federico Leva (Nemo) > wrote: > > > Fæ, 01/05/2014 14:59: > >> Instead, a good example of

Re: [Glamtools] Advice on uploading a batch from a GLAM when individuals have already uploaded some of that GLAMs images?

2014-05-02 Thread
I have had many issues around this in the past. If the images are the same in quality/resolution then avoid duplicating what is currently on Commons. However if your versions are, in your view, better quality then there is no problem uploading them as they are not true duplicates. Digitally identic

Re: [Glamtools] Advice on uploading a batch from a GLAM when individuals have already uploaded some of that GLAMs images?

2014-05-03 Thread
On 03/05/2014, Brian Wolff wrote: > On May 2, 2014 5:40 AM, "Fæ" wrote: >> >> I have had many issues around this in the past. If the images are the > same in quality/resolution then avoid duplicating what is currently on > Commons. However if your versions are, in

Re: [Glamtools] Advice on uploading a batch from a GLAM when individuals have already uploaded some of that GLAMs images?

2014-05-03 Thread
On 03/05/2014, Estermann Beat wrote: > Hi Dan, > > It probably should still output a warning and list all identical files, so > they can be tackled manually after the upload. > Giving preference to the media file from the GLAM probably makes sense, but > you still want to substitute any other iden

Re: [Glamtools] Glamtools: how can I get Commons filenames to preserve my brackets and commas?

2014-05-04 Thread
The GWT allows quotes and normal brackets, I have not tested other Commons allowed characters. For example: https://commons.wikimedia.org/wiki/File:%22No.1_Seminary_Building._No._3_Morgan_Hall_Completed_1875._No.2_Library_Building_Dedicated_May_8th_1872._Auburn_Theologigal_Seminary,_%28Incorporated

Re: [Glamtools] input author xml metadata garbled in output author in {{Information}}

2014-05-05 Thread
On 5 May 2014 13:01, Hans Muller wrote: ... > Author Creator:Bert en Lilian Mellink Pieter Christiaansz Bor ... This is a bug that has been a problem on my NYPL uploads (which I have yet to fix). What *probably* should be happening, is that the field should be checked for a possible match to

Re: [Glamtools] input author xml metadata garbled in output author in {{Information}}

2014-05-05 Thread
n due to a new GWT upload. Once the bug is fixed this should be irrelevant. Fae On 5 May 2014 13:21, Fæ wrote: > On 5 May 2014 13:01, Hans Muller wrote: > ... >> Author Creator:Bert en Lilian Mellink Pieter Christiaansz Bor > ... > > This is a bug that has been a problem

Re: [Glamtools] Advice on uploading a batch from a GLAM when individuals have already uploaded some of that GLAMs images?

2014-05-06 Thread
duplicates the GWT has allowed in the NYPL uploads, this is now a longer term post-upload housekeeping issue. Fæ -- fae...@gmail.com https://commons.wikimedia.org/wiki/User:Fae ___ Glamtools mailing list Glamtools@lists.wikimedia.org https

Re: [Glamtools] input author xml metadata garbled in output author in {{Information}}

2014-05-06 Thread
On 5 May 2014 13:59, Fæ wrote: > I'll get Faebot to run through the entire set in a bit. I'll not run > this automatically, so drop a note on my talk page if you need it to > re-run due to a new GWT upload. Once the bug is fixed this should be > irrelevant. On request I re

Re: [Glamtools] GWT best practice/rules for file name characters

2014-05-06 Thread
On 2 May 2014 08:00, Brian Wolff wrote: ... > The encoding should be utf8 with NFC, but even if its not quite correct > gw/mw should convert it. ... SUMMARY Not a bug. Most headaches with the NYPL upload seem to be due to the source data. ANALYSIS Based on more tranches of uploads from NYPL an

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-05-07 Thread
On 7 May 2014 02:37, Gergo Tisza wrote: ... > files on Commons is about 5 files per minute (very unscientific number, I > just looked at the recent changes page), so a full-speed GWToolset run would ... The consequent estimates appear off here. In practice, I have run 5 threads using GWT and the

Re: [Glamtools] GWToolset might be broken on Commons

2014-05-07 Thread
The bug was fixed and I'm using GWT okay this morning. Just uploaded: https://commons.wikimedia.org/wiki/File:Virginie,_Maryland_-_en_2_feuilles_-_par_Fry_et_Jefferson;_traduit,_corrig%C3%A9,_augment%C3%A9._NYPL434441.tiff Fae On 6 May 2014 21:38, Gergo Tisza wrote: > Just a heads-up that GWTool

Re: [Glamtools] GWToolset might be broken on Commons

2014-05-07 Thread
No worries. GWT has been running an upload tranche of the NYPL maps for the last 10 hours (averaging 100MB of files every minute), so it seems to be okay right now. Fae On 7 May 2014 19:19, Gergo Tisza wrote: > On Wed, May 7, 2014 at 1:05 AM, Fæ wrote: >> >> The bug was fixed an

Re: [Glamtools] Maximum filename length for GWT = 206?

2014-05-07 Thread
Thanks for looking into this Dan. Two thoughts where other volunteers on this list might help out: USER GUIDE - EXAMPLE FILE NAME FORMATS The user guide would really benefit from example formats for file names used in our projects. I tend to use: "File: ." with the project abbreviations normally b

[Glamtools] GWT interesting error warning

2014-05-09 Thread
After the 3 file "preview" when running GWT on a fresh batch, it is possible to get the following odd looking error: Internal error [8292dd50] 2014-05-09 22:50:47: Fatal exception of type MWException It looks disasterous, but actually (in my experience of two instances) indicates that GWT has en

[Glamtools] GWT application of Artwork is at variance with Information

2014-05-10 Thread
I am at the final test run stage of a Rijksmuseum upload and getting annoying results when using the Artwork template in GWT, compared to my experience when using the Information template. FILENAME The filename used appears based on + + {mime-type}. This is incredibly annoying as I would like to

[Glamtools] GWT Help: Rijksmuseum upload final checks

2014-05-11 Thread
If anyone has a moment today and would like to comment, I would appreciate extra eyes on the test upload of images on beta at: http://commons.wikimedia.beta.wmflabs.org/w/index.php?title=Category:Images_from_the_Rijksmuseum The way GWT handles the Artwork template is not ideal, and things like the

Re: [Glamtools] GWT Help: Rijksmuseum upload final checks

2014-05-11 Thread
On 11 May 2014 14:07, James Heald wrote: > While you're running the clean-up script to fix all the apostrophes, you > might also prefer the neater (and warning-free) {{PD-old-70-1923}} rather > than {{PD-US}}{{PD-old}} -- though in fact, for these images of 3D objects, > the license needed is sur

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-05-12 Thread
Thanks for the update. I think the only GWT project that has caused the loading issue has been the NYPL maps upload, due to both the large file sizes and the tiff format used. I do have a project in the wings that has similar tiff sizes (100MB+)... some collections from the Library of Congress that

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-05-12 Thread
On 12 May 2014 11:27, dan-nl wrote: > first, i suggest that we put off all large image uploads, > 10mb ( unless we > have a concrete value that would work ), until we resolve the thumbnail issue. > > during the zürich hackathon i spoke with aaron schultz, faiden liambotis, and > brion vibber reg

Re: [Glamtools] GWT Help: Rijksmuseum upload final checks

2014-05-12 Thread
On 12 May 2014 13:24, Hay (Husky) wrote: > Seconded here. Surely the objects are PD, but the photographs are > definitely not, so you probably want to add the CC-zero template as > well. It's a bit of a shame that the metadata isn't very clear on the > license, but for now i guess just checking if

[Glamtools] Beta running?

2014-05-19 Thread
I have been getting "Our servers are currently experiencing a technical problem" when using GWT there. Is there a current known problem, or it is me? Fae -- fae...@gmail.com https://commons.wikimedia.org/wiki/User:Fae ___ Glamtools mailing list Glamtoo

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-05-19 Thread
Hi Gergo, Was there a conclusion from discussions elsewhere? I have some rather nice >100MB tiffs to upload from the Library of Congress, so would like to be able to run these through GWT without making more waves. :-) Fae On 14 May 2014 19:23, Gergo Tisza wrote: > Just a quick note that techn

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-05-20 Thread
On 20 May 2014 13:12, Gilles Dubuc wrote: >> I have some rather nice >100MB tiffs > > How large is that batch? > > We're still working on technical changes, nothing has been merged since the > last outage. It should be small, as it is the exception that is over 50MB, let alone 100MB. Some of it i

Re: [Glamtools] Beta running?

2014-05-21 Thread
-object Thanks, Fae On 19 May 2014 18:19, Fæ wrote: > I have been getting "Our servers are currently experiencing a > technical problem" when using GWT there. Is there a current known > problem, or it is me? > > Fae > -- > fae...@gmail.com https://commons.wikimed

Re: [Glamtools] Beta running?

2014-05-21 Thread
On 21 May 2014 11:10, Fæ wrote: > Anyone know more about the GWT outage? It has not been working for me > on production or beta servers for a couple of days now. > hey fae, > > after logging into beta, i was able to browse to > http://commons.wikimedia.beta.wmflabs.org/wiki

Re: [Glamtools] Image resizing in beta test uploads

2014-05-23 Thread
On 23/05/2014, Crockford, Ally wrote: ... > Example link: > http://commons.wikimedia.beta.wmflabs.org/wiki/File:Imaginative_depiction_of_the_completed_Forth_Rail_Bridge.jpg This looks like a possible bug worth reporting if you can duplicate this error. I have not seen this on my beta tests. I am

Re: [Glamtools] Using an open FTP server

2014-05-26 Thread
The FTP site would need to be added to the tool's whitelist. Follow the link on the first form of the GWTool - or the link below (you will need to set up a bugzilla account). Link * https://bugzilla.wikimedia.org/enter_bug.cgi?assigned_to=wikibug...@lists.wikimedia.org&attach_text=&blocked=58224&

[Glamtools] Request: GWT eliminating the source link

2014-06-24 Thread
I have an unusual scenario where the GLAM might make images available either under a one-shot deal where the source website might vanish after upload (possible a transient FTP share) or where I am allowed to systematically use high resolution links that are not normally directly declared but only p

Re: [Glamtools] [GWToolset] “Nothing happened” + Logging?

2014-06-26 Thread
Sorry, no. It is highly likely that the upload fell over on the first record due to a problem with the xml, such as missing the xml header, or not being wrapped in a parent tag. Debugging is a pain due to this user unfriendliness. My tips would be: 1. Run the xml through the validator (linked on

[Glamtools] GWT: What volume of uploads is too much for the WMF?

2014-06-30 Thread
This is a simple sounding question, but I have two uploads going on in parallel right now, one using 8 processing threads and the other using 16, so a total of 24. None of these files is huge, they seem to be under 15mb, with an occasional outlier around 45mb (though quite a few drawing scans break

Re: [Glamtools] GWT: What volume of uploads is too much for the WMF?

2014-07-01 Thread
On 01/07/2014, Gergo Tisza wrote: >> Any thoughts? If what I'm doing is somehow self-regulating, I would be >> tempted to add another job and bump the "volume" to 40 or more >> threads, as this particular upload has over 100,000 images >> (potentially 200,000) and I'd rather it didn't take over a

Re: [Glamtools] Help with error message in the test servers

2014-07-02 Thread
On 02/07/2014, Monica Mora wrote: > There was a problem processing the metadata file. > > There is no block for template Information. > > If possible, add a block > > to the template, or to the template's Templa

Re: [Glamtools] Help with error message in the test servers

2014-07-02 Thread
On 02/07/2014, Jean-Frédéric wrote: ... > Reproduced and {{fixed}} ... > The {{Information}} template, as well as its documentation, on the Beta > Cluster was imported from Commons a long time ago, at a time where there > was no TemplateData there. Thanks JF, that looked complicated. I'm glad to

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-07-10 Thread
On 10/07/2014, Gergo Tisza wrote: > Hi all, > > with bug 65691 fixed > (the last patch was deployed today), now might be a good time to test large > TIFF uploads again (the patch is limited to TIFF files for now). I was > thinking of the follo

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-07-12 Thread
On 10/07/2014, Fæ wrote: > or if there is some trick to forcing the thumbnail recreation. If > these work well, then I'll stop creating PNGs for the TIFF drawings Ah, rereading the bugzilla ticket (oops, I should have paid more attention) shows that this will not fix the thumbnail cre

Re: [Glamtools] GWToolset operational issues for very large TIFFs

2014-07-18 Thread
No worries. I will have time to set something up in a couple of weeks. The Wellcome uploads will be a last minute priority before Wikimania, my focus will probably be on that. On 18 Jul 2014 19:43, "Gergo Tisza" wrote: > On Thu, Jul 10, 2014 at 3:08 PM, Gergo Tisza wrote: > >> Hi all, >> >> wit

[Glamtools] GWT may have breaks in uploads for an hour or more

2014-07-23 Thread
Hi, Could someone explain what is happening when large uploads have long breaks? My long term HABS upload (250,000 files) has had notable pauses, mostly around an hour, though this morning has seen a 4 hour break in uploading. I had thought that the upload has completed, but as soon as I started a

Re: [Glamtools] Uploading books via GWtoolset - query

2014-07-23 Thread
On 22/07/2014, Crockford, Ally wrote: > The NLS has some digitised books that I am keen to upload to Commons where > appropriate and I'd like to do so as separate image files (for a number of > reasons - can go into them if you'd like but would rather save time). > > My question is whether anyone

Re: [Glamtools] GWT may have breaks in uploads for an hour or more

2014-07-24 Thread
This appears to be a longer running issue, raised a bug request at: https://bugzilla.wikimedia.org/show_bug.cgi?id=68506 Fae On 23/07/2014, Fæ wrote: > Hi, > > Could someone explain what is happening when large uploads have long > breaks? My long term HABS upload (250,000 files) has

Re: [Glamtools] GWT may have breaks in uploads for an hour or more

2014-07-25 Thread
On 24 July 2014 13:15, Fæ wrote: > This appears to be a longer running issue, raised a bug request at: > https://bugzilla.wikimedia.org/show_bug.cgi?id=68506 Has anyone been successfully using GWT for uploads in the last few days? I believe this bug effectively makes it offline. Fae -

Re: [Glamtools] hyphens in xml Re: =?iso-8859-1?Q?=C2=AB_GWToolset_fails_to_upload_files_and_throws_no_warni?=ng »

2014-07-25 Thread
On 25/07/2014, Hans Muller wrote: > Hi Jean-Frederic, > > spurious hyphens > > Perhaps a trivial remark: in your xml-file attached to the bug report in > the first lines there are unnecessary hyphens (-) which cause the > XML-validator (step 5 GWTools) to fail. I have run into th

Re: [Glamtools] GWToolset upload silently fails

2014-09-01 Thread
Nothing jumps out at me. You may want to double check for odd use of "&" or "--" which may cause issues. If you header line were wrongly formed this might also cause it to drop out, or there might be an issue that is related to the particular way you are mapping to the Commons template. I suggest

Re: [Glamtools] GWToolset upload silently fails

2014-09-02 Thread
On 02/09/2014, Ari Häyrinen wrote: > Thanks for your response. > > Here is the XML. It is valid and files exists. You can download it > here: http://opendimension.org/misc/first_test.xml This appears to be a bug in the beta cluster version, but not in the production environment. We could raise th

Re: [Glamtools] Problem with wiki syntax

2014-10-23 Thread
2014-10-23 14:15 GMT+01:00 Nicolas Rück : > Hello, > > we just uploaded some files with the GWToolset, but wiki syntax like > '''xxx''' for bold text and html-tags don't work. > > The description text should look like this: > https://commons.wikimedia.org/wiki/File:Siegelmarke_Schulvorstand_zu_Nerc

Re: [Glamtools] Change of direction in GWT 2.0 scoping

2014-11-04 Thread
Ideally what we would like as a next stage is a user interface improvement, the underpinning technology works and can be refined to match. This solves the usability issues. For example rather than talking xml, it makes more sense to point a tool at your online archive (or hard disk collection) and

Re: [Glamtools] GWToolset uploads stalled?

2014-12-01 Thread
On 1 December 2014 at 19:03, Jean-Frédéric wrote: ... > Should we open a Phabricator task for this ? Not sure this list is monitored > by the relevant folks. Please do. This is a tiny list for users, and stuff that pops up we ought to log so that it can be planned for. It's unfair to rely on Dan

Re: [Glamtools] Seeking feedback. Please read the GWT 2.0 grant application draft

2015-02-19 Thread
On 19 February 2015 at 14:37, MZMcBride wrote: > I personally think we must give a lot more thought to the broad strategy > being used here. For example, if you want to upload millions of files, why > not put them on a hard drive and ship them? It will be dramatically faster > and a lot less waste

Re: [Glamtools] Toolset hiccoughs?

2015-05-11 Thread
It is not a guaranteed service. Training materials would sensibly have worked cases sufficient, if it is offline at that moment. I would be happy to walk through my own pragmatic belt and braces approach to large uploads, for an old GLAM friend of mine such as the BL. Fae On 11 May 2015 16:43, "Al

Re: [Glamtools] Domain whitelisting

2015-05-12 Thread
I suggest contacting odder or dropping onto IRC. In any plan the white list is best requested a few days in advance to allow for questions. Fae On 12 May 2015 13:36, "Ally Crockford" wrote: > Hi all > > Training is progressing well here at the BL, and there are several > curators with XML files

Re: [Glamtools] Domain whitelisting

2015-05-12 Thread
James, these were avoidable issues. Had anyone reviewed the timetable with me, or asked for my assistance, based on my experience as the primary user of GWT, I would have suggested how to avoid these foreseeable potholes. Fae On 12 May 2015 21:56, "James Heald" wrote: > Hi Brian, > > Yes, https

Re: [Glamtools] Hello + language template enquiry

2015-05-20 Thread
Hi Nora, The Commons language policy can be found at < http://commons.wikimedia.org/wiki/Commons:Language_policy>. To put any language in an artwork or information template, you just add a template around the text like {{en|1=}} for english or {{zh|1=}} for chinese. The language templates are list

Re: [Glamtools] Hello + language template enquiry

2015-05-20 Thread
On 20 May 2015 at 13:40, McGregor, Nora wrote: ... > Romanised Japanese (?) ... > Romanised Chinese, Specifically Pinyin (?) With regard to these specific language challenges, if there are no suitable templates listed at , then it may

Re: [Glamtools] log file for each batch?

2015-06-01 Thread
This was discussed (last year I think). It would make for a good small improvement to GWT, especially if the user could tick a box and be emailed a results report. It is very frustrating when a minor flaw in your XML input file makes the GWT fall over and you have to debug without any immediate fee

Re: [Glamtools] WMCH server slowed down and GWToolset still trying to download

2015-06-02 Thread
There's no kill button. The job would have to be killed by an operator (a phabricator request is probably in order). As a general option, it would be a bad idea to have a general kill process, what would be more useful would be a process for an end user to kill/pause a job request using a given job

Re: [Glamtools] WMCH server slowed down and GWToolset still trying to download

2015-06-03 Thread
On 3 June 2015 at 15:10, Brian Wolff wrote: ... > I think this suggests that gwtoolset should have some sort of back-off > feature when things start to fail (particularly due to "HTTP request > timed out.") to slow down the request rate. When I used to analyse big stuff like oil rigs, part of the

Re: [Glamtools] Mass Upload

2015-07-27 Thread
On 27 July 2015 at 15:14, Jason J. Evans wrote: > Sorry to keep asking so many questions, but i have noticed with our upload of > Welsh landscapes that several different images often have the same file name. > Each upload is just replacing the previous file by that name. Is there some > way to

Re: [Glamtools] New at this and trying to configure our API

2015-09-11 Thread
On 11 September 2015 at 10:43, Jane Darnell wrote: ... > Here is a painting that I was viewing a few minutes ago: > https://commons.wikimedia.org/wiki/File:De_molen_bij_Wijk_bij_Duurstede_Rijksmuseum_SK-C-211.jpeg Hey, I uploaded that. :-) My approach to these initial Rijks uploads was to just g

Re: [Glamtools] Can somebody kill my GWT-job?

2015-09-17 Thread
Ouch. For my very large uploads, I ran a maximum of 5,000 at a time, even when the process was stable. This meant I could have a quick check of each "tranche" and if the worst came to the worst, I could identify the tranche and stick them in their own category or even get them deleted post-upload.

Re: [Glamtools] upload from Rijksmuseum has stalled?

2015-09-22 Thread
On 22 September 2015 at 12:06, Hans Muller wrote: > Hi Lizzy and Sandra, > > Just check the GWToolsetbox "Upload again/Opnieuw opladen" just before > submitting the job. If not, error messages like Lizzy's ensue because the > requested filenames are in use already. GWToolset doesn't dare to > over

Re: [Glamtools] restart Re: upload from Rijksmuseum has stalled?

2015-09-28 Thread
I have seen unexplained drop-outs for my direct uploads (not GWT) since 25 Sept and I see 4 files of mine created today with empty image text pages. I'd agree with leaving it for a day and trying again rather than spending a lot of time investigating. If there is a pattern of failures of the same

Re: [Glamtools] upload from Rijksmuseum has stalled?

2015-09-29 Thread
On 29 September 2015 at 10:10, Lizzy Jongma wrote: ... > We also encountered some character set issues: I cannot explain where that > came from but I will research it a bit more. Character encoding problems can be very tricky. There is a short bit of guidance in the GWT manual on XML sensitive t

Re: [Glamtools] TIFF error when using Bulk Uploader

2015-10-27 Thread
Large TIFFs can be problematic. For the some of my past projects relying on TIFFs, I have created PNG files and JPEGs to help with their reuse, but always uploaded the original TIFF. Note that at the current time TIFFs cannot be browsed in the media player, so they are not ideal if you want the ima

Re: [Glamtools] TIFF error when using Bulk Uploader

2015-10-27 Thread
On 27 October 2015 at 19:49, Brian Wolff wrote: ... > Re Fae's comment: I'm not sure what you mean. Tiffs work fine in media > viewer for me. By this, I mean that you can't view the fullsize TIFF within the viewer or browser, while other formats work as you might expect. Fae -- fae...@gmail.com

Re: [Glamtools] Upload from hard disk

2015-11-17 Thread
If you get stuck, I'll be happy to upload them from a USB stick or via an online drop, so long as the metadata was easy to sync with filenames and this was around Dec/Jan time. A custom upload of 30k medium size files, with no special tools, would take me about a week. Fae

[Glamtools] Rights for GLAM group accounts

2015-11-19 Thread
Link: https://commons.wikimedia.org/wiki/Commons:Bureaucrats'_noticeboard#Rights_for_GLAM_group_accounts I have raised this question on the Bureaucrat's noticeboard. As not that many people watch that page, I'm flagging it on this email list, though if you would like to join in with discussion on

Re: [Glamtools] GWToolSet stopped 25 nov 2015 15:36?

2015-11-26 Thread
The XML input is hidden in the image pages (even parts not displayed on the output page). These could be datamined for wikidata or external use, even if the uploader does not have the time or experience to do it themselves... Maybe worth discussing on a wikidata noticeboard? :-) Fae On 26 Nov 201

Re: [Glamtools] Overwriting possible if file not edited by other users Re: Same user? Re: Yes Re: Batch re-uploading images to Commons in higher resolution?

2015-12-18 Thread
This is a good option for how GWT should handle "upgrades". Where other volunteers have edited it makes sense to tidy these up manually or by on-project negotiation. As an example, many of the 100,000 Wellcome images I uploaded were plate illustrations, and I did go through a process of uploading

Re: [Glamtools] [GWToolset] Uploading 350 books - via command line or API?

2016-02-15 Thread
The easiest way is to merge your XML files into one large XML, then put this through the GWT. Though I can imagine ways of automating the front end of GWT, it would be a clumsy way of going about it. If your concern is that you want to create separate book categories, then add a category field in

Re: [Glamtools] adding 3134 captions after upload?

2016-05-09 Thread
There's no 'painless' solution. I have made similar housekeeping updates to very large batch uploads post-upload using pywikibot using short scripts. Amongst other things, pywikibot can pull up the specific image page history and tell which past edits have been made and by whom, then make intellige

[Glamtools] ParameterAssertionException

2016-06-04 Thread
Does anyone know what this error indicates? When I get to the stage of previewing uploads on GWT, I get a page with just this: Exception encountered, of type "Wikimedia\Assert\ParameterAssertionException" I've tried swapping around parameters and I'm using the most basic information template, whic

Re: [Glamtools] ParameterAssertionException

2016-06-04 Thread
xed. Fae On 4 June 2016 at 22:06, bawolff wrote: > Its caused by the Linker rewrite. If someone adds an invalid page name > to a LinkBatch object, it triggers this issue. > > I'll look into fixing it. > > On Sat, Jun 4, 2016 at 1:10 PM, Fæ wrote: >> Does anyone know

Re: [Glamtools] GWToolset labs instance

2016-07-15 Thread
I do not remember this coming up as part of the GWT Steering Group discussions. I suspect it was used as a website to focus information during the early test phase and has been superseded by the on-wiki manual. I'm not confident enough to officially say it can be purged, but I think it's low risk

Re: [Glamtools] GWToolset User Right

2016-09-15 Thread
Hi Tom, You need to add your request to https://commons.wikimedia.org/wiki/Commons:Bureaucrats%27_noticeboard Make sure you have checked the advice of the manual. Fae On 15 Sep 2016 16:17, "Derrick, Tom" wrote: > Good afternoon, > > > > Please may I request GWToolset User Right on Beta Common

Re: [Glamtools] GWToolset User Right

2016-09-15 Thread
Oh, I'll add you to beta in a moment though. Fae On 15 Sep 2016 16:20, "Fæ" wrote: > Hi Tom, > > You need to add your request to > https://commons.wikimedia.org/wiki/Commons:Bureaucrats%27_noticeboard > > Make sure you have checked the advice of the manual.

Re: [Glamtools] GWToolset User Right on Beta Commons

2016-09-19 Thread
Hi Adi, What's your account name there? Fae On 19 Sep 2016 10:29 a.m., "Keinan-Schoonbaert, Adi" < adi.keinan-schoonba...@bl.uk> wrote: > Hi there, > > > > Could I please request GWToolset User Right on Beta Commons? > > > > Many thanks, > > Adi > -- > > [image: Desc

Re: [Glamtools] Wikimedia Commons | GWToolset user rights for the National Museum of Australia [SEC=UNCLASSIFIED]

2016-10-17 Thread
Hi Beck, The GWT rights have been added to https://commons.wikimedia. beta.wmflabs.org/wiki/User:NatMuseumAust. There are useful cases in the manual https://www.mediawiki. org/wiki/Help:Extension:GWToolset to get you started. If you need to white-list new sites to upload from, I suggest raising a

Re: [Glamtools] wiki markup and XML

2016-12-15 Thread
It may be easiest to try {{italics|Your text here.}} As you are embedding within xml, most characters will end up escaped, but I think brackets and pipes are okay as they don't interfere with xml standards. Once you have something working, it may be worth adding the solution to the GWT manual. F

[Glamtools] Future of the GLAMtools email list

2017-05-10 Thread
* https://lists.wikimedia.org/mailman/listinfo/glamtools * The glamtools list we created a couple of years ago, after launching the GLAM wiki toolset, has recently had very few users asking questions about tools. On behalf of the list mods, I invite feedback on whether the list should be re-promo

Re: [Glamtools] Future of the GLAMtools email list

2017-05-17 Thread
Based on feedback the plan is to close the list, keeping the list archive available for reference. The comment on the Commons Village Pump was useful, highlighting that Commons-l is a good place for Commons upload tool discussion, especially if the topic is indicated, i.e. put "[Tools]" or similar

Re: [Glamtools] Title-field in GWtoolset?

2017-05-27 Thread
Hi Jesse, We are closing down this list, could you try re-posting to commons-l with a topic of [TOOLS]? There should be enough folks around with GWT experience there. Thanks, Fae On 26 May 2017 at 12:39, Jesse de Vos wrote: > Hi all, > > It's been a while since I last used the GWtoolset. I'm u