Re: [Toolserver-l] Data wipe tomorrow
On 11/12/14 10:59, Marlen Caemmerer wrote: Hello, tomorrow is the day I will start to completely throw away all data on the Toolserver. After this nothing can be copied anymore. If you feel you still need anything please let me know asap. Cheers nosy Are http://svn.toolserver.org/ repositories kept somewhere? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] sql command
On 18/12/13 18:17, Merlijn van Deen wrote:
On 18 December 2013 17:52, John phoenixoverr...@gmail.com wrote:
I am in the process of migrating to labs and the command sql
performs differently between the two platforms, Can I get the source
for the toolserver version of it so that we can make the labs
version work the same.
Check /opt/local/bin/sql (on solaris) or /usr/local/bin/sql (on linux).
The file is a publicly readable shell script, but for the benefit of the
archives, I'm attaching it.
#! /bin/ksh
PATH=$(getconf PATH):$PATH
usage() {
echo 2 usage: $0 [-r | -u] database [command]
echo 2 -rConnect to round-robin database
echo 2 -uConnect to user database
exit 1
}
uflag=0
rflag=0
while getopts ru opt; do
case $opt in
u) uflag=1;;
r) rflag=1;;
*) usage;;
esac
done
shift $(( OPTIND - 1 ))
if [[ $rflag = 1 $uflag = 1 ]]; then
echo 2 -u and -r cannot both be specified
usage
fi
if [[ $# = 0 ]]; then
usage
fi
if [[ $# 1 ]]; then
query=$2
fi
if [[ $1 == toolserver ]]; then
server=sql-toolserver
elif echo $1 | grep -q '^[up]_'; then
server=sql
elif [[ $1 == centralauth_p ]]; then
server=z-dat-s7-a
else
db=$(mysql -BN -hsql-toolserver -e select server from wiki where
dbname='$1'; toolserver)
if [[ -z $db ]]; then
echo 2 $0: no such database \$1\
exit 1
fi
if (( $uflag )); then
server=sql-s${db}-user
elif (( $rflag )); then
server=sql-s${db}-rr
else
server=sql-s${db}
fi
fi
if [[ -z $query ]]; then
mysql -h $server --comments $1
else
mysql -h $server --comments $1 -e $query
fi
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] FYI: Toolserver account expiry in early January
the second tool deletes all this data as well as expires your account. The tools migrated to labs should be 301-redirecting to the tools new home, not showing an Account expired message. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cron on nightshade not running since Nov 2nd?
On 06/11/13 09:59, Federico Leva (Nemo) wrote: I don't know if cronie is recommended, but https://wiki.toolserver.org/view/Cron#Conversion Nemo On nightshade, crontab *is* cronie. The cron server is running and -after a quick test- working. The problem seems to lie with emails, which are not sent. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cron on nightshade not running since Nov 2nd?
On 07/11/13 01:30, Christian Thiele wrote: Hello, Am 06.11.2013, 22:49 Uhr, schrieb Platonides platoni...@gmail.com: On nightshade, crontab *is* cronie. The cron server is running and -after a quick test- working. The problem seems to lie with emails, which are not sent. I spoke about willow, and there crontab didn't work (I don't receive any mails from cron (everything goes to /dev/null), but I write log files). I don't know what's the case with nightshade. I didn't thought about it because I changed my cron and on another MMP project crontab never worked on willow (only cronie), so I thought it's some problem with crontab. So I thought it was only my problem and switched to cronie for my user account, too (and it worked again). But then a mail was written by Danny. So it seems to be a general problem with crontab on willow. Chris / APPER Well, Danny was talking about nightshade. It is indeed the case that in willow crontab and cronie are different processes, and sometimes crontab dies (seems to be running, though). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Need Tool Labs migration support?
On 08/10/13 00:34, Marc A. Pelletier wrote: On 10/07/2013 06:27 PM, Magnus Manske wrote: * Forcibly port the most important, unmaintained tools to Labs Sadly, that's only possible when the code is unambiguously licensed as Open Source, which is often not the case (much of the code running on the Toolserver has no licensing information at all). -- Marc Then the default license applies. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Pywikibot, GIT and TS
On 10/08/13 13:05, Dr. Trigon wrote: If you read the instructions (https://www.mediawiki.org/wiki/Manual:Pywikipediabot/Gerrit), it recommends you only clone with a --depth of 3 (or anything smaller), and it will take up very little space. Yes of course there is a solution - but it again one thing that can go wrong and another paramter you have to remeber and use. By the way it would be a good thing if it would be possible to develop from TS also. And then all the 200MB are needed, aren't they? No, they aren't. You can happily develop from a shadow clone. You would only have problems if someone sent you a patchset based on an old version you didn't clone (but when working from gerrit, it won't happen). You would of course not have the commits previous to the point you set, so you won't be able to properly view the commit list or blame. But that's not a big difference with svn, where those requiring downloading the full file history from the net (I went to ViewVC for those actions). In git they are nice with a full clone, but they won't work properly with a shallow clone (eg. they will attribute everything “old” to the earliest commit you cloned). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] TS and/vs. Labs
On 11/08/13 08:36, Silke Meyer wrote: * Most tools will just run when you copy them to Tool Labs. Just in case: Need help adapting a tool? Ask me and I'll do what I can to get you help, e.g. from my colleague Johannes Kroll. Provide a toolserver.wiki table please. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Login issues
On 23/07/13 22:18, Emilio J. Rodríguez-Posada wrote: Hi; I'm having login issues since some days ago. I'm trying ssh emi...@login.toolserver.org mailto:emi...@login.toolserver.org and get password failures. I'm almost sure I have not changed my password... Can you verify that my account is OK? Thanks, emijrp Users can't do password authentcation. Do you have your ssh key loaded? Your account is not expired. I see a strange statement when fingering you: Last login Tue Jul 23 21:55 on null So perhaps it wasn't able to allocate a pty? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Large core dumps found in home directory
On 19/06/13 00:12, Derrick Coetzee wrote: Hi all, I noticed today I had several large core dumps in my home directory, and two buried under my public_html directory. The directories I found them in contained no executables and as far as I know were not working directories for executables (just PHP scripts used by the web server), so I can't imagine how they got there. -rw--- 1 dcoetzee users 191463424 May 5 06:56 core -rw--- 1 dcoetzee users 38797312 Sep 29 2012 core This exceeded my hard quota without my knowledge. Any idea what's up with this? Can I prevent it from reoccurring? Thanks. -- Derrick Coetzee User:Dcoetzee The php fastcgi script crashed and left a useless core. This has happened for a long time, see https://jira.toolserver.org/browse/TS-1505 but it had (apparently) been fixed, so perhaps it got reintroduced when the web servers were reconfigured? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] toolserver web down
toolserver.org isn't listening on port 80: willow ~ $ telnet toolserver.org 80 Trying 2a02:ec80:101::1:4... telnet: connect to address 2a02:ec80:101::1:4: Connection refused Trying 185.15.59.214... telnet: Unable to connect to remote host: Connection refused Connecting directly to the web servers works. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] commonswiki_p corrupted
Commons copy on s3, s4 and s6 are corrupted. The versions at s1 and s5 are not affected. Test case: mysql -h $server commonswiki_p -e SELECT page_title FROM page where page_id=21683917 Good: +--+ | page_title | +--+ | Dworzec_główny_w_Gdyni.jpg | +--+ Bad: +--+ | page_title | +--+ | Dworzec_główny.jpg | +--+ It looks like a range of statements from November 2012 weren't replicated. Other tests are: SELECT log_id, log_title, log_type, log_action FROM logging_ts_alternative where log_namespace=6 AND log_title='Dworzec_główny.jpg' The wrong copies don't have «| 48917341 | Dworzec_główny.jpg | move | move |» (the other 7 entries appear in both) SELECT rev_id, rev_timestamp, rev_comment FROM revision where rev_page=21683917 The wrong copies don't have the last 4 revisions (all of them from 2012-11-03) 'Dworzec_główny_w_Gdyni_4.jpg', 'Dworzec_główny_w_Gdyni_3.jpg', 'Dworzec_główny_w_Gdyni_2.jpg' are similarly affected, shown as just Dworzec_główny_0X.jpg in the wrong dbs. Comparing the number of revisions per day, it seems to have fixed on 2013-11-04, with the 3 being the most noticeable (27746 revisions where they should have been 53974), a difference of 80 revisions the previous day, 200 on 2012-10-27... It should be possible to just copy the db from s1. s1 and s5 are probably sane due to the reimports on January and April. As an unexpected thing, sql-s2 and sql-s7 don't have a commonswiki_p copy. sql-s3 and sql-s6 don't have a commonswiki_p.revision table/view. I created https://jira.toolserver.org/browse/TS-1667 ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Encoding issue using SGE
On 23/05/13 00:38, Hercule Hercule wrote:
Hello,
I deleted one of my script, and try to recreate it. But I now have an
encoding issue when I run it with SGE. I searched a solution during all
this evening and can't find any idea to solve it. I hope one of you will
be able to help me.
I created the file /home/hercule/batch/LienFr.sh with this content :
#-*- coding: utf-8 -*-
#!bin/bash
#! /usr/bin/python
These shebangs are crazy. You would want just #!/bin/bash as the first
line.
#$ -l h_rt=1:30:00
#$ -l virtual_free=500M
#$ -l arch='*'
#$ -m as
#$ -o /home/hercule/logs/LienFr.log
#$ -j y
#$ -wd /home/hercule/
python /home/hercule/template.py Lien Lien/Conversion automatique
-cat:Page utilisant Lien pour un article existant -summary:[Bot] :
transformation de liens avec le modèle {{Lien}} en lien interne, suite à
la création de l'article correspondant -assubst -pt:0 -always
mutt -s Fin de LienFr hercule /home/hercule/logs/LienFr.log
When I simply run /home/hercule/batch/LienFr.sh in the terminal the
result is correct :
http://fr.wikipedia.org/w/index.php?title=Uttar_Pradeshdiff=93370887oldid=93370870
But when I use the command line
qcronsub -N LienFr /home/hercule/batch/LienFr.sh
I have a problem with the summary :
http://fr.wikipedia.org/w/index.php?title=Uttar_Pradeshdiff=93370948oldid=93370932
Do you know how to solve this ?
Regards
Hercule
Seems the string was converted to utf-8 twice. What did you use to edit
the file?
Try converting it with iconv:
iconv -f utf-8 -t iso-8859-1 LienFr.sh LienFr2.sh
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Maintenance: Rebooting ortelius web server
On 09/05/13 18:57, Tim Landscheidt wrote: Marlen Caemmerer marlen.caemme...@wikimedia.de wrote: I would like to reboot ortelius, one of the web servers at tomorrow, Tuesday 1830 UTC Apparently, wolfsbane rebooted today as well: | timl@wolfsbane:~$ uptime | 16:49pm up 5:00, 2 users, load average: 1.16, 1.24, 1.47 | timl@wolfsbane:~$ Perhaps related to that, SGE queues on ortelius and wolfs- bane are in state au (alarm, unknown): Yes, sge_execd seems not to be running on them. Plus medium and longrun queues in yarrow are in error state. I tried cleaning them, but they failed again. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver limitation to come?
On 02/05/13 10:03, Marlen Caemmerer wrote:
Hey,
On Wed, 1 May 2013, Ryan Lane wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt
t...@tim-landscheidt.dewrote:
There were never answers to this, so I bring it up here
again:
1. How were almost /all/ of the problems the TS have had
with replication caused by that redundancy and trying
to keep it synced?
Thanks for this question :) - I also want to know.
From my perspective it does not look like this and even the data
inconsistencies appear when we have no commons copy on a mysql instance.
And: DaB experimented with federated tables for commons too and we
decided to not do this since it does not perform from the start.
Probably nowadays when I planned something new in this area (which does
not seem to make sense for TS) I'd really give Galera a try -
http://codership.com/content/using-galera-cluster
FWIW, my {{ref needed}} phrase in the log was also intended to that
statement by Coren, not to multichill reply.
Additionally, I rembember you data inconsistencies happen even with
native mysql replication (as informed by Nosy earlier).
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Survey: Moving to Labs
On 01/05/13 21:53, DaB. wrote: Hello all, until now I had the impression that we (you, the authors, and me) fight together against WMF and WMDE for keeping the Toolserver and against Labs. Some mails and discussion in the last days gives me now the impression that this was wrong and (at least some of) you are eager to leave the toolserver as soon as possible. It does look at times as if they wanted to remove the toolserver from behind us, but it shouldn't be considered a fight. In this situation, the interest for that new-über-replacement it's completely normal. It doesn't mean that they love one more than the other. There is no point to beg the WMDE for new hardware and to invest much more time if 2 weeks after Labs is ready the toolserver will be empty. For this reason I created a survey at [1] that starts at midnight. Please take a moment of your time and place your nick in the section that suits you. Sincerely, DaB. Even if labs being ready happens in 2018? ready will vary for each tool, but I foresee a process like this: 1. labs provides all the resources needed for $TOOL 2. $AUTHOR signs up in labs, gets added to the projects, etc. 3. $AUTHOR tests (benchmarks) labs and finds it acceptable for $TOOL 4. $AUTHOR learns all the labs-specific details. 5. $AUTHOR allocates some time for installing $TOOL in labs 6. Fix bugs in $TOOL when run in labs (aka. adapt $TOOL to labs) 7. (Optional) Redirect toolserver/$TOOL to labs/$TOOL For the majority of tools, we haven't reached #1 yet. Once labs provides (almost) everything available at toolserver, you can start talking about when to stop supporting TS. But doing otherwise is premature. #2 is the only step that could take place before #1. Then there is the 'lazy factor' for #2-7, although it is also known as too busy to fix this program which works ok in TS Some people may skip #3, while others will want to be damn sure that it will work correctly. The time required by #4 can be reduced making labs very similar to toolserver. If labs environment for the programs is very different, such as needing to do the joins manually inside the program, that will increase #6 a lot. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Doxygen and coverage
On 28/04/13 13:44, Dr. Trigon wrote: Hello Admins! Could please someone take care of TS-1301 [1]? This ticket is now open for quite a while and having the possibility to generate docs and coverage reports on the TS would be very favourable. [1] https://jira.toolserver.org/browse/TS-1301 Thanks a lot and Greetings! DrTrigon I tried to add a puppet entry for that in the devel manifest, but my commit was rejected (access to the file forbidden). I left a question there regarding the coverage pkg. PS: It seems that there was a temporary LDAP failure yesterday night, I had a number of jobs in error status with reason «can't get password entry for user platonides. Either the user does not exist or NIS error!» and there were ‘temporarily not available’ queues. I have reenabled the affected queues (short-...@wolfsbane.toolserver.org, longrun...@yarrow.toolserver.org, medium-...@wolfsbane.toolserver.org, medium...@mayapple.toolserver.org) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] I can't login any more
On 24/04/13 10:54, K. Peachey wrote: IIRC that should still log you in, but you get a You account has been disabled message and your web folder should show a this account has expired message. I'm sorry, but it does seem disabled: http://toolserver.org/~alebot/ Your account was set to expire on on 2013-04-05 (15800). Open a jira bug (or otherwise bug a toolserver admin) to request it is enabled again. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] I can't login any more
On 24/04/13 16:25, Alex Brollo wrote: Thanks - really I felt that I should get the renew notice but I obviously missed/deleted the mail; I've been confused from some toolserver stop-and-go notices, and from some messages by other user, with login troubles. Yes, you are true: I found the notice Your account will expire tomorrow into the group of deleted mails I apologyze for your wasted time. Me stupid. Alex Being busy is different than being stupid. Plus this was easy to determine, don't worry. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronie jobs with specified ENV are not executed
On 24/04/13 20:16, Adam Klimont wrote: My cronie task list looks like this: PATH=/opt/ts/gnu/bin:/sge/GE/bin/sol-amd64:/usr/bin:/bin 5 1 * * * qcronsub -N 'wymowa.py' -b y -l h_rt=4:00:00 -l virtual_free=50M /usr/bin/env PYTHONPATH=$HOME/wikt/pywikipedia /usr/bin/python2 $HOME/wikt/moje/wymowa.py /dev/null 6 2 * * * qcronsub -N 'frequency.py' -b y -l h_rt=0:30:00 -l virtual_free=20M /usr/bin/env PYTHONPATH=$HOME/wikt/pywikipedia /usr/bin/python2 $HOME/wikt/moje/frequency.py /dev/null ...and so on It stopped working around 2-3 weeks ago. Now I get an error message: /usr/bin/env: No such file or directory What should I change in these lines in order for the scripts to work again? alkamid There's no python2 binary on debian nor solaris. Just replace 'python2' with 'python'. In both OS you will get a python 2 release (2.6.6 and 2.7.1 respectively). Although I agree it would be nice to have python2 as a symlink. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] I can't login any more
On 24/04/13 00:15, Alex Brollo wrote: Thanks, but server closes connection as soon as I try to login - both with PuTTY and by WinSCP. No matter; I don't want to waster your time any more; I'll search for help y some friends into it.wikisource. Alex What's your username? Did your perhaps allow your account to expire? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Qcronsub and qstat trickery
On 21/04/13 17:21, Theopolisme wrote: The full content of the script is [https://github.com/theopolisme/theobot/blob/master/bsr_notifier.py]. Nowhere in the script is any sort of image processing. I'm suitably mystified. The shebang got ignored, somehow, (did bsr_notifier.py have execute permissions?), and sge tried to be execute it as a shell script. Thus the import lines called import(1) from imagemagick (this program tries to open the X server to take a screenshot) On linux there's also a from command which prints out the mail header lines from the invoker's mailbox, which produced the /var/mail errors. See the beginning of the file annotated with the errors given: #! /usr/bin/env python - Comment import mwclient - import: unable to open X server `' @ error/import.c/ImportImageCommand/362. import re - import: unable to open X server `' @ error/import.c/ImportImageCommand/362. import sys - import: unable to open X server `' @ error/import.c/ImportImageCommand/362. import datetime - import: unable to open X server `' @ error/import.c/ImportImageCommand/362. import pickle - import: unable to open X server `' @ error/import.c/ImportImageCommand/362. from theobot import bot - from: can't read /var/mail/theobot from theobot import password - from: can't read /var/mail/theobot # CC-BY-SA Theopolisme - Comment # Task 9 on [[User:Theo's Little Bot]] - Comment def sokay(donenow): - Syntax error: ( unexpected I see you recently changed the shebang in order to add a space before the path: https://github.com/theopolisme/theobot/commit/fe58c45b6e8997177d27f49ef9f25dba42171ee5 Both shebangs are ok. I will send you privately an email regarding a problem I noticed in your tool. Regards ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Qcronsub and qstat trickery
On 21/04/13 19:22, Theopolisme wrote: Okay, I found this useful old message in the toolserver-l archives: [http://osdir.com/ml/toolserver-l/2011-01/msg00015.html]. I configured everything like the message suggested [https://github.com/theopolisme/theobot/commit/302e7383be5aac9de9afc5fe17160232bff5e132], then ran and got this error: /home /theo/pysub[3]: /sge62/default/common/settings.sh: not found/ Looks like an issue with the bash code itself. Thoughts? /sge62 is an old folder. It is empty now (why wasn't it removed completely?) Replace . /sge62/default/common/settings.sh with . /sge/settings.sh Your previous way should have worked, too. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Qcronsub and qstat trickery
I have been trying several variations of: qcronsub -l h_rt=00:00:20 -l virtual_free=10M -l arch=lx ./helloworld.py and it is always working. Is the original qcronsub method still failing for you? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Wikidata tables
On 19/04/13 01:19, DaB. wrote: as you may know there is a rev_text_id-field in the revision-table. This field points to the text-table where the actual text is – or should be. Because the WMF doesn’t store the text here, but only a pointer (DB://cluster25/11458305 for example). If you query different wikis you will see that most of them point to the same cluster or one with a number short by. That says me (and I was also told so before) that all text of all wmf-projects are stored together. The task would now to separate wikidata from the rest – but the storage-area has no clue from where a text is which makes the separating very hard. And there is another problem: Deleted texts are also in this area, so even more filtering would be needed. I very doubt that this situation will change at the TS and I also doubt that it will be different for WikiLabs. So I guess your best bet is the API here. Sincerely, DaB. I think the only hope would be if wikidata was stored under its own cluster (for easier differenciation) and at least one server of that group (the master?) only had that (so toolserver could get its binlogs). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Wikidata tables
Sorry, but your mail doesn't make much sense. El 19/04/13 11:14, Patricia Pintilie escribió: Ok so how about we recocnize what the overal goal is first . Then establish the point that its trying to convay. Only then can we meet in the middle and set a plan in motion. I can only assist when a plan of action is clear with a definite plan without it im lost on where to begin. It seems as if im doing my natural instinct research then I get mail from the people im reading about... Very interesting this is because im left to think my mind is linked to the problems at hand. TS is my old signature, my server os will pull up my IP searches. Which leads me to believe this is why I am always being brought up in the middle of these outstanding conversations you guys are having lol. Please send detailed instructions as to how I can help,there should be a file known as Mila.eu also known as ro.eula. Find it and run whatever it has, thanks. -patiently waiting your responce. -MilaStarX-TS TS was being used in this thread as an abbreviature of ToolServer. This mailing list is about the Wikimedia Toolserver, so no wonder that it's mentioned a lot here. :) I don't know if you have a toolserver account, or even if you're a wikimedian. I don't know what you refer to with “my server os will pull up my IP searches” nor where is that “file known as Mila.eu also known as ro.eula” supposed to be. If you were asking for someone to make a query for you in the toolserver, try including the request in the email, or at least a link to what you want. Best regards ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Output files
On 18/04/13 02:09, Avocato wrote:
Hello all. I use this way
https://wiki.toolserver.org/view/Submit.toolserver.org#resource_definition@script
for running crons. I make a file named /something.sh /for example,
encluding the following:
/#!/bin/sh
#$ -j y
#$ -o /dev/null
cd pywikipedia
python anyscript.py/
Then, I put a cron like:
/00 21 * * * cronsub -s something % sh $HOME/something.sh/
The problem is that I want my crons to stop producing output files at
home folder inside my account, I don't want it to produce outputs at
all. How can I do that?
Remove the -s flag of cronsub. It is giving qsub the parameters -j y -o
$HOME/${JOBNAME}.out and it seems to be overriding the script ones.
So you would put just
/00 21 * * * cronsub something % sh $HOME/something.sh/
By the way, cronsub is deprecated in favor of qcronsub
Cheers
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] srwiki_p database incosistency
On 12/04/13 10:12, Marlen Caemmerer wrote: Hello, On Tue, 9 Apr 2013, Горан Обрадовић wrote: Date: Tue, 9 Apr 2013 23:55:28 Around 2000 (~0.01%) revisions from replicated srwiki_p database seem to be missing on Toolserver. For example, revision 5242120 exists in database[1], but doesn't exist on Toolserver. Does anybody know the reason for this anomaly? This reminds me of a big bug I was not able to track down. It seems we have database inconsistencies quite soon after we setup fresh data from WMF db servers. What kind of inconsistencies? Is it always on the same fields? This problem occurs even if we only use the replication integrated into MySQL. I wrote to the database gurus at WMF about that some weeks ago but I got no mail back and I dont know how to resolve this issue. Barring a mysql bug (hopefully unlikely), and given that WMF slaves are -supposedly- unaffected I can only think in a configuration difference which is affecting in the interpretation of the replicated statements. Is toolserver ignoring some tables? I remember a mysql bug where actions on ignored tables corrupted replication (but I thought it was fixed!). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] login.toolserver.orgIn-Reply
I suspect the above means I can't login into the toolserver. I have problems, too (it is asking for a password). Some scripts are also randomly failing. Perhaps LDAP is down? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Short downtime for s2 today
On 15/02/13 21:38, DaB. wrote: I admit that there is still room for optimization (z-dat-s2-b is our first Debian-db-server and I started with a clean config), but if you look at the monthly or yearly graph you can see that there were also these peaks before. I will look what I can do, but there are several things I have to care for in parallel at the moment. An option would be to clone the configuration used for wmf servers. It must be puppetized, and it should apply quite well into debian. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] New Rule: SGE-constraint for bots
On 11/01/13 23:01, DaB. wrote: c.) if you start a bot by hand for testing (no screen, no cron, no while). Does this forrbid to run a bot by hand with an attached screen? Ie. you have run screen because you are afraid your local computer/connection could reset, not to ‘run forget’. You would be attached eg. 90% of time, and pay (some) attention at the screen output... ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] s2/s5 will be read-only at Saturday
On 09/02/13 18:46, DaB. wrote: A more unpleasant news: I received a SMS by Nosy telling me that she is in the hospital for at least 1 week. Let us all hope that she and her child will be better soon! :( Do you know who was who needed hospitalization? I hope it was not her baby. I wish all your family gets healthy soon, Nosy. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] New Rule: SGE-constraint for bots
On 11/01/13 23:43, DaB. wrote: Hello, At Friday 11 January 2013 23:42:03 DaB. wrote: Is TS-1479[1] a valid exception B for not using SGE for some specific scripts? yes. But I will speak with Merlissimo about a fix for this (AFAIR he had a problem with the given patch). It looks good to me (note it is *not* using embedded quotes as suggested in comment #1). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cron on willow
On 11/01/13 13:45, Carl (CBM) wrote: Has anyone else noticed this? I am having problems with cron on willow, which does not seem to be executing my crontab. I tried to test it by adding a line to my crontab that should execute every minute: * * * * * /home/cbm/touch.sh That script runs correctly from the command line, but cron does not seem to execute it. Some of my other other cronjobs appear not to have run in a couple days, even though they were working fine for months before. - Carl Are you using Solaris crontab or cronie? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cron on willow
On 11/01/13 15:01, Carl (CBM) wrote: On Fri, Jan 11, 2013 at 8:33 AM, Platonides platoni...@gmail.com wrote: /usr/sbin/cron does seem to be running :S It is, but something is not right. The line * * * * * /home/cbm/touch.sh at the beginning of my crontab is not being executed. The script runs fine from the command line on the same host (and a script error should give an email from cron rather than silence, anyway). I know, I know. Wolfgang ten Weges may be right in that it's overloaded. fyi, cronie is working fine. That's not particularly relevant for people who have set crontabs with /usr/bin/crontab - Carl It was given as an aside, not as a solution although I guess you could migrate: crontab -l | cronie ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Crons overflow/server overload on Willow (other)
For the record, sun crontab seems to be working again since 15:50 UTC. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Can't explain queries?
On 04/01/13 21:05, OQ wrote: On Fri, Jan 4, 2013 at 1:55 PM, Aaron Halfaker aaron.halfa...@gmail.com wrote: Hello, I've been working on a new script that will access the MySQL database. In testing this script, I was worried about the complexity of some of my queries, so I naturally went to the MySQL client and ran them with explain. It appears that I do not have permission to do so. Is this intended? Is something wrong? EXPLAIN can leak information from underlying tables, so if you dont have permissions for those tables the explain will fail with the error message provided. It was possible until the last mysql update. And yes, being able to EXPLAIN queries is *very* useful. Otherwise the already-a-guessing-game of making efficient queries is even harder. I don't think an EXPLAIN can leak information not already available to us by querying the view (possibly multiple times). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Merry Christmas
+20 Merry Christmas to 255.255.255.255 :) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] JIRA session loss
On 14/12/12 16:42, DaB. wrote: sorry, no news yet. If the situation doesn't change until the new year we will switch to another software like bugzilla (can anybody recommend one?) Sincerely, DaB. What features should it have? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Result of the general member meeting of WMDE
On 04/12/12 09:07, Chris Grant wrote: On Tue, Dec 4, 2012 at 5:05 AM, Magnus Manske wrote: Doubleplusgood for DaB staying in the fight for another round!!! Seconded! Following that analogy, I hope he knocks a few teeth out! (I'm joking, I'm joking - the last thing we want is an aggressive relationship between the toolserver and everyone else) -- Chris So, you are proposing a relationship in the ministry of Love? ;) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Result of the general member meeting of WMDE
On 26/11/12 19:33, Dr. Trigon wrote: Another question; I just learned about Wikimedia Polska toolserver [1], how are we related to them? Are we at all? Is it the same as the german one but paid from poland wikimedia? (May be you can get admin there, if you like?) AFAIK it's a more simple server. Like the German Toolserver at the beginning. Noticeably, with no db access, which is what makes TS special. (Why would he want being an admin there? Not enough problems here? :D Do they actually *need* an admin?) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] SGE queue waiting forever?
On 24/11/12 17:37, Wolfgang Faust wrote: Logging in to submit.toolserver.org http://submit.toolserver.org takes a really long time recently (starting a few days ago). Clematis doesn't seem to have any load though, so I don't know what's going on. Thousands of processes running df -k /mnt/user-store? :) $ ssh submit ps -ef | grep -c 'df -k /mnt/user-store' 15408 Seems we have /mnt/user-store problems again. clematis ~ $ time ls /mnt/user-store NFS server thyme not responding still trying NFS getattr failed for server thyme: error 16 (RPC: Failed (unspecified error)) ls: cannot access /mnt/user-store: Connection timed out But those instances are stuck. There are processes since Nov 12. Seems that nosy just killed them. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] SGE queue waiting forever?
On 24/11/12 21:38, Dr. Trigon wrote: @All: If you are working on big files please copy them to local temp first (on sge $TMP contains an individual temp dir for the job). E.g. piping big files to other slow programs causes much nfs load because data must be read in small packages which cause high load on servers. That's why sge cannot schedule new jobs on nightshade since days. What is a big file? Is it ok if the file is in user-home? Thanks and greetings DrTrigon /home is also mounted with nfs Although it's strange that reading from big files overloads the servers. stdio or the equivalent functionality in the language they are made should be making it work in blocks. Looking at willow mounts, /shared and /home are mounted with nfsv3 over udp. But /mnt/user-store and /install don't show it, so they are probably using nfsv4 over tcp. Is that intended? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Defective database s7
On 16/11/12 21:44, Marlen Caemmerer wrote: Now I tried several thing to check which table might be corrupted. Innodbchecksum reported everything fine. Mysqlcheck crashed the mysql daemon when accessing centralauth.localnames. Oh? Why? Checking the table again crashes mysql. Hm. Tried a repair table - storage engine does not support this...hm. The log says InnoDB: Page lsn 268 3672100478, low 4 bytes of lsn at page end 3672100478 InnoDB: Page number (if stored to page already) 192520, InnoDB: space id (if created with = MySQL-4.1.1 and stored already) 428 InnoDB: Page may be an index page where index id is 0 1174 InnoDB: (index PRIMARY of table centralauth.localnames) InnoDB: Error in page 192520 of index PRIMARY of table centralauth.localnames 121116 13:02:46 - mysqld got signal 11 ; I tried to remove the index to rebuild it but this does not work due to innodb_force_recovery = 3. You may need to set innodb_force_recovery back to 0 before being able to drop the index (we got lucky there if it's just the index). http://dev.mysql.com/doc/refman/5.1/en/forcing-innodb-recovery.html: The database must not otherwise be used with any nonzero value of innodb_force_recovery. As a safety measure, InnoDB prevents users from performing INSERT, UPDATE, or DELETE operations when innodb_force_recovery is greater than 0. (I would have expected DROP INDEX to work, as DROP or CREATE tables are allowed in innodb_force_recovery, but seems it's not) Mysqldump fails - crashes the mysql daemon too. I expect it only crashes when dumping that table :) You may be able to get the values out of the table by forcing usage of the other index (ln_name, ln_wiki). Although it could be safer to get a new dump for that able from wmf, or a mixture of them (such as getting a dump at both sides, then rsyncing). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] thyme – postmortem
On 13/11/12 01:46, DaB. wrote: Thyme also carried my wikidata-replication-program which failed too (so the replag of wikidata everywhere increased). I moved it to another server now. A strange thing is that the mysql-process on thyme is still running; even replication is working so the replag will not increase there. What about puppetd ? If it's still running, it could provide a way to restart the server. It looks as if sshd had simply died. I would had blamed the OOM killer but thyme is Solaris, and it affected both sshd and the nfsd at the same time... (but not mysqld). Maybe an error with the filesystem umounting itself, but in that case sshd should still work. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] SQL-S1 maintenance
On 11/11/12 17:37, DaB. wrote: I will switch sql-s1-user to read-only at tomorrow, Monday after 20:05 UTC and than dump the user-databases (and import them on thyme). I do not know how long this will take, but I guess the hole European night. When the import is done somewhen on Tuesday I will switch sql-s1-user to thyme and switch it to read/write. Later (or in parallel to the user-database-dumping, not sure yet) Nosy or I will dump commons from rosemary too for a later re-import. When everything is dumped and imported I will re-setup rosemary with the fresh dump too. Sincerely, DaB. Remember that there is a trick where you dump the user-databases with the binlog enabled (and noting its position), leaving the database writable. You then transfer and import the dump. Finally you make it readonly, copy the changes from the binlog (which will be much smaller and thus results in a shorter read-only interval) and point the dns to the other server. Regards ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] SQL-S1 maintenance
On 12/11/12 00:04, DaB. wrote: Hello, At Sunday 11 November 2012 23:54:39 DaB. wrote: Remember that there is a trick where you dump the user-databases with the binlog enabled (and noting its position), leaving the database writable. You then transfer and import the dump. Finally you make it readonly, copy the changes from the binlog (which will be much smaller and thus results in a shorter read-only interval) and point the dns to the other server. AFAIK this would need a complete LOCK of all tables during the dumping (to get a consistent state) which would be the same as switching the server read-only. Also somebody would have to write a program that read the bin-log and filter out all non-user-database-entries (enwp, commons and heartbeat in this case). The 3rd disadvantage would be that you would have to wait for the dump to complete before you can start the import, The only advantage I see it that sql-s1-user would be writable between the end of the dump and the time I awake in the morning. Sincerely, DaB. You are completely right. Still, I suspect it *is* possible to do so efficiently. Starting a global read transaction and then resuming writes (I am assuming that all will be InnoDB, which is very likely). Maybe it needs a special tool. I am copying Asher in case he can enlighten us. There are tools for the binlog filtering, but it is not a problem in this case, since we could just stop replication before the export and let the lag rise during the backup. In the general case we would reenable them and let the server pick up after it finishes, but in this case we are just going to overwrite it :) How much data is in user tables and how long does it take to import the (uncompressed, ready) wiki tablespaces? It might turn out to be faster to switch the tablespaces than moving the dbs around. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] mdbtools
On 10/11/12 16:58, Andre Koopal wrote: Hi, Today I am working on a project where I need to convert an access database dump to something else (mysql likely). Most things I found on the web are not really suitable to script this process (it will need to run every month), but mdbtools looks to be promising. However I tried to compile that on toolserver all I got out of it are coredumps :-(. (Tried both on willow and nightshade). The ubuntu pacakge does work on my home server. Did anybody ever look into mdbtools and have it working? Is it perhaps possible to have it globally installed on toolserver, as it probably is usefull to some of the glam-projects as well? I was able to compile mdbtools-0.6pre1 for linux, see the result at /mnt/user-store/mdbtools I admit that compilation was a bit tricky. When it complains that backend.c:31: error: static declaration of 'mdb_backends' follows non-static declaration, remove the static keyword from that line. yacc and flex are not installed in nightshade, I worked around that by running those of willow. The flex rule expected an output file of .c, so I ran ln -s lex.yy.c .c (otherwise you get an empty lexer.c) However, the generated binaries didn't output any table for your file. I then realised that my local mdbtools does work with yout file. Seems that mdbtools-0.7 is in GitHub, but not in SourceForge: https://github.com/brianb/mdbtools However, in order for just running autoconf.sh, it needs libtool, which is not installed in Linux or Solaris. If people want an mdb file to test with, the file I need to convert is at /mnt/user-store/rce-nl-data. Regards, Andre ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] rosemary's copy of enwiki_p also corrupt?
On 04/11/12 00:56, DaB. wrote: Shortly after Asher Feldman provided a dump in the new xtrabackup-binary- format the wmf uses for dumps. It took ~2 days to transmit this dump to the toolserver. Was the rate-limiting issue fixed? It is the first time the toolserver has to handle this new format and it took some days to find the needed tools (the wmf uses Ubuntu for their database- servers while we uses Solaris). This may be a good time to migrate one of rosemary/thyme to Debian. Asher told me that the dump is compressed in some way, but I could not decompress it. While I tried to figure out how to decompress the file, I accidentally overwritten the file at Thursday. Failures happen. Looking aroound, it isn't obvious the way they get compressed. Given you could not decompress it initially, it probably used xbstream (the other common compression format with xtrabackup seems to be using a gzipped tar). I would have expected Asher to provide you a full page documenting how to restore the backups, though. How to restore a backup is a critical piece which can't be left undocumented (or untested). Thanks for the update, DaB. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] rosemary's copy of enwiki_p also corrupt?
On 04/11/12 02:29, Erik Moeller wrote: My most recent inquiry to you and Nosy was unrelated (I asked for the configs so we can compare them as we begin setting up replication in Labs.) Erik It's the first notice I get that labs replication is being set up. Who is handling this? Asher? I would expect Ryan to be involved, but he is currently on vacation... ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Maintenance of a Switch tomorrow
On 25/10/12 14:59, Marlen Caemmerer wrote: Hello, tomorrow a switch in one rack will be replaced. This means some downtime of about 30 min probably tat will occur between 9 am and 13 am utc. Sorry for not having a better time frame but the exact arrival of the onsite technician is not know to me. Cheers nosy Please send out a notice when it's done. It is planned to open the WLM survey (and thus start advertising it) tomorrow at 15:00 UTC. It would be sad that the technician came very late and the maintenance took place just when lots of users are getting notified to go to the TS to fill out the survey. :) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Database connection problem
On Mon, Oct 22, 2012 at 1:40 PM, ant antof...@gmail.com wrote: Hi, I'm new to the toolserver and I'm trying to install a web application[1]. I created a database on sql-s1-user to which I can connect via my PHP code. Also I have a C program that needs to connect as well. If I call the C program from the command line, it works perfectly. However, if it is called from the PHP script using shell_exec() by the web server user, I get the following error message: The program is compiled for solaris? Access denied for user 'ant'@'damiana-bge0.esi.toolserver.org' (using password: NO) This is clearly wrong, as you have to provide a password in order to connect. Maybe your program is relying in $HOME being set? I think you need to get it from getpwuid() ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Mail forwarding not working
On 17/10/12 22:53, Tim Landscheidt wrote: Hi, for those of you not having seen TS-1553, mail forwarding seems to have stopped working. So if you haven't received the usual job reports that you were expecting, you might want to login to all servers and check if there is mail for you. You can query all servers by: | for SERVER in clematis hawthorn nightshade ortelius willow wolfsbane yarrow; do | ssh $USER@$SERVER.toolserver.org ls -l /var/mail/$USER | done replacing $USER with your username. Tim Actually, no need to replace it. $USER is set automatically. You can run: $ for SERVER in clematis hawthorn nightshade ortelius willow wolfsbane yarrow; do ssh $USER@$SERVER.toolserver.org test -s /var/mail/$USER echo $SERVER; done and it will list for you the servers where you have mail. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Mail forwarding not working
On 17/10/12 23:23, Tim Landscheidt wrote: Actually, no need to replace it. $USER is set automatically. You can run: $ for SERVER in clematis hawthorn nightshade ortelius willow wolfsbane yarrow; do ssh $USER@$SERVER.toolserver.org test -s /var/mail/$USER echo $SERVER; done and it will list for you the servers where you have mail. Actually, that's only true for those whose username on the toolserver is the same as on their private box :-). I'd have to name my username on the toolserver either explicitly or rely on my ~/.ssh/config: | Host *.toolserver.org | User timl and always escape the second $USER so that it is expanded on the toolserver. Tim I had run the above in the login server. Hadn't noticed you were doing so from local. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Query killer and replag
On 14/10/12 13:53, John wrote: Either tsbot or the Query killer are broken I just got an email about a killed query (Not the issue) and query killer said the lag was around 14000 seconds while tsbot and the relag graph are showing almost no lag. The enwiki db has pretty much no lag in the user server, but a lag of 5h in the round-robin one (yes, that's quite strange). BTW, may I ask why are you using 17520 as limit? Seems an odd choice. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] How to have qsub mail output?
On 01/10/12 13:21, Tim Landscheidt wrote: (anonymous) wrote: Please see the documentation on Toolserver wiki [1] to set this up. It uses a command of -m with another variable depending on when you want mail sent. [1] https://wiki.toolserver.org/view/Batch_job_scheduling#arguments_to_qsub/qcronsub [...] That just sends notifications about the job's status, not the job's output itself. Tim No, there's no such option. I looked into it recently. We could get that by adding a final script which mails the user the output file if qstat -j $JOB_ID | grep -q mail_options:.*e. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] How to have qsub mail output?
On 01/10/12 14:19, Tim Landscheidt wrote: Thanks. I don't want to fiddle to much with SGE's in- testines, so I will probably either use | mail timl in my script or have my MUA insert the log in the status mail. I looked if I could submit this totally fascinating and innovative idea of mailing the output as a RFE upstream, but amazingly I didn't see a bugtracker at Oracle :-). I would even have had another idea: Impromptu jobs à la echo true | at now :-). Tim Oracle closed-sourced it. There are a number of forks. Quick link: http://gridengine.org/blog/2011/11/23/what-now/ ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Future of the toolserver
On 30/09/12 03:31, Krinkle wrote: On Sep 29, 2012, at 9:41 PM, Samuel Klein meta...@gmail.com wrote: And why is the WMF considering not providing db replication for it? [citation needed] I think you misunderstood. -- Krinkle Written by Erik Moeller the 25th Sep: Chapters are autonomous organizations, and it's WM-DE's call how much / whether it wants to continue to invest in infrastructure of any kind (and the decision of funding bodies like the FDC to accept or reject that proposition). However, for our part, we will not continue to support the current arrangement (DB replication, hosting in our data-center, etc.) indefinitely. He is writing with his WMF hat, the above 'we' refers to the WMF. So no, there's no misunderstanding: The WMF has threaten to stop providing the db replication. That would be a really bad move to do, of course (and a gratuitous one, since there aren't technical or financial reasons for doing that). And I hope they won't do that. - http://lists.wikimedia.org/pipermail/toolserver-l/2012-September/005294.html ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Reasons for not migrating to Tool Lab
On 27/09/12 17:21, Andrei Cipu wrote: Additionally, we have a number of volunteer driven projects. Here's a few choice ones: * Bots * Deployment-prep * Maps (for OpenStreetMaps) * Wikistats * Wikitrust * Signwriting * Phabricator * Metavidwiki * Huggle * Glam * Wiki loves monuments * Blamemaps * Counter vandalism network Where can we find more information about these projects, especially OSM and WLM? You can go to https://labsconsole.wikimedia.org/ List projects, take a look at project members and bug them to tell you their evil plans :) The project for WLM is actually one for making a tool for judging the images (Wlmjudging). There are a couple of VMs, I don't know if it produced something. I'd ask Ynhockey about it. All real work for Wiki Loves Monuments is done in the toolserver (plus http://wlm.wikimedia.org/ which has a copy of the data produced at TS). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Reasons for not migrating to Tool Lab
On 27/09/12 01:07, Ryan Lane wrote: We currently have no plans for having the user databases on the same servers as the replicated databases. Direct joins will not be possible, so tools will need to be modified. -50 It's such a useful feature, that it would be worth making a local mysql slaves for having them. I know, the all-powerful labs environment is unable to run a mysql instance, but we could use MySQL cluster, trading memory (available) to get joins (denied). I'm not the one setting up the databases. If you want information about why this won't be available, talk to Asher (binasher in #wikimedia-operations on Freenode). Maybe he can be convinced otherwise. Of course, in the production cluster we don't do joins this way. We handle the joins in the app logic, which is a more appropriate way of doing this. I disagree. In production you can just create a new table in the wiki db. We can't create new tables there in the toolserver (the dbs are a mirror or what there is in production). Thus, we create a new db in the same server and use a cross-db join instead of joining a new table. Joining several wiki tables is probably more strange, with the exception of commons, which is more often joined to others, as the commons images are also at the local wikis. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Future of the toolserver
On 26/09/12 08:27, Federico Leva (Nemo) wrote: Yes, I know about MMP. It was introduced much later, after people were already very used to doing things in an individualized manner. From what I'm told MMPs aren't used much. So you think that even new users (I believe many users arrived after MMP were introduced) are not using MMP due to bad old habits of the other users? I've had a hard time trying to convince some users to use a MMP; I think it's not trivial to understand what models actually work and are liked. The reason is as simple as being much more easier, run mkdir and start coding vs open a jira request to get a new MMP for a project which may or may not be interesting (heh, most tools were probably born after coding for a couple of hours after having a good idea) and which nobody is wanting to maintain with you, probably. Ryan wrote: Toolserver's model is fundamentally different. It's based on an old concept of shared hosting. Labs is built on a model more like a VPS (really more like EC2). Due to that, it's possible to give users far more rights. labs model didn't exist when the toolserver started. This route was the only 'normal' one to follow, specially with a single server. It even looks too new for labs right now. If you guys want to build the exact same Toolserver environment as a Labs project, go for it. I have a good feeling you'll start doing things differently when you see the affordances given by having more rights, though. I have a few ideas on how to improve it on webtools, based on toolserver experience, but without big changes. MMPs aren't a very elegant solution. I'd prefer not to force an inelegant solution into a system that allows much more elegant approaches. Actually, I'm unsure on how to replicate MMP accounts there. It shouldn't need to manually create them everywhere. The elegant solution is probably to have those accounts in LDAP... Any ideas? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Future of the toolserver
On 26/09/12 12:40, Sumurai8 (DD) wrote: 2012/9/26 Platonides: On 26/09/12 08:27, Federico Leva (Nemo) wrote: Yes, I know about MMP. It was introduced much later, after people were already very used to doing things in an individualized manner. From what I'm told MMPs aren't used much. So you think that even new users (I believe many users arrived after MMP were introduced) are not using MMP due to bad old habits of the other users? I've had a hard time trying to convince some users to use a MMP; I think it's not trivial to understand what models actually work and are liked. The reason is as simple as being much more easier, run mkdir and start coding vs open a jira request to get a new MMP for a project which may or may not be interesting (heh, most tools were probably born after coding for a couple of hours after having a good idea) and which nobody is wanting to maintain with you, probably. It is not hard to start coding in your own workspace and move code from your newly created directory to a MMP. A MMP doesn't necessarely have to be maintained by multiple users. While it's convenient having someone in the project that can jump in in case your tool breaks badly and you are not around to fix it, it is not by any means necessary. Right. I was explaining why there aren't so many MMPs out there. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Future of the toolserver
On 25/09/12 00:51, DaB. wrote: Hello all, in these days WMDE (the chapter that finance the toolserver) is discussing the budget for the next year (2013); you can find it at [1]. At the moment there is no money for new toolserver-hardware in this budget and the CEO Pavel Richter is unwilling to change this ([2] in german) – because he fears that there will be a Wikilabs in 2014. It is not possible for me to run the toolserver for another year with the current hardware – you all know why. For this reason I will request a change of the budget at the general meeting at November, so there will be a vote about. If this vote should fail (and we get no money for new hardware), I am going to retire from my job as root at 30. December 2012. I'm not longer able to tolerant the behavior of the german chapter and the WMF in matter of the toolserver; I do this for free and for fun, and it is not longer fun. Sincerely, DaB. So, out of fear that there may be a better alternative in two years time, he decides to stop supporting it now? I know labs, and I'm not sure it will really be a better alternative. Currently, it isn't. The toolserver is much more reliable and flexible. I don't think labs will win in the near future, either. Although it might change in two years. Specially if no attention is payed to the toolserver for that time. I see two big problems: 1) If labs really becomes the perfect tool hosting in 2014, What happens before we reach that? Yes, your tool doesn't work, migrate to labs next year 2) We risk ending up with no good alternative at that time. labs is still not good enough, toolserver has degraded so much it's unusable. Not to mention the migration cost that would need to be payed by tool authors if forced to move. Although perhaps he doesn't care. P.S: If you are in a board of a chapter that gives money to WMDE for the toolserver: Make sure that it will be spend for hardware. [1] http://meta.wikimedia.org/wiki/Wikimedia_Deutschland/2013_annual_plan_draft/en [2] meta.wikimedia.org/wiki/Talk:Wikimedia_Deutschland/2013_annual_plan_draft/de#Toolserver ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Future of the toolserver
On 25/09/12 20:48, Erik Moeller wrote: 1) WMF is a technology organization. Hosting the core infrastructure for Wikimedia projects is very much what we do. This includes data center operation, monitoring and backups, software deployments, software/service upgrades, code versioning infrastructure, bug tracking infrastructure, additional support systems and services (like this mailing list), etc. Toolserver is in fact hosted by the Wikimedia Foundation today, in our Amsterdam data-center. We provide space, power and racks for the toolserver cluster, at a cost of about $65,000/year to WMF according to our Director of TechOps. Something we should all be grateful for. We also maintain the database replication on our end which enables tools to function. A replication which WMF is already doing for itself, adding an extra slave hosted by a trusted party doesn't make a techical difference. While WMF gains a backup server (at some point the toolserver was the only externally hosted backup). It has also inadvertedly dropped the binlogs needed by TS when doing server cleanup. So while very welcome, the WMF side of the db replication is not the most important piece of the toolserver (having a db copy still makes the TS better than any other alternative available at the time, though). The value is probably in how WMDE was able to open that database and many fruitful tools emerged. We can't provide the same level of service for the toolserver infrastructure as we do for core operations, and it makes no sense for a chapter to build out the required staffing and expertise to do so (set up/maintain all or some of the aforementioned functions). Even with slightly increased investment, toolserver would always suffer from being second or third tier infrastructure. labs is also a second class citizen. Moreover, it is explicitely stated not to be for production-like level. What will happen if a really successful tool reaches to a point where it de-facto needs it ? (eg. a WLM tool, tools to Request to get an Account Created, or Request an Unblock appeal...) 2) We're not comfortable hosting the toolserver infrastructure as-is. There are too many idiosyncratic aspects of its configuration; it has its own wiki, Since when is that a problem? To take an obvious example, labs also began by making its own wiki, instead of incorporating itself into an existing one. its own (closed source) version control system, The vcs isn't closed source, it's just plain subversion. The *repository viewer* is closed source, although with a an Open Source license. I don't know if people really use it too much. its own (closed source) issue tracker. I'm not a fan of jira, but this is the silliest reason to drop the toolserver :) There are hacks like TUSC that we want to replace with better systems/services (e.g. OpenID/OAuth). OpenID nor OAuth are currently available. If they were, TUSC wouldn't have been developed in the first time. When they appear, they will -slowly at first- take over TUSC, as it should. This ball is at WMF side. Not to mention, if these «idiosyncratic aspects» were really a problem, they could easily be changed. Chapters are autonomous organizations, and it's WM-DE's call how much / whether it wants to continue to invest in infrastructure of any kind (and the decision of funding bodies like the FDC to accept or reject that proposition). However, for our part, we will not continue to support the current arrangement (DB replication, hosting in our data-center, etc.) indefinitely. This sounds as forcing them to go this route. The timeline we've discussed with Wikimedia Germany is roughly as follows: - Wind down new account creation on toolserver by Q2 of 2013 calendar year - Decommission toolserver by December 2013 WMF can't commit to providing technical support for tool transition (there are too many tools), so if there's any area where I think it would make sense to ramp up investments on WM-DE's part, it's in engineering capacity to support tool developers in porting tools to Labs. So the tool authors are on their own, while forced to move out. What is the WMF *offering*? That said, there may be a need for emergency purchases/investments to keep TS in a usable state until December 2013 (and perhaps allow for some buffer room beyond that). That's not our call to make. But you are refraining WMDE from investing to it now. Would for instance WMF be willing to lend a few servers to the toolserver until December 2013? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Future of the toolserver
On 25/09/12 23:38, Federico Leva (Nemo) wrote: Indeed the first answers on https://www.mediawiki.org/wiki/Wikimedia_Labs/Toolserver_features_wanted_in_Tool_Labs seem to indicate that all tools and services currently on Toolserver will just be trashed and people forced to reinvent them from scratch because a completely different approach is thought to be better. Nemo Both approaches have its merits. The WMF could have copied the toolserver recipe much more easily than gooing the route of labs. They were brave pursuing the VM based system, and it allows solving different problems. But both of them have their use. It's not a case where one should win the other, or the toolserver is bad because it is currently hosted by WMDE instead of WMF. Organizations trying to defeat one another will end up with one big loser: the community. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Web servers unresponsive
On 24/09/12 15:03, emijrp wrote: Can you kill them? I always connect to login.toolserver.org http://login.toolserver.org. How do I change to rosemary? rosemary is a sql server. You'd do something like this: $ mysql -h rosemary mysql show full processlist; +---++-+--+-+--+---+---+ | Id| User | Host| db | Command | Time | State | Info | +---++-+--+-+--+---+---+ | 142886093 | platonides | willow.toolserver.org:62886 | NULL | Query | 0 | NULL | show full processlist | +---++-+--+-+--+---+---+ 1 row in set (0.00 sec) mysql kill 142886093; Of course, you would have some problems doing it when you can't even connect to the server :) I had the same problem on Saturday with s2-rr, even after killing all php.fcgi instances on the webservers (the ones from which those 15 connections could have originated), the server wasn't allowing me to connect. While we're blaming the sql servers, I had a script failing this night with «Lost connection to MySQL server at 'reading initial communication packet', system error: 0 (sql-s3-rr.toolserver.org)» but it is working to now. Regards ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] qcronsub warning: Please add maximum runtime
On 24/09/12 18:07, Krinkle wrote: Can someone decode this? What is this? -- Krinkle Begin forwarded message: *From: *r...@toolserver.org mailto:r...@toolserver.org (Cron Daemon) *Subject: **Cron krinkle@hawthorn qcronsub -N dbbot_wm -m n -j y -b y -l h_rt=INFINITY -l virtual_free=90M $HOME/bots/dbbot-wm-start.sh* *Date: *September 24, 2012 6:05:07 PM GMT+02:00 *To: *krin...@toolserver.org mailto:krin...@toolserver.org warning: Please add maximum runtime by adding parameter [33m-l arch=[0msol|lx The text asks you to place a time limit. The parameter (embedded in posix colors despite not being output to a terminal) to specify if it needs a linux or solaris server. However, if I try to execute it, I get a much saner message: $ qcronsub -N dbbot_wm -m n -j y -b y -l h_rt=INFINITY -l virtual_free=90M /home/krinkle/bots/dbbot-wm-start.sh Unable to run job: Script not executable: /home/krinkle/bots/dbbot-wm-start.sh. Exiting. warning: Please add the os this job can run on by adding parameter -l arch='*'|sol|lx For more information read documentation at https://wiki.toolserver.org/view/Job_scheduling As this is a php script, your parameter would be «-l arch='*'» ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Web servers unresponsive
On 23/09/12 21:07, DaB. wrote: Hello, All error out on with no response and a time out. I can still SSH into wolfsbane and ortelius from willow, though. I will now investigate this. Until now the only problem I found is that hemlock is down. I restored the web-access now. As far as I see hemlock lost its external array and became out of memory around 2:30 UTC. I have no idea why this influence our webserver. I rebooted hemlock to free the memory and restarted the webserver on ortelius and wolfsbane; the webpages are back AFAIS. Maybe they got blocked trying to access something at /mnt/user-store ? Following a symlink, perhaps. The last successful answer by both webservers was at 23/Sep/2012:02:35 before the restart. This NFS failure was quite bad since there was no timeout when trying to access /mnt/user-store, just being blocked forever. I guess TS-1519 and TS-1520 may be closed now. Thanks ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] z-dat-s4-a (s4-user) is down (was: Re: Reboot
On 20/09/12 13:12, DaB. wrote: Hello all, At Thursday 20 September 2012 13:10:18 DaB. wrote: I'm noticing that some data has gone missing from s4 database(s). It appears that this goes back at least a few days; do we know how far, and if any of this data is recoverable? (Of less immediate concern is why this happened...) I told Nosy to use our backup from the 18 (that's the day the mistake happened and so only few data should be lost). But only she knows that she has done. @Nosy: Can you please respond to Hersfold? Sincerely, DaB. According to my logs, the backup was done shortly after 2012-09-17T03:36 ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When to execute cron-tasks
On 14/09/12 01:42, Dr. Trigon wrote: What about a tool that gather some statistics from all users cron(ie)tab files and show them on a public (or for logged-in users only) page/place. The idea is that user can get an clue on what times have high or low job loads. Something like: 00 - used xx times - xx% of all jobs 01 - used xx times - xx% of all jobs 02 - ... 03 Might also include hours and more info, BUT NO per user data just overall averages and counts. Greetings DrTrigon Ideally, you could mark a task as being daily-I-don't-care-when, or perhaps run each 20-28h, and cron would choose the time that best suited itself, taking all registered jobs into acocunt. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] sql.toolserver.org will become deprecated
What about adding user entries to the switchserver table? Currently, if you want to access an arbitrary wiki table, you need to look it up first in toolserver.wiki. Seems appropiate to extend that to user databases. Frameworks setting db connections would need little changes to also handle user dbs like they do with wmf views. (I'd actually put it into a new table eg. toolserver.databases, but changing the db name used in the framework at the same time -keeping wiki for other uses, of course- should be trivial) Regards ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Web services are down
On 02/09/12 17:37, Magnus Manske wrote: Me too. Occasionally, it's 502 bad gateway though. Just in the middle of testing stuff and thought it was my broken code. Frustration. wolfsbane has a near-full /tmp Use http://ortelius.toolserver.org/ until Dschwen or a root gets to it. Jira bug https://jira.toolserver.org/browse/TS-1504 Maybe we should use a LRU filesystem on the webservers /tmp ? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Why does Perl use so much more vmem on Linux than on Solaris?
To begin with, the perl used in Solaris is 32 bit and the Linux one 64 bit. They are also different versions (v5.12.3 vs v5.10.1). But the main difference is probably in the linux version mapping /usr/lib/locale/locale-archive, a 86M file, while in Solaris it seems to be using /usr/lib/locale/en_US.UTF-8/en_US.UTF-8.so.3, /usr/lib/locale/en_US.UTF-8/en_US.UTF-8.so.3 and /usr/lib/locale/en_US.UTF-8/libc.so.1 instead. From locale-gen(8): Locale data files can be stored either in a single archive file, /usr/lib/locale/locale-archive, or in a deep tree where individual files are stored under /usr/lib/locale/locale_name/LC_*. As all processes will be sharing the same locale-archive file, the two methods shouldn't matter too much for memory usage, although locale-archive will probably bite us when dumping core. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Web services are down
On 29/08/12 09:22, Marlen Caemmerer wrote: Hello, wolfsbane ran out of space which means one of the web servers was not able to deliver any more web pages which has the effect of sometimes the page works - sometimes not. Should be fixed now - if you still see errors please let us know. Cheers nosy Thanks nosy. Shouldn't the balancer have automatically considered wolfsbane down and direct all requests to ortelius? Cheers ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] sql-s1 will be read-only on Wednesday
On Thu, Aug 16, 2012 at 1:33 PM, DaB. w...@daniel.baur4.info wrote: Hello, At Thursday 16 August 2012 13:31:27 DaB. wrote: Is it possible that such db belongs to anyone else and got missasigned in some db move? You don't need to import that, DaB. I can not guarantee that there wasn't a mistake somewhen in the past, but I doubt it. But I would say we solve it in the practical way: Just drop what you don't need :-) (if somebody would miss it, he/she would have already tell us, wouldn't he/she?) Maybe I created them for some temporary thing, then forget it. Part of the reason of this mail was to see if someone came up saying those are my tables! Tables dropped from rosemary. Regards ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] sql-s1 will be read-only on Wednesday
On Wed, Aug 15, 2012 at 5:44 PM, DaB. w...@daniel.baur4.info wrote: 68616734 u_platonides.dump.gz I seem to have a db in thyme listing the hashes of enwiki images and their size, but I don't remember setting up that thing. Is it possible that such db belongs to anyone else and got missasigned in some db move? You don't need to import that, DaB. Regards ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] s1 replag update, suggestion, and question
On Wed, Aug 8, 2012 at 3:46 PM, Russell Blau russb...@imapmail.org wrote: There is a possible workaround. The TS could treat this like a server outage; copy user databases from thyme to rosemary and then point sql-s1-user to rosemary, which currently has no replag. Rosemary would then have to handle twice the load, but thyme should start to recover very quickly with no user-generated queries hitting it. Once thyme has recovered, point sql-s1-rr to it. Downsides: (1) this would require several hours of downtime for sql-s1-user while the user databases are copied; all tools that require access to user databases would be offline entirely for this period. (2) it would have to wait until our volunteer TS admins have time to do it. Actually, it could probably be reduced from downtime to readonly user databases. If thyme were writing at the binlog, it could probably stay accepting writes for the most part of it. This comes at the expense of TS admin time, of course. (3) the added load on rosemary could cause replag to grow there, although I doubt it would come anywhere near the 14+ days replag we are dealing with now on thyme. Depending on the insert speed without queries, another option would be the time needed for copying the db from rosemary to thyme. (I'm assuming it would be much slower than the downtime moving user dbs but it's just a guess, if it weren't this could replace that move). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Wpipe
I'm not convinced about its utility. What tools would need combining?
If I just need the results of a SQL query, it may be easier for me than
using this system. Maybe a better interface would help.
The use case I see more interesting is for taking a tool which outputs a
list of pages and provide for input of another tool. Some page/user to
work with seems to be the most common input.
Maybe we should just standarize the input parameters and let some tools
chain to another, simply using a special format parameter.
For instance I usually use names like: art, lang, project such as:
$_REQUEST += array('art'='', 'lang'='en', 'project'='wikipedia' );
Magnus Manske wrote:
Right now, a tool is started by this page via nohup ; that could
change to the job submission system, if that's possible from the web
servers, but right now it seems overly complicated (runtime
estimation? memory estimation? sql server access? whatnot)
The web page then returns the reserved output asset ID, while the
actual tool is running; another tool could thus be watching
asynchronously, by pulling the status every few seconds.
Yes, it can be called. I use it in a script for scheduling a cleanup of
the created temporary files.
The relevant code:
$dt = new DateTime( now, new DateTimeZone( UTC ) );
$tmpdir = dirname( __FILE__ ) . /tmp;
@mkdir( $tmpdir, 0711 );
$shell = mktemp -d --tmpdir= . escapeshellarg($tmpdir) .
catdown.;
$tmpdir2 = trim( `$shell` );
// Program the folder destruction
// Note that qsub is 'slow' to return, so we perform it in the background
$dt-add( new DateInterval( PT1H ) );
exec( SGE_ROOT=/sge/GE qsub -a . $dt-format(YmdHi.s) . -wd .
escapeshellarg( $tmpdir ) . -j y -b y /bin/rm -r . escapeshellarg(
$tmpdir2 ) . 21 );
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Update of SGE on Thursday 5. July
On 15/07/12 14:39, DaB. wrote: -There is -l arch=lx now which will run your task at a linux-host. Wouldn't a naming of arch=linux be preferable? Being simple to understand seems more important than the 3 characters saved. -Hawthorn and Clematis are not longer (available) submit-hosts. Connecting to submit.toolserver.org I still arrive at clematis... ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Update of SGE on Thursday 5. July
Am 15/07/12 20:32, DaB. schrieb: -Hawthorn and Clematis are not longer (available) submit-hosts. Connecting to submit.toolserver.org I still arrive at clematis... Yes, you are right. I choose the wrong term. There not longer sge-execution- hosts. That makes much more sense. Thanks. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Quota
On 11/07/12 23:44, Dr. Trigon wrote: Sorry Trigon, but it doesn't work. It is calculating the percentage of the hard limit at which your soft limit is set. Not really what you wanted :) May be we do not have the same 'quota -v' output, mine is: Disk quotas for drtrigon (uid 1340): Filesystem usage quota limittimeleft files quota limittimeleft /home 308.2M 500.0M 1000.0M 6016 0 0 You have to consider that '1340' from '(uid 1340)' does also match the regex, so it should work fine. At least for me it does for shure since I got mail as expected. :) I stand corrected. I had only tested with the second line. Cheers ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] s1 replag, some observations
On 11/07/12 16:14, Russell Blau wrote: I'm glad to know that the appearance was deceiving. Having data from rosemary now, I come up with an estimated completion date there of July 30, assuming that the rate at which updates are received from WMF does not change. I thought it would be worse. How are you measuring the sha1-populated boundaries? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Quota
On 09/07/12 20:52, Dr. Trigon wrote: Sorry, this copy'n'paste is useless - thus I attached the file here. On 09.07.2012 19:20, Dr. Trigon wrote: So no answer at all... that's a pitty! To have at least some information, I wrote a small script: Sorry Trigon, but it doesn't work. It is calculating the percentage of the hard limit at which your soft limit is set. Not really what you wanted :) Moreover, even if you fixed it ( float(size[0])/float(size[1]) ), it has problems with eg. using 600.0K out of the 500.0M quota. Also, no need to hardcode the email address. You could simply run: (echo Subject: You are reaching your quota quota -v ) | mail $USER ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Dude, where is my logfile?
On 25/06/12 20:44, Hazard-SJ wrote: So anyone can create folders in /mnt/user-store right? Hazard-SJ Yes. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Dude, where is my logfile?
On 22/06/12 00:16, Platonides wrote: On 21/06/12 23:12, Andre Koopal wrote: I had a quick look over the MMP's I have access to, and found some core-files, that is perhaps one you can scan for as well. I would say it is save to remove all core-files older then a month. Regards, Andre Why are they generated by default? *Removes two php-cgi cores 30M each, plus 8M python core* /me removes two php-cgi cores, 12M and 160M, being now in quota limits. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Dude, where is my logfile?
And then I see your message that (albeit it not appearing so) the quota has indeed been enabled for everyone now. Why? Now I can't even try to clean up, because I can't even edit a big file and replace it with Temporarily disabled. I can't remove 100MB to add a small .ini file. I can't comment out things that are breaking stuff. My account is completely locked and anything I try to touch is immediately wiped. Error/warnings are absent. I have also been bitten by this. On the one hand, it forced me to go through some archived logs that had gone wild. On the other hand, I wanted to a three-line fix to a tool, and that was a blocker. Not even overwriting a bigger file. Delete a file to free space? Nope, it doesn't allow you to write anything. cp over an existing file? No. dd over an existing file? No, that still usess O_TRUNC. Edit the file with nano and paste the new contents? No dd conv=notrunc *does* work. But it's not trivial to come up. Also, if you got the report: NFS server ha-nfs.esi not responding still trying NFS server ha-nfs.esi ok Does it mean everything was processed correctly on disk? I have a process running for 14 hours checking a tar file which took 20 minutes to create. Seems a bit odd :/ On 23/06/12 03:04, Krinkle wrote: I did not connect the problems I was hearing about from all over with the quota that was going to come after the weekend. The reason being that I did not get any emails regarding limits being reached on /home/krinkle (or the home of the MMPs) or any errors when trying to write to a file. e.g. $ echo Hello World test.txt .. works fine without errors. But looking it up shows it is size 0 bytes. Interesting. I first suspected echo (1) wasn't reporting the problem, but there's no error reported to the application: $ truss /usr/bin/echo Hello World test.txt (...) write(1, H e l l o W o r l d\n, 12)= 12 _exit(0) OTOH, cat reports the error: $ cat /tmp/test.txt test.txt cat: write error: Disc quota exceeded The system call is failing there: write(1, H e l l o W o r l d\n, 12)Err#49 EDQUOT Why such discrepancy? Edit: actually, it seems that it sometimes the write succeeds and in others it fails, with echo exiting in such case with non-zero status (/usr/bin/echo doesn't provide a message in stderr, but bash builtin does). write(1, H e l l o W o r l d\n, 12)Err#49 EDQUOT _exit(1) I think NFS is partly at fault here for the random behavior. The logic seem to be that if the file doesn't exist, you get no error on write, and later close() returns the Err#49 EDQUOT. But if the file already exists, the error is returned directly on the write(). Unlike echo, cat is performing a close (both binary and builtin), which allows it to detect that condition. echo doesn't need to do such close(), but as bash isn't reporting the error on close, it silently fails. Workaround, replace your echo Hello World test.txt calls to echo Hello World | cat test.txt and have fun when someone mocks of you for writing silly code :) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Running weblinkchecker.py on Toolserver
On 22/06/12 11:38, Yishen Miao wrote: Hello everyone, I set my bot (Mys_bot) to run weblinkchecker.py on toolserver. It needs to check all pages on zh_wikipedia for the first time through api. This task would be done much more efficiently if instead of grabbing all the wikipedia links from the api, it used the toolserver copy (externallinks table). Nonetheless, checking lots of foreign servers will still be very slow. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Dude, where is my logfile?
On 21/06/12 23:12, Andre Koopal wrote: I had a quick look over the MMP's I have access to, and found some core-files, that is perhaps one you can scan for as well. I would say it is save to remove all core-files older then a month. Regards, Andre Why are they generated by default? *Removes two php-cgi cores 30M each, plus 8M python core* ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Anoter SGE question
On 16/06/12 23:33, Dr. Trigon wrote: I have a bunch of cronie jobs calling qcronsub for several times with very similar settings (just the language of the wiki used changes). In total there are 5 jobs - regurarly (about ever 2nd day) one of those jobs does not get executed and it is always the same one. I do not get any error mail. Any idea? Is it by chance your last cron entry? Remember that there must be a newline finalising your crontab or the last command won't get executed. You should get this output: $ crontab -l | tail -c1 | od -c 000 \n 001 ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Write attempts to wiki API from TS timeout
On 12/06/12 21:31, Dr. Trigon wrote: Hello everybody! Since 7th (may be 6th) of june my bot is not able to write pages on dewiki larger than ~130KB. The bot code was not changed but now occurs MaxTriesExceeded errors with urlopen error timed out all the time. Befor it was able to write pages 600KB. It looks to like the uplink is has slowed down (bottleneck?) or the timeout was reduced... I am using pywikipedia framework and know of at least 1 other user having the same issue. Any idea what might be the problem here? Thanks a lot and greetings! DrTrigon You are now editing through IPv6. Maybe the IPv6 link is slower, the parameters are different or toolserver is being throttled. Open a bug in https://bugzilla.wikimedia.org/ with the ops keyword. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] SGE qcronsub auto-flush?
On 12/06/12 21:31, Dr. Trigon wrote: Hello everybody! I experience a behaviour with SGE that I cannot understand. Due to logging my scripts output I redirect stdout and stderr. When running the script from console (e.g. on willow) all write actions to stderr are passed as they are, but runned through qcronsub (SGE) results in all writes splitted up at newlines '\n' as if there was some kind of auto-flush in background? Changing the usage of param 'j' in SGE did not help. I am using python. Is there something in qcronsub that has in influence to this? Any idea? Thanks a lot and greetings! DrTrigon That looks like line buffering in stdio. You can try prepending the python command with: stderr -e0 (despite the fact that stderr should be unbuffered by default...) I'm unsure if it's being buffered at python or if SGE is doing caching there, thoguh. It _should_ be simply passing the file descriptor but, who knows? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Crontab not running?
On 09/06/12 18:02, Hersfold wrote: Hello, I'm having some strange issues with my crontab. I've got this line in there, which has worked for months: 0 */2 * * * cronsub -s HersfoldArbClerkBot % java -jar HersfoldArbClerkBot.jar HersfoldArbClerkBot.out Last night I replaced the jar in question with a new version (same name, location, etc.); since then, it doesn't appear that it's been running or even attempting to run. Usually when it fails to run I get an email, or at the very least something should be entered in that .out file; unfortunately I got no emails, and the file didn't exist, yet my bot didn't make any edits until I started it manually a short while ago. It should have run just now at 16:00 UTC, but again, no dice. Is there something wrong with cron where it's stopped functioning correctly? User:Hersfold hersfoldw...@gmail.com Maybe the new HersfoldArbClerkBot.jar has some dependency which is only fullfilled in the login server? (thus it dies in the new version) What's the point of that % character? That redirection is also confusing, since cronsub would use HersfoldAhrbClerkBot.out as output file, and you're also using a redirection to the same file. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Login servers
On 08/06/12 22:54, DaB. wrote: Hello, At Friday 08 June 2012 22:43:54 DaB. wrote: hostbased authentication should be as easy as configuring the yarrow key at the other servers. What's the problem? the problem is, that that is exactly what I did (and changing various ssh- settings) and that it still not work for unknown reasons. My next testing- setup will test if the hoping between linux-hosts is possible (then it could be a linux-solaris-problem) or not (then it could be a config-problem). Sincerely, DaB. It should work. Moreover, both Solaris and Linux ssh come from OpenSSH. yarrow is trying hostbased authentication (that's the first thing I suspected could be wrong), I'd look at the sshd log for the reason of the refusal. Maybe the sshd server needs to be restarted to pick the new yarrow host key? (let's remember that its key changed with the reinstall) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Update on IPv6
On 07/06/12 16:04, Brad Jorsch wrote: At the very least, all pywikipedia bots on the toolserver will edit over IPv6 - and I think a lot of the non-pywikipedia bots also, unless they resolve IP addresses manually. Of course, they won't edit anonymously, so you'll only see it in the access logs. Speaking of the toolserver, does anyone happen to know which IPv6 addresses belong to it? Just looking in DNS, it seems the named servers are currently in 2620:0:862:101::2:0/124, with some other toolserver.org addresses resolving in 2620:0:862:101::1:3/124 and 2620:0:862:101::3:0/124. A little playing shows: hemlock2620:0:862:101:0:0:2:0 clematis* 2620:0:862:101:0:0:2:1 zedler 2620:0:862:101:0:0:2:2 nightshade 2620:0:862:101:0:0:2:3 willow*2620:0:862:101:0:0:2:4 hawthorn* 2620:0:862:101:0:0:2:5 wolfsbane* 2620:0:862:101:0:0:2:6 ortelius* 2620:0:862:101:0:0:2:7 yarrow*2620:0:862:101:0:0:2:8 daphne 2620:0:862:101:0:0:2:9 damiana2620:0:862:101:0:0:3:0 2620:0:862:101:0:0:3:1 2620:0:862:101:0:0:3:2 2620:0:862:101:0:0:3:3 turnera2620:0:862:101:0:0:3:4 2620:0:862:101:0:0:3:5 2620:0:862:101:0:0:3:6 2620:0:862:101:0:0:3:7 thyme 2620:0:862:301:0:0:2:0 ptolemy2620:0:862:301:0:0:2:1 hemlock2620:0:862:301:0:0:2:2 [outdated dns entry?] rosemary 2620:0:862:301:0:0:2:3 hyacinth 2620:0:862:301:0:0:2:4 cassia 2620:0:862:301:0:0:2:5 adenia 2620:0:862:301:0:0:2:6 scs-oe10 2620:0:862:301:0:0:2:7 (Cisco) scs-oe16 2620:0:862:301:0:0:2:8 2620:0:862:101::1:0, 2620:0:862:101::1:1, 2620:0:862:101::1:2, 2620:0:862:101::1:3, 2620:0:862:101::1:4 point to www.toolserver.org, ha-dns-auth.toolserver.org, ha-mail.toolserver.org, ha-www.toolserver.org, ha-lb.toolserver.org which seem to mean they are handled by the damiana-turnera pair. You will probably only view toolserver edits from the server marked with * (plus nightshade when it gets setup again) until the cluster grows. I have tested for those that the listed ips are indeed those viewed as source by the wikis. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Day of the open linux-server
On 10/05/12 12:09, DaB. wrote: Please remind that after 22:00 UTC the box will be rebooted and so all programs will stop – yarrow is only for testing stuff not productive yet. Seems it has been happily running since that reboot. Can it begin to be used now? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Labs replication?
On 05/06/12 22:16, Danny B. wrote: Would it be possible to replicate labs? Danny B. What do you want to replicate? There's nothing worth to be replicated there. It would be better for labs to replicate toolserver dbs! ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver user phe account expired
On 25/05/12 23:12, Marcin Cieslak wrote: If some of you needs help moving a php tool into a MediaWiki extension, drop me a line. :) . o O (only not to be allowed to run it on toolserver later?:) Perhaps we may be able to make it a cross-tool, which can either be embedded in MediaWiki or standalone (I'm not aware of any extension doing that, but it's something I have considered several times myself). ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver user phe account expired
On 22/05/12 11:22, Merlijn van Deen wrote: How do we make sure tools do not disappear? By making multi-maintainer projects for them. However, we see that this doesn't happen enough - see also the thread about the expiration of soxred93's account. Options for improvement: - better communication with wikis - which tools are used a lot and *thus* should be moved to mmp's? - easier creation of mmp's? I can imaging people don't move their tools because it takes time to organise everything. It's relatively hard to create a MMP. Compare that with the complexity of doing a mkdir for creating a project in your account. Add to that the relatively low interest of other people for maintaining external projects (as shown by Magnus mail). There's little reason to create a MMP in advance. Plus, each of is coding using different languages, conventions and frameworks (helper functions). Maybe we should use a model where stable tools are available in a repository where all users can commit. The code can only be updated through that. As an alternative, each project could be in either open-gate or closed-gate model. In the first one, anyone can commit there. In the second one, there's just a subset of users which can directly commit (commits by others must be approved by a project member). If the accounts for all the project members expire (it gets orphan), the tool automatically changes to open-gate mode. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver user phe account expired
On 22/05/12 19:16, DaB. wrote: that's the reason why there is a toolserver: Because the WMF (and the mediawiki-develeopers/techs) can not do everything themself and there are several people with good ideas outside the WMF. So the idea was we give them a (yes 1 ;-)) server and they can hosts their stuff there (in opposite to host it somewhere external) – that part worked quite well; there was also a second part and than their stuff can move into mediawiki when it is done and the code is good enough (and it has to be php…). – that part never worked (at least I do not know an example). If some of you needs help moving a php tool into a MediaWiki extension, drop me a line. :) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Spam problem on Toolserver wiki
On 06/05/12 23:34, MZMcBride wrote: Hi. https://wiki.toolserver.org/view/Special:Log/block I think there's already a Math CAPTCHA enabled, but clearly this script/bot/whatever has defeated that. Not sure what needs doing. MZMcBride MathCaptcha is not a mean to prevent a bot designed to overcome it. I'd place a QuestyCaptcha asking for the text subtitle located in the top, at the right of the logo. (Ie. the text of https://wiki.toolserver.org/w/images/7/78/Headerbg.png) ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
