Between 2016-10-20 and 2016-11-04, the Wikimedia Foundation ran a direct response user survey of registered Tool Labs users. 175 email recipients participated in the survey which represents 18% of those who were contacted.
Based on responses to demographic questions, the average* respondent: - Has used Tool Labs for 1-3 years - Developed & maintains 1-2 tools - Spends an hour or less a week using Tool Labs - Programs using PHP and/or Python - Does the majority of their work locally - Uses source control - Was not a developer or maintainer on Toolserver [1]: "Average" here means a range of responses covering 50% or more of responses to the question. This summarization is coarse, but useful as a broad generalization. Detailed demographic response data is available on wiki <https://meta.wikimedia.org/wiki/Research:Annual_Tool_Labs_Survey/2016> . Qualitative questions: - 87% agree that services have high reliability (up time). Up from 64% last year. - 71% agree that it is easy to write code and have it running on Tool Labs. Up from 69% last year. - 71% agree that they feel they are supported by the Tool Labs team when they contact them via labs-l mailing list, #wikimedia-labs IRC channel, or phabricator. Up from 67% last year. - 46% agree that they receive useful information via labs-announce / labs-l mailing lists. Down from 53% last year. - 49% disagree that documentation is easy-to-find. Down from 52% last year. - 89% find the support they receive when using Tool Labs as good or better than the support they received when using Toolserver. Up from 71% last year. - 50% agree that Tool Labs documentation is comprehensive. Up from 46% last year. - 48% agree that Tool Labs documentation is clear. Down from 50% last year. Service usage: - 39% use LabsDB often. Down from 45% last year. - 49% use webservices often. Down from 60% last year. - 42% use cronjobs often. Down from 54% last year. - 66% never use redis. Up from 75% last year. - 48% never use continuous jobs. Down from 41% last year. The survey included several free form response sections. Survey participants were told that we would only publicly share their responses or survey results in aggregate or anonymized form. The free form responses include comments broadly falling into these categories: - Documentation (54 comments) - Workflow (22 comments) - Stability & performance (11 comments) - Software versions (8 comments) - Monitoring (8 comments) - Kubernetes / PaaS (6 comments) - Storage (4 comments) - Support (4 comments) - Discovery (3 comments) - Other (15 comments) Documentation: Over a third of the comments concerned the state of Tool Labs documentation. Given the survey scores on documentation related questions this is not surprising. Many suggestions for improvement focus on step by step guides, task based tutorials, and new user on-boarding guides. Better coverage of database related issues also received multiple mentions. Workflow: Workflow related comments included several calls for easier methods of uploading and downloading files in shared tool accounts. Others hoped for some form of automated deployment related to GitHub repositories and web-based editing capabilities. Several people commented on difficulties learning and using Linux commands with a Windows background. Stability & performance: Three comments asked for overall stability to be a focus for the coming year's work. Increased performance of Tool Labs provided services such as database servers and the hosted tools themselves also received several mentions. Software versions: Java 8, PHP 7, and general software and library upgrades were requested. Monitoring: Services for monitoring and alerting on webservice uptime, grid jobs, and log files were included in the comments. Kubernetes / PaaS: Comments asked for expanded Kubernetes services, Docker support, and Platform as a Service rollout. Storage: Storage comments mentioned expanded capacity for database and large file storage. NFS speed improvements were also mentioned. Support: Comments on the topic of support called for expanded staffing by the WMF and additional coverage in the #wikimedia-labs irc channel. Discovery: Better systems to track and search existing tools are desired. Thanks to all who participated. Comments are welcome on the meta talk page <https://meta.wikimedia.org/wiki/Research_talk:Annual_Tool_Labs_Survey/2016> . Bryan -- Bryan Davis Wikimedia Foundation <[email protected]> [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855
_______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
