Re: [Wikitech-l] Video Uploads to Commons
On Wed, Feb 4, 2015 at 1:28 AM, Nkansah Rexford seanmav...@gmail.com wrote: Its very interesting for me to know similar tools exist in the system. I wish a tool like https://tools.wmflabs.org/videoconvert/ was listed on the ways to upload videos on commons page. I would have used it right away. I added a section Online conversion tools to https://commons.wikimedia.org/wiki/Help:Converting_video -- =S Page WMF Tech writer ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
Thank you for adding that. I'm sure when my tool is ready using OAuth, it might be added to the list. Is it possible to run/host a python based application on wmflabs? On Feb 6, 2015 7:25 PM, S Page sp...@wikimedia.org wrote: On Wed, Feb 4, 2015 at 1:28 AM, Nkansah Rexford seanmav...@gmail.com wrote: Its very interesting for me to know similar tools exist in the system. I wish a tool like https://tools.wmflabs.org/videoconvert/ was listed on the ways to upload videos on commons page. I would have used it right away. I added a section Online conversion tools to https://commons.wikimedia.org/wiki/Help:Converting_video -- =S Page WMF Tech writer ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
On Fri, Feb 6, 2015 at 12:07 PM, Nkansah Rexford nkansahrexf...@gmail.com wrote: Is it possible to run/host a python based application on wmflabs? Sure, there are a bunch there. https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Quick_start (get a labs account, create a new tools account or maybe join the existing videoconvert tool account). -- =S Page WMF Tech writer ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
Thanks for all the ideas and suggestions. Will start with enabling oauth right away. I'm dead in php so I always look for python related hooks. I think this will do for the oauth? http://pythonhosted.org/mwoauth/ As regards the conversion to webm instead of ogv, I tried but couldn't get avconv to process files into the webm format. Perhaps I'm not putting things together right. Will read more on that. Its very interesting for me to know similar tools exist in the system. I wish a tool like https://tools.wmflabs.org/videoconvert/ was listed on the ways to upload videos on commons page. I would have used it right away. Really appreciate all your thoughts. With MediaWiki I wish I was a bit good in php. Thus I always look around for python related packages to interact with the API. But with the oauth module found, I think authentication won't be the 'crude' way I'm doing it now. Will reach out to 'real' users on VP soon too. Hi, there is the tool by Holger which does that using WebM - but it is hosted on WMF Labs. It uses OAuth for unified login and moving the files to Commons. Then, a more elaborate tool for * storing raw material at the Internet Archive * generating patent free WebM proxy clips for editing * rendering high-quality videos * moving these rendered videos to Commons directly is the Video Editing Server, developed by some Wikipedians and a MLT developer, hosted by the Internet Archive: https://wikimedia.meltvideo.com/ It also uses OAuth for login and moving files to Commons. The workflow with this: # upload all your raw files to the server for ## long-term storage ## to make them available to other editors ## to let the server use them in the rendering process # the server transcodes all files into WebM proxy clips # editors download the WebM proxy clips ## do the editing on your computer ## create an MLT project file (eg. using kdenlive or another MLT-based video editor) # upload the project file ## server will replace proxy clips with raw material ## server will render video project ## server will move generated file to Commons It comes with a search engine, meta data forms... it's still pretty new (development started in December '14) but can be used. We plan to add some more features like tagging using Wikidata QIDs (hence allowing multilingual / localised tagging / searching, adding more project file formats and renderer, making old project file revisions available for download, give it a nice vector-based theme, give it a better domain name and SSL certificate... Play with it and have fun! For source code or any issues refer to GitHub: https://github.com/ddennedy/wikimedia-video-editing-server also there see the wiki for the specs and a deployment guide: https://github.com/ddennedy/wikimedia-video-editing-server/wiki /Manuel -- Wikimedia CH - Verein zur Förderung Freien Wissens Lausanne, +41 (21) 34066-22 - www.wikimedia.ch ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
Hi Magnus, On 02/04/2015 04:40 PM, Magnus Manske wrote: I wrote a PHP class to specifically deal with the Wikimedia OAuth, including uploads (not chunked though). May be helpful. https://bitbucket.org/magnusmanske/magnustools/src/9dc80c2479a41239b9661b35504dcaaaedf367f7/public_html/php/oauth.php?at=master very good! Because I have written a PHP library to use the MW library - it does not OAuth, because it is command-line based, but it can do chunked uploads! Maybe there is a way to add OAuth to it, then I will shamelessly steal from your code. My code is here: https://github.com/masterssystems/phpapibot I know that at least Dan used some it for inspiration for the video editing server upload component ;-) /Manuel -- Wikimedia CH - Verein zur Förderung Freien Wissens Lausanne, +41 (21) 34066-22 - www.wikimedia.ch ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
I wrote a PHP class to specifically deal with the Wikimedia OAuth, including uploads (not chunked though). May be helpful. https://bitbucket.org/magnusmanske/magnustools/src/9dc80c2479a41239b9661b35504dcaaaedf367f7/public_html/php/oauth.php?at=master On Wed Feb 04 2015 at 09:29:28 Nkansah Rexford seanmav...@gmail.com wrote: Thanks for all the ideas and suggestions. Will start with enabling oauth right away. I'm dead in php so I always look for python related hooks. I think this will do for the oauth? http://pythonhosted.org/mwoauth/ As regards the conversion to webm instead of ogv, I tried but couldn't get avconv to process files into the webm format. Perhaps I'm not putting things together right. Will read more on that. Its very interesting for me to know similar tools exist in the system. I wish a tool like https://tools.wmflabs.org/videoconvert/ was listed on the ways to upload videos on commons page. I would have used it right away. Really appreciate all your thoughts. With MediaWiki I wish I was a bit good in php. Thus I always look around for python related packages to interact with the API. But with the oauth module found, I think authentication won't be the 'crude' way I'm doing it now. Will reach out to 'real' users on VP soon too. Hi, there is the tool by Holger which does that using WebM - but it is hosted on WMF Labs. It uses OAuth for unified login and moving the files to Commons. Then, a more elaborate tool for * storing raw material at the Internet Archive * generating patent free WebM proxy clips for editing * rendering high-quality videos * moving these rendered videos to Commons directly is the Video Editing Server, developed by some Wikipedians and a MLT developer, hosted by the Internet Archive: https://wikimedia.meltvideo.com/ It also uses OAuth for login and moving files to Commons. The workflow with this: # upload all your raw files to the server for ## long-term storage ## to make them available to other editors ## to let the server use them in the rendering process # the server transcodes all files into WebM proxy clips # editors download the WebM proxy clips ## do the editing on your computer ## create an MLT project file (eg. using kdenlive or another MLT-based video editor) # upload the project file ## server will replace proxy clips with raw material ## server will render video project ## server will move generated file to Commons It comes with a search engine, meta data forms... it's still pretty new (development started in December '14) but can be used. We plan to add some more features like tagging using Wikidata QIDs (hence allowing multilingual / localised tagging / searching, adding more project file formats and renderer, making old project file revisions available for download, give it a nice vector-based theme, give it a better domain name and SSL certificate... Play with it and have fun! For source code or any issues refer to GitHub: https://github.com/ddennedy/wikimedia-video-editing-server also there see the wiki for the specs and a deployment guide: https://github.com/ddennedy/wikimedia-video-editing-server/wiki /Manuel -- Wikimedia CH - Verein zur Förderung Freien Wissens Lausanne, +41 (21) 34066-22 - www.wikimedia.ch ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 This is the tool be prolineserver: http://tools.wmflabs.org/videoconvert/ -BEGIN PGP SIGNATURE- Version: APG v1.1.1 iQJABAEBCgAqBQJU0bgrIxxNYXRhbnlhIE1vc2VzIDxtYXRhbnlhQGZvc3MuY28u aWw+AAoJEKzSGXfsOI0vItEP+weGjpdbmtlUFxzaBoRhCiF6PpMPhWd2d5BMA2x/ Qa2NeDy0N9Xfqixt60W5iHFi0JZ1DUmFlWd9KotEptRNRKzTw/7Mt/hQKqWyhDFo GO9XKND/9b/omLm/VsMx4jombx3hMbqoMZtx1SCFxTedAfdbszDX6bwnaUPGoSVB FDHMc43PoNTGxAF6H8UY1nx1gcA+jJ0u+vzApH6hvauay5T1unZ5E7bkvrYfA8fl qtklcZIDFOOU0iK4IHtC5gKY4Ds1XD7CCj51g1h+cBIgNd+oD+x1GxS7M02kdJDn jHwUwALVN1XBH3+iUL2DhpPn2e2RFdH8qTpdmLZ0n6Xu03/78ycWuhrmYcZ51awq j4E2IVyu7jytFBM9QiAXpEzoEm8O8rY77xBo4JDYSO7kZmygGVC8TusjNs9P0JT+ asOMakbKg4z0RxT/QwCUR2S74PBlpG5ON+V9bMXVMF52ZkR36XlaVA7nx+FvwCAX eCcgVGTbpRLa4Cpw2zQandQ4Hhf2U7dzw5A/ijwBeryVbCxOztR2lHdGFlfz+97S v6rnM5YmU7L3GCARq0ulFqUw5/uS4ZVWi9zFhdtqTLS8lVoruhVDFFrBe9DePhvQ XV+5EtJNcKzMvSxUH9Kl+iXw0PjyC14bna99IyU2aWv6bHdqxtneJfdfDJCGas/Z nixK =C7uz -END PGP SIGNATURE- ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
On Tue, Feb 3, 2015 at 5:19 PM, Brian Wolff bawo...@gmail.com wrote: On Feb 3, 2015 8:43 PM, Nkansah Rexford seanmav...@gmail.com wrote: I couldn't find a tool to convert my videos from whatever format into .ogv outside my PC box before pushing to Commons. I guess there might exist something like that, but perhaps I can't find it. But I made one for myself. Maybe it might be useful to others too. I call it CommonsConvert. Upload any video format, enter username and password, and get the video converted into .ogv format plus pushed to Commons all in one shot. You upload the original video in whatever format, and get it on Commons in .ogv Some rough edges currently, such as can't/don't know/no available means to append license to uploaded video among other issues. Working on the ones I can asap. It uses mwclient module via Django based on Python. Django gives user info to Python, Python calls avconv in a subprocess, and the converted file is relayed to Commons via mwclient module via Media Wiki API. I think not everyone has means/technical know how/interest/time converting videos taken on their PC to an Ogg-Vorbis-whatever format before uploading to Commons. Doing the conversion on a server leaves room for user to focus on getting videos than processing them. I don't know if this is or will be of any interest that someone might wanna use, but I personally would enjoy having a server sitting somewhere convert my videos I want to save onto Commons, than using my local computer doing that boring task. In an email to this list a week or so ago, I 'ranted' about why commons wants a specific format (which if not for commons, I never come across that format anywhere), but has no provision for converting any videos thrown at it into that format of its choice. Well Tool can be found here: khophi.co http://khophi.co/commonsconvert/ http://khophi.co/commonsconvertcommonsconvert http://khophi.co/commonsconvert And this is sample video uploaded using the tool. https://commons.m.wikimedia.org/wiki/File:Testing_file_for_upload_last.ogv (will be deleted soon, likely) What I do not know or have not experimented yet is whether uploading using the api also has the 100mb upload restriction. Will appreciate feedback. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l Cool. Thanks for working on this sort of thing. The uploading videos process certainly benefit from some love. May i suggest using webm (of the vorbis/vp8 variety) as the output format? Webm will give higher quality videos at lower file size, so is probably a better choice if converting from another format. For the 100mb thing - there are multiple ways to upload things with the api. The chunked method has a file size limit of 1gb. All the other methods have the 100mb limit. See https://commons.wikimedia.org/wiki/Commons:Chunked_uploads for more information. If you havent already, id encourage mentioning this on [[Commons:VP]]. Real users would probably be able to give much more specific feedback. Cheers, Bawolff P.s. im not sure, but i think user:Prolineserver might have been working on something similar, in case you are looking for collaborators. That's https://tools.wmflabs.org/videoconvert/ - seems to be down right now, but usually it works very well for converting to WebM, and can also upload the result directly to Commons using OAuth. It's just not very fast (can take several hours to generate a 200MB WebM). Results: https://commons.wikimedia.org/wiki/Category:Uploaded_with_videoconvert -- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
Hi, there is the tool by Holger which does that using WebM - but it is hosted on WMF Labs. It uses OAuth for unified login and moving the files to Commons. Then, a more elaborate tool for * storing raw material at the Internet Archive * generating patent free WebM proxy clips for editing * rendering high-quality videos * moving these rendered videos to Commons directly is the Video Editing Server, developed by some Wikipedians and a MLT developer, hosted by the Internet Archive: https://wikimedia.meltvideo.com/ It also uses OAuth for login and moving files to Commons. The workflow with this: # upload all your raw files to the server for ## long-term storage ## to make them available to other editors ## to let the server use them in the rendering process # the server transcodes all files into WebM proxy clips # editors download the WebM proxy clips ## do the editing on your computer ## create an MLT project file (eg. using kdenlive or another MLT-based video editor) # upload the project file ## server will replace proxy clips with raw material ## server will render video project ## server will move generated file to Commons It comes with a search engine, meta data forms... it's still pretty new (development started in December '14) but can be used. We plan to add some more features like tagging using Wikidata QIDs (hence allowing multilingual / localised tagging / searching, adding more project file formats and renderer, making old project file revisions available for download, give it a nice vector-based theme, give it a better domain name and SSL certificate... Play with it and have fun! For source code or any issues refer to GitHub: https://github.com/ddennedy/wikimedia-video-editing-server also there see the wiki for the specs and a deployment guide: https://github.com/ddennedy/wikimedia-video-editing-server/wiki /Manuel -- Wikimedia CH - Verein zur Förderung Freien Wissens Lausanne, +41 (21) 34066-22 - www.wikimedia.ch ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Video Uploads to Commons
I couldn't find a tool to convert my videos from whatever format into .ogv outside my PC box before pushing to Commons. I guess there might exist something like that, but perhaps I can't find it. But I made one for myself. Maybe it might be useful to others too. I call it CommonsConvert. Upload any video format, enter username and password, and get the video converted into .ogv format plus pushed to Commons all in one shot. You upload the original video in whatever format, and get it on Commons in .ogv Some rough edges currently, such as can't/don't know/no available means to append license to uploaded video among other issues. Working on the ones I can asap. It uses mwclient module via Django based on Python. Django gives user info to Python, Python calls avconv in a subprocess, and the converted file is relayed to Commons via mwclient module via Media Wiki API. I think not everyone has means/technical know how/interest/time converting videos taken on their PC to an Ogg-Vorbis-whatever format before uploading to Commons. Doing the conversion on a server leaves room for user to focus on getting videos than processing them. I don't know if this is or will be of any interest that someone might wanna use, but I personally would enjoy having a server sitting somewhere convert my videos I want to save onto Commons, than using my local computer doing that boring task. In an email to this list a week or so ago, I 'ranted' about why commons wants a specific format (which if not for commons, I never come across that format anywhere), but has no provision for converting any videos thrown at it into that format of its choice. Well Tool can be found here: khophi.co http://khophi.co/commonsconvert/ http://khophi.co/commonsconvertcommonsconvert http://khophi.co/commonsconvert And this is sample video uploaded using the tool. https://commons.m.wikimedia.org/wiki/File:Testing_file_for_upload_last.ogv (will be deleted soon, likely) What I do not know or have not experimented yet is whether uploading using the api also has the 100mb upload restriction. Will appreciate feedback. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
Very nice! However, according to https://meta.wikimedia.org/wiki/Terms_of_use, users' passwords should never be disclosed to any third party. Please read OAuth/For Developers https://www.mediawiki.org/wiki/OAuth/For_Developers and convert your tool to use OAuth instead ;-) Il 04/02/2015 01:43, Nkansah Rexford ha scritto: I couldn't find a tool to convert my videos from whatever format into .ogv outside my PC box before pushing to Commons. I guess there might exist something like that, but perhaps I can't find it. But I made one for myself. Maybe it might be useful to others too. I call it CommonsConvert. Upload any video format, enter username and password, and get the video converted into .ogv format plus pushed to Commons all in one shot. You upload the original video in whatever format, and get it on Commons in .ogv Some rough edges currently, such as can't/don't know/no available means to append license to uploaded video among other issues. Working on the ones I can asap. It uses mwclient module via Django based on Python. Django gives user info to Python, Python calls avconv in a subprocess, and the converted file is relayed to Commons via mwclient module via Media Wiki API. I think not everyone has means/technical know how/interest/time converting videos taken on their PC to an Ogg-Vorbis-whatever format before uploading to Commons. Doing the conversion on a server leaves room for user to focus on getting videos than processing them. I don't know if this is or will be of any interest that someone might wanna use, but I personally would enjoy having a server sitting somewhere convert my videos I want to save onto Commons, than using my local computer doing that boring task. In an email to this list a week or so ago, I 'ranted' about why commons wants a specific format (which if not for commons, I never come across that format anywhere), but has no provision for converting any videos thrown at it into that format of its choice. Well Tool can be found here: khophi.co http://khophi.co/commonsconvert/ http://khophi.co/commonsconvertcommonsconvert http://khophi.co/commonsconvert And this is sample video uploaded using the tool. https://commons.m.wikimedia.org/wiki/File:Testing_file_for_upload_last.ogv (will be deleted soon, likely) What I do not know or have not experimented yet is whether uploading using the api also has the 100mb upload restriction. Will appreciate feedback. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Video Uploads to Commons
On Feb 3, 2015 8:43 PM, Nkansah Rexford seanmav...@gmail.com wrote: I couldn't find a tool to convert my videos from whatever format into .ogv outside my PC box before pushing to Commons. I guess there might exist something like that, but perhaps I can't find it. But I made one for myself. Maybe it might be useful to others too. I call it CommonsConvert. Upload any video format, enter username and password, and get the video converted into .ogv format plus pushed to Commons all in one shot. You upload the original video in whatever format, and get it on Commons in .ogv Some rough edges currently, such as can't/don't know/no available means to append license to uploaded video among other issues. Working on the ones I can asap. It uses mwclient module via Django based on Python. Django gives user info to Python, Python calls avconv in a subprocess, and the converted file is relayed to Commons via mwclient module via Media Wiki API. I think not everyone has means/technical know how/interest/time converting videos taken on their PC to an Ogg-Vorbis-whatever format before uploading to Commons. Doing the conversion on a server leaves room for user to focus on getting videos than processing them. I don't know if this is or will be of any interest that someone might wanna use, but I personally would enjoy having a server sitting somewhere convert my videos I want to save onto Commons, than using my local computer doing that boring task. In an email to this list a week or so ago, I 'ranted' about why commons wants a specific format (which if not for commons, I never come across that format anywhere), but has no provision for converting any videos thrown at it into that format of its choice. Well Tool can be found here: khophi.co http://khophi.co/commonsconvert/ http://khophi.co/commonsconvertcommonsconvert http://khophi.co/commonsconvert And this is sample video uploaded using the tool. https://commons.m.wikimedia.org/wiki/File:Testing_file_for_upload_last.ogv (will be deleted soon, likely) What I do not know or have not experimented yet is whether uploading using the api also has the 100mb upload restriction. Will appreciate feedback. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l Cool. Thanks for working on this sort of thing. The uploading videos process certainly benefit from some love. May i suggest using webm (of the vorbis/vp8 variety) as the output format? Webm will give higher quality videos at lower file size, so is probably a better choice if converting from another format. For the 100mb thing - there are multiple ways to upload things with the api. The chunked method has a file size limit of 1gb. All the other methods have the 100mb limit. If you havent already, id encourage mentioning this on [[Commons:VP]]. Real users would probably be able to give much more specific feedback. Cheers, Bawolff P.s. im not sure, but i think user:Prolineserver might have been working on something similar, in case you are looking for collaborators. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
