On Wed, Apr 11, 2012 at 9:40 PM, Carl Karsten <[email protected]> wrote: > On Wed, Apr 11, 2012 at 9:51 PM, Brian Matherly <[email protected]> wrote: >> >> >> Dan, >> >> I'd like to offer a proposal for a repository to keep all the scripts >> related to MLT: >> https://github.com/pez4brian/mlt-scripts >> >> I would like to start assembling various test scripts in the "test" >> directory. We could make one script per module - and each script could just >> launch melt with different options to exercise the module under test. >> >> The "teamcity" scripts would be used by the build server and would allow us >> to keep the wrapper scripts from being buried in the teamcity database. It >> would also make it easier to test the wrapper scripts before they are >> deployed on teamcity. >> >> The repository could be hosted anywhere - github, gitorious or mltframework >> - I don't have a preference. >> >> >> Let me know your thoughts. > > Hey Brian, > > Thanks for what you are doing - I am more of a mlt user than dev, but > I am coder and completely appreciate the value of unit tests. > > I have a lengthy and tangled wad of python code that tests a bunch of > stuff. In it's current state it isn't a reliable test - when it fails > I have to go track down what failed, and 1/2 the time it's essentially > a bug in the test. > > I would be up for making it fit for human consumption, but there is a > blocker: the debian pocketsphinx-utils package is deprecated - there > currently is no maintainer and what's currently packaged is pretty out > of date. I took a crack at packaging it, failed. so waiting for > someone else. > > Here is what my script does: > > renders the text ABCDE into .dv files, uses melt to mux them with an > audio file of a voice saying "go forward ten meters" and encode, the > demuxes the audio back out to a file and pulls a frame out, runs it > though gocr and looks for the string ABCDE, and runs the audio though
interesting use of gocr. Along these lines, other tests can use the new vqm transition to ensure an output matches its reference closely enough (e.g, SSIM > 0.92 (1.0 is perfect)). That could check for all sorts things. The basic idea is to have stuff that is similar to what is in the demo directory and manually render and verify the result to serve as the reference. Then, after each build, render/encode all of them and compare their results to the reference using -transition vqm piping the output through awk. > sphinx, text comes out, if all is well "go forward ten meters" > (don't be surprised, the input file is a sample from the sphinx > codebase) > > So it tests that melt can render text, encode and end up with a result > that looks and sounds acceptable. (basically. I am sure there are > cases where it may look/sound terrible but the automated test will be > smart enough to figure out the strings anyway.) > > It also tests that my django code can connect to the currently > configured database, and a bunch of other stuff that has nothing to do > with testing melt. again, not fit for human consumption. > > https://github.com/CarlFK/veyepar/blob/master/dj/scripts/run_tests.py > > Because I can no longer apt-get install sphinx (or whatever the > package is called, it no longer exits in the repo) I disabled the > audio tests from my script. > > So if someone can package current sphinx (or any other speech > recognition system) I'll rip my code apart into components and submit > them to your collection. I could add PSNR for each audio channel to the vqm transition. ------------------------------------------------------------------------------ For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 _______________________________________________ Mlt-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/mlt-devel
