Thanks for the image-processing suggestions. I have lots of pictures - "million" is not an exaggeration - and I'd like to do some kind of automated recognition and aesthetic ranking of them.
On Tue, Dec 5, 2017 at 7:36 PM, Scott Locklin <[email protected]> wrote: > On Dec 04, 2017; 7:37pm Devon McCormick wrote: > > > I searched the J wiki for "quadratic" and found a mention from a 2005 > > NYCJUG meeting where we were lamenting that other environments simply > > have > > quadratic solvers available but J does not. Looking into it more, I was > > unsure "quadratic" means in "quadratic solver"; from "Numerical > > Recipes", I > > gather that it may refer to quadratic convergence. On further > > reflection, > > I'm not exactly sure what this means either but suspect it means > > something > > like O(sqrt(n)) solution time but am not sure. I'm also unclear if this > > non-linear solver in J - http://code.jsoftware.com/wiki/Scripts/nlls - > is > > an example of a quadratic solver (with this follow-up page I put > > together - > > http://code.jsoftware.com/wiki/NYCJUG/2010-11-09/ > Levenberg-MarquardtAlgorithm > > ). > > Perhaps someone more acquainted with this domain can shed some light. > > My ultimate goal for these sort of tools is to do something with the > > million or so photos I have but I'm seeing from what I've learned so far > > that the real work is in feature-extraction, so that at least is > > something > > where I can use J to good effect. > > > > Image processing is one of the reasons people have moved to GPUs; it's > pretty heavy duty stuff for regular processors. Particularly on full > sized photographic quality images. "Do something with photos" is also a > REALLY open ended statement. What do you want to do? Identify faces in > the images? Identify similar shapes in an image database? SVM isn't very > good at any of these things, though it's OK at small stuff like MNIST > (classifying tiny cropped images of handwritten number digits). > > Identifying something is a face: use Viola-Jones (adaboost on Haar > wavelet summaries). > https://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_ > detection_framework > > Identifying a specific face in a bag of faces; after you've identified > the faces in your images, Eigenfaces is actually pretty easy to code up > in J: > https://en.wikipedia.org/wiki/Eigenface > > Identifying similar images... I dunno, maybe locality sensitive hashing. > > Feature extraction, maybe SIFT or HOG descriptors (along with Haar > wavelets): > https://en.wikipedia.org/wiki/Histogram_of_oriented_gradients > > If you still want to build a native J SVM (which you shouldn't try to > use on large photographic data), the appropriate quadratic programming > is this thing: > https://en.wikipedia.org/wiki/Quadratic_programming > > Most ML (and optimization such as QP) algorithms are fiddley and hard > enough to code up, people almost always use libraries. I mean, people > still use LAPACK which is more or less code from the 60s! Pretty sure > the quadprog algo in R is of similarly ancient vintage. > > flann is a good example of this: there are basically three useful open > source general purpose KD-tree-like data structures available to the > general public for doing low dimensional KNN kinds of things. It was a > Ph.D. thesis worth of work. It would be brilliant to have one in > native-J, but FFIing out isn't cheating! It's what everyone does! > > -SL > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > -- Devon McCormick, CFA Quantitative Consultant ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
