[Scikit-learn-general] Using RBM to infer missing data

Is it easy to perform collaborative filtering with scikit's "Restricted Boltzmann Machines" (RBM)? Assume given are user ratings of 3 movies and for the fourth movie the rating is missing. Clearly, using RBMs the reconstruction values constitute the missing data. But, I'm not sure how the RBM

Re: [Scikit-learn-general] Using RBM to infer missing data

Thank you Kyle for clearing my doubts about this approach. Nice concise explanation! ~Issam On 9/13/2013 2:42 AM, Kyle Kastner wrote: Take a look at the "missing values" section in Hinton's guide (http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf <http://www.cs.toronto.e

[Scikit-learn-general] Does scikit RBM support continuous values?

Reading the documentation, it seems that scikit's RBM does not support continuous values or, more precisely, values that are larger than one or less than zero; since logistic is the output function. This paper, "http://www.ee.nthu.edu.tw/~hchen/pubs/iee2003.pdf

Re: [Scikit-learn-general] Does scikit RBM support continuous values?

com/qa/questions/12157/problems-with-dbn-for-audio-data . Maybe I could start working on the simple approach - 'Gaussian visible nodes'. On 9/16/2013 10:59 PM, Andreas Mueller wrote: On 09/16/2013 09:54 PM, Issam wrote: Reading the documentation, it seems that scikit's

Re: [Scikit-learn-general] Does scikit RBM support continuous values?

Thanks guys! --Issam On 9/17/2013 12:30 PM, Olivier Grisel wrote: > 2013/9/17 Vlad Niculae : >> And under the current implementation, implementing them involves >> changing only the sampling and energy computation, I think. >> >> I discussed this with Gabriel Synna

[Scikit-learn-general] Reporting coverage level of a test file against a feature file

Hi scikit community, I'm writing a feature for scikit and I'm looking to find my test suite coverage. In short, I would like to know how I can use the 'coverage' library to see how much a test file, say test_feature.py, is covering the feature file, say feature.py. test_feature.py has metho

Re: [Scikit-learn-general] Reporting coverage level of a test file against a feature file

Thank you @Olivier, your explanation is elaborate and complete. :) On 9/22/2013 9:20 PM, Olivier Grisel wrote: > Just run: > > $ nosetests --with-coverage > > And consider only the lines related to your module in the report. > > Offcourse you can always run a specific test and see how it covers >

Re: [Scikit-learn-general] Neural Networks for supervised classification

quired. My intuition, however, tells me that its working perfectly :P. You could help us out by reporting any difficulties you might face. Thanks! Cheers! ~Issam On 9/23/2013 2:58 PM, Luca Cerone wrote: Thanks Olivier! Since I am new to github, has the pull request been added to the master,

Re: [Scikit-learn-general] Neural Networks for supervised classification

Thanks Olivier! >> Since I am new to github, has the pull request been added to the master, or >> are you waiting for it to be improved before merging? > The pull request is still open, hence not merged yet. The review is > still underway. > > Issam: if you think it's ready

[Scikit-learn-general] Gaussian RBM

Hi scikit, I coded Gaussian RBM in scikit form. Testing the code, however, on the digits dataset gave me unreliable performance. Though I followed references and py2learn's G-RBM to the T, the scores suggest that my implementation is wrong. Applying logistic regression on the raw pixels gave

Re: [Scikit-learn-general] Gaussian RBM

Thanks Lars. Nice idea, I'll give it a try! :) On 9/30/2013 8:07 PM, Lars Buitinck wrote: > 2013/9/30 Issam : >> I coded Gaussian RBM in scikit form. Testing the code, however, on the >> digits dataset gave me unreliable performance. Though I followed references >> and

[Scikit-learn-general] Suggestion to add author names/emails at the bottom of module documentations

Hi scikit-learn editors, Any documentation can have mistakes, but it's important to address them quickly and efficiently. One plausible way is to contact the author of an erroneous text to have him make proper changes. But, for all I know, scikit-learn's documentation lacks authors' informatio

[Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

Greedy layer-wise training algorithm for deep learning, extending MLP to allow for more than one hidden layer, where weights are initialized using Sparse Auto-encoders or RBM. How will this suit for GSoC? I thank you earnestly in advance... Best Regards, --Issam

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

I'm also still hopeful there. Unfortunately I will definitely be unable to mentor. About pretraining: that is really out of style now ;) Afaik "everybody" is now doing purely supervised training using drop-out. Implementing pretrained deep nets should be fairly easy for a user if we support mo

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

On 2/26/2014 3:29 PM, Lars Buitinck wrote: > We have a PR that implements them, but in too convoluted a way. My > personal choice for implementing these would be a transformer doing a > random projection + nonlinear activation. That way, you can stack any > linear model on top (think SGDClassifier

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

lp mentoring for the GSoC. If these could fill in the PSF sign up form, > it would be great: > > https://docs.google.com/forms/d/1wEaF22w2sKQY4iWxzFKmuYPf6ZEgV4uUKEKssHFSTBg/vie > wform > > Cheers, > > Gaël --Issam

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

Thanks Olivier, I will upload the proposal very soon. While doing so, I will strengthen my proposal by implementing a basic version of each of the proposed algorithms, which I will cite in my proposal. Cheers. :) On 3/14/2014 5:38 PM, Olivier Grisel wrote: > Issam if I am not mistaken

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

wrote: On Sat, Mar 15, 2014 at 6:59 PM, Issam <mailto:issamo...@gmail.com>> wrote: Thanks Olivier, I will upload the proposal very soon. While doing so, I will strengthen my proposal by implementing a basic version of each of the proposed algorithms, which I will cite in m

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

Hi all, I uploaded the Neural Network proposal to this link, https://github.com/scikit-learn/scikit-learn/wiki/GSoC-2014:-Extending-Neural-Networks-Module-for-Scikit-learn Please see if it is detailed enough as a promising proposal. Thank you. ~Issam On 3/19/2014 9:00 PM, Jaidev Deshpande

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

True that, I uploaded the proposal as a gist to this link, https://gist.github.com/IssamLaradji/9660324 Thank you. ~Issam Hi Issam, Looks OK at first glance, but please add it as a gist. Those are much easier to comment on. Thanks -- JD

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

Hi all, I uploaded the proposal for Neural Networks to melange, here is the public link. http://www.google-melange.com/gsoc/proposal/public/google/gsoc2014/issamou/5668600916475904 Thank you. Regards, ~Issam On 3/20/2014 10:47 AM, Jaidev Deshpande wrote: On Thu, Mar 20, 2014 at 5:54

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

h non-regularized and regularized ELM? If the former tends to overfit, I would keep only the latter. And why do you need 2 weeks for implementing the regularized variant? Is the algorithm fundamentally different from the non-regularized variant? Mathieu On Fri, Mar 21, 2014 at 4:44 AM,

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

hms I proposed, meaning I will not face unexpected obstacles :). Thanks, ~Issam On 3/21/2014 1:14 PM, Arnaud Joly wrote: > Hi Issam, > > > Why not starting by improving multilayer neural network before adding > new algorithms ? > > To neural network expert, is it

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

The thesis is due April 30, 2014, which is 19 days before GSoC starts :). > On Fri, Mar 21, 2014 at 01:26:00PM +0300, Issam wrote: >> You are right, I was supposed to finish my MLP PR before the summer, but >> my thesis took over my time, which, fortunately, I am completing t

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

How about assigning the first week to finalizing the PR? Because the documentation haven't been thoroughly reviewed yet. Thanks On 3/21/2014 2:28 PM, Jaidev Deshpande wrote: > > I'm sorry, I did not intend to +1 that. Does it make sense for Issam > to allocate two weeks of

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

Hi all, I updated the Neural Network proposal in melange, http://www.google-melange.com/gsoc/proposal/public/google/gsoc2014/issamou/5668600916475904 Thank you. ~Issam -- Learn Graph Databases - Download FREE O'R

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

the algorithms' outputs with that of the MATLAB's versions of the implementations, and theano's implementation of deep networks. :) 3. If you are just now completing your Masters degree on such models, you might want to mention that in your proposal's "Past Work" sec

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

> > I get that if you have 10, 000 samples and 150 features, then your > system is over-determined. > Where I think you go wrong is in worrying about a large number of > unique solutions. Over-determined typically means 0 solutions! (Have > another look at that page you linked, it's the under-d

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

e.pdf Thanks. ~Issam -- Learn Graph Databases - Download FREE O'Reilly Book "Graph Databases" is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in t

Re: [Scikit-learn-general] GSoC - Completing my Neural Network PRs and more

ithms I proposed, meaning I will not face unexpected obstacles :). Thanks, ~Issam On 3/21/2014 1:14 PM, Arnaud Joly wrote: Hi Issam, Why not starting by improving multilayer neural network before adding new algorithms ? To neural network expert, is it interesting to have layer configurat

[Scikit-learn-general] Accepted for GSoC 2014: Neural Networks

ct :). We are looking at a fun, fruitful summer just ahead of us. :) Best, ~Issam -- Start Your Social Network Today - Download eXo Platform Build your Enterprise Intranet with eXo Platform Software Java Based O

Re: [Scikit-learn-general] Accepted for GSoC 2014: Neural Networks

Hi James, and Gael, Thank you! I am very excited to be supervised by you and Oliver. This will be a wonderful summer :). Best regards. --Issam On 4/23/2014 3:23 PM, James Bergstra wrote: Congrats Issam, looking forward to working with you! On Wed, Apr 23, 2014 at 2:24 AM, Gael Varoquaux

Re: [Scikit-learn-general] Classifier for Neural Networks

Hi Eric, My GSoC 2014 project requires me to finalize the "vanilla feed-forward net" by next week. Here is the pull-request, https://github.com/scikit-learn/scikit-learn/pull/2120 The code only needs a final review and the I am working on completing the documentation. Thank

[Scikit-learn-general] About weekly posts for GSoc 2014

updated my blog for GSoc 2014 here, http://issamlaradji.blogspot.com/ Now my questions are as follows, 1) should I only post highlights of my weekly progress? or 2) should I provide a thorough description of the algorithms I am implementing? Thank you very much. --Issam

Re: [Scikit-learn-general] About weekly posts for GSoc 2014

least weekly. So don't worry, but do keep posting regularly! On 2 June 2014 09:28, Issam mailto:issamo...@gmail.com> <mailto:issamo...@gmail.com <mailto:issamo...@gmail.com>>> wrote: Hi scikit-team I am sorry tha

[Scikit-learn-general] Midterm summary for GSoC 2014

Hi, I posted my mid-term summary blog post for GSoC 2014 :). http://issamlaradji.blogspot.com/ Thanks. -- HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data

[Scikit-learn-general] GSoC 2014 neural networks - blog post update

Hi there, Posted an update on my progress in GSoC 2014 that requires implementing multi-layer perceptron, multi-layer perceptron with pretraining, and extreme learning machines. http://issamlaradji.blogspot.com/ Thank you. Best regards, --Issam

Re: [Scikit-learn-general] sparse datasets loading

Hi Eustache, Although this might be more time consuming than needed, I load a .csv file using `read_csv` in the `pandas` library (http://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.parsers.read_csv.html). You will get a dataframe, say DATAFRAME that you can convert to a numpy arra

[Scikit-learn-general] Final summary for GSoC 2014 posted

Hi all, I finished writing the final summary of my work for GSoC 2014. It is posted here: http://issamlaradji.blogspot.com/ Thank you! Best regards, --Issam Laradji

[Scikit-learn-general] Why Gaussian Naive Bayes is not working as a base classifier?

owing error after executing "In[67]" : *TypeError: fit() got an unexpected keyword argument 'sample_weight'* Any reason why I might be getting this? PS: I frequently use adaboosting with Navie Bayes as a base classifier in *WEKA**, *hence the concern*:) *Thank you very much!

Re: [Scikit-learn-general] Why Gaussian Naive Bayes is not working as a base classifier?

Thank you Peter, your answer if very helpful! Best, --Issam On 03/08/2013 01:40 PM, Peter Prettenhofer wrote: > Issam, > > currently, GaussianNB does not support sample weights thus it cannot > be used w/ Adaboost. > In Weka, if a classifier does not support sample weights they

[Scikit-learn-general] (Deep learning) pre-proposal for (GSOC) 2013

ndwriting, to breast cancer prediction using mammograms. I'm in my second semester as a graduate student (MSc), and machine learning is my research area. My thesis would involve deep learning, which i will apply on bioinformatics and face recognition. I would be more than happy to work with a

[Scikit-learn-general] (GSOC) Would scikit consider expanding to add reinforcement learning?

le suggestion, I would be definitely proposing it for GSOC. Thank you! --Issam -- Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET Get 100% visibility into your production application - at no cost. C

Re: [Scikit-learn-general] (GSOC) Would scikit consider expanding to add reinforcement learning?

Hi Andy, That is true, RL requires a whole new API, just didn't find PyBrain friendly enough :). Thanks, --Issam On 5/1/2013 12:56 PM, Andreas Mueller wrote: > Hi Issam. > I don't think adding RL is a good idea, as it requires a whole different > API. > Also, we try t

Re: [Scikit-learn-general] (Deep learning) pre-proposal for (GSOC) 2013

tion should be used, etc. It is early to go > into it too deeply (pun intended) but I think the proposal can benefit > by your embracing the skeptic side. > > Hope this helps, > Vlad > > > On Thu, May 2, 2013 at 5:20 PM, Vlad Niculae wrote: >> Hi Issam, >> >

Re: [Scikit-learn-general] (Deep learning) pre-proposal for (GSOC) 2013

p models do you recommend I should focus on within the given time frame? Thanks a lot, yours truly --Issam On 5/2/2013 5:20 PM, Frédéric Bastien wrote: Hi, I have no dough you are a great programmer, but even people in my lab that is specialized in deep learning won't be able to do

Re: [Scikit-learn-general] (Deep learning) pre-proposal for (GSOC) 2013

implementation of a DL algorithm I would ensure that its highly usable, bug free and efficient, so the schedule could change based on that. If you have some feedback on the updated version, it would be highly appreciated. Thank you! --Issam On 5/2/2013 8:35 PM, Frédéric Bastien wrote: > I j

Re: [Scikit-learn-general] (Deep learning) pre-proposal for (GSOC) 2013

to follow, here is the updated proposal, https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/issamou/1# Thank you, --Issam On 5/3/2013 4:38 PM, James Bergstra wrote: > "A flag into your hands" as they say, Issam. But do bear in mind that > many of those

Re: [Scikit-learn-general] (Deep learning) pre-proposal for (GSOC) 2013

Hi Andy, The main idea behind proposing to use GPU techniques is to have an efficient implementation, it doesnt need to be GPU techniques, rather any technique such as Numba Pro that speeds up deep learning algorithms which demand a lot of computation. So are you suggesting that there would b

Re: [Scikit-learn-general] (Deep learning) pre-proposal for (GSOC) 2013

:). Best regards, --Issam On 5/4/2013 11:08 AM, Andy Mueller wrote: > Hi Issam. > Please stay on the list ;) > > Sorry I have been so critical of your proposal. > Merging the two PRs is a good proposal. > > I just don't think the "speeding up" is realistic. >

Re: [Scikit-learn-general] Deep Learning Proposal

Hi Andy, You are absolutely right, I'm proposing something which I'm not familiar with. It would be very difficult to learn and integrate it to scikit in a two months time. However, the comments were more than helpful as to what I'm facing :). Thanks a lot! yours truly,

[Scikit-learn-general] Goal: Finalizing Multi-layer perceptron

Hi folks, This is my first time contributing to scikit, so please give me a chance if some of my suggestions are not plausible :) I want to finalize the Multi-Layer Perceptron (MLP) implementation found here: https://github.com/scikit-learn/scikit-learn/pull/1653 I'm thinking of adding two maj

[Scikit-learn-general] installing backprop_sgd.pyx

Hi scikit team, I'm trying to make larsmans MLP code work in my machine, its the pull request found here, https://github.com/scikit-learn/scikit-learn/pull/1395/files It's great that he used cython for the backpropagation step, the speed is expected to be folds faster than if numpy was used. Ho

Re: [Scikit-learn-general] installing backprop_sgd.pyx

Thanks Joel for the help! On 6/23/2013 4:00 AM, Joel Nothman wrote: Hi Issam, Try `make cython inplace` in a terminal, in the directory of your git clone. - Joel On Sun, Jun 23, 2013 at 10:54 AM, Issam <mailto:issamo...@gmail.com>> wrote: Hi scikit team, I'm

[Scikit-learn-general] Adding Sparse Autoencoder to Scikit

Hi Scikit, I'm planning to push a Sparse Autoencoder code I implemented, but since the scikit page says, "To avoid duplicating work, it is highly advised that you contact the developers on the mailing list before starting work on a non-trivial feature" , I decided to ask whether pushing Sparse

Re: [Scikit-learn-general] Adding Sparse Autoencoder to Scikit

, I'm not very familiar with Cython. :(. So, would numpy suffice? or is Cython implementation enforced? Or I could start with numpy and as the review progresses and see that more performance is necessary, then I transition the code to Cython. If thats feasible. Thanks a lo

Re: [Scikit-learn-general] Adding Sparse Autoencoder to Scikit

Very helpful information! Thanks @Olivier! I'll do my best! -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ Scik

Re: [Scikit-learn-general] Adding Sparse Autoencoder to Scikit

Thanks! that does sound very easy. I'll get into Cython soon! I have pushed a draft version of the Sparse Autoencoder to scikit's github. Hopefully I have sent the pull request correctly :). Thanks! On 6/26/2013 2:28 AM, Robert Layton wrote: The basics of cython are, and I'm not kidding her

Re: [Scikit-learn-general] Adding Sparse Autoencoder to Scikit

Hi @Hannes, how about using scipy.optimize.fmin_l_bfgs_b for optimizing the weights? I found it to be very efficient and fast (I even found it to be faster than MATLAB's minFunc), it's also widely used for neural networks type of optimization like in Prof. Andrew's courses and even in Deep Lear

Re: [Scikit-learn-general] Adding Sparse Autoencoder to Scikit

ub.com/scikit-learn/scikit-learn/pull/2099 I will now work on larsmans MLP and work on devising/reusing an SGD that enables efficient partial_fit() Thanks again! ~Issam -- This SF.net email is sponsored by Windows: Bui

Re: [Scikit-learn-general] Adding Sparse Autoencoder to Scikit

It would be interesting to use on-line updating for feature extraction, its main uses would be in cases where data change over time (PCA and autoencoders are very close to each other). It was a poor choice of words when I used 'naturally batch', stochastic updating can definitely be supported,

Re: [Scikit-learn-general] Some concerns about the MLP pull request

On 7/8/2013 12:53 PM, Lars Buitinck wrote: > cost = np.sum(np.einsum('ij,ji->i', diff, diff.T)) / (2 * n_samples) Thanks for all the remarks! I found out that the `einsum` can be replaced simply by 'cost = np.sum(diff**2)/ (2 * n_samples)' which is faster and more readable.

Re: [Scikit-learn-general] Neural Network in Scikit-learn

appreciated! Thanks a lot! ~Issam > Hi Joel, > > Thanks for the cookery instructions. Looking my magit-log, > I can see I have the right branch (if that is > the correct git terminology). I will take up Olivier's advice as well. > > Dominic. > > Commits in HEAD > cd383f

Re: [Scikit-learn-general] Neural Network in Scikit-learn

`neural network` folder is present A subset of the path is something like this 'sklearn\neural_network' ~Issam On 7/11/2013 4:37 PM, Dominic Steinitz wrote: Many thanks for all the help so far. I am running the tests after installation. I get the output below. Should I be worried? Al

Re: [Scikit-learn-general] Neural Network in Scikit-learn

rn 2) cd scikit-learn/ 3) git fetch origin refs/pull/2120/head:mlp 4) git checkout mlp Cheers. ~Issam -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application

[Scikit-learn-general] Using .sgd_fast. SquaredLoss for numpy arrays

Hi, I noticed that .sgd_fast. SquaredLoss works only on scalars as can be done by the code below: `SquaredLoss().loss(12,3)` Is there a way to make it work for arrays? For example, `SquaredLoss().loss(np.array([12,11]), np.array([4,5]))`? It says it only supports scalars Thanks in advance...

Re: [Scikit-learn-general] Using .sgd_fast. SquaredLoss for numpy arrays

Thanks Lars! On 8/4/2013 11:04 AM, Lars Buitinck wrote: > The loss functions in sgd_fast are intended to be used with "true" > SGD, i.e. not the minibatch version. But in the minibatch case, you > can just use NumPy vectorized operations. > ---

[Scikit-learn-general] Getting training score in multi-label classification problems

Hi, Does scikit have an implemented metrics that computes the auc score for multi-label classification? That is if a sample can have one or more associated classes Thanks. -- Get your SQL database under version control

Re: [Scikit-learn-general] Getting training score in multi-label classification problems

@federico haha! thanks for the motivation! @Josh, I'm not aware of polling in Github, but it sounds very convenient, a polling feature would be a great addition to scikit ;) @Gael, I also thought that AUC is not suitable for multi-labels, but if you check the latest kaggle competitions such as

[Scikit-learn-general] Counting the number of comparisons done by nearest neighbor algorithms

exity of approximately O(dlogn) but the complexity could become O(dn) if nothing is pruned. So it would be interesting to empirically observe how likely it is to have that worst case complexity. Cheers! --Issam -- Dive

Re: [Scikit-learn-general] Why don't we support Neural Network Algorithms?

f-the-shelf neural net pull requests in the process of review, notably those by Issam Laradji for GSoC 2014. Extreme Learning Machines and Multilayer Perceptrons should be merged Real Soon Now. On 7 May 2015 at 14:58, Ronnie Ghose wrote: neural nets are already well supported in other p