Hm, if you set n_jobs>1, then I think it’s using multiprocessing, which will pass a copy of the input data to each process. That could be one reason for the relatively large memory consumption.
> On Mar 16, 2017, at 12:46 AM, Carlton Banks <nofl...@gmail.com> wrote: > > The ndarray (6,3,3) => (row, col,color channels) > > I tried fixing it converting the list of numpy.ndarray to numpy.asarray(list) > > but this causes a different problem: > > is grid use a lot a memory.. I am running on a super computer, and seem to > have problems with memory.. already used 62 gb ram.. > >> Den 16. mar. 2017 kl. 05.30 skrev Sebastian Raschka <se.rasc...@gmail.com>: >> >> Sklearn estimators typically assume 2d inputs (as numpy arrays) with >> shape=[n_samples, n_features]. >> >>> list of Np.ndarrays of shape (6,3,3) >> >> I assume you mean a 3D tensor (3D numpy array) with shape=[n_samples, >> n_pixels, n_pixels]? What you could do is to reshape it before you put it >> in, i.e., >> >> data_ary = your_ary.reshape(n_samples, -1).shape >> >> then, you need to add a line at the beginning your CNN class that does the >> reverse, i.e., data_ary.reshape(6, n_pixels, n_pixels).shape. Numpy’s >> reshape typically returns view objects, so that these additional steps >> shouldn’t be “too” expensive. >> >> Best, >> Sebastian >> >> >> >>> On Mar 16, 2017, at 12:00 AM, Carlton Banks <nofl...@gmail.com> wrote: >>> >>> Hi… >>> >>> I currently trying to optimize my CNN model using gridsearchCV, but seem to >>> have some problems feading my input data.. >>> >>> My training data is stored as a list of Np.ndarrays of shape (6,3,3) and my >>> output is stored as a list of np.array with one entry. >>> >>> Why am I having problems parsing my data to it? >>> >>> best regards >>> Carl B. >>> _______________________________________________ >>> scikit-learn mailing list >>> scikit-learn@python.org >>> https://mail.python.org/mailman/listinfo/scikit-learn >> >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn@python.org >> https://mail.python.org/mailman/listinfo/scikit-learn > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
