On 08/04/2015 01:49 PM, Maria Gorinova wrote:
On 4 August 2015 at 18:25, Ronnie Ghose <ronnie.gh...@gmail.com
<mailto:ronnie.gh...@gmail.com>> wrote:
are you able to make a np.ones stand alone of that size?
Yes, I can create a np.ones array of size 100 000 000 approximatelly.
On 4 August 2015 at 18:26, Andreas Mueller <t3k...@gmail.com
<mailto:t3k...@gmail.com>> wrote:
That array would take about 700mb of ram. Do you have that much
available?
Btw, you could work around this issue probably by using
HashingVectorizer instead of CountVectorizer.
Yes, I've got plenty of memory, even that Windows limits single
processes to 2GB. I've tried HashingVectorizer before and it gives me
another numpy memory error. Here is the trace:
I would imagine the text takes up a lot of the memory, and there is not
enough left of the 2gb.
------------------------------------------------------------------------------
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
