On 28 March 2013 18:19, Jacob Vanderplas <[email protected]> wrote:
> On Thu, Mar 28, 2013 at 10:10 AM, Lars Buitinck <[email protected]>wrote:
>
>> 2013/3/28 Mathieu Blondel <[email protected]>:
>> > Encoding missing values with np.nan doesn't scale to very
>> > high-dimensional problems with mostly missing values.
>> > Personally, for encoding missing data, I just use sparse matrices.
>> > Values which are actually zero can be stored explicitly in the .data
>> > attribute.
>>
>> +1 for not storing missing values, but will scipy.sparse matrices work
>> correctly when .data has zeros, and will conversion between formats
>> retain them?
>>
>
> It depends on the conversion. Some sparse matrix conversions keep
> explicit zeros, some don't. We dealt with this in scipy.sparse.csgraph:
> there are some utilities there that do the matrix conversions and make sure
> missing entries & zero entries are distinguished correctly. The functions
> are a bit graph-specific, but it might be useful to look at for some ideas.
>
But in general, I don't think we can "force" the user to use sparse
matrices. They are an absolute pain to work with because of the
inconsistencies of interface with ndarray and conversion between sparse and
dense can be time consuming. Hence, I think we need to find something that
works both with dense and sparse matrices.
N
> Jake
>
>
>>
>> --
>> Lars Buitinck
>> Scientific programmer, ILPS
>> University of Amsterdam
>>
>>
>> ------------------------------------------------------------------------------
>> Own the Future-Intel® Level Up Game Demo Contest 2013
>> Rise to greatness in Intel's independent game demo contest.
>> Compete for recognition, cash, and the chance to get your game
>> on Steam. $5K grand prize plus 10 genre and skill prizes.
>> Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d
>> _______________________________________________
>> Scikit-learn-general mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>
>
>
> ------------------------------------------------------------------------------
> Own the Future-Intel® Level Up Game Demo Contest 2013
> Rise to greatness in Intel's independent game demo contest.
> Compete for recognition, cash, and the chance to get your game
> on Steam. $5K grand prize plus 10 genre and skill prizes.
> Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
Own the Future-Intel® Level Up Game Demo Contest 2013
Rise to greatness in Intel's independent game demo contest.
Compete for recognition, cash, and the chance to get your game
on Steam. $5K grand prize plus 10 genre and skill prizes.
Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
