What would be the advantage for using a shared vocabulary for Count
Vectorizer??
When I read about FeatureUnion, what I understood was that, the given list
of transformers would process the given data set completely. Could we use
it to selectively process different features?? Or is my understanding of
the concept not clear??
Regards,
Abijith
On Sat, Jun 21, 2014 at 7:12 PM, Andy <[email protected]> wrote:
> Yes, you can use CountVectorizer.
> Do you want the different features to share the same vocabulary?
> To use the Count Vectorizer, you probably have to either get all the
> values (for a shared vocabulary)
> or learn one CountVectorizer per key (you could use FeatureUnion for that).
>
> So there is a little bit of code to write to handle the fact that you have
> multiple text fields.
>
> Hth,
> Andy
>
>
>
> On 06/21/2014 03:35 PM, Abijith Kp wrote:
>
> Hi,
>
>
> Initially, one of my feature list looks like: {"a":"3", "b":"random1",
> "c":"", "d":"random2 text"}.
> The random text contains names of people, email ids, some description,
> numbers and goes on.
>
> When I used DictVectorizer, I could not get an accurate clustering.
>
> I wanted know if I could get any method similar to DictVectorizer, which
> could process a dictionary of string features, correctly.
>
> Regards,
> Abijith
>
>
> On Sat, Jun 21, 2014 at 6:51 PM, Andy <[email protected]> wrote:
>
>> Hi Abijith.
>>
>> It depends on how you want to interpret the strings.
>> If they are texts and you want to interpret them based on their content,
>> Brians suggestion is the right one.
>> If you want to consider each possible string as a distinct feature, the
>> OneHotEncoder would be the right choice.
>>
>> Could you give an example of what the strings and the semantics of the
>> strings are?
>>
>> Andy
>>
>>
>>
>>
>> On 06/20/2014 06:05 PM, Abijith Kp wrote:
>>
>> Can anyone help me with the problem of dealing with feature which are
>> both strings of varying length(say from 0 to 100-150 characters) and
>> numbers?
>>
>> What will be the most widely used techniques in such kind of situations?
>> And can it be solved using only scikit-learn?
>>
>> PS: Initially I have to convert a json file to a feature's list, and
>> then use it.
>>
>> Any help is appreciated.
>>
>> Regards,
>> Abijith
>>
>> --
>> Abijith KP
>> github.com/abijith-kp
>> kpabijith.wordpress.com
>>
>>
>>
>> ------------------------------------------------------------------------------
>> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
>> Find What Matters Most in Your Big Data with HPCC Systems
>> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
>> Leverages Graph Analysis for Fast Processing & Easy Data
>> Explorationhttp://p.sf.net/sfu/hpccsystems
>>
>>
>>
>> _______________________________________________
>> Scikit-learn-general mailing
>> [email protected]https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
>> Find What Matters Most in Your Big Data with HPCC Systems
>> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
>> Leverages Graph Analysis for Fast Processing & Easy Data Exploration
>> http://p.sf.net/sfu/hpccsystems
>> _______________________________________________
>> Scikit-learn-general mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>
>
> --
> Abijith KP
> github.com/abijith-kp
> kpabijith.wordpress.com
>
>
> ------------------------------------------------------------------------------
> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
> Find What Matters Most in Your Big Data with HPCC Systems
> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
> Leverages Graph Analysis for Fast Processing & Easy Data
> Explorationhttp://p.sf.net/sfu/hpccsystems
>
>
>
> _______________________________________________
> Scikit-learn-general mailing
> [email protected]https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
>
> ------------------------------------------------------------------------------
> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
> Find What Matters Most in Your Big Data with HPCC Systems
> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
> Leverages Graph Analysis for Fast Processing & Easy Data Exploration
> http://p.sf.net/sfu/hpccsystems
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
--
Abijith KP
github.com/abijith-kp
kpabijith.wordpress.com
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
