Hi Sheila,
The purpose of my comment was to help you fix your experimental setup, not
improve accuracy.
In fact, scaling the entire data before splitting is expected to work
better, but this is cheating.
In your case, you can just give up scaling your data, since the "natural"
scale works better.
Mathieu
On Thu, Jul 10, 2014 at 5:33 PM, Sheila the angel <from.d.pu...@gmail.com>
wrote:
> I have changed the code, still I don't see much difference.
> The non-scaled data-set is giving more accuracy then scaled.
> Should I apply dimension selection first?
> And what are the easy methods to start with?
>
> Thanks
> --
> Sheila
>
> On 8 July 2014 17:02, Mathieu Blondel <math...@mblondel.org> wrote:
>
>>
>>
>>
>> On Tue, Jul 8, 2014 at 11:27 PM, Sheila the angel <from.d.pu...@gmail.com
>> > wrote:
>>
>>> First I scaled the complete data-set and then splitting it in test and
>>> train data.
>>>
>>
>> You should not pre-process the data before splitting it. Just ask
>> yourself how you would use your model in practice. In a real-world setting,
>> you wouldn't have access to test data (unseen data) ahead of time. This
>> will also lead to overly optimistic accuracy results. You should use the
>> usual transformer API: fit_transform method to scale training data and
>> transform method to scale new data.
>>
>> Mathieu
>>
>>
>> ------------------------------------------------------------------------------
>> Open source business process management suite built on Java and Eclipse
>> Turn processes into business applications with Bonita BPM Community
>> Edition
>> Quickly connect people, data, and systems into organized workflows
>> Winner of BOSSIE, CODIE, OW2 and Gartner awards
>> http://p.sf.net/sfu/Bonitasoft
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>
------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
