Re: [Scikit-learn-general] Problem with parallel processing in randomSearch

2016-02-24 Thread Stylianos Kampakis 
Could be, but I don't think so in this particular case. The dataset is like
a few thousand rows and around 20 features.

2016-02-24 18:51 GMT+00:00 Sebastian Raschka :

> I am not using XGBoost and honestly don't much about the way its
> implemented. However, have you checked if memory swapping could be an issue
> if you are doing multiprocessing? Maybe it's using so much RAM due to
> creating multiple copies that it just feels like it stopped? Just a thought
> :P
>
> On Feb 24, 2016, at 11:42 AM, Stylianos Kampakis <
> stylianos.kampa...@gmail.com> wrote:
>
> Hi Joel,
>
> I ran lots of tests today and I realized that it is XGBRegressor (a
> wrapper for XGboost) that is guilty. However, it doesn't always stop. It
> does only some times. Any ideas why this might be happening?
>
> 2016-02-24 0:38 GMT+00:00 Joel Nothman :
>
>> What estimator(s) are you searching over? How big is your data?
>>
>> On 24 February 2016 at 06:15, Stylianos Kampakis <
>> stylianos.kampa...@gmail.com> wrote:
>>
>>> Hi everyone,
>>>
>>> Sometimes, when I am using random search with n_jobs>1 the processing
>>> stops. I am on a Mac. I went through some discussions on Github where
>>> people said it relates joblib and this problem is more common on Mac.
>>> However, I couldn't find the answer to two questions I have:
>>>
>>> 1) Why the processing stops only some times and not every single time?
>>>
>>> 2) Have any people managed to find a workaround?
>>>
>>> Thank you all in advance,
>>>
>>>
>>> --
>>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>>> Monitor end-to-end web transactions and take corrective actions now
>>> Troubleshoot faster and improve end-user experience. Signup Now!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
>>> ___
>>> Scikit-learn-general mailing list
>>> Scikit-learn-general@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>>
>>>
>>
>>
>> --
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
>> ___
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
>
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
> ___
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general

Re: [Scikit-learn-general] Problem with parallel processing in randomSearch

2016-02-24 Thread Sebastian Raschka 
I am not using XGBoost and honestly don't much about the way its implemented. 
However, have you checked if memory swapping could be an issue if you are doing 
multiprocessing? Maybe it's using so much RAM due to creating multiple copies 
that it just feels like it stopped? Just a thought :P

> On Feb 24, 2016, at 11:42 AM, Stylianos Kampakis 
>  wrote:
> 
> Hi Joel,
> 
> I ran lots of tests today and I realized that it is XGBRegressor (a wrapper 
> for XGboost) that is guilty. However, it doesn't always stop. It does only 
> some times. Any ideas why this might be happening?
> 
> 2016-02-24 0:38 GMT+00:00 Joel Nothman  >:
> What estimator(s) are you searching over? How big is your data?
> 
> On 24 February 2016 at 06:15, Stylianos Kampakis 
> > wrote:
> Hi everyone,
> 
> Sometimes, when I am using random search with n_jobs>1 the processing stops. 
> I am on a Mac. I went through some discussions on Github where people said it 
> relates joblib and this problem is more common on Mac. However, I couldn't 
> find the answer to two questions I have:
> 
> 1) Why the processing stops only some times and not every single time?
> 
> 2) Have any people managed to find a workaround?
> 
> Thank you all in advance,
> 
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140 
> 
> ___
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net 
> 
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general 
> 
> 
> 
> 
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140 
> 
> ___
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net 
> 
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general 
> 
> 
> 
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general

Re: [Scikit-learn-general] Problem with parallel processing in randomSearch

2016-02-24 Thread Stylianos Kampakis 
Hi Joel,

I ran lots of tests today and I realized that it is XGBRegressor (a wrapper
for XGboost) that is guilty. However, it doesn't always stop. It does only
some times. Any ideas why this might be happening?

2016-02-24 0:38 GMT+00:00 Joel Nothman :

> What estimator(s) are you searching over? How big is your data?
>
> On 24 February 2016 at 06:15, Stylianos Kampakis <
> stylianos.kampa...@gmail.com> wrote:
>
>> Hi everyone,
>>
>> Sometimes, when I am using random search with n_jobs>1 the processing
>> stops. I am on a Mac. I went through some discussions on Github where
>> people said it relates joblib and this problem is more common on Mac.
>> However, I couldn't find the answer to two questions I have:
>>
>> 1) Why the processing stops only some times and not every single time?
>>
>> 2) Have any people managed to find a workaround?
>>
>> Thank you all in advance,
>>
>>
>> --
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
>> ___
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
> ___
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general

Re: [Scikit-learn-general] Problem with parallel processing in randomSearch

2016-02-23 Thread Joel Nothman 
What estimator(s) are you searching over? How big is your data?

On 24 February 2016 at 06:15, Stylianos Kampakis <
stylianos.kampa...@gmail.com> wrote:

> Hi everyone,
>
> Sometimes, when I am using random search with n_jobs>1 the processing
> stops. I am on a Mac. I went through some discussions on Github where
> people said it relates joblib and this problem is more common on Mac.
> However, I couldn't find the answer to two questions I have:
>
> 1) Why the processing stops only some times and not every single time?
>
> 2) Have any people managed to find a workaround?
>
> Thank you all in advance,
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
> ___
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140___
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general