Re: Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-06 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136291
---


Ship it!




Ship It!

- Sumit Mohanty


On June 6, 2016, 4:29 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 6, 2016, 4:29 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-06 Thread Andrew Onischuk 


> On June 6, 2016, 4:21 p.m., Sumit Mohanty wrote:
> > ambari-agent/conf/unix/ambari-agent.ini, line 37
> > 
> >
> > Whats the default behavior - looks like by default it should be 
> > disabled.
> > 
> > What is the memory consumption for an agent instance that is not 
> > leaking - assuming 20+ master/slave components on the host.

done


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136283
---


On June 6, 2016, 4:29 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 6, 2016, 4:29 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-06 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/
---

(Updated June 6, 2016, 4:29 p.m.)


Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17030
https://issues.apache.org/jira/browse/AMBARI-17030


Repository: ambari


Description
---

Add an agent capability to auto-restart when the memory threshold for the
agent process exceeds a limit. This should be a configurable property (e.g.
agent_memory_threshold) where any positive value, say in MB, is a limit at
which agent should get restarted.

The restart itself should be executed when there are no
EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
reached for agent auto restart, a flag should be set to restart when agent is
not doing any work.

A hard limit should also be set where the agent will be restarted even if
there are commands in pipeline.


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
  ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
  ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 

Diff: https://reviews.apache.org/r/48207/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-06 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136283
---




ambari-agent/conf/unix/ambari-agent.ini (line 37)


Whats the default behavior - looks like by default it should be disabled.

What is the memory consumption for an agent instance that is not leaking - 
assuming 20+ master/slave components on the host.


- Sumit Mohanty


On June 6, 2016, 4:13 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 6, 2016, 4:13 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-06 Thread Andrew Onischuk 


> On June 3, 2016, 8:15 p.m., Alejandro Fernandez wrote:
> > ambari-agent/src/main/python/ambari_agent/Controller.py, line 290
> > 
> >
> > Restarting is dangerous to do, especially if it happens during 
> > kerberization or RU/EU. If there's a memory leak, then that's the problem 
> > we should be fixing.
> 
> Sumit Mohanty wrote:
> Agree. Andrew can you open an JIRA and add details of investigations 
> going on for memory leak

Opened an apache jira https://issues.apache.org/jira/browse/AMBARI-17065 to 
track the actual issue


The concern about RU/EU/kerberos should not be a problem because in this patch 
we do restart only if there are no commands queued on the agent.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136102
---


On June 6, 2016, 4:13 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 6, 2016, 4:13 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-06 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/
---

(Updated June 6, 2016, 4:13 p.m.)


Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17030
https://issues.apache.org/jira/browse/AMBARI-17030


Repository: ambari


Description
---

Add an agent capability to auto-restart when the memory threshold for the
agent process exceeds a limit. This should be a configurable property (e.g.
agent_memory_threshold) where any positive value, say in MB, is a limit at
which agent should get restarted.

The restart itself should be executed when there are no
EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
reached for agent auto restart, a flag should be set to restart when agent is
not doing any work.

A hard limit should also be set where the agent will be restarted even if
there are commands in pipeline.


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
  ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
  ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 

Diff: https://reviews.apache.org/r/48207/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-06 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/
---

(Updated June 6, 2016, 4:04 p.m.)


Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17030
https://issues.apache.org/jira/browse/AMBARI-17030


Repository: ambari


Description
---

Add an agent capability to auto-restart when the memory threshold for the
agent process exceeds a limit. This should be a configurable property (e.g.
agent_memory_threshold) where any positive value, say in MB, is a limit at
which agent should get restarted.

The restart itself should be executed when there are no
EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
reached for agent auto restart, a flag should be set to restart when agent is
not doing any work.

A hard limit should also be set where the agent will be restarted even if
there are commands in pipeline.


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
  ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
  ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 

Diff: https://reviews.apache.org/r/48207/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-03 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136065
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On June 3, 2016, 5:38 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 3, 2016, 5:38 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 48207: Agents should automatically restart if there is a memory leak

2016-06-03 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17030
https://issues.apache.org/jira/browse/AMBARI-17030


Repository: ambari


Description
---

Add an agent capability to auto-restart when the memory threshold for the
agent process exceeds a limit. This should be a configurable property (e.g.
agent_memory_threshold) where any positive value, say in MB, is a limit at
which agent should get restarted.

The restart itself should be executed when there are no
EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
reached for agent auto restart, a flag should be set to restart when agent is
not doing any work.

A hard limit should also be set where the agent will be restarted even if
there are commands in pipeline.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
  ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 

Diff: https://reviews.apache.org/r/48207/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk