Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-21 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/
---

(Updated March 21, 2016, 10:48 a.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Bugs: AMBARI-15482
https://issues.apache.org/jira/browse/AMBARI-15482


Repository: ambari


Description
---

During an upgrade, the HDFS NameNode(s) are restarted with the 
{{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
upgrade and decide to "Finalize Later".

This allows the cluster to run in the upgraded state before committing to the 
upgrade. Full cluster control is returned via the Ambari web client.

Administrators can then decide to restart a NameNode. Upon restarting the 
NameNode, it will produce an error that it was not started with the 
{{rollingUpgrade}} flag. 

It seems that as long as an upgrade has not been finalized, the NameNode(s) 
must be started with the {{rollingUpgrade}} to allow them to function properly. 

STR:
- Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
there is a major version change.
- Before finalization, say "Finalize Later". All services should be up and 
green.
- Now restart a NameNode


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 9ea541e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 4ef215c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 303f3a4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 07061e1 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
 3f1a52b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
ed3c772 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 1c7ff61 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 1e69db7 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql e01e693 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql a07c6fc 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b2b450a 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql cec122e 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
96fc720 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql c425d6f 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2a89e26 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
 e4c8c9c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 02905ec 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
 a154b73 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 905802f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 17c52d2 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 2dceccc 
  ambari-web/app/utils/ajax/ajax.js 29d0715 

Diff: https://reviews.apache.org/r/45035/diff/


Testing (updated)
---

Tests run: 3968, Failures: 0, Errors: 0, Skipped: 33

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 32:16 min
[INFO] Finished at: 2016-03-18T13:41:19-04:00
[INFO] Final Memory: 37M/701M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-21 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/
---

(Updated March 21, 2016, 10:48 a.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Changes
---

Updating after merge conflicts resolved.


Bugs: AMBARI-15482
https://issues.apache.org/jira/browse/AMBARI-15482


Repository: ambari


Description
---

During an upgrade, the HDFS NameNode(s) are restarted with the 
{{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
upgrade and decide to "Finalize Later".

This allows the cluster to run in the upgraded state before committing to the 
upgrade. Full cluster control is returned via the Ambari web client.

Administrators can then decide to restart a NameNode. Upon restarting the 
NameNode, it will produce an error that it was not started with the 
{{rollingUpgrade}} flag. 

It seems that as long as an upgrade has not been finalized, the NameNode(s) 
must be started with the {{rollingUpgrade}} to allow them to function properly. 

STR:
- Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
there is a major version change.
- Before finalization, say "Finalize Later". All services should be up and 
green.
- Now restart a NameNode


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 9ea541e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 4ef215c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 303f3a4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 07061e1 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
 3f1a52b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
ed3c772 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 1c7ff61 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 1e69db7 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql e01e693 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql a07c6fc 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b2b450a 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql cec122e 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
96fc720 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql c425d6f 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2a89e26 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
 e4c8c9c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 02905ec 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
 a154b73 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 905802f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 17c52d2 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 2dceccc 
  ambari-web/app/utils/ajax/ajax.js 29d0715 

Diff: https://reviews.apache.org/r/45035/diff/


Testing
---

Pending...


Thanks,

Jonathan Hurley

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-20 Thread Alejandro Fernandez 


> On March 18, 2016, 5:50 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java,
> >  line 453
> > 
> >
> > I think adding a new column is overkill. If EU is suspended while 
> > stopping services, then HDFS has not yet been prepared or restarted with 
> > the -rollingUpgrade option.
> > 
> > I wonder if there's a way to detect on the python side that NameNode is 
> > in the middle of an upgrade and hence should be restarted with that flag.
> > Could even use a marker file that gets deleted after Finalize.
> 
> Jonathan Hurley wrote:
> I don't think it is; it's useful for querying and allows us to deliver 
> this state information via commands to any service that needs it; not just 
> HDFS. 
> 
> The marker file is the hack approach - easily can cause problems.
> 
> Jonathan Hurley wrote:
> Hmmm - I see what you're saying though. What if HDFS hasn't even started 
> yet. And then they pause the upgrade and try to restart NameNode. OK, so how 
> about a combination then; we can use a marker file, but we also use the 
> suspended nature of the upgrade in combination with it?

I would say just use the marker file, no need for any java changes.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/#review124232
---


On March 18, 2016, 8:56 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45035/
> ---
> 
> (Updated March 18, 2016, 8:56 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-15482
> https://issues.apache.org/jira/browse/AMBARI-15482
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade, the HDFS NameNode(s) are restarted with the 
> {{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
> upgrade and decide to "Finalize Later".
> 
> This allows the cluster to run in the upgraded state before committing to the 
> upgrade. Full cluster control is returned via the Ambari web client.
> 
> Administrators can then decide to restart a NameNode. Upon restarting the 
> NameNode, it will produce an error that it was not started with the 
> {{rollingUpgrade}} flag. 
> 
> It seems that as long as an upgrade has not been finalized, the NameNode(s) 
> must be started with the {{rollingUpgrade}} to allow them to function 
> properly. 
> 
> STR:
> - Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
> there is a major version change.
> - Before finalization, say "Finalize Later". All services should be up and 
> green.
> - Now restart a NameNode
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  9ea541e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
>  4ef215c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  303f3a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  07061e1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  3f1a52b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ed3c772 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  1c7ff61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql e01e693 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql fb24c89 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 2e092b2 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 5f5df3c 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 3d71f1f 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 9d70fbb 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 715000e 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
>  e4c8c9c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  02905ec 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
>  a154b73 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  905802f 
>   
>

Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-19 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/
---

Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Bugs: AMBARI-15482
https://issues.apache.org/jira/browse/AMBARI-15482


Repository: ambari


Description
---

During an upgrade, the HDFS NameNode(s) are restarted with the 
{{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
upgrade and decide to "Finalize Later".

This allows the cluster to run in the upgraded state before committing to the 
upgrade. Full cluster control is returned via the Ambari web client.

Administrators can then decide to restart a NameNode. Upon restarting the 
NameNode, it will produce an error that it was not started with the 
{{rollingUpgrade}} flag. 

It seems that as long as an upgrade has not been finalized, the NameNode(s) 
must be started with the {{rollingUpgrade}} to allow them to function properly. 

STR:
- Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
there is a major version change.
- Before finalization, say "Finalize Later". All services should be up and 
green.
- Now restart a NameNode


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 9ea541e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 4ef215c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 303f3a4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 07061e1 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
 3f1a52b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
ed3c772 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 1c7ff61 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 84bb9f3 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
ab6dc93 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
 e4c8c9c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 02905ec 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 905802f 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 2dceccc 
  ambari-web/app/utils/ajax/ajax.js 29d0715 

Diff: https://reviews.apache.org/r/45035/diff/


Testing
---

Pending...


Thanks,

Jonathan Hurley

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-19 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/
---

(Updated March 18, 2016, 4:56 p.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Changes
---

Using a marker file in addition to the upgrade state to figure out how to 
handle NN restart.


Bugs: AMBARI-15482
https://issues.apache.org/jira/browse/AMBARI-15482


Repository: ambari


Description
---

During an upgrade, the HDFS NameNode(s) are restarted with the 
{{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
upgrade and decide to "Finalize Later".

This allows the cluster to run in the upgraded state before committing to the 
upgrade. Full cluster control is returned via the Ambari web client.

Administrators can then decide to restart a NameNode. Upon restarting the 
NameNode, it will produce an error that it was not started with the 
{{rollingUpgrade}} flag. 

It seems that as long as an upgrade has not been finalized, the NameNode(s) 
must be started with the {{rollingUpgrade}} to allow them to function properly. 

STR:
- Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
there is a major version change.
- Before finalization, say "Finalize Later". All services should be up and 
green.
- Now restart a NameNode


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 9ea541e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 4ef215c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 303f3a4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 07061e1 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
 3f1a52b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
ed3c772 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 1c7ff61 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 84bb9f3 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql e01e693 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql fb24c89 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 2e092b2 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 5f5df3c 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
3d71f1f 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 9d70fbb 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 715000e 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
 e4c8c9c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 02905ec 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
 a154b73 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 905802f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 17c52d2 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 2dceccc 
  ambari-web/app/utils/ajax/ajax.js 29d0715 

Diff: https://reviews.apache.org/r/45035/diff/


Testing
---

Pending...


Thanks,

Jonathan Hurley

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-19 Thread Jonathan Hurley 


> On March 18, 2016, 1:50 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java,
> >  line 453
> > 
> >
> > I think adding a new column is overkill. If EU is suspended while 
> > stopping services, then HDFS has not yet been prepared or restarted with 
> > the -rollingUpgrade option.
> > 
> > I wonder if there's a way to detect on the python side that NameNode is 
> > in the middle of an upgrade and hence should be restarted with that flag.
> > Could even use a marker file that gets deleted after Finalize.

I don't think it is; it's useful for querying and allows us to deliver this 
state information via commands to any service that needs it; not just HDFS. 

The marker file is the hack approach - easily can cause problems.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/#review124232
---


On March 18, 2016, 11:20 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45035/
> ---
> 
> (Updated March 18, 2016, 11:20 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-15482
> https://issues.apache.org/jira/browse/AMBARI-15482
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade, the HDFS NameNode(s) are restarted with the 
> {{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
> upgrade and decide to "Finalize Later".
> 
> This allows the cluster to run in the upgraded state before committing to the 
> upgrade. Full cluster control is returned via the Ambari web client.
> 
> Administrators can then decide to restart a NameNode. Upon restarting the 
> NameNode, it will produce an error that it was not started with the 
> {{rollingUpgrade}} flag. 
> 
> It seems that as long as an upgrade has not been finalized, the NameNode(s) 
> must be started with the {{rollingUpgrade}} to allow them to function 
> properly. 
> 
> STR:
> - Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
> there is a major version change.
> - Before finalization, say "Finalize Later". All services should be up and 
> green.
> - Now restart a NameNode
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  9ea541e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
>  4ef215c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  303f3a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  07061e1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  3f1a52b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ed3c772 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  1c7ff61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> ab6dc93 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
>  e4c8c9c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  02905ec 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  905802f 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> 2dceccc 
>   ambari-web/app/utils/ajax/ajax.js 29d0715 
> 
> Diff: https://reviews.apache.org/r/45035/diff/
> 
> 
> Testing
> ---
> 
> Pending...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/#review124232
---




ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 (line 441)


I think adding a new column is overkill. If EU is suspended while stopping 
services, then HDFS has not yet been prepared or restarted with the 
-rollingUpgrade option.

I wonder if there's a way to detect on the python side that NameNode is in 
the middle of an upgrade and hence should be restarted with that flag.
Could even use a marker file that gets deleted after Finalize.


- Alejandro Fernandez


On March 18, 2016, 3:20 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45035/
> ---
> 
> (Updated March 18, 2016, 3:20 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-15482
> https://issues.apache.org/jira/browse/AMBARI-15482
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade, the HDFS NameNode(s) are restarted with the 
> {{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
> upgrade and decide to "Finalize Later".
> 
> This allows the cluster to run in the upgraded state before committing to the 
> upgrade. Full cluster control is returned via the Ambari web client.
> 
> Administrators can then decide to restart a NameNode. Upon restarting the 
> NameNode, it will produce an error that it was not started with the 
> {{rollingUpgrade}} flag. 
> 
> It seems that as long as an upgrade has not been finalized, the NameNode(s) 
> must be started with the {{rollingUpgrade}} to allow them to function 
> properly. 
> 
> STR:
> - Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
> there is a major version change.
> - Before finalization, say "Finalize Later". All services should be up and 
> green.
> - Now restart a NameNode
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  9ea541e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
>  4ef215c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  303f3a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  07061e1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  3f1a52b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ed3c772 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  1c7ff61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> ab6dc93 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
>  e4c8c9c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  02905ec 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  905802f 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> 2dceccc 
>   ambari-web/app/utils/ajax/ajax.js 29d0715 
> 
> Diff: https://reviews.apache.org/r/45035/diff/
> 
> 
> Testing
> ---
> 
> Pending...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-18 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/#review124217
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 (line 461)


formatting :)


- Nate Cole


On March 18, 2016, 11:20 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45035/
> ---
> 
> (Updated March 18, 2016, 11:20 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-15482
> https://issues.apache.org/jira/browse/AMBARI-15482
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade, the HDFS NameNode(s) are restarted with the 
> {{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
> upgrade and decide to "Finalize Later".
> 
> This allows the cluster to run in the upgraded state before committing to the 
> upgrade. Full cluster control is returned via the Ambari web client.
> 
> Administrators can then decide to restart a NameNode. Upon restarting the 
> NameNode, it will produce an error that it was not started with the 
> {{rollingUpgrade}} flag. 
> 
> It seems that as long as an upgrade has not been finalized, the NameNode(s) 
> must be started with the {{rollingUpgrade}} to allow them to function 
> properly. 
> 
> STR:
> - Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
> there is a major version change.
> - Before finalization, say "Finalize Later". All services should be up and 
> green.
> - Now restart a NameNode
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  9ea541e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
>  4ef215c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  303f3a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  07061e1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  3f1a52b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ed3c772 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  1c7ff61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> ab6dc93 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
>  e4c8c9c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  02905ec 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  905802f 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> 2dceccc 
>   ambari-web/app/utils/ajax/ajax.js 29d0715 
> 
> Diff: https://reviews.apache.org/r/45035/diff/
> 
> 
> Testing
> ---
> 
> Pending...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>