subject:"\[DISCUSS\] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0"

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

2018-02-14 Thread Rene Moser

On 02/14/2018 06:21 PM, Daan Hoogland wrote:
> the -x would only add it to the comment making it harder to find. As for
> multiple stable branches; merging forward always folows all branches
> forward so a fix on 4.9 would be merged forward to 4.10 and then 4.10 would
> be merged forward again to 4.11 and finally to master. of course there is
> always work to do in terms of solving merge conflicts but these are
> generally less then port work as the order of any commits to the
> intemediats is always preserved.

I don't say this workflow is "bad" or does not work "technically".

To me, it looks "hard" to make a decision to which branch should a fix
go in the first place. And in this workflow, you basically have to
decided it _before_ the merge: To 4.11? or even 4.10? And if I should
have merged to 4.10 but merged it to 4.11, now what?

In contrast of the cherry-pick workflow: you decide _after_ to which
versions the fix should be backported to.

To me, this seems much convenient. But can live with that.

René

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

2018-02-14 Thread Daan Hoogland

the -x would only add it to the comment making it harder to find. As for
multiple stable branches; merging forward always folows all branches
forward so a fix on 4.9 would be merged forward to 4.10 and then 4.10 would
be merged forward again to 4.11 and finally to master. of course there is
always work to do in terms of solving merge conflicts but these are
generally less then port work as the order of any commits to the
intemediats is always preserved.

On Wed, Feb 14, 2018 at 6:15 PM, Rene Moser  wrote:

> Hi Daan
>
> On 02/14/2018 05:26 PM, Daan Hoogland wrote:
> > Rene,
> >
> > The issue is certainly not due the git workflow but to upgrade schemes we
> > have.
> >
> > The result of this workflow for us is that it is easier to find to which
> > branches a particular commit is added as by merging forward the commit id
> > of the actual fix doesn't change. so instead of looking in each branch
> for
> > a bit of code you can look for a commit id on a branches log.
>
> Ah I see.
>
> However, the same can be achieved by adding -x to cherry-picks (to add
> the origin commit id), without the downside that a fix can "only" go
> into one stable branch.
>
> Keep in mind, we certainly do have more than one stable branch at a time
> (4.11-lts, 4.12). A fix should be applicable to any stable branch.
>
> Or how would this work with the current workflow?
>
> René
>
>
>
>
>
>

-- 
Daan

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

2018-02-14 Thread Rene Moser

Hi Daan

On 02/14/2018 05:26 PM, Daan Hoogland wrote:
> Rene,
> 
> The issue is certainly not due the git workflow but to upgrade schemes we
> have.
> 
> The result of this workflow for us is that it is easier to find to which
> branches a particular commit is added as by merging forward the commit id
> of the actual fix doesn't change. so instead of looking in each branch for
> a bit of code you can look for a commit id on a branches log.

Ah I see.

However, the same can be achieved by adding -x to cherry-picks (to add
the origin commit id), without the downside that a fix can "only" go
into one stable branch.

Keep in mind, we certainly do have more than one stable branch at a time
(4.11-lts, 4.12). A fix should be applicable to any stable branch.

Or how would this work with the current workflow?

René

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

2018-02-14 Thread Daan Hoogland

Rene,

The issue is certainly not due the git workflow but to upgrade schemes we
have.

The result of this workflow for us is that it is easier to find to which
branches a particular commit is added as by merging forward the commit id
of the actual fix doesn't change. so instead of looking in each branch for
a bit of code you can look for a commit id on a branches log.


On Wed, Feb 14, 2018 at 4:54 PM, Rene Moser  wrote:

> Hi all
>
> Thanks for taking care of this issue.
>
> However, I wonder how we can prevent such issues in the future and
> wondering if this "incident" has its orginos by our current git workflow.
>
> I find the workflow "commit to stable branch --> merge back to master"
> is inconvenient. Because at one point, the branches divert too much
> (think of refactorings on master branch) and a merge back to the master
> would not work or would break.
>
> In most of the project I work with, there are stable-branches as well
> but any fix (and feature of course) has to be on master first. Any
> serious fix could be cherry picked to stable branches (there may be more
> than one) if applicable or an alternative fix could be made. stable
> branches never receives merges from or to master branch.
>
> To be honest, I haven't seen any project adopting this workflow, I
> wondered why and how come we have this workflow.
>
> Thanks for clarification.
>
> René
>
>
>
> On 02/12/2018 02:29 PM, Rohit Yadav wrote:
> > All,
> >
> >
> > Some of us have discussed and found an upgrade path issue that only
> affects the 4.10.0.0 users who may see missing columns in certain tables
> post upgrading to 4.11.0.0 version.
> >
> >
> > The issue is/was that at the time 4.10.0.0 was released, PRs were merged
> to the 'then' master branch that made changes to the 4.9.3.0->4.10.0.0
> upgrade path instead of the 4.10.0.0->4.11.0.0 upgrade path. One of such
> change was an ALTER statement that added a new column `service_package_id`,
> and if 4.10.0.0 version is upgraded to 4.11.0.0 such environments may
> report sql related errors related to this column.
> >
> >
> > (My colleagues Bobby and Ernie may chip in their findings and test
> results as well)
> >
> >
> > Pull request: https://github.com/apache/cloudstack/pull/2452
> >
> > Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-10285
> >
> >
> > The proposed workaround is to move the changes to the expected upgrade
> path of 4.10.0.0->4.11.0.0. This is *not ideal* given 4.11.0.0 is released
> but at least 4.10.0.0 users who may want to upgrade to 4.11.1.0 or later
> won't face the issue.
> >
> >
> > The proposed workaround for current 4.10.0.0 users who may want to
> upgrade to 4.11.0.0 is to run the following sql statements from the PR
> above before upgrading to 4.11.0.0: (we may discuss and update the
> release notes website as well)
> >
> >
> > ### start sql ###
> >
> >
> > use cloud;
> >
> >
> > CREATE TABLE IF NOT EXISTS `cloud`.`netscaler_servicepackages` (
> >   `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
> >   `uuid` varchar(255) UNIQUE,
> >   `name` varchar(255) UNIQUE COMMENT 'name of the service package',
> >   `description` varchar(255) COMMENT 'description of the service
> package',
> >   PRIMARY KEY  (`id`)
> > ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
> >
> > CREATE TABLE IF NOT EXISTS `cloud`.`external_netscaler_controlcenter` (
> >   `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
> >   `uuid` varchar(255) UNIQUE,
> >   `username` varchar(255) COMMENT 'username of the NCC',
> >   `password` varchar(255) COMMENT 'password of NCC',
> >   `ncc_ip` varchar(255) COMMENT 'IP of NCC Manager',
> >   `num_retries` bigint unsigned NOT NULL default 2 COMMENT 'Number of
> retries in ncc for command failure',
> >   PRIMARY KEY  (`id`)
> > ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
> >
> > ALTER TABLE `cloud`.`sslcerts` ADD COLUMN `name` varchar(255) NULL
> default NULL COMMENT 'Name of the Certificate';
> > ALTER TABLE `cloud`.`network_offerings` ADD COLUMN `service_package_id`
> varchar(255) NULL default NULL COMMENT 'Netscaler ControlCenter Service
> Package';
> >
> >
> > DROP VIEW IF EXISTS `cloud`.`user_view`;
> > CREATE VIEW `cloud`.`user_view` AS
> > select
> > user.id,
> > user.uuid,
> > user.username,
> > user.password,
> > user.firstname,
> > user.lastname,
> > user.email,
> > user.state,
> > user.api_key,
> > user.secret_key,
> > user.created,
> > user.removed,
> > user.timezone,
> > user.registration_token,
> > user.is_registered,
> > user.incorrect_login_attempts,
> > user.source,
> > user.default,
> > account.id account_id,
> > account.uuid account_uuid,
> > account.account_name account_name,
> > account.type account_type,
> > account.role_id account_role_id,
> > domain.id domain_id,
> > domain.uuid

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

2018-02-14 Thread Rene Moser

Hi all

Thanks for taking care of this issue.

However, I wonder how we can prevent such issues in the future and
wondering if this "incident" has its orginos by our current git workflow.

I find the workflow "commit to stable branch --> merge back to master"
is inconvenient. Because at one point, the branches divert too much
(think of refactorings on master branch) and a merge back to the master
would not work or would break.

In most of the project I work with, there are stable-branches as well
but any fix (and feature of course) has to be on master first. Any
serious fix could be cherry picked to stable branches (there may be more
than one) if applicable or an alternative fix could be made. stable
branches never receives merges from or to master branch.

To be honest, I haven't seen any project adopting this workflow, I
wondered why and how come we have this workflow.

Thanks for clarification.

René



On 02/12/2018 02:29 PM, Rohit Yadav wrote:
> All,
> 
> 
> Some of us have discussed and found an upgrade path issue that only affects 
> the 4.10.0.0 users who may see missing columns in certain tables post 
> upgrading to 4.11.0.0 version.
> 
> 
> The issue is/was that at the time 4.10.0.0 was released, PRs were merged to 
> the 'then' master branch that made changes to the 4.9.3.0->4.10.0.0 upgrade 
> path instead of the 4.10.0.0->4.11.0.0 upgrade path. One of such change was 
> an ALTER statement that added a new column `service_package_id`, and if 
> 4.10.0.0 version is upgraded to 4.11.0.0 such environments may report sql 
> related errors related to this column.
> 
> 
> (My colleagues Bobby and Ernie may chip in their findings and test results as 
> well)
> 
> 
> Pull request: https://github.com/apache/cloudstack/pull/2452
> 
> Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-10285
> 
> 
> The proposed workaround is to move the changes to the expected upgrade path 
> of 4.10.0.0->4.11.0.0. This is *not ideal* given 4.11.0.0 is released but at 
> least 4.10.0.0 users who may want to upgrade to 4.11.1.0 or later won't face 
> the issue.
> 
> 
> The proposed workaround for current 4.10.0.0 users who may want to upgrade to 
> 4.11.0.0 is to run the following sql statements from the PR above before 
> upgrading to 4.11.0.0: (we may discuss and update the release notes website 
> as well)
> 
> 
> ### start sql ###
> 
> 
> use cloud;
> 
> 
> CREATE TABLE IF NOT EXISTS `cloud`.`netscaler_servicepackages` (
>   `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
>   `uuid` varchar(255) UNIQUE,
>   `name` varchar(255) UNIQUE COMMENT 'name of the service package',
>   `description` varchar(255) COMMENT 'description of the service package',
>   PRIMARY KEY  (`id`)
> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
> 
> CREATE TABLE IF NOT EXISTS `cloud`.`external_netscaler_controlcenter` (
>   `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
>   `uuid` varchar(255) UNIQUE,
>   `username` varchar(255) COMMENT 'username of the NCC',
>   `password` varchar(255) COMMENT 'password of NCC',
>   `ncc_ip` varchar(255) COMMENT 'IP of NCC Manager',
>   `num_retries` bigint unsigned NOT NULL default 2 COMMENT 'Number of retries 
> in ncc for command failure',
>   PRIMARY KEY  (`id`)
> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
> 
> ALTER TABLE `cloud`.`sslcerts` ADD COLUMN `name` varchar(255) NULL default 
> NULL COMMENT 'Name of the Certificate';
> ALTER TABLE `cloud`.`network_offerings` ADD COLUMN `service_package_id` 
> varchar(255) NULL default NULL COMMENT 'Netscaler ControlCenter Service 
> Package';
> 
> 
> DROP VIEW IF EXISTS `cloud`.`user_view`;
> CREATE VIEW `cloud`.`user_view` AS
> select
> user.id,
> user.uuid,
> user.username,
> user.password,
> user.firstname,
> user.lastname,
> user.email,
> user.state,
> user.api_key,
> user.secret_key,
> user.created,
> user.removed,
> user.timezone,
> user.registration_token,
> user.is_registered,
> user.incorrect_login_attempts,
> user.source,
> user.default,
> account.id account_id,
> account.uuid account_uuid,
> account.account_name account_name,
> account.type account_type,
> account.role_id account_role_id,
> domain.id domain_id,
> domain.uuid domain_uuid,
> domain.name domain_name,
> domain.path domain_path,
> async_job.id job_id,
> async_job.uuid job_uuid,
> async_job.job_status job_status,
> async_job.account_id job_account_id
> from
> `cloud`.`user`
> inner join
> `cloud`.`account` ON user.account_id = account.id
> inner join
> `cloud`.`domain` ON account.domain_id = domain.id
> left join
> `cloud`.`async_job` ON async_job.instance_id = user.id
> and async_job.instance_type = 'User'
> and

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

2018-02-14 Thread Rohit Yadav

All,


Given there was no objection to the approach, the concerned PR has been merged 
now. I've added a note to the release notes website for 4.10.0.0 users only.


4.11.1.0 will contain the fix, so in case 4.10.0.0 users want to upgrade, they 
will not be required to manually run the sql/upgrade-path workaround. Thanks.


- Rohit

<https://cloudstack.apache.org>




From: Rohit Yadav <rohit.ya...@shapeblue.com>
Sent: Monday, February 12, 2018 2:29:18 PM
To: dev@cloudstack.apache.org; us...@cloudstack.apache.org
Cc: Boris Stoyanov; Ernie Janse van Rensburg
Subject: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 
4.11.0.0

All,


Some of us have discussed and found an upgrade path issue that only affects the 
4.10.0.0 users who may see missing columns in certain tables post upgrading to 
4.11.0.0 version.


The issue is/was that at the time 4.10.0.0 was released, PRs were merged to the 
'then' master branch that made changes to the 4.9.3.0->4.10.0.0 upgrade path 
instead of the 4.10.0.0->4.11.0.0 upgrade path. One of such change was an ALTER 
statement that added a new column `service_package_id`, and if 4.10.0.0 version 
is upgraded to 4.11.0.0 such environments may report sql related errors related 
to this column.


(My colleagues Bobby and Ernie may chip in their findings and test results as 
well)


Pull request: https://github.com/apache/cloudstack/pull/2452

Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-10285


The proposed workaround is to move the changes to the expected upgrade path of 
4.10.0.0->4.11.0.0. This is *not ideal* given 4.11.0.0 is released but at least 
4.10.0.0 users who may want to upgrade to 4.11.1.0 or later won't face the 
issue.


The proposed workaround for current 4.10.0.0 users who may want to upgrade to 
4.11.0.0 is to run the following sql statements from the PR above before 
upgrading to 4.11.0.0: (we may discuss and update the release notes website as 
well)


### start sql ###


use cloud;


CREATE TABLE IF NOT EXISTS `cloud`.`netscaler_servicepackages` (
  `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
  `uuid` varchar(255) UNIQUE,
  `name` varchar(255) UNIQUE COMMENT 'name of the service package',
  `description` varchar(255) COMMENT 'description of the service package',
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `cloud`.`external_netscaler_controlcenter` (
  `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
  `uuid` varchar(255) UNIQUE,
  `username` varchar(255) COMMENT 'username of the NCC',
  `password` varchar(255) COMMENT 'password of NCC',
  `ncc_ip` varchar(255) COMMENT 'IP of NCC Manager',
  `num_retries` bigint unsigned NOT NULL default 2 COMMENT 'Number of retries 
in ncc for command failure',
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

ALTER TABLE `cloud`.`sslcerts` ADD COLUMN `name` varchar(255) NULL default NULL 
COMMENT 'Name of the Certificate';
ALTER TABLE `cloud`.`network_offerings` ADD COLUMN `service_package_id` 
varchar(255) NULL default NULL COMMENT 'Netscaler ControlCenter Service 
Package';


DROP VIEW IF EXISTS `cloud`.`user_view`;
CREATE VIEW `cloud`.`user_view` AS
select
user.id,
user.uuid,
user.username,
user.password,
user.firstname,
user.lastname,
user.email,
user.state,
user.api_key,
user.secret_key,
user.created,
user.removed,
user.timezone,
user.registration_token,
user.is_registered,
user.incorrect_login_attempts,
user.source,
user.default,
account.id account_id,
account.uuid account_uuid,
account.account_name account_name,
account.type account_type,
account.role_id account_role_id,
domain.id domain_id,
domain.uuid domain_uuid,
domain.name domain_name,
domain.path domain_path,
async_job.id job_id,
async_job.uuid job_uuid,
async_job.job_status job_status,
async_job.account_id job_account_id
from
`cloud`.`user`
inner join
`cloud`.`account` ON user.account_id = account.id
inner join
`cloud`.`domain` ON account.domain_id = domain.id
left join
`cloud`.`async_job` ON async_job.instance_id = user.id
and async_job.instance_type = 'User'
and async_job.job_status = 0;


### end sql ###


Thoughts, perhaps other ways to tackle this?


- Rohit

<https://cloudstack.apache.org>



rohit.ya...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue




rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue

[DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

2018-02-12 Thread Rohit Yadav

All,


Some of us have discussed and found an upgrade path issue that only affects the 
4.10.0.0 users who may see missing columns in certain tables post upgrading to 
4.11.0.0 version.


The issue is/was that at the time 4.10.0.0 was released, PRs were merged to the 
'then' master branch that made changes to the 4.9.3.0->4.10.0.0 upgrade path 
instead of the 4.10.0.0->4.11.0.0 upgrade path. One of such change was an ALTER 
statement that added a new column `service_package_id`, and if 4.10.0.0 version 
is upgraded to 4.11.0.0 such environments may report sql related errors related 
to this column.


(My colleagues Bobby and Ernie may chip in their findings and test results as 
well)


Pull request: https://github.com/apache/cloudstack/pull/2452

Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-10285


The proposed workaround is to move the changes to the expected upgrade path of 
4.10.0.0->4.11.0.0. This is *not ideal* given 4.11.0.0 is released but at least 
4.10.0.0 users who may want to upgrade to 4.11.1.0 or later won't face the 
issue.


The proposed workaround for current 4.10.0.0 users who may want to upgrade to 
4.11.0.0 is to run the following sql statements from the PR above before 
upgrading to 4.11.0.0: (we may discuss and update the release notes website as 
well)


### start sql ###


use cloud;


CREATE TABLE IF NOT EXISTS `cloud`.`netscaler_servicepackages` (
  `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
  `uuid` varchar(255) UNIQUE,
  `name` varchar(255) UNIQUE COMMENT 'name of the service package',
  `description` varchar(255) COMMENT 'description of the service package',
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `cloud`.`external_netscaler_controlcenter` (
  `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
  `uuid` varchar(255) UNIQUE,
  `username` varchar(255) COMMENT 'username of the NCC',
  `password` varchar(255) COMMENT 'password of NCC',
  `ncc_ip` varchar(255) COMMENT 'IP of NCC Manager',
  `num_retries` bigint unsigned NOT NULL default 2 COMMENT 'Number of retries 
in ncc for command failure',
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

ALTER TABLE `cloud`.`sslcerts` ADD COLUMN `name` varchar(255) NULL default NULL 
COMMENT 'Name of the Certificate';
ALTER TABLE `cloud`.`network_offerings` ADD COLUMN `service_package_id` 
varchar(255) NULL default NULL COMMENT 'Netscaler ControlCenter Service 
Package';


DROP VIEW IF EXISTS `cloud`.`user_view`;
CREATE VIEW `cloud`.`user_view` AS
select
user.id,
user.uuid,
user.username,
user.password,
user.firstname,
user.lastname,
user.email,
user.state,
user.api_key,
user.secret_key,
user.created,
user.removed,
user.timezone,
user.registration_token,
user.is_registered,
user.incorrect_login_attempts,
user.source,
user.default,
account.id account_id,
account.uuid account_uuid,
account.account_name account_name,
account.type account_type,
account.role_id account_role_id,
domain.id domain_id,
domain.uuid domain_uuid,
domain.name domain_name,
domain.path domain_path,
async_job.id job_id,
async_job.uuid job_uuid,
async_job.job_status job_status,
async_job.account_id job_account_id
from
`cloud`.`user`
inner join
`cloud`.`account` ON user.account_id = account.id
inner join
`cloud`.`domain` ON account.domain_id = domain.id
left join
`cloud`.`async_job` ON async_job.instance_id = user.id
and async_job.instance_type = 'User'
and async_job.job_status = 0;


### end sql ###


Thoughts, perhaps other ways to tackle this?


- Rohit





rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

Re: [DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

[DISCUSS] DB upgrade issue workaround for 4.10.0.0 users upgrading to 4.11.0.0

7 matches

Site Navigation

Mail list logo

Footer information