Re: [openstack-dev] [Fuel] [shotgun] New shotgun2 command: short-report

[Alex Schultz]

On Mon, Mar 21, 2016 at 3:15 PM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> Alex,
>
> >This should have been created before removing the thing providing this
> information previously.
>
> Once again, using version.yaml you could NOT reproduce the env, because
> what was really installed on the Fuel node had nothing in common with what
> it was written in version.yaml. The information you think was actual was,
> in fact, not actual.
>


So it could be, that's the difference. Yes it could be out of date for long
running environments where people have updated/upgraded. But in the case of
BVT failures or testing, it was more likely then not to be correct.  From
my standpoint as a developer working on non-released versions where the
packages were not likely to be updated, this information was used to work
on pre-release issues. Look once again, the choice to remove this
information had impacts far beyond what was understood when the choice was
made to remove it. Fine, we're dealing with it now and hopefully we can
improve things going forward.  I'm just asking for more information about
what people are intending with these types of tools so that others can
understand how to use them.



>
> Having 'shotgun report' output you can see real SHA sums, not ephemeral
> (like in version.yaml).
>
> And without removal version.yaml we could not implement some features in
> make system, so, this new build/test package based approach could not have
> been implemented before version.yaml removal. We are moving towards better
> developer experience NOT making things worse. In fact version.yaml removal
> and substitution it with 'shotgun report' was a fix for broken version.yaml
> content.
>

I've voiced my issues with shotgun2 report and the quality of the
information provided.


>
> As for attaching short report to bugs instead of long report, I don't know
> what motivation was for this change except their convenience. Probably
> short report is to be used for internal QA team purposes only.
>
>
>
Once again, I'm just asking for more information as to the intended usage
of this new command and asking that this information be made public.

Thanks,
-Alex


>
>
>
> Vladimir Kozhukalov
>
> On Mon, Mar 21, 2016 at 11:20 PM, Alex Schultz 
> wrote:
>
>>
>> On Mon, Mar 21, 2016 at 1:59 PM, Vladimir Kozhukalov <
>> vkozhuka...@mirantis.com> wrote:
>>
>>> Alex,
>>>
>>> That is just a short report that QA team needs for their convenience.
>>> Please consider this letter as just FYI, nothing more.
>>>
>>>
>> If a bug only contains the short report, how do we work on fixing the
>> bug?  My question is valid and should be answered.  What good is this
>> information if I cannot reproduce an environment with this information? How
>> does one translate or locate specific package versions for reproducing
>> issues?  Is this documented?  If not, will it be?  I'd like to know what
>> the point of this short report is since I'm not sure how it could be
>> consumed by QA/Devs?
>>
>>
>>
>>> 'shotgun report' allows you to see commit SHA from which a package was
>>> built. It is even more information than it was available in version.yaml
>>> and this information is actual unlike the content of version.yaml.
>>>
>>> I know you were opposing getting rid of version.yaml but the thing is
>>> Fuel now can be installed on any CentOS 7.2 node directly from RPM
>>> repository. You don't even need the Fuel ISO, and thus version.yaml could
>>> not be an artifact that we could rely on (no ISO build id any more, no sha
>>> sums). Instead, now we rely on packages that are currently installed on the
>>> master node. The only issue with this approach is the ability to easily
>>> reproduce the env having just this list of packages attached to a bug. But
>>> it is not worse that it was with version.yaml.
>>>
>>>
>> I was only opposed to getting rid of it without a proper replacement
>> which is why I keep asking the same questions as an equivalent replacement
>> does not seem to exist.  Also it is much worse now that we don't have
>> version.yaml because I may have no way to locate these mystical package
>> versions that keep getting reported.  At least with git hashes I could at
>> least look at the same code base across everything. Now without this
>> information I have no way of building a complete picture of the code being
>> utilized in the environment.
>>
>>
>>> I'm currently working on design draft about modular data driven
>>> functional testing. This could also help for troubleshooting. In a nutshell
>>> the developer experience will be like:
>>>  1) you look at log files (`shotgun dump`) and roughly locate the issue
>>> (that allows you to choose respective test case)
>>>  2) you run script passing some data to it (data are to come from
>>> 'shotgun report --machinereadable' or smth like this)
>>>  3) this script builds testing/experimental env for you (env is to
>>> include only those components that are respective 

Re: [openstack-dev] [Fuel] [shotgun] New shotgun2 command: short-report

[Vladimir Kozhukalov]

Alex,

>This should have been created before removing the thing providing this
information previously.

Once again, using version.yaml you could NOT reproduce the env, because
what was really installed on the Fuel node had nothing in common with what
it was written in version.yaml. The information you think was actual was,
in fact, not actual.

Having 'shotgun report' output you can see real SHA sums, not ephemeral
(like in version.yaml).

And without removal version.yaml we could not implement some features in
make system, so, this new build/test package based approach could not have
been implemented before version.yaml removal. We are moving towards better
developer experience NOT making things worse. In fact version.yaml removal
and substitution it with 'shotgun report' was a fix for broken version.yaml
content.

As for attaching short report to bugs instead of long report, I don't know
what motivation was for this change except their convenience. Probably
short report is to be used for internal QA team purposes only.





Vladimir Kozhukalov

On Mon, Mar 21, 2016 at 11:20 PM, Alex Schultz 
wrote:

>
> On Mon, Mar 21, 2016 at 1:59 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com> wrote:
>
>> Alex,
>>
>> That is just a short report that QA team needs for their convenience.
>> Please consider this letter as just FYI, nothing more.
>>
>>
> If a bug only contains the short report, how do we work on fixing the
> bug?  My question is valid and should be answered.  What good is this
> information if I cannot reproduce an environment with this information? How
> does one translate or locate specific package versions for reproducing
> issues?  Is this documented?  If not, will it be?  I'd like to know what
> the point of this short report is since I'm not sure how it could be
> consumed by QA/Devs?
>
>
>
>> 'shotgun report' allows you to see commit SHA from which a package was
>> built. It is even more information than it was available in version.yaml
>> and this information is actual unlike the content of version.yaml.
>>
>> I know you were opposing getting rid of version.yaml but the thing is
>> Fuel now can be installed on any CentOS 7.2 node directly from RPM
>> repository. You don't even need the Fuel ISO, and thus version.yaml could
>> not be an artifact that we could rely on (no ISO build id any more, no sha
>> sums). Instead, now we rely on packages that are currently installed on the
>> master node. The only issue with this approach is the ability to easily
>> reproduce the env having just this list of packages attached to a bug. But
>> it is not worse that it was with version.yaml.
>>
>>
> I was only opposed to getting rid of it without a proper replacement which
> is why I keep asking the same questions as an equivalent replacement does
> not seem to exist.  Also it is much worse now that we don't have
> version.yaml because I may have no way to locate these mystical package
> versions that keep getting reported.  At least with git hashes I could at
> least look at the same code base across everything. Now without this
> information I have no way of building a complete picture of the code being
> utilized in the environment.
>
>
>> I'm currently working on design draft about modular data driven
>> functional testing. This could also help for troubleshooting. In a nutshell
>> the developer experience will be like:
>>  1) you look at log files (`shotgun dump`) and roughly locate the issue
>> (that allows you to choose respective test case)
>>  2) you run script passing some data to it (data are to come from
>> 'shotgun report --machinereadable' or smth like this)
>>  3) this script builds testing/experimental env for you (env is to
>> include only those components that are respective to chosen test case)
>>  5) you run some tests against this lab and manually do some experiments
>> to kill the bug
>>
>>
>>
>
> This should have been created before removing the thing providing this
> information previously. Yes I know I sound like a broken record on this,
> but it's very hard to address issues if you cannot reproduce the
> environment they occur on.  I'm trying to make sure we are providing all
> the information to aid in reproducing issues to get them fixed and not just
> providing more information that is ultimately ignored because it's useless.
>
> Thanks,
> -Alex
>
>
>>
>>
>> Vladimir Kozhukalov
>>
>> On Mon, Mar 21, 2016 at 5:28 PM, Alex Schultz 
>> wrote:
>>
>>>
>>>
>>> On Mon, Mar 21, 2016 at 7:21 AM, Volodymyr Shypyguzov <
>>> vshypygu...@mirantis.com> wrote:
>>>
 Hi, all

 Just wanted to inform you, that shotgun2 now has new command
 short-report, which allows you to receive shorter and cleaner output for
 attaching to bug description, sharing, etc.

 Usage: shotgun2 short-report
 Example output: http://paste.openstack.org/show/491256/


>>> How will we be able to find those specific packages and how will 

Re: [openstack-dev] [Fuel] [shotgun] New shotgun2 command: short-report

[Alex Schultz]

On Mon, Mar 21, 2016 at 1:59 PM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> Alex,
>
> That is just a short report that QA team needs for their convenience.
> Please consider this letter as just FYI, nothing more.
>
>
If a bug only contains the short report, how do we work on fixing the bug?
My question is valid and should be answered.  What good is this information
if I cannot reproduce an environment with this information? How does one
translate or locate specific package versions for reproducing issues?  Is
this documented?  If not, will it be?  I'd like to know what the point of
this short report is since I'm not sure how it could be consumed by
QA/Devs?



> 'shotgun report' allows you to see commit SHA from which a package was
> built. It is even more information than it was available in version.yaml
> and this information is actual unlike the content of version.yaml.
>
> I know you were opposing getting rid of version.yaml but the thing is Fuel
> now can be installed on any CentOS 7.2 node directly from RPM repository.
> You don't even need the Fuel ISO, and thus version.yaml could not be an
> artifact that we could rely on (no ISO build id any more, no sha sums).
> Instead, now we rely on packages that are currently installed on the master
> node. The only issue with this approach is the ability to easily reproduce
> the env having just this list of packages attached to a bug. But it is not
> worse that it was with version.yaml.
>
>
I was only opposed to getting rid of it without a proper replacement which
is why I keep asking the same questions as an equivalent replacement does
not seem to exist.  Also it is much worse now that we don't have
version.yaml because I may have no way to locate these mystical package
versions that keep getting reported.  At least with git hashes I could at
least look at the same code base across everything. Now without this
information I have no way of building a complete picture of the code being
utilized in the environment.


> I'm currently working on design draft about modular data driven functional
> testing. This could also help for troubleshooting. In a nutshell the
> developer experience will be like:
>  1) you look at log files (`shotgun dump`) and roughly locate the issue
> (that allows you to choose respective test case)
>  2) you run script passing some data to it (data are to come from 'shotgun
> report --machinereadable' or smth like this)
>  3) this script builds testing/experimental env for you (env is to include
> only those components that are respective to chosen test case)
>  5) you run some tests against this lab and manually do some experiments
> to kill the bug
>
>
>

This should have been created before removing the thing providing this
information previously. Yes I know I sound like a broken record on this,
but it's very hard to address issues if you cannot reproduce the
environment they occur on.  I'm trying to make sure we are providing all
the information to aid in reproducing issues to get them fixed and not just
providing more information that is ultimately ignored because it's useless.

Thanks,
-Alex


>
>
> Vladimir Kozhukalov
>
> On Mon, Mar 21, 2016 at 5:28 PM, Alex Schultz 
> wrote:
>
>>
>>
>> On Mon, Mar 21, 2016 at 7:21 AM, Volodymyr Shypyguzov <
>> vshypygu...@mirantis.com> wrote:
>>
>>> Hi, all
>>>
>>> Just wanted to inform you, that shotgun2 now has new command
>>> short-report, which allows you to receive shorter and cleaner output for
>>> attaching to bug description, sharing, etc.
>>>
>>> Usage: shotgun2 short-report
>>> Example output: http://paste.openstack.org/show/491256/
>>>
>>>
>> How will we be able to find those specific packages and how will we be
>> able to correlate them with the equivalent commit in the git repository?
>>
>> Thanks,
>> -Alex
>>
>>
>>> Regards,
>>> Volodymyr
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [Fuel] [shotgun] New shotgun2 command: short-report

[Vladimir Kozhukalov]

Alex,

That is just a short report that QA team needs for their convenience.
Please consider this letter as just FYI, nothing more.

'shotgun report' allows you to see commit SHA from which a package was
built. It is even more information than it was available in version.yaml
and this information is actual unlike the content of version.yaml.

I know you were opposing getting rid of version.yaml but the thing is Fuel
now can be installed on any CentOS 7.2 node directly from RPM repository.
You don't even need the Fuel ISO, and thus version.yaml could not be an
artifact that we could rely on (no ISO build id any more, no sha sums).
Instead, now we rely on packages that are currently installed on the master
node. The only issue with this approach is the ability to easily reproduce
the env having just this list of packages attached to a bug. But it is not
worse that it was with version.yaml.

I'm currently working on design draft about modular data driven functional
testing. This could also help for troubleshooting. In a nutshell the
developer experience will be like:
 1) you look at log files (`shotgun dump`) and roughly locate the issue
(that allows you to choose respective test case)
 2) you run script passing some data to it (data are to come from 'shotgun
report --machinereadable' or smth like this)
 3) this script builds testing/experimental env for you (env is to include
only those components that are respective to chosen test case)
 5) you run some tests against this lab and manually do some experiments to
kill the bug




Vladimir Kozhukalov

On Mon, Mar 21, 2016 at 5:28 PM, Alex Schultz  wrote:

>
>
> On Mon, Mar 21, 2016 at 7:21 AM, Volodymyr Shypyguzov <
> vshypygu...@mirantis.com> wrote:
>
>> Hi, all
>>
>> Just wanted to inform you, that shotgun2 now has new command
>> short-report, which allows you to receive shorter and cleaner output for
>> attaching to bug description, sharing, etc.
>>
>> Usage: shotgun2 short-report
>> Example output: http://paste.openstack.org/show/491256/
>>
>>
> How will we be able to find those specific packages and how will we be
> able to correlate them with the equivalent commit in the git repository?
>
> Thanks,
> -Alex
>
>
>> Regards,
>> Volodymyr
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] [shotgun] New shotgun2 command: short-report

[Alex Schultz]

On Mon, Mar 21, 2016 at 7:21 AM, Volodymyr Shypyguzov <
vshypygu...@mirantis.com> wrote:

> Hi, all
>
> Just wanted to inform you, that shotgun2 now has new command short-report,
> which allows you to receive shorter and cleaner output for attaching to bug
> description, sharing, etc.
>
> Usage: shotgun2 short-report
> Example output: http://paste.openstack.org/show/491256/
>
>
How will we be able to find those specific packages and how will we be able
to correlate them with the equivalent commit in the git repository?

Thanks,
-Alex


> Regards,
> Volodymyr
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] [shotgun] New shotgun2 command: short-report

[Volodymyr Shypyguzov]

Hi, all

Just wanted to inform you, that shotgun2 now has new command short-report,
which allows you to receive shorter and cleaner output for attaching to bug
description, sharing, etc.

Usage: shotgun2 short-report
Example output: http://paste.openstack.org/show/491256/

Regards,
Volodymyr
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev