Re: [OpenStack-Infra] [gear] Making Gear easier to consume ( less .encode() and .decode() )

[Morgan Fainberg]

On Tue, Jun 21, 2016 at 3:16 PM, James E. Blair  wrote:

> Morgan Fainberg  writes:
>
> > As I have been converting Zuul and NodePool to python3, I have had to do
> a
> > bunch of changes around encode() and decode() of strings since gear is
> > (properly) an implementation of a protocol that requires binary data
> > (rather than text_strings).
> >
> > What this has highlighted is that gear should be made a bit more friendly
> > to use in the python world. We already explicitly assume a utf-8 encoding
> > for when things are turned into "binary" when crafting the job object in
> > certain cases [1].  I have discussed this with Jim Blair, and we both
> agree
> > that the ability to still reference attributes such as "job.name" (in a
> > simple manner that is straight forward), is important.
>
> Thanks for this email -- with this and a little browsing of the gear
> code, I think I've absorbed most of the context and am able to process
> this.  :)
>
> > Here is the outline of the change I'm proposing:
> >
> > * The main consumable part of gear (public classes) will convert the
> > "string" data we assign ( name[2], unique[3]) into utf-8-encoded bytes
> via
> > @property, @property.setter, and @property.getter for public consumption.
>
> This seems good.  I think we can make the assertion that in the API
> these things should always be utf-8 encoded strings, and do what's
> needed behind the scenes to encode/decode to bytes.
>
> I think this is the bulk of what we need to do to make gear
> user-friendly.
>
>
++ This is where I am going to focus.


> > * Arguments are explicitly supposed to be a binary_blob [4]. I am unsure
> if
> > this should also be automatically converted *or* if it should be the
> > inverse, .arguments and .arguments_string ?
>
> This is the thing we can't handle automatically.  At least, we can on
> the input side (if the user passes in a string, we could auto-encode but
> leave bytes alone -- but on the output side since the protocol is
> un-typed, we wouldn't know what to do).  So maybe this is a place where
> we should force the user to encode/decode.
>
> Looking at the gear code, I think that was the intent in a lot of
> places, but I think we may have gotten ahead of ourselves and written
> code assuming that isinstance('string', bytes) is False (which is the
> case in python3, but not in python2):
>
>
> https://github.com/openstack-infra/gear/blob/59d29104cb9c370e49b44f313e568bd43b172e1b/gear/__init__.py#L589
>
> That line of code explains a lot to me.
>
> I think we should stick close to the idea that arguments are binary
> blobs.
>
> From that, it seems to follow that reading job.arguments should get you
> a bytes object.
>
> On the one hand, I don't think it's unreasonable to say that users must
> encode/decode on that boundary.
>
> On the other hand, I think we could easily have a job.arguments_string
> property that decodes that from utf-8 for convenience, so why not?  I
> think I'm in favor of this.
>
> If we do that, then I think the question is: do we have gear
> automatically encode job arguments (in the Job constructor) if the user
> passes in a string?  I tend to favor this as well because it should
> almost always do the right thing in both python2 and python3 without
> inhibiting usage for either strings or binary data.  Does that sound
> reasonable?
>
>
I've spent a lot of time thinking about this, I'm going to say that I
agree, but I
feel like the _string property will be mostly unused. I want to see how
things look
before we add it in. If it makes things a lot better we can always add it.


> > * Internally gear will reference the encoded bits via the underlying
> > _binary form, which will allow direct access to a non-"string" form
> > of the data itself in the case that there is interesting things that need
> > to be handled via binary packing (for example) instead of "stringified"
> > versions.
>
> Sounds good.  Maybe we should call it "_bytes" though for clarity?
> /bikeshed
>
>
HAH! Sure. But as long as the bikeshed is cherenkov radiation blue.


> > * For compatibility the main @property.setter will handle either
> > binary_type or string_type (so we don't break everyone).
>
> Most of these are set in the constructor -- having a setter do that
> should make the constructor do the right thing, but in case there are
> cases where we need to remove type validation, I think it's worth saying
> that our intent is to have the constructor handle both forms as well for
> the same reason.
>
> > * The "_binary" will enforce that the data with be a binary_type
> > only.
> >
> >
> > I think this can be done in a single release of gear with minimal impact
> to
> > those using it. For what it is worth, it is unlikely that anyone has used
> > gear extensively in python3 as of yet because of recent bug fixes that
> > addressed py2->py3 compat issues around dict.values() and similar list()
> ->
> > iter() changes.
>
> Agreed.
>
> 

Re: [OpenStack-Infra] JJB V2.0 planning

[Darragh Bailey]

It's starting to sound like Friday would suit people best, I'm happy enough
with that day as well, may have to leave slightly before 17hr utc, but
should be able to be present for most of the time.

Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool" - unknown
On 5 Jul 2016 17:21, "Wayne Warren"  wrote:

>
>
> On Tue, Jul 5, 2016 at 6:32 AM, Darragh Bailey 
> wrote:
>
>>
>>
>> On 5 July 2016 at 05:44, Kien Ha  wrote:
>>
>>> HI Darragh,
>>>
>>> I would like to help where I can in the reviews, but Tuesdays and
>>> Thursdays are my busiest days. I can only guarantee that I am free after
>>> 19:00 UTC on Tuesdays and Thursdays. Monday and Fridays are days where I am
>>> free from summer class obligations but I am not too sure if that will work
>>> with everyone else.
>>>
>>
>> I can do Monday, assuming Friday is a little more awkward for people in
>> European timezones as they plan to wrap up for the day earlier.
>>
>> Woud 13:00-1700 UTC suit on those days?
>>
>>
>>>
>>> Regards,
>>> Kien Ha
>>>
>>> On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma  wrote:
>>>
 Hey Darragh,



 I agree with your thought and would like to participate to the reviews,
 although the time slots is a little late for me but it works.



>>>
>> Would moving the time slot an hour earlier on either Mon/Fri suit you
>> better?
>>
>
> For the proposed time slots so far, Friday would be better for me--I have
> a team meeting I cannot miss between UTC 1530 and UTC 1630 (although it
> usually runs a half hour short).
>
> However, if Monday 1300 - 1700 UTC ends up being the best time for
> everyone else, I can attempt to split my attention between the JJB 2.0
> review session and that meeting, or set aside dedicated time outside of the
> review session to respond to comments and iterate on the 2.0 patches
> (really I will want to do this anyway).
>
>
>> --
>> Darragh Bailey
>> "Nothing is foolproof to a sufficiently talented fool"
>>
>> ___
>> OpenStack-Infra mailing list
>> OpenStack-Infra@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>
>>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-dev] [StoryBoard] Thanks for the bugsquash, plus a new-things roundup

[David Moreau Simard]

I appreciate all the great work that's been going into StoryBoard lately.

Thanks everyone involved.

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Tue, Jul 5, 2016 at 12:50 PM, Zara Zaimeche
 wrote:
> Hi all,
>
> A big thank you to everyone who came and helped out in the spectacular*
> StoryBoard bug squash! We look forward to the next. :) Here are some
> hilights from the last couple of weeks:
>
> * BEAUTIFUL NEW COMMENTS AND EVENTS TIMELINE
>
> It's so beautiful, it requires all-caps. SotK has transformed the barebones
> events timeline into an elegant swan. Well, that's a weird mixed metaphor,
> but it *is* lovely! Furthermore, this magnificent gentleman has removed
> pagination so that comments are no longer lost on the second page of the
> results, and has made it possible to link comments directly. Extra thanks to
> ttx for fixing some of the css during the bugsquash! :) Here's an example:
>
> https://storyboard.openstack.org/#!/story/2000464#comment-7029
>
> There is a WIP patch in review for editing one's own comments, for anyone
> interested in trying it out and giving feedback:
>
> https://review.openstack.org/#/c/333418/
>
>
> * Email threading
>
> The kindly pedroalvarez has worked some magic on the emails StoryBoard
> sends, so that they are threaded according to story. It should now be easier
> to see what an email refers to at a glance.
>
>
> * API Docs example commands
>
> anteaya has made it easier for people to interact with StoryBoard via the
> API with these examples. This should be good news for anyone who wants to
> use scripts with StoryBoard. You can see them here:
>
> http://docs.openstack.org/infra/storyboard/webapi/v1.html#stories
>
>
> * Gerrit integration for storyboard-dev
>
> Review-dev can now post comments on storyboard-dev (our test instance)!
> Thanks so much, zaro! You can see an example patch here:
> https://review-dev.openstack.org/#/c/5454/
>
>
> * Tags search upgraded
>
> Tags search now suggests existing tags! This should making searching-by-tag
> much easier.
>
> I hope to build on this to change task-statuses in the next couple of weeks.
>
>
>
> It's been a pretty busy time... which is why I'm over a week late with this
> email \o/. Anyway, yes, thanks again to everyone who helped out. If you'd
> like to get involved in the project, we're always available in #storyboard
> on freenode; the project is a mix of python and angularjs. We have a
> worklist of stories that contain easy tasks here:
> https://storyboard.openstack.org/#!/worklist/76 , so you can see if anything
> takes your interest, then it's best to ask in the channel for the specifics.
> :)
>
> Hope to see you there! If I've missed anything, please let me know.
>
> Best Wishes,
>
> Zara
>
> *I haven't personally written any interesting patches of late, so I am
> allowed to call it 'spectacular'. :)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] JJB V2.0 planning

[Wayne Warren]

On Tue, Jul 5, 2016 at 6:32 AM, Darragh Bailey 
wrote:

>
>
> On 5 July 2016 at 05:44, Kien Ha  wrote:
>
>> HI Darragh,
>>
>> I would like to help where I can in the reviews, but Tuesdays and
>> Thursdays are my busiest days. I can only guarantee that I am free after
>> 19:00 UTC on Tuesdays and Thursdays. Monday and Fridays are days where I am
>> free from summer class obligations but I am not too sure if that will work
>> with everyone else.
>>
>
> I can do Monday, assuming Friday is a little more awkward for people in
> European timezones as they plan to wrap up for the day earlier.
>
> Woud 13:00-1700 UTC suit on those days?
>
>
>>
>> Regards,
>> Kien Ha
>>
>> On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma  wrote:
>>
>>> Hey Darragh,
>>>
>>>
>>>
>>> I agree with your thought and would like to participate to the reviews,
>>> although the time slots is a little late for me but it works.
>>>
>>>
>>>
>>
> Would moving the time slot an hour earlier on either Mon/Fri suit you
> better?
>

For the proposed time slots so far, Friday would be better for me--I have a
team meeting I cannot miss between UTC 1530 and UTC 1630 (although it
usually runs a half hour short).

However, if Monday 1300 - 1700 UTC ends up being the best time for everyone
else, I can attempt to split my attention between the JJB 2.0 review
session and that meeting, or set aside dedicated time outside of the review
session to respond to comments and iterate on the 2.0 patches (really I
will want to do this anyway).


> --
> Darragh Bailey
> "Nothing is foolproof to a sufficiently talented fool"
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] JJB V2.0 planning

[Darragh Bailey]

On 5 July 2016 at 05:44, Kien Ha  wrote:

> HI Darragh,
>
> I would like to help where I can in the reviews, but Tuesdays and
> Thursdays are my busiest days. I can only guarantee that I am free after
> 19:00 UTC on Tuesdays and Thursdays. Monday and Fridays are days where I am
> free from summer class obligations but I am not too sure if that will work
> with everyone else.
>

I can do Monday, assuming Friday is a little more awkward for people in
European timezones as they plan to wrap up for the day earlier.

Woud 13:00-1700 UTC suit on those days?


>
> Regards,
> Kien Ha
>
> On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma  wrote:
>
>> Hey Darragh,
>>
>>
>>
>> I agree with your thought and would like to participate to the reviews,
>> although the time slots is a little late for me but it works.
>>
>>
>>
>
Would moving the time slot an hour earlier on either Mon/Fri suit you
better?

-- 
Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool"
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra