Re: [edk2] TianoCore Community Meeting Minutes
Hello Jeremiah,
On 10/19/18 18:09, Jeremiah Cox wrote:
> I'd like to better understand assertions of GitHub "lock in". GitHub
> provides a comprehensive REST API that you can easily drive via Python
> ("pip install PyGithub").
> I recall one assertion that GitHub holds PR comments hostage, but it
> is trivial to dump out all Comments on all PRs in a repo.
>
>
>g = Github( )
>repo = g.get_repo("chipsec/chipsec")
>pulls = repo.get_pulls(sort='created', base='master')
>for pr in pulls:
> for comment in pr.get_review_comments():
> print(pr.number)
> print(comment.body)
> ...
>
> Can folks help me better understand examples of GitHub "lock in"? I
> think it would be trivial to author a daemon that listens for PRs and
> comments, and forward those to a mailing list, if that is preferred
> for archival.
> With respect to web-based, we believe it facilitates convenient,
> multi-platform access, and GitHub's REST APIs can be leveraged to
> provide a command-line driven experience. I believe Sean's point was
> that GitHub provides a 1-stop shop for code reviews and issue tracking
> such that there is no need to spend TianoCore resources set up,
> maintain, and update separate Bugzilla and Gerrit services, VMs, OSs,
> & machines. The REST APIs could be used to maintain a mailing list or
> other mirror. I am not asserting that GitHub provides the best code
> review and issue tracker experience, our team finds Azure Dev Ops is
> superior, but GitHub is sufficient for most use cases while handling
> the infrastructure so that we can focus on getting things done.
>
> I think it would be helpful to construct a PROs/CONs table for each of
> the proposed end-to-end solution (source control, code review, gates,
> CI/CD, issue tracking, testing, ...). Perhaps this table is on a wiki
> to enable faster iteration than emails on the DL.
> We think of "Pull Request" as a contribution & review process akin to
> GitHub, GitLab, or Azure DevOps. Feedback, responses, automated
> check-in gates, and signoff can be handled within each PR ticket.
> Policies can be assigned, automated tests that must pass, enforcement
> of community wide rules (# of reviewers, reviewer list, all comments
> "resolved"). Tests could start as simple as successful build and over
> time could grow to a more comprehensive set of boot, functional,
> static analysis, ...
> By leveraging a popular hosting provider's built-in workflow we enjoy
> significant efficiencies. Its fully documented and known to most. New
> capabilities are being added daily. No TianoCore resources would be
> required to manage systems, OS patches, network issues, etc and can
> instead be focused on TianoCore code, building tests, and driving
> process improvements.
I don't have anything against GitHub specifically. I also don't have any
"favorite" WebUI (beyond Bugzilla, of course :) ). I'm concerned about a
number of general anti-patterns that might not be obvious, and might not
be easy to prevent with any given modern web application. Perhaps I can
word this as a personal wish list:
* Full email audit trail (all changes mailed out, even own ones), about
patch submission, PRs, patch review comments, and bug tracker actions
alike.
One should be able to establish the full history of a bug or a pull
request just from the emails. (This would be for human readers, not
for machines.) Review comments made on the web, in reference to
specific parts of a patch, should preserve that context in email form.
The messages generated for a given bug or PR should form a thread on
the mailing list. (Proper use of In-Reply-To and References headers.)
The generated emails should be readable without HTML support (i.e. as
plain text). As a counter-example for this, I'll mention JIRA --
JIRA's emails are practically unreadable without HTML rendering
support in the MUA.
If a WebUI can offer such an email gateway as a feature, that's
awesome. It is fine if the gateway is one-way only (web to email). The
service should not be an external one however, in order to minimize
email lossage when the WebUI works but the gateway goes offline for
some reason. The gateway should be tightly integrated.
* The comment editing interface on the WebUI should support, in addition
to the default rich editing interface, a plaintext mode that uses
monospace font, interprets no markdown sequences, and preserves the
exact layout of the plaintext comment (including indentation and line
wrapping).
Counter-examples abound. Perhaps somewhat unexpectedly, I'll call out
Launchpad. Among other things:
(a) Launchpad truncates comments it thinks are "too long", in the full
bug display -- and then readers have to open the bug comment
separately, to see it in its full length;
(b) Launchpad inserts invisible breaks into words it thinks are "too
long" -- and then, if someone double clicks a 40-nibble
[edk2] TianoCore Community Meeting Minutes
Finally finding time to respond to an older thread...
>>> The was a concern raised over potential lock-in to Github's, specifically in
>>> regards to history retention.
>>> Several Github users brought up that this shouldn't be an issue.
>>
>> Hopefully they said more than that.
>> What does "shouldn't be an issue" mean.
>> Were these users from multiple organisations?
>
>I will ask for more details in the next meeting, but the comments were
>that the API is quite robust and lends itself to readily accessible data.
I'd like to better understand assertions of GitHub "lock in". GitHub provides
a comprehensive REST API that you can easily drive via Python ("pip install
PyGithub").
I recall one assertion that GitHub holds PR comments hostage, but it is trivial
to dump out all Comments on all PRs in a repo.
g = Github( )
repo = g.get_repo("chipsec/chipsec")
pulls = repo.get_pulls(sort='created', base='master')
for pr in pulls:
for comment in pr.get_review_comments():
print(pr.number)
print(comment.body)
...
Can folks help me better understand examples of GitHub "lock in"? I think it
would be trivial to author a daemon that listens for PRs and comments, and
forward those to a mailing list, if that is preferred for archival.
>>
>>> Shawn mentioned some benefits to stock Github such as
>>> it is always up to date, it includes APIs to extract data, pull
>>> requests
>>
>> Since we are discussing multiple different development systems, can we
>> try to be a bit more explicit? This is referring to github's web-based
>> branch-based ticketing system, yes?
>
>Yes that's correct.
With respect to web-based, we believe it facilitates convenient, multi-platform
access, and GitHub's REST APIs can be leveraged to provide a command-line
driven experience. I believe Sean's point was that GitHub provides a 1-stop
shop for code reviews and issue tracking such that there is no need to spend
TianoCore resources set up, maintain, and update separate Bugzilla and Gerrit
services, VMs, OSs, & machines. The REST APIs could be used to maintain a
mailing list or other mirror. I am not asserting that GitHub provides the best
code review and issue tracker experience, our team finds Azure Dev Ops is
superior, but GitHub is sufficient for most use cases while handling the
infrastructure so that we can focus on getting things done.
I think it would be helpful to construct a PROs/CONs table for each of the
proposed end-to-end solution (source control, code review, gates, CI/CD, issue
tracking, testing, ...). Perhaps this table is on a wiki to enable faster
iteration than emails on the DL.
>> I know language drift and all, but
>>
>> is what pull request means to at least 3 participants in this conversation.
>
>I think this is an important distinction to make and I'm sure it will
>come up in future discussions. I'll be sure to bring this up.
We think of "Pull Request" as a contribution & review process akin to GitHub,
GitLab, or Azure DevOps. Feedback, responses, automated check-in gates, and
signoff can be handled within each PR ticket. Policies can be assigned,
automated tests that must pass, enforcement of community wide rules (# of
reviewers, reviewer list, all comments "resolved"). Tests could start as
simple as successful build and over time could grow to a more comprehensive set
of boot, functional, static analysis, ...
By leveraging a popular hosting provider's built-in workflow we enjoy
significant efficiencies. Its fully documented and known to most. New
capabilities are being added daily. No TianoCore resources would be required
to manage systems, OS patches, network issues, etc and can instead be focused
on TianoCore code, building tests, and driving process improvements.
>>
>> It seems somewhat less than ideal to me that all of the github
>> proponents were on the opposite call to me and Laszlo (and Ard). Were
>> any of them Asia-based or could we try to get them on the call with
>> Europe next time? I'm sure me and Laszlo could be somewhat more
>> accommodating than your 7AM, but we're not going to stay up for a 3/4AM
>> meeting about source control.
>
>That's understandable. I'm not sure where Shawn and Nate are located,
>but I believe it is in the US. We could certainly move the AM call a bit
>later if it makes it easier to get more folks from the US on that call.
Agreed, pushing the US/EU call out a couple hours (8 or 9am Pacific) would
likely increase Pacific participation.
Thanks,
Jeremiah
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes (stephano)
Thank you for the feedback. Comments below. On 10/15/2018 4:58 PM, Jeremiah Cox wrote: Some additional feedback... General --- We would like to have a discussion around goals, what are the top issues we are trying to solve with these solutions? We believe a primary challenge is getting code integrated downstream. We would like to see security patches flow to more systems, faster, with higher confidence. The same applies to new UEFI features. Part of making downstream integration efficient is getting downstream changes upstreamed, thus we support efforts to improve upstream contribution efficiency & quality. This is high on our list of concerns, along with generally making life easier for current developers. We would like to address ease of use for newcomers, but I've made it clear in calls that making our current community's workflow more streamlined and efficient takes priority. Patch Workflow Improvement -- We would like to propose adding Azure DevOps (previously Visual Studio Online) to the list. Azure DevOps is free for OSS and more feature rich than GitHub: https://azure.microsoft.com/en-us/pricing/details/devops/azure-devops-services/ Public CI - With respect to using Azure DevOps for CI, we have an example of this working here: https://dev.azure.com/projectmu/mu/_build?definitionId=4 I've added Azure DevOps to the list of items to be discussed in our CI email, along with Cirrus and (I'm sure) Jenkins/Travis. Andrew started that and I'll be pulling it out into its own thread shortly. Repos & Submodules -- In an upcoming meeting, we would like to discuss code layout and repositories. We propose to introduce layering and separation for a variety of reasons, best articulated by visiting the following: https://microsoft.github.io/mu/ I'd like to specifically call out Project Mu on our next call. That will be happening next week, after the Plugfest in Taipei. Kind regards, Jeremiah Cox ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes (stephano)
Some additional feedback... General --- We would like to have a discussion around goals, what are the top issues we are trying to solve with these solutions? We believe a primary challenge is getting code integrated downstream. We would like to see security patches flow to more systems, faster, with higher confidence. The same applies to new UEFI features. Part of making downstream integration efficient is getting downstream changes upstreamed, thus we support efforts to improve upstream contribution efficiency & quality. Patch Workflow Improvement -- We would like to propose adding Azure DevOps (previously Visual Studio Online) to the list. Azure DevOps is free for OSS and more feature rich than GitHub: https://azure.microsoft.com/en-us/pricing/details/devops/azure-devops-services/ Public CI - With respect to using Azure DevOps for CI, we have an example of this working here: https://dev.azure.com/projectmu/mu/_build?definitionId=4 Repos & Submodules -- In an upcoming meeting, we would like to discuss code layout and repositories. We propose to introduce layering and separation for a variety of reasons, best articulated by visiting the following: https://microsoft.github.io/mu/ Kind regards, Jeremiah Cox ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes
On 10/12/2018 6:27 AM, Leif Lindholm wrote:> I know at least one organisation that would be prevented from using it until they had explicit approval (which could take a while), and it puts me in the position of having to keep a spare x86 machine on my desktop in order to participate. Solutions I know would work in this scenario (i.e. work without plugins or downloads in any modern browser) are: - google hangouts (requires accounts, not ideal for China) - Skype (not SFB though, only plain old Skype) - Bluejeans (requires organiser account) We needed an organizer account for Zoom since our meeting was 1 hour long. So switching to something paid is not an issue. I've used Bluejeans with success in the past. I'll look into switching to this platform if no one else has any objections. I'd like to stay away from my other preferred google hangouts because so many of our members are in PRC. The was a concern raised over potential lock-in to Github's, specifically in regards to history retention. Several Github users brought up that this shouldn't be an issue. Hopefully they said more than that. What does "shouldn't be an issue" mean. Were these users from multiple organisations? I will ask for more details in the next meeting, but the comments were that the API is quite robust and lends itself to readily accessible data. Shawn mentioned some benefits to stock Github such as it is always up to date, it includes APIs to extract data, pull requests Since we are discussing multiple different development systems, can we try to be a bit more explicit? This is referring to github's web-based branch-based ticketing system, yes? Yes that's correct. I know language drift and all, but https://www.kernel.org/doc/html/latest/maintainer/pull-requests.html is what pull request means to at least 3 participants in this conversation. I think this is an important distinction to make and I'm sure it will come up in future discussions. I'll be sure to bring this up. lend themselves to simple(r) automation, and it has a much more modern web UI than Gerrit. It seems somewhat less than ideal to me that all of the github proponents were on the opposite call to me and Laszlo (and Ard). Were any of them Asia-based or could we try to get them on the call with Europe next time? I'm sure me and Laszlo could be somewhat more accommodating than your 7AM, but we're not going to stay up for a 3/4AM meeting about source control. That's understandable. I'm not sure where Shawn and Nate are located, but I believe it is in the US. We could certainly move the AM call a bit later if it makes it easier to get more folks from the US on that call. -How can we better track the code quality of BaseTools and the current build system on the whole. Should we add a "requires documentation change" flag to BZ so that it will be easier to compile a list of required doc changes. Could we also have a thread on someting that you sort of mention in the text, but not the bullet points?: - Suggestions for improvements to BaseTools. Of course. I'll add this to the list of "community conversations" that I will start up on the list this week. Cheers, Stephano ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes
> On Oct 11, 2018, at 10:43 AM, stephano > wrote: > > Thank you all for a great set of meetings! > > This is an overview of the topics discussed and the tasks that were assigned. > Please feel free to send me any questions or comments. > ... > Improvements to the Build Process > > We would like to gather a list of concrete specific proposals. Nate mentioned > that there will always be some specialized tooling because of the nature of > BIOS. Shawn mentioned that we need to keep in mind that making development > easier for current developers should be a priority over making the processes > easier for newcomers. I will send out emails asking for specific feedback on > some of these topics: > > -Which toolchains are being used, which are validated, and which are known to > create reproducible builds. > -When toolchains are known to be orphaned, should they be archived or simply > removed. > -Could we add some kconfig-like tool that allows introspection into what type > of builds are available. > -How can we better track the code quality of BaseTools and the current build > system on the whole. Should we add a "requires documentation change" flag to > BZ so that it will be easier to compile a list of required doc changes. > Stephano, I'll help start the conversation on the build. 1) Making the process easier for newcomers. a) Getting it to build the 1st time I think we could aspire to a more Unix like flow: git clone ./configure make ovmf make ovmf.run The ./config step would automate resolving dependencies. Things like figuring out what compiler is installed, what extra tools need to be installed etc. It could be interactive? make would build all the build tools, and do what ever local config is required to get the build working. Then build the requested target. make .run is more of an optional step. It could do things like launch Ovmf with your new ROM you just constructed. b) Debugging failures. Better quality error messages. Making the build.log from the build command a default setting(or have an abstracted defined database that can be queried), and add documentation on how to use it. Maybe make the output easier to consume? c) Making changes. That can be a phase 2. 2) Build performance, complexity, and maintainability. I'm about to complain about the build system, but I was around when it was first invented so I'm to blame as much as anyone We have way too much custom Python that is very complex, hard to maintain, and slow. One thing we did better on the EDK was break up the build tools into smaller chunks (maybe too small in some cases) and I always found I could figure things out, but then again all the code was C. Another interesting data point is we added parallel build, via makefiles, to the EDK. When we moved from the EDK to the edk2 we noticed the builds slowed down a lot. I did some profiling on the edk2 build and I noticed 1,000,000 calls to regex (part of that long . sequence on a build of a complex platform). It turns out we have custom Python code that is generating the makefile dependencies so it was scanning all the C code. The other issue I see is build parallelism is controlled from the Python code. It also seems the edk2 build system is not a complete solution and a lot of full platform implementations have shell scripts or batch files, that wrap the calls to build. So I'd like to see a build system: 1) Start with a top level makefile in the root of the tree. 2) Replaces custom batch or shell scripts with makefiles to handle pre and post build tasks. 3) Use a tool to construct a parser for the ekd2 build files (INF, DSC, DEC). Use this data to construct the makefiles and build a database for other build components to use. Chris Roberts has messed around and built an Earley parser using the EBNF in the build documents. There are some problematic issues in the grammar, but some grammar streamlining might make it possible to use tooling to construct a parser vs. writing one from scratch in Python. 4) Move to using the compiler to generate dependencies. gcc/clang support this via -M*. I think it may be a little more complex for NMAKE but there is a /showIncludes compiler flag. 5) Have the build tool terminate after parallel makefile generation 6) Top level makefile can invoke parallel build on generated makefiles. 7) Invoke build tool to generate FVs and FDs from makefile. 8) Invoke custom post processing steps in the makefile. Thanks, Andrew Fish ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes
> On Oct 12, 2018, at 11:30 AM, Kinney, Michael D > wrote: > >> -Original Message- >> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org >> <mailto:edk2-devel-boun...@lists.01.org>] >> On Behalf Of Leif Lindholm >> Sent: Friday, October 12, 2018 11:06 AM >> To: Laszlo Ersek mailto:ler...@redhat.com>> >> Cc: edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> >> Subject: Re: [edk2] TianoCore Community Meeting Minutes >> >> On Fri, Oct 12, 2018 at 06:07:01PM +0200, Laszlo Ersek >> wrote: >>> On 10/12/18 15:27, Leif Lindholm wrote: >>>> On Thu, Oct 11, 2018 at 10:43:57AM -0700, stephano >> wrote: >>> >>>>> Switching to Standard C Types >>>>> - >>>>> Both Shawn and Nate mentioned that the current >> system has been in place for >>>>> a long time and some people prefer the current >> setup. I can start an email >>>>> discussion around this issue specifically if anyone >> feels strongly that we >>>>> should be using standard types. >>>> >>>> So, I don't think we made it this far down the agenda >> on the US-EU >>>> call. >>>> >>>> One way would be to simply explicitly permit it, >> possibly with the >>>> constraint that every module needs to pick one and >> stick with it, >>>> unless people object. >>>> >>>> I think we'll want to discuss this in a US-EU call as >> well. >>> >>> I'm playing devil's advocate here -- because, in >> general, I'm a fan of >>> sticking with standard C as much as possible --, but I >> see a big >>> obstacle in the way. >>> >>> That obstacle is "Table 5. Common UEFI Data Types", in >> the UEFI spec. >>> Until a good portion of that table is expressed in >> terms of standard C >>> types as well (expanding upon the current definitions), >> possibly in an >>> edk2-level spec (i.e. not necessarily in the UEFI spec >> itself), I think >>> there's no chance to enable standard C types in edk2 >> *meaningfully*. >>> >>> Because, as soon as you have to call a PI or UEFI >> interface, you'll have >>> to stick with the PI/UEFI spec types anyway. >> >> I don't necessarily see that as an issue. But it is a >> good point that >> it can't just be the codebase changing. >> >> Since we are however extremely specificly not looking to >> change the >> underlying storage types, all it would take would be to >> make a >> 2-column table into a 3-column table in both specs. Or >> just add a >> separate table for the mapping. Then edk2 could adopt the >> "permitted" >> rule as soon as the specs were out. Arguably (because >> we're not >> changing underlying types) we could do it before, but we >> _are_ >> supposed to be the reference implementation, so it would >> be poor form. > > I agree that it would be best if the specs list synonymous > type names. We would have to guarantee in the edk2 implementation > that they types are identical. One potential issue is support > for really old compilers. If we can decide to only support > compilers that fully support the synonymous types, then that > would be clean. Not sure what the ANSI C equivalents are for > INTN/UINTN on all compilers. > Mike, INTN -> intptr_t UINTN -> uintptr_t If I understand correctly the types we are talking about are defined via stdint.h. Thus if we are a freestanding (__STDC_HOSTED__ == 0) project and we did not include stdint.h we would need to define them in the edk2 headers? If the edk2 App/Driver is also including stdint.h we would need some way to avoid conflicts. Likely including stdint.h from ProcessorBind.h in place of defining the values. Thanks, Andrew Fish >> >>>>> Using Git Submodules (like we do with OpenSSL) >>>>> >>>> >>>> We didn't make it here either. What would we use it >> _for_? >>>> I think the openssl case makes a lot of sense, but >> what else? >>> >>> We embed a bunch of other projects (libraries, mainly): >>> - Oniguruma >>> - Brotli >>> - fdt >>> - LZMA SDK >>> - ... >> >> Sure. But do we know that is what was meant? >> >> I certainly recall the "each package should be a >> submodule" idea from >> a (much) earlier conversation, and would like to ensure >> we're not >> resurrecting that. > > Yes. Those other projects was the brief discussion. > Not submodule per package. > >> >> Regards, >> >> Leif >> ___ >> edk2-devel mailing list >> edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> >> https://lists.01.org/mailman/listinfo/edk2-devel >> <https://lists.01.org/mailman/listinfo/edk2-devel> > ___ > edk2-devel mailing list > edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> > https://lists.01.org/mailman/listinfo/edk2-devel > <https://lists.01.org/mailman/listinfo/edk2-devel> ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes
> On Oct 12, 2018, at 9:07 AM, Laszlo Ersek wrote: > > On 10/12/18 15:27, Leif Lindholm wrote: >> On Thu, Oct 11, 2018 at 10:43:57AM -0700, stephano wrote: > >>> Switching to Standard C Types >>> - >>> Both Shawn and Nate mentioned that the current system has been in place for >>> a long time and some people prefer the current setup. I can start an email >>> discussion around this issue specifically if anyone feels strongly that we >>> should be using standard types. >> >> So, I don't think we made it this far down the agenda on the US-EU >> call. >> >> One way would be to simply explicitly permit it, possibly with the >> constraint that every module needs to pick one and stick with it, >> unless people object. >> >> I think we'll want to discuss this in a US-EU call as well. > > I'm playing devil's advocate here -- because, in general, I'm a fan of > sticking with standard C as much as possible --, but I see a big > obstacle in the way. > > That obstacle is "Table 5. Common UEFI Data Types", in the UEFI spec. > Until a good portion of that table is expressed in terms of standard C > types as well (expanding upon the current definitions), possibly in an > edk2-level spec (i.e. not necessarily in the UEFI spec itself), I think > there's no chance to enable standard C types in edk2 *meaningfully*. > > Because, as soon as you have to call a PI or UEFI interface, you'll have > to stick with the PI/UEFI spec types anyway. > Lazlo, Given there is also a C ABI for each supported processor architecture in the UEFI spec it should be possible to define the EFI types in terms of C types. The only potential issue is I'm not sure BOOLEAN maps to bool in all cases? https://github.com/tianocore/edk2/blob/master/MdePkg/Include/X64/ProcessorBind.h#L188 So typedef __int64UINT64; or typedef long long INT64; becomes: typedef int64_tINT64; Thus we could move the definition of UINT64, INT64, UINT32, INT32, UINT16, INT16, UINT8, INT8, CHAR, and CHAR16 to Base.h and have ProcessorBind.h just define the C standard types for a given architecture. The tricky part is when there are different answers to the question are you using a standard C lib. Basically the definitions in MdePkg/Include/X64/ProcessorBind.h could conflict with stdint.h. Almost seems like you would need a build flag to control if you use stdint.h. I think the bigger question is what problem are we trying to solve? If some one is programming with uint32_t are they going to expect printf(), memcpy(), etc. We solve that problem today StdLib? I guess we only have an option of the full pedantic C lib (I can't remember if StdLib depends on the shell?), or nothing. Are we trying to define a light weight C lib that works in the firmware code? How much of the C lib do we need to make that useful? Thanks, Andrew Fish PS Speaking of printf() != Print() or DEBUG()... I see a lot of people botching Print() in EFI. It would nice if we could get compiler warnings for miss use. You can add printf warnings to any functing in int my_printf (void *my_object, const char *my_format, ...) __attribute__ ((format (printf, 2, 3))); It would be awesome if we could add edk2_print to at least gcc and clang. Given they are open source projects anything is possible. Not sure how this works on VC++? > >>> Using Git Submodules (like we do with OpenSSL) >>> >> >> We didn't make it here either. What would we use it _for_? >> I think the openssl case makes a lot of sense, but what else? > > We embed a bunch of other projects (libraries, mainly): > - Oniguruma > - Brotli > - fdt > - LZMA SDK > - ... > > Thanks > Laszlo > ___ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes
> -Original Message- > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] > On Behalf Of Leif Lindholm > Sent: Friday, October 12, 2018 11:06 AM > To: Laszlo Ersek > Cc: edk2-devel@lists.01.org > Subject: Re: [edk2] TianoCore Community Meeting Minutes > > On Fri, Oct 12, 2018 at 06:07:01PM +0200, Laszlo Ersek > wrote: > > On 10/12/18 15:27, Leif Lindholm wrote: > > > On Thu, Oct 11, 2018 at 10:43:57AM -0700, stephano > wrote: > > > > >> Switching to Standard C Types > > >> - > > >> Both Shawn and Nate mentioned that the current > system has been in place for > > >> a long time and some people prefer the current > setup. I can start an email > > >> discussion around this issue specifically if anyone > feels strongly that we > > >> should be using standard types. > > > > > > So, I don't think we made it this far down the agenda > on the US-EU > > > call. > > > > > > One way would be to simply explicitly permit it, > possibly with the > > > constraint that every module needs to pick one and > stick with it, > > > unless people object. > > > > > > I think we'll want to discuss this in a US-EU call as > well. > > > > I'm playing devil's advocate here -- because, in > general, I'm a fan of > > sticking with standard C as much as possible --, but I > see a big > > obstacle in the way. > > > > That obstacle is "Table 5. Common UEFI Data Types", in > the UEFI spec. > > Until a good portion of that table is expressed in > terms of standard C > > types as well (expanding upon the current definitions), > possibly in an > > edk2-level spec (i.e. not necessarily in the UEFI spec > itself), I think > > there's no chance to enable standard C types in edk2 > *meaningfully*. > > > > Because, as soon as you have to call a PI or UEFI > interface, you'll have > > to stick with the PI/UEFI spec types anyway. > > I don't necessarily see that as an issue. But it is a > good point that > it can't just be the codebase changing. > > Since we are however extremely specificly not looking to > change the > underlying storage types, all it would take would be to > make a > 2-column table into a 3-column table in both specs. Or > just add a > separate table for the mapping. Then edk2 could adopt the > "permitted" > rule as soon as the specs were out. Arguably (because > we're not > changing underlying types) we could do it before, but we > _are_ > supposed to be the reference implementation, so it would > be poor form. I agree that it would be best if the specs list synonymous type names. We would have to guarantee in the edk2 implementation that they types are identical. One potential issue is support for really old compilers. If we can decide to only support compilers that fully support the synonymous types, then that would be clean. Not sure what the ANSI C equivalents are for INTN/UINTN on all compilers. > > > >> Using Git Submodules (like we do with OpenSSL) > > >> > > > > > > We didn't make it here either. What would we use it > _for_? > > > I think the openssl case makes a lot of sense, but > what else? > > > > We embed a bunch of other projects (libraries, mainly): > > - Oniguruma > > - Brotli > > - fdt > > - LZMA SDK > > - ... > > Sure. But do we know that is what was meant? > > I certainly recall the "each package should be a > submodule" idea from > a (much) earlier conversation, and would like to ensure > we're not > resurrecting that. Yes. Those other projects was the brief discussion. Not submodule per package. > > Regards, > > Leif > ___ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes
On Fri, Oct 12, 2018 at 06:07:01PM +0200, Laszlo Ersek wrote: > On 10/12/18 15:27, Leif Lindholm wrote: > > On Thu, Oct 11, 2018 at 10:43:57AM -0700, stephano wrote: > > >> Switching to Standard C Types > >> - > >> Both Shawn and Nate mentioned that the current system has been in place for > >> a long time and some people prefer the current setup. I can start an email > >> discussion around this issue specifically if anyone feels strongly that we > >> should be using standard types. > > > > So, I don't think we made it this far down the agenda on the US-EU > > call. > > > > One way would be to simply explicitly permit it, possibly with the > > constraint that every module needs to pick one and stick with it, > > unless people object. > > > > I think we'll want to discuss this in a US-EU call as well. > > I'm playing devil's advocate here -- because, in general, I'm a fan of > sticking with standard C as much as possible --, but I see a big > obstacle in the way. > > That obstacle is "Table 5. Common UEFI Data Types", in the UEFI spec. > Until a good portion of that table is expressed in terms of standard C > types as well (expanding upon the current definitions), possibly in an > edk2-level spec (i.e. not necessarily in the UEFI spec itself), I think > there's no chance to enable standard C types in edk2 *meaningfully*. > > Because, as soon as you have to call a PI or UEFI interface, you'll have > to stick with the PI/UEFI spec types anyway. I don't necessarily see that as an issue. But it is a good point that it can't just be the codebase changing. Since we are however extremely specificly not looking to change the underlying storage types, all it would take would be to make a 2-column table into a 3-column table in both specs. Or just add a separate table for the mapping. Then edk2 could adopt the "permitted" rule as soon as the specs were out. Arguably (because we're not changing underlying types) we could do it before, but we _are_ supposed to be the reference implementation, so it would be poor form. > >> Using Git Submodules (like we do with OpenSSL) > >> > > > > We didn't make it here either. What would we use it _for_? > > I think the openssl case makes a lot of sense, but what else? > > We embed a bunch of other projects (libraries, mainly): > - Oniguruma > - Brotli > - fdt > - LZMA SDK > - ... Sure. But do we know that is what was meant? I certainly recall the "each package should be a submodule" idea from a (much) earlier conversation, and would like to ensure we're not resurrecting that. Regards, Leif ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes
On 10/12/18 15:27, Leif Lindholm wrote: > On Thu, Oct 11, 2018 at 10:43:57AM -0700, stephano wrote: >> Switching to Standard C Types >> - >> Both Shawn and Nate mentioned that the current system has been in place for >> a long time and some people prefer the current setup. I can start an email >> discussion around this issue specifically if anyone feels strongly that we >> should be using standard types. > > So, I don't think we made it this far down the agenda on the US-EU > call. > > One way would be to simply explicitly permit it, possibly with the > constraint that every module needs to pick one and stick with it, > unless people object. > > I think we'll want to discuss this in a US-EU call as well. I'm playing devil's advocate here -- because, in general, I'm a fan of sticking with standard C as much as possible --, but I see a big obstacle in the way. That obstacle is "Table 5. Common UEFI Data Types", in the UEFI spec. Until a good portion of that table is expressed in terms of standard C types as well (expanding upon the current definitions), possibly in an edk2-level spec (i.e. not necessarily in the UEFI spec itself), I think there's no chance to enable standard C types in edk2 *meaningfully*. Because, as soon as you have to call a PI or UEFI interface, you'll have to stick with the PI/UEFI spec types anyway. >> Using Git Submodules (like we do with OpenSSL) >> > > We didn't make it here either. What would we use it _for_? > I think the openssl case makes a lot of sense, but what else? We embed a bunch of other projects (libraries, mainly): - Oniguruma - Brotli - fdt - LZMA SDK - ... Thanks Laszlo ___ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Re: [edk2] TianoCore Community Meeting Minutes
Hi Stephano, Thanks for pulling this together. Some comments below. On Thu, Oct 11, 2018 at 10:43:57AM -0700, stephano wrote: > Thank you all for a great set of meetings! > > This is an overview of the topics discussed and the tasks that were > assigned. Please feel free to send me any questions or comments. > > Licensing > - > Licensing was discussed and no questions or comments came up. Please feel > free to contact me directly if you would like more info. > > Community Meetings > -- > Community meetings will be held regularly. We will plan public bug scrubs > and design meetings. Meeting formats will use zoom and be held 2 times per > day to accommodate all time zones. So I didn't comment on the call (which I dialled in to, using a phone, because that was the only thing that wasn't going to delay me 10 minutes), but I would prefer a conferencing system that does not require binary downloads to participate in. I know at least one organisation that would be prevented from using it until they had explicit approval (which could take a while), and it puts me in the position of having to keep a spare x86 machine on my desktop in order to participate. Solutions I know would work in this scenario (i.e. work without plugins or downloads in any modern browser) are: - google hangouts (requires accounts, not ideal for China) - Skype (not SFB though, only plain old Skype) - Bluejeans (requires organiser account) Sure, I could attend via the Android app for zoom, but it would make it tedious to paste links to/from chat and put me on a tiny screen. > Invites will be posted to the mailing > list and the wiki. Since the mailing list does not allow attachments, I will > work on getting an "add to calendar" link attached to the wiki page. > > I will send out a poll to see if the community would be interested in a > Slack channel. I'm happy to admin it if there is interest. > > I will also be setting up a specific meeting, probably next month, to > discuss general code and commit message standards. > > > Patch Workflow Improvement > -- > We would like to find a more modern, user friendly patch workflow that fits > (as best as it can) most company's workflow. > It is recommended that the > community familiarize themselves with some of the options that are out there > so that we can pose useful questions / comments at the next meeting. Please > be sure to research up-to-date information and not rely on old info. Options > and tools discussed included: > -Github > -Gitlab > -Phabricator > -Gerrit > > The was a concern raised over potential lock-in to Github's, specifically in > regards to history retention. > Several Github users brought up that this shouldn't be an issue. Hopefully they said more than that. What does "shouldn't be an issue" mean. Were these users from multiple organisations? > Shawn mentioned some benefits to stock Github such as > it is always up to date, it includes APIs to extract data, pull > requests Since we are discussing multiple different development systems, can we try to be a bit more explicit? This is referring to github's web-based branch-based ticketing system, yes? I know language drift and all, but https://www.kernel.org/doc/html/latest/maintainer/pull-requests.html is what pull request means to at least 3 participants in this conversation. > lend themselves to simple(r) automation, and it has a much more modern web > UI than Gerrit. It seems somewhat less than ideal to me that all of the github proponents were on the opposite call to me and Laszlo (and Ard). Were any of them Asia-based or could we try to get them on the call with Europe next time? I'm sure me and Laszlo could be somewhat more accommodating than your 7AM, but we're not going to stay up for a 3/4AM meeting about source control. > Several community members including Marcin and Rebecca have used Phabricator > and can recommend it. Intel has used Gerrit internally and can recommend its > use. I don't think we had anyone from ARM on either of the calls, but I know they use gerrit internally. For those who weren't on the call I was on, I am _not_ a fan of gerrit - but if the deciding factor (as suggested above) is to best align with the internal processes of contributing organisations, then gerrit wins hands down. > No one voiced any issues with Bugzilla and several from the community > thought that we should continue to use it. > > Improvements to the Build Process > > We would like to gather a list of concrete specific proposals. Nate > mentioned that there will always be some specialized tooling because of the > nature of BIOS. Shawn mentioned that we need to keep in mind that making > development easier for current developers should be a priority over making > the processes easier for newcomers. I will send out emails asking for > specific feedback on some of these topics: > > -Which toolchains are being used, which are
[edk2] TianoCore Community Meeting Minutes
Thank you all for a great set of meetings! This is an overview of the topics discussed and the tasks that were assigned. Please feel free to send me any questions or comments. Licensing - Licensing was discussed and no questions or comments came up. Please feel free to contact me directly if you would like more info. Community Meetings -- Community meetings will be held regularly. We will plan public bug scrubs and design meetings. Meeting formats will use zoom and be held 2 times per day to accommodate all time zones. Invites will be posted to the mailing list and the wiki. Since the mailing list does not allow attachments, I will work on getting an "add to calendar" link attached to the wiki page. I will send out a poll to see if the community would be interested in a Slack channel. I'm happy to admin it if there is interest. I will also be setting up a specific meeting, probably next month, to discuss general code and commit message standards. Patch Workflow Improvement -- We would like to find a more modern, user friendly patch workflow that fits (as best as it can) most company's workflow. It is recommended that the community familiarize themselves with some of the options that are out there so that we can pose useful questions / comments at the next meeting. Please be sure to research up-to-date information and not rely on old info. Options and tools discussed included: -Github -Gitlab -Phabricator -Gerrit The was a concern raised over potential lock-in to Github's, specifically in regards to history retention. Several Github users brought up that this shouldn't be an issue. Shawn mentioned some benefits to stock Github such as it is always up to date, it includes APIs to extract data, pull requests lend themselves to simple(r) automation, and it has a much more modern web UI than Gerrit. Several community members including Marcin and Rebecca have used Phabricator and can recommend it. Intel has used Gerrit internally and can recommend its use. No one voiced any issues with Bugzilla and several from the community thought that we should continue to use it. Improvements to the Build Process We would like to gather a list of concrete specific proposals. Nate mentioned that there will always be some specialized tooling because of the nature of BIOS. Shawn mentioned that we need to keep in mind that making development easier for current developers should be a priority over making the processes easier for newcomers. I will send out emails asking for specific feedback on some of these topics: -Which toolchains are being used, which are validated, and which are known to create reproducible builds. -When toolchains are known to be orphaned, should they be archived or simply removed. -Could we add some kconfig-like tool that allows introspection into what type of builds are available. -How can we better track the code quality of BaseTools and the current build system on the whole. Should we add a "requires documentation change" flag to BZ so that it will be easier to compile a list of required doc changes. Changing to LF instead of CRLF -- It was agreed that this is a good idea but that we should create a test repo to verify this change before implementing it. It was noted that this will cause some overhead with tasks like git blame, however the benefits outweigh the costs. Switching to Standard C Types - Both Shawn and Nate mentioned that the current system has been in place for a long time and some people prefer the current setup. I can start an email discussion around this issue specifically if anyone feels strongly that we should be using standard types. Using Git Submodules (like we do with OpenSSL) Many in the group liked this idea. I will engage with the communities to gauge their interest in taking patches or helping with our integration of their software. Public CI - This subject was not discussed at length but there seems to be quite a bit of community interest. We will discuss this more in the next meeting. Please familiarize yourself with some possible solutions, with a focus on supporting Windows + Linux + Mac. Some possible solutions include: https://azure.microsoft.com/en-us/blog/introducing-azure-devops/ https://cirrus-ci.org/ Mike has tried out Cirrus with some success. Some positive points to Cirrus include: -Build time of ~ 7 Minutes (clone edk2, apt-get all dependent tools, build BaseTools, and build 3 versions of OVMF) -Caching of VM contents so reduce build times of subsequent builds -We could use platforms like OVMF to boot FW and provide test results from tools like MicroPythonTestFramework -Cirrus includes integration with docker Cheers, Stephano Stephano Cetola TianoCore Community Manager stephano.cet...@linux.intel.com
