Re: The new Apache NuttX Logo
Hi, I like the mascot idea, and all the animals proposed. But I liked specially the turtle idea, somehow embedded :) Also, the idea of the "nut" and the origins in Costa Rica but somehow universal animal lead me to think in hermit crabs, specifically juvenile coconut crabs: it seems they use coconut shells as house to live in. Somehow they "match" some of the ideas of NuttX: hermit crabs are quite ubiquitous, the are open to new ideas and hacking friendly to adapt whatever available to build his house, which is small or big but always tight to the needs (like different target chip/boards), also usually externally is a shell, or even a "nuttshell" (like coconut shells for crabs in Costa Rica). Also crabs looks quite "robotic" to me, and some crustaceans like mantis shrimps are very fast and precise, yet simple and not very dangerous animals, though definitively not the typical pet. So a crab happily embedded in his coconut shell could be a funny idea of a logo to me, if someone artistically inclined want to extend it :) Cheers, /Miguel El vie., 19 jun. 2020 21:05, Erdem MEYDANLI escribió: > >>> > My favorite animal, my "totem animal" is a turtle. Unfortunately, > a turtle is not a good representation for a real time operating system! > > > A Mutant Ninja Turtle. Noted. > > >> > NuttX was created in Costa Rica so an animal from Costa Rica does not > seem inappropriate. > << > > Indeed, before you refer to it, I have been considering something > originating in 'Costa Rica' in detail. As a cat lover, Margay Cats are on > my radar. Please watch this: https://www.youtube.com/watch?v=0RewQMq13pw > > BR, > Erdem > > Gregory Nutt , 19 Haz 2020 Cum, 15:25 tarihinde şunu > yazdı: > > > I once was told that it was Linus Torvalds who insisted on the Pengquin > > mascot. My favorit animal, my "totem animal" is turtle. Unfortunately, > > a turtle is not a good representation for a real time operating system! > > Unless it were a Mutant Ninja turtle ;) > > > > > >> NuttX is a very international project. The sun never sets on NuttX > > >> developers. We are on every continent (except perhaps Antarctica). So > if > > >> the mascot is an animal, I think we don't want something indigenous to > > only > > >> one place. But I don't know what to suggest here... > > NuttX was created in Costa Rica so an animal from Costa Rica does not > > seem in appropriate. Here the most beloved animals are (1) the > > rainforest frogs, the red-eyed rain forest frog in particular: > > > > > https://kids.nationalgeographic.com/content/dam/kids/photos/animals/Amphibians/Q-Z/red-eyed-tree-frog-on-leaves-3-2.ngsversion.1568211399457.adapt.1900.1.jpg > > , (2) the toucan: > > > > > http://www.pacifico-costarica.com/wp-content/uploads/2015/07/live-belize-keel-billed-toucan-national-bird.jpg > , > > > > and (2) the three-toed sloth. The slot is, again, not an appropriate > > image since it implies slow and lazy. > > > Maybe we should then consider an animal from Antarctica, or one that's > > > known to have a species there, to at least have a mascot from that > > > continent. > > > Maybe puffins? [1] They are small, fast and can live in both sea and > > land. > > > They also to some extent, and with some imagination, resemble little > > > penguins, which may also convey the "think tiny Linux" motto. > > > > I like Puffins. They also share the habitat with Penguins which is a > > good thing. > > > > I think of all of those, however, the red-eyed rainforest frog appeals > > to me most. He is already a caricature of himself. > > > > Greg > > > > > > >
Re: squashing commits or not
On Thu, Mar 5, 2020 at 5:30 PM David Sidrane wrote: > Isn't that is just more noise? > Noise that you can filter is less noisy. If you do have a PR with B1 and B2 commits and generates in master: A1 <- B1 <- B2 <- C1 <- master you cannot filter it easily. If you generate with branch merges: A <- B <- C <- master ^- B1 <- B2 <-/ then using git log gives: C B B1 B2 A but git log --merges gives: C B A so it is way less noisy but you do not remove from the repo the subcommit information (a.k.a. as noise by the people that don't need/care that information) Anyway, if the common used tool (aka Github web interface) does not provide that option, maybe yes, it is always noise for most people. So I probably agree with you in this case and anyway I will use whatever the community decide to use. > -Original Message- > From: Miguel Ángel Herranz [mailto:mig...@midokura.com.INVALID] > Sent: Thursday, March 05, 2020 8:16 AM > To: dev@nuttx.apache.org > Subject: Re: squashing commits or not > > Why not just leave all the commits and generate on master a single merge > commit? > > In git CLI you can easily filter the non-merge commits using `git log > --merges` (same sequence as squashed PR) and then inspect the involved > commits looking at the non-master commit parent. > > Not sure if the problem is that Github UI is not able to filter this way. > > On Thu, Mar 5, 2020 at 5:00 PM Gregory Nutt wrote: > > > We have to trust that committers will do the best job that they can. We > > have no control over how committers do that job. There is no mechanism > > for such control. This is an area where there is no option but to trust > > the judgement of the committers. > > > > Discussing good practice is fine, but there is no authority to control > > the behavior of the committers. The committers are free to do as they > > choose. The only "punishment" is peer pressure and social acceptance. > > > > > > > > -- > > Miguel Ángel Herranz Trillo > > Software developer > > mig...@midokura.com > > All rights reserved. All logos and descriptions are property of their > respective owners > > > > > > LEGAL NOTICE: This mail and its attached files are confidential and are > exclusively intended to their addresses. In case you may receive this mail > not being its address, we beg you to let us know the error by reply and to > proceed to delete it. The circulation by any mean of this mail could be > penalized in accordance with the applicable legislation. The use of both > the transmitter and the addresses with a commercial aim, or in order to be > incorporated to automated files, is not authorized. > -- Miguel Ángel Herranz Trillo Software developer mig...@midokura.com All rights reserved. All logos and descriptions are property of their respective owners LEGAL NOTICE: This mail and its attached files are confidential and are exclusively intended to their addresses. In case you may receive this mail not being its address, we beg you to let us know the error by reply and to proceed to delete it. The circulation by any mean of this mail could be penalized in accordance with the applicable legislation. The use of both the transmitter and the addresses with a commercial aim, or in order to be incorporated to automated files, is not authorized.
Re: squashing commits or not
Why not just leave all the commits and generate on master a single merge commit? In git CLI you can easily filter the non-merge commits using `git log --merges` (same sequence as squashed PR) and then inspect the involved commits looking at the non-master commit parent. Not sure if the problem is that Github UI is not able to filter this way. On Thu, Mar 5, 2020 at 5:00 PM Gregory Nutt wrote: > We have to trust that committers will do the best job that they can. We > have no control over how committers do that job. There is no mechanism > for such control. This is an area where there is no option but to trust > the judgement of the committers. > > Discussing good practice is fine, but there is no authority to control > the behavior of the committers. The committers are free to do as they > choose. The only "punishment" is peer pressure and social acceptance. > > > -- Miguel Ángel Herranz Trillo Software developer mig...@midokura.com All rights reserved. All logos and descriptions are property of their respective owners LEGAL NOTICE: This mail and its attached files are confidential and are exclusively intended to their addresses. In case you may receive this mail not being its address, we beg you to let us know the error by reply and to proceed to delete it. The circulation by any mean of this mail could be penalized in accordance with the applicable legislation. The use of both the transmitter and the addresses with a commercial aim, or in order to be incorporated to automated files, is not authorized.
Re: userspace/kernel isolation question
On Tue, Feb 25, 2020 at 5:27 PM Gregory Nutt wrote: > > > ... How are syscalls dealt in other RTOS using MPU? > > For the case of the MPU, there is no general answer to that. I don't > think that there is any standard model for the use of the MPU in a > Unix-like system. > > NuttX uses the MPU to create a Kernel space with the OS and critical > board logic in it and a separate user space for applications. That is > inspired by the standard Unix kernel + applications model. > > But every use of the MPU that I am familiar with in deeply embedded > bare-metal or tiny RTOS systems are totally custom solutions. > > For the MMU (which you might think of as an MPU with address mapping), > the Unix model is the only one I am familiar with. As in Linux, a > kernel blob and applications loaded into RAM from a file system. > > Greg > > Thanks for the info. I did a quick search and find some examples: *Zephyr*: https://docs.zephyrproject.org/1.13.0/kernel/usermode/mpu_userspace.html (emphasis added by me): "During system calls, the user mode thread’s access to the system call and the *passed-in parameters are all validated*. The user mode thread is then elevated to privileged mode, the stack is switched to use the privileged stack, and the call is made to the specified kernel API. On return from the kernel API, the thread is set back to user mode and the stack is restored to the user stack." Not sure what kind of validation is done though. *FreeRTOS*: some documentation related to MPU memory regions (but nothing said about syscall params): https://freertos.org/FreeRTOS-MPU-memory-protection-unit.html I would like to check the code, but if someone already know about it would be great :) Cheers, Miguel
Re: userspace/kernel isolation question
On Tue, Feb 25, 2020 at 5:08 PM Nathan Hartman wrote: > ... > If you need the sort of "security" that makes it possible to run > totally untrusted code, then maybe you need a full blown operating > system, which also comes with a full blown computer and not an > embedded microcontroller. > Despite it could be an "overkill" for most use cases, for other use cases it may be needed and not impacting too much in performance. It could be disabled if not needed. At least, these limitations/issues should be well documented, since it is not unusual that people, if not explicitly stated, assume that the same restriction as in Linux or other common OS apply to NuttX. How are syscalls dealt in other RTOS using MPU? I have little experience in that regard. Cheers, Miguel
Re: Jenkins CI
Thanks for your summary. Some comments: On Thu, Feb 13, 2020 at 5:16 AM Haitao Liu wrote: > Apache Jenkins >|Github action > > > ... > Restrict accounts access to update jenkins configs >| Free to Make PRs to update githubaction workflow > That is not exactly true: accounts access is restricted to create the _Jenkins jobs_. The logic executed in these jobs, once created, is stored in a Jenkinsfile or in Groovy pipeline script, so the workflow can be updated by a PR as in Github. Moreover, if using shared libraries, the logic can reside in another repository, as Greg requested. I don't know if that is possible with Github. > So, I think we could consider to use apache jenkins ci as nightly build > and use github action ci as PR check build accordingly. > I agree with that, but mostly due to apparently saturation of AFS CI service. 1. Apache Jenkins CI for Nightly Build > > There is an existing freestyle NuttX Nightly Build do cron job at > midnight each day. However, it's very simple. And there is still flock > issue to resolve to make parallel build all pass. > > > https://builds.apache.org/view/Incubator%20Projects/job/NuttX-Nightly-Build/ > > Thanks to @Maht, he made multibranch jenkins pipeline PR which would make > the Nightly build more robust. But it may still need some time to switch to > it because of Apache Jenkins access control. The PRs as below: > > https://github.com/apache/incubator-nuttx-testing/pull/2 > > https://github.com/apache/incubator-nuttx/pull/174 > > Well, my multibranch proposal is originally intended to manage PR triggers (each PR is a branch). For nightly, as long there is only one branch now (master), there is less added value. Still, the pipeline logic could be modified to be more "parallel" (to be done, though). Also, there is still travis CI pull request form @yamt, we could integrate > it into incubator-nuttx-testing as maht do. > > https://github.com/apache/incubator-nuttx/pull/183 > > < > https://builds.apache.org/view/Incubator%20Projects/job/NuttX-Nightly-Build/ > > > Yes, I think the idea of having the CI/testing stuff in a separate repo is good, as Greg requested, despite maybe some minimal file info has to be kept in the main repo. But I won't consider it a strong requirement for the short term if achieving it would take a lot of effort to implement outside of the typical usage of Github Actions/CircleCI/TravisCI/etc. 2. Github action CI for Check Build > > I have just make two PRs to incubator-nuttx and inucbator-nuttx-apps > accordingly as below: > > https://github.com/apache/incubator-nuttx/pull/261 > > https://github.com/apache/incubator-nuttx-apps/pull/65 > > The two PRs are not intended to be merged right now. They still need align > to check testlist update under incubator-nuttx-testing. > > For example, Consider have a minimal check testlist set that always run. > > As for github action check build CI, there are still more space to > optimize: > > 1. Matrix builds to allow concurrent jobs and even multi-platorms > > To speed up the check build time, use multiple jobs in github action > workflow to do parallel builds as @btashton and @davids suggest. > That would be a very nice addition, but I am not sure how to do it Github Actions, and have no much time this week to explore. > 2. Use docker images with tools preinstalled > > Now it takes several minitues to install all essential tools in ubuntu vm. > > we may build nuttx with docker images which tools preinstalled in the > future. > @davids suggested to use PX4 Docker images, and I think they are very good to use/reference: https://hub.docker.com/r/px4io/px4-dev-nuttx > Anyway, These are just some thoughts from our previous discussions, > welcome more reviews and feedbacks from community. > Thanks! > Thanks for the effort and count on me if you need some feedback. Cheers, Miguel
Re: Jenkins CI
Hi Haitao, I will take a look. By the way, do Apache Jenkins slave includes Docker? Or alternatively, does Apache Jenkins have Docker Slave [1] plugin available? If we could use either launch Docker containers in the slave or launch the jobs in a Docker container we could use an image with all the tools pre installed. Please someone with read privileges on Jenkins check that whenever possible (Xiang? Justin?). Thanks, Miguel [1] https://plugins.jenkins.io/docker-slaves On Tue, Feb 4, 2020 at 12:04 PM Haitao Liu wrote: > Thanks Xiang! > Another PR https://github.com/apache/incubator-nuttx-testing/pull/4 since > Apache jenkins slave haven't gperf preinstalled, > so switch to use our owned prebuilt gperf. Xiang and Miguel please help to > review. > Or in another way, may I ask Justin Apache Infra team to apt-get install > gperf instead? > > Xiang Xiao 于2020年2月4日周二 下午2:36写道： > > > I will merge it. > > > > On Tue, Feb 4, 2020 at 12:50 PM Haitao Liu wrote: > > > > > > Hi, Greg, > > > Thanks to Miguel's careful review, could you please help to finish the > PR > > > merge? > > > Then I and Miguel can continue the development with more PRs and > > refinement. > > > > > > Gregory Nutt 于2020年1月31日周五 下午11:30写道： > > > > > > > > > > > > I have just added the CI scripts and testlist files in > > > > > https://github.com/apache/incubator-nuttx-testing/pull/3 > > > > > And asked XiaoXiang help to create Nuttx Nightly build job in > > > > > builds.apache.org before. But it's just a freestyle CI job > > > > > which call cibuild.sh simply. Once CI script and testlist file > verify > > > > well. > > > > > It would be better if it could be integrated into the > > > > > pipeline definition you mentioned. You could help to use the > pipeline > > > > > way > > > > > if access granted. Let's work together on it :) > > > > > > > > Perhaps Miguel could provide the review of PR3. I will be happy to > > > > finish the merge after some review is provided. > > > > > > > > Once it has been merged, both you and Miguel can continue the > > > > development with PRs. But we need to find someone competent to > provide > > > > review. > > > > > > > > Greg > > > > > > > > > > > > > > >
Re: Jenkins CI
On Fri, Jan 31, 2020 at 4:30 PM Gregory Nutt wrote: > > Perhaps Miguel could provide the review of PR3. I will be happy to > finish the merge after some review is provided. > > I can review the script, but maybe mostly generic (not too specific to NuttX though). > Once it has been merged, both you and Miguel can continue the > development with PRs. But we need to find someone competent to provide > review. > > Greg > Maybe @yamt (see https://github.com/apache/incubator-nuttx/pull/183/commits/bc07eb680e09db3d07ddf0ab1da655661c066bef) can take a look also, as he created that config for CircleCI (and maybe integrate their changes later in Jenkins too). I will try to give a review asap. Cheers, Miguel
Re: [DISCUSS] Wrapping up the Workflow document
On Tue, Jan 28, 2020 at 6:01 PM Nathan Hartman wrote: ... > Please, feel free to improve the text. (Let us know if you need someone to > set up Confluence editing access for you.) > > Thank you very much, > Nathan > My pleasure. I have been busy the last couple of days, but I will try to help complete some of the sections, after reviewing any new comment/email about them. Cheers, /Miguel
Re: [DISCUSS] Wrapping up the Workflow document
On Tue, Jan 28, 2020 at 5:52 PM Gregory Nutt wrote: > We need to get you on the PPMC where it would be easier for you to make > contributions > Thanks, let's see :) I think for now is ok without becoming PPMC, but anyway I have sent the ICLA in case it could help to contribute in more areas.
Re: Jenkins CI
On Tue, Jan 28, 2020 at 8:58 PM Justin Mclean wrote: ... > You would need to submit an ICLA. The CCLA is for more for your employer > benefit not ours. Hi Justin, sorry for the delay in answering. In that case there is no need to delay I guess. I have just sent the ICLA and waiting for confirmation about it. Thanks, /Miguel
Re: Jenkins CI
Hi, after some research about new releases of Jenkins, it seems that my proposal can be improved towards a simpler setup, so I modified it. I have included some code as pull requests that could be useful to discuss upon: https://github.com/apache/incubator-nuttx-testing/pull/2 and https://github.com/apache/incubator-nuttx/pull/174 The actual logic from Haitao scripts can be used inside the pipeline definition, so we could have a mostly functional "Github pull request based workflow", once a comitter with enough rights could create the jobs in builds.apache.org (I am still waiting confirmation for the CCLA, sorry, and anyway the access is not granted). Cheers, Miguel On Tue, Jan 21, 2020 at 1:11 PM Miguel Ángel Herranz wrote: > > > We probably can get infra to add you, submitting an ICLA might help as > that could be used to create an account. > > Thanks, good to know. I am asking my employer about including me in a > corporate one (CCLA [1]), but if it is too much paperwork I guess I can > just submit the ICLA. Let's see. > > [1] https://www.apache.org/licenses/contributor-agreements.html > > On Tue, Jan 21, 2020 at 2:01 AM Justin Mclean > wrote: > >> Hi, >> >> > Oh, I did check that wiki page but I was not aware that it was a >> > requirement to be a comitter or PPMC member to have an LDAP account, >> but it >> > makes sense, thanks anyway. >> >> We probably can get infra to add you, submitting an ICLA might help as >> that could be used to create an account. >> >> Thanks, >> Justin >> >> >>
Re: [DISCUSS] Wrapping up the Workflow document
Hi Nathan, I reviewed the document and added some inlines comments (not sure if they are not recommended for use though). I haven't edited/added any text though, but I will be glad to help if needed. Cheers, Miguel On Fri, Jan 24, 2020 at 4:29 PM Nathan Hartman wrote: > Hi all, > > The proposed Workflow document (see [1]) has a substantial amount of > good information in it. > > It is not yet 100% percent complete: > (1) There are several "REVISIT" notes throughout. > (2) A few sections toward the end aren't written yet. > > However, by the 80-20 rule, I think the most important parts have been > written, even if some things are rough around the edges. > > I have had a LOT of overtime at my $dayjob lately, which has kept me > from working much on NuttX. However, I don't want to let the workflow > fall by the wayside. I would like to get it wrapped up so we can vote > on it soon. > > Please, could we have some willing volunteers proofread the document, > fix some of the "REVISIT" parts, and help push this document that last > little bit to get it into a "shippable" state? > > [1] > > https://cwiki.apache.org/confluence/display/NUTTX/Code+Contribution+Workflow+--+Brennan+Ashton > ) > > Thanks, > Nathan >
Re: Jenkins CI
> We probably can get infra to add you, submitting an ICLA might help as that could be used to create an account. Thanks, good to know. I am asking my employer about including me in a corporate one (CCLA [1]), but if it is too much paperwork I guess I can just submit the ICLA. Let's see. [1] https://www.apache.org/licenses/contributor-agreements.html On Tue, Jan 21, 2020 at 2:01 AM Justin Mclean wrote: > Hi, > > > Oh, I did check that wiki page but I was not aware that it was a > > requirement to be a comitter or PPMC member to have an LDAP account, but > it > > makes sense, thanks anyway. > > We probably can get infra to add you, submitting an ICLA might help as > that could be used to create an account. > > Thanks, > Justin > > >
Re: Jenkins CI
> I can give people Jenkin admin access but they need to be in LDAP. [1] that means you need to be a committer or PPMC member. Oh, I did check that wiki page but I was not aware that it was a requirement to be a comitter or PPMC member to have an LDAP account, but it makes sense, thanks anyway. In that case I will just try to set up a local Jenkins server for testing, using plugins usually available by default and doing a pull request against incubator-nuttx-testing so other people can review or even adapt it and try in the real server.
Re: Jenkins CI
> You should have permission now. Thanks Justin. I have created a draft proposal (postfixing my name like the other one by Brennan Ashton, in case others want to add another proposal from scratch, but contributions are welcome): https://cwiki.apache.org/confluence/display/NUTTX/Continuous+integration+--+Miguel+Herranz It is incomplete still, of course, expecting some feedback. Also, a request: maybe it would be good to test if Jenkins in builds.apache.org support pipelines or any Jenkins plugin is missing, so if I could get the rights to access create a proof-of-concept pipeline to check everything is available. Cheers, Miguel
Re: Jenkins CI
Hi Justin and Haitao, so good you liked the idea. I tried to create the page in Confluence, but it seems I don't have enough rights. Can someone else give me the needed rights to add it or create and give edit permissions? My Confluence user in cwiki.apache.org is 'maht' Cheers, Miguel On Fri, Jan 17, 2020 at 4:40 AM Haitao Liu wrote: > Thanks Greg and Abdelatif. I am going to use the Pinguino toolchain at > first to let it work, and try the buildroot way later with free time. > > Gregory Nutt 于2020年1月16日周四 下午10:26写道： > > > > > > In addition, I notice http://www.nuttx.org/Documentation/NuttX.html > say > > gnu > > > toolchains based on buildroot > > > for many architectures. > > > > There are configurations at > > https://bitbucket.org/nuttx/buildroot/src/master/configs/ for: > > > > avr-defconfig-4.3.3 > > avr-defconfig-4.5.2 > > > > I am not sure of the state of those, however. > > > > > So for AVR gcc toolchains, I just found > > > > > > https://www.microchip.com/mplab/avr-support/avr-and-arm-toolchains-c-compilers > > > , > > > but not right for nuttx. Do I need to build based on builtroot? > > > > I don't know about this, but the toolchain from AtmelStudio is quite > nice. > > > > Notice that there is a build test for AVR (and MIPS) here: > > https://bitbucket.org/nuttx/tools/src/master/buildtest/ . If you look in > > doavr.sh, you an see that the test is configured to use the AtmelStudio > > toolchain. > > > > I am not certain if AVR will build without AtmelStudio. I think it > > expects some resource from AtmelStudio, but I might be wrong. > > > > Greg > > > > > > > > > > >
Re: Jenkins CI
TLDR: I think we need to make a "CI requirements and design" Draft Proposal (https://cwiki.apache.org/confluence/display/NUTTX/Draft+Proposals) --- Hi, I think that in order to get a working CI as soon as possible we need to please define a plan in a coordinated way. I think "perfection" is the enemy of "good", so I propose to have two stages: 1st stage: Defining a "Good CI" to be ready ASAP, so we could include it as a requirement for the also-to-be-defined workflow (CI should go in the Criteria for acceptance of workflow https://cwiki.apache.org/confluence/display/NUTTX/Code+Contribution+Workflow+--+Brennan+Ashton#CodeContributionWorkflow--BrennanAshton-CriteriaForAcceptance ). 2nd stage: Defining a "Perfect CI" to be agree to future nice-to-have-but-not-essential parts of the CI, that can be extended later upon the basis of "Good CI". So, from now on I will talk about "Good CI" exclusively. I appreciate Haitao is preparing the scripts, but as being said, it should not be the only one defining it. I don't know Haitao and probably nobody knows me, so every decision is better done by the merits of the proposal itself and not the person. I think that for a "Good CI" we need to understand the needs for NuttX: some feedback from the people that have worked all these years with NuttX about how to test it and what to test to run to conclude if something has quality or not. So I would suggest to create a "Draft Proposal" ( https://cwiki.apache.org/confluence/display/NUTTX/Draft+Proposals) called something like "CI requirements and design" where to develop that proposal. In the proposal we should address what parts of the features of a CI we want for the "Good CI" among the ones one could expect in a CI (for example, from the ones described in https://en.wikipedia.org/wiki/Continuous_integration#Common_practices) . Despite being commong for CIs all over the world, we need to agree on a minimum. But more important even, as I said, is to know what is needed _specifically_ for NuttX. For example: - What sanity checks are mandatory? what tools are we going to use to enforce them? - What architectures we want to include: all of them? (list them) some of them can be deferred until "Perfect CI"? do Apache Jenkins have the machine for that architectures? if not ... are there emulators? and the toolchains? options, pros/cons - What are the stages of a NuttX build? What are the artifacts generated? can these artifacts be reused in different tests? how does that relate to the number of jobs? is the job pipeline a line, or some artifacts are fed to parallel jobs with different configurations to accelerate the testing? - Are we going to test the interface with the applications? Are the applications being tested in the CI pipeline using end-to-end tests? - What is the general CI "pipeline/job graph"? What are the triggers? What are the outputs? - What are the minimum level of logs to help debugging problems? - What artifacts will be used for the Releases? I have little experience with NuttX still, so more experienced people need to address these things. This document would include little or code at all. Having this document would be of great use, since we all could agree on a CI structure, even non programmers, and then, Haitao, myself or any other could help on the nitty-gritty details of translating that to specific CI (Apache infra Jenkins by now). Also, of course Haitao and others can continue to develop the implementations in parallel. The more stones he remove from the path the better. And sorry for the long email :) Miguel On Thu, Jan 16, 2020 at 3:26 PM Gregory Nutt wrote: > > > In addition, I notice http://www.nuttx.org/Documentation/NuttX.html say > gnu > > toolchains based on buildroot > > for many architectures. > > There are configurations at > https://bitbucket.org/nuttx/buildroot/src/master/configs/ for: > > avr-defconfig-4.3.3 > avr-defconfig-4.5.2 > > I am not sure of the state of those, however. > > > So for AVR gcc toolchains, I just found > > > https://www.microchip.com/mplab/avr-support/avr-and-arm-toolchains-c-compilers > > , > > but not right for nuttx. Do I need to build based on builtroot? > > I don't know about this, but the toolchain from AtmelStudio is quite nice. > > Notice that there is a build test for AVR (and MIPS) here: > https://bitbucket.org/nuttx/tools/src/master/buildtest/ . If you look in > doavr.sh, you an see that the test is configured to use the AtmelStudio > toolchain. > > I am not certain if AVR will build without AtmelStudio. I think it > expects some resource from AtmelStudio, but I might be wrong. > > Greg > > > > >
Re: Jenkins CI
On Fri, Jan 10, 2020 at 2:12 PM Justin Mclean wrote: > Hi, > > > One request: I checked the Jenkins and it seems that the job > configuration > > is not accessible without having an account, and you have to be a > committer > > and must be a member of the hudson-jobadmin group > > I can add people to that as needed. I created jobs and given access o > people in the past. > Ok, I would like to be added in order to collaborate, if other people involved agree. Not urgent by now. > Could it be possible to make the configuration read-only, even without > > account or with a restricted one? > > I'm not 100% sure that can be done, but you have a history of changes in > Jenkins so it’s not hard to manage. > Ok, if not possible, I guess is ok to give a normal account, and just rely on accountability of changes. Another request: is there any document describing the things we are expecting to check in the CI for each commit/PR and for each release? I guess that at the very least we want to test the tests in https://github.com/apache/incubator-nuttx-apps/tree/master/testing and in the examples, for the most important architectures. Thanks, Miguel
Re: Jenkins CI
On Fri, Jan 10, 2020 at 4:24 AM Haitao Liu wrote: > Thanks Miguel share your approach. We'll look into it, the aim for NuttX > is keep nuttx/ and apps/ user-end projects > clean and do not mess up with CI stuffs. Thanks for the feedback. And yes, separate repo for CI stuff is a reasonable approach. And now prepare to use Apache > Infra Jenkins CI firstly. > Yes, probably the best approach is to prepare CI jobs in a traditional way, and once it is almost stable and you and people involved have less work with it start to translating it to something versionable as the Jenkins Job Builder definitions. This way the transition curve is easier and avoid interfering with the priority goal of having a working CI. One request: I checked the Jenkins and it seems that the job configuration is not accessible without having an account, and you have to be a committer and must be a member of the hudson-jobadmin group (according to https://cwiki.apache.org/confluence/display/INFRA/Jenkins#Jenkins-HowdoIgetanaccount?). Could it be possible to make the configuration read-only, even without account or with a restricted one? This way it would be easier for me and other people to help and review job definitions and I could help to create job builder definition templates in the meanwhile, based on the jobs the committer is creating manually with the UI.
Re: Jenkins CI
Hi, I have configured Jenkins CI pipelines in the past and aside the scripts running in the jobs, managing the jobs definition in the UI can be a pain, since is easy to apply untrackable changes that impact the pipeline. So I am wondering if you are going to use any kind of solution like Jenkins Job Builder (https://docs.openstack.org/infra/jenkins-job-builder/). It was created by OpenStack to manage Jenkins jobs. Basically, aside of bash/python/etc scripts to be run by the jobs, you define your pipeline/jobs using YAML files that could be easily versioned in a public repo, and put your Jenkins user/pass in a private file. Then invoking the tool that will contact the Jenkins service and setup everything as declared in the YAML files. This way to replicate all the pipeline and jobs to another Jenkins instance should be easy. I successfully used that approach in previous projects and I think is really useful to managing complex situations. If there is interest I can help, since I have previous experience with it. Cheers, Miguel On Thu, Jan 9, 2020 at 10:32 AM Haitao Liu wrote: > Or https://github.com/NuttX/tools would be better to hold the jenkins CI > scripts in this period? > > Haitao Liu 于2020年1月9日周四 下午4:08写道： > > > In my opinion, Jenkins CI scripts could be version controlled firstly > > locally or my github. > > Once the whole process is functional, I think the best place is to put > > them into nuttx testing project if available. > > > > Adam Feuer 于2020年1月9日周四 下午12:51写道： > > > >> Haitao, > >> > >> Will the Jenkins CI scripts live in version control somewhere? > >> > >> cheers > >> adam > >> > >> On Wed, Jan 8, 2020 at 7:44 PM Haitao Liu wrote: > >> > >> > Ok, get it. Thanks! > >> > > >> > Justin Mclean 于2020年1月9日周四 上午11:07写道： > >> > > >> > > Hi, > >> > > > >> > > > Thanks, I think we can firstly get start in Linux and Windows. > Then > >> > > > consider how to setup Mac later, donate slave or other. > >> > > > Two other question, nuttx uses some cross compile toolchains etc. > In > >> > > > addition, Cygwin or MinGW is in need once nuttx built under > Windows > >> > > slaves. > >> > > > Should we ask for Apache Infra to help install in slave machines? > >> > > > >> > > If it's not already installed yet you need to ask for it to be > >> installed. > >> > > Best way to do is to raise an Infra JIRA. > >> > > > >> > > Thanks, > >> > > Justin > >> > > >> > >> > >> -- > >> Adam Feuer > >> > > >
