Hi everyone, I have for some time been considering the state of the embedded Linux community, and some of the challenges facing us. As an embedded Linux developer myself, I am interested in improving both my own working conditions and my efficiency and effectiveness in my own projects. But also I think we could be having more fun, and learning and growing more if our community were stronger.
In this message I am writing some of my own ideas, and would like to solicit feedback on these and get other people's ideas as well. I think the greatest challenge of developing embedded products using open source software is fragmentation. Fragmentation is not new to open source, or software in general. I have been called out for saying that the issues that embedded contributors have, relative to mainlining code, are different from those of other developers. That is, I've asserted that embedded is somehow "special", and people (like Greg KH) have disagreed. I think every group developing Linux has to identify those areas of commonality that they can work on together, and separate them from the areas of divergence, where there is not enough common ground that it makes sense to collaborate or even attempt to push things upstream. In this regard (to disagree with Greg), I think embedded *is* different than other Linux use areas. I think the very nature of embedded, which at it's core consists of making customized solutions, tends to run counter to the nature of open source, which tries to leverage commonality and sharing. And so I worry sometimes about how to increase our commonality, and in particular how to encourage more sharing and discussion within our community. As a concrete example, I have been working lately on 4k stacks for the ARM kernel. The particular use case for this is very special within Sony -- or at least it appears to me to be a rather uncommon usage scenario. But here's the thing... How do I know there's not some other developer somewhere struggling with something similar? - maybe not the identical problem, but one where 4K stacks would help their situation? I attempted last year to submit my ARM 4K patches upstream. However, I got some pushback, and I didn't really see others with similar use cases that required this, and so I didn't push it as hard as needed to get it mainlined. There will always be some weird things that really shouldn't go upstream (and should be kept in someone's local repository -- something that if they decide is needed can be maintained out-of-tree.) However, I've been bothered by the idea that possibly I just didn't connect with the right people or find those who might also benefit from this work. And this led me back to thinking about the community. One of the foundational premises of open source is that of using network effects to overcome shared challenges. Eric Raymond's famous "to enough eyes, all bugs are shallow" comes to mind. If one can find enough people with the same interests, open source offers a way to collaborate on solutions rather than create and maintain them all by oneself. I'm worried that even after years of trying to build community ties, through things such as conferences, summits, the elinux wiki, this mailing list, and the CE Linux Forum (and it's successor the CE Workgroup), the embedded Linux developers still haven't "come together" as a community as well as they could. To be honest, I think that many of our previous efforts, while good in their own right, have not caught fire the way we had hoped. For example, elinux.org has some great areas dedicated to BeagleBoard and Raspberry Pi, and the section on boot time has lots of good material. But the content in some other areas is outdated or otherwise lacking. We have had occasional bursts of energy at improving the site (sometimes spurred by a contest or conference event), but overall we have not gotten a sustained population of contributors to the site. eLinux.org is just one example. There are numerous other examples of places where there should be vibrant communication of embedded Linux information and discussion of issues. Among the best things we have, I believe, is our conferences, (ELC, ELCE, the Japan Jamborees, and various other events). I'm always excited to meet with other developers and hear about what they're working on. But only a small number of developers, out of the literally tens of thousands of embedded Linux developers worldwide, can attend those. And even then we only have room for 40 to 50 presentation topics - which are, themselves, pretty specialized. We try to spread the information, through things like publishing the presentations, and videos. But, as worthwhile as those are, it doesn't increase the interactivity of the community. So, I have a few ideas I'd like to float out there: 1) I'd like to resurrect this list, and encourage people to use it to discuss general topics of interest for embedded Linux. I will be trying over the next few months to discuss things related to system size on this list, and in particular the projects I'm working on to reduce kernel size. I don't want this list to become LKML, where the traffic is so high that everyone ignores it. Hopefully we can increase the traffic, but still keep it limited and relevant for those who are working with general embedded Linux problems (you know, the classics: size, boot-time, file systems, real-time, security, graphics, embedded distros, etc.) 2) I'd like to try to get more focused groups at upcoming conferences. Specifically, at ELC Europe and ELC I'd like to try to steer people with common interests toward each other. I'm not sure the exact mechanism to use for this. Other sub-communities tend to use mini-summits for this. The problem with these is that they are limiting in terms of number of participants, and you kind of have to know ahead of time who to invite. I'd like to use a more open approach. Sometimes just identifying the people who attended a particular presentation will help you find those who are interested in or actively working on a particular topic area. The "hallway tracks" of conferences are often where the most interesting and fruitful discussions occur. Somehow I'd like to introduce people of like interests, and let them initiate their hallway tracks on their own, without a lot of pre-invitations or formal setup. Does anyone have ideas on how to do this? To be less abstract: I'm leaning towards having a "size" summit at ELC. What's the best way to get people there who are (still) interested in reducing the kernel and user-space size for embedded Linux systems? 3) I'm looking for ways to invigorate the elinux.org wiki site. I have a few projects that I'll (hopefully) be announcing in the next few weeks (and, also hopefully, making a major push for at ELCE in about 6 weeks). Finally, 4) I'm also looking for ways to increase the involvement in and use of the LTSI kernel. I have a sense that it is starting to get visibility and usage, but I have no solid indicators that tell me yet whether a community is building or not around that kernel version. I'll start working on these by making a concerted effort to invite people to the celinux-dev mailing list. For historical reasons, it doesn't have the most relevant name for what I'd like to achieve. But it's a place to start (and, thankfully, it's administered by someone besides me!). Also, I'll try to post more often on this list myself, to start to encourage the type of communication I'd like to see more of in the future. Am I the only one who feels like this? (If so, it may be a frustrating couple of months, with a feeling like I'm pushing on a string.) Do you have ideas for how to build and improve the embedded Linux community? Or, do you have ideas for what might be blocking it? I've got lots more to say, but this is already long enough. (Just wait until you hear my philosophy of kernel size reduction... ;-) Thanks for reading this far... Let me know what you think. -- Tim P.S. Please encourage your embedded Linux developer buddies to subscribe to this list!! http://lists.celinuxforum.org/mailman/listinfo/celinux-dev ============================= Tim Bird Architecture Group Chair, CE Workgroup of the Linux Foundation Senior Staff Engineer, Sony Network Entertainment ============================= _______________________________________________ Celinux-dev mailing list Celinux-dev@lists.celinuxforum.org https://lists.celinuxforum.org/mailman/listinfo/celinux-dev
