Re: Roadmap 2019
Dear list, To reflect on this year's progress, its been a pleasure to watch the Genode desktop evolve from a ramshackle init configuration into Sculpt and the launcher system. The lack of interest from the outside world is disappointing, but I blame a pessimistic zeitgeist rather than the lameness of Genode. As for concrete goals for next year, I have one laptop that I use exclusively with Sculpt, but I also have a NixOS VPS that I use as my always-on OS. Therefore I would like a headless Sculpt that I can use remotely. For this to happen I would imagine a nested sculpt_manager so that I could share the same machine with friends, a Curses-like implementation of menu_view, and some secure networking tunneling. Its my perception that the greatest impedement to new users and contributers is the singular C++ system API. Perhaps additional Sculpt shells could be implemented in interpreted languages such as Python or Lisp. I've known a few "power-users" that would appreciate booting into their favorite language intepreter, but couldn't care less about how to use Bash. Python seems the surest choice, the digital artists I know work with Python and there was a time when I used IPython daily. As for my personal roadmap, in the past few weeks I've begun work on a distributed non-hierarchal storage layer. The intention is to support a subset of the de facto file-system semantics with quick composition using the formalism of set logic. This is a pivot of my roadmap from a year ago and reuses code and ideas from another storage project of mine, so progress is swift. The use-case is distributed and redundant storage for things like my permanent music collection as well as the package depot. Best wishes for next year, Emery ___ Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
Re: Roadmap 2019
Congratulations on a highly successful year! Thanks for this opportunity to chime in. There's a lot to process here; for brevity's sake I will only share a few initial thoughts at this time. On 12/23/18 2:46 PM, Norman Feske wrote: Hallo everyone, hereby, I'd like to follow up on our tradition to discuss Genode's road map on the mailing list. Let's take the turn of the year to recapture the events of 2018 and make plans for the next twelve months. Please feel welcome to chime in! By mid of January, I'll finalize Genode's official road map and would like to take your input into account. [snip] For me personally, this ride was certainly the most rewarding period in Genode's history so far. Now, when looking at the result, I am overwhelmed about what we achieved together! Whenever I have the chance to showing off Sculpt running on my laptop, Sculpt doesn't fail to impress. I completely agree with this assessment. I think the Year Of Sculpt has tied enough loose ends together that the framework has reached the "plateau" phase, and the focus now shifts to building upon it. That said, during the course of the year, this positive sense of achievement eventually blended with the rather dull feeling that our hard work remains largely unnoticed outside the inner circle of Genode enthusiasts. The public at large remains quite indifferent (e.g., I was unable to capture the interest of FOSDEM to feature a talk about Sculpt OS on their main track). For a long time, I tricked myself into believing that once we overcome all technical road blocks, Genode will eventually become widely recognized automatically. There was always a technical challenge to take on. With Sculpt, we have reached a point where this excuse doesn't hold anymore. There is no technical piece missing. I agree that there is no technical piece missing. But as an old refugee from the Amiga, etc., I'm painfully aware that the best technology doesn't always get the mind- or market-share. (The FOSDEM news is surprising, though.) This leaves me with the question of how to make Genode relevant at a larger scale? Since this is not a technical question, I admittedly struggle to find a good answer. When thinking of the overall leitmotif for 2019, I always come back to this question. Plan for 2019 I see three directions to help Genode to become more widely recognized. [snip] 1) Widening the audience of Sculpt OS Sculpt works well, but it is arguably still too hard to use for non-technical users and it lacks a lot of software that users take for granted today. Consequently, we should improve its ease of use. In particular, I'd love to explore the more consequent use of Sculpt's component graph as an intuitive user interface for both the composition and the configuration of components. The ultimate goal would be to eliminate the need for editing any text files. With respect to software availability, I have high hopes in the Genode SDK that we introduced in Genode 18.11. Let's follow this route further. I don't have any big ideas on this subject, but I just want to support the idea of focusing on the Component Graph. It is a powerful (and unique) feature, and I can't wait to use it for configuration also. 2) Fostering the community spirit around our project Our project enjoys a healthy community of users and contributors. But the community is rather scattered and the various groups work pretty much in isolation. As a result, the work of one group is often invisible to the others, not to speak of any public visibility. Could we possibly establish an instrument that would help each participant to gain more visibility and thereby help the community at large to become more relevant? I'll come up with a concrete proposal for such an instrument soon. This should really help us hit critical mass. It sounds like some interesting (and unexpected) ideas are percolating already! One technical element may be a feature of Sculpt that would allow the user to browse the software depots provided by various community members and install/deploy packages by just a few clicks. This would vastly ease the discoverability of the available software and highlight the roles of the respective developers. I think this is a really important idea also. [snip] (I need to put more thought into the questions at the end, so I'll respond to those separately.) As a final thought, I'm really looking forward to what both Genode Labs and the community do in the upcoming year, and to being part of it! Happy Holidays! John J. Karcher devu...@alternateapproach.com ___ Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
Re: Roadmap 2019
*Desktop Environment* I also see having a desktop environment as priority, and I have already made some progress in that direction. I have my plans along those lines, and if anyone wants to discuss details, I'd be happy to do so in another thread on this mailing list. However, I have run into some hurdles that I could use some help with. In particular, to enable easy IPC with libc-based applications, a named pipe (FIFO) VFS plugin would be extremely useful. I wrote one myself, only to discover that the VFS and File_system interfaces do not adequately support partial writes, as needed by such a plugin. With those interfaces reworked, I would happily modify and provide my VFS FIFO plugin code, as well as my Libc pipe plugin that uses it as the backend. My other main technical hurdle for creating a desktop environment has been converting the init component into a library to allow for easy customization beyond what is possible through configuration files alone. The current method of writing a monitor component to read init's state and modify its configuration works fine for simple cases, but it has performance penalties that can range from mild to severe. The configuration parsing overhead grows with the number of child components, which isn't ideal, but the real show-stopper that I see is dynamic quota management. Each quota upgrade has to wait for a new report, and reports happen at regular intervals, usually every second. Thus multiple quota upgrades, e.g. from opening one or more new web browser windows/tabs can take several seconds. With an init library, such upgrades could be handled very quickly. I can handle this particular issue myself as soon as it becomes a high enough priority. *Hardware Support* Aside from a desktop environment, hardware support is another likely show-stopper for new users. I've been working on this myself, and hope to be able to make Genode/Sculpt usable on my new desktop computer in 2019. I accept that support for my hardware is mostly my issue, but improvements in debugging tools and documentation would help a lot with this. *Debugging* As for debugging tools, I suggest building both remote (serial and/or (W)LAN) and on-screen GDB component debugging scenarios. This would help with many other goals, and might also help make Genode more attractive to developers. *Package Management* I'd like to second and add to what ttcoder said about package management. It would be very helpful to provide a way for Sculpt users to discover and add depots. This would increase usability, and would encourage developers outside Genode Labs to write and port more software for Genode. Currently, depots don't provide package or launcher lists. Ideally, I would like to see Sculpt have an option to fetch the latest launchers from the repositories selected by the user, allowing them to easily discover components. This would also fix the issue I've seen with Sculpt launchers expecting package versions only available locally on the build machine. Currently, when I modify a component used by Sculpt and update its recipe version, Sculpt expects my new version to be available from depot.genode.org. If a launcher list is fetched from the Genode Labs depot (possibly with a correct default launcher list provided in the Sculpt image), this problem will be solved. *Performance* All current Genode scenarios (AFAIK), including Sculpt, run almost everything on a single hardware thread. An SMP balancing component would dramatically increase performance (and likely responsiveness) on modern multi-core CPUs. If nobody else gets around to it first, I'll probably write such a component, but if anyone else here more familiar with Genode's scheduling mechanisms is interested enough, they can probably write a much better implementation than I would. *Summary* In short, I'd like to see the following things on the 2019 roadmap: 1. Rework VFS and File_system interfaces to support partial writes, as needed by FIFOs 2. Interactive local and remote GDB debugging scenarios for real hardware 3. Depot and Launcher discovery for Sculpt 4. SMP balancing component 5. Improved documentation We don't all see eye-to-eye on how a desktop environment should operate, so I'm particularly interested in getting help with (1) so I can make more progress on writing my own, and with (3) so I can make it easily available so others can try it out and give me feedback. On Mon, Dec 24, 2018 at 8:42 AM wrote: > > As a newcomer I should probably step up to the plate, while the 'first > impressions' are still fresh in my mind. > > I'm still making baby steps; but based on past experience I should hit the > ascending part of the learning S-curve soon and become > productive. > Still have tons of documentation to read before I'll feel half-literate > about Genode (it seems to be made of just a handful of ground- > breaking principles, but in practice they have such far-reaching > consequences that it takes some time to get "fluent" with al
Re: Roadmap 2019
As a newcomer I should probably step up to the plate, while the 'first impressions' are still fresh in my mind. I'm still making baby steps; but based on past experience I should hit the ascending part of the learning S-curve soon and become productive. Still have tons of documentation to read before I'll feel half-literate about Genode (it seems to be made of just a handful of ground- breaking principles, but in practice they have such far-reaching consequences that it takes some time to get "fluent" with all ramifications). Hopefully some of the below makes sense anyway. This much is sure -- as we say in french, if Genode didn't exist, someone'd have to invent it *g*. My whole C.S. perspective (and thus also this message) is haiku centric. So each question & concern will be addressed from that perspective. Hopefully this will bring value to the discussion in at least a few of the cases, if not all. Criticism welcome. Not much I can add (other than a deep sigh *s*) as to why people don't adopt superior technology in higher numbers.. As a frequent 'early adopter' type I've seen the same thing over time. People stick with the familiar as being "good enough".. Been guilty of that myself too. To be fair though, people who download and run Scult/VC currently are greeted with the leitzencentral, not with a desktop (or what in Haiku I call "a userland"); so the Genode team would need to develop a 'userland' on top of leitzencentral.. In order to go beyond the technological demo, and instead produce a general-purpose OS demo. More on that below. I'll now 'do my part' with Genode though: will open a micro-blog in a few days about my technical forays into Genode, my mistakes and misunderstandings and how I solve them.. So that others will save time. Including some additional thoughts on this email thread -- to help keep this message short(er than a novel). Once I have more exciting news items in store, I'll give my micro-blog wider circulation, on the haiku forums (they don't have a gigantic readorship, but they are very enthusiastic early-adopter types so likely worth the effort). Waiting until I announce "our flagship app runs on Genode" will carry more weight. When doing (low key) advocacy in haiku forums, I'll leverage the cultural aspect: in the haiku crowd, they/we adopted Neil Stephenson's saying that we're using the "batmobile" of operating systems: I'll build on that by saying Genode is potentially the "UFO" of operating systems, out of this world :-) Will give back to the Genode community probably in a similar way I did with Haiku over the years (tickets e.g.). If I had to help out with the "challenges" page later, I suppose it might be with the MAID-SAFE part (decentralized internet). But even if within the edge of my abilities, I would have to get a lot of ducks in a row first. Their litterature mentions developing in something called "electron" (?) whereas I'm most productive in C++ coding against my beloved "Interface Kit". *** Finding the way: > The unconquered design space seemed vast, which was > both exciting but also - at times - a bit paralyzing. This might relate to a similar topic I have in mind these days: if e.g. the genode team does not come up with an "official" desktop but leaves it to the community to do (several) implementations, we might end up with the fragmentation that is prevalent in the Linux world: there are many "distros", which I hear are more or less (sometimes "less") compatible with each other. The way this decision making difficulty has always been addressed in the Be/Haiku world is something like, "choose reasonable defaults", "allow some (limited) easy customisations", and "keep power-user customisation accessible but under the hood". On a related note, I've also noticed over the years that OS designers are often bad application designers, and vice-versa. I guess the point I'm trying to make is something like, "if you want a rich userland, make an SDK that is a joy to use and help app developers help you" :-) Of course any SDK design decision has to be weighted with the absolute Genode priorities: security, stability, separation. * Concrete ideas for "spreading the word" on packages: The single biggest issue I have with the 'depot' system and packages in Genode, is the lack of an introductory paragraph, explaining what each package is about. Small potatoes, right? Power users probably know software names by heart in their sleep? But there are lots of intermediate-style users who don't. It does make a difference. Do note -- said texts can be simply copy-pasted, as they are in the example below with haiku recipes. There's no word-smith work involved at all. Just a matter of extending the depot format with a new field, and pasting pre-existing texts into that field. Especially important, taking into account that the genode depots are destined to grow into the hundreds of packages (I'd hope
Re: Roadmap 2019
Hi Norman and the whole Genode team, First, I recognize the feeling that you are enthusiastic about your work and nobody seems to care. And how devastating that can feel. I know that feeling from my attempt at designing and promoting my authentication protocol that (I believe) could decimate most phishing and identity theft attacks. I got so disappointed at the lack of response that I disabled web statistics so I didn't have to see how little visitors my site got. Now I've more or less given up on it. Yet every time I see a news article about phishing or stolen (and abused) passwords I'm reminded that I came up with something that could have (helped to) prevent it. Second, it's hard to create a new platform from scratch. As a rule of thumb, for any new invention it takes roughly 15 years from invention to market-readyness. So far, Genode Labs succeeded where others gave up. Thanks for not giving up. Third, it's very hard to bring innovation into the IT-world. The IT community is very conservative. Sandboxing to contain viruses is something that HP-Labs did 15 years ago for Windows XP [1]. Regrettably HP didn't succeed in selling it widely. Just last week, Microsoft has added a sandbox to Win10 in an attempt to contain malware. An example of the conservatism: I explained some ideas of Genode (micro-kernel, sandboxing, separation, pola) to a 30yo manager of a software development company that advertised itself for writing secure software. He found the ideas interesting but did not want to spend time to research it. Not even reading on the topic. He said it was "academic stuff that eventually shows up in Linux". I couldn't convince him that it might be a good idea to be ahead of the curve. He was risk averse, as are most IT-people. Genode needs to find people willing to take a risk and reap the reward. Every time I read about crypto malware holding a user or company hostage for ransom I'm reminded that had they used Genode it could have (helped to) prevent it. I believe there is a huge market out there. The difficulty is getting there. I hope my answers to this mail give some ideas how to proceed. Now about your goals and questions: > 1) Widening the audience of Sculpt OS > Consequently, we should improve its ease of use. For me a 1000 times this. To me, the learning curve of Genode is steep. The learning curve of (changing) Sculpt is steep too. Although I know most of the concepts of the platform, I find it still hard to develop in. Lately I went deep into changing the configuration of Virtualbox in Sculpt. What should be a few hours took me more than a week. And the result was still hacky :-( Although there are many examples, I'd like to see them listed from simple to complex, each explaining a concept of Genode, builing on top of the previous ones. It could grow into a Genode for Dummies. Other wish: please describe how I can enable all kinds of debugging modes. For example, when I make a XML syntax error in an init.config, the app doesn't start but the log remains quiet. Other example, at one point I added printf-logging to init-components to trace the routing decisions so I could check that my config was correct. It would be great if there was a config option for that. > 2) Fostering the community spirit around our project I think you have a nice (but small) community already, with your team ready to answer any question quicky. Just keep doing that. You've seen my recent write-up on adding a raw partition to virtualbox in Sculpt. I thought of making it a blog so others can learn from it too. (I'm not sure if that example makes a nice showcase of the ease of Sculpt :-). > 3) Marketing of Genode-based products I like the idea of cross-promotion between Genode and some companies that use it. Put some showcases on your site. Perhaps with a small testimonial from each company why they chose Genode. Given the impossibility of convincing (conservative) people why Genode is better than their current system, don't focus on technical details. Instead focus on concrete benefits for (end) users: - safe by design; - protects privacy; - little errors do not become catastrophes; - robust against malware; - no need for regular updates; - updates don't break existing functionality; - easy to use, also for non-computer users (see my dream system below). > * What are your ambitions for 2019? I still have my wish for running Genode on my server, running a webserver for static and dynamic content. I ran a static web site on Genode a few years ago. It lasted a week until it crashed due to a resource leak, (memory, VFS-file descriptors). I could not debug it, so the box runs Linux again. I'd like to run a (small) dynamic web site on it. I.e. start with a blog and comments section and private messaging. For parsing I use the a composable parser generator such as the Hammer library [2] from the Langsec [3] community. I think Lan
