Re: [rust-dev] real-time programming? usability?
On Sat, Oct 19, 2013 at 12:45 AM, Jerry Morrison jhm...@gmail.com wrote: On Fri, Oct 18, 2013 at 6:13 PM, Daniel Micay danielmi...@gmail.comwrote: On Fri, Oct 18, 2013 at 7:37 PM, Jerry Morrison jhm...@gmail.com wrote: (1) Rust has great potential for *real-time programming* as well as secure programming. Perhaps this just needs a library call to fork real-time threads that never do garbage collection. (Real-time means predictably meeting time deadlines, e.g. in avionics, video capture, medical x-ray, and robotics. Speed is important but the crucial point is reliably meeting deadlines. Garbage collectors, JIT compilers, and run-time optimizers are bad for predictability.) Maybe Rust already has this. What determines whether a thread-local managed heap uses reference counting? Ref-counting may be OK in a real-time thread if it doesn't deallocate a variable-length chain of nodes at an inconvenient time. Rust leaves memory allocation up to the user as much as C++. Most code can simply use unboxed values and lightweight references, with occasional usage of owned boxes (single-owner heap allocations, with lifetime tied to scope). Ideally, forking a real-time thread could ensure that (1) no GC pauses the thread just to find out there's no garbage to collect, and (2) something would catch the mistake if it ever calls code (directly or indirectly) that evolves to using GC memory. Rust doesn't currently have a way to forbid features only in some parts of the code but not others. Avoiding garbage collection as a hard requirement will likely mean using rust-core (or whatever it evolves to, maybe a special profile of the standard library). If you need shared ownership, it's available via the `Rc`/`RcMut` types. Every split of ownership is explicit via a `clone` call since moves are the default. There will be a task-local garbage collector providing garbage collected pointers, but it still needs to be implemented. Sounds good, although I don't understand the details. Does the program pick between GC allocation and ref counting allocation via the choice of pointer sigils? Via the choice of library calls? Reference counting is implemented as a library type, so you would make a reference counted allocation with `Rc::new(value)` and make an explicit reference count with `box.clone()` (the implementation: https://github.com/mozilla/rust/blob/master/src/libstd/rc.rs). There are managed pointers sigils for garbage collection (@, @mut) but I think consensus is trending towards replacing the syntax with a type similar to `Rc` allowing cycles and implicit copies. The compiler support could be exposed via attributes hooks like the operator traits, rather than syntax. The standard library implements N:M concurrency and exposes a blocking API for asynchronous I/O. I don't think it would be suitable for a real-time application, but you can avoid the standard library: https://github.com/thestinger/rust-core What's N:M concurrency? The concurrency model implemented in the standard library treats real threads as schedulers, and maps cooperatively scheduled tasks onto them. If the task doesn't make calls to functions with yields, it will monopolize a scheduler thread. It's a very good model for I/O or for high throughput on many CPU-bound batch jobs (https://www.threadingbuildingblocks.org/), but it's not going to be suitable for real-time needs. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] real-time programming? usability?
On Fri, Oct 18, 2013 at 11:02 PM, Daniel Micay danielmi...@gmail.comwrote: On Sat, Oct 19, 2013 at 12:45 AM, Jerry Morrison jhm...@gmail.com wrote: On Fri, Oct 18, 2013 at 6:13 PM, Daniel Micay danielmi...@gmail.comwrote: On Fri, Oct 18, 2013 at 7:37 PM, Jerry Morrison jhm...@gmail.comwrote: (1) Rust has great potential for *real-time programming* as well as secure programming. Perhaps this just needs a library call to fork real-time threads that never do garbage collection. (Real-time means predictably meeting time deadlines, e.g. in avionics, video capture, medical x-ray, and robotics. Speed is important but the crucial point is reliably meeting deadlines. Garbage collectors, JIT compilers, and run-time optimizers are bad for predictability.) Maybe Rust already has this. What determines whether a thread-local managed heap uses reference counting? Ref-counting may be OK in a real-time thread if it doesn't deallocate a variable-length chain of nodes at an inconvenient time. Rust leaves memory allocation up to the user as much as C++. Most code can simply use unboxed values and lightweight references, with occasional usage of owned boxes (single-owner heap allocations, with lifetime tied to scope). Ideally, forking a real-time thread could ensure that (1) no GC pauses the thread just to find out there's no garbage to collect, and (2) something would catch the mistake if it ever calls code (directly or indirectly) that evolves to using GC memory. Rust doesn't currently have a way to forbid features only in some parts of the code but not others. Avoiding garbage collection as a hard requirement will likely mean using rust-core (or whatever it evolves to, maybe a special profile of the standard library). So with a suitable library, real-time threads can avoid garbage collection pauses. Can it enforce linking to this library for all the code it calls, or else arrange for the standard GC memory allocator to fail!() if it accidentally called? If you need shared ownership, it's available via the `Rc`/`RcMut` types. Every split of ownership is explicit via a `clone` call since moves are the default. There will be a task-local garbage collector providing garbage collected pointers, but it still needs to be implemented. Sounds good, although I don't understand the details. Does the program pick between GC allocation and ref counting allocation via the choice of pointer sigils? Via the choice of library calls? Reference counting is implemented as a library type, so you would make a reference counted allocation with `Rc::new(value)` and make an explicit reference count with `box.clone()` (the implementation: https://github.com/mozilla/rust/blob/master/src/libstd/rc.rs). There are managed pointers sigils for garbage collection (@, @mut) but I think consensus is trending towards replacing the syntax with a type similar to `Rc` allowing cycles and implicit copies. The compiler support could be exposed via attributes hooks like the operator traits, rather than syntax. Nice. The standard library implements N:M concurrency and exposes a blocking API for asynchronous I/O. I don't think it would be suitable for a real-time application, but you can avoid the standard library: https://github.com/thestinger/rust-core What's N:M concurrency? The concurrency model implemented in the standard library treats real threads as schedulers, and maps cooperatively scheduled tasks onto them. If the task doesn't make calls to functions with yields, it will monopolize a scheduler thread. It's a very good model for I/O or for high throughput on many CPU-bound batch jobs (https://www.threadingbuildingblocks.org/), but it's not going to be suitable for real-time needs. Thanks. That makes sense. -- Jerry @1fish2 ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] real-time programming? usability?
On Sat, Oct 19, 2013 at 2:33 PM, Jerry Morrison jhm...@gmail.com wrote: So with a suitable library, real-time threads can avoid garbage collection pauses. Can it enforce linking to this library for all the code it calls, or else arrange for the standard GC memory allocator to fail!() if it accidentally called? There's no dynamic checking needed; you just need to include the #[deny(managed_heap_memory)] attribute in the crate file. (There's also a way to do the same thing with a command-line flag.) This will result in a compile-time failure if any code might use garbage collection (including code in external libraries that's called by your code). Cheers, Tim -- Tim Chevalier * http://catamorphism.org/ * Often in error, never in doubt Being queer is not about a right to privacy; it is about the freedom to be public, to just be who we are. -- anonymous, June 1990 ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] real-time programming? usability?
On Sat, Oct 19, 2013 at 2:36 PM, Tim Chevalier catamorph...@gmail.comwrote: On Sat, Oct 19, 2013 at 2:33 PM, Jerry Morrison jhm...@gmail.com wrote: So with a suitable library, real-time threads can avoid garbage collection pauses. Can it enforce linking to this library for all the code it calls, or else arrange for the standard GC memory allocator to fail!() if it accidentally called? There's no dynamic checking needed; you just need to include the #[deny(managed_heap_memory)] attribute in the crate file. (There's also a way to do the same thing with a command-line flag.) This will result in a compile-time failure if any code might use garbage collection (including code in external libraries that's called by your code). Cheers, Tim Fantastic. Thanks, Tim! For a program with some real-time threads plus other code like UI, I assume the other code can be in a crate that allows managed_heap_memory. -- Jerry ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] real-time programming? usability?
On Sat, Oct 19, 2013 at 5:43 PM, Jerry Morrison jhm...@gmail.com wrote: On Sat, Oct 19, 2013 at 2:36 PM, Tim Chevalier catamorph...@gmail.comwrote: On Sat, Oct 19, 2013 at 2:33 PM, Jerry Morrison jhm...@gmail.com wrote: So with a suitable library, real-time threads can avoid garbage collection pauses. Can it enforce linking to this library for all the code it calls, or else arrange for the standard GC memory allocator to fail!() if it accidentally called? There's no dynamic checking needed; you just need to include the #[deny(managed_heap_memory)] attribute in the crate file. (There's also a way to do the same thing with a command-line flag.) This will result in a compile-time failure if any code might use garbage collection (including code in external libraries that's called by your code). Cheers, Tim Fantastic. Thanks, Tim! For a program with some real-time threads plus other code like UI, I assume the other code can be in a crate that allows managed_heap_memory. -- Jerry The `managed_heap_memory` lint will prevent using any types containing managed pointers, but won't strictly prevent calling code using garbage collection. In a `#[no_std]` codebase, the compiler will prevent using managed pointers unless the hooks for them are defined. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
[rust-dev] real-time programming? usability?
Hello! While reading the Rust tutorial I'm really excited to see a new programming language that can replace C/C++ for performance-critical programming. C/C++ is very entrenched but extremely tricky and getting more complicated with each revision. The world needs a replacement that's not so perilous -- before it causes big disasters. I'd like to suggest some (hopefully useful) ideas. (1) Rust has great potential for *real-time programming* as well as secure programming. Perhaps this just needs a library call to fork real-time threads that never do garbage collection. (Real-time means predictably meeting time deadlines, e.g. in avionics, video capture, medical x-ray, and robotics. Speed is important but the crucial point is reliably meeting deadlines. Garbage collectors, JIT compilers, and run-time optimizers are bad for predictability.) Maybe Rust already has this. What determines whether a thread-local managed heap uses reference counting? Ref-counting may be OK in a real-time thread if it doesn't deallocate a variable-length chain of nodes at an inconvenient time. (2) I'd like to see programming language designers follow other domains by doing usability testing. As Alan Kay said, The users are not like us. If you're a language designer or implementer, you can be sure that few language users will have your deep understanding of it. Few programmers read the entire language reference manual or even learn all the core semantics. E.g. what percent of Java programmers know about precise exceptions and their implications for optimization? What percent can declare bounded generic types? What percent understand volatile and lock ordering? Many real-time programs are *safety-critical*, which has more or less the same language needs as security-critical: fewer failure modes, easy to understand and reason about, less error-prone, easier to learn, easier to document. We could measure this for a language by the number of traps, pitfalls, and puzzlers books. There are shelves of such books for C++ and one puzzlers book for Java. Can Rust get it down to a one-pager? Rust eliminates many C++ failure modes, including memory access errors and octal constants that look like decimals. Could it systematically eliminate the failure modes listed in C++ traps pitfalls books? Some examples: - Make the everyday int and uint types portable, that is, produce the same results on all platforms. int size does matter when it varies between 16 and 64 bits. If Rust needs an integer type that’s the size of a pointer, call it intptr_t and only use it for that special case. - Security holes happen whenever a program does exploitably unexpected things. When the unexpected results happen in rarely-tested cases like integer overflow, it’s bad for security. So consider arbitrary precision int. - Use postfix syntax for pointer dereference, like in Pascal: (~rect).area() becomes rect~.area() . That reads left-to-right with nary a precedence mistake. While Rust’s auto-dereference feature and type checker will sometimes catch that mistake, it's better to just fix the failure mode. All security holes in the field got past the type checker and unit tests. - Don’t let ; default its second operand. Require an explicit value, even if (). That fixes an opportunity to goof that might be frequent among programmers used to ending every statement with a ;. - AIUI, let mut x, y defines 2 mutable variables but |mut x, y| defines mutable x and immutable y. This is harder to learn and easier to goof. - Drop C-compatible struct layout support? Is such a goal practical considering that C struct layouts vary by CPU, compiler, and compiler switches (enum size, pointer size, integer alignment, bit-field padding, etc.)? C can’t reliably pass a struct to a dynamically-loaded module (that has burned me) or to another computer (that, too). A better approach is a library like Protocol Buffers or Cap’n Proto. Thanks for listening! -- Jerry ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] real-time programming? usability?
On 19/10/13 10:37, Jerry Morrison wrote:
* Use postfix syntax for pointer dereference, like in Pascal:
(~rect).area() becomes rect~.area() . That reads left-to-right
with nary a precedence mistake.
While Rust’s auto-dereference feature and type checker will
sometimes catch that mistake, it's better to just fix the failure
mode. All security holes in the field got past the type checker
and unit tests.
Do you realise that `~rect` means create an owned box [a pointer] that
contains `rect` and isn't not a dereference in Rust? (That is performed
by `*rect`.)
* Don’t let ; default its second operand. Require an explicit value,
even if (). That fixes an opportunity to goof that might be
frequent among programmers used to ending every statement with a ;.
`;` isn't a binary operator, and anyway `a; b` doesn't affect the
behaviour of `b` at all. Could you describe what you mean a little more?
(Also the type system means that writing `fn foo() - int { 1; }` (with
the extra incorrect semicolon) is caught immediately.)
* AIUI, let mut x, y defines 2 mutable variables but |mut x, y|
defines mutable x and immutable y. This is harder to learn and
easier to goof.
`let mut x, y` doesn't exist at the moment precisely because of the
confusion; one has to write `let mut x; let mut y;`. (Relatedly There is
a proposal to make `mut ident` a valid pattern, so that `let (mut x,
y)` is valid: https://github.com/mozilla/rust/issues/9792 )
Thanks for listening!
--
Jerry
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] real-time programming? usability?
* Use postfix syntax for pointer dereference, like in Pascal: (~rect).area() becomes rect~.area() . That reads left-to-right with nary a precedence mistake. While Rust?s auto-dereference feature and type checker will sometimes catch that mistake, it's better to just fix the failure mode. All security holes in the field got past the type checker and unit tests. Do you realise that `~rect` means create an owned box [a pointer] that contains `rect` and isn't not a dereference in Rust? (That is performed by `*rect`.) I would add that if pointer dereference were a suffix operator, it would have to be changed from * to (e.g.) ~. Why? Because if foo* means dereference foo then it becomes ambiguous whether foo * - bar means (foo*)-bar dereference foo and subtract bar or foo*(-bar) multiply foo by negated bar. Due to this ambiguity, it is difficult for a language to simultaneously have 1. Operators that can be prefix or infix, and 2. Operators that can be infix or suffix. To avoid the ambiguity, one can introduce an operator like ~ that is prefix or suffix but never infix. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] real-time programming? usability?
On Fri, Oct 18, 2013 at 6:13 PM, Daniel Micay danielmi...@gmail.com wrote: On Fri, Oct 18, 2013 at 7:37 PM, Jerry Morrison jhm...@gmail.com wrote: (1) Rust has great potential for *real-time programming* as well as secure programming. Perhaps this just needs a library call to fork real-time threads that never do garbage collection. (Real-time means predictably meeting time deadlines, e.g. in avionics, video capture, medical x-ray, and robotics. Speed is important but the crucial point is reliably meeting deadlines. Garbage collectors, JIT compilers, and run-time optimizers are bad for predictability.) Maybe Rust already has this. What determines whether a thread-local managed heap uses reference counting? Ref-counting may be OK in a real-time thread if it doesn't deallocate a variable-length chain of nodes at an inconvenient time. Rust leaves memory allocation up to the user as much as C++. Most code can simply use unboxed values and lightweight references, with occasional usage of owned boxes (single-owner heap allocations, with lifetime tied to scope). Ideally, forking a real-time thread could ensure that (1) no GC pauses the thread just to find out there's no garbage to collect, and (2) something would catch the mistake if it ever calls code (directly or indirectly) that evolves to using GC memory. If you need shared ownership, it's available via the `Rc`/`RcMut` types. Every split of ownership is explicit via a `clone` call since moves are the default. There will be a task-local garbage collector providing garbage collected pointers, but it still needs to be implemented. Sounds good, although I don't understand the details. Does the program pick between GC allocation and ref counting allocation via the choice of pointer sigils? Via the choice of library calls? The standard library implements N:M concurrency and exposes a blocking API for asynchronous I/O. I don't think it would be suitable for a real-time application, but you can avoid the standard library: https://github.com/thestinger/rust-core What's N:M concurrency? Some examples: - Make the everyday int and uint types portable, that is, produce the same results on all platforms. int size does matter when it varies between 16 and 64 bits. If Rust needs an integer type that’s the size of a pointer, call it intptr_t and only use it for that special case. There are fixed-size integers and floats and then the pointer-sized integers. I agree that renaming the pointer-size ones to `intptr`/`uintptr` would be great. I just filed https://github.com/mozilla/rust/issues/9940about this, so feel free to voice support there :). Will do. - Security holes happen whenever a program does exploitably unexpected things. When the unexpected results happen in rarely-tested cases like integer overflow, it’s bad for security. So consider arbitrary precision int. A relatively slow big integer type exists in the standard library. There are two issues blocking it from being as first-class as the built-in types: * https://github.com/mozilla/rust/issues/4169 * https://github.com/mozilla/rust/issues/6023 - AIUI, let mut x, y defines 2 mutable variables but |mut x, y| defines mutable x and immutable y. This is harder to learn and easier to goof. This syntax has been removed. The `mut` keyword will eventually be usable on variable bindings in patterns, like `let mut x = 5` or `let (x, mut y) = (1, 2). - Drop C-compatible struct layout support? Is such a goal practical considering that C struct layouts vary by CPU, compiler, and compiler switches (enum size, pointer size, integer alignment, bit-field padding, etc.)? C can’t reliably pass a struct to a dynamically-loaded module (that has burned me) or to another computer (that, too). A better approach is a library like Protocol Buffers or Cap’n Proto. The `struct` layout must be compatible with the platform ABI. There's no way around the need to be interoperable with existing kernels, libraries and languages, and it needs to be efficient. Rust doesn't provide any guarantees for `enum` layout, and doesn't implement bit fields. The guarantee is not intended to be used directly for serialization. Ah. It focuses on the core cases. A case that crashed on me was calling libjpeg.so when compiled with a gcc switch that set small enums. A student put that switch in the make file because he got burned by enum sizes when calling the OS on a Chumby. -- Jerry ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
