Re: [rust-dev] std::num::pow() is inadequate / language concepts
For the specific issue of exponentiation, you might be interested in https://github.com/rust-lang/rfcs/pull/172 On Fri, Jul 25, 2014 at 9:26 AM, Gregor Cramer rema...@gmx.net wrote: Hi Marijn, Firstly, blanket statements like This makes generic programming impossible and it does not allow proper software design are unneccesary hyperbole, and do not help the discussion in any way. You're not right, my statement wasn't blanket, it was my result after I tried to overwork the big integer library, and I have mentioned this: I gave up at all. (I'm doing software design and implementation since more than 30 years, and I never accept compromises, this is the way how to develop magnificient software). Traits provide a more well-defined, easier to reason about alternative to overloading. They do require the author of an algorithm to decide ahead of time whether this algorithm needs to be specializeable, which I guess C++-style overloading does not. Yes, the traits are great, I'm impressed, as I said before, and in fact Rust is really great, despite a few facts, otherwise I wouldn't subscribe to this mailing list. And my goal is to be constructive, don't worry if I'm a bit euphoric, such things happens. Nethertheless, it gave up to overwork the big integer libary because I cannot specialize std::num::pow(). There is no way to proceed with a proper design. Whether that is a good or a bad thing is debatable, but it is not true that Rust lacks a feature for specialization. There is a lack in the current language concept, std::num::pow() is inadequate due to this language concept, and std::num::pow() is only one example for this fact. I will repeat the problem with signatures. Currently pow() is declared as following: pub fn powT: One + MulT, T(mut base: T, mut exp: uint) - T; That't 100% ok. The user will call this function in this way: pow(a) // a is i32 Perfect. Now I need a specialized function for BigInt: [#overload] pub fn pow(base: BigInt, mut exp: uint) - T; There's a problem (beside the missing overloading feature): the specialized version requires a reference. Same problem if I'm calling this function: pow(a) // a is BigInt The user has to know how to call a function, depending on the type. But a proper function specialization would be: [#overload] pub fn pow(base: BigInt, mut exp: uint) - T; And so the function call is as expected, like with other numeric types: pow(a) // a is BigInt But there is now a problem in this function definition, BigInt is given as a copy, and this is a software design issue (superfluous memory allocation). And this currently happens if the user is calling std::num::pow() with a numeric type like BigInt (apart from other performance penalties in pow()). That's what I've mentioned that the compiler should decide whether an argument is given by reference or by value. In this way the latter approach works. And in the case that a function willl modify an argument (in-out value), for example: fn mul_vec(acc : mut [BigDigit], base: mut [BigDigit], mut exp:uint) the call of this function would be: mul_vec(a, b, exp) This concept will not change, because here it has to be clear that an argument will be changed (furthermore the compiler should give a warning if a function is not changing a mutable argument). I think that this approach is even superior to the 'const' concept of C++, and it fit's with the great overall concept of Rust (especially with the owner/borrower concept). I try to show the problems if function specialization (overloading) is not supported. A stable software design is problematic. Adding a new module, which will use existing function declarations, is impossible in some cases. Currently I cannot implement a specialized version of pow() for BigInt, adding a new function for a different numeric type is only a hack, and moving this function into a trait is not solving the general problem, because pow() is only one example. (Beside: it's not my decision to move pow() into a trait.) Cheers, Gregor ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Anyone in NYC?
I'd be in. On Thursday, March 13, 2014, Clark Gaebel cg.wowus...@gmail.com wrote: Hey I'm in NYC and think some sort of rust meet-up would be neat. Anyone out there? ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] reader.lines() swallows io errors
Can you point to any scripting langs whose lines equivalent just silently
ignores errors? I'm not aware of any; even perl will at least populate $!.
I opened https://github.com/mozilla/rust/issues/12130 a little while ago
about if_ok!/try! not being usable from main and the limitations for simple
use cases that can cause. Forgive a possibly dumb question, but is there a
reason main has to return ()? Could Rust provide an 'ExitCode' trait that
types could implement that would provide the exit code that the process
would spit out if it were returned from main? IoResult's impl would just
be `match self { Ok(_) = 0, Err(_) = 1 }` and your example would look like
fn main() - IoResult~str {
for line in io::stdin().lines() {
print!(“received: {}”, try!(line));
}
}
On Wed, Feb 19, 2014 at 5:50 PM, Kevin Ballard ke...@sb.org wrote:
On Feb 19, 2014, at 2:34 PM, Lee Braiden leebr...@gmail.com wrote:
Then we could introduce a new struct to wrap any Reader that translates
non-EOF errors into EOF specifically to let you say “I really don’t care
about failure”.
It sounds like a very specific way to handle a very general problem.
People like (modern, complete) scripting languages because they handle this
sort of intricacy in elegant, ways, not because they gloss over it and make
half-baked programs that don't handle errors. It's just that you can, say,
handle IOErrors in one step, at the top of your script, except for one
particular issue that you know how to recover from, six levels into the
call stack. Exceptions (so long as there isn't a lot of boilerplate around
them) let you do that, easily. Rust needs a similarly generic approach to
propagating errors and handling them five levels up, whether that's
exceptions or fails (I don't think they currently are flexible enough), or
monads, or something else.
In my experience, exceptions are actually a very *inelegant* way to
handle this problem. The code 5 levels higher that catches the exception
doesn’t have enough information about the problem in order to recover.
Maybe it just discards the entire computation, or perhaps restarts it. But
it can’t recover and continue.
We already tried conditions for this, which do let you recover and
continue, except that turned out to be a dismal failure. Code that didn’t
touch conditions were basically just hoping nothing went wrong, and would
fail!() if it did. Code that did try to handle errors was very verbose
because conditions were a PITA to work with.
As for what we’re talking about here. lines() is fairly unique right now
in its discarding of errors. I can’t think of another example offhand that
will discard errors. As I said before, I believe that .lines() exists to
facilitate I/O handling in a fashion similar to scripting languages,
primarily because one of the basic things people try to do with new
languages is read from stdin and handle the input, and it’s great if we can
say our solution to that is:
fn main() {
for line in io::stdin().lines() {
print!(“received: {}”, line);
}
}
It’s a lot more confusing and off-putting if our example looks like
fn main() {
for line in io::stdin().lines() {
match line {
Ok(line) = print!(“received: {}”, line),
Err(e) = {
println!(“error: {}”, e);
break;
}
}
}
or alternatively
fn main() {
for line in io::stdin().lines() {
let line = line.unwrap(); // new user says “what is .unwrap()?”
and is still not handling errors here
print!(“received: {}”, line);
}
}
Note that we can’t even use try!() (née if_ok!()) here because main()
doesn’t return an IoResult.
The other thing to consider is that StrSlice also exposes a .lines()
method and it may be confusing to have two .lines() methods that yield
different types.
Given that, the only reasonable solutions appear to be:
1. Keep the current behavior. .lines() already documents its behavior;
anyone who cares about errors should use .read_line() in a loop
2. Change .lines() to fail!() on a non-EOF error. Introduce a new wrapper
type IgnoreErrReader (name suggestions welcome!) that translates all errors
into EOF. Now the original sample code will fail!() on a non-EOF error, and
there’s a defined way of turning it back into the version that ignores
errors for people who legitimately want that. This could be exposed as a
default method on Reader called .ignoring_errors() that consumes self and
returns the new wrapper.
3. Keep .lines() as-is and add the wrapper struct that fail!()s on errors.
This doesn’t make a lot of sense to me because the struct would only ever
be used with .lines(), and therefore this seems worse than:
4. Change .lines() to fail!() on errors and add a new method
.lines_ignoring_errs() that behaves the way .lines() does today. That’s
kind of verbose though, and is a specialized form of suggestion
Re: [rust-dev] Proposal: Change Parametric Polymorphism Declaration Syntax
I'm not a huge fan of this proposal. It makes declarations longer, and it
removes the visual consistency of FooT,U everywhere, which I think
introduces its own pedagogical issue.
The recent addition of default type parameters, though, makes me think
there's a reasonable change that increases consistency and shortens
declarations in a few common cases.
From what I understand, the reason we can't just have
impl TraitT for FooT,U
is because it's ambiguous whether T and U are intended to be concrete or
generic type names; i.e.,
implT TraitT for FooT,U
tells the compiler that we expect U to be a concrete type name.
Our new default type parameter declarations look like:
struct FooT,U=Bar
So what if to actually make generic types concrete, we always used the '='?
struct FooT,U=Bar
impl TraitT for FooT, U=Derp
This saves a character over 'implT TraitT for FooT, Derp', solves the
greppability problem, and makes intuitive sense given how defaults are
declared.
It also has a nice parallel with how ':' is used - ':' adds restrictions,
'=' fully locks in place. So what is today something like
implT:Ord TraitT for FooT, Derp
would become
impl TraitT:Ord for FooT, U=Derp
The rule would be that the first use of a type variable T would introduce
its bounds, so for instance:
impl TraitT:Ord for FooZ:Clone, U=Derp
would be fine, and
impl TraitT for FooT:Clone, U=Derp
would be an error.
More nice fallout:
struct FooA,B
impl FooA,B=Bar {
fn one(a: A) - B
fn two(a: A) - B
fn three(a: A) - B
}
means that if I ever want to go back and change the name of Bar, I only
have to do it in one place, or if Bar is actually some complicated type, I
only had to write it once, like a little local typedef.
I'm sure this has some glaring obvious flaw I'm not thinking of. It would
be nice to have less syntax for these declarations, but honestly I'm ok
with how it is now.
On Sat, Feb 1, 2014 at 5:39 PM, Corey Richardson co...@octayn.net wrote:
Hey all,
bjz and I have worked out a nice proposal[0] for a slight syntax
change, reproduced here. It is a breaking change to the syntax, but it
is one that I think brings many benefits.
Summary
===
Change the following syntax:
```
struct FooT, U { ... }
implT, U TraitT for FooT, U { ... }
fn fooT, U(...) { ... }
```
to:
```
forallT, U struct Foo { ... }
forallT, U impl TraitT for FooT, U { ... }
forallT, U fn foo(...) { ... }
```
The Problem
===
The immediate, and most pragmatic, problem is that in today's Rust one
cannot
easily search for implementations of a trait. Why? `grep 'impl Clone'` is
itself not sufficient, since many types have parametric polymorphism. Now I
need to come up with some sort of regex that can handle this. An easy
first-attempt is `grep 'impl(.*?)? Clone'` but that is quite
inconvenient to
type and remember. (Here I ignore the issue of tooling, as I do not find
the
argument of But a tool can do it! valid in language design.)
A deeper, more pedagogical problem, is the mismatch between how `struct
Foo... { ... }` is read and how it is actually treated. The
straightforward,
left-to-right reading says There is a struct Foo which, given the types
...
has the members This might lead one to believe that `Foo` is a single
type, but it is not. `Fooint` (that is, type `Foo` instantiated with type
`int`) is not the same type as `Foounit` (that is, type `Foo`
instantiated
with type `uint`). Of course, with a small amount of experience or a very
simple explanation, that becomes obvious.
Something less obvious is the treatment of functions. What does `fn
foo...(...) { ... }` say? There is a function foo which, given types ...
and arguments ..., does the following computation: ... is not very
adequate.
It leads one to believe there is a *single* function `foo`, whereas there
is
actually a single `foo` for every substitution of type parameters! This
also
holds for implementations (both of traits and of inherent methods).
Another minor problem is that nicely formatting long lists of type
parameters
or type parameters with many bounds is difficult.
Proposed Solution
=
Introduce a new keyword, `forall`. This choice of keyword reads very well
and
will not conflict with any identifiers in code which follows the [style
guide](https://github.com/mozilla/rust/wiki/Note-style-guide).
Change the following declarations from
```
struct FooT, U { ... }
implT, U TraitT for FooT, U { ... }
fn fooT, U(...) { ... }
```
to:
```
forallT, U struct Foo { ... }
forallT, U impl TraitT for FooT, U { ... }
forallT, U fn foo(...) { ... }
```
These read very well. for all types T and U, there is a struct Foo ...,
for
all types T and U, there is a function foo ..., etc. These reflect that
there
are in fact multiple functions `foo` and structs `Foo` and implementations
of
`Trait`, due to monomorphization.
[0]:
Re: [rust-dev] let mut - var
3 extra characters isn't doing anything to stop consenting adults. Nobody's saying get rid of mutable variables, just that it seems like a waste of limited resources to figure out how to streamline them when in general their use should be limited to where necessary. Python isn't busy trying to figure out how to make 'look before you leap' easier. They have a culture of discouraging it and favoring 'forgiveness over permission'. Consenting adults can do all the looking they like, but that doesn't mean Python is going to bend over backwards to make it pleasant for them. On Thursday, January 30, 2014, Donaldo Fastoso donquest...@rocketmail.com wrote: I like python's rational of consenting adults: Give people the tools to do the right thing, if they still want to hurt themselves, they may have a good reason, or they are better of dead! ;-) I would argue, that people choosing Rust over C/C++ are choosing it BECAUSE of safety measures like immutability and wouldn't need overbearing lectures. In all honesty it's not the TYPING of three additional chars let mut, but the READING. It interrupts the flow of reading, or better: it's a bump in the flow of scanning. Source-Code is not prosa, you have to actively follow the train of thought and guess the intentions of the author. So improving READABILITY would really be nice, especially for people coming from other languages. They would probably try to learn by reading the source from experienced programmers. In this case i would also advice against the use of var, instead of let mut, but omitting let and leave it to mut would be much easier to read and understand. so let mut x, y; would become: mut x; let y; which would take a possible interpretation-ambiguity away from the single let mut x, y, which can be read either as let mut for x and y, or let mut x and let y! So imho let mut has at least two pitfalls: 1) read-bump 2) ambiguity. AFAIK you did a remarkable good job so far, and i have all the faith you are considering all arguments before coming to a decision. Even if some thoughts of the thoughts come form the bad smelling lurker-fraction, which do nothing but making comments about things they possible can't understand! ;-) Regards, Don ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] let mut - var
You *should* get sick of writing 'let mut' all over the place, not just b/c
of the syntax but b/c you're using mutable variables all over the place.
Casual mutability kills maintainability.
Affordances matter. I'm convinced that the reason Option.unwrap() is used
so frequently is b/c it's the shortest method name and requires the
fewest explicit decisions, and so is the easiest thing to reach for. If it
were unwrap_or_fail(reason), forcing you to both type more and to think
about a fail message, unwrap_or and unwrap_or_else wouldn't look as
difficult in comparison. or and or_else would be even better, or
'do' syntax now that the keyword's free again.
Make the Right Thing the easy thing, and don't put effort into making the
Wrong Thing as easy or easier.
On Wednesday, January 29, 2014, Samuel Williams
space.ship.travel...@gmail.com wrote:
I agree that it is syntactic salt and that the design is to discourage
mutability. I actually appreciate that point as a programmer.
w.r.t. this specific issue: I think what concerns me is that it is quite a
high burden for new programmers (I teach COSC1xx courses to new students so
I have some idea about the level of new programmers). For example, you need
to know more detail about what is going on - new programmers would find
that difficult as it is one more concept to overflow their heads.
Adding var as a keyword identically maps to new programmer's
expectations from JavaScript. Writing a program entirely using var
wouldn't cause any problems right? But, could be optimised more
(potentially) if using let for immutable parts.
Anyway, I'm not convinced either way, I'm not sure I see the entire
picture yet. But, if I was writing code, I'd certainly get sick of writing
let mut over and over again - and looking at existing rust examples, that
certainly seems like the norm..
On 30 January 2014 15:59, Samuel Williams
space.ship.travel...@gmail.comjavascript:_e({}, 'cvml',
'space.ship.travel...@gmail.com');
wrote:
I guess the main gain would be less typing of what seems to be a
reasonably common sequence, and the formalisation of a particular semantic
pattern which makes it easier to recognise the code when you visually
scanning it.
On 30 January 2014 15:50, Kevin Ballard ke...@sb.org javascript:_e({},
'cvml', 'ke...@sb.org'); wrote:
On Jan 29, 2014, at 6:43 PM, Brian Anderson
bander...@mozilla.comjavascript:_e({}, 'cvml', 'bander...@mozilla.com');
wrote:
On 01/29/2014 06:35 PM, Patrick Walton wrote:
On 1/29/14 6:34 PM, Samuel Williams wrote:
Perhaps this has been considered already, but when I'm reading rust
code
let mut just seems to stick out all over the place. Why not add a
var keyword that does the same thing? I think there are lots of
good
and bad reasons to do this or not do it, but I just wanted to propose
the idea and see what other people are thinking.
`let` takes a pattern. `mut` is a modifier on variables in a pattern.
It is reasonable to write `let (x, mut y) = ...`, `let (mut x, y) = ...`,
`let (mut x, mut y) = ...`, and so forth.
Having a special var syntax would defeat this orthogonality.
`var` could potentially just be special-case sugar for `let mut`.
To what end? Users still need to know about `mut` for all the other uses
of patterns. This would reserve a new keyword and appear to duplicate
functionality for no gain.
-Kevin
___
Rust-dev mailing list
Rust-dev@mozilla.org javascript:_e({}, 'cvml',
'Rust-dev@mozilla.org');
https://mail.mozilla.org/listinfo/rust-dev
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
[rust-dev] RFC: New Rust channel proposal
open() feels like the clear winner here. Channel::new_pipe is annoying because it's long and because channels and pipes are different things ( http://en.m.wikipedia.org/wiki/Pipe_flow), And aren't we down to naming? I thought the design sounded mostly settled from the last conversation, and brson revived the thread specifically responding to a question about naming. On Thursday, January 23, 2014, Tony Arcieri basc...@gmail.com wrote: On Thu, Jan 23, 2014 at 7:29 PM, Benjamin Striegel ben.strie...@gmail.com wrote: And Chan::open() doesn't map to anything that's as intuitive. Like File::open? :P -- Tony Arcieri ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] RFC: New Rust channel proposal
Also, +1 for source and sink, I still get port and channel mixed up. On Thursday, January 23, 2014, Jason Fager jfa...@gmail.com wrote: open() feels like the clear winner here. Channel::new_pipe is annoying because it's long and because channels and pipes are different things ( http://en.m.wikipedia.org/wiki/Pipe_flow), And aren't we down to naming? I thought the design sounded mostly settled from the last conversation, and brson revived the thread specifically responding to a question about naming. On Thursday, January 23, 2014, Tony Arcieri basc...@gmail.com wrote: On Thu, Jan 23, 2014 at 7:29 PM, Benjamin Striegel ben.strie...@gmail.com wrote: And Chan::open() doesn't map to anything that's as intuitive. Like File::open? :P -- Tony Arcieri ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Unbounded channels: Good idea/bad idea?
If you're pushing to an unbounded vec in a tight loop you've got
fundamental design issues. If you're pushing to a channel, you've got
something like a server under load. Use cases matter.
About the deadlock scenario, why aren't non-blocking sends sufficient to
address that concern? I'd personally argue just as strenuously as for
bounded channels that robust systems shouldn't have
senders that block indefinitely (nothing like waking up to a production
server that's hung on a socket someone removed the timeout from).
Also, again: unbounded channels aren't, their bound is just arbitrary and
they fail catastrophically when it's hit.
Even if you don't OOM, channels that are too large are themselves a major
problem. If you actually use your hardware you only have so much capacity.
If your channel's getting backed up it's probably for a reason; when that
reason gets resolved you're going to need to go back to handling your
normal load plus everything you've been back-logging on your unbounded
channels. That's one of the main things you tune with bounded channels:
how backed up can this reasonably be before I just can't catch up anymore?
In a browser, users might not want you to throw away events, but they
probably don't want to deal with their browser OOMing, or pausing for five
minutes and then machine-gun responding, either. Again, anytime consumers
lag producers, you're screwed. The question is, how can you respond to
mitigate?
On Tuesday, December 31, 2013, Patrick Walton wrote:
On 12/31/13 3:15 PM, György Andrasek wrote:
On 12/31/2013 10:41 PM, Patrick Walton wrote:
Unbounded channels have defined behavior as well. Undefined behavior has
a precise definition and OOM is not undefined behavior.
OOM is not a behavior. It's a DoS attack on the rest of the system.
When we speak of eliminating undefined behavior in Rust, we aren't
speaking of taming code like:
let mut v = ~[];
loop {
v.push(1)
}
Eliminating this hazard simply isn't one of the goals of the language.
Patrick
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Unbounded channels: Good idea/bad idea?
I work on a system that handles 10s of billions of events per day, and we do a lot of queueing. Big +1 on having bounded queues. Unbounded in-memory queues aren't, they just have a bound you have no direct control over and that blows up the world when its hit. The only reason to have a queue size greater than 1 is to handle spikes in the producer, short outages in the consumer, or a bit of out-of-phaseness between producers and consumers. If your consumers habitually can't keep up with your producers, you're screwed regardless, but unbounded queues make your problem grow from simply being 'consumer can't keep up' to 'consumer can't keep up, I'm OOMing, and I just lost a boatload of messages'. If the only option that's available are unbounded queues, there are going to be a lot of explicit semaphores sitting around in production Rust code. I would also hope that Rust doesn't decide to silently drop messages on the floor. It's very true that systems that can tolerate dropped messages are more robust, but not all messages are created equal, and not all messages can just be dropped (i.e., if they come from a 3rd party). I need to have control over what happens to a message that I can't send, whether that means slow-pathing it to something on-disk, passing an error back to my source, or just deciding to drop it. The twitter conversation has discussion of blocking sends leading to deadlocks: yes, blocking indefinitely is also bad, so don't do that. If it's blocking it should have a timeout, or as others have mentioned you can keep the sends non-blocking and just have them fail if the queue is full. On Wed, Dec 18, 2013 at 9:29 PM, Tony Arcieri basc...@gmail.com wrote: Some context: https://twitter.com/mentalguy/status/284776872452173824 As someone who knows a lot of people who use Erlang in production (Erlang has unbounded mailboxes), and the maintainer of my own actor-based concurrency framework (Celluloid, which started with unbounded mailboxes and is in the process of moving to bounded ones), I have come to the Hard Won Knowledge(TM) that unbounded queues/mailboxes/channels are a bad idea. In terms of production users of these sorts of systems, my personal experience is that nobody likes unboundedness and most have experienced some sort of production outage because of it. This isn't just a minor nitpick. This is the sort of decision that makes or breaks the reliability of systems under load. The main problem with unboundedness is that users systems based on unbounded queues fail to adequately build mechanisms for providing backpressure into their code. They then start flooding their systems with messages, and get confused why they're performing so poorly when the answer is they have a huge backlog of unprocessed messages. Processes overloaded with too many messages will slow down and grow in memory until they eventually exhaust system resources and crash. Adding bounds to a channel doesn't require that sends block, and I think Rust is doing the Right Thing(TM) here in regard to non-blocking sends and I would never ask you to change that. There are other options for bounding channels which don't involve a blocking send though: 1) Drop messages on the floor: This falls into the category of at most once message semantics that actor systems are typically described as having (although there's a fun discussion about this right now on the friam mailing list). My personal opinion is that systems that can tolerate the loss of messages are more robust by design 2) Crash the sender: This works similarly to the above in that it results in the messages being discarded, but can loop in Erlang-style fault tolerance to recover from an overloaded system. I definitely find this less preferable than simply dropping messages on the floor though. This is a particularly invasive option that I think doesn't translate well to distributed systems. 3) Make sends to a full channel an error: I'm really not a fan of this option at all but I'm listing it for completeness. We could make everyone check every message send for success. Ick. No thanks. Bounded channels offer a lot of advantages over unbounded ones, IMO. Fixed-size data structures confer a natural performance advantage over elastic ones that need to be resized to accomodate growing numbers of messages. Depending on what data structure you use, you either take an up-front performance hit to provide variable capacity, or take an invisible performance hit whenever you hit some cap and need to resize. Adding a bound makes it easier to reason about the use cases and requirements and will generally allow you to leverage better data structures (e.g. ring buffers) that will confer maximum performance. -- Tony Arcieri ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev
Re: [rust-dev] Unbounded channels: Good idea/bad idea?
Okay, parallelism, of course, and I'm sure others. Bad use of the word 'only'. The point is that if your consumers aren't keeping up with your producers, you're screwed anyways, and growing the queue indefinitely isn't a way to get around that. Growing queues should only serve specific purposes and make it easy to apply back pressure when the assumptions behind those purposes go awry. On Thursday, December 19, 2013, Patrick Walton wrote: On 12/19/13 6:31 AM, Jason Fager wrote: I work on a system that handles 10s of billions of events per day, and we do a lot of queueing. Big +1 on having bounded queues. Unbounded in-memory queues aren't, they just have a bound you have no direct control over and that blows up the world when its hit. The only reason to have a queue size greater than 1 is to handle spikes in the producer, short outages in the consumer, or a bit of out-of-phaseness between producers and consumers. Well, also parallelism. Patrick ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Unbounded channels: Good idea/bad idea?
So what do you do when you OOM? A network traffic spike beyond a particular threshold is exactly why you want a bounded queue, b/c it gives you an opportunity to actually handle it and recover, even if the recovery is just drop messages I can't handle. Backpressure doesn't make sense on an edge server handling traffic you don't control, but spill-to-disk or discarding messages does. Having a bound on your queue size and statically allocating a gigantic channel buffer are orthogonal issues. You can bound a linked list. On Thu, Dec 19, 2013 at 1:23 PM, Kevin Ballard ke...@sb.org wrote: Here’s an example from where I use an infinite queue. I have an IRC bot, written in Go. The incoming network traffic of this bot is handled in one goroutine, which parses each line into its components, and enqueues the result on a channel. The channel is very deliberately made infinite (via a separate goroutine that stores the infinite buffer in a local slice). The reason it’s infinite is because the bot needs to be resilient against the case where either the consumer unexpectedly blocks, or the network traffic spikes. The general assumption is that, under normal conditions, the consumer will always be able to keep up with the producer (as the producer is based on network traffic and not e.g. a tight CPU loop generating messages as fast as possible). Backpressure makes no sense here, as you cannot put backpressure on the network short of letting the socket buffer fill up, and letting the socket buffer fill up with cause the IRC network to disconnect you. So the overriding goal here is to prevent network disconnects, while assuming that the consumer will be able to catch up if it ever gets behind. This particular use case very explicitly wants a dynamically-sized infinite channel. I suppose an absurdly large channel would be acceptable, because if the consumer ever gets e.g. 100,000 lines behind then it’s in trouble already, but I’d rather not have the memory overhead of a statically-allocated gigantic channel buffer. -Kevin On Dec 19, 2013, at 10:04 AM, Jason Fager jfa...@gmail.com wrote: Okay, parallelism, of course, and I'm sure others. Bad use of the word 'only'. The point is that if your consumers aren't keeping up with your producers, you're screwed anyways, and growing the queue indefinitely isn't a way to get around that. Growing queues should only serve specific purposes and make it easy to apply back pressure when the assumptions behind those purposes go awry. On Thursday, December 19, 2013, Patrick Walton wrote: On 12/19/13 6:31 AM, Jason Fager wrote: I work on a system that handles 10s of billions of events per day, and we do a lot of queueing. Big +1 on having bounded queues. Unbounded in-memory queues aren't, they just have a bound you have no direct control over and that blows up the world when its hit. The only reason to have a queue size greater than 1 is to handle spikes in the producer, short outages in the consumer, or a bit of out-of-phaseness between producers and consumers. Well, also parallelism. Patrick ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] select on std::comm::Port and different types
I solved these problems somewhat clunkily using an enum:
https://github.com/jfager/d3cap/blob/master/multicast.rs
It's not pretty but it gets the job done until the various issues around
this get worked out.
On Thursday, November 14, 2013, Diego Ongaro wrote:
Hi all,
My program starts a bunch of tasks, then I want the main task to both
receive ctrl-c signals and receive results from the children. The
signal will come from a std::rt::io::signal::Listener's port, which is
an std::comm::PortSignum. The child results will come from a
std::comm::Port~[uint].
My first problem is that std::comm::Port doesn't implement
std::select::Select. It looks like std::rt::comm::Port does, and
std::comm::Port is just a small wrapper around that, but
std::comm::Port makes its internal std::rt::comm::Port private. Is
there any way to select on a std::comm::Port? (And what's the
difference between a std::rt::comm::Port and a std::comm::Port?)
My second problem is that std::select::select() doesn't seem to
support selecting from ports with different types. Naively, I tried
select([p1, p2]), but that expects p1 and p2 to have the same type:
error: mismatched types: expected
`std::rt::comm::Portstd::rt::io::signal::Signum` but found
`std::rt::comm::Port~[uint]` (expected enum
std::rt::io::signal::Signum but found vector)
It'll need dynamic dispatch, so I tried: select([p1 as Select, p2 as
Select]). However, this doesn't work since Select doesn't implement
Select:
error: failed to find an implementation of trait std::select::Select
for std::select::Selectno-bounds
There's some commented out code that may be related in select.rs,
though it's hard for me to know where this stands:
/* FIXME(#5121, #7914) This all should be legal, but rust is not
clever enough yet.
impl 'self Select for 'self mut Select {
fn optimistic_check(mut self) - bool { self.optimistic_check() }
fn block_on(mut self, sched: mut Scheduler, task: BlockedTask) -
bool {
self.block_on(sched, task)
}
fn unblock_from(mut self) - bool { self.unblock_from() }
}
...
How can I select on two ports of different types?
Thanks,
Diego
___
Rust-dev mailing list
Rust-dev@mozilla.org javascript:;
https://mail.mozilla.org/listinfo/rust-dev
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
[rust-dev] About owned pointer
Let me ask another way: what's wrong with thinking of ~ just as meaning allocate to the heap and subject to move semantics? When would that simplification bite me in the ass regarding owned boxes vs strs/vecs? I get that I should very rarely want that for things that aren't dynamic containers or recursive data structures. But beyond that, am I ever going to get myself in trouble not remembering the details of the difference between how ~T works vs ~[T]? On Thursday, November 7, 2013, Daniel Micay wrote: On Thu, Nov 7, 2013 at 7:49 PM, Jason Fager jfa...@gmail.com wrote: Can you speak a little to the practical differences between owned boxes and ~[T]/~str? How does the difference affect how I should use each? I wrote the section on owned boxes in the tutorial currently in master, so I would suggest reading that. It's very rare for there to be a use case for an owned box outside of a recursive data structure or plugin system (traits as objects). The coverage in the tutorial of vectors/strings is not only lacking in depth but is also *incorrect*, so I understand why there's a lot of confusion about them. Vectors/strings are containers, and aren't connected to owned boxes any more than HashMap/TreeMap/TrieMap. I would prefer it if the syntactic sugar didn't exist and we just had generic container literals, because it seems to end up causing a lot of confusion. ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] If and pattern match
Doesn't seem like enough bang for the buck to me. In your first example you
save 3 vertical lines but get a really wide one in return, and lose
some indentation
levels but add more syntax and conceptual overhead to the language.
Might be my lack of imagination, but the feature doesn't seem to expand out
to many other use cases, either.
Your second case you could write as:
let foo = get_option(foo);
let bar = get_option(bar);
if foo.is_some() bar.is_some() {
use(foo.unwrap(), bar.unwrap());
}
On Monday, September 23, 2013, Oren Ben-Kiki wrote:
A question / proposed syntax... How about allowing writing something like:
if (Some(foo), Some(bar)) ~~ (get_option(foo), get_option(bar)) {
use(foo, bar);
}
Instead of having to write the more combersome:
match (get_option(foo), get_option(bar)) {
(Some(foo), Some(bar)) = {
use(foo, bar);
},
_otherwise = {},
}
Not to mention:
let foo_bar: bool = (Some(_foo), Some(_bar)) ~~ (get_option(foo),
get_option(bar));
match (get_option(foo), get_option(bar)) {
(Some(foo), Some(bar)) = {
use(foo, bar);
},
_otherwise = {},
}
Instead of the very cumbersome:
let foo_bar: bool =
match (get_option(foo), get_option(bar)) {
(Some(_foo), Some(_bar)) = true,
_otherwise = false,
}
}
So, in general allow `pattern ~~ expression` to be a boolean expression
and if it is used in an if statement allow it to introduce the matched
variables to the then scope.
The operator doesn't have to be ~~, it could be anything unique (though
using ~ for matching has a lot of precedence in other languages).
Thoughts?
Oren Ben-Kiki
___
Rust-dev mailing list
Rust-dev@mozilla.org
https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Proposal for clarifying the iterator protocol
Not confused, I understand your point about for loops not caring about what
happens with additional 'next' calls.
But as a user of an iterator, I have the expectation that a for loop
exhausts the elements available from an iterator unless I return early or
break. Designing an iterator that intentionally sidesteps that expectation
seems like a bad idea. Principle of least astonishment, etc.
And yes, of course, iterators already return Option. But they return
Option to satisfy the *iterator protocol*, not the use cases you
described. I'm talking about adding another layer of Option
So say I want to implement non-blocking io that plays nice w/ the iterator
protocol. Using an implementation taking advantage of option#3 look
something like:
loop {
for i in iter {
foo(i);
}
// other stuff
if(noReallyImDone) {
break;
}
}
While hoisting the Iterator's type into another layer of Option looks like:
for i in iter {
match i {
Some(i) = foo(i);
None = {
//other stuff
}
}
}
The outer Option allows the iterator protocol to work as expected, i.e.
iterate over all available elements in the iterator, and the inner
implements the non-blocking protocol you're looking for.
On Sun, Aug 4, 2013 at 8:12 PM, Kevin Ballard ke...@sb.org wrote:
I suspect you're confused about something.
The for loop doesn't care in the slightest what an iterator does after
it's returned None. All 3 approaches work equally well as far as the for
loop is concerned.
And I'm not sure what you mean by design Iterators that would take
advantage of the undefined behavior. If an iterator defines how it behaves
after returning None, then it's defined behavior. If you're using iterators
and you know your entire iterator pipeline, then you can use whatever
behavior the iterators involved define. You only need to restrict yourself
to what the iterator protocol defines if you don't know what iterator
you're consuming.
I also don't understand your suggestion about using Option. Iterators *
already* return an Option.
-Kevin
On Aug 4, 2013, at 4:45 PM, Jason Fager jfa...@gmail.com wrote:
Of course. I think I'm reacting more to the possible use cases you
described for option 3 than the actual meaning of it. It seems like a
really bad idea to design iterators that would take advantage of the
undefined behavior, not least b/c it's unexpected and not supported by the
most pervasive client of the iterator protocol (the for loop, in the sense
of actually iterating through all elements available through the iterator),
but that doesn't mean option 3 is in itself the wrong thing to do.
But addressing the use cases you mentioned, if you need that kind of
functionality, shouldn't you be hoisting the iterator's return type into
its own Option? i.e., an IteratorT should be become an
IteratorOptionT?
On Sun, Aug 4, 2013 at 6:23 PM, Kevin Ballard ke...@sb.org wrote:
The new for loop works with all 3 of these. Your output shows that it
queried .next() twice, and got a single Some(1) result back. Once it gets
None, it never calls .next() again, whereas the 3 behaviors stated
previously are exclusively concerned with what happens if you call .next()
again after it has already returned None.
-Kevin
P.S. I changed the email address that I'm subscribed to this list with,
so apologies for any potential confusion.
On Aug 4, 2013, at 6:18 AM, Jason Fager jfa...@gmail.com wrote:
The new for loop already assumes #2, right?
let x = [1,2,3];
let mut it = x.iter().peek_(|x| printfln!(*x)).scan(true, |st, x| { if
*st { *st = false; Some(x) } else { None } });
for i in it {
printfln!(from for loop: %?, i);
}
Which produces:
1
from for loop: 1
2
On Sun, Aug 4, 2013 at 1:49 AM, Daniel Micay danielmi...@gmail.comwrote:
On Sat, Aug 3, 2013 at 9:18 PM, Kevin Ballard kball...@gmail.com
wrote:
The iterator protocol, as I'm sure you're aware, is the protocol that
defines the behavior of the Iterator trait. Unfortunately, at the
moment the
trait does not document what happens if you call `.next()` on an
iterator
after a previous call has returned `None`. According to Daniel Micay,
the
intention was that the iterator would return `None` forever. However,
this
is not guaranteed by at least one iterator adaptor (Scan), nor is it
documented. Furthermore, no thought has been given to what happens if
an
iterator pipeline has side-effects. A trivial example of the
side-effect
problem is this:
let x = [1,2,3];
let mut it = x.iter().peek_(|x| printfln!(*x)).scan(true, |st, x|
{ if
*st { *st = false; Some(x) } else { None } });
(it.next(), it.next(), it.next())
This results in `(Some(1), None, None)` but it prints out
1
2
3
After giving it some thought, I came up with 3 possible definitions for
behavior in this case:
1. Once `.next()` has returned `None
Re: [rust-dev] deriving Clone on a struct with a static vector
Yeah, that is a cool feature. They're called newtype structs, after newtypes in Haskell, discussed in the tutorial at http://static.rust-lang.org/doc/tutorial.html#tuple-structs btw, updated that gist w/ a hacky impl of Clone that uses copy, which I think I've heard is going away in the near future. Works for me for now, though. On Sat, Jul 6, 2013 at 11:31 AM, Ashish Myles marci...@gmail.com wrote: On Sat, Jul 6, 2013 at 5:45 AM, Jason Fager jfa...@gmail.com wrote: I've started implementing traits for fixed-length vectors with a few macros: https://gist.github.com/jfager/5936197 I don't have Clone yet, but it should be easy to add. As a side note, looking through your code, this is cool: struct Foo([u8,..2]); Foo([1u8,2u8]) I had no idea one could define single-item/wrapper structs that way; i.e. like an anonymous member. This is going in my cool tidbits collection. Ashish ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
