Re: Stream programming
Kiuhnm wrote:
[snip]
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
It reads as
take a list of numbers - save it - compute the average and named it
'med' - restore the flow - create two streams which have, respect.,
the numbers less than 'med' and those greater or equal to 'med' - do
the /entire/ 'same' process on each one of the two streams - concat
the resulting streams - name all this /entire/ process 'same'.
Not readable enough? Replace 'same' with 'qsort'.
Is that readable or am I going crazy? [note: that's a rhetorical
question whose answer is That's very readable!]
Kiuhnm
Here's a rhetorical answer to your question : whatever you're taking, I
want some !
JM
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 3/26/2012 11:27, Jean-Michel Pichavant wrote:
Kiuhnm wrote:
[snip]
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
It reads as
take a list of numbers - save it - compute the average and named it
'med' - restore the flow - create two streams which have, respect.,
the numbers less than 'med' and those greater or equal to 'med' - do
the /entire/ 'same' process on each one of the two streams - concat
the resulting streams - name all this /entire/ process 'same'.
Not readable enough? Replace 'same' with 'qsort'.
Is that readable or am I going crazy? [note: that's a rhetorical
question whose answer is That's very readable!]
Kiuhnm
Here's a rhetorical answer to your question : whatever you're taking, I
want some !
LOL. I should have titled my post A new obfuscation technique then :)
Kiuhnm
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 3/24/2012 4:23, Steven D'Aprano wrote:
On Fri, 23 Mar 2012 17:00:23 +0100, Kiuhnm wrote:
I've been writing a little library for handling streams as an excuse for
doing a little OOP with Python.
I don't share some of the views on readability expressed on this ng.
Indeed, I believe that a piece of code may very well start as complete
gibberish and become a pleasure to read after some additional
information is provided.
[...]
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
Ok, we're at the complete gibberish phase.
Time to give you the additional information.
There are multiple problems with your DSL. Having read your explanation,
and subsequent posts, I think I understand the data model, but the syntax
itself is not very good and far from readable. It is just too hard to
reason about the code.
Your syntax conflicts with established, far more common, use of the same
syntax: you use - to mean call a function and | to join two or more
streams into a flow.
You also use () for calling functions, and the difference between - and
() isn't clear. So a mystery there -- your DSL seems to have different
function syntax, depending on... what?
The semantics are unclear even after your examples. To understand your
syntax, you give examples, but to understand the examples, the reader
needs to understand the syntax. That suggests that the semantics are
unclear even in your own mind, or at least too difficult to explain in
simple examples.
Take this example:
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
What the hell does that mean? The reader initially doesn't know what
*any* of push, by(2), pop or val('double') means. All they see is an
obfuscated series of calls that starts with a stream as input, makes a
copy of it, and doubles the entries in the copy: you make FIVE function
calls to perform TWO conceptual operations. So the reader can't even map
a function call to a result.
With careful thought and further explanations from you, the reader (me)
eventually gets a mental model here. Your DSL has a single input which is
pipelined through a series of function calls by the - operator, plus a
separate stack. (I initially thought that, like Forth, your DSL was stack
based. But it isn't, is it?)
It seems to me that the - operator is only needed as syntactic sugar to
avoid using reverse Polish notation and an implicit stack. Instead of the
Forth-like:
[1,2,3,4] dup 2 *
your DSL has an explicit stack, and an explicit - operator to call a
function. Presumably [1,2] push would be a syntax error.
I think this is a good example of an inferior syntax. Contrast your:
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
with the equivalent RPL:
[1,2,3,4] dup 2 *
I was just explaining how push and pop work.
I also said that
[1,2,3,4] - [id,by(2)]
would be the recommended way to do it.
Now *that* is a pleasure to read, once you wrap your head around reverse
Polish notation and the concept of a stack. Which you need in your DSL
anyway, to understand push and pop.
I don't see why. Push and pop are not needed. They're just handful
mainly to modify a flow, collect a result, and go back to how the flow
was before the push.
It has nothing to do with RPN (which RPL is based on).
You say that this is an easier way to get the same result:
[1,2,3,4] - [id, by(2)]
but it isn't, is it? The more complex example above ends up with two
streams joined in a single flow:
[1,2,3,4]|[2,4,6,8]
whereas the shorter version using the magic id gives you a single
stream containing nested streams:
[[1,2,3,4], [2,4,6,8]]
Says who?
Here are the rules again:
A flow can be transformed:
[1,2] - f = [f(1),f(2)]
([1,2] | [3,4]) - f = [f(1,3),f(2,4)]
([1,2] | [3,4]) - [f] = [f(1),f(2)] | [f(3),f(4)]
([1,2] | [3,4]) - [f,g] = [f(1),f(2)] | [g(3),g(4)]
[1,2] - [f,g] = [f(1),f(2)] | [g(1),g(2)]
Read the last line.
What's very interesting, is that [f,g] is an iterable as well, so your
functions can be generated as needed.
So, how could you make this more readable?
* Don't fight the reader's expectations. If they've programmed in Unix
shells, they expect | as the pipelining operator. If they haven't, they
probably will find easy to read as a dataflow operator. Either way,
they're probably used to seeing a|b as meaning or (as in this stream,
or this stream) rather than the way you seem to be using it (this
stream, and this stream).
Here's my first attempt at improved syntax that doesn't fight the user:
[1,2,3,4] push by(2) 'double' pop val('double')
There are problems with your syntax.
Mine:
[...]+[...] - f + [...] - g - h + [...] - i + [...]
Yours:
((([...]+[...] f) + [...] g h) + [...] i) + [...]
I first tried to use '' and '' but '+' and '-' are much better.
push and pop are poor choices of words. Push does not actually push
Stream programming
I've been writing a little library for handling streams as an excuse for
doing a little OOP with Python.
I don't share some of the views on readability expressed on this ng.
Indeed, I believe that a piece of code may very well start as complete
gibberish and become a pleasure to read after some additional
information is provided.
I must say that imposed indentation is a pain when one is trying to
define some sort of DSL (domain specific language). Also, Python's
operator overloading is a bit limited, but that makes for a more
rewarding experience in my case.
Here's an example of what you can write:
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
Ok, we're at the complete gibberish phase.
Time to give you the additional information.
I will use = to mean is equivalent to. That's not part of the DSL.
A flow has one or more streams:
1 stream:
[1,2,3]
2 streams:
[1,3,5] | [2,4,6]
Two flows can be concatenated:
[1,2,3] + [4,5,6] = [1,2,3,4,5,6]
[0] + ([1,2] | [3,4]) + [10] = [0,1,2,10] | [0,3,4,10]
([1,2] | [10,20]) + ([3,4] | [30,40]) = [1,2,3,4] | [10,20,30,40]
A flow can be transformed:
[1,2] - f = [f(1),f(2)]
([1,2] | [3,4]) - f = [f(1,3),f(2,4)]
([1,2] | [3,4]) - [f] = [f(1),f(2)] | [f(3),f(4)]
([1,2] | [3,4]) - [f,g] = [f(1),f(2)] | [g(3),g(4)]
[1,2] - [f,g] = [f(1),f(2)] | [g(1),g(2)]
Some functions are special and almost any function can be made special:
[1,2,3,4,5] - filter(isprime) = [2,3,5]
[[],(1,2),[3,4,5]] - flatten = [1,2,3,4,5]
Note that 'filter' is not really necessary, thanks to 'flatten'.
Flows can be named, remembered and used
as a value:
[1,2,3,4,5] - 'flow' + val('flow') = [1,2,3,4,5]*2
as a transformation chain:
[1,2,3] - skipfirst - 'again' | [4,5,6] - func('again')
= [2,3] | [5,6]
Recursion is also possible and stops when a function is applied to an
empty sequence.
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
There are easier ways to achieve the same result, of course:
[1,2,3,4] - [id, by(2)]
Let's go back to our example. I didn't tell you anything but you should
be able to understand it anyway.
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
It reads as
take a list of numbers - save it - compute the average and named it
'med' - restore the flow - create two streams which have, respect., the
numbers less than 'med' and those greater or equal to 'med' - do the
/entire/ 'same' process on each one of the two streams - concat the
resulting streams - name all this /entire/ process 'same'.
Not readable enough? Replace 'same' with 'qsort'.
Is that readable or am I going crazy? [note: that's a rhetorical
question whose answer is That's very readable!]
Kiuhnm
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 3/23/2012 17:00, Kiuhnm wrote:
I've been writing a little library for handling streams as an excuse for
doing a little OOP with Python.
I don't share some of the views on readability expressed on this ng.
Indeed, I believe that a piece of code may very well start as complete
gibberish and become a pleasure to read after some additional
information is provided.
I must say that imposed indentation is a pain when one is trying to
define some sort of DSL (domain specific language). Also, Python's
operator overloading is a bit limited, but that makes for a more
rewarding experience in my case.
Here's an example of what you can write:
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
Ok, we're at the complete gibberish phase.
Time to give you the additional information.
I will use = to mean is equivalent to. That's not part of the DSL.
A flow has one or more streams:
1 stream:
[1,2,3]
2 streams:
[1,3,5] | [2,4,6]
Two flows can be concatenated:
[1,2,3] + [4,5,6] = [1,2,3,4,5,6]
[0] + ([1,2] | [3,4]) + [10] = [0,1,2,10] | [0,3,4,10]
([1,2] | [10,20]) + ([3,4] | [30,40]) = [1,2,3,4] | [10,20,30,40]
A flow can be transformed:
[1,2] - f = [f(1),f(2)]
([1,2] | [3,4]) - f = [f(1,3),f(2,4)]
([1,2] | [3,4]) - [f] = [f(1),f(2)] | [f(3),f(4)]
([1,2] | [3,4]) - [f,g] = [f(1),f(2)] | [g(3),g(4)]
[1,2] - [f,g] = [f(1),f(2)] | [g(1),g(2)]
Some functions are special and almost any function can be made special:
[1,2,3,4,5] - filter(isprime) = [2,3,5]
[[],(1,2),[3,4,5]] - flatten = [1,2,3,4,5]
Note that 'filter' is not really necessary, thanks to 'flatten'.
Flows can be named, remembered and used
as a value:
[1,2,3,4,5] - 'flow' + val('flow') = [1,2,3,4,5]*2
as a transformation chain:
[1,2,3] - skipfirst - 'again' | [4,5,6] - func('again')
= [2,3] | [5,6]
Recursion is also possible and stops when a function is applied to an
empty sequence.
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
There are easier ways to achieve the same result, of course:
[1,2,3,4] - [id, by(2)]
Let's go back to our example. I didn't tell you anything but you should
Ops... *everything*.
Kiuhnm
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
I will use = to mean is equivalent to. That's not part of the DSL.
A flow has one or more streams:
1 stream:
[1,2,3]
2 streams:
[1,3,5] | [2,4,6]
Two flows can be concatenated:
[1,2,3] + [4,5,6] = [1,2,3,4,5,6]
[0] + ([1,2] | [3,4]) + [10] = [0,1,2,10] | [0,3,4,10]
([1,2] | [10,20]) + ([3,4] | [30,40]) = [1,2,3,4] | [10,20,30,40]
Algebraically, your concatenation rules don't really make sense - your
flows are both distributive and non distributive. You also make the
implicit assumption of an order over streams in a flow, but disregard
the implications of that assumption in some cases. I understand what
you're trying to communicate, so I think you need to be a little more
strict and explicit in your definitions.
A flow can be transformed:
[1,2] - f = [f(1),f(2)]
([1,2] | [3,4]) - f = [f(1,3),f(2,4)]
([1,2] | [3,4]) - [f] = [f(1),f(2)] | [f(3),f(4)]
([1,2] | [3,4]) - [f,g] = [f(1),f(2)] | [g(3),g(4)]
[1,2] - [f,g] = [f(1),f(2)] | [g(1),g(2)]
Given the examples you pose here, it is clear that you are assuming
that the streams are synchronized in discrete time. Since you do not
provide any mechanism for temporal alignment of streams you are also
assuming every stream will have an element at every time point, the
streams start at the same time and are of the same length. Is this
what you want? These seem like pretty rough simplifying assumptions.
Some functions are special and almost any function can be made special:
[1,2,3,4,5] - filter(isprime) = [2,3,5]
[[],(1,2),[3,4,5]] - flatten = [1,2,3,4,5]
Note that 'filter' is not really necessary, thanks to 'flatten'.
This implies that your transformations again produce flows. You
should explicitly state this.
Flows can be named, remembered and used
as a value:
[1,2,3,4,5] - 'flow' + val('flow') = [1,2,3,4,5]*2
Is this a flow with two identical streams, or a flow with one long
stream formed by concatenation?
as a transformation chain:
[1,2,3] - skipfirst - 'again' | [4,5,6] - func('again')
= [2,3] | [5,6]
Recursion is also possible and stops when a function is applied to an empty
sequence.
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
There are easier ways to achieve the same result, of course:
[1,2,3,4] - [id, by(2)]
You are grasping at an algebra here, a sort of calculus of temporal
observations. You need to step back and make it rigorous before you
worry about issues such as a readable syntax.
Nathan
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 23/03/2012 16:33, Nathan Rice wrote: I will use = to mean is equivalent to. That's not part of the DSL. A flow has one or more streams: 1 stream: [1,2,3] 2 streams: [1,3,5] | [2,4,6] Two flows can be concatenated: [1,2,3] + [4,5,6]= [1,2,3,4,5,6] [0] + ([1,2] | [3,4]) + [10]= [0,1,2,10] | [0,3,4,10] ([1,2] | [10,20]) + ([3,4] | [30,40])= [1,2,3,4] | [10,20,30,40] Algebraically, your concatenation rules don't really make sense - your flows are both distributive and non distributive. You also make the implicit assumption of an order over streams in a flow, but disregard the implications of that assumption in some cases. I understand what you're trying to communicate, so I think you need to be a little more strict and explicit in your definitions. When concatenating, either there are the same number of streams, or one of them is a single stream which is duplicated. Therefore, in this example: [0] + ([1, 2] | [3, 4]) you're concatenating a single stream with a pair, so the single stream is duplicated: ([0] | [0]) + ([1, 2] | [3, 4]) and then they can be concatenated: ([0, 1, 2] | [0, 3, 4]) However, this: ([0, 1] | [2, 3]) + ([4, 5] | [6, 7] | [8, 9]) won't work. -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
I will use = to mean is equivalent to. That's not part of the DSL. A flow has one or more streams: 1 stream: [1,2,3] 2 streams: [1,3,5] | [2,4,6] Two flows can be concatenated: [1,2,3] + [4,5,6]= [1,2,3,4,5,6] [0] + ([1,2] | [3,4]) + [10]= [0,1,2,10] | [0,3,4,10] ([1,2] | [10,20]) + ([3,4] | [30,40])= [1,2,3,4] | [10,20,30,40] Algebraically, your concatenation rules don't really make sense - your flows are both distributive and non distributive. You also make the implicit assumption of an order over streams in a flow, but disregard the implications of that assumption in some cases. I understand what you're trying to communicate, so I think you need to be a little more strict and explicit in your definitions. When concatenating, either there are the same number of streams, or one of them is a single stream which is duplicated. Therefore, in this example: [0] + ([1, 2] | [3, 4]) you're concatenating a single stream with a pair, so the single stream is duplicated: ([0] | [0]) + ([1, 2] | [3, 4]) and then they can be concatenated: ([0, 1, 2] | [0, 3, 4]) However, this: ([0, 1] | [2, 3]) + ([4, 5] | [6, 7] | [8, 9]) won't work. I understand how he wants the system to work in this case; my point was that it isn't consistent. He stated flows can be concatenated, so [0] is just a flow of a single stream. Because he clearly intends that streams in a flow are ordered, that indicates that he wants some sort of algebraic module. He could get close if he can extend a stream to meet the requirements of a ring, then a flow would be a module over the ring of streams. The problem he is going to run into is that operations on streams as he defines them are not commutative. Identity elements for the two binary operations are also not immediately obvious to me. He could just be smart and use the pi calculus, it is rigorously developed and can model his desired behavior, if he reformulates it slightly. Another option is just to give up on being rigorous. Given the abstract treatment he attempted I would be disappointed, but if his only goal is to get practice writing code and he isn't interested in exploring the conceptual space of the problem domain it would be the right decision. -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 3/23/2012 17:33, Nathan Rice wrote:
I will use = to mean is equivalent to. That's not part of the DSL.
A flow has one or more streams:
1 stream:
[1,2,3]
2 streams:
[1,3,5] | [2,4,6]
Two flows can be concatenated:
[1,2,3] + [4,5,6]= [1,2,3,4,5,6]
[0] + ([1,2] | [3,4]) + [10]= [0,1,2,10] | [0,3,4,10]
([1,2] | [10,20]) + ([3,4] | [30,40])= [1,2,3,4] | [10,20,30,40]
Algebraically, your concatenation rules don't really make sense - your
flows are both distributive and non distributive.
?
You also make the
implicit assumption of an order over streams in a flow, but disregard
the implications of that assumption in some cases.
?
I understand what
you're trying to communicate, so I think you need to be a little more
strict and explicit in your definitions.
No, I don't think you understand what I meant.
A flow can be transformed:
[1,2] - f= [f(1),f(2)]
([1,2] | [3,4]) - f= [f(1,3),f(2,4)]
([1,2] | [3,4]) - [f]= [f(1),f(2)] | [f(3),f(4)]
([1,2] | [3,4]) - [f,g]= [f(1),f(2)] | [g(3),g(4)]
[1,2] - [f,g]= [f(1),f(2)] | [g(1),g(2)]
Given the examples you pose here, it is clear that you are assuming
that the streams are synchronized in discrete time. Since you do not
provide any mechanism for temporal alignment of streams you are also
assuming every stream will have an element at every time point, the
streams start at the same time and are of the same length. Is this
what you want?
Yes. I thought that streams as an alternative to functional programming
were widely known.
Some functions are special and almost any function can be made special:
[1,2,3,4,5] - filter(isprime)= [2,3,5]
[[],(1,2),[3,4,5]] - flatten= [1,2,3,4,5]
Note that 'filter' is not really necessary, thanks to 'flatten'.
This implies that your transformations again produce flows. You
should explicitly state this.
Isn't that obvious? BTW, those are not rigorous definitions. I thought I
was talking to people who regularly works with streams.
Flows can be named, remembered and used
as a value:
[1,2,3,4,5] - 'flow' + val('flow')= [1,2,3,4,5]*2
Is this a flow with two identical streams, or a flow with one long
stream formed by concatenation?
What does
[1,2,3,4,5]*2
mean in Python?
Those are Python's lists/arrays.
as a transformation chain:
[1,2,3] - skipfirst - 'again' | [4,5,6] - func('again')
= [2,3] | [5,6]
Recursion is also possible and stops when a function is applied to an empty
sequence.
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
There are easier ways to achieve the same result, of course:
[1,2,3,4] - [id, by(2)]
You are grasping at an algebra here, a sort of calculus of temporal
observations. You need to step back and make it rigorous before you
worry about issues such as a readable syntax.
Nathan
I don't agree. Sorry.
Kiuhnm
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 3/23/2012 20:23, Nathan Rice wrote: I will use = to mean is equivalent to. That's not part of the DSL. A flow has one or more streams: 1 stream: [1,2,3] 2 streams: [1,3,5] | [2,4,6] Two flows can be concatenated: [1,2,3] + [4,5,6]=[1,2,3,4,5,6] [0] + ([1,2] | [3,4]) + [10]=[0,1,2,10] | [0,3,4,10] ([1,2] | [10,20]) + ([3,4] | [30,40])=[1,2,3,4] | [10,20,30,40] Algebraically, your concatenation rules don't really make sense - your flows are both distributive and non distributive. You also make the implicit assumption of an order over streams in a flow, but disregard the implications of that assumption in some cases. I understand what you're trying to communicate, so I think you need to be a little more strict and explicit in your definitions. When concatenating, either there are the same number of streams, or one of them is a single stream which is duplicated. Therefore, in this example: [0] + ([1, 2] | [3, 4]) you're concatenating a single stream with a pair, so the single stream is duplicated: ([0] | [0]) + ([1, 2] | [3, 4]) and then they can be concatenated: ([0, 1, 2] | [0, 3, 4]) However, this: ([0, 1] | [2, 3]) + ([4, 5] | [6, 7] | [8, 9]) won't work. I understand how he wants the system to work in this case; my point was that it isn't consistent. He stated flows can be concatenated, so [0] is just a flow of a single stream. Because he clearly intends that streams in a flow are ordered, that indicates that he wants some sort of algebraic module. He could get close if he can extend a stream to meet the requirements of a ring, then a flow would be a module over the ring of streams. The problem he is going to run into is that operations on streams as he defines them are not commutative. Identity elements for the two binary operations are also not immediately obvious to me. Instead of talking of what I wasn't trying to do and, indeed, I didn't do, you should try to understand what I wanted to do and, in fact, I did. I'm afraid your cup is too full to understand simple things as the one I wrote in my OP. Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
I understand what you're trying to communicate, so I think you need to be a little more strict and explicit in your definitions. No, I don't think you understand what I meant. I don't agree. Sorry. Yes. I thought that streams as an alternative to functional programming were widely known. Streams aren't really a paradigm of computation. They're a semantic element of a computational system which cuts across paradigms. If you want to retract that and say you were talking about dataflow programming (which is much larger than streams, and actually has a cohesive definition), I won't hold it against you. Ultimately though, functional programming and dataflow programming are closely linked, the main difference being the use of queues based rather than stacks. Isn't that obvious? BTW, those are not rigorous definitions. I thought I was talking to people who regularly works with streams. That is the GPU mailing list, down the hall on the left. Instead of talking of what I wasn't trying to do and, indeed, I didn't do, you should try to understand what I wanted to do and, in fact, I did. I'm afraid your cup is too full to understand simple things as the one I wrote in my OP. Clearly, because I didn't explicitly include the possibility that you are just writing throwaway code with no attempt at development of ideas for the purpose of practicing writing code in the paragraph after the one you quoted. If your goal is to learn to code, instead of posting a message stating that you have a superior way to compose code, you might want to try to solve a problem and ask others their opinion of the structure and techniques in your code. -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
Kiuhnm wrote: On 3/23/2012 17:33, Nathan Rice wrote: Given the examples you pose here, it is clear that you are assuming that the streams are synchronized in discrete time. Since you do not provide any mechanism for temporal alignment of streams you are also assuming every stream will have an element at every time point, the streams start at the same time and are of the same length. Is this what you want? Yes. I thought that streams as an alternative to functional programming were widely known. Don't know about widely, but I can say I am unfamiliar with the way you are using them. This implies that your transformations again produce flows. You should explicitly state this. Isn't that obvious? BTW, those are not rigorous definitions. I thought I was talking to people who regularly works with streams. Why would you think that? This list is composed of all types that use Python. I've seen occasional discussion of functional programming, but I've only seen anything this confusing maybe twice before... granted, I don't read *everything*, but I do read quite a bit -- especially the stuff that looks like it might be interesting... like stream programming, for example. ;) After the discussion I've seen so far, I still have no idea how I would use your code or what it's good for. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On Fri, Mar 23, 2012 at 05:00:23PM +0100, Kiuhnm wrote:
I've been writing a little library for handling streams as an excuse for
doing a little OOP with Python.
I don't share some of the views on readability expressed on this ng.
Indeed, I believe that a piece of code may very well start as complete
gibberish and become a pleasure to read after some additional
information is provided.
I must say that imposed indentation is a pain when one is trying to
define some sort of DSL (domain specific language). Also, Python's
operator overloading is a bit limited, but that makes for a more
rewarding experience in my case.
Here's an example of what you can write:
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
Ok, we're at the complete gibberish phase.
Time to give you the additional information.
I will use = to mean is equivalent to. That's not part of the DSL.
A flow has one or more streams:
1 stream:
[1,2,3]
2 streams:
[1,3,5] | [2,4,6]
Two flows can be concatenated:
[1,2,3] + [4,5,6] = [1,2,3,4,5,6]
[0] + ([1,2] | [3,4]) + [10] = [0,1,2,10] | [0,3,4,10]
([1,2] | [10,20]) + ([3,4] | [30,40]) = [1,2,3,4] | [10,20,30,40]
A flow can be transformed:
[1,2] - f = [f(1),f(2)]
([1,2] | [3,4]) - f = [f(1,3),f(2,4)]
([1,2] | [3,4]) - [f] = [f(1),f(2)] | [f(3),f(4)]
([1,2] | [3,4]) - [f,g] = [f(1),f(2)] | [g(3),g(4)]
[1,2] - [f,g] = [f(1),f(2)] | [g(1),g(2)]
Some functions are special and almost any function can be made special:
[1,2,3,4,5] - filter(isprime) = [2,3,5]
[[],(1,2),[3,4,5]] - flatten = [1,2,3,4,5]
Note that 'filter' is not really necessary, thanks to 'flatten'.
Flows can be named, remembered and used
as a value:
[1,2,3,4,5] - 'flow' + val('flow') = [1,2,3,4,5]*2
as a transformation chain:
[1,2,3] - skipfirst - 'again' | [4,5,6] - func('again')
= [2,3] | [5,6]
Recursion is also possible and stops when a function is applied to an
empty sequence.
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
There are easier ways to achieve the same result, of course:
[1,2,3,4] - [id, by(2)]
Let's go back to our example. I didn't tell you anything but you should
be able to understand it anyway.
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
It reads as
take a list of numbers - save it - compute the average and named it
'med' - restore the flow - create two streams which have, respect., the
numbers less than 'med' and those greater or equal to 'med' - do the
/entire/ 'same' process on each one of the two streams - concat the
resulting streams - name all this /entire/ process 'same'.
Not readable enough? Replace 'same' with 'qsort'.
Is that readable or am I going crazy? [note: that's a rhetorical
question whose answer is That's very readable!]
This sounds like a concatenative programming and i've found many cool
examples from Haskell and Ruby. I was pleased with Python's rigid
off-side rule at first but frustrated when I realized that this is a
two-edged blade as the rule makes a DSL difficult.
Sorry for former mail to you (my damn Android gmail client doesn't
understand mailing lists well).
--
Ray
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 3/23/2012 22:12, Ethan Furman wrote: Kiuhnm wrote: On 3/23/2012 17:33, Nathan Rice wrote: Given the examples you pose here, it is clear that you are assuming that the streams are synchronized in discrete time. Since you do not provide any mechanism for temporal alignment of streams you are also assuming every stream will have an element at every time point, the streams start at the same time and are of the same length. Is this what you want? Yes. I thought that streams as an alternative to functional programming were widely known. Don't know about widely, but I can say I am unfamiliar with the way you are using them. That's good! It means I'm saying something new (and, hopefully, interesting). This implies that your transformations again produce flows. You should explicitly state this. Isn't that obvious? BTW, those are not rigorous definitions. I thought I was talking to people who regularly works with streams. Why would you think that? This list is composed of all types that use Python. I've seen occasional discussion of functional programming, but I've only seen anything this confusing maybe twice before... granted, I don't read *everything*, but I do read quite a bit -- especially the stuff that looks like it might be interesting... like stream programming, for example. ;) After the discussion I've seen so far, I still have no idea how I would use your code or what it's good for. The idea is simple. Flows or streams let you be more declarative without being too functional :) In imperative progamming you write statements or commands. In functional programming (FP) you write expressions. In streaming programming (SP) you create flows. Basically, if in FP you write h9(h8(h7(.(h1)))) in SP you write h1-h2-h3-...-h9 which I greatly prefer because that's the way I think. I think that SP could be an interesting alternative to FP (in ten years, maybe). Mine was just a little experiment. Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 3/23/2012 22:18, Nathan Rice wrote: I understand what you're trying to communicate, so I think you need to be a little more strict and explicit in your definitions. No, I don't think you understand what I meant. I don't agree. Sorry. You could just point out those inconsistencies that you found. Yes. I thought that streams as an alternative to functional programming were widely known. Streams aren't really a paradigm of computation. They're a semantic element of a computational system which cuts across paradigms. If you want to retract that and say you were talking about dataflow programming (which is much larger than streams, and actually has a cohesive definition), I won't hold it against you. I wasn't talking of dataflow programming. Flow programming is much closer to functional programming than dataflow programming is. Instead of talking of what I wasn't trying to do and, indeed, I didn't do, you should try to understand what I wanted to do and, in fact, I did. I'm afraid your cup is too full to understand simple things as the one I wrote in my OP. Clearly, because I didn't explicitly include the possibility that you are just writing throwaway code with no attempt at development of ideas for the purpose of practicing writing code in the paragraph after the one you quoted. x != 'a' doesn't imply that x == 'b'. If your goal is to learn to code, instead of posting a message stating that you have a superior way to compose code, you might want to try to solve a problem and ask others their opinion of the structure and techniques in your code. Never said it was superior. We've been talking about readability a lot, haven't we? I was just proposing something different. Kiuhnm -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On 3/24/2012 0:32, Ray Song wrote:
On Fri, Mar 23, 2012 at 05:00:23PM +0100, Kiuhnm wrote:
I've been writing a little library for handling streams as an excuse for
doing a little OOP with Python.
I don't share some of the views on readability expressed on this ng.
Indeed, I believe that a piece of code may very well start as complete
gibberish and become a pleasure to read after some additional
information is provided.
I must say that imposed indentation is a pain when one is trying to
define some sort of DSL (domain specific language). Also, Python's
operator overloading is a bit limited, but that makes for a more
rewarding experience in my case.
Here's an example of what you can write:
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
Ok, we're at the complete gibberish phase.
Time to give you the additional information.
I will use = to mean is equivalent to. That's not part of the DSL.
A flow has one or more streams:
1 stream:
[1,2,3]
2 streams:
[1,3,5] | [2,4,6]
Two flows can be concatenated:
[1,2,3] + [4,5,6]= [1,2,3,4,5,6]
[0] + ([1,2] | [3,4]) + [10]= [0,1,2,10] | [0,3,4,10]
([1,2] | [10,20]) + ([3,4] | [30,40])= [1,2,3,4] | [10,20,30,40]
A flow can be transformed:
[1,2] - f= [f(1),f(2)]
([1,2] | [3,4]) - f= [f(1,3),f(2,4)]
([1,2] | [3,4]) - [f]= [f(1),f(2)] | [f(3),f(4)]
([1,2] | [3,4]) - [f,g]= [f(1),f(2)] | [g(3),g(4)]
[1,2] - [f,g]= [f(1),f(2)] | [g(1),g(2)]
Some functions are special and almost any function can be made special:
[1,2,3,4,5] - filter(isprime)= [2,3,5]
[[],(1,2),[3,4,5]] - flatten= [1,2,3,4,5]
Note that 'filter' is not really necessary, thanks to 'flatten'.
Flows can be named, remembered and used
as a value:
[1,2,3,4,5] - 'flow' + val('flow')= [1,2,3,4,5]*2
as a transformation chain:
[1,2,3] - skipfirst - 'again' | [4,5,6] - func('again')
= [2,3] | [5,6]
Recursion is also possible and stops when a function is applied to an
empty sequence.
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
There are easier ways to achieve the same result, of course:
[1,2,3,4] - [id, by(2)]
Let's go back to our example. I didn't tell you anything but you should
be able to understand it anyway.
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
It reads as
take a list of numbers - save it - compute the average and named it
'med' - restore the flow - create two streams which have, respect., the
numbers less than 'med' and those greater or equal to 'med' - do the
/entire/ 'same' process on each one of the two streams - concat the
resulting streams - name all this /entire/ process 'same'.
Not readable enough? Replace 'same' with 'qsort'.
Is that readable or am I going crazy? [note: that's a rhetorical
question whose answer is That's very readable!]
This sounds like a concatenative programming and i've found many cool
examples from Haskell and Ruby. I was pleased with Python's rigid
off-side rule at first but frustrated when I realized that this is a
two-edged blade as the rule makes a DSL difficult.
Yes, there is that. I also noticed that I couldn't use the comparison
operators the way I liked.
Precedence is also an issue. I had to use '+' and '-' instead of '+' and
'' or '' because of that. I didn't want to introduce extra parentheses.
Sorry for former mail to you (my damn Android gmail client doesn't
understand mailing lists well).
Don't worry, my email address is invalid!
Kiuhnm
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
On Fri, 23 Mar 2012 17:00:23 +0100, Kiuhnm wrote:
I've been writing a little library for handling streams as an excuse for
doing a little OOP with Python.
I don't share some of the views on readability expressed on this ng.
Indeed, I believe that a piece of code may very well start as complete
gibberish and become a pleasure to read after some additional
information is provided.
[...]
numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
- ['same', 'same'] - streams(cat) - 'same'
Ok, we're at the complete gibberish phase.
Time to give you the additional information.
There are multiple problems with your DSL. Having read your explanation,
and subsequent posts, I think I understand the data model, but the syntax
itself is not very good and far from readable. It is just too hard to
reason about the code.
Your syntax conflicts with established, far more common, use of the same
syntax: you use - to mean call a function and | to join two or more
streams into a flow.
You also use () for calling functions, and the difference between - and
() isn't clear. So a mystery there -- your DSL seems to have different
function syntax, depending on... what?
The semantics are unclear even after your examples. To understand your
syntax, you give examples, but to understand the examples, the reader
needs to understand the syntax. That suggests that the semantics are
unclear even in your own mind, or at least too difficult to explain in
simple examples.
Take this example:
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
What the hell does that mean? The reader initially doesn't know what
*any* of push, by(2), pop or val('double') means. All they see is an
obfuscated series of calls that starts with a stream as input, makes a
copy of it, and doubles the entries in the copy: you make FIVE function
calls to perform TWO conceptual operations. So the reader can't even map
a function call to a result.
With careful thought and further explanations from you, the reader (me)
eventually gets a mental model here. Your DSL has a single input which is
pipelined through a series of function calls by the - operator, plus a
separate stack. (I initially thought that, like Forth, your DSL was stack
based. But it isn't, is it?)
It seems to me that the - operator is only needed as syntactic sugar to
avoid using reverse Polish notation and an implicit stack. Instead of the
Forth-like:
[1,2,3,4] dup 2 *
your DSL has an explicit stack, and an explicit - operator to call a
function. Presumably [1,2] push would be a syntax error.
I think this is a good example of an inferior syntax. Contrast your:
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
with the equivalent RPL:
[1,2,3,4] dup 2 *
Now *that* is a pleasure to read, once you wrap your head around reverse
Polish notation and the concept of a stack. Which you need in your DSL
anyway, to understand push and pop.
You say that this is an easier way to get the same result:
[1,2,3,4] - [id, by(2)]
but it isn't, is it? The more complex example above ends up with two
streams joined in a single flow:
[1,2,3,4]|[2,4,6,8]
whereas the shorter version using the magic id gives you a single
stream containing nested streams:
[[1,2,3,4], [2,4,6,8]]
So, how could you make this more readable?
* Don't fight the reader's expectations. If they've programmed in Unix
shells, they expect | as the pipelining operator. If they haven't, they
probably will find easy to read as a dataflow operator. Either way,
they're probably used to seeing a|b as meaning or (as in this stream,
or this stream) rather than the way you seem to be using it (this
stream, and this stream).
Here's my first attempt at improved syntax that doesn't fight the user:
[1,2,3,4] push by(2) 'double' pop val('double')
push and pop are poor choices of words. Push does not actually push
its input onto the stack, which would leave the input stream empty. It
makes a copy. You explain what they do:
Flows can be saved (push) and restored (pop)
so why not just use SAVE and RESTORE as your functions? Or if they're too
verbose, STO and RCL, or my preference, store and recall.
[1,2,3,4] store by(2) 'double' recall val('double')
I'm still not happy with for the join operator. I think that the use of
+ for concatenate and for join is just one of those arbitrary choices
that the user will have to learn. Although I'm tempted to try using a
colon instead.
[1,2,3]:[4,5,6]
would be a flow with two streams.
I don't like the syntax for defining and using names. Here's a random
thought:
[1,2,3,4] store by(2) @double recall double
Use @name to store to a name, and the name alone to retrieve from it. But
I haven't given this too much thought, so it too might suck.
Some other problems with your DSL:
A flow can be transformed:
[1,2] - f =
