Re: [proto] : Proto transform with state
On 1/29/2011 7:49 PM, Eric Niebler wrote: > Bills be damned. I just committed to trunk an implementation of > proto::let, along with tests and reference docs. End-user docs are still > todo. As often happens, I woke up this morning knowing this code was broken, so I pulled it. I think I finally know how to fix it, though. -- Eric Niebler BoostPro Computing http://www.boostpro.com signature.asc Description: OpenPGP digital signature ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 11/18/2010 4:56 AM, Eric Niebler wrote:
> I think Proto transforms need a "let" statement for storing intermediate
> results. Maybe something like this:
>
> struct RenumberFun
> : proto::fold<
> _
> , make_pair(fusion::vector0<>(), proto::_state)
> , let<
> _a( Renumber(_, second(proto::_state))> )
> , make_pair(
> push_back(
> first(proto::_state)
> , first(_a)
> )
> , second(_a)
> )
> >
> >
> {};
>
> I haven't a clue how this would be implemented.
>
> It's fun to think about this stuff, but I wish it actually payed the bills.
Bills be damned. I just committed to trunk an implementation of
proto::let, along with tests and reference docs. End-user docs are still
todo.
--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Eric Niebler wrote:
> On 12/7/2010 3:13 PM, Thomas Heller wrote:
>> Eric Niebler wrote:
>>> Now they do: T()(e,s,d). Inside T::impl, D had better be the type of d.
>>> Nowhere does the _data transform appear in this code, so changing _data
>>> to be "smart" about environments and scopes won't save you if you've
>>> monkeyed with the data parameter.
>>
>> Very true. Something like proto::transform_env_impl could help. Introduce
>> a new type of primitive transform which is aware of this environment. The
>> usual transform_impl can still be used.
>> By calling T()(e,s,d) you just create a 2-tuple. The first parameter is
>> the state, second data.
>> Just thinking out loud here ...
>
> So transform_impl strips the data parameter of the let scope stuff, and
> the local variables like _a don't use transform_impl and see the scope
> with the locals?
>
> Well, it's not that simple Consider:
>
> make< int(_a) >
>
> If make::impl uses transform_impl, it strips the let scope before _a
> gets to see the local variables. If this is to work, most of Proto's
> transforms must be "special" and pass the let scope through. That means
> proto::call, proto::make, proto::lazy and proto::when, at least. But ...
> it just might work. Good thinking!
>
>
>
> No, wait a minute. Look again:
>
> struct T : transform {
>
> template
> struct impl : transform_impl {
>
> // do something with D
>
> };
>
> };
>
> T::impl gets passed D *before* transform_impl has a change to fix it up.
> Any existing transform that actually uses D and assumes it's what they
> passed in is going to be totally hosed. In order to make existing
> transforms let-friendly, they would all need to be changed. That's no
> good.
>
> Bummer, I was excited for a minute there. :-/
Yes ... this would basically mean a complete rewrite ... no good.
--> Proto V5, you might save the thought for the C++0x rewrite :)
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 12/7/2010 3:13 PM, Thomas Heller wrote:
> Eric Niebler wrote:
>> Now they do: T()(e,s,d). Inside T::impl, D had better be the type of d.
>> Nowhere does the _data transform appear in this code, so changing _data
>> to be "smart" about environments and scopes won't save you if you've
>> monkeyed with the data parameter.
>
> Very true. Something like proto::transform_env_impl could help. Introduce a
> new type of primitive transform which is aware of this environment. The
> usual transform_impl can still be used.
> By calling T()(e,s,d) you just create a 2-tuple. The first parameter is the
> state, second data.
> Just thinking out loud here ...
So transform_impl strips the data parameter of the let scope stuff, and
the local variables like _a don't use transform_impl and see the scope
with the locals?
Well, it's not that simple Consider:
make< int(_a) >
If make::impl uses transform_impl, it strips the let scope before _a
gets to see the local variables. If this is to work, most of Proto's
transforms must be "special" and pass the let scope through. That means
proto::call, proto::make, proto::lazy and proto::when, at least. But ...
it just might work. Good thinking!
No, wait a minute. Look again:
struct T : transform {
template
struct impl : transform_impl {
// do something with D
};
};
T::impl gets passed D *before* transform_impl has a change to fix it up.
Any existing transform that actually uses D and assumes it's what they
passed in is going to be totally hosed. In order to make existing
transforms let-friendly, they would all need to be changed. That's no good.
Bummer, I was excited for a minute there. :-/
--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Eric Niebler wrote:
> On 12/7/2010 2:58 PM, Thomas Heller wrote:
>> Eric Niebler wrote:
>>
>>> On 12/7/2010 12:57 PM, Thomas Heller wrote:
*Dough* misunderstanding here. I didn't mean to clean up the phoenix
scope expressions with the help of proto::let. I was thinking, maybe
proto::let can borrow something from phoenix scopes on a conceptual
level.
>>>
>>> Oh, sure. How does Phoenix handle let scopes? Are local variables
>>> statically or dynamically scoped? How is it accomplished?
>>
>> Phoenix in general has an abstract concept of an environment. This
>> environment is used to store the arguments of a lambda expression in a
>> tuple. This leads to the only requirement this environment must have:
>> access the arguments with a compile-time index in that tuple, using
>> fusion::at_c(env).
>> When having a let statement, A new environment is created which acts as a
>> map like data structure, indexed by the local names. Additionally the
>> above requirements are fulfilled.
>> Local variables are dynamically scoped. Additionally you can access
>> locals defined in some outer let scope.
>>
>> This is basically how it is working. I was thinking, maybe, proto can
>> adapt this concept of an environment. This would allow pretty nifty
>> things, like not being tied to only state and data, which really can be a
>> limitation, sometimes. Backwards compatibility can be provided by
>> transparently change proto::_state and proto::_data to do the right
>> thing.
>
> That is not sufficient to preserve back-compat. Imagine someone has
> defined a PrimitiveTransform T:
>
> struct T : transform {
>
> template
> struct impl : transform_impl {
>
> // do something with D
>
> };
>
> };
>
> Now they do: T()(e,s,d). Inside T::impl, D had better be the type of d.
> Nowhere does the _data transform appear in this code, so changing _data
> to be "smart" about environments and scopes won't save you if you've
> monkeyed with the data parameter.
Very true. Something like proto::transform_env_impl could help. Introduce a
new type of primitive transform which is aware of this environment. The
usual transform_impl can still be used.
By calling T()(e,s,d) you just create a 2-tuple. The first parameter is the
state, second data.
Just thinking out loud here ...
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 12/7/2010 2:58 PM, Thomas Heller wrote:
> Eric Niebler wrote:
>
>> On 12/7/2010 12:57 PM, Thomas Heller wrote:
>>>
>>> *Dough* misunderstanding here. I didn't mean to clean up the phoenix
>>> scope expressions with the help of proto::let. I was thinking, maybe
>>> proto::let can borrow something from phoenix scopes on a conceptual
>>> level.
>>
>> Oh, sure. How does Phoenix handle let scopes? Are local variables
>> statically or dynamically scoped? How is it accomplished?
>
> Phoenix in general has an abstract concept of an environment. This
> environment is used to store the arguments of a lambda expression in a
> tuple. This leads to the only requirement this environment must have:
> access the arguments with a compile-time index in that tuple, using
> fusion::at_c(env).
> When having a let statement, A new environment is created which acts as a
> map like data structure, indexed by the local names. Additionally the above
> requirements are fulfilled.
> Local variables are dynamically scoped. Additionally you can access locals
> defined in some outer let scope.
>
> This is basically how it is working. I was thinking, maybe, proto can adapt
> this concept of an environment. This would allow pretty nifty things, like
> not being tied to only state and data, which really can be a limitation,
> sometimes. Backwards compatibility can be provided by transparently change
> proto::_state and proto::_data to do the right thing.
That is not sufficient to preserve back-compat. Imagine someone has
defined a PrimitiveTransform T:
struct T : transform {
template
struct impl : transform_impl {
// do something with D
};
};
Now they do: T()(e,s,d). Inside T::impl, D had better be the type of d.
Nowhere does the _data transform appear in this code, so changing _data
to be "smart" about environments and scopes won't save you if you've
monkeyed with the data parameter.
--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Eric Niebler wrote: > On 12/7/2010 12:57 PM, Thomas Heller wrote: >> >> *Dough* misunderstanding here. I didn't mean to clean up the phoenix >> scope expressions with the help of proto::let. I was thinking, maybe >> proto::let can borrow something from phoenix scopes on a conceptual >> level. > > Oh, sure. How does Phoenix handle let scopes? Are local variables > statically or dynamically scoped? How is it accomplished? Phoenix in general has an abstract concept of an environment. This environment is used to store the arguments of a lambda expression in a tuple. This leads to the only requirement this environment must have: access the arguments with a compile-time index in that tuple, using fusion::at_c(env). When having a let statement, A new environment is created which acts as a map like data structure, indexed by the local names. Additionally the above requirements are fulfilled. Local variables are dynamically scoped. Additionally you can access locals defined in some outer let scope. This is basically how it is working. I was thinking, maybe, proto can adapt this concept of an environment. This would allow pretty nifty things, like not being tied to only state and data, which really can be a limitation, sometimes. Backwards compatibility can be provided by transparently change proto::_state and proto::_data to do the right thing. ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 12/7/2010 12:57 PM, Thomas Heller wrote: > > *Dough* misunderstanding here. I didn't mean to clean up the phoenix scope > expressions with the help of proto::let. I was thinking, maybe proto::let > can borrow something from phoenix scopes on a conceptual level. Oh, sure. How does Phoenix handle let scopes? Are local variables statically or dynamically scoped? How is it accomplished? -- Eric Niebler BoostPro Computing http://www.boostpro.com signature.asc Description: OpenPGP digital signature ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Eric Niebler wrote: > On 12/6/2010 4:50 PM, Thomas Heller wrote: >> Eric Niebler wrote: >>> I played with the let transform idea over the weekend. It *may* be >>> possible to accomplish without the two problems I described above. See >>> the attached let transform (needs latest Proto trunk). I'm also >>> attaching the Renumber example, reworked to use let. > >> >> Without having looked at it too much ... this looks a lot like the >> environment in phoenix. Maybe this helps in cleaning it out a bit. > > I tend to doubt it would help clean up the implementation of Phoenix > environments. These features exist on different meta-levels: one > (proto::let) is a feature for compiler-construction (Proto), the other > (phoenix::let) is a language feature (Phoenix). The have roughly the > same purpose within their purview, but as their purviews are separated > by one great, big Meta, it's not clear that they have anything to do > with each other. *Dough* misunderstanding here. I didn't mean to clean up the phoenix scope expressions with the help of proto::let. I was thinking, maybe proto::let can borrow something from phoenix scopes on a conceptual level. ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 12/6/2010 4:50 PM, Thomas Heller wrote: > Eric Niebler wrote: >> I played with the let transform idea over the weekend. It *may* be >> possible to accomplish without the two problems I described above. See >> the attached let transform (needs latest Proto trunk). I'm also >> attaching the Renumber example, reworked to use let. > > Without having looked at it too much ... this looks a lot like the > environment in phoenix. Maybe this helps in cleaning it out a bit. I tend to doubt it would help clean up the implementation of Phoenix environments. These features exist on different meta-levels: one (proto::let) is a feature for compiler-construction (Proto), the other (phoenix::let) is a language feature (Phoenix). The have roughly the same purpose within their purview, but as their purviews are separated by one great, big Meta, it's not clear that they have anything to do with each other. -- Eric Niebler BoostPro Computing http://www.boostpro.com ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Eric Niebler wrote: > On 11/18/2010 3:31 PM, Eric Niebler wrote: >> On 11/18/2010 1:45 PM, Thomas Heller wrote: >>> Eric Niebler writes: It's REALLY hard. The let context needs to be bundled with the Expr, State, or Data parameters somehow, but in a way that's transparent. I don't actually know if it's possible. >>> >>> Very hard ... yeah. I am thinking that we can maybe save these variables >>> in the transform? >> >> I'm thinking we just stuff it into the Data parameter. We have a >> let_scope template that is effectively a pair containing: >> >> 1. The user's original Data, and >> 2. A Fusion map from local variables (_a) to values. >> >> The let transform evaluates the bindings and stores the result in the >> let_scope's Fusion map alongside the user's Data. We pass the let_scope >> as the new Data parameter. _a is itself a transform that looks up the >> value in Data's Fusion map. The proto::_data transform is changed to be >> aware of let_scope and return only the original user's Data. This can >> work. We also need to be sure not to break the new >> proto::external_transform. >> >> The problems with this approach as I see it: >> >> 1. It's not completely transparent. Custom primitive transforms will see >> that the Data parameter has been monkeyed with. >> >> 2. Local variables like _a are not lexically scoped. They are, in fact, >> dynamically scoped. That is, you can access _a outside of a let<> >> clause, as long as you've been called from within a let clause. >> >> Might be worth it. But as there's no pressing need, I'm content to let >> this simmer. Maybe we can think of something better. > > I played with the let transform idea over the weekend. It *may* be > possible to accomplish without the two problems I described above. See > the attached let transform (needs latest Proto trunk). I'm also > attaching the Renumber example, reworked to use let. > > This code is NOT ready for prime time. I'm not convinced it behaves > sensibly in all cases. I'm only posting it as a curiosity. You're insane > if you use this in production code. Etc, etc. Without having looked at it too much ... this looks a lot like the environment in phoenix. Maybe this helps in cleaning it out a bit. ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 11/18/2010 3:31 PM, Eric Niebler wrote:
> On 11/18/2010 1:45 PM, Thomas Heller wrote:
>> Eric Niebler writes:
>>> It's REALLY hard. The let context needs to be bundled with the Expr,
>>> State, or Data parameters somehow, but in a way that's transparent. I
>>> don't actually know if it's possible.
>>
>> Very hard ... yeah. I am thinking that we can maybe save these variables in
>> the
>> transform?
>
> I'm thinking we just stuff it into the Data parameter. We have a
> let_scope template that is effectively a pair containing:
>
> 1. The user's original Data, and
> 2. A Fusion map from local variables (_a) to values.
>
> The let transform evaluates the bindings and stores the result in the
> let_scope's Fusion map alongside the user's Data. We pass the let_scope
> as the new Data parameter. _a is itself a transform that looks up the
> value in Data's Fusion map. The proto::_data transform is changed to be
> aware of let_scope and return only the original user's Data. This can
> work. We also need to be sure not to break the new
> proto::external_transform.
>
> The problems with this approach as I see it:
>
> 1. It's not completely transparent. Custom primitive transforms will see
> that the Data parameter has been monkeyed with.
>
> 2. Local variables like _a are not lexically scoped. They are, in fact,
> dynamically scoped. That is, you can access _a outside of a let<>
> clause, as long as you've been called from within a let clause.
>
> Might be worth it. But as there's no pressing need, I'm content to let
> this simmer. Maybe we can think of something better.
I played with the let transform idea over the weekend. It *may* be
possible to accomplish without the two problems I described above. See
the attached let transform (needs latest Proto trunk). I'm also
attaching the Renumber example, reworked to use let.
This code is NOT ready for prime time. I'm not convinced it behaves
sensibly in all cases. I'm only posting it as a curiosity. You're insane
if you use this in production code. Etc, etc.
--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
#ifndef BOOST_PP_IS_ITERATING
///
/// \file let.hpp
/// Contains definition of the let transform.
//
// Copyright 2010 Eric Niebler. Distributed under the Boost
// Software License, Version 1.0. (See accompanying file
// LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
#ifndef BOOST_PROTO_TRANSFORM_LET_HPP_EAN_12_04_2010
#define BOOST_PROTO_TRANSFORM_LET_HPP_EAN_12_04_2010
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
//#include
#include
#include
#include
#include
#include
#include
#include
#include
namespace boost { namespace proto
{
// Fwd declarations to be moved to proto_fwd.hpp
template<
BOOST_PP_ENUM_PARAMS_WITH_A_DEFAULT(BOOST_PROTO_MAX_ARITY, typename
Local, void)
, typename Transform = void
>
struct let;
template
struct local;
namespace detail
{
// A structure that holds both a map of local variables as
// well as the original Data parameter passed to the let transform
template
struct let_scope
{
typedef LocalMap local_map_type;
typedef Data data_type;
let_scope(LocalMap l, Data d)
: locals(l)
, data(d)
{}
LocalMap locals;
Data data;
private:
let_scope &operator=(let_scope const &);
};
template
struct init_local_map;
// A transnform that fetches the original data parameter out of a
let_scope
struct _get_data : transform<_get_data>
{
template
struct impl;
template
struct impl &>
: transform_impl &>
{
typedef Data result_type;
Data operator()(
typename impl::expr_param
, typename impl::state_param
, typename impl::data_param d
) const
{
return d.data;
}
};
};
template
struct rewrite_transform;
template
struct rewrite_args
{
typedef T type;
};
// rewrite_object if T is not a template
template<
typename T
BOOST_MPL_AUX_LAMBDA_ARITY_PARAM(long Arity =
mpl::aux::tem
Re: [proto] : Proto transform with state
On 11/18/2010 1:45 PM, Thomas Heller wrote:
> Eric Niebler writes:
>> On 11/18/2010 6:09 AM, Thomas Heller wrote:
>>> Here goes the renumbering example:
>>> http://codepad.org/K0TZamPb
>
>> Unfortunately, this doesn't actually solve the reevaluation problem ...
>> it just hides it.
>
> Yes, exactly.
>
>> If "_a" gets replaced eagerly everywhere with
>> "Renumber(_, second(proto::_state))", then that transform will actually
>> get run every time. What I want is a way to evaluate the transform,
>> store the result in a temporary, and have "_a" refer to that temporary.
>
> Yes, I thought of doing this ... but could not find a solution. The question
> is,
> do we really need this behaviour? Is replacement not enough?
> Could you make up a usecase? I can not think of one ... Regarding that
> counting
> example ... it can be solved by this pair hack.
The Renumber example is itself a use-case. I had to hack around the lack
of this feature with the silly type_of transform:
struct RenumberFun
: proto::fold<
_
, make_pair(fusion::vector0<>(), proto::_state)
, make_pair(
push_back(
first(proto::_state)
, first(Renumber(_, second(proto::_state)))
)
, type_of
)
>
{};
That should simply be:
struct RenumberFun
: proto::fold<
_
, make_pair(fusion::vector0<>(), proto::_state)
, let<
_a(Renumber(_, second(proto::_state)))
, make_pair(
push_back(first(proto::_state), first(_a))
, second(_a)
)
>
>
{};
See? If Renumber gets invoked once and the result "stored" in _a, then
we can just do first(_a) and then second(_a) and not incur any runtime
overhead. type_of was a hack that took advantage of the fact that the
result of second(_a) contains no interesting runtime state (it's just a
MPL int_). There will be times when that's not the case.
>> It's REALLY hard. The let context needs to be bundled with the Expr,
>> State, or Data parameters somehow, but in a way that's transparent. I
>> don't actually know if it's possible.
>
> Very hard ... yeah. I am thinking that we can maybe save these variables in
> the
> transform?
I'm thinking we just stuff it into the Data parameter. We have a
let_scope template that is effectively a pair containing:
1. The user's original Data, and
2. A Fusion map from local variables (_a) to values.
The let transform evaluates the bindings and stores the result in the
let_scope's Fusion map alongside the user's Data. We pass the let_scope
as the new Data parameter. _a is itself a transform that looks up the
value in Data's Fusion map. The proto::_data transform is changed to be
aware of let_scope and return only the original user's Data. This can
work. We also need to be sure not to break the new
proto::external_transform.
The problems with this approach as I see it:
1. It's not completely transparent. Custom primitive transforms will see
that the Data parameter has been monkeyed with.
2. Local variables like _a are not lexically scoped. They are, in fact,
dynamically scoped. That is, you can access _a outside of a let<>
clause, as long as you've been called from within a let clause.
Might be worth it. But as there's no pressing need, I'm content to let
this simmer. Maybe we can think of something better.
--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
signature.asc
Description: OpenPGP digital signature
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Eric Niebler writes: > On 11/18/2010 6:09 AM, Thomas Heller wrote: > > Here goes the renumbering example: > > http://codepad.org/K0TZamPb > Unfortunately, this doesn't actually solve the reevaluation problem ... > it just hides it. Yes, exactly. > If "_a" gets replaced eagerly everywhere with > "Renumber(_, second(proto::_state))", then that transform will actually > get run every time. What I want is a way to evaluate the transform, > store the result in a temporary, and have "_a" refer to that temporary. Yes, I thought of doing this ... but could not find a solution. The question is, do we really need this behaviour? Is replacement not enough? Could you make up a usecase? I can not think of one ... Regarding that counting example ... it can be solved by this pair hack. > It's REALLY hard. The let context needs to be bundled with the Expr, > State, or Data parameters somehow, but in a way that's transparent. I > don't actually know if it's possible. Very hard ... yeah. I am thinking that we can maybe save these variables in the transform? ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 11/18/2010 6:09 AM, Thomas Heller wrote:
> Ok ... I implemented let!
You've been busy. :-)
> Here goes the renumbering example:
> http://codepad.org/K0TZamPb
>
> The change is in line 296 rendering RenumberFun to:
> struct RenumberFun
> : proto::fold<
> _
> , make_pair(fusion::vector<>(), proto::_state)
> , let<
> _a(Renumber(_, second(proto::_state)))
> , make_pair(
> push_back(
> first(proto::_state)
> , first(_a)
> )
> , type_of
> )
> >
> >
> {};
>
> The implementation of let actually was quite easy ... here is how it works:
>
> let is a transform taking definitions of local variables
> and
> the transform these locals will get applied to.
> A local definition is in the form of: LocalName(LocalTransform)
> If the specified transform has LocalName embedded, it will get replaced by
> LocalTransform.
Unfortunately, this doesn't actually solve the reevaluation problem ...
it just hides it. If "_a" gets replaced eagerly everywhere with
"Renumber(_, second(proto::_state))", then that transform will actually
get run every time. What I want is a way to evaluate the transform,
store the result in a temporary, and have "_a" refer to that temporary.
It's REALLY hard. The let context needs to be bundled with the Expr,
State, or Data parameters somehow, but in a way that's transparent. I
don't actually know if it's possible.
--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 11/18/2010 12:31 PM, Dave Abrahams wrote: > On Wed, Nov 17, 2010 at 1:46 PM, Eric Niebler wrote: > >> See the attached code. I wish I had a better answer. It sure would be >> nice to generalize this for other times when new state needs to bubble >> up and back down. > > Forgive me if I'm telling you something you already know, but there's > probably already an abstraction for this out there; it sounds like > classic attribute-grammar stuff. If you have more insights to share, I'm all ears. I don't know enough about attribute-grammar to see the applicability. -- Eric Niebler BoostPro Computing http://www.boostpro.com ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Thomas Heller writes:
> Thomas Heller writes:
> > Eric Niebler writes:
>
> > > Notice that the Renumber algorithm needs to be invoked twice with the
> > > same arguments. In this case, we can avoid the runtime overhead of the
> > > second invocation by just using the type information, but that's not
> > > always going to be the case. There doesn't seem to be a way around it,
> > > either.
> > >
> > > I think Proto transforms need a "let" statement for storing intermediate
> > > results. Maybe something like this:
> > >
> > > It's fun to think about this stuff, but I wish it actually payed the
bills.
> >
> > Ok ... I implemented let!
> >
> > Here goes the renumbering example:
> > http://codepad.org/K0TZamPb
> >
> > The change is in line 296 rendering RenumberFun to:
> > struct RenumberFun
> > : proto::fold<
> > _
> > , make_pair(fusion::vector<>(), proto::_state)
> > , let<
> > _a(Renumber(_, second(proto::_state)))
> > , make_pair(
> > push_back(
> > first(proto::_state)
> > , first(_a)
> > )
> > , type_of
> > )
> > >
> > >
> > {};
> >
> > The implementation of let actually was quite easy ... here is how it works:
> >
> > let is a transform taking definitions of local variables
> and
> > the transform these locals will get applied to.
> > A local definition is in the form of: LocalName(LocalTransform)
> > If the specified transform has LocalName embedded, it will get replaced by
> > LocalTransform.
> > I also implemented the definition of more than one local ... this is done
> > by
> > reusing proto::and_:
> >
> > let,
> > Transform>
> > The replacement is done from the end to the beginning, making it possible
> > to
> > refer in a LocalTransformN to a LocalNameN-1, this gets replaced
> automatically!
>
> Ok, the implementation in the previous post had some bugs.
> Here is a updated one: http://codepad.org/ljjBYqHr
>
> With the relevant example, showing of reuse of locals:
> struct RenumberFun
> : proto::fold<
> _
> , make_pair(fusion::vector<>(), proto::_state)
> , let<
> proto::and_<
> _a(second(proto::_state))
> , _b(Renumber(_, _a))
> >
> , make_pair(
> push_back(
> first(proto::_state)
> , first(_b)
> )
> , type_of
> )
> >
> >
> {};
Ok ... it was pointed out that you might want to give _a a meaningful name.
So here it goes:
struct RenumberFun
: proto::fold<
_
, make_pair(fusion::vector<>(), proto::_state)
, let<
Renumber(Renumber(_, second(proto::_state)))
, make_pair(
push_back(
first(proto::_state)
, first(Renumber)
)
, type_of
)
>
>
{};
Which aliases Renumber to Renumber(_, second(proto::_state)) which is quite
cool. It turns out that you can have any type as the local name ;)
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Thomas Heller writes:
>
> Eric Niebler writes:
> >
> > Notice that the Renumber algorithm needs to be invoked twice with the
> > same arguments. In this case, we can avoid the runtime overhead of the
> > second invocation by just using the type information, but that's not
> > always going to be the case. There doesn't seem to be a way around it,
> > either.
> >
> > I think Proto transforms need a "let" statement for storing intermediate
> > results. Maybe something like this:
> >
> > struct RenumberFun
> > : proto::fold<
> > _
> > , make_pair(fusion::vector0<>(), proto::_state)
> > , let<
> > _a( Renumber(_, second(proto::_state))> )
> > , make_pair(
> > push_back(
> > first(proto::_state)
> > , first(_a)
> > )
> > , type_of
> > )
> > >
> > >
> > {};
> >
> > I haven't a clue how this would be implemented.
> >
> > It's fun to think about this stuff, but I wish it actually payed the bills.
>
> Ok ... I implemented let!
>
> Here goes the renumbering example:
> http://codepad.org/K0TZamPb
>
> The change is in line 296 rendering RenumberFun to:
> struct RenumberFun
> : proto::fold<
> _
> , make_pair(fusion::vector<>(), proto::_state)
> , let<
> _a(Renumber(_, second(proto::_state)))
> , make_pair(
> push_back(
> first(proto::_state)
> , first(_a)
> )
> , type_of
> )
> >
> >
> {};
>
> The implementation of let actually was quite easy ... here is how it works:
>
> let is a transform taking definitions of local variables
and
> the transform these locals will get applied to.
> A local definition is in the form of: LocalName(LocalTransform)
> If the specified transform has LocalName embedded, it will get replaced by
> LocalTransform.
> I also implemented the definition of more than one local ... this is done by
> reusing proto::and_:
>
> let LocalNameN(LocalTransformN)>,
> Transform>
> The replacement is done from the end to the beginning, making it possible to
> refer in a LocalTransformN to a LocalNameN-1, this gets replaced
automatically!
>
> Hope that helps!
>
> Thomas
>
>
Ok, the implementation in the previous post had some bugs.
Here is a updated one: http://codepad.org/ljjBYqHr
With the relevant example, showing of reuse of locals:
struct RenumberFun
: proto::fold<
_
, make_pair(fusion::vector<>(), proto::_state)
, let<
proto::and_<
_a(second(proto::_state))
, _b(Renumber(_, _a))
>
, make_pair(
push_back(
first(proto::_state)
, first(_b)
)
, type_of
)
>
>
{};
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Eric Niebler writes:
>
> On 11/17/2010 2:18 PM, joel falcou wrote:
> > On 17/11/10 19:46, Eric Niebler wrote:
> >> See the attached code. I wish I had a better answer. It sure would be
> >> nice to generalize this for other times when new state needs to bubble
> >> up and back down.
> >
> > Just chiming in. We had the exact same problem in quaff where needed to
> > carry on a process ID over the trasnform of parallel statement. If it can
> > make you worry less Eric, we ended with the exact same workaround.
>
> There's another issue. Look here:
>
> // don't evaluate T at runtime, but default-construct
> // an object of T's result type.
> template
> struct type_of
> : proto::make >
> {};
>
> struct RenumberFun
> : proto::fold<
> _
> , make_pair(fusion::vector0<>(), proto::_state)
> , make_pair(
> push_back(
> first(proto::_state)
> //--1
> , first(Renumber(_, second(proto::_state)))
> )
> //---2
> , type_of
> )
> >
> {};
>
> Notice that the Renumber algorithm needs to be invoked twice with the
> same arguments. In this case, we can avoid the runtime overhead of the
> second invocation by just using the type information, but that's not
> always going to be the case. There doesn't seem to be a way around it,
> either.
>
> I think Proto transforms need a "let" statement for storing intermediate
> results. Maybe something like this:
>
> struct RenumberFun
> : proto::fold<
> _
> , make_pair(fusion::vector0<>(), proto::_state)
> , let<
> _a( Renumber(_, second(proto::_state))> )
> , make_pair(
> push_back(
> first(proto::_state)
> , first(_a)
> )
> , type_of
> )
> >
> >
> {};
>
> I haven't a clue how this would be implemented.
>
> It's fun to think about this stuff, but I wish it actually payed the bills.
Ok ... I implemented let!
Here goes the renumbering example:
http://codepad.org/K0TZamPb
The change is in line 296 rendering RenumberFun to:
struct RenumberFun
: proto::fold<
_
, make_pair(fusion::vector<>(), proto::_state)
, let<
_a(Renumber(_, second(proto::_state)))
, make_pair(
push_back(
first(proto::_state)
, first(_a)
)
, type_of
)
>
>
{};
The implementation of let actually was quite easy ... here is how it works:
let is a transform taking definitions of local variables and
the transform these locals will get applied to.
A local definition is in the form of: LocalName(LocalTransform)
If the specified transform has LocalName embedded, it will get replaced by
LocalTransform.
I also implemented the definition of more than one local ... this is done by
reusing proto::and_:
let,
Transform>
The replacement is done from the end to the beginning, making it possible to
refer in a LocalTransformN to a LocalNameN-1, this gets replaced automatically!
Hope that helps!
Thomas
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
Eric Niebler wrote:
> On 11/17/2010 2:18 PM, joel falcou wrote:
>> On 17/11/10 19:46, Eric Niebler wrote:
>>> See the attached code. I wish I had a better answer. It sure would be
>>> nice to generalize this for other times when new state needs to bubble
>>> up and back down.
>>
>> Just chiming in. We had the exact same problem in quaff where needed to
>> carry on a process ID over the trasnform of parallel statement. If it can
>> make you worry less Eric, we ended with the exact same workaround.
>
> There's another issue. Look here:
>
> // don't evaluate T at runtime, but default-construct
> // an object of T's result type.
> template
> struct type_of
> : proto::make >
> {};
>
> struct RenumberFun
> : proto::fold<
> _
> , make_pair(fusion::vector0<>(), proto::_state)
> , make_pair(
> push_back(
> first(proto::_state)
> //--1
> , first(Renumber(_, second(proto::_state)))
> )
> //---2
> , type_of
> )
> >
> {};
>
> Notice that the Renumber algorithm needs to be invoked twice with the
> same arguments. In this case, we can avoid the runtime overhead of the
> second invocation by just using the type information, but that's not
> always going to be the case. There doesn't seem to be a way around it,
> either.
>
> I think Proto transforms need a "let" statement for storing intermediate
> results. Maybe something like this:
>
> struct RenumberFun
> : proto::fold<
> _
> , make_pair(fusion::vector0<>(), proto::_state)
> , let<
> _a( Renumber(_, second(proto::_state))> )
> , make_pair(
> push_back(
> first(proto::_state)
> , first(_a)
> )
> , type_of
> )
> >
> >
> {};
>
> I haven't a clue how this would be implemented.
>
> It's fun to think about this stuff, but I wish it actually payed the
> bills.
>
WOW! A let statement this would indeed make proto even more awesome!
Let's see what we can do ;)
Btw, i just finished implementing the unpack feature we were talking about
...
Short description:
Calling some_callable with the expression unpacked:
proto::when
Calling some_callable with an arbitrary sequence unpacked:
proto::when
Calling some_callable with an arbitrary sequence unpacked, and apply a proto
transform before:
proto::when
Additionally it is possible to have arbitrary parameters before or after
unpack. The implementation is located at:
http://svn.boost.org/svn/boost/sandbox/SOC/2010/phoenix3/boost/phoenix/core/unpack.hpp
Just a whole mess of preprocessor generation ... this is not really fast to
compile at the moment, PROTO_MAX_ARITY of 5 is fine, everything above will
just blow the compiler :(
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 11/17/2010 2:18 PM, joel falcou wrote:
> On 17/11/10 19:46, Eric Niebler wrote:
>> See the attached code. I wish I had a better answer. It sure would be
>> nice to generalize this for other times when new state needs to bubble
>> up and back down.
>
> Just chiming in. We had the exact same problem in quaff where needed to
> carry on a process ID over the trasnform of parallel statement. If it can
> make you worry less Eric, we ended with the exact same workaround.
There's another issue. Look here:
// don't evaluate T at runtime, but default-construct
// an object of T's result type.
template
struct type_of
: proto::make >
{};
struct RenumberFun
: proto::fold<
_
, make_pair(fusion::vector0<>(), proto::_state)
, make_pair(
push_back(
first(proto::_state)
//--1
, first(Renumber(_, second(proto::_state)))
)
//---2
, type_of
)
>
{};
Notice that the Renumber algorithm needs to be invoked twice with the
same arguments. In this case, we can avoid the runtime overhead of the
second invocation by just using the type information, but that's not
always going to be the case. There doesn't seem to be a way around it,
either.
I think Proto transforms need a "let" statement for storing intermediate
results. Maybe something like this:
struct RenumberFun
: proto::fold<
_
, make_pair(fusion::vector0<>(), proto::_state)
, let<
_a( Renumber(_, second(proto::_state))> )
, make_pair(
push_back(
first(proto::_state)
, first(_a)
)
, type_of
)
>
>
{};
I haven't a clue how this would be implemented.
It's fun to think about this stuff, but I wish it actually payed the bills.
--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
___
proto mailing list
proto@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 17/11/10 19:46, Eric Niebler wrote: See the attached code. I wish I had a better answer. It sure would be nice to generalize this for other times when new state needs to bubble up and back down. Just chiming in. We had the exact same problem in quaff where needed to carry on a process ID over the trasnform of parallel statement. If it can make you worry less Eric, we ended with the exact same workaround. ___ proto mailing list proto@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/proto
Re: [proto] : Proto transform with state
On 11/17/2010 1:47 AM, Manjunath Kudlur wrote:
> I am trying to create a transform where I replace occurrences of
> proto::terminal with proto::terminal > >. I
> want N to increment for every proto::terminal encountered. So
> far I have the following code :
> I pass in mpl::int_<0>() as the initial state and call mpl::next when
> I match proto::terminal. As the output would indicate, this is
> not working. Every time I think I understand proto transforms,
> something basic like this stumps me. I will be grateful for any hints
> on how to accomplish this.
Don't feel bad. This is EXTREMELY hard to do in Proto. :-( Most
algorithms only return one piece of information at each step. This task
requires returning two: the transformed expression and the new state.
Worse, after transforming a left child, the new state needs to be used
as input while transforming the right. Like I said, EXTREMELY hard.
Since your algorithm needs to return two pieces of information, it needs
to return a std::pair containing the new expression and the updated
state variable. And when transforming a non-terminal, you need to use
the fold algorithm to propagate the state from one invocation to the
next (again, bundling and unbundling into a pair as necessary). Fold can
build a fusion::vector of transformed children, which then needs to be
unpacked into an expression. (This step wouldn't be necessary if Proto
expressions were extensible Fusion sequences. )
See the attached code. I wish I had a better answer. It sure would be
nice to generalize this for other times when new state needs to bubble
up and back down.
--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
//#include "stdafx.h"
#include
#include
#include
#include
#include
#include
#include
namespace mpl = boost::mpl;
namespace proto = boost::proto;
namespace fusion = boost::fusion;
using proto::_;
struct term
{
friend std::ostream &operator<<(std::ostream &sout, term const &)
{
return sout << "term";
}
};
template
struct newterm
{
friend std::ostream &operator<<(std::ostream &sout, newterm const &)
{
return sout << "newterm<" << I::value << ">";
}
};
proto::terminal::type const a = {{}};
proto::terminal::type const b = {{}};
proto::terminal::type const c = {{}};
struct make_pair : proto::callable
{
template
struct result;
template
struct result
{
typedef std::pair type;
};
template
std::pair operator()(First const &first, Second const
&second) const
{
return std::make_pair(first, second);
}
};
struct first : proto::callable
{
template
struct result;
template
struct result
{
typedef typename Pair::first_type type;
};
template
typename Pair::first_type operator()(Pair const &pair) const
{
return pair.first;
}
};
struct second : proto::callable
{
template
struct result;
template
struct result
{
typedef typename Pair::second_type type;
};
template
typename Pair::second_type operator()(Pair const &pair) const
{
return pair.second;
}
};
struct push_back : proto::callable
{
template
struct result;
template
struct result
: fusion::result_of::push_back<
typename boost::add_const::type>::type
, typename boost::remove_const::type>::type
>
{};
template
typename fusion::result_of::push_back::type
operator ()(Seq const &seq, T const &t) const
{
return fusion::push_back(seq, t);
}
};
struct unpack_expr : proto::callable
{
template
struct result;
template
struct result
: proto::result_of::unpack_expr
{};
template
typename proto::result_of::unpack_expr::type
operator ()(Tag, Seq const &seq) const
{
return proto::unpack_expr(seq);
}
};
#if BOOST_WORKAROUND(BOOST_MSVC, BOOST_TESTED_AT(1600))
#define _make_terminal(x) call
#define Renumber(x,y) proto::call
#define push_back(x,y) proto::call
#define make_pair(x,y) proto::call
#define unpack_expr(x,y) proto::call
#define first(x) proto::call
#define second(x) proto::call
#endif
struct Renumber;
// don't evaluate T at runtime, but default-construct an object
// of T's result type.
template
struct type_of
: proto::make >
{};
struct RenumberFun
: proto::fold<
_
, make_pair(fusion::vector<>(), proto::_state)
, make_pair(
push_back(
first(proto::_state)
, first(Renumber(_, second(proto::_state)))
)
, type_of
)
>
{};
struct Renumber
: proto::or_<
proto::when<
proto::terminal
, make_pair(
proto::_make_terminal(newterm())
, mpl::next()
)
>
, proto::when<
proto::terminal<_>
, make_pair(proto::_byval(_), proto::_state)
>
, proto::otherwise<
make
