Re: [julia-users] Re: Organizing code around Jump models
I understand and concur with your comment regarding the proper use of
abstract types. And yes, defining appropriate abstractions is challenging.
A few more iterations will likely be required for us. For now, most of my
hesitations concern how to extend a given object and its functionality: say
using the scheme above or, and I am now only starting to experiment with
this, using the hooks provided by the base type. In the example above, we
could have proceeded by using the *JuMP* Model *ext* dictionary to store a
data structure providing what we need to pair with the JuMP model, ( i.e.:
a demand scenario, initial conditions, a list of contingencies, etc), as is
more or less done in the *JuMPStoch* and *JuMPeR* modules. But it is not
clear to me how we could then use polymorphism on the calls to *setGenCapa!,
setRamping!*, etc. Perhaps I should look more closely at the implementation
of *MathProgBase*' solver interface.
Thanks again for your comments!
Le jeudi 11 juin 2015 15:45:40 UTC-5, Miles Lubin a écrit :
There's a lot going on here, but a few observations:
- In the (unreleased) development version of JuMP, we store a dictionary
of variables used in the model (
https://github.com/JuliaOpt/JuMP.jl/pull/446), so you won't need to
manually keep track of this e.g., with the var dictionary above.
- That said, there's a bit of a performance penalty for going through
dictionaries. Don't worry about this until it becomes a bottleneck, though.
- Abstract types specify what do can do with a certain object, but not
what fields are available in an object. So it's a bit of poor style to
access the fields of an object in generic code (in setLineCapa!, you
access m.mip, but m is not guaranteed to have this field). I admit to
doing this myself sometimes, though. The suggested workaround is to define
accessor methods like mip(m::Model) = m.mip. Then the delegation/proxy
pattern above is replaced by defining new methods, e.g., mip(m::Model) =
m.uc.mip.
Does this help? It's always a challange to design good abstractions, and
we're definitely open to making it easier to build modular models like this
in JuMP. Are there Julia language features which you feel are making this
more difficult than you expect?
Best,
Miles
On Thu, Jun 4, 2015 at 10:28 PM, Francois Gilbert francois@gmail.com
javascript: wrote:
Hi Miles,
It would be easier for me to give some excerpts of the Julia sources I am
working with. I think it would suffice to illustrate the mechanic we are
trying to implement.
Say we have two abstract types:
abstract BasicUC
abstract CCModel:BasicUC
These are declared in separate modules. BasicUC stands for basic
unit-commitment and types that derive it have access to functionalities
such as:
function setLineCapa!(m::BasicUC)
JuMP.@addConstraint(m.mip, _contraint1[l = keys(m.net.branch), t
= m.timeIndex],
m.var[:flow][l,t] = m.net.branch[l].Fmax)
...
end
CCModel stands for an extended unit-commitment formulation, where robust
reserve requirement are implemented. Details are irrelevant. A type that
derives CCModel has access to, say:
function setRobustReserve!(m::CCModel)
...
end
Extending the functionality specific to BasicUC - let a type “Model” such
that
type Model{T:BasicUC} : CCModel
uc# Hook to BasicUC
# Proxies to BasicUC:
demand # exogenous stuff
mip # JuMP model
. . .
function Model(net,scenario)
this=new()
this.uc=T(net,scenario)
. . .
# Delegation to BasicUC
this.demand=this.uc.demand
this.mip=this.uc.mip
this.var=this.uc.var
. . .
# Specific to CCModel
this.contingencies=scenario[:contingencies]
JuMP.@defVar(this.mip, _post_gen[keys(this.net.gen),
keys(this.contingencies),
this.timeIndex])
this.var[:post_gen]= _post_gen
. . .
return this
end
end
Now, let a concrete type BasicModel be defined in, say module
BasicUnitCommitment, and such that :
BasicModel:BasicUC
Type Model{T} above can then be instantiated and tailored, with:
m=Model{BasicUnitCommitment.BaseModel}(net,scenario)
# from the base model (BasicUC)
setBalanceEq!(m)
setLineCapa!(m)
setInitRamping!(m)
# these are overloaded by CCModel
setObjective!(m)
setGenCapa!(m)
setRamping!(m)
# these are specific to CCModel
setPostBalanceEq!(m)
setRobustReserve!(m)
. . .
So that's the idea. Perhaps I am overdoing this, but I think having some
kind of encapsulation is essential for developing larger applications.
Emulating inheritance, as is done above, is rather straightforward, but
perhaps a little laborious, and there might be a better way to go about
it.
Re: [julia-users] Re: Organizing code around Jump models
There's a lot going on here, but a few observations:
- In the (unreleased) development version of JuMP, we store a dictionary of
variables used in the model (https://github.com/JuliaOpt/JuMP.jl/pull/446),
so you won't need to manually keep track of this e.g., with the var
dictionary above.
- That said, there's a bit of a performance penalty for going through
dictionaries. Don't worry about this until it becomes a bottleneck, though.
- Abstract types specify what do can do with a certain object, but not
what fields are available in an object. So it's a bit of poor style to
access the fields of an object in generic code (in setLineCapa!, you access
m.mip, but m is not guaranteed to have this field). I admit to doing this
myself sometimes, though. The suggested workaround is to define accessor
methods like mip(m::Model) = m.mip. Then the delegation/proxy pattern
above is replaced by defining new methods, e.g., mip(m::Model) = m.uc.mip.
Does this help? It's always a challange to design good abstractions, and
we're definitely open to making it easier to build modular models like this
in JuMP. Are there Julia language features which you feel are making this
more difficult than you expect?
Best,
Miles
On Thu, Jun 4, 2015 at 10:28 PM, Francois Gilbert
francois.gilbert...@gmail.com wrote:
Hi Miles,
It would be easier for me to give some excerpts of the Julia sources I am
working with. I think it would suffice to illustrate the mechanic we are
trying to implement.
Say we have two abstract types:
abstract BasicUC
abstract CCModel:BasicUC
These are declared in separate modules. BasicUC stands for basic
unit-commitment and types that derive it have access to functionalities
such as:
function setLineCapa!(m::BasicUC)
JuMP.@addConstraint(m.mip, _contraint1[l = keys(m.net.branch), t
= m.timeIndex],
m.var[:flow][l,t] = m.net.branch[l].Fmax)
...
end
CCModel stands for an extended unit-commitment formulation, where robust
reserve requirement are implemented. Details are irrelevant. A type that
derives CCModel has access to, say:
function setRobustReserve!(m::CCModel)
...
end
Extending the functionality specific to BasicUC - let a type “Model” such
that
type Model{T:BasicUC} : CCModel
uc# Hook to BasicUC
# Proxies to BasicUC:
demand # exogenous stuff
mip # JuMP model
. . .
function Model(net,scenario)
this=new()
this.uc=T(net,scenario)
. . .
# Delegation to BasicUC
this.demand=this.uc.demand
this.mip=this.uc.mip
this.var=this.uc.var
. . .
# Specific to CCModel
this.contingencies=scenario[:contingencies]
JuMP.@defVar(this.mip, _post_gen[keys(this.net.gen),
keys(this.contingencies),
this.timeIndex])
this.var[:post_gen]= _post_gen
. . .
return this
end
end
Now, let a concrete type BasicModel be defined in, say module
BasicUnitCommitment, and such that :
BasicModel:BasicUC
Type Model{T} above can then be instantiated and tailored, with:
m=Model{BasicUnitCommitment.BaseModel}(net,scenario)
# from the base model (BasicUC)
setBalanceEq!(m)
setLineCapa!(m)
setInitRamping!(m)
# these are overloaded by CCModel
setObjective!(m)
setGenCapa!(m)
setRamping!(m)
# these are specific to CCModel
setPostBalanceEq!(m)
setRobustReserve!(m)
. . .
So that's the idea. Perhaps I am overdoing this, but I think having some
kind of encapsulation is essential for developing larger applications.
Emulating inheritance, as is done above, is rather straightforward, but
perhaps a little laborious, and there might be a better way to go about
it.
François
Le mercredi 3 juin 2015 16:12:21 UTC-5, Miles Lubin a écrit :
Hi Francois,
Could you give an example of how you might organize this code in a more
namespace heavy language like C++ or Java? I think this is quite a general
question on how to structure large Julia applications, not really specific
to JuMP.
Best,
Miles
On Wednesday, June 3, 2015 at 9:07:03 PM UTC+2, Francois Gilbert wrote:
Hi all,
Is anyone aware of any effort building large OR applications? I am
currently working at organizing a relatively large code base making use of
the JuMP libraries. The context is unit-commitment and transmission
planning. Flexibility and code re-use is among our priorities.
Here are some of the design choices we made so far:
1) JuMP models are encapsulated in types that hold:
a) references to the variable names, constraints (when multipliers are
needed) and indices. These are otherwise difficult to extract directly
from the JuMP object
b) data structure that are required for model specification (generator
characteristics, transmission lines, etc)
c) exogenous variable values (demand scenarios,
[julia-users] Re: Organizing code around Jump models
Hi Miles,
It would be easier for me to give some excerpts of the Julia sources I am
working with. I think it would suffice to illustrate the mechanic we are
trying to implement.
Say we have two abstract types:
abstract BasicUC
abstract CCModel:BasicUC
These are declared in separate modules. BasicUC stands for basic
unit-commitment and types that derive it have access to functionalities
such as:
function setLineCapa!(m::BasicUC)
JuMP.@addConstraint(m.mip, _contraint1[l = keys(m.net.branch), t =
m.timeIndex],
m.var[:flow][l,t] = m.net.branch[l].Fmax)
...
end
CCModel stands for an extended unit-commitment formulation, where robust
reserve requirement are implemented. Details are irrelevant. A type that
derives CCModel has access to, say:
function setRobustReserve!(m::CCModel)
...
end
Extending the functionality specific to BasicUC - let a type “Model” such
that
type Model{T:BasicUC} : CCModel
uc# Hook to BasicUC
# Proxies to BasicUC:
demand # exogenous stuff
mip # JuMP model
. . .
function Model(net,scenario)
this=new()
this.uc=T(net,scenario)
. . .
# Delegation to BasicUC
this.demand=this.uc.demand
this.mip=this.uc.mip
this.var=this.uc.var
. . .
# Specific to CCModel
this.contingencies=scenario[:contingencies]
JuMP.@defVar(this.mip, _post_gen[keys(this.net.gen),
keys(this.contingencies),
this.timeIndex])
this.var[:post_gen]= _post_gen
. . .
return this
end
end
Now, let a concrete type BasicModel be defined in, say module
BasicUnitCommitment, and such that :
BasicModel:BasicUC
Type Model{T} above can then be instantiated and tailored, with:
m=Model{BasicUnitCommitment.BaseModel}(net,scenario)
# from the base model (BasicUC)
setBalanceEq!(m)
setLineCapa!(m)
setInitRamping!(m)
# these are overloaded by CCModel
setObjective!(m)
setGenCapa!(m)
setRamping!(m)
# these are specific to CCModel
setPostBalanceEq!(m)
setRobustReserve!(m)
. . .
So that's the idea. Perhaps I am overdoing this, but I think having some
kind of encapsulation is essential for developing larger applications.
Emulating inheritance, as is done above, is rather straightforward, but
perhaps a little laborious, and there might be a better way to go about
it.
François
Le mercredi 3 juin 2015 16:12:21 UTC-5, Miles Lubin a écrit :
Hi Francois,
Could you give an example of how you might organize this code in a more
namespace heavy language like C++ or Java? I think this is quite a general
question on how to structure large Julia applications, not really specific
to JuMP.
Best,
Miles
On Wednesday, June 3, 2015 at 9:07:03 PM UTC+2, Francois Gilbert wrote:
Hi all,
Is anyone aware of any effort building large OR applications? I am
currently working at organizing a relatively large code base making use of
the JuMP libraries. The context is unit-commitment and transmission
planning. Flexibility and code re-use is among our priorities.
Here are some of the design choices we made so far:
1) JuMP models are encapsulated in types that hold:
a) references to the variable names, constraints (when multipliers are
needed) and indices. These are otherwise difficult to extract directly
from the JuMP object
b) data structure that are required for model specification (generator
characteristics, transmission lines, etc)
c) exogenous variable values (demand scenarios, initial conditions, etc)
2) Most models we use are derived from a basic unit commitment model. The
derivation is implemented using
composition: the derived type manages a reference to the base type, and
provides a number of proxies to some of the base type data structure.
3) The objective and constraints are specified through function calls,
using polymorphism on the basis of an underlying abstract types hierarchy.
I also toyed for a while with submodules to incorporate functionality
that only make sense within the scope of a given base module. But from the
language point of view, I did not see much benefit in doing so. That is, a
Julia submodule does not see anything of the base module, unless it is
explicitly use/imported, and is thus in the same position as any other
module defined outside the scope of the base module. Are there any Julia
packages making use submodules?
Any suggestions/comments are very welcome,
François
[julia-users] Re: Organizing code around Jump models
Hi Francois, Could you give an example of how you might organize this code in a more namespace heavy language like C++ or Java? I think this is quite a general question on how to structure large Julia applications, not really specific to JuMP. Best, Miles On Wednesday, June 3, 2015 at 9:07:03 PM UTC+2, Francois Gilbert wrote: Hi all, Is anyone aware of any effort building large OR applications? I am currently working at organizing a relatively large code base making use of the JuMP libraries. The context is unit-commitment and transmission planning. Flexibility and code re-use is among our priorities. Here are some of the design choices we made so far: 1) JuMP models are encapsulated in types that hold: a) references to the variable names, constraints (when multipliers are needed) and indices. These are otherwise difficult to extract directly from the JuMP object b) data structure that are required for model specification (generator characteristics, transmission lines, etc) c) exogenous variable values (demand scenarios, initial conditions, etc) 2) Most models we use are derived from a basic unit commitment model. The derivation is implemented using composition: the derived type manages a reference to the base type, and provides a number of proxies to some of the base type data structure. 3) The objective and constraints are specified through function calls, using polymorphism on the basis of an underlying abstract types hierarchy. I also toyed for a while with submodules to incorporate functionality that only make sense within the scope of a given base module. But from the language point of view, I did not see much benefit in doing so. That is, a Julia submodule does not see anything of the base module, unless it is explicitly use/imported, and is thus in the same position as any other module defined outside the scope of the base module. Are there any Julia packages making use submodules? Any suggestions/comments are very welcome, François
