Re: [ot-users] Linear least squares always insert a constant
Hi Sofiane,
Thanks for the trick : I did not know this constructor.
However, my particular need was really to get over the linear model. My
particular applications are the calibration of a model based on the BLUE or
ridge regression. In these cases, we really need to solve the basic linear
least squares problem, without any link to a functional basis.
How would you do that ?
Best regards,
Michaël
Michaël BAUDIN
Ingénieur - Chercheur
EDF – R
Département PRISME
6, quai Watier
78401 CHATOU
michael.bau...@edf.fr<mailto:michael.bau...@edf.fr>
Tél. : 01 30 87 81 82
Fax : 01 30 87 82 13
De : sofiane_had...@yahoo.fr [mailto:sofiane_had...@yahoo.fr]
Envoyé : mardi 2 avril 2019 09:00
À : users@openturns.org; BAUDIN Michael
Objet : Re: [ot-users] Linear least squares always insert a constant
Hi Michael,
The LinearModelAlgorithm indeed relies on a linear basis (if no one is
provided).
However you can define your own basis and provide it to the algorithm class :
"""
import openturns as ot
ot.RandomGenerator.SetSeed(0)
g = ot.SymbolicFunction(['x', 'y'], ['0.5+sin(x)-2*y'])
npoints = 50
x = ot.Uniform(-2,2).getSample(npoints)
x.stack(ot.Normal(0, 3).getSample(npoints))
x.setDescription(["x", "y"])
y = g(x)
# Create basis B = ( [x,y]-->x, [x,y]-->y )
input_description = x.getDescription()
basis = ot.Basis([ot.SymbolicFunction(input_description, [description_i]) for
description_i in input_description])
algo = ot.LinearModelAlgorithm(x, basis, y)
algo.run()
...
"""
The trend coefficients's size is 2 corresponding to the basis size
Sofiane
Le jeudi 28 mars 2019 à 20:09:52 UTC+1, BAUDIN Michael
mailto:michael.bau...@edf.fr>> a écrit :
Dear all !
The LinearLeastSquares class in OT 1.12 always inserts a constant in the model,
be it wanted by the user or not:
https://github.com/openturns/openturns/blob/d0802a1b17b60bd86afa234662a047bc4f04492f/lib/src/Base/MetaModel/LinearLeastSquares.cxx#L105
In the API, this term corresponds to the getConstant() method.
The same is true for LinearModelFactory.
In the demo script in PS, I use linear least squares to approximate the sine
function with the polynomial basis 1, x, x^2, x^3. The linear model involves 4
coefficients. An intercept is always added leading to 5 estimated coefficients,
that I do not want.
g = ot.SymbolicFunction(['x'], ['0.5+sin(x)'])
npoints = 50
x=ot.Uniform(-2,2).getSample(npoints).sort()
y = g(x)
# Create input
basis = ot.SymbolicFunction(['x'], ['1','x','x^2','x^3'])
inputData = basis(x)
With LinearLeastSquares, I get a constant:
myLeastSquares = ot.LinearLeastSquares(inputData, y)
myLeastSquares.run()
beta0 = myLeastSquares.getConstant()[0]
With LinearModelFactory, I get 5 coefficicents instead of 4:
LMF = ot.LinearModelFactory()
linearModel = LMF.build(inputData, y)
beta = linearModel.getRegression()
As far as I can see, the LinearModelAlgorithm in OT 1.13 has the same behaviour:
https://github.com/openturns/openturns/blob/ce1bc890a907faeecde495f5528ed42e401153c7/lib/src/Uncertainty/Algorithm/MetaModel/LinearModel/LinearModelAlgorithm.cxx#L65
I assume that the constant is always there, so that the method prevents from
having a bias in the estimate. But in cases where you want really to perform
linear least squares, then there is an issue.
As far as I can see, the LeastSquaresMethod is the right tool. Unfortunately,
this cannot be used from the Python API.
Am I correct ?
Best regards,
Michaël
PS
import openturns as ot
from openturns.viewer import View
g = ot.SymbolicFunction(['x'], ['0.5+sin(x)'])
npoints = 50
x=ot.Uniform(-2,2).getSample(npoints).sort()
y = g(x)
# Create input
basis = ot.SymbolicFunction(['x'], ['1','x','x^2','x^3'])
inputData = basis(x)
# Solve
myLeastSquares = ot.LinearLeastSquares(inputData, y)
myLeastSquares.run()
beta0 = myLeastSquares.getConstant()[0]
print("beta0=%s" % (beta0))
beta = myLeastSquares.getLinear()
print("beta=%s" % (beta))
# Check
responseSurface = myLeastSquares.getResponseSurface()
ypredicted = responseSurface(inputData)
#
graph = ot.Graph("Linear Model","x","y",True,"topleft")
curve = ot.Curve(x,ypredicted)
curve.setLegend("Linear Model")
graph.add(curve)
cloud = ot.Cloud(x,y)
cloud.setColor("red")
cloud.setLegend("Data")
graph.add(cloud)
View(graph)
#
ot.ResourceMap.SetAsString('R-executable-command','bla\\bla\\R.exe')
LMF = ot.LinearModelFactory()
linearModel = LMF.build(inputData, y)
beta = linearModel.getRegression()
print(beta)
Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à
l'intention exclusive des destinataires et les informations qui y figurent sont
strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination, toute diffusion ou toute publicatio
Re: [ot-users] Linear least squares always insert a constant
Hi,
Sorry I missed the msg
You can: 1 - Wait for the next release ;-) 2 - Implement easily your own class
that solve the linear problem without the ìntercept` 3 - Might rely on python
modules (statsmodels, scikit...)
Regards,Sofiane
Le mardi 2 avril 2019 à 17:03:33 UTC+2, BAUDIN Michael
a écrit :
#yiv3869854319 #yiv3869854319 -- _filtered #yiv3869854319
{font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv3869854319
{panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv3869854319
{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}#yiv3869854319
#yiv3869854319 p.yiv3869854319MsoNormal, #yiv3869854319
li.yiv3869854319MsoNormal, #yiv3869854319 div.yiv3869854319MsoNormal
{margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;font-family:New
serif;}#yiv3869854319 a:link, #yiv3869854319 span.yiv3869854319MsoHyperlink
{color:blue;text-decoration:underline;}#yiv3869854319 a:visited, #yiv3869854319
span.yiv3869854319MsoHyperlinkFollowed
{color:purple;text-decoration:underline;}#yiv3869854319 p
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New
serif;}#yiv3869854319 p.yiv3869854319msonormal, #yiv3869854319
li.yiv3869854319msonormal, #yiv3869854319 div.yiv3869854319msonormal
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New
serif;}#yiv3869854319 p.yiv3869854319msochpdefault, #yiv3869854319
li.yiv3869854319msochpdefault, #yiv3869854319 div.yiv3869854319msochpdefault
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:New
serif;}#yiv3869854319 span.yiv3869854319msohyperlink {}#yiv3869854319
span.yiv3869854319msohyperlinkfollowed {}#yiv3869854319
span.yiv3869854319emailstyle17 {}#yiv3869854319 p.yiv3869854319msonormal1,
#yiv3869854319 li.yiv3869854319msonormal1, #yiv3869854319
div.yiv3869854319msonormal1
{margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;font-family:sans-serif;}#yiv3869854319
span.yiv3869854319msohyperlink1
{color:#0563C1;text-decoration:underline;}#yiv3869854319
span.yiv3869854319msohyperlinkfollowed1
{color:#954F72;text-decoration:underline;}#yiv3869854319
span.yiv3869854319emailstyle171
{font-family:sans-serif;color:windowtext;}#yiv3869854319
p.yiv3869854319msochpdefault1, #yiv3869854319 li.yiv3869854319msochpdefault1,
#yiv3869854319 div.yiv3869854319msochpdefault1
{margin-right:0cm;margin-left:0cm;font-size:12.0pt;font-family:sans-serif;}#yiv3869854319
span.yiv3869854319EmailStyle28
{font-family:sans-serif;color:#1F497D;}#yiv3869854319
.yiv3869854319MsoChpDefault {font-size:10.0pt;} _filtered #yiv3869854319
{margin:70.85pt 70.85pt 70.85pt 70.85pt;}#yiv3869854319
div.yiv3869854319WordSection1 {}#yiv3869854319
Hi Sofiane,
Thanks for the trick : I did not know this constructor.
However, my particular need was really to get over the linear model. My
particular applications are the calibration of a model based on the BLUE or
ridge regression. In these cases, we really need to solve the basic linear
least squares problem, without any link to a functional basis.
How would you do that ?
Best regards,
Michaël
Michaël BAUDIN
Ingénieur - Chercheur
EDF – R
Département PRISME
6, quai Watier
78401 CHATOU
michael.bau...@edf.fr
Tél. : 01 30 87 81 82
Fax : 01 30 87 82 13
De : sofiane_had...@yahoo.fr [mailto:sofiane_had...@yahoo.fr]
Envoyé : mardi 2 avril 2019 09:00
À : users@openturns.org; BAUDIN Michael
Objet : Re: [ot-users] Linear least squares always insert a constant
Hi Michael,
The LinearModelAlgorithm indeed relies on a linear basis (if no one is
provided).
However you can define your own basis and provide it to the algorithm class :
"""
import openturns as ot
ot.RandomGenerator.SetSeed(0)
g = ot.SymbolicFunction(['x', 'y'], ['0.5+sin(x)-2*y'])
npoints = 50
x = ot.Uniform(-2,2).getSample(npoints)
x.stack(ot.Normal(0, 3).getSample(npoints))
x.setDescription(["x", "y"])
y = g(x)
# Create basis B = ( [x,y]-->x, [x,y]-->y )
input_description = x.getDescription()
basis = ot.Basis([ot.SymbolicFunction(input_description, [description_i]) for
description_i in input_description])
algo = ot.LinearModelAlgorithm(x, basis, y)
algo.run()
...
"""
The trend coefficients's size is 2 corresponding to the basis size
Sofiane
Le jeudi 28 mars 2019 à 20:09:52 UTC+1, BAUDIN Michael
a écrit :
Dear all !
The LinearLeastSquares class in OT 1.12 always inserts a constant in the model,
be it wanted by the user or not:
https://github.com/openturns/openturns/blob/d0802a1b17b60bd86afa234662a047bc4f04492f/lib/src/Base/MetaModel/LinearLeastSquares.cxx#L105
In the API, this term corresponds to the getConstant() method.
The same is true for LinearModelFactory.
In the demo script in PS, I use linear least squares to approximate the sine
function with the polynomial basis 1
Re: [ot-users] Linear least squares always insert a constant
Hi Michael,
The LinearModelAlgorithm indeed relies on a linear basis (if no one is
provided).
However you can define your own basis and provide it to the algorithm class :
"""import openturns as ot
ot.RandomGenerator.SetSeed(0)g = ot.SymbolicFunction(['x', 'y'],
['0.5+sin(x)-2*y'])
npoints = 50
x = ot.Uniform(-2,2).getSample(npoints)x.stack(ot.Normal(0,
3).getSample(npoints))x.setDescription(["x", "y"])y = g(x)
# Create basis B = ( [x,y]-->x, [x,y]-->y )input_description =
x.getDescription()basis = ot.Basis([ot.SymbolicFunction(input_description,
[description_i]) for description_i in input_description])
algo = ot.LinearModelAlgorithm(x, basis, y)algo.run()..."""
The trend coefficients's size is 2 corresponding to the basis size Sofiane
Le jeudi 28 mars 2019 à 20:09:52 UTC+1, BAUDIN Michael
a écrit :
Dear all !
The LinearLeastSquares class in OT 1.12 always inserts a constant in the model,
be it wanted by the user or not:
https://github.com/openturns/openturns/blob/d0802a1b17b60bd86afa234662a047bc4f04492f/lib/src/Base/MetaModel/LinearLeastSquares.cxx#L105
In the API, this term corresponds to the getConstant() method.
The same is true for LinearModelFactory.
In the demo script in PS, I use linear least squares to approximate the sine
function with the polynomial basis 1, x, x^2, x^3. The linear model involves 4
coefficients. An intercept is always added leading to 5 estimated coefficients,
that I do not want.
g = ot.SymbolicFunction(['x'], ['0.5+sin(x)'])
npoints = 50
x=ot.Uniform(-2,2).getSample(npoints).sort()
y = g(x)
# Create input
basis = ot.SymbolicFunction(['x'], ['1','x','x^2','x^3'])
inputData = basis(x)
With LinearLeastSquares, I get a constant:
myLeastSquares = ot.LinearLeastSquares(inputData, y)
myLeastSquares.run()
beta0 = myLeastSquares.getConstant()[0]
With LinearModelFactory, I get 5 coefficicents instead of 4:
LMF = ot.LinearModelFactory()
linearModel = LMF.build(inputData, y)
beta = linearModel.getRegression()
As far as I can see, the LinearModelAlgorithm in OT 1.13 has the same behaviour:
https://github.com/openturns/openturns/blob/ce1bc890a907faeecde495f5528ed42e401153c7/lib/src/Uncertainty/Algorithm/MetaModel/LinearModel/LinearModelAlgorithm.cxx#L65
I assume that the constant is always there, so that the method prevents from
having a bias in the estimate. But in cases where you want really to perform
linear least squares, then there is an issue.
As far as I can see, the LeastSquaresMethod is the right tool. Unfortunately,
this cannot be used from the Python API.
Am I correct ?
Best regards,
Michaël
PS
import openturns as ot
from openturns.viewer import View
g = ot.SymbolicFunction(['x'], ['0.5+sin(x)'])
npoints = 50
x=ot.Uniform(-2,2).getSample(npoints).sort()
y = g(x)
# Create input
basis = ot.SymbolicFunction(['x'], ['1','x','x^2','x^3'])
inputData = basis(x)
# Solve
myLeastSquares = ot.LinearLeastSquares(inputData, y)
myLeastSquares.run()
beta0 = myLeastSquares.getConstant()[0]
print("beta0=%s" % (beta0))
beta = myLeastSquares.getLinear()
print("beta=%s" % (beta))
# Check
responseSurface = myLeastSquares.getResponseSurface()
ypredicted = responseSurface(inputData)
#
graph = ot.Graph("Linear Model","x","y",True,"topleft")
curve = ot.Curve(x,ypredicted)
curve.setLegend("Linear Model")
graph.add(curve)
cloud = ot.Cloud(x,y)
cloud.setColor("red")
cloud.setLegend("Data")
graph.add(cloud)
View(graph)
#
ot.ResourceMap.SetAsString('R-executable-command','bla\\bla\\R.exe')
LMF = ot.LinearModelFactory()
linearModel = LMF.build(inputData, y)
beta = linearModel.getRegression()
print(beta)
Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à
l'intention exclusive des destinataires et les informations qui y figurent sont
strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination, toute diffusion ou toute publication totale ou partielle, est
interdite sauf autorisation expresse.
Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le
copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si
vous avez reçu ce Message par erreur, merci de le supprimer de votre système,
ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support
que ce soit. Nous vous remercions également d'en avertir immédiatement
l'expéditeur par retour du message.
Il est impossible de garantir que les communications par messagerie
électronique arrivent en temps utile, sont sécurisées ou dénuées de toute
erreur ou virus.
This message and any attachments (the 'Message') are intended solely for the
addressees. The information contained in this Message is confidential.
