A large Leave-One-Out error

UQ with UQLab Community Q&A and How To
1

Dear colleagues,
I Applied a series of simultations by using 42 collocation points where for each collocation point, 5 random variables and 2 outputs were defined.
I generated the PCE from this set of data but the LOO error is very large and I do not understand why. I would like to know if the written code is right:

clearvars
rng(100,‘twister’)
uqlab

IOpts.Marginals(1).Type = ‘uniform’;
IOpts.Marginals(1).Parameters = [0.0634 , 0.146];
IOpts.Marginals(2).Type = ‘uniform’;
IOpts.Marginals(2).Parameters = [1.064 , 1.464];
IOpts.Marginals(3).Type = ‘uniform’;
IOpts.Marginals(3).Parameters = [1.561 , 3.000];
IOpts.Marginals(4).Type = ‘uniform’;
IOpts.Marginals(4).Parameters = [1.29999 , 1.30001];
IOpts.Marginals(5).Type = ‘uniform’;
IOpts.Marginals(5).Parameters = [0.840 , 1.365];

myInput = uq_createInput(IOpts);

Input = nan(42,5);

PCE_IIord = importdata(‘SPW_PCE_IIord.txt’);

Input(:,1) = PCE_IIord(:,1);
Input(:,2) = PCE_IIord(:,2);
Input(:,3) = PCE_IIord(:,3);
Input(:,4) = PCE_IIord(:,4);
Input(:,5) = PCE_IIord(:,5);

Output1 = PCE_IIord(:,6);
Output2 = PCE_IIord(:,7);

MetaOpts1.Type = ‘Metamodel’;
MetaOpts1.MetaType = ‘PCE’;
MetaOpts1.ExpDesign.X = Input;
MetaOpts1.ExpDesign.Y = Output1;
MetaOpts1.Degree = 3:20;
myPCE1 = uq_createModel(MetaOpts1);

uq_print(myPCE1)

If necessary I can give the set of data.

2

There are several factors why that could happen. From the top of my head, it may be that 42 training data points are simply not enough to train a sufficiently accurate PCE. In that case, you should try increasing the size of the training data set and monitor the LOO error. Ideally, it will decrease for increasing training data points. However, it might also be that your input-output map is not sufficiently smooth for a polynomial approximation, in which case the PCE is simply a poor model choice.

3

Dear @SimoneRomani12

There can be indeed various explanations. As a first check, you could use a scatterplot (Matlab plotmatrix command with a matrix containing your 42 lines, and one each line 5 inputs and the two outputs) to make sure the points cover well the uniform ranges you use in IOpts.

With such size of experimental design the degrees should be varied from 1 to 5-6 at most. If nothing looks suspicious, you should try PC-Kriging which works well when there is little data.

Good luck!
Best regards