Hi Everyone, below is my PCE result:
PCE metamodel validation error: 7.5371e-01
PCE metamodel LOO error: 1.9341e-08
For 9 dimensions, 250 samples got by latin hypercube method are split to 100 training samples and 150 validation samples.
LOO error is ok, validation error 7.5371e-01 is bad.
But why?
Dear @AmosJoseph,
Since you have a good LOO and a relatively high validation error, I suspect you are close to overfitting your PCE. What method and truncation scheme did you use to build your PCE? Depending on the size of your basis, the amount of data needed to train your model increases. If only a small budget is available, you should favour sparse solvers.
Additionally, have you tried to switch the size of your training and validation set? Maybe training your model with 150 samples and validating it with the remaining 100 may improve the PCE model.
Best regards,
Anderson
Hi, @ Anderson_Pires, thanks a lot for your help!
I use the same 100 samples for training, and obtain 150 new samples for Validation. And build sparse PCE:
Number of input variables: 9
Maximal degree: 3
q-norm: 1.00
Size of full basis: 220
Size of sparse basis: 65
Full model evaluations: 100
Leave-one-out error: 1.9340998e-08
Validation error: 8.8987533e-08
In this figure, FEM prediction is totally covered by PCE prediction. Is this OK?
As Leave-one-out error is 1.9340998e-08 and Validation error is 8.8987533e-08, there is high precision and small error. But I wonder if the result is correct.
Dear @AmosJoseph ,
I am sorry for the long silence. Concerning your question, these results suggest that the metamodel was successfully trained and can now be used as your surrogate.
Best regards,
Anderson
Many thanks for your help！