I am new to UQ and working on the analysis of a building structural model under seismic loading. There are 15 random variables (material parameters) in the model and the output is the median seismic response of the structure.
I have already got a data set with the size of 1000 (inputs were obtained by Latin Hypercube Sampling). I trained a PCE model with 800 data samples and used the remaining 200 as a validation set. The errors and the a plot of validation is shown below:
PCE metamodel validation error: 1.0273e-01
PCE metamodel LOO error: 7.6733e-02
I have also run the UQLab code with different training sizes up to 1000, a plot of training size v.s. LOO error is shown below:
As recommended by Professor Sudret (Acceptable LOO Error), a LOO error less than 0.01 is sufficient for obtaining accurate moments or sensitivity indices of the output. So based on the second plot, it seems that I need a lot more data to achieve a LOO error of 0.01.
My questions are:
(1) Does the second plot look normal? The LOO error decreases very slowly with the increase of the training size.
(2) If I cannot achieve a low value of the LOO error, is there anyway to prove that the metamodel is reliable?
Thank you very much!