Large discrepancy between the leave-one-out error and validation error creating sparse PCEs

Dear UQLab community,

First of all I would like to say thanks for the freely available UQLab-Framework! I have been trying for a while to create sparse PCEs for different output variables from FE calculations using random fields and different material models. It works very well for some material models, but not so well for other material models so far.

However, for all the calculations, I have noticed a large discrepancy between the leave-one-out error and the validation error. In most cases a LOO error was reported which was one or more powers of 10 smaller. I observed this using linear-elastic calculations as well as non-linear calculations.

Based on the equations for both error measures in the user manual, I can’t explain the differences since both equations do almost the same normalization and I always chose my number of validation samples to be at least 200-500.

Since the adaptive choice of p and q are based on the LOO error, I have often manually experimented to find out with which p and q I get the lowest validation error for this reason. This gives me different “optimal” values for p and q than if I let p and q be automatically selected adaptively.
Is this difference between the error measures normal, or am I doing something fundamentally wrong?

I would be very happy about your comments. Thank you in advance!