How to find proper metamodel method in a surrogate-assisted RBDO example

UQ with UQLab Community Q&A and How To
,
1

Dear UQLab

First of all, thanks for this enjoyable software.
Then, according to the discussion in a precious work done by @moustapha, “Surrogate-assisted reliability-based design optimization: a survey and a unified modular framework”, Its important to calibrate metamodels in the context of surrogate-assisted RBDO.
As discussed in detail in the Acceptable Loo Error, the LOO error is a powerful tool in finding a suitable metamodel.In this regard, I have two questions:

  1. I was wondering how to derive the LOO error in a surrogate assisted RBDO example (I can’t find the acceptable command)?
  2. Is it logical to select a proper metamodel (i.e, PCE, Kriging and etc.) for a particular example based on the LOO error or our only criteria to find a proper metamodel is the cost value (Fstar)?

Finally, I really appreciate any help/guidance in this regard from the community and/or developers.

Best regards
Ali

2

Dear @ali,

Currently the LOO error is not used directly in the surrogate-assisted RBDO framework implemented in UQLab. As Bruno said in the post you have mentioned, the LOO error is used for estimating the global error of the surrogate model, whereas the algorithms implemented in the module seek to find surrogates that are only locally accurate (that is in the close vicinity of the limit-state surface).

Regarding your questions:

  1. You can find the LOO error in the field myMetamodel.Error.LOO. As of now you can’t use that information within the RBDO module but what you can do is build externally a surrogate model with your own custom enrichment scheme until the desired LOO error is reached. Then you can directly pass that model as the limit-state for the RBDO analysis (you shouldn’t specify any metamodel then)

RBDOOpts.LimitState.Model = myMetamodel;

  1. a. I wouldn’t recommend selecting a surrogate based on the LOO error as that is a global error and reaching a good accuracy for RBDO would probably require an extremely large number of model evaluations. You would be better off with adaptively enriching the metamodel.
    b. As of which surrogate is better, I would recommend going for an interpolating one (that is, PCK or Kriging) rather than a regression model (e.g. PCE, SVR) as in the latter the rate of improvement of the local accuracy by adding experimental design points is much slower than in interpolation.

Generally speaking, I think it’s a good idea to add the LOO error as convergence criteria in the enrichment process. As of now, UQLab allows for custom convergence function but unfortunately the generic template function does not take as input the current metamodel so the LOO error can’t be accessed internally (if you really need to do that, I can tell you how to modify the uq_activeMetamodel.m file to achieve this). We will update to allow for this in a later release. Actually, the current enrichment scheme in the RBDO module is quite basic but we plan to introduce more efficient methods… But that’s probably for v1.5

Cheers,
Moustapha

1 Like
3

Dear @moustapha

First of all, I really appreciate your detailed answer.
Second, I guess, I could find the LOO error without constructing the “myMetamodel” which you mentioned in your answer by the below command:

    for i=1:N ;% N=Number of Constraints
      myRBDO_LOO(i)= myRBDO.Internal.Constraints.Model.Error(i).LOO; %LOO Error
    end

I was wondering to know my try to find the LOO error leads to the right answer or not?
Third, It’s worth mentioning that I am a little bit curious to know if there is any other surrogate-assisted example like “Section 6.2: Short column under oblique bending” of your mentioned paper to share with us to learn more in this context?
Last but not least, I am most grateful for any help from developers and other community members.

Best regards
Ali