Meaning of total and second order sobol indices being equal

Hello all,

I have a question about sensitivity analysis using Sobol indices.

I performed sensitivity analysis of a 12 input parameter blackbox function, using two methods:

  • (OLS) PCE coefficient-based calculation of the total and first order Sobol indices
  • Kriging-based MC type calculation, with 100000 Kriging model evaluations

(both metamodels were built using 120 blackbox function evaluations)

The results are shown below:

In the PCE coefficient-based case, the total and first order indices for each parameter are equal (I checked the exact values and they are equal to the last available digit in Matlab). In the Kriging case, the total and first order indices are very similar but not equal.

Note: I also quickly tried an OLS PCE non-coefficient-based sensitivity analysis (by setting ‘SobolOpts.Sobol.PCEBased = 0’) and this method yields results similar to the Kriging-based calculation (total and first order indices for each parameter are close but not equal).

My questions are:

  • Is it normal to have total and first order indices which are equal? From my understanding, this means that higher order interactions are equal to zero, which in turn means that varying two or more input parameters simultaneously has no additional effect compared to varying each one independently. But, is this possible, and if it is, what does this result say about my function from a physical point of view?

  • Is there some kind of uqlab sensitivity analysis setup mistake/problem which could induce such behavior in the PCE coefficient-based method only?

Thank you in advance.

Hi @achille,

Thanks for the post. For the two questions, I would like to share some ideas as follows:

  1. If the first-order and total Sobol’ indices are equal, interactive effects among your input variables are 0. Moreover, it suggests that the model is additive. In other words, your surrogate model (of M input variables) can be expressed as f(X_1,\ldots,X_M) = \sum_{i=1}^{M}f_i(X_i). In general, according to the sparsity-of-effects principle, higher-order effects are usually small. As a result, I would not consider the results abnormal. Nevertheless, since you estimate the Sobol’ indices based on surrogate models, I would suggest first verifying the accuracy of the surrogate models by looking at the leave-one-out error. If it is low (say <1%), the sensitivity analysis suggests that your computational model is close to additive.
  2. Because of the orthonormality of the basis functions, PCE allows for calculating the Sobol’ indices analytically by postprocessing the coefficients (see this paper for more details). When using Monte Carlo simulations, the results (sample statistics) are estimators, which are uncertain (though with increasing sample size, the estimators should converge to the true values). This explains the slight discrepancy you observed between the coefficient-based method and Monte Carlo simulation.

I hope this helps.

Best regards,
Xujia

Hi @xujia, thank you for your detailed response! I think I have a better understanding of PCE-based Sobol indices now.

For reference, the leave-one-out error is 0.4% for the PCE model, and 1.6% for the Kriging model, here.