I am using the Sobol sensitivity analysis for my model. I was able to use them well with my previous simplified model and currently with much more complex model and more input uncertainties, I have been getting output of the indices especially the total sobol indices more than 1.2 to 1.9 and the sobol first order are all negative values. I am not sure if I understand the issue. Is it might be that I have to implement the dependent input SA types? If my understanding on the dependent inputs is correct, my input variables that is uncertain is not dependent and hence I am not sure. Any comment on this? If I need to try the dependent input SA, any suggestion to which type?
Perhaps you could describe your problem more? How much more complex does your model actually become? With how many more inputs? What was the result of the simpler model? Did you model your inputs with a dependence structure, say using a Gaussian copula?
And when it comes to computation: How do you compute the Sobol’ indices? Is it via a Monte Carlo simulation? If so, with how many sample points? In some cases, problems (such as negative but very small values for the first-order Sobol’ indices) might happen due to convergence issues.
Moreover, it might be indeed the case that your inputs would be realistically dependent, but if you don’t model any dependence in your inputs (that is, you just assume that they are independent), then there should be no problem in computing the Sobol’ indices (at least numerically). It’s just that the conclusion of the sensitivity analysis might be wrong.
I might be wrong, but I am not sure if your problem here is because you don’t model a dependence structure in your inputs.
I was running simple isothermal column model with only 5 input uncertainties. Now my model is not isothermal and has energy balance and the solution to the model will be by solving the mass and energy balance simultaneously. Currently for this model, I have 8 uncertainty inputs. I did not use any dependence structure like in my previous case.
For the Sobol indices computation, I used Monte Carlo simulation. I usually test run for 100 points to test it out but I understood that I may need to run at more sampling points to get accurate result and it might be the case. I am not sure.
I have assume they are independent to each other. Each input variable uncertainty propagate through submodels that is used to compute the mass and energy balances. So each submodels, could have two input UC playing role for example or even 3. So I dont have the feeling about it whether these interactions also play some role.
I am going to try to run with more sampling points as I did previously with 10000 samples but as the model is complex its going to take time. However I know there is this PCE method to be applied and it will much quicker. I will see to try them then.
I think 100 points are a bit too few to estimate the indices via MC. I would run with much more than that and see if it gives more reasonable results. Maybe also check the convergence of the estimates (whether the estimates are stable with increasing number of sample points).
If computational load is a concern then for sure try using metamodels (say PCE or LRA that can give you Sobol’ indices as by-products).
Hopefully, that’s what (global) sensitivity analysis of a complex model should help you find out . Now, whether the inputs are dependent or independent is one of the first assumption we make before doing a sensitivity analysis; so the conclusion of the analysis would depend on that assumption.
Hi @Vishashalu, were you able to fix your problem? Does a large number of sample still gives strange results? Were you able to use metamodels to compute the Sobol’ indices?
No I was not successful in solving my problem. After running 10000samples, the results for Total sobol had more than value of 1 for some parameters and others in the range of 0.8 to 0.9 and for the first order, I had some negative values.
Also, I was not successful in implementing PCE based method for my column model. There is some error perhaps with the numerical method of ODE’s and array issue which I am not able to solve. The model is quite complex and the best I feel is definitely to implement surrogate model based method but yes I am little stuck.
Would it be possible for you to create a reproducible example here? Say by posting or attaching your model and analysis scripts? Thanks!
Hi @Vishashalu, by the way, have you checked the distribution of the output of interests of this 10000 runs? I think it would be a nice diagnostic step.
I will do that. Just give me some time. Instead of this complex model that I have, I would like to implement in a simple model to see if it works. Otherwise, I might be doing something wrong and I will get back to you on this. Thanks for your help!