Arbitrary PCE with dependence structure

Dear all,

I have read the paper ’ Data-driven polynomial chaos expansion for machine learning regression’. And it seems that when the input data has dependence structures, it is better to ignore the dependence and simply to bulid arbitrary PCE w.r.t. each variable.

This phenomenon is also validated in my own dataset. I am wondering if this is always the case that ignoring dependence can produce good approxiamtions. If we just neglect the dependence, why can we still obtain good approximations? And if so, why should many papers today focus on how to take the dependence into consideration when building PCE?

Any help will be appreciated. Thanks in advance.

Hi @YuanXi_Wu,

Very interesting questions! In PCE, we usually want to use polynomials that are orthonormal with respect to the joint distribution of the input variables. Then we get convergence results (Ernst et al 2012) and, if we do regression-based PCE, the regression matrix has the favorable property of having in expectation orthogonal columns. This helps in computing the coefficients (see e.g. Alemazkoor and Meidani (2018) for a nice overview of relevant literature). Also, mean and variance of the output can be computed from the PCE coefficients using the well-known formulas.

If we just neglect the dependence, why can we still obtain good approximations?

We can also view PCE as “just” performing polynomial regression, where we usually use some fancy polynomial families. In principle, it would even work if we simply used monomials (1, x, x^2, …), just that then the regression matrix can become very ill-conditioned and we might run into numerical issues when trying to compute the coefficients.

Heuristic explanation why neglecting the dependence might work well: The idea with “ignoring the dependence and building arbitrary PCE w.r.t. each variable” is that this specific independent distribution always contains the support of the dependent distribution and, if the dependence is not too strong, might actually be quite close to the dependent distribution. In that sense, we don’t lose too much orthogonality by using the “wrong” polynomials. This approach is called aPCEonX by Torre et al. (2018) and dominating by Jakeman et al. (2019).

Heuristic explanation of why considering the dependence might not work that well: On the other hand, when the basis is constructed to be orthonormal w.r.t. the dependent distribution, some difficulties might arise. There could be numerical issues e.g. from Gram-Schmidt orthogonalization. Furthermore, the orthogonalized polynomials don’t have tensor-product structure anymore, but they mix different input variables, which can be unphysical and can lead to less sparse solutions, which then require more data points to be computed accurately.

Whether the dominating approach works well or not also depends on the case study. P. Schnabel recently wrote a report summarizing and comparing several methods for building dependent PCE (available here), it could be of interest to you.

And if so, why should many papers today focus on how to take the dependence into consideration when building PCE?

I would say, because it’s an interesting open problem, and it has the potential of improving the resulting PCE (if the numerical issues during construction of the basis do not overshadow the improvements due to orthogonality). But for sure, more comparisons should be performed to find out whether (and in which cases) there is a benefit over the dominating approach.

If you know of other papers about PCE for dependent inputs than the ones mentioned in P. Schnabel’s report, let us know!

1 Like

Dear @YuanXi_Wu

Thanks for the question. Nora already gave all the answers! I would just comment on your question:

And if so, why should many papers today focus on how to take the dependence into consideration when building PCE?

In my opinion, there is enough empirical evidence in the above-mentioned publications that disregarding the dependence in the construction of orthogonal polynomials is actually the best option. It may look like a boring conclusion, but that’s how it is!
People still try to develop orthogonal bases w.r.t. to dependent inputs for the sake of solving this mathematical question, not for the sake of getting accurate surrogates!

By the way, if you use these aPCE for uncertainty propagation, don’t forget that you cannot anymore compute the mean and variance of the model output from the PCE coefficients: the polynomial expansion should be used as a mere response surface. Monte Carlo simulation using a sample distributed according to the dependent input distribution should be used to compute output moments.

Best regards
Bruno

1 Like

Thank you so much @nluethen. Your answer and the report really helps me a lot and makes me understand better about the PCE.

Dear @bsudret

Thank you so much for the explanation and for your research in this domain.

So, in engineering applications, we can state that it is insignificant to try to build orthonormal polynomials w.r.t. the dependent probability measures since we can just ignore the dependence, right? I’ve seen many research papers in electrical engineering UQ which focused on how to take dependence into consideration. Their numerical results always show superiority to the method ignoring dependence. But in my case, my method shows only marginal advantages over the method ignoring dependence, and that makes my proposed method less appealing and less meaningful. Now from explanations of yours and @nluethen, I think the phenomenon in my case is quite normal and should be anticipated.

Thanks again.

Best regards,
Yuanxi Wu

Dear @YuanXi_Wu

Thanks for your message.

To be clear: I didn’t say that “dependence between inputs should be ignored in general in uncertainty quantification studies”. This is the opposite!

I only meant that, when you try to build a polynomial chaos surrogate model, it is usually more accurate to disregard the dependence for the construction of the surrogate. But as I said in my previous message, if you want to compute afterwards, e.g., mean and variance of the output, you should use the obtained PCE with samples from the input with dependence.

I clarify this so that people who would read this series of posts later would not misinterpret our discussion.

Good luck with PCE and UQ!
Best regards
Bruno

1 Like

Dear @bsudret

Thanks so much for your detailed explanation.