Expected Imrpovement in Kriging Based Optimization

UQ with UQLab
1

Dear All,
Does anyone have any experience in applying the Expected Improvement (EI) as a method of Infill Criteria in Kriging Optimization ?

2

Dear Shayan

Welcome to UQWorld!

EI is a standard infill criterion when building Kriging models adaptively for optimization. You can easily define it in UQLab, as the criterion depends on the current mean predictor and variance, which are two quantities available as functions as an outcome of uq_createModel(myKriging). Then it is a matter of picking the best point to maximize EI out of a random population, and looping on the procedure.

Best regards
Bruno

3

Dear Bruno;
Thank you so much for your Information.
I have tried a code which was written by myself in my UQLAB kriging code using the Mean and Variance which where the outcomes of my `uq_createModel(myKriging). I used the BFGS as the optimization method , and Cross Validation (CV) as my Estimation method.

The code I have used is in the following:

%%
%function Expected Improvement

y_min= min(Yval);

gamma= (((y_min -Ymu_CV_BFGS))/sqrt(abs(Yvar_CV_BFGS)));

Phi= cdf(‘Normal’, gamma, gamma, 1);
Lambda= pdf(‘Normal’, gamma, gamma, 1);

% Expected improvement

if Yvar_CV_BFGS==0
ExpImp=0;

else
ExpImp= ((y_min -Ymu_CV_BFGS)).*Phi + sqrt(abs(Yvar_CV_BFGS)).*Lambda;
end

figure;

uq_plot(Xval, Yval, ‘k’,…
Xval, Ymu_CV_BFGS, ‘b’,…
X, Y, ‘ko’,…
Xval,ExpImp,’-.r’,…
Xval,ExpImp2,’-.c’);

uq_legend({'True model', 'Kriging, optim. method: BFGS',...
   'Observations','EI-1','EI-2'},...
'Location','southwest')

% Set axes limits
xlim([0 10])
ylim([-2 2])

What I understand from the EI, it should make the kriging prediction more accurate and closer to the real function value. That is why we use the Y-min of the main function and Y-mean of the Kriging predicted. When I draw the new ExpImp function on a graph in MATLAB, the curve is produced outside the Confidence Interval , and it does not seem to be something between the main function and the estimated kriging.

I also saw that a sensitivity analysis is used in the Kriging Prediction which is called SOBOL. By increasing the number of Observation Points we have more accurate estimated kriging function while we are adopting Sobol.

What I feel is that SOBOL should make the same effect as EI (am I right?) .

Would you please help me in finding the best solution for my EI code in my Kriging Prediction? I am using it for many functions , and after that I will share the result here with you.

Looking forward to hearing from you.
Best Regards.
Keep Healthy.
Shayan