I suggest to ignore the statement ``… due to its sequential nature, the algorithm cannot be parallelized which makes it comparably slow.‘’ at the end of Section 1.3.4 Affine invariant ensemble algorithm in the Bayesian inference UQ-Lab User manuel. and to try at least to implement a parallel update of the AIES schema.
I think that even if the algorithm is sequential, one could speed it up by performing in each step the work for updating the walkers in parallel, e.g. if one has C walkers and 8 processors one could request that
processor 1 updates walkers 1,…,C/8,
processor 2 updates walkers C/8+1,…,C/4,
…
processor 8 updates walkers 7C/8+1,…,C.
Of course, one could use a more clever way to divide the work between the processors and would have to save the updates values for a walker in a new array but this could be a starting point for an parallel update for the AIES scheme.
Dear Olaf,
thank you as usual for your insight, feedback and feature request.
The parallelization of AIES is not as straightforward as running multiple blocks at the same time, as there are convergence considerations to make, and may/may not be as efficient as running a properly parallelized implementation.
There are multiple recently proposed parallelization schemes in the literature, all of which start by observing that parallelization is not straightforward, and providing different implementation alternatives e.g.:
At the this stage, it may be more convenient to simply submit multiple jobs with different random seeds with the current implementation, rather than using a simple slicing as you suggest.
We have had this feature in mind for quite some time, and it is in our ticketing system in the backlog of features we would like to implement. However, at the moment we don’t have the resources to validate, benchmark, implement and document them in the short term, as priority is given to other upcoming projects.
