I wanted to share with you a new web application that I’ve developed during my PhD at the University of Warwick in computational modelling. This app aims to simplify uncertainty quantification (UQ) and sensitivity analysis for a wide range of models, and I’m excited to share it with anyone who may find it useful.
Input Your Own Models: The app allows you to input your own mathematical models directly into an integrated code editor. There’s no need to write additional code for UQ and sensitivity analyses.
Comprehensive Analyses: Once your model is entered, the app performs a suite of UQ and sensitivity analyses, including Monte Carlo simulations, Sobol indices, Morris method, Taylor expansion, and more.
AI-Driven Interpretation: A standout feature is the AI-driven interpretation of results. The app leverages advanced language models (defaulting to Google’s Gemma 2) to provide insightful explanations and recommendations based on your analysis results. This integration aims to bridge the gap between the produced data and actionable insights.
These examples should be familiar to many of you who have used UQLab or participated in UQWorld’s benchmark analyses.
Why Did I Create This App?
During my PhD research, I frequently needed to perform UQ on various models without the overhead of constantly creating new code and re-interpreting the results for my sponsor company. I developed this app to streamline that process, making it easier for anyone to run UQ and sensitivity analyses on their models quickly and efficiently.
Key Features
(Maybe?) User-Friendly Interface: Paste or write your model code directly into the app’s editor or modify one of the existing examples. The app handles the rest, running analyses and generating results without additional coding. Please check the readme on how to define custom models but hopefully it is somewhat intuitive.
AI Integration for Insights: The app doesn’t just crunch numbers—it interprets them. By integrating AI for results analysis and interpretation, it provides insights and recommendations (of course, LLMs can lie etc etc), improving your understanding of the model’s behavior under uncertainty.
Built with Known UQ Libraries: The backend utilizes robust Python libraries like OpenTURNS and SALib, ensuring reliable and accurate computations.
Free and Accessible: The app is completely free to use. I’m hosting (on streamlit) and maintaining it, and I’m considering making it open source if there’s enough interest from the community.
The idea here is to make UQ more accessible and insightful for wide audiences as it is not easy to start with UQ (my opinion). Whether you’re a researcher, practitioner, or just curious, I hope this tool proves useful in your work.
Feel free to reach out with any questions, suggestions, or if you’re interested in contributing.
congratulations on your work/app! This is really impressive. I was wondering how good the AI-generated messages are in general, based on your experience. Especially when it comes to use advanced techniques such as Shapley effects for instance.
Thanks
Bruno
Thank you so much for your kind words! It means a lot coming from someone whose work I’ve followed and greatly appreciate in the field of uncertainty quantification.
Regarding your question about the AI-generated messages, I’ve found that LLMs can be quite helpful when it comes to interpreting results from traditional UQ methods like the Sobol indices. The key is that the AI isn’t performing the uncertainty quantification itself—it’s interpreting the results that are computed using established tools like OpenTURNS or UQLab.
In my experience, LLMs are generally good at acting as data analysts, providing explanations and insights based on the data you feed them. However, as you rightly pointed out, they can sometimes struggle with more advanced techniques like Shapley effects or grasping the nuances between first-order and total-order Sobol indices, especially when using smaller models like my case. Also, smaller models are much faster that the large ones (that maybe just ok out of the box), which I think is very important for user experience/adoption.
To address this, I’ve fine-tuned the models integrated into the app using data and reports from my work with my sponsor company. With proper prompting and adjustments, they can provide interpretations that are sensible and helpful. Of course, there are still gaps, and if you use the app a bit more, you’ll probably notice areas where it could improve.
Overall, I’m optimistic about the potential of integrating LLMs with UQ tools. When I first introduced these UQ techniques to my sponsor company—a large steel manufacturer—they were really impressed. It made me realize that many people and companies might not be aware of how powerful uncertainty quantification and sensitivity analysis can be. By combining these methods with AI-driven interpretations, the hope is to make them more accessible, especially to those who might not have a dedicated UQ practitioner on their team.
I’d love to connect with you and learn more about the work you’re doing. Maybe there’s an opportunity to exchange ideas or even collaborate down the line.
Thanks for sharing your work; it’s very interesting!
I was wondering, in what form do you feed the UQ/Sensitivity results to the LLM? Do you provide the numerical data, images, or both? And do I understand correctly that the LLM infers the case study (e.g., a cantilever beam) solely from the source code (e.g., function_of_interest)? If it relies only on the source, would the results improve if more information, such as comments, were provided in the code? Did you explore how the quality of the results changes depending on the amount and type of information provided? That would be especially interesting for non-standard problems. I assume that for most examples you provide in the app, there is abundant information available on the web, which the LLM has likely encountered before.
Thank you for your thoughtful questions and interest in the project.
Here is the repository for the code—I’d really welcome your contributions if you have the time: UncertaintyCat Repository.
To answer your question about how I feed UQ/Sensitivity results to the LLM:
For the API calls, I only supply numerical data; no images are included in the API call.
For example, when performing Sobol Sensitivity Analysis, the app generates a prompt based on the analysis results and sends it to the Groq API to receive an interpretation from a fine-tuned gemma-2 model (the other models in the drop-down menu are not fine-tuned so the responses may not be as “scientific”). Here’s a simplified example of how the API call works:
from modules.api_utils import call_groq_api
def sobol_sensitivity_analysis(...):
# Perform Sobol analysis
# Prepare the prompt with analysis results
prompt = f"""
Please interpret the following Sobol indices:
First-order indices:
{first_order_md_table}
Total-order indices:
{total_order_md_table}
Second-order indices:
{second_order_md_table}
Provide insights based on these results.
"""
# Call the Groq API to get AI-driven interpretation
interpretation = call_groq_api(prompt, model_name='gemma2-9b-it')
# Display the interpretation in the app
st.markdown(interpretation)
In this snippet:
call_groq_api: This utility function sends the prepared prompt to the Groq API and retrieves the AI-generated markdown interpretation.
model_name: This parameter specifies the language model to use (in this case, Google’s Gemma 2).
interpretation: The returned interpretation is displayed in the Streamlit app using st.markdown().
Regarding your question about adding comments to the model definition:
Yes, comments do help because the model definition (including comments) is passed in full to the LLM for interpretation. You can test this by inputting a complex or convoluted model into the code editor—once without comments and once with comments. From my experience, the LLM’s interpretation improves significantly when comments are included.
On how the quality of results changes with the information provided:
The LLM’s interpretation tends to improve for more well-known models that it has likely encountered before. However, most of the analysis in this app is driven by numerical results that are contextualized within the model. For non-standard problems, the quality of interpretation remains strong as long as the problem is well-defined and includes comments explaining the context.
That said, in the very first part of the analysis (Model Interpretation), where no data is supplied apart from the Python code describing the model, the interpretation is likely to be better for “well-known” UQ problems. I believe this limitation can be mitigated by providing extra comments in the code.
