por The problem: no true transparency without the right tools
When we discuss Explainable AI (XAI) and Trustworthy Algorithms, our attention almost always gravitates toward designing the underlying mathematical model. However, there is a structural vulnerability in software engineering that we frequently overlook: the development and debugging environment.
An algorithm cannot be genuinely «trustworthy» or transparent if the tools engineers use to program, test, and debug it operate as black boxes. For an intelligent system to be fully auditable, we require strict algorithmic observability. A researcher must be able to halt execution, inspect internal variable states, visualize decision trees in real time, and ensure that another independent scientist can accurately reproduce that exact computational scenario. Without this infrastructure, explainability remains a purely theoretical promise.
What do we propose in this work?
To bridge this critical infrastructure gap, we published research introducing JGraphs, a software toolset designed specifically for working with Monte-Carlo Tree Search (MCTS) algorithms.
In short, the paper demonstrates how JGraphs builds the necessary foundation for algorithmic auditing through three core capabilities:
- Absolute Traceability: The framework implements a dedicated traceability subsystem that allows for step-by-step execution of tree search simulations. A human operator can pause the algorithm at any given node to understand exactly why the system is evaluating a specific path or decision.
- Visual Interpretation: Complex mathematical trees are fundamentally unreadable in plain text. We integrated automated visualization capabilities (leveraging Graphviz) so that the algorithm’s internal reasoning can be extracted and visually audited.
- Scientific Reproducibility: Through a robust serialization and persistence subsystem (exporting runtime states to JSON), JGraphs allows developers to save the exact footprint of a computational run. This ensures that any algorithmic anomaly or specific edge case can be independently verified—the core foundation of the scientific method.
The connection to trustworthy algorithms
This work addresses a core, pragmatic principle of trustworthy computing: the necessity of opening the black box directly at the infrastructure level. Trust in software is not established by writing terms of service or abstract declarations of intent; it is engineered through the debugging, logging, and monitoring tools used to build the code. JGraphs demonstrates that to move toward software that respects user autonomy, we must first equip engineers with the tools necessary to make algorithmic reasoning completely visible and evaluable.
Read the full research
If you want to dive deeper into the software architecture of the toolset, how we designed the statistical monitoring subsystems, or how to integrate it into your own MCTS-driven developments, the full article has been published in the International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems.
