The promise of XAI (eXplainable AI)

The current limitations of the technology are so severe that I believe XAI will not be achieved — Will AI systems supervise cloud computing?

Will AI systems oversee large-scale critical infrastructures?

I just wrote this email to a friend who works in the stock market in Wall Street and has helped me a great deal in finding my way in AI standards activities. I am sharing it here now.

It is, as always, just my opinion.

Hope you are doing well. You have been so generous to me in the past that I wanted to share this opinion with you first. Bear in mind it is just an opinion.

I’ve recently interacted with GPT a lot — for a boring science application I had to write. (For details: this was GPT 4o, and I think I pay a subscription of USD 20 per month). I never use the models so I believed they were able to do a lot more than what I actually saw. This is my opinion after this one week of intense interaction:

The current limitations of the technology are so severe that I believe XAI will not be achieved.

At any rate not in any near future. It is my belief that cascade failures (like Crowdstrike or other) will continue to happen, with more severe financial losses, until companies realise that they cannot leave AI to “babysit their infrastructures”.

In short, generative models will not be able to do patch management.

In my opinion it is a matter of time until current investors in AI companies loose interest in the promise of enterprise software applications. Around 14 months ago (maybe less) you wrote that we need to remove human operators from monitoring system infrastructures because the industry was overloaded, and employees were burning out. That did make a lot of sense, and it still does. However I see that the models today are so limited in their capabilities that, in my opinion,

The promise of “AI-based oversight of large critical infrastructures” will, I fear, remain just a promise.

Physical models, which interact in the physical world are, unfortunately, a different story altogether. Because there we just need one government official to believe that the models can reason, and release physical systems onto the public sphere with a `law-enforcement’ goal. The damage there will be physical more than (or as well as) financial.

This text represents my opinion alone. I do not have proof of these opinions.