The cause that you speak of is that the IVR is the window into the company from the outside world. Problems elsewhere will always be manifested through the window. In this case, the window is the messenger, a victim of a badly performing system elsewhere.
I don't think there is a cure that will work systematically. You can have fault logs, with published resolutions etc but quite often it is the first impression that counts. People either blame the IVR maliciously because it is a convenient way of diverting attention from what might really be their own problem, or else they just blame it out of ignorance.
The previous suggestions of education, visiblity, updating stakeholders directly are all valid approaches, though sadly time consuming. In the end, I think that this is something that goes with the territory. You cannot be a good doctor if you dislike sick people. This is the same.
But there are ways to make it better. You could publish a quarterly report with a headline pie chart showing the outcome of all faults through root cause analysis. E.g. 20% back end db, 30% LAN letency issues, 12% telephony congestion, whatever. It has to be simple, colourful and punchy. It also has to be regular. People will soon get the idea, especially if it is sent to the correct level of seniority downwards. What if senior people aren't interested. Well, quite often its because they don't know they should be interested. But get it right and things soon change for the better.