While in the past, most economic assessments of countries focused on indicators such as GDP growth, inflation, unemployment, the fiscal deficit, and the current account of the balance of payments, these days they tend to also include assessments of a country’s governance framework and the strength of its institutions. 

The standard macroeconomic indicators have been in common use for many decades and are well understood. On the other hand, the new governance indicators are somewhat more obscure. This has tended to limit the scope of discussion on these indicators to a simple partisan sparring game.

A recent study published by the Central Bank of Malta sheds light on two frequently used governance set of indicators, namely that developed by the World Bank and the one published by the ratings agency, Moody’s.

The World Bank’s indicators

The World Bank’s Worldwide Governance indicators is the more comprehensive of the two, spanning six different elements ranging from political stability to the control of corruption. The data to compile these indicators come from about 33 sources, comprising public sector organisations, NGOs, commercial business information providers and surveys of households and firms.

Herein lies the main defect of governance indicators. While macroeconomic data are compiled according to harmonised statistical methods by national statistical offices that operate within an international statistical professional framework, governance indicators are more of a case of mix and match at present. Not all countries are analysed in the same detail by NGOs and international institutions. As one can suspect, small countries are not devoted that much attention and resources.

Running population surveys is expensive, and for most small countries, instead of doing this, most frequently data compilers just resort to consulting the opinion of “country experts”. Moreover, even when surveys are held, the study conducted by the Central Bank notes that “the reliability and accuracy of the data they provide is largely dependent on the number of respondents, which in some cases is rather low”.

To give a practical example, a major source for Malta is the business executive survey conducted by the World Economic Forum as part of its Global Competitiveness Survey. While there were over 15,000 active companies in Malta in 2019, the results of this survey are based on the responses of a mere 65 firms, with no details given as to their size and effective coverage.    

The World Bank indicators rely greatly on NGO’s views, and “in supplying information about a country’s governance quality, some sources do not even provide any justification for changes in scores between one year and the next”. This is not the world of balance of payments statistics where highly trained statisticians pore over hefty manuals detailing exactly how they can operate. There is no Eurostat zooming in, auditing rigorously nationally compiled data with on-site visits and quality audits.

The analysis conducted in the Central Bank study is very detailed in its assessment of the various sources of data on Malta and its conclusion is that “it appears that for the most part, the decline in many indicators derives from changes in assessments made by a small number of commercial and non-governmental organisations. By contrast the results of more comprehensive surveys carried out among local households do not generally show similar trends.”

The “expert” opinion vs Moody’s credit quality

This is quite an interesting conclusion by the Central Bank. Assessments made by the Economist Intelligence Unit, which is typically done by a single individual, point one way, while broader surveys, such as the Gallup World Poll, carried out in a scientific way among the Maltese population point the exact opposite. At the same time, the research carried out by the Central Bank notes that “expert” opinion can vary greatly.

For instance, while the Economic Intelligence Unit’s assessment of governance is negative – so much so that in 2019 Malta was relegated from being a full democracy to being a flawed democracy, according to Moody’s, Malta’s credit quality is “positively impacted by its strong governance framework, largely due to positive perceptions of Malta’s budget management, transparency and disclosure, and policy credibility and effectiveness.”

This rather confusing picture does not mean that we should ignore governance indicators. However, we need to be cautious when we look at them. Even economic data needs to be interpreted with caution, with the focus being put on trends rather than in single year movements and by looking at various indicators rather than one.

In the case of governance indicators where one is relying mostly on subjective assessments many times made by few individuals, and not always in a transparent way, the need for caution is even more pronounced.