Forgot your password?
Document Actions
How are we doing in Afghanistan, part I: the dangers of statistics...
There are widely varying degrees of positive and negative assessment regarding the progress of Afghanistan. Many of the conclusions of progress against the Taliban are based on assessment of the numbers of attacks taking place across the country. At best, stats can serve as very crude and loose indicators. At worst they can be misleading, distorting and something to get distracted by. Beware of premature declarations of victory...
Everywhere you go in Afghan analytical and media reporting circles it seems there are widely varying degrees of positive and negative assessment regarding the progress of Afghanistan. In the same week in a British newspaper, the Daily Telegraph, you can read how Taliban militants have begun infiltrating districts around Kabul, adopting Iraq-style insurgency tactics on the one hand, and how, according to the commander of the British-led Helmand Task Force, the insurgents are on the brink of defeat.
Many of the conclusions of progress against the Taliban are based on assessment of the numbers of attacks taking place across the country. There is an interesting debate going on at the ICGA blogspot at the moment regarding whether the claims of Regional Command (East) to be defeating the insurgency in their part of the country are accurate. Much of this argument is based on data showing the number of Taliban attacks increasing against the same period last year.
It may be to state the obvious, but it is worth injecting a note of caution against reading victory or defeat of the Taliban into sets of statistics. At best, stats can serve as very crude and loose indicators. At worst they can be misleading, distorting and something to get distracted by – all too easy to get hung up on them if they are proving the thing you want to believe in. Definitions of what an incident is, who was involved and who initiated the incident are all part of the statistics minefield, if you will pardon the poor choice of expression, and have been interpreted and reinterpreted over the years. The data referred to on the ICGA blog shows incidents by “Taliban and Anti-Government Elements”, (my italics) so I guess we may be looking at figures that include local warlords who have fired a rocket because they are fed up with counter-narcotics activities in their neighbourhood.
In addition, when an incident occurs it is often more revealing to examine how the nature of the attacks are evolving, particularly the effectiveness of the attacks. Are the attacks being conducted at long range by haphazard firing of inaccurate long-range rockets, or are they increasingly becoming “complex” ambushes with the insurgents developing the confidence to get up close for longer and press home the attack?
It is also about how the information is presented, or “spun” that is crucial - and where the most damage, certainly politically, can be done. If, by the end of the summer, attack levels (however defined), are declared to have been less than last year, we will immediately hear the claims from NATO thus: “that's it, we've won, the Taliban are now beaten”. If attack levels (however defined) stay about the same, the message will be the same “we have now got the Taliban contained and on the run”. If the attack levels (however defined) go up by about 10-20% on last year (which seems to be the “standard” increase rate for insurgent activity) the message will be: “everything is fine, we have them contained”.
I just have the nagging feeling that I have heard declarations of victory against the Taliban before...
lies, damn lies...
Seems like we are still, seven years on, sticking to the "glass half full, glass half empty" school of assessment...
Have you checked out the Taliban statements and "stats" at the ref in my other posting, by the way? Would welcome your thoughts...
Can conflict ever be an indicator of success?
Looked at from this angle, might there even be a proportion of "incidents" that could be viewed as a sign of success? The media forever bemoan the confinement of Karzai's government to Kabul, leaving me to wonder whether clashes outside the city aren't evidence to the contrary. I'd like to hope that after seven years the Afghan Army has a greater presence in the south, that those involved in the drug trade are more concerned about the viability of their business ventures, and that local leaders opposed to rule from central government are either feeling the heat or have left power vacuums behind. All these factors and more should be considered in any explanation of incident trends.
The motivation behind this simplistic analysis of statistics seems to be a desire to reduce a complex situation into a single question: "are the Taliban winning or losing?" or perhaps, to put it another way "are we winning or losing". But this frame of reference misses the scale of what is being attempted in Afghanistan; the challenge has never been to "defeat" the Taliban. Acheiving long term stability implies a complete reordering of Afghan's social and political structure. Leaving aside ideological opposition to the government/western forces, it is difficult to imagine how upheavel on this scale could not also bring a degree of conflict. If creating a country with stable institutions and political structure takes decades, looking at incident trends on a year by year basis seems rather like tracking share prices by the day.
That said, a man who's invested a good portion of his life savings in the stock market cannot help but be fixated as the indices rise and fall; we are bound to look for signs of success or failure. But I confess, when reading reports about progress or the lack of it in Afghanistan, I remain mystified. Are things getting worse? I agree that the nature of incidents reveals more about opposition capability than statistics do - so what is your view, are the Taliban more effective than they were before? What does polling data reveal? Is there a greater risk of a significant swing of support towards the Taliban? Does the central government have more influence over decisions at local level or are local leaders sympathetic towards the Taliban? And what about the long term goals - is the central government making progress in building the institutions that people care about, what are the prospects for those government departments tasked with delivering physical security, education, and basic health care?
lies, damn lies and statistics
Nice comments on a long-running theme. You and I both have spent many hours both criticizing statistical indicators of "success" or "progress," but at the same time having to rely on them because there are so few other quantifiable indicators. Afghanistan may prove the point that the numbers _do_ lie, and that the "ground truth" is usually a lie.
A single and basic indicator, such as "incidents" is clearly inadequate for all the reasons you spell out. But could a more nuanced picture be presented with a variety of them -- tracking not just kinetic incidents, casualties, etc., but slightly squishier things such as Taliban intimidation, criminal activity, reconstruction projects, social indicators, opinion polling, presence and activities of Afghan forces and NGOs, etc.? Obviously the data would be difficult to obtain, but could this present a slightly more accurate picture of district or provincial-level stability?
It's also interesting to note -- as your other posting notes -- that, for all the rhetoric of their propaganda, the Taliban are also just as reliant on their own statistics to bolster their claims of victory. The thought of some poor insurgent having to do PowerPoint charts, and being questioned the Quetta Shura about his methodology, has seen me through some tough times.....