6 Apr 2014

Lies and Lives

“All’s fair in love and war”, goes an old saying. The current political discourse in our country has perhaps moved much closer to a war-like situation than it has ever been and hence it is but natural for political opponents to rake up non-issues, spin-doctor them and then present them as gospel truth as they wage this war upon their political opponents. One can view such episodes with a general mistrust of all politicians, but given the 24*7 media (including social media) attention, every such move gets wide-spread publicity and ends up influencing a section of the electorate. The truth, often, gets lost somewhere in the din.

Politicians would continue doing this to score brownie points. None other that Mr. Vajpayee told this, in 1991, to the then finance minister Mr. Manmohan Singh. (Apparently Mr Singh, a newly-turned politician, took opposition’s criticism of his first budget to heart and communicated to PM Mr. Narsimha Rao his desire to resign. PM Rao mentioned this to Mr. Vajpayee, and the latter shared the above wisdom with him and requested not to resign on this issue). 

We as a country celebrate kootneeti, the assortment of political tricks-of-trade that “grants” certain leeways to the politicians in their public utterances and private deeds. We gave this world Chanakya. However, there’s always a line to be drawn somewhere. Much of the past few months have been spent in politicians crying crocodile tears at the current status of farmers. Allegations and counter-allegations have ensued. The most public spat on this came when Gujarat Govt. was accused of being responsible for hundreds / thousands of farmer suicides (funny numbers, as we would see later), and the state govt. in question released a counter-claim that only one farmer committed suicide due to crop failure (even funnier).

Playing politics over lives of the largest down-trodden segment of the society is disgusting. Especially when it is suicide – the most gruesome result when the alternative usually is living a life of utter humiliation or penury. On top of it: cherry-picking of facts, selective accusations and gross generalizations for short term political gains. This was enough persuasion for this blog to explore what’s what, and so here it goes.

AAP website, in its chargesheet-equivalent against Gujarat Govt, published farmer suicide numbers as evidence of the Gujarat Govt’s mismanagement and lies, when it comes to farmers. It’s well known that AAP is gunning after Modi, since Modi seems to have himself become the agenda of these elections. It’s only fair (and welcome) that an opposition party asks tough questions to someone who is openly campaigning for the top job and seems to be in strong position to succeed as well. AAP quoted numbers from a reliable 3rd party in this chargesheet - conclusive evidence, one would think so, but turns out it’s not the case.

We, on this blog, in our pursuit of truth and the complete truth, rely on objective and comprehensive analysis to draw on any conclusions. We strive to base such analysis on reliable public information from non-conflicted sources. We did this in our last analysis The Most "Power"-full States, when assessing the claim of 24*7 electricity in Gujarat and the counter-claims; we would attempt to do something similar here to the extent data permits.

(I recommend reading this introductory piece before moving forward.)

In its charge-sheet, AAP quotes suicide numbers from National Crime Records Bureau (NCRB), a part of Ministry of Home Affairs under Central Govt of India. A research of their website seems to confirm their numbers would broadly pass the reliability test we hold dear. The below graph summarizes the number of farmer suicides for the year 2012 by state, using the same data.  We have considered all 17 states with population of 2.5 crore or more (as of March-11), and  have excluded very small states and UTs.

Number of farmer suicides by state, in 2012


Notes: 
1. NCRB reports number of suicides victims by profession. The suicide numbers reported under profession: self-employed - farming/agriculture, have been taken as equivalent of farmers. 

2. For W. Bengal, data for 2011 taken, since 2012 unavailable. The same is added to NCRB reported overall number of suicides for India for 2012

3. For Chhattisgarh, data for 2010 used since 2011, 2012 is reported as nil & four suicides respectively, which is very low as compared to 1000+ suicides as seen every year from 2002 to 2010: seemingly data reporting issue. The 2010 suicide data for the state has also been added to NCRB reported overall number of suicides for India for 2012. However if indeed the reported data for 2011 and 2012 for Chhattisgarh is correct data then our apologies & heartiest congratulations to the state Govt.

Source: NCRB website. Exact link of the pdf file here

As evident from the graph, six states have dismal performance to show – with 1000+ farmer suicides in year 2012. Gujarat isn’t one of these six states, in fact Gujarat is 9th in the list. This table is important to showcase why cherry-picking as done by AAP is just not done. On a standalone basis, 500-600 farmer suicides in Gujarat and Tamil Nadu seem terrible. However, on a relative basis compared to other states, not so much. We need to see this, as with any other human indicator, in the perspective of whole of India and other comparable states. Indeed, when almost all states show farmer suicides in hundreds and thousands (India, at the end of the day, still remains a poor country where farmers still rely on rainfall for irrigation and don't have reliable access to credit from formal banking channels. These numbers sadly remind us of the same.), it’s important to benchmark states on a relative basis to each other. When someone singles out a state CM (Modi for Gujarat, for eg), its only fair the state’s performance be compared to other states. While state governments need to be pulled up for even one farmer suicide, it’s equally foolish to debunk the entire economic model of such states based on such absolute numbers alone without any perspective or relative benchmarking.

However, the above graph also does not tell us the whole story. As common sense will tell you, we need to further probe the above data on two more fronts:
  • How have states performed in terms of suicide rate (i.e. no. of farmer suicides per 1 lac farmers). Clearly, large state would have somewhat higher suicides than smaller states, even if everything else is equal in these two states.
  • How has the performance been over the past decade. A state might be performing very bad 10 years back, but if it has improved over the last decade, credit needs to be given.
The table below attempts to answer both the above questions. We have divided the farmer suicide numbers by number of farmers (as per Census 2011) to arrive at farmer suicide rate for last several years, and then taken a 3-year average of the same beginning of the decade (2002/03/04) and ending of the decade (2010/11/12) for comparison.

Comparison of Farmer Suicide Rate by state (Now vs. 10 years back)

Notes: 1. Farmer Suicide rate = No. of farmer suicides per one lac farmers. 
2. Farmer Suicides data is as per the previous graph, and comments for West Bengal and Chhatisgarh as noted in that graph apply here as well. 
3. Number of farmers is taken to be all such workers are categorized as "Cultivators" or "Agricultural Labourers" under Census 2001 and Census 2011. 2011 data was used for 2010-12 3-yr-avg, a linear extrapolation from 2001 to 2011 was used to estimate number of farmers as of 2003 which was used for 2002-04 3-y- avg.  
4. Three year averages are taken to smoothen year-on-year variations.

Source: NCRB, as in the above graph, for farmer suicides. Census 2011 for number of farmers.

I have divided these 17 states into 6 categories based on their performance:
  1. Bihar/Jharkhand/UP/Punjab/Assam: This category states have far less suicide rate compared to other states. These states also have the benefit of large rivers (Ganga/Yamuna/Brahamputra) flowing through them, which provide ample irrigation water, thereby reducing chances of crop failure. Still, nothing to take away from these states, except that the performance has deteriorated over the last decade. Not good signs, but still ok by and large even today. Not many farmers are committing suicide over there, except Assam where the suicide rates were not as low to begin with as compared to other states in this category.
  2. Karnataka / Maharashtra / Andhra Pradesh / Chhattisgarh / Kerala: These category states have far higher suicide rate compared to other states, historically and recently as well. Suicide rates are in double digits (10%+) which is in far excess of the all India average of 5-6%. Worse, the suicide rates are not declining rapidly, which should be the goal given such high suicide rates to begin with; in fact for Andhra Pradesh the suicide rate has worsened over the decade. Governments in these states need to prioritize this area and reduce this shame. Kerala seems to have the highest farmer suicide, much higher than any other state - we don't rule out data mis-recording in this case since Kerala is otherwise known to be economically much better-off than other states. 
  3. Haryana: A separate category was required for this small state which had a low-to-mid suicide rate a decade back, and has deteriorated very significantly ~82%). It should be safely assumed that this state enjoys adequate river+rain-water for irrigation (same as category 1), but shows post worse performance than those states.
  4. West Bengal / Madhya Pradesh: This category states have performance similar/slightly better than national averageBoth states are large, and have farmer suicide rate closer to all-India average 10 years back as well as today. Hence, the 25-30% decline in the farmer suicide rate as seen these 2 states is also not significantly better than the national average of 20%. I would categorize this as a good performance, especially considering both states are weak on the overall economic front: having moved slightly better than all-India average is a good achievement for these states.
  5. Tamil Nadu / Rajasthan / Odisha: This category states have the performance record, which would bring a smile to your face. Slightly high (Tamil Nadu) / Low (Rajasthan/Odisha) farmer suicide rates to begin with a decade back, and today all these three states have a suicide rate far lower than the national average. Improvement levels of ~60% are commendable achievements.
  6. Gujarat: And then we have Gujarat. It doesn't fit any of the above categories. A state which has a ~33% lower suicide rate to begin with a decade back as compared to all-India average (5.0 vs 7.4), and the 9% improvement it posts is lesser than national average of 20%, albeit on a smaller base. This is not a best-in-class performance as we have seen in water and electricity, this isn’t a performance that warrants opening the champagne. Having said that, the condition is much better than most other states to begin with, and is not deteriorating thankfully unlike many other states. There is definitely a scope for improvement, as the states in category 5 have showcased. However, it is also not a case where the state government should be singularly castigated and the opposition bays for blood.
So there you go folks, Lies and Lives. Our politicians don’t care much for the latter in their pursuit of spreading the former. We would strive to discredit the former in our pursuit of the opposite, the truth. Job well done - we would say to TN, Rajasthan and Odisha, need to continue doing the good work till suicide rates are brought even lower. Gujarat ain't a demon when it comes to farmer suicides it's suicide rate is lower than national average and the condition is not worsening, unlike many many other states. Still, more needs to be done, and should have been done. Gujarat is probably the 5th or 6th best state amongst the 17 states we have considered in this analysis. 

So next time you hear someone screaming at the top of his head on how his political opponent doesn't care about the farmers, you know where to look. Don't forget to speak out.

P.S. The final piece of the puzzle, however, remains missing. The above analysis is for overall farmer suicides, and no further categorization on underlying cause of suicide is available. Indeed, while crop failure and resultant inability to pay debt and/or feed stomach is expected to be a primary cause, there would be farmer suicides which have nothing do with the occupation: family disputes, marital tensions, mental health etc. There are sample based primary research studies available which try to answer these questions, however only for a few states and at very defined time-points, and their methodologies don't give us much confidence. 

Update 1NCRB has the data on underlying cause of suicide by state, however doesn't provide the same for farmers separately. RTI?

Update 2: Further research reveals that the above-mentioned data on underlying cause of suicide (it would have been insightful to see, at a state level, how many farmer suicides are happening due to crop failure and not just absolute numbers) is not very useful - different states adopt different terminologies while recording such causes of suicides, and hence any such data (even if obtained through RTI from NCRB) would be not generate reasonably accurate insights incremental to what we already have.

Update 3For calculation of suicide rate for the two time periods of 3 years, i.e. 2002-04 and 2010-11, the total number of farmers (denominator) was taken as of 2011 in both cases. It was disclosed in a reply to one of the comments as well. While this approximation is ok for calculation of average suicide rate over 2010-2012 (2011 being the mid year), the same is not accurate for 2002-2004, where ideally number of farmers as of 2003 should have been taken. The reasons the approximation was done were a) the number of farmers data is taken from census, and there was no census on 2003, the closest census was done was in 2001 b) the census 2001 data on number of farmers was not available with the author at the time of analysis, and c) given the objective was to do a relative benchmarking of states with each other and with the same states themselves over a time-period, 80-20 rule was applied and 2011 data on number of farmers was taken as denominator for the period 2002-04 as well.

After a furore over the same, and the article being called biased/incomplete/inaccurate, the author has managed to get hands on 2001 data on number of farmers. Using this data as of 2001, along with 2011 data on number of farmers, a linear extrapolation was done to estimate number of farmers as of 2003 for each state (as is now included in the table above). The whole analysis was refreshed using this data - the graphs, tables as well as numbers quoted in the language were updated. As had been maintained by the author, while this refinement helped updating the numbers on an "absolute" basis, the inferences drawn from the analysis remain exactly the same. The performance based categorization of the sates remains the same: The states of TN, Rajasthan and Odisha were seen to be doing an excellent job earlier, and are still doing the same based on the updated analysis as well. Gujarat earlier was showing an improvement of 1% relative to all-India improvement of 18%, now this state is showing 9% improvement relative to all-India improvement of 20%, it still is a separate category in itself. The states seen to be doing bad earlier, are still in the same bucket. To repeat: absolute numbers have changed, relative positioning of states vis-a-vis each other as well as over a timeline remains the same. The 80-20 rule is often abused by statisticians to hide more than to show, however when used honestly and appropriately this rule helps arrive at insights in a much faster manner when the perfect data is not available. That said, the author concedes this irrelevant debate could have been avoided altogether by striving more to get hands on 2001 data.

Update 4: The number of farmer suicides data for overall India has been updated for 2011 and 2012: West Bengal didn't report 2012 farmer suicides numbers and hence while 2011 data for the same was used when benchmarking WB relative to other states, the overall number of suicides for India wasn't corrected, which has now been done. Similarly Chattisgarh was showing nil and four suicides in 2011 and 2012, whereas it had historically recorded 1000+ suicides for 8 years during 2002-2010. As such, the 2011 and 2012 data seemed erroneous and hence 2010 data for Chhattisgarh has been added to overall India farmer suicides for 2011 and 2012.

Update 5: Kerala was inadvertently omitted in the table of suicide rates earlier, it has been included now - it falls in category 2, probably the worst category to be in for a state. However, as noted earlier, Kerala data might not be accurate due to such negative-outlier performance by a state which otherwise is known to be economically and socially well-off.

Update 6: Punjab, Jharkhand and Assam were moved from Category 3 to Category 1, it seems they better fit there. Cosmetic re-categorization, doesn't affect the overall inferences

10 comments:

  1. Can you please provide the link for Census 2011 (number of farmers segmented by state). Found 2001 data and couldn't replicate your suicide rate for 2002-2004 for Uttar Pradesh. Btw, was there a census in 2002?

    ReplyDelete
    Replies
    1. Census was held in 2001. Also, have used 2011 data on number of farmers as denominator for both 2002-04 and 2010-12 (I could not find 2001 data and hence used same denominator). Numbers would change a bit, but inferences would remain the same due to large differences in suicide rates across states.

      Delete
    2. Whoa! Isn't that a HUGE flaw in the analysis? I looked at the 2001 census numbers (painstakingly extracted the main cultivator & main labourer numbers for each state) and compared it with the 2011 numbers -- the number of people engaged in agricultural activity has dropped drastically in a decade. So, if Gujarat has the same number of reported suicides per 100,000 "farmers" in 2001 & 2011, while the number of farmers has reduced by 16%, guess what that does to a COMPLETELY UNSCIENTIFIC, AD-HOC metric?

      That's right. Take a look at my analysis at https://docs.google.com/spreadsheet/ccc?key=0AhirOfU5ZZtQdDdlemJlQ2NsZll3NWttVU5PcmNURmc&usp=sharing Suicide rate per 100,00 farmers has increased by 28% in Gujarat. Holy-fucking-abki-baar-modi-ki-paad Batman!

      Now, my conclusion is not that this proves that Gujarat sucks. My conclusion is that this PROVES NOTHING. We can draw absolutely no inferences [1]

      If your main point is that AAP is bullshitting by cherry-picking numbers (which may be true), the way to "prove" it is certainly NOT by cooking up ad-hoc metrics in Excel and adding to the BS in public debate.

      [1] Even my analysis could be flawed. Someone should verify it. Try getting any data out of the Census website. How something is reported in 2001 is NOT the same way it's reported in 2011. For example, take a look at http://www.censusindia.gov.in/Tables_Published/A-Series/A-Series_links/t_00_009.aspx -- are the numbers on this for "main" workers or "marginal" workers or a total? I don't know. Do you?

      Delete
    3. It's a fashion to cry murder at first sight of anything red. Logical discussions are welcome, and in that spirit below response:

      1. As mentioned clearly, denominator used was 2011 data only is not 100% accurate, but given the large differences across states, even if one were to tale 2001 data for the 3 year average 2002-04, an improvement of 10% might become 12-13% and so on, but the relative differences among the states would still remain. The best states would remain at the top, the worst states would continue languishing at the bottom. A rank 5 state might become rank 4 or rank 6 by a marginal improvement / deterioration, it doesn't change the answer. 80-20 is a great tool when applied appropriately. Please do consider that putting together that dataset that you have, required you to "painstaingly extract" data from census website. This analysis uses several such datasets from Census and other sources as mentioned.

      2. Your analysis is wrong becuase it considers only "main" agri workers, and doesn't take into account marginal workers engaged in agri. Marginal workers make up for 20-50% of total agri population, and differs widely across states. Hence, to draw any inference from the data set you have uploaded would be very risky

      3. The point of this blog is not to prove Gujarat sucks or otherwise. Indeed, Gujarat has been clearly showcased not to the the best state on this metric, there are a few states above it (Category 5 definitely, and category 4 as well to some extent). Rather, the objective of this article (or this blog for that matter), is to attempt an objective analysis based on public, reliable data for debates raging on MSM and SM. Needless to say, given elections 2014 and Modi's PM candidacy and AAP's passionate campaign against Modi, most debates today are centred around Gujarat model.

      Having said that, your comment has pushed me to get detailed data or 2001 as well (I have approx numbers, rounded off in millions, I am also not sure if 100% definition consistency). If you find that, do share.

      Finally, unparliamentary language is not encouraged. Please keep f-words and similar narrative at bay.

      Delete
    4. Needs to be mentioned that first while you asked for data tables, then rescinded the request saying that you have got all data tables, which was clearly not the case. Further, it was clarified that both main and marginal workers are considered, but you have still taken only main workers.

      One of the objectives of this bog is to also discourage incomplete analysis and drawing incorrect conclusions. We would request not to share such analysis on this blog, you of course are free to advertise your analysis (even if faulty / incomplete) on your digital properties. We are being blunt here not because your analysis is faulty (everyone can make mistakes), rather because it overlooked clarifications already provided.

      Delete
    5. Article updated, as per observations.

      Delete
  2. Let it be. Found all the data tables. So, for total number of farmers, did you sum up cultivators and agricultural labourers?

    ReplyDelete
    Replies
    1. I added both cultivators and agri labourers as I noted in the article. Further, both "Main" workers (full-time) and "Marginal" workers (part-time) were included. Results remain directionally the same you consider both or only Main.

      Delete
  3. Biased , ignores base effect issues, time groupings do not cater to any rainfall led increase, if any. Given concentration of crops across states and variability of weather, among-states comparison may not be as correct as a delta improvement for the same state

    ReplyDelete
    Replies
    1. 1. Would welcome to discuss why you would consider it to be biased.

      2. Base effect issues are discussed, for eg in category 2 states which have very high bases to begin with.

      3. Further, time grouping being uniform across states means rainfall led increase/decrease is average to a large extent (we have taken 3 year average). If you think other time groupings are warranted, lets discuss why

      4. Among-states comparison can never be perfect due to factors you pointed out such as variations in crop and weather. In fct, most important factor is availability of water for irrigation from sources other than rainfall. This issue is mentioned clearly when discussin UP and Bihar, since both states have the advantage of Ganga/Yamuna running through them. For this reason, this analysis just doesnt do a State A vs State B analysis, rather also compares State A 2010-12 average with State A itself 2002-04 average

      Comments are welcome

      Delete

Comments are not moderated. Civil language is expected. Extremely abusive posts can invite deletion, however only after having displayed potential to snowball into a slug-fest.