Fact Check

Swachh Survekshan- Step By Step

STORY
Varanasi cleaner than Bhubaneswar? Swachh Survekshan needs to come clean on city ranking

Hindustan Times, May 7, 2017

The article raises concerns about the methodology used in the Swachh Survekshan survey to rank cities on cleanliness.

THE PROBLEM

There seems to be a lack of understanding about the ranking system used by the survey. The writer compares the ranking of cities in the 2017 survey with that of the 2016 survey. That is a fallacy, since the number of cities covered in 2017 was 434 while only 73 cities featured in 2016. It also assumes that data submitted by municipalities may have been accepted without proper verification.

THE TRUTH

  • Purpose of Swachh Survekshan was to foster healthy competition among cities for improving cleanliness standards
  • Cities which made more of an effort to improve sanitation standards in the year in question scored higher than others
  • 434 cities featured in the 2017 survey while only 73 were covered by the 2016 survey
  • Some cities slipped in ranking primarily because the sample size was much bigger in the 2017 survey
  • 45% weightage given to self-declaration by municipalities
  • After data was collected from municipalities, it was verified by a two-step process:
    Step 1: Independent Validation: Municipal data was matched against citizens’ responses to a questionnaire
    Step 2: Observation: To validate the data provided by municipalities, surveyors recorded observations and findings along with photographs. To ensure authenticity, location, time and date were tagged to all pictures
  • This independent validation and observation had a weightage of 25 per cent
  • Citizen feedback had 30% weightage
  • Citizen feedback cannot be questioned, since 1,000 samples were taken regardless of the population of a city to preclude any unfair advantage to big cities
  • The survey was democratised through the Swachhta App; scores were given based on the data on App downloads, number of complaints received, resolved and rejected by municipalities
  • By this parameter, a city with more proactive citizens scored higher
  • Civic bodies that promoted the App aggressively and made efforts towards citizen engagement managed to get an edge

A look at the above facts appears to belie the article’s concerns about the methodology and ranking system used by the Swachh Survekshan survey. At each step, the survey used a process to check the authenticity of the data gathered from the preceding phase. The reasons why a city may have done better or worse than the 2016 survey are also tend to adhere to the logic of the methodology. How a city performed depended on the initiative of its citizens and municipality.

KEY TAKEAWAYS

  • Aim was competition among cities for improving cleanliness
  • 34 cities featured in 2017 survey, only 73 in 2016
  • Sample size much bigger in 2017
  • Larger sample size contributed to some cities slipping
  • Data collected directly from stakeholders
  • Data from municipalities verified by a two-step process
  • Standard 1,000 samples for citizen feedback to minimise big-city advantage
  • Scored on Swachhta App downloads, complaints received, resolved & rejected
  • Cities with more proactive citizens scored higher
  • Technology utilised to minimise subjectivity

Unlike the usual methodologies employed by surveys, Swachh Survekshan 2017 collected data directly from many of the stakeholders. Besides, technology was optimally utilised to minimise the problem of subjectivity. Before judging the credibility or lack thereof of the survey, the above facts should be taken into account.

Share