Frequently Asked Questions About Who Speaks for Islam?

Frequently Asked Questions

How did you define the "politically radicalized" and "moderates" in Who Speaks for Islam? What a Billion Muslims Really Think?

In the book Who Speaks for Islam?, we define the "politically radicalized" as respondents who A) answered a "5" when asked to rate the extent that 9/11 could be morally justified on a 5-point scale, where "1" is "cannot be justified at all" and "5" is "completely justifiable," and B) said they view the United States unfavorably. A population-weighted average of 7% fit these criteria. We labeled those who said 9/11 could not be completely justified as "moderates." We further broke this group down into those who were pro-United States and those who were anti-United States.

The decision as to where to break out the "politically radicalized" from the rest was data-driven. It was based on several analyses of where the data clustered for a natural breaking point. The analyses showed that the people who responded with a "5" (completely justifiable) to the question on the justifiability of 9/11 as a group were distinctly different from the groups who responded with a "1", "2", "3" or "4." The graphic below provides an illustrative example: It shows the percentage of people in each of the 5 groups who said "sacrificing one's life for a cause one believes in" is completely justifiable. The group that responded to the 9/11 question with a "5" look distinctly different from the groups that responded with a "1" to "4."

For our widely read November 2006 Foreign Policy article "What Makes a Muslim Radical?", we analyzed the data we had available from nine countries. At that time, our sample clustered in a different way from how it ultimately would when we expanded the number of countries in our database to more than 35 and ran new analysis in 2007. We defined "radicals" (as opposed to the "politically radicalized") as those who answered a "4" or a "5" to the 9/11 justification question, and compared this group to those who answered with a "1" or a "2" (who, for the sake of the analysis, we labeled "moderates"). In 2007, we ran new analyses with the larger dataset, which resulted in new data clusters. The new results from the new clustering based on the larger sample were what we included in our book. As our work continues and more countries are added and more trended data become available, we will update all of our analyses to reflect the latest findings.

How many people answered a "4", "3", "2," or "1" to the 9/11 question, and how can you say someone who answers a "4" (mostly justified) is "moderate"?

The percentages of people who answered "1" to "4" to the question about 9/11 are listed below.

The term "moderate" is more of a placeholder label than a value judgment. It is similar to calling one clustering in the data "group A" and another "group B." We simply used labels that a broad audience can easily understand and remember. Some have also asked how we can call someone a "radical" simply because they thought 9/11 was justified and actually had not *done* anything. The idea here is not that we are judging who or what a "moderate" or "radical" is, but rather assigning labels to statistical groups that we clearly define.

How does one access the raw data?

Core data used in Who Speaks for Islam? can be found in the Core Data area of this website. Data and insights from the Gallup World Poll are available on both a public and proprietary basis. A selection of key findings regularly appears on the Gallup Web site and the Muslim-West Facts Initiative Web site. and may be accessed for free. However, the full data set of global and country-specific data for this project, as for the entire Gallup World Poll, is accessible only for a fee. Gallup maintains its nonpartisanship and independence -- by self-funding its multimillion-dollar global research endeavor and never accepting money from special-interest groups or political parties. It is also how Gallup is able to support its independent research and continue the World Poll in coming years.

How do you decide what questions to ask?

Gallup has been in the business of asking people questions for a long time -- we know which questions and question formats will yield valid answers. Starting first with a knowledge base of the region, culture, or issue, we develop a research question based on something we want to know more about and measure over time. We then determine how to convert it into questions that people will know how to answer across cultures -- questions that measure public values, perceptions, attitudes, and desires. These questions possibly could have never been asked before in a poll, or could be longstanding questions never asked of a certain group.

How do you get permission from authorities to poll in a country, region, city, town, or village?

It varies. In several countries and regions, such as the United States, Europe, or in sub-Saharan Africa, there is no formal policy. Some countries require government approval of the questionnaire (including China and some countries in the Middle East); sometimes a government may require that certain questions be cut. In some cases, the questions we ask will determine whether we get permission to poll in that country.

In most rural areas, to show respect and gain credibility as strangers in a village, we request permission of the village elder. In other cases, we inform the local police. It's a balance of what's legally possible and what's practically possible.

How do you translate the questionnaire?

We have a rigorous and consistently applied translation process.

• When designing the English version, we first focus on clearly defining the essence of the question so that the language translation can capture it. Then, we account for education and culture, and word the question to the lowest common denominator, with simple vocabulary that's to the point.
• When translating into another language: Two people independently translate the survey from English into the native language -- one of whom understands survey design. Those versions are compared and vetted into a third version by a non-researcher, an "everyday person" to ensure the language preserves the spirit of the original question while being understood by "the common man on the street." The third version is then "back-translated" from the native language into English -- to ensure that the concepts still line up with the original questionnaire.

How do you design the survey to elicit people's real feelings and not the "socially desirable" answer? How do you know if people are telling you the truth?

The Science

• We use techniques to develop question wording that is unbiased and unthreatening. We give respondents chances to "opt out" by saying "I don't know" or "Don't want to answer" so the respondent doesn't feel forced to answer. We give respondents the sense that there's no right or wrong answer by presenting all response choices as completely valid and acceptable.
• When an item is controversial, we distance the respondent from it by saying, "Some people say x, others say y -- Which is closer to your view?" We tend to ask more controversial questions later in the interview, after comfort and trust have been established.
• We do extensive pre-testing of the questionnaire, including testing items as open-ended questions first.

The Interviewers

• Gallup uses local people, respondents' fellow citizens, as interviewers, versus government representatives. We train the interviewers to ask the questions in an unbiased manner.
• As in a journalistic interview, we work to create optimal conditions for eliciting truthful responses by establishing rapport; we ask respondents basic questions about their lives and what's important to them, and get them to realize we are sincerely interested in their views. We often have people react with, "What an interesting question -- no one has ever asked my opinion about that before."

Are there some topics that you cannot address in a survey process? What are those?

• While there are no standard rules, in countries where polls can "ask anything" there will inevitably be topics that make people feel uncomfortable and are seemingly controversial. Designing a robust survey really requires good knowledge of the local/national culture. Questions that may be socially taboo to ask in one region or culture may not be in another. There could even be unexpected nuances such as offending someone by asking if they have "running water" vs. "drinking water" in their home.
• In countries where permission to poll is required, there may be cases in which the leadership doesn't want us to ask questions it doesn't want to know the answers to.

What about the differences in some cultures, the way women/men talk to women/men, youth to elders, etc.? Are you able to get candid responses from women in certain male-dominated societies?

• We employ a diverse body of interviewers that include men, women, the young, and older people. In regions where there is gender sensitivity, we often send interviewers in pairs -- one man and one woman -- so that we can match the gender of the interviewer and respondent (as in Saudi Arabia).
• In some cases, family members may insist on staying in the room, listening, or trying to provide their own opinions. However, because of the very specific method of our random sampling process, we only want to record the responses of the person we've selected. Our interviewers are trained to politely focus only on the respondent.

How can a sample of 1,000 be representative of a nation with 50 million people? Don't you need more for a large country than a small country?

• We start with a sampling frame that is representative of 50 million people, break it down to identify each region and each area of land in that sampling frame, so that each land area accounts for all the people in that region, and each person therefore has an equally random chance of being selected.
• The key is ensuring that the selection process is truly random and conducted based on probability. The number of cases over 1,000 doesn't matter as much as the randomness of the sampling. Twenty thousand cases of literate adults aged 20 to 50 in a solely urban area will not be as representative of a population as are 1,000 cases across a nation that captures urban/rural, young/old, and literate/nonliterate.

What's makes up a representative sample?

• Approximately 1,000 people in a country that captures urban/rural, young/old, and literate/non-literate who are selected randomly by a process of categorizing a country into segments so that all people in that country are accounted for. We then drill down to a region, to an area, to a city/town/village, to a household, and finally to a person in the household.
• Compare it to blood testing: You don't need to take more blood from a bigger animal than a smaller one to test what's in the blood.

What about people who aren't at home much or if no one answers the phone or answers the door? Aren't there sections of the population that get missed?

• We make three attempts to re-contact the respondent and follow specific procedures to randomly select a replacement.
• Sections that do get missed include: those who are hospitalized, institutionalized, in prison, or deployed out of the country.

What about the people who refuse to answer the questions?

• The response rate for face-to-face interviewing is usually high, approximately 90% in sub-Saharan Africa.
• Flat-out refusals are typically only about 1% to 2%.

How far does Gallup go to ensure the randomness of the sampling? To reach hard-to-access, extremely rural areas?

• Once the home has been identified, Gallup interviewers will take whatever transportation means necessary -- including walking -- to reach that household.

How do you hire and train interviewers?

• Interviewers are trained extensively on the survey to the point at which they can deliver it as though it were a naturally flowing conversation; this helps to establish rapport and trust among respondents.
• Interviewers are also trained extensively on how to draw people out, as with journalistic interviews, as well as manage challenging or difficult situations.
• They conduct mock interviews in a classroom and are then observed conducting live interviews by a supervisor.
• Interviewers are trained extensively on how to accurately and consistently execute the random sample methodology to ensure a representative sample.

What does it mean to be "statistically significant"?

• A difference between two percentages is statistically significant when the difference is not due to chance. When tested again and again -- 98 or 99 times out of 100 that difference would repeat itself. The World Poll uses a 95% confidence level interval.