All life is an experiment, the more experiments you make the better – Ralph Waldo Emerson

The short answer to this question — yes. Now let’s get to the long answer.

As we have discussed before in one of the earliest posts on this blog, the GMAT is an adaptive test. The Quant and Verbal sections will start with test-takers being posed a question of moderate difficulty and will proceed based on the test-taker’s response to that question. Depending upon whether the test-taker answers it correctly or incorrectly, the subsequent question will be easier or tougher.

The entire mechanism of an adaptive test thus rests on equating the level of difficulty of questions posed to the ability of the test-taker. For the system to work perfectly the level of difficulty of a question has to be defined accurately.

How is the level of difficulty of a GMAT question determined?
Defining level of difficulty can be tricky process. Firstly, who should define it? Can those who develop questions define it? Should the definitions be qualitative — easy, medium and difficult — or numerical?

The answers to these two questions hold the key to the presence of experimental questions on the GMAT. Firstly, since the GMAT attempts to equate the level of difficulty of the questions with the ability of the test-taker, the level of difficulty of the questions should be defined with respect the test-taker.

Secondly, qualitative classifications like Easy, Medium and Difficult do not have the granularity of the numerical classification since they are very broadly defined.

The difficulty level of a question is usually defined by its p-value — the percentage of test-takers likely to answer the question correctly. Once a certain number of test-takers answer a particular question — say a 1000 or a 2000 or whatever the test-setters define as a minimum required for statistically validity — the p-value of that question becomes fixed.

So in order to define the p-value of a question as per test-takers’ ability, the GMAT administers experimental questions as part of the test.

The experiment is on the test-taker
Since technically the GMAT is administered everyday, the GMAC has to generate new questions of varying level of difficulty on a regular basis. These new questions need to have a defined p-value. The way this done is by administering these questions randomly to test-takers as part of the section – Quant or Verbal.

These experimental questions are not used for scoring — performance on these questions does not have any bearing on your score. For example, a test-taker gets a scored question and he answers it correctly, the next question has to be slightly tougher. If the next question is an experimental question the answer to that will have no bearing on the test. The scored question that follows it will be tougher than the previous scored question.

Once a certain number of test-takers answer a particular experimental question and its p-value becomes defined it becomes a question that can be used for scoring. For example, the question developers would have broadly designed as question to be of moderate difficulty. After it is administered and as an experimental question, its level of difficultly  gets defined accurately. The p-value can be .40, .45, .53, .67  and so on. It then becomes a scored question which gets administered at a particular point in the test.

Scored questions are administered on a rotational basis for a period of time before they are  retired — they will no longer appear on the test. The questions in the GMAT Official Guide, The Verbal Review and the Quantitative Review are retired GMAT questions.

How many experimental questions are posed on the GMAT?
On the Quant and Verbal sections the number of experimental questions will be approximately 7-8 and 11-12 respectively.

These questions can appear anywhere in the test and the test-takers cannot distinguish between them and scored questions. This is the only method by which the level of difficulty of questions can be measured most accurately.

Technically this should have no bearing on test-takers. While answering a question posed to you, all you have to do is try to answer it to the best of your abilities.

For example, while taking the test if you encounter a very easy question you should not wonder:

• I think I answered the previous one correctly, then why am I getting this easy question; I am beginning to wonder whether I answered the previous incorrectly?
• This is such an easy question; it must be an experimental question, I will just mark something, move ahead and save time for tougher, scored questions

All of this information is just to help test-takers understand the nature of the test better.

Your job as I always say is very simply cut out – to answer the question in front of you in 2 minutes if it is a Quant question and 1 min. 49 sec. if it is a Verbal question.