By the mercy of Allaah to His slaves, as one of the beauties of this perfect religion, Allaah has prescribed that His slaves may make up for shortcomings and mistakes that they make in their worship and cannot avoid completely, by performing supererogatory (naafil) acts of worship, praying for forgiveness and so on. 

One of the things that Allaah has prescribed for His slaves to make up for shortcomings that may occur in their prayer is the prostration of forgetfulness, but it is only prescribed to make up for certain things; it does not make up for everything nor is it prescribed for everything. 

Shaykh Ibn ‘Uthaymeen (may Allaah have mercy on him) was asked about the reasons for doing the prostration of forgetfulness, and he replied as follows: 

The prostration of forgetfulness in prayer is generally prescribed for three reasons: 

1-Doing something extra

2-Omitting something

3-Being uncertain

Doing something extra: for example, doing an extra bow (rukoo’), prostration (sujood), standing or sitting. 

Omitting something: such as omitting an essential part of the prayer, or omitting one of the obligatory parts of the prayer. 

Being uncertain: such as when a person is not sure how many rak’ahs he has prayed, whether it was three or four, for example. 

In the case of doing something extra, if a person adds something to his prayer – bowing, prostrating, standing or sitting – deliberately, then his prayer becomes invalid, because when he added it, that means that he did not do the prayer in the manner ordained by Allaah and His Messenger (peace and blessings of Allaah be upon him). The Prophet (peace and blessings of Allaah be upon him) said: “Whoever does an action that is not part of this matter of ours (i.e., Islam), will have it rejected.” Narrated by Muslim, 1718. 

But if he did that extra thing by mistake, then his prayer is not invalidated, but he should do the prostration of forgetfulness after saying the salaam. The evidence for that is the hadeeth of Abu Hurayrah (may Allaah be pleased with him) who said that when the Prophet (peace and blessings of Allaah be upon him) said the salaam after two rak’ahs in one of the two afternoon prayers, either Zuhr or ‘Asr, and they told him about that, he (peace and blessings of Allaah be upon him) did the rest of the prayer, then he said the salaam, then he prostrated twice (the prostration of forgetfulness) after saying the salaam. Narrated by al-Bukhaari, 482; Muslim, 573. 

And Ibn Mas’ood (may Allaah be pleased with him) narrated that the Prophet (peace and blessings of Allaah be upon him) led them in praying Zuhr and he prayed five rak’ahs. When he had finished it was said to him: “Has something been added to the prayer?” 

He said: “Why are you asking that?” 

They said: “You prayed five (rak’ahs).” 

So he turned towards the qiblah and prostrated twice. Narrated by al-Bukhaari, 4040; Muslim, 572. 

In the case of omitting something, if a person omits one of the essential parts of the prayer, one of the following two scenarios must apply: 

Either he remembers it before he reaches the same point in the following rak’ah, so he has to go back and do that essential part of the prayer and whatever came after it, 

Or he does not remember it until he reaches the same point in the following rak’ah, in which case the following rak’ah takes the place of the one in which he omitted that essential part, and he should make up the invalid rak’ah. 

In either of these two cases, he should do the prostration of forgetfulness after saying the salaam. 

For example: a man stood up after doing the first prostration of the first rak’ah and did not sit or do the second prostration. When he started to recite Qur’aan he remembers that he had not done the second prostration or sat between the two prostrations. In that case he should go back and sit as between the two prostrations, then prostrate, then stand up and do whatever is left of his prayer, and do the prostration of forgetfulness after saying the salaam. 

An example of one who did not remember until after he reached the same point in the following rak’ah is a man who stood up following the first prostration in the first rak’ah and did not do the second prostration or sit between the two prostrations, but he did not remember that until he sat between the two prostrations in the second rak’ah. In this case he should make the second rak’ah the first rak’ah, and add another rak’ah to his prayer, then say the salaam then do the prostration of forgetfulness. 

Omitting an obligatory part of the prayer – such as if he omits an obligatory part and moves on to the next part of the prayer. For example, he forgets to say Subhaan Rabbiy al-A’la (Glory be to my Lord Most High) and he does not remember until he has got up from prostrating. In this case he has omitted an obligatory part of the prayer by mistake, so he should carry on with his prayer then do the prostration of forgetfulness before saying the salaam, because when the Prophet (peace and blessings of Allaah be upon him) omitted the first tashahhud he carried on with his prayer and did not go back and repeat anything, then he did the prostration of forgetfulness before saying the salaam. 

In the case of being uncertain, such as if a person is not sure if he has done something extra or omitted something, e.g., he is not sure whether he has prayed three rak’ahs or four, one of the following two scenarios must apply: 

Either he thinks that one or the other is more likely to be case, whether it is doing more or omitting something, in which case he should proceed on the basis of what he thinks is more likely to be the case, complete his prayer accordingly, then do the prostration of forgetfulness after saying the salaam; 

Or neither appears more likely to be the case, so he should proceed on the basis of what is certain, namely the lesser amount, and complete his prayer accordingly, then do the prostration of forgetfulness before saying the salaam. 

For example: a man prays Zuhr then he is not sure whether he is praying the third or fourth rak’ah, but he thinks it more likely that it is the third. So he should pray another rak’ah, then say the salaam, then do the prostration of forgetfulness. 

An example of when neither seems to him more likely to be the case is when a man is praying Zuhr and is not sure whether he is in the third rak’ah or the fourth, and neither seems to him more likely to be the case. In that case he should proceed on the basis of what is certain, which is the lesser amount. So he should regard it as the third rak’ah, then do another rak’ah and do the prostration of forgetfulness before saying the salaam. Thus it becomes clear that this should be done before the salaam if you have omitted one of the obligatory parts of the prayer or if you are not sure how many rak’ahs you have done, and neither of the two choices seems more likely to be the case. And it should be done after the salaam if you have added something extra to the prayer or you are not certain but one of the two choices seems more likely to be the case. 

See Majmoo’ Fataawa al-Shaykh, 14/14-16 

And Allaah is the Source of strength.