The imam sat briefly after the third rak’ah and the people praying behind him alerted him to that; should he do the prostration of forgetfulness?
If the matter is as you describe, then the members of the congregation do not have to do anything for standing up before the imam, thinking that the imam had said the takbeer for standing up for the fourth rak’ah.
As for the imam, he should have done the prostration of forgetfulness, because he sat with the intention of reciting the tashahhud, not with the intention of resting.
It says in Kashshaaf al-Qinaa’ (1/395): If the worshipper deliberately does some additional action in the prayer, such as standing, sitting, bowing or prostrating, then the prayer is invalidated according to scholarly consensus, because he has spoiled the order of the prayer and changed its form.
But if he adds that by mistake, even if the sitting that he added at the wrong place was the length of sitting for a rest immediately after a rak’ah, such as if he sat immediately after it for the tashahhud – whether or not we say that it is mustahabb to sit for a rest – because he did not intend that when he sat, rather he intended to recite the tashahhud by mistake, then it is obligatory for him to do the prostration of forgetfulness, because the Prophet (peace and blessings of Allaah be upon him) said, according to the hadeeth of Ibn Mas’ood (may Allaah be pleased with him): If a man adds or omits something in his prayer, let him prostrate twice.” Narrated by Muslim. End quote.
But if the imam did not do the prostration of forgetfulness, thinking that he did not have to do it, or because he forgot to do it, then his prayer is still valid.
And Allaah knows best.