He died and left behind his sister and his brother’s children
Praise be to Allah.
If a person dies and did not have any children, and also did not have a father or a grandfather, then this case is called kalaalah, which refers to the one who dies and does not leave behind any parent or child.
Kalaalah is mentioned in the Qur’an, in the verse in which Allah, may He be exalted, says (interpretation of the meaning):
“They ask you for a legal verdict. Say: ‘Allah directs (thus) about al-Kalaalah (those who leave neither descendants nor ascendants as heirs). If it is a man that dies, leaving a sister, but no child, she shall have half the inheritance. If (such a deceased was) a woman, who left no child, her brother takes her inheritance. If there are two sisters, they shall have two-thirds of the inheritance; if there are brothers and sisters, the male will have twice the share of the female. (Thus) does Allah makes clear to you (His Law) lest you go astray. And Allah is the All-Knower of everything”
[an-Nisa’ 4:176].
And Allah, may He be exalted, says (interpretation of the meaning):
“If the man or woman whose inheritance is in question has left neither ascendants nor descendants, but has left a brother or a sister, each one of the two gets a sixth; but if more than two, they share in a third; after payment of legacies he (or she) may have bequeathed or debts, so that no loss is caused (to anyone). This is a Commandment from Allah; and Allah is Ever All-Knowing, Most Forbearing”
[an-Nisa’ 4:12].
Shaykh Muhammad al-Ameen ash-Shinqeeti (may Allah have mercy on him) said: The correct view is that what is meant by kalaalah is leaving no ascendants or descendants, as an-Naazim said:
They ask you about al-kalaalah… it definitely refers to lack of descendants,
No surviving parent or offspring, so there are no sons, (no parents) and no grandparents.
End quote from Adwa’ al-Bayaan (1/228)
Based on that, if that person who died did not have any father or son, and the rest of the heirs are as mentioned, then there inheritance is to be divided in the following manner:
The full sister gets half, as the share allocated to her by sharee‘ah, because Allah, may He be exalted, says (interpretation of the meaning):
“They ask you for a legal verdict. Say: ‘Allah directs (thus) about al-Kalaalah (those who leave neither descendants nor ascendants as heirs). If it is a man that dies, leaving a sister, but no child, she shall have half the inheritance…”
[an-Nisa’ 4:176].
The two sons of the full brother get the rest of the estate, because the Prophet (blessings and peace of Allah be upon him) said: “Give the shares of inheritance to those who are entitled to them, and whatever is left goes to the closest male relative.”
Narrated by al-Bukhaari (6746) and Muslim (1615)
The two daughters of the full brother (the two nieces) do not inherit anything of the estate.
The division of the estate mentioned above is based on the assumption that the sister mentioned in the question is the full sister of the deceased or his sister through the father; the same is assumed in the case of her brother.
For more information, please see the answer to question no. 160948
And Allah knows best.