Firstly: 

Spending money to help one’s brother or anyone else perform Hajj is a great righteous deed, because it is helping him to perform this important act of worship which will lead to his being raised in status and his bad deeds being erased. But it is not permissible for this help to be a cause of you falling into that which Allaah has forbidden, such as taking a loan from a riba-based bank, for riba is a serious matter, and warnings have been issued concerning it that have not be issued concerning any other sin. 

For more information please see question no. 6847 and 9054. 

What you have to do is repent to Allaah from dealing in riba. 

You should note that it is not permissible to deposit money in a riba-based bank except if there is fear for one’s wealth and there is no Islamic bank. In that case you should deposit in a non-interest-bearing account, because one of the basic principles of sharee’ah is that necessity may make that which is forbidden permissible, but only as much as is necessary. 

Secondly: 

Your brother’s Hajj is valid in sha Allaah, because he took the wealth from you in a permissible manner, either as charity, a gift or a good loan (i.e. one without interest). 

Some scholars are of the view that the wealth which a person acquires by haraam means – such as riba – is haraam for the one who acquires it only, and it is not haraam for the one who takes it from him subsequently in a permissible manner, such as selling, gift-giving and so on. We have discussed this in the answer to question no. 45018. 

The prohibition applies only to you having taking a loan with interest, not to your brother. 

May Allaah help us and you to do that which He loves and which pleases Him. 

And Allaah knows best.