Qitzur Shulchan Arukh – 65:2
Someone who stumbled and did take interest is obligated to return it; except for an increase collected before the loan, or one collected after the loan, as we shall see in law #6.
Given that the violation is a lack of fraternal feeling, it is possible that we would think that after the fact, if the interest were paid, it would be the property of the lender. However, this halakhah teaches us that the money should be repaid. Presumably (in other words: I’m guessing) because in the long run, it will foster that unity even if the loan is made interest-free after the fact.
The second clause excludes gifts given before the loan in an attempt to convince the lender to lend the money, or gifts given after the pay-back in appreciation for the loan. (This aspect of the prohibition is discussed in full in 65:6.) But actual interest must be returned to the borrower.