Qitzur Shulchan Arukh – 182:12
Someone whose utensils were switched in the pub or the like, he should not use the utensils that came to his hand that weren’t his. When the owner of the item comes, he must return it — even if his own item is lost. Similarly a washer-woman who washes [clothes] for the community and brings him a shirt which is not his, it is prohibited to wear it. Rather, he must return it to its owners — even if his own was lost.
However, if it rests with him many days, until it is impossible that the owners didn’t search in the meantime for their own, then it is permitted for him to wear it. Because by default [you may assume that] the washer-woman cleared [the matter] with its owners and paid for this shirt.
This situation comes up in shul pretty often. There aren’t that many different styles of men coat, and it sometimes happens that someone looks through the coat room and realizes that the only remaining coat was one similar to theirs. Someone who left already took with the wrong coat.
I hadn’t heard, though, of a rabbi telling the person stuck in this situation that he is not permitted to wear the accidentally exchanged coat home.