Deleted Users Appearing in DMs

[3asop]

I have a user report an issue where all the deleted user team chats keep appearing in his DM and Chat channels. There is no option to remove them except to clear chat history in their in-app settings nor the admin center.

We seem to be able to create a folder and move them there, but that isn't an ideal solution.