Speaker View freeze/crash

Hi! Since a couple of months Zoom keeps crashing when using the Speaker View. No issues in Gallery View.

Issue persists since multiple Zoom updates (currently on 5.10.1).

I only use Zoom for our team meeting with approx. 15 participants, all with Video enabled. I am the only one with this issue. When in Speaker View the speaker window freezes after a couple of minutes (usually 1-10 Minutes), the smaller video feeds on the top still work fine for a couple minutes longer. Doesn't matter if someone is screen-sharing or not. The GUI becomes unresponsive and the audio starts to stutter from time to time. When I click somewhere in the app, then Windows gives me the "The app has stopped responding" window and asks me to kill the app. If I switch to Gallery View some time before the speaker window freezes, then no issues occur during the 1-2 hr meeting.

I tried reinstalling Zoom and updated Nvidia drivers.

Specs:
Zoom 5.10.1 (happened on multiple previous versions as well)

Windows 10 Edu, 21H2 (build 19044.1645)

Intel i7-7820X (8-Core), 32GB Ram, Nvidia RTX 3080 10GB

Nvidia Broadcast Microphone Filter activated

Any help would be greatly appreciated!