Any way to change what the headphones "play" button does (from starting Siri)?

I've had this issue on multiple Mac laptops and multiple pairs of headphones, so I don't think it is a hardware issue. In all cases, the "play" button on the headset gets changed to "Siri" whenever Zoom is open, even when I am not in a meeting. I've used everything from MonoPrice BT headphones to Beats Studio3s and Apple AirPods (Pro and now Max). They all behave the same.

Is there any control over this? It would be great if, in a meeting, the "play" button could mute/unmute, but I can't find any such controls in the Zoom settings spaghetti. If I can't control it, can I at least tell Zoom to not hijack the button?