So I'm still a little unclear... if you need to use the volume control on the task bar, is it not available to you when using Zoom? Or it's available but moving it doesn't change the volume?
If you're using Windows 10, click on the volume control in the task bar and select the Volume Mixer from the pop-up menu:
here:

then here:

In the volume mixer, locate your Bluetooth speaker, and you should be able to adjust the volume of the speaker (and of the individual apps) there as well.
Hopefully that gets you closer to a solution, @alex65