Closed caption: why all participants see it?

I've attempted to use Zoom with closed captions more than half a year ago.

When one of the participant has enabled it, all participants have seen it and for some it bothered.

Although all were understanding when figuring out it's due to one of the participants being hard of hearing, and not just a feature someone enabled out of "nice to have", it clearly bothered some because in a tech meeting with tech and internal terminologies, the quality was poor and thus a bit disturbing.

To my surprise there seemed to be no way to enable it selectively only for a single user.

I'm in a new org now, and thinking about adopting Zoom, it's important to understand whether it's still the case. 

Unfortunately the Zoom website doesn't seem to be clear on this important aspect.