How to conference call with Computer Video & Loudspeaker with only Vocal Mic?

When in budget situation for conference call

• AV system has Vocal Microphone
• Local participants spread over conference room
• Computer is connected to AV system
  and hardened to prevent tampering from non-computer-literate users

With poor audibility from vocal microphone, I’m wondering what if local participants use:

• Computer Zoom App connected to AV Projector Screen and Room Ceiling Loudspeaker. 
• Own Smartphone App for Microphone 

When Virtual participants speak, local participants view and listen via AV Projector Screen and Room Ceiling Loudspeaker ; mute own Smartphone (Audio and Video). 

 When one of the local participants speak, 

• Walk over to AV system to lower Room Ceiling Loudspeaker Volume 
• Unmute his/her Smartphone Zoom Audio
• Other locals use his/her Smartphone Zoom to listen
• Everyone view via AV Projector Screen

The main disadvantage is hearing un-synchronised Sound from neighbouring Smartphone Zoom; will connecting Smartphones to same WiFi network improves sound synchronicity?

Is there a better alternative?