differentiate non-speakers from speakers

In a language class, where interaction is expected and happening throughout, some people are also only listening. It is hard to tell the listeners from the speakers, so we waste time calling on people who aren't going to be speaking. Looking for a solution.