How to set a maximum breakout group size?

You can create the breakout rooms in advance when you create the meeting and there set the number of groups and participants in each one

Go to your Teams calendar and open the meeting invite.
Go to Breakout rooms and select Create rooms.
Choose the number of rooms you want (up to 50) and select Add rooms. 
Select Assign participants.

Greetings,
James

You can create up to 100 breakout rooms and assign up to 1,000 total participants.