Anything more than five and the conversation does not work so well: one or two people tend to dominate; the conversation breaks into two, even three; frequently one person is totally cut out of the interaction and there is little energy in the group.
This research paper (via Keith de la Rue) confirms my observations.
Current communication models draw a broad distinction between communication as dialogue and communication as monologue. The two kinds of models have different implications for who influences whom in a group discussion.
The experiments reported in this paper show that in small, 5-person groups, the communication is like dialogue and members are influenced most by those with whom they interact in the discussion.
However, in large, 10-person groups, the communication is like monologue and members are influenced most by the dominant speaker.
The difference in mode of communication is explained in terms of how speakers in the two sizes of groups design their utterances for different audiences.