I'd say the easiest way is by an explanation of sexual selection akin to that of birds of prey, in which the females were originally as big and strong as males, but overtime, they started grouping together during mating periods and sexually selecting smaller and weaker males which were safer and less aggressive to mate with, while ganging up on and killing stronger males which tried to force their way with them. Overtime, this resulted in a society with slightly weaker and smaller males than females. Additionally, having a matriarchal society, we could have a cultural poligamic scenario in which only a certain number of males is kept at a time, with all other males born once the number is at its maximum being either killed or kept, should one of the members of the male group be already at death's door. This wouldn't be that hard to think, as we had a similar scenario in Sparta (with the selection term being deformities rather than sex).
Lastly, they could be a group in between the Neanderthals and the sapiens, likely successful "hybrids" which inherited the best of both groups, and thus, like the Neanderthals could, be completely capable of interbreeding with normal humans.