The neuropeptides vasopressin (VP) and oxytocin (OT) and their receptors in the brain are involved in the regulation of various social behaviors and have emerged as drug targets for the treatment of social dysfunction in several sex-biased neuropsychiatric disorders. Sex differences in the VP and OT systems may therefore be implicated in sex-specific regulation of healthy as well as impaired social behaviors. We begin this review by highlighting the sex differences, or lack of sex differences, in VP and OT synthesis in the brain. We then discuss the evidence showing the presence or absence of sex differences in VP and OT receptors in rodents and humans, as well as showing new data of sexually dimorphic V1a receptor binding in the rat brain. Importantly, we find that there is lack of comprehensive analysis of sex differences in these systems in common laboratory species, and we find that, when sex differences are present, they are highly brain region- and species-specific. Interestingly, VP system parameters (VP and V1aR) are typically higher in males, while sex differences in the OT system are not always in the same direction, often showing higher OT expression in females, but higher OT receptor expression in males. Furthermore, VP and OT receptor systems show distinct and largely non-overlapping expression in the rodent brain, which may cause these receptors to have either complementary or opposing functional roles in the sex-specific regulation of social behavior. Though still in need of further research, we close by discussing how manipulations of the VP and OT systems have given important insights into the involvement of these neuropeptide systems in the sex-specific regulation of social behavior in rodents and humans.