Mid-19th century Irish identities divided along lines of class, religion and gender but it could be argued that all were constructed in an atmosphere of the negative characterization of the island and its inhabitants by the British elite. Race and low “moral character” were blamed for the endemic poverty of the island. The Irish poor were portrayed as a “race apart” whose inherent failings were at least partly to blame for the mortality they suffered during the Great Famine of 1845–1852. Recent excavations at Kilkenny workhouse and Spike Island convict prison have produced skeletal assemblages from this critical period. These collections have enabled bioarchaeological analysis of parameters mentioned by the Victorians as indicative of the distinctiveness of the Irish poor: stature, interpersonal violence, and tobacco-use. We argue that the differences between Irish and British populations in stature and risk of violence were exaggerated. Such characterizations, we suggest, were part of a strategy of “othering” that served to legitimize colonial domination. This exertion of power did not go uncontested, and we argue that aspects of both the material culture and the skeletal evidence may be indicative of forms of passive resistance.