The Gardener's Quarters began life as a turkey house on the farm lot southeast of the mansion. Together with the Garden House. it was moved to the edge of the yard in the late 1940s and remodeled into an office for Chippokes' proprietor, Victor Stewart. Later, in the 1950s, it served as the residence of the principal gardener. A modest one-story building consisting of two one-room-plan blocks, it was heavily remodeled in the 1980s and now serves as public toilets.