In 1772 Peter Olcott of Bolton, Connecticut, settled in Norwich on a south-running ridge between Bloody Brook and the Connecticut River. Olcott played an important role in shaping his town and the Republic of Vermont (1777–1791), holding town meetings in his house between 1774 and 1780 and serving as brigadier general of the Third Vermont Militia and on the governor's council. In 1789 and 1791, when as lieutenant governor he was at the peak of his influence, he hired local framers and carpenters to build a two-story, hipped-roof house, as well as a cow house and a hay barn. Today, the house is a landmark of Georgian style due to its lavish, paneled interior. Also remarkable is the survival of two eighteenth-century barns on one farm, a rarity in Vermont.
The house has a center-chimney layout, with one side slightly deeper than the other and a relatively plain, clapboarded exterior. About 1850 the Georgian double door of its south-facing, central entrance was replaced and the surround modified to include sidelights. Inside, the entrance and stair and the paneled parlor and dining room are untouched from Olcott's time. Their sophistication led Herbert W. Congdon to rhapsodize in Old Vermont Houses (1940) that the work was “the finest paneling I have seen in Vermont.” Especially noteworthy are the raised arched panels, fluted wall pilasters, and an elegant wine closet with an arched window light in the dining room.
According to Olcott's ledgers, the 1789 wood-frame, English cow barn was framed by Ashbil Pasavil of Norwich and built by Ebenezer Broughton and Silas Carpenter. It was originally located east of and somewhat closer to the road than the house in a typical eighteenth-century arrangement. It was moved behind the house and connected to a larger c. 1930 dairy barn. A second wood-frame English barn was constructed behind the house in 1791. It remains intact with its great wagon-drive doors facing east and west.