Gilman was not only interested in education. He also initiated the Northeast Harbor Library and Reading Room, and was instrumental in the organization of the Neighborhood House. Gilman "realized the need of a social meeting place throughout the year, where books, games, and lectures could be enjoyed and to this cause he gave liberally of his time and money." After the Neighborhood House was constructed, Miss Eleanor Blodgett donated all the furnishings for the new building.