Marshall Public Library has its roots in a private lending library serving members of five women’s clubs dating back to the end of the 19th century. The lending library serving these women’s groups was organized into the “Marshall Library” in 1902. In 1904, as the Marshall Library Association, it obtained a 50-year state charter governing its operation. That private library served the white community for 69 years. During this period, the African American community was served by the Carnegie Library at Wiley College .

In the mid-1960s, a movement was begun in Marshall to establish a municipally operated library open to all citizens. This movement was led by the Marshall Chamber of Commerce , which passed a resolution urging the City to create a library department within city government.

The movement gained momentum when the Andrew Norman Foundation offered a large matching contribution if the community could raise an equal amount of money. With the support of the Marshall Chamber of Commerce , the Friends of a Public Library orginization was established in August 1969.

A fundraising effort was organized to raise money to build a city library.  In December 1969, the city commission voted to establish a public library department. The Marshall Public Library was opened in September 1970. The following year, the Marshall Library Association transferred its land, building, collection of 15,000 volumes and endowment of $27,000 to the Marshall Public Library. The Friends of a Public Library's fundraising effort was successful. Construction on the new public library building began in December 1972. The building was completed in the Fall of 1973. Dedication of the new building and its contents was held in October of that year.