If the container stays on private property, which for most jobs means your own driveway or a cleared spot beside the house, you need nothing from the City of Columbia. That covers the overwhelming majority of deliveries here.
Street placement is different, and Columbia is one of the clearer cities on it. The City's right-of-way page says that encroachments into the City right-of-way that may raise issues of safety or accessibility, or affect facilities used and enjoyed by the public, may require a permit, and it lists dumpsters among its examples alongside outdoor seating, sidewalk closures, scaffolding and street parking closures. The application is published as a PDF on the City's right-of-way encroachment page. No fee figure is published there, so ask when you apply.
There is a wrinkle that catches contractors in Columbia. State routes inside the city limits are maintained by the state transportation department, not by City Public Works, and the City names 43 Bypass, Nashville Highway, Bear Creek Pike, Carmack Boulevard, Trotwood Avenue, James Campbell Boulevard and Santa Fe Pike among them. If your frontage is on one of those, the right-of-way is not the City's to permit, so raise it early and allow extra time.
On the collection side, Columbia is open. The City Recorder publishes the full list of permits the City issues, covering alarm systems, beer, business licenses, fireworks, hotel and motel, liquor, mass gatherings, peddlers, taxicabs, temporary vending, transient vendors and yard sales. There is no waste or hauler permit anywhere on it, and nothing in the City's published material conditions a container on an approval from the City. Your account manager still confirms the address before booking. City Hall is at 700 North Garden Street and Public Works answers on 931-388-8650.