Conversely, most of my "sandboxing" happens in game, at the table, and in the moment. I think that’s the best place for it to happen, because you get the instant feedback of the players to gauge where things are going and if they are playing along. You also get to react to their reactions, making hurried notes if one of them coughs up a good idea. You maybe didn’t plan for their stay at the inn to be eventful, but suddenly, all of the players are fiddling with their phones and you’re about to lose them to the Facebook. That’s when the two guys in the corner who have been talking quietly all night suddenly throw their table aside and draw weapons, and everyone in the bar roars their approval, and the bartender says, “Dammit, not again!” Sometimes you need to insert a little action while the players find their way.
That push and pull, or better yet, that back and forth rocking motion between “here’s the module I have prepped” and “what weird-ass thing do y’all want to do now?” is the essence of great DMing, and that’s the point I’ve been trying to make in my overly-long-winded way. Open play, all of the time, would beat me down and drive me crazy, not to mention creatively exhaust me. Likewise, sticking to the script and excising all of the NPC byplay and not letting the players breathe would also be zero fun.
The real trick is knowing when to switch modes. There are days when you don’t want them lollygagging about, and there are days when everyone is having the most fun gambling in the tavern. If you have the energy and the inclination, give them a “by week” and let them make friends, make enemies, and find clever ways to part them from their gold. If you don’t have the energy for a big thing, you can downplay the encounter by keeping personality off of the table. Instead of infusing the local blacksmith with an Irish brogue and an ailing wife—for that would certainly be player bait—instead say, “The blacksmith only has common weapons for sale, at the prices listed in the player’s handbook. Nothing exotic. If you want a new sword, there’s a rack of them on the south wall.”
90% of the time, they will take the hint. Buy your equipment and get ready to move out. There’s an owlbear to massacre.
I find the best time to free-wheel an NPC or two is after a mission or a dungeon crawl. It’s a nice palate cleanser from getting hit in the face with fireballs and digging through crypts. Combat is intense, especially at levels 1-5. Everyone is acutely aware of hit points and spell slots. Short rests are crucial. So, when that’s over, it’s good to let them sit back and just talk out a problem, or try to get the best deal for all of the loot they swiped.
In gaming, as well as in creative writing, there’s nothing wrong with some boundaries. A little structure makes play (and creating) much better by limiting choices and allowing you to focus in on the best outcome.