Here's how it worked on my current WIP. I found the bones of a story (the possibly apocryphal origins of the Claddagh ring design) with a number of associated names and places. I did some research into the era, and realised that I could create a rich and exciting world around this story, filling in the blanks that history had left me. (I also realised that Buffy may have created a market for this story, since that was the ring that Angel gave her.).
Here's how it worked with the other story that I'm incubating. I took a trip to a place that was once a smuggler's haven (Kinsale, Ireland), and then created characters that would work in that setting. The story comes out of the setting and characters (there's a widow and a pirate. Swashes will buckle).
So in one case it was story first, setting later. While in the other case it was setting first, then story.