Short answer is it depends.
Starting point is the contract with your present publisher. It's not uncommon for your publisher to have some rights in the next book in a series. You might have a fair amount of room or none at all.
Assuming you've got some freedom, the next question is how big a run did your first publisher print and how well did they sell? (Print run obviously disappears with epublishers.) Numbers matter.
If you've got the ability to change publishers and you have some decent numbers (at least four figures), that's your lead in your query. You'd start your query with something like: I am looking for an agent to represent my book SQUIRRELS MAKE LOVE TO QUERIES, the sequel to SQUIRRELS SCREW UP QUERIES. My first book, published by Pressed Acorns, sold over 5,000 copies and I think an agent would help me get a larger publisher interested in the next one. Then the next paragraph is the story.
If you've got the ability to change publishers (or self-published), and don't have the numbers, you mention this in the last paragraph of the query, knowing it's not going to help you one iota, but you've got to be honest. You can also try to sell the query as a stand alone book, and not mentioning your previous book until the phone call.
Best of luck,
Jim Clark-Dawe