I am an overwriter. If you read my posts here at AW you might see that this is true. And so when I subbed my first book and an agent was interested, she would only offer rep after I cut 10 000 words.
I only actually cut one chapter. I later re-wrote a sequence in the book, but that had nothing to do with word count. I managed to cut 10 000 words out of my book without getting rid of any characters, or scenes (aside from that one chapter, and that truly was an extraneous chapter, I didn't mind in the end). I simply read my paragraphs and realised that I simply didn't need that many words to describe what I was describing. My dialogue that I thought was oh so witty, actually just went on a bit too long.
And after I'd cut everything, well you'd be hard pressed to pinpoint what I'd cut, because the voice, the quality of the writing everything was exactly the same.
The one big difference? The book was SO much better. It was tighter, it didn't have extra padding. It read, and this is going to sound weird, like a real book.
I bet you could cut at least ten words per page. You could definitely cut more than that. But let's look at ten words per page. If your book is say 300 pages . . . that's 3000 words. Already. A huge dent.
Also, I agree with the others that aside from the wordcount and possibly saturated market for your work (which market is it btw?), it just might be the query. Even if you enclosed a chapter 1, that doesn't mean that any agent actually read it. If the query isn't strong enough, doesn't capture an agent's interest, then they won't move onto the pages. Fixing a query is tough, but it isn't impossible. And it's worth it.
So to answer your question, no, no you shouldn't give up on this work just yet. If anything you should take advantage of this situation and use it to learn and improve your craft. So many people move on so quickly when things aren't going 100% the way they want them to, they never get the chance to really work at something and grow. It's worth taking the time with this simply to grow as a writer. Even if in the end this isn't the book you end up selling.