A first-half goal from Justin Richards secured Vale a place in the third round for the first time in five seasons.
And Coleman said: "We didn't turn up for the first half-hour and if you give a team who are at the top of the league a goal start you're asking for trouble.
"We gave away a stupid goal because of a stupid mistake and I'm really upset to go out of the Cup like that. Port Vale have gone through without having to get out of bed and I'm not happy to say the least.
"We had a go in the second-half, but it was a case of too little, too late. I thought we had scored when the shot (by Chris Turner) was cleared off the line at the end, but it wasn't to be.
"There are no positives because we're out of the Cup. We probably needed the money more than they did, so that adds to the disappointment."