Cav's lost the Gold Jersey at the Tour of Britain after a wind hit end to the third stage.
Lars Boom and Michael Matthews took first and second for Rabobank in Stoke-on-Trent.
A late attack by Sky's Alex Dowsett was chased down by the leading pack in the face of strong winds.
The uphill sprint to the finish turned into a bit of a melee with Dutchman Boom taking the line honours, Matthews second, Geraint Thomas third and Stephen Cummings fourth. Cavendish was fifth.
Boom's win edges Cavendish out at the top of the sprint leaderboard but he still starts Wednesday's Welshpool to Caerphilly stage in the points jersey.
Bikes aplenty planned for Jurby Day 2026
FC Isle of Man on their travels to face Trafford
Cycling: Bostock bound for 2026 Tour of Britain
Cricket: IoM men finish fifth at T20 World Cup qualifiers