It's true that the speed gap isn't double, but Manabu has a fully-haxxed analytical prediction from a feat where he predicted someone's attack fully while having his eyes covered with snow, this someone scales to a speed roughly equal to what Dexter does and Manabu had never fought this person or seen them do an attack ever, he simply predicted it via a large range of stuff which is narratively unrevealed.
Plus, Manabu is a highly skilled fighter, in fact, above the level of a genius, he has 5th dan in karate (requires like 15-20 years at least) and 4th dan in aikido (requires like 10-15 years at least) at the age of just 17-18. This should probably indicate that he isn't really that dumb to actually be hit by Dexter's any of the attacks for the most part.
And Manabu also has a higher stamina advantage, which should help him to restore energy for attacks much faster.
And in my opinion, 1.6x speed gap is still a notable gap, anything above 1.5x is kind of crazy honestly. I personally draw the line at 1.1-1.2x of speed. Dexter still has a notable gap to overcome in their speeds. I would have probably disagreed and said that Manabu shouldn't be able to dodge every attack here if the speeds were roughly equal, but 1.6x is a gap where it helps him a lot, even if he can't readily blitz Dexter.