No. The latency settings for Dante Virtual Soundcard can be set at different parameters than the rest of your network and will not affect the playout to other devices. For example, the rest of the network could be set at 1ms latency for live audio while Dante Virtual Soundcard remains set for a higher value for recording.
More info here: Dante Latency