When I walked with the randonneurs in CM, the leader would say, "on y va" after a stop for a drink of water and/or a pee as a way of rounding everyone up and a signal to get ready to move off.
I also noticed the one or two occasions when I have watched a table tennis match when OH was playing, that the arbitre (is that spelling right?) would say, "on y va" which I translate as OK, if you are both ready, we'll start the match.
Anyway, "on y va" just slips off the tongue, so I tend to say that.
If I'm just ready to leave and no one seems to want to move, I say "allez, je m'en vais!" which I feel is not too abrupt or, if I am the one holding everyone else up, I think it's a way to reassure people that I shan't be lingering!
N'allez pas trop vite - Proust