If you are undecided and want to try living and working in France before burning your bridges I suggest you look into whether or not you can work under what was the E101 form. This allows you to continue being taxed, and to pay NI, in Britain but be accepted into the French systems on a temporary basis - and completely legally despite some of the comments above. I believe the former E101 is now one of the various A1 forms.
You have to meet certain conditions, but it sounds as if you should be OK. This gives you a year, and if all the authorities agree you can renew for another year. By then you should be able to decide whether to stay, and register your own business in France, or you may discover working in France isn't really for you.
intentionally blank signature