The only really easy way to leave a coach is to actually leave town, and sometimes I think skating families actually consider this in order to avoid telling the coach there needs to be a change.
In the US skating families are hobbled by what I call the gag rule--it's very difficult to seek a new coach without the old coach finding out, because they're required by PSA rules to rat you out. If you're not a competitive skater the way around this is to quit for three months--just do classes and practice; as far as I'm concerned this cleans the slate. You no longer have a coach and can talk to anyone you want without them having to talk to the other guy. When you quit, make sure you have a final bill in writing, and that the coach physically signs it off as paid when you pay it. This ends your obligation and overrides the gag rule. You now have no coach. Sigh.
If you're competitive it's a little trickier because you can't be without a coach for that long. Of course, you can always just be honest and upfront and tell the coach that you want to change and that you'll be talking to other coaches. Let him know why-- we don't feel the relationship is working/the commute is too long/rink politics are making us uncomfortable/whatever.
Then the current coach can prove you right by being a jerk and making the skater miserable, or by refusing to teach you anymore, or s/he can be a mensch and support your search.
Since the gag rule is so stupid, I cannot believe coaches really follow it. I really think there have to be an awful lot of families out there talking to coaches and asking them to please not tell the current coach until they are ready to switch. Frankly, as far as I'm concerned this is the logical way to do this--talk to any coach you want to and prohibit them from telling your coach--after all talking to a coach is not the same thing as hiring a new coach. It's really no one's business but your own until you actually make the change, at which point you really do have to tell everyone.
It's never easy to "fire" someone, and you shouldn't put it like that; unless the coach has engaged in criminal or abusive behavior (rare) there is no need to burn the bridge. A coach being a jerk is not enough of a reason to destroy a relationship.
So, back to the original question: When do you tell the coach you're leaving?
The second you think it's going to get out anyway.