Not illegal in itself
but bear in mind that they are legally entitled to exceed the speed limit, and go through red lights whilst on blue lights / sirens, therefore if Mr Plod sees you do it, they may take a dim view ("you were overtaking an ambulance, could you have been speeding sir?"), and if you have an accident, you may have a hard time proving your innocence for that reason (ie, try explaining it to your insurance co, or in a court).
It does depend heavily on the exact circumstances (type of road, traffic levels etc) so its not really possible to provide a "one size fits all" type answer.
I personally would be inclined to sit behind it because a) traffic may behave unpredictably on hearing the siren (I don't want to be the poor sod that needs the ambulance), and you never know where its going; ask yourself "could it be going to attend to a member of my family?"