Orbit separation and invariant theory : degree bounds and algorithms