I guess it's okay as long as it doesn't happen all the time and the scene in its entirety is still understandable.
You could also mark it as a flashback by writing it in italics. That is done very often.