Harry and Meghan looked unusually casual during their latest royal outing