Tony Goldwyn says it's “especially fun” to see Scandal’s Fitz be “completely out to sea”