"I mean, we were starving after Game of Thrones," he tells InStyle. "I couldn't get work. It's very challenging when you have babies and you're completely in debt."