PM says he suggested political rival join him in Washington because 'we should get as broad a consensus as possible'