Although residents of NYC's Inspir Carnegie Hill mostly liked the performer's solo Broadway show 'Just for Us,' they were less impressed when he showed up at the facility in person