The correct answer is man. Adjective clauses are relative clauses that come immediately after the noun they modify and are introduced by a relative pronoun (that, which, who, whose, etc.) which connects the clause to the noun being described. In the example, the adjective clause whose portrait hangs comes after the noun man; consequently, it modifies the word man.
