Don't use a SmallSet for returned attribute inference

Suggested post-commit by David Majnemer on IRC (following-up on a pre-commit
review comment).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@275033 91177308-0d34-0410-b5e6-96231b3b80d8
1 file changed