Add .rc scripts tokenizer.

This extends the shell of llvm-rc tool with the ability of tokenization
of the input files. Currently, ASCII and ASCII-compatible UTF-8 files
are supported.

Thanks to Nico Weber (thakis) for his original work in this area.

Differential Revision: https://reviews.llvm.org/D35957

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@310621 91177308-0d34-0410-b5e6-96231b3b80d8
7 files changed