In order to use special characters in variable definitions you have to include the actual character in the BookVars file rather than the special character sequence that you would use in a FrameMaker dialog.
For example, when you enter a “\m” in the internal editor, you'll see that it actually turns into an emdash after you save, close, and re-edit the BookVars file. As long as you’re using the internal editor, thiss conversion will work fine. However, if you use an external editor, you need to create a temporary file using the internal editor in order to create the actual character that will exist in the file. After creating the temporary file using the internal editor, you can open that file in an external editor, then copy and paste the character to other files as needed. Note that this character may not look like the character you’re trying to use.