If I ever work on Yapps again, I should take a look at using this “undocumented” feature of the SRE engine. I’m currently going through a list of regular expressions and trying to match each one. For example, I might have a list r'if', r'\w+', r'\d+'. With this new feature, I can combine all the regular expressions into one with alternation (r'(if)|(\w+)|(\d+)') and then the regexp engine will tell me which one of them matched. This should speed up the scanner greatly.