If I ever work on Yapps again, I should take a look at using this “undocumented” feature of the SRE engine. I’m currently going through a list of regular expressions and trying to match each one. For example, I might have a list
r'\d+'. With this new feature, I can combine all the regular expressions into one with alternation (
r'(if)|(\w+)|(\d+)') and then the regexp engine will tell me which one of them matched. This should speed up the scanner greatly.