A few weeks ago I posted how to make Mac OS Spotlight index source code. Mac OS is already indexing my C, C++, ObjC, Java, Fortran, Shell, Python, Perl, Ruby, Pascal, Ada, Javascript, and HTML files, but it doesn't get the other languages and text files I use. Here's the followup post.

Problem to solve: I want to search the contents of *.css and other text files with Spotlight.

In my previous blog post, I went into the Spotlight configuration file and told it to index certain file types like dyn.ah62d4rv4ge80e8drru, which I had determined was *.css. This wasn't satisfying. Instead of telling Spotlight, I wanted a solution at the Mac OS level.

Mac OS determines a "UTI" file type based on various things, including the filename extension. By default everything gets a generic UTI type of public.data. If the UTI type is public.plain-text or a few other types then it'll get indexed by Spotlight. UTI types can inherit from each other. A type like public.source-code inherits from public.plain-text so when Spotlight grabs all plain text files, it also gets the source code. The built-in type hierarchy is described here. Note that Spotlight does not index public.text by default; it only gets public.plain-text.

Solution: Tell Mac OS that *.css files inherit from public.plain-text.

The trouble is that only applications can create an association between file extensions and UTI types. What's the easiest way to make an application? I decided to make an Automator bundle.

Step 1. I created a dummy Automator "Application" called "IndexTextFiles" that has no actions. I saved it to ~/Applications/IndexTextFiles.app.

Step 2. I went into ~/Applications/IndexTextFiles.app/Contents/Info.plist and found the UTImportedTypeDeclarations section (this is apparently the newer way to do things, and replaces CFBundleTypeExtension):

<key>UTImportedTypeDeclarations</key>
<array/>

and changed it to:

<key>UTImportedTypeDeclarations</key>
<array>
    <dict>
        <key>UTTypeIdentifier</key>
        <string>public.css</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.w3.org/Style/CSS/</string>
        <key>UTTypeDescription</key>
        <string>CSS Document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>css</string>
            </array>
            <key>public.mime-type</key>
            <string>text/css</string>
        </dict>
    </dict>
</array>

Yay! There now exists an application on my Mac that claims *.css files are public.plain-text. I don't know what the allowed icon names are so I just set it to "document". Step 3. I need to tell Mac OS to re-read that application's Info.plist:

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -f ~/Applications/IndexTextFiles.app

The *.css files don't get updated with the new information. Step 4. I need to run mdimport:

mdls -name kMDItemContentTypeTree main.css
# prints dyn.ah62d4rv4ge80e8drru
mdimport main.css
mdls -name kMDItemContentTypeTree main.css
# prints public.css, public.plain-text

Great!!

I can do the same with the other file types I want: *.el *.org *.md *.rst *.yml *.xslt *.scss *.as *.ts *.hx. I used Jonathan Wight's UTI list as a reference where I could, and made up the rest. Step 5. I re-did step 2 with these type declarations: (update: [2016-06-11] since the original blog post, *.css became indexed by default, so I changed the following to add *.scss but not *.css)

<key>UTImportedTypeDeclarations</key>
<array>
    <dict>
        <key>UTTypeIdentifier</key>
        <string>public.org-document</string>
        <key>UTTypeReferenceURL</key>
        <string>http://orgmode.org/</string>
        <key>UTTypeDescription</key>
        <string>Emacs Org document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>org</string>
            </array>
            <key>public.mime-type</key>
            <string>text/org</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>public.emacs-lisp</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.gnu.org/software/emacs/</string>
        <key>UTTypeDescription</key>
        <string>Emacs Lisp source</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.source-code</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>el</string>
            </array>
            <key>public.mime-type</key>
            <string>text/x-script.elisp</string>
        </dict>
    </dict>

    <!-- http://daringfireball.net/linked/2011/08/05/markdown-uti -->
    <dict>
        <key>UTTypeIdentifier</key>
        <string>net.daringfireball.markdown</string>
        <key>UTTypeReferenceURL</key>
        <string>http://daringfireball.net/projects/markdown/</string>
        <key>UTTypeDescription</key>
        <string>Markdown document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>md</string>
            </array>
            <key>public.mime-type</key>
            <string>text/x-markdown</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>net.sourceforge.docutils.rst</string>
        <key>UTTypeReferenceURL</key>
        <string>http://docutils.sourceforge.net/rst.html</string>
        <key>UTTypeDescription</key>
        <string>reStructuredText document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>rst</string>
            </array>
            <key>public.mime-type</key>
            <string>text/x-rst</string>
        </dict>
    </dict>
    
    <dict>
        <key>UTTypeIdentifier</key>
        <string>org.yaml.yaml</string>
        <key>UTTypeReferenceURL</key>
        <string>http://yaml.org</string>
        <key>UTTypeDescription</key>
        <string>YAML Document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>yaml</string>
                <string>yml</string>
            </array>
            <key>public.mime-type</key>
            <string>text/x-yaml</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>org.w3.xsl</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.w3.org/Style/XSL/</string>
        <key>UTTypeDescription</key>
        <string>XSL Stylesheet</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.xml</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>xsl</string>
                <string>xslt</string>
            </array>
            <key>public.mime-type</key>
            <string>application/xslt+xml</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>public.scss</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.w3.org/Style/CSS/</string>
        <key>UTTypeDescription</key>
        <string>SCSS Document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.css</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>scss</string>
            </array>
            <key>public.mime-type</key>
            <string>text/css</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>com.adobe.actionscript</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.adobe.com/devnet/actionscript.html</string>
        <key>UTTypeDescription</key>
        <string>ActionScript Source Code</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.source-code</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>as</string>
            </array>
            <key>public.mime-type</key>
            <string>application/ecmascript</string>
        </dict>
    </dict>
    
    <dict>
        <key>UTTypeIdentifier</key>
        <string>com.microsoft.typescript</string>
        <key>UTTypeReferenceURL</key>
        <string>http://typescript.codeplex.com/</string>
        <key>UTTypeDescription</key>
        <string>TypeScript Source Code</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.source-code</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>ts</string>
            </array>
            <key>public.mime-type</key>
            <string>application/ecmascript</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>org.haxe</string>
        <key>UTTypeReferenceURL</key>
        <string>http://haxe.org/</string>
        <key>UTTypeDescription</key>
        <string>Haxe Source Code</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.source-code</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>hx</string>
            </array>
            <key>public.mime-type</key>
            <string>application/ecmascript</string>
        </dict>
    </dict>

</array>

Then I re-did steps 3 and 4: I had to run lsregister again and then ran mdimport on my home directory. Now most of these files are treated as text. I can use Spotlight and mdfind with them.This also simplifies my Emacs setup to find files anywhere. The exceptions are *.as and *.ts files, which Mac already has registered. For those, I will needed the previous hacks to Spotlight's rich text indexer.

Labels: ,

7 comments:

Amit wrote at Friday, June 6, 2014 at 4:09:00 PM PDT

I still have a conflict with .ts and .as files that I don't know how to resolve.

Toby wrote at Tuesday, July 26, 2016 at 10:34:00 AM PDT

I know this is an older post, but this helped me get files indexed properly! I use backed up Cisco configs (*.ios) and this works perfectly.

Thanks &
Cheers,

Toby

Amit wrote at Wednesday, July 27, 2016 at 9:40:00 AM PDT

Glad to hear it Toby! I still use this configuration and am happy with it. Every once in a while, something fails to index, and I use a command line like this to repair it:

mdfind -onlyin $HOME "(kMDItemFSName == '*.ios') && (kMDItemContentTypeTree != 'public.text')" | xargs -n 1 mdimport

Unknown wrote at Wednesday, August 24, 2016 at 4:11:00 PM PDT

These instructions were very useful, thanks!

I had to make a few tweaks to make this work properly for typescript

1. I disabled System Integrity Protection and removed the built in definition for .ts files from /System/Library/CoreServices/CoreTypes.bundle/Contents/Info.plist (after using plutil to convert it from binary to text). This caused mdls to correctly detect these files as typescript.
2. I modified the UTI definitions listed above to give typescript a type of text/ecmascript instead of application/ecmascript. This caused the RichText spotlight importer to start considering their contents.

Amit wrote at Thursday, August 25, 2016 at 2:35:00 PM PDT

TheoSpears — thanks! I didn't know how to solve the built-in definitions.

Unknown wrote at Monday, December 11, 2017 at 4:28:00 AM PST

Can this be done without disabling System Integrity Protection?

Amit wrote at Monday, December 11, 2017 at 8:03:00 AM PST

@Kees the dummy application can be anywhere and can be added without disabling SIP. It's the same mechanisms Mac OS wants apps to use to make documents searchable by Spotlight.