A few weeks ago I posted how to make Mac OS Spotlight index source code. Mac OS is already indexing my C, C++, ObjC, Java, Fortran, Shell, Python, Perl, Ruby, Pascal, Ada, Javascript, and HTML files, but it doesn't get the other languages and text files I use. Here's the followup post.

Problem to solve: I want to search the contents of *.css and other text files with Spotlight.

In my previous blog post, I went into the Spotlight configuration file and told it to index certain file types like dyn.ah62d4rv4ge80e8drru, which I had determined was *.css. This wasn't satisfying. Instead of telling Spotlight, I wanted a solution at the Mac OS level.

Mac OS determines a "UTI" file type based on various things, including the filename extension. By default everything gets a generic UTI type of public.data. If the UTI type is public.plain-text or a few other types then it'll get indexed by Spotlight. UTI types can inherit from each other. A type like public.source-code inherits from public.plain-text so when Spotlight grabs all plain text files, it also gets the source code. The built-in type hierarchy is described here. Note that Spotlight does not index public.text by default; it only gets public.plain-text.

Solution: Tell Mac OS that *.css files inherit from public.plain-text.

The trouble is that only applications can create an association between file extensions and UTI types. What's the easiest way to make an application? I decided to make an Automator bundle.

Step 1. I created a dummy Automator "Application" called "IndexTextFiles" that has no actions. I saved it to ~/Applications/IndexTextFiles.app.

Step 2. I went into ~/Applications/IndexTextFiles.app/Contents/Info.plist and found the UTImportedTypeDeclarations section (this is apparently the newer way to do things, and replaces CFBundleTypeExtension): 

<key>UTImportedTypeDeclarations</key>
<array/>

and changed it to: 

<key>UTImportedTypeDeclarations</key>
<array>
    <dict>
        <key>UTTypeIdentifier</key>
        <string>public.css</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.w3.org/Style/CSS/</string>
        <key>UTTypeDescription</key>
        <string>CSS Document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>css</string>
            </array>
            <key>public.mime-type</key>
            <string>text/css</string>
        </dict>
    </dict>
</array>

Yay! There now exists an application on my Mac that claims *.css files are public.plain-text. I don't know what the allowed icon names are so I just set it to "document". Step 3. I need to tell Mac OS to re-read that application's Info.plist: 

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -f ~/Applications/IndexTextFiles.app

The *.css files don't get updated with the new information. Step 4. I need to run mdimport: 

mdls -name kMDItemContentTypeTree main.css
# prints dyn.ah62d4rv4ge80e8drru
mdimport main.css
mdls -name kMDItemContentTypeTree main.css
# prints public.css, public.plain-text

Great!!

I can do the same with the other file types I want: *.el *.org *.md *.rst *.yml *.xslt *.scss *.as *.ts *.hx. I used Jonathan Wight's UTI list as a reference where I could, and made up the rest. Step 5. I re-did step 2 with these type declarations: (update: [2016-06-11] since the original blog post, *.css became indexed by default, so I changed the following to add *.scss but not *.css) 

<key>UTImportedTypeDeclarations</key>
<array>
    <dict>
        <key>UTTypeIdentifier</key>
        <string>public.org-document</string>
        <key>UTTypeReferenceURL</key>
        <string>http://orgmode.org/</string>
        <key>UTTypeDescription</key>
        <string>Emacs Org document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>org</string>
            </array>
            <key>public.mime-type</key>
            <string>text/org</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>public.emacs-lisp</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.gnu.org/software/emacs/</string>
        <key>UTTypeDescription</key>
        <string>Emacs Lisp source</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.source-code</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>el</string>
            </array>
            <key>public.mime-type</key>
            <string>text/x-script.elisp</string>
        </dict>
    </dict>

    <!-- http://daringfireball.net/linked/2011/08/05/markdown-uti -->
    <dict>
        <key>UTTypeIdentifier</key>
        <string>net.daringfireball.markdown</string>
        <key>UTTypeReferenceURL</key>
        <string>http://daringfireball.net/projects/markdown/</string>
        <key>UTTypeDescription</key>
        <string>Markdown document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>md</string>
            </array>
            <key>public.mime-type</key>
            <string>text/x-markdown</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>net.sourceforge.docutils.rst</string>
        <key>UTTypeReferenceURL</key>
        <string>http://docutils.sourceforge.net/rst.html</string>
        <key>UTTypeDescription</key>
        <string>reStructuredText document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>rst</string>
            </array>
            <key>public.mime-type</key>
            <string>text/x-rst</string>
        </dict>
    </dict>
    
    <dict>
        <key>UTTypeIdentifier</key>
        <string>org.yaml.yaml</string>
        <key>UTTypeReferenceURL</key>
        <string>http://yaml.org</string>
        <key>UTTypeDescription</key>
        <string>YAML Document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.plain-text</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>yaml</string>
                <string>yml</string>
            </array>
            <key>public.mime-type</key>
            <string>text/x-yaml</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>org.w3.xsl</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.w3.org/Style/XSL/</string>
        <key>UTTypeDescription</key>
        <string>XSL Stylesheet</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.xml</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>xsl</string>
                <string>xslt</string>
            </array>
            <key>public.mime-type</key>
            <string>application/xslt+xml</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>public.scss</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.w3.org/Style/CSS/</string>
        <key>UTTypeDescription</key>
        <string>SCSS Document</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.css</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>scss</string>
            </array>
            <key>public.mime-type</key>
            <string>text/css</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>com.adobe.actionscript</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.adobe.com/devnet/actionscript.html</string>
        <key>UTTypeDescription</key>
        <string>ActionScript Source Code</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.source-code</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>as</string>
            </array>
            <key>public.mime-type</key>
            <string>application/ecmascript</string>
        </dict>
    </dict>
    
    <dict>
        <key>UTTypeIdentifier</key>
        <string>com.microsoft.typescript</string>
        <key>UTTypeReferenceURL</key>
        <string>http://typescript.codeplex.com/</string>
        <key>UTTypeDescription</key>
        <string>TypeScript Source Code</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.source-code</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>ts</string>
            </array>
            <key>public.mime-type</key>
            <string>application/ecmascript</string>
        </dict>
    </dict>

    <dict>
        <key>UTTypeIdentifier</key>
        <string>org.haxe</string>
        <key>UTTypeReferenceURL</key>
        <string>http://haxe.org/</string>
        <key>UTTypeDescription</key>
        <string>Haxe Source Code</string>
        <key>UTTypeIconFile</key>
        <string>document.icns</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.source-code</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>hx</string>
            </array>
            <key>public.mime-type</key>
            <string>application/ecmascript</string>
        </dict>
    </dict>

</array>

Then I re-did steps 3 and 4: I had to run lsregister again and then ran mdimport on my home directory. Now most of these files are treated as text. I can use Spotlight and mdfind with them.This also simplifies my Emacs setup to find files anywhere. The exceptions are *.as and *.ts files, which Mac already has registered. For those, I will needed the previous hacks to Spotlight's rich text indexer.

Labels: ,

– Amit – Friday, June 06, 2014

7 comments:

Amit wrote at Friday, June 6, 2014 at 4:09:00 PM PDT

I still have a conflict with .ts and .as files that I don't know how to resolve.

Toby wrote at Tuesday, July 26, 2016 at 10:34:00 AM PDT

I know this is an older post, but this helped me get files indexed properly! I use backed up Cisco configs (*.ios) and this works perfectly.

Thanks &
Cheers,

Toby

Amit wrote at Wednesday, July 27, 2016 at 9:40:00 AM PDT

Glad to hear it Toby! I still use this configuration and am happy with it. Every once in a while, something fails to index, and I use a command line like this to repair it:

mdfind -onlyin $HOME "(kMDItemFSName == '*.ios') && (kMDItemContentTypeTree != 'public.text')" | xargs -n 1 mdimport

Anonymous wrote at Wednesday, August 24, 2016 at 4:11:00 PM PDT

These instructions were very useful, thanks!

I had to make a few tweaks to make this work properly for typescript

1. I disabled System Integrity Protection and removed the built in definition for .ts files from /System/Library/CoreServices/CoreTypes.bundle/Contents/Info.plist (after using plutil to convert it from binary to text). This caused mdls to correctly detect these files as typescript.
2. I modified the UTI definitions listed above to give typescript a type of text/ecmascript instead of application/ecmascript. This caused the RichText spotlight importer to start considering their contents.

Amit wrote at Thursday, August 25, 2016 at 2:35:00 PM PDT

TheoSpears — thanks! I didn't know how to solve the built-in definitions.

Unknown wrote at Monday, December 11, 2017 at 4:28:00 AM PST

Can this be done without disabling System Integrity Protection?

Amit wrote at Monday, December 11, 2017 at 8:03:00 AM PST

@Kees the dummy application can be anywhere and can be added without disabling SIP. It's the same mechanisms Mac OS wants apps to use to make documents searchable by Spotlight.

Post a Comment

Subscribe to: Post Comments (Atom)