Amit's Thoughts: howto

From Amit’s Thoughts

Using an LLM to query my notes #

I occasionally play with LLMs (large language models). Last time I had tried Simon Willison's llm software to search over my notes. Today I tried amaiya/onprem to ask questions about my notes. The documentation is pretty good but I made a few changes for my configuration, so I decided to write them down here. I'm running these on my Mac M1 and not on my Linux or Windows machines.

First step, get the thing installed:

cd ~/Projects/machine-learning/
mkdir onprem
cd onprem

# need to use python 3.9 through 3.11 for torch support; 3.12 doesn't support it yet
# check the version I have installed
python3 --version
# if python3 isn't a reasonable version, then install or choose a different python
# before doing the next step

python3.11 -m venv venv
source venv/bin/activate

# install onprem itself, which also installs torch
pip install onprem
mkdir data

Labels: howto

Mac keyboard with hidutil #

[Update: 13-Apr-2023 with some of the background context and some things I learned from hackernews]

[Update: 18-Dec-2023] This hidutil stopped working for me in MacOS 13.6 and 14.2. Also see other people having problems too.

[Update: 28-Jan-2024] Some people say hidutil works again in Mac OS 14.3.

On my Mac, I've used KeyRemap4Macbook, Karabiner, Karabiner Elements, and FunctionFlip for some of my key remapping needs. The main things I want:

1. On the laptop keyboard, some media keys should be function keys but other media keys should stay media keys. When using fn I want to get the other version of the key.
2. On external keyboards, make the function keys act like the laptop keyboard.
3. On external keyboards, the numpad should act like "numlock off". For example, 4 should be Left Arrow.
4. On external keyboards, the Windows key should act as Option, and the Alt key should act as Command. This can usually be set in the System Preferences. (Except it doesn't work on one of my keyboards)
5. I also use those same external keyboards with my Windows and Linux machines, so I prefer to make these changes through software rather than firmware.

Labels: howto, mac

Firefox 89 tab appearance #

Firefox 89 has a new appearance, removing features (ugh) and annoying some people while delighting others. It can be disabled by setting browser.proton.enabled to false:

I mostly like the new design though, but I strongly dislike some elements, especially the tabs:

Labels: firefox, howto

OpenTTD on Windows 10 + Steam #

Transport Tycoon / OpenTTD is my favorite game of all time, and I wanted to play it on Steam. I kept getting "file not writable" error. I tracked it down to the Windows security system blocking it. To unblock:

1. Start →
2. Settings →
3. Update & Security →
4. Windows security →
5. Open Windows Security →
6. Virus & threat protection →
7. Manage settings →
8. Controlled folder access →
9. Manage Controlled folder access →
10. Allow an app through Controlled folder access →
11. Add an allowed app →
12. openttd.exe

I am not a fan of making exceptions like this but I didn't find anything else that worked. If you have an alternate method of solving the permissions error, please let me know.

Labels: howto

Building Emscripten on Apple ARM M1 #

Update: [2021-01-24] The workarounds in this blog post are now obsolete, as openjdk, yuicompressor, and emscripten are now available in ARM homebrew.

On Mac, Emscripten is available in the Intel version of Homebrew, but not the ARM version of Homebrew. I use Emscripten rarely enough that this doesn't bother me. However, I thought I'd try compiling it on the ARM version.

I couldn't get openjdk working in ARM homebrew, but I was able to get emscripten installed in ARM homebrew by using the Intel/Rosetta version of openjdk:

Labels: howto, mac

Building Emacs 27 on Apple ARM M1 #

On Mac, I sometimes run a prebuilt binary and sometimes compile my own. For Apple M1 (ARM) I started by using the x86 binaries from emacsformacosx.com but I wanted to try compiling a native ARM version too. I saw that the work branch of Mitsuharu Yamamoto's Mac port has M1 patches applied, so I decided to try it.

Labels: emacs, howto, mac

Emacs Org mode and KaTeX #

Emacs Org mode can handle LaTeX math, and exports it to HTML using MathJax. On my pages I have been using KaTeX instead of MathJax. It loads faster but doesn't support as many features. Org mode doesn't have direct support for KaTeX but for the simple things I do on my pages, I can redirect MathJax to KaTeX using KaTeX autorender:

(setq org-html-mathjax-template
"<link rel=\"stylesheet\" href=\"https://cdn.jsdelivr.net/npm/katex@0.10.0/dist/katex.min.css\" integrity=\"sha384-9eLZqc9ds8eNjO3TmqPeYcDj8n+Qfa4nuSiGYa6DjLNcv9BtN69ZIulL9+8CqC9Y\" crossorigin=\"anonymous\"/>
<script defer=\"defer\" src=\"https://cdn.jsdelivr.net/npm/katex@0.10.0/dist/katex.min.js\" integrity=\"sha384-K3vbOmF2BtaVai+Qk37uypf7VrgBubhQreNQe9aGsz9lB63dIFiQVlJbr92dw2Lx\" crossorigin=\"anonymous\"></script>
<script defer=\"defer\" src=\"https://cdn.jsdelivr.net/npm/katex@0.10.0/dist/contrib/auto-render.min.js\" integrity=\"sha384-kmZOZB5ObwgQnS/DuDg6TScgOiWWBiVt0plIRkZCmE6rDZGrEOQeHM5PcHi+nyqe\" crossorigin=\"anonymous\" onload=\"renderMathInElement(document.body);\"></script>")

(it looks ugly but it's straight from the KaTeX autorender page, but with quotes backslashed)

This seems to work on the small tests I've done so far.

Labels: emacs, howto

Emacs: prettier tabbar #

Back in 2007 I posted that I was going to try buffer tabs for Emacs. At the time I wasn't sure if I'd like it. People told me that there's no point in displaying them visually like in other editors, that Emacs had better ways of switching buffers. I ended up abandoning Tab Bar mode. But some time later, I started using it again, and I found the new version worked really well for my needs.

This post is mostly about the visual customization I use. I make the current tab blue, the other tabs gray, and the background dark gray. I use powerline to make the tabs look more like tabs:

Labels: emacs, howto

Emacs: find files anywhere, updated for 2017 #

Back in 2016 I posted my emacs setup for finding files globally. It unifies find-file and switch-to-buffer, and also lets me find files without switching folders. I've been improving it since then and wanted to post an update.

The biggest change is speed. I profiled the code and found two bottlenecks in the construction of the global file list. The list is roughly 15000 elements, so I precomputed as much of that as I could, and the startup time went down from 0.8 seconds to 0.1 second. It feels instant now.

Labels: emacs, howto

Building Mac OS X Emacs 26 #

On 11 Oct 2017 Emacs 26.0.90 pretest was released. For Mac, there's a build on emacsformacosx.com. I have been using the "Mac port" from Mitsuharu Yamamoto because it feels smoother and faster on my system, and I decided to try his development branch. Here's what I ran:

brew install gnutls texinfo libxml2
cd ~/Projects/src
git clone --branch work --depth 1 https://bitbucket.org/mituharu/emacs-mac.git
cd emacs-mac
./autogen.sh
export PATH=/usr/local/opt/texinfo/bin/:$PATH
export PKG_CONFIG_PATH=/usr/local/opt/libxml2/lib/pkgconfig
./configure --with-mac \
     --prefix=/PersonalApplications/Emacs.app/Contents/MacOS \
     --enable-mac-app=/PersonalApplications
make
make install

It runs nicely!

A minor note: the byte-compiled .elc files compiled under Emacs 26 are not compatible with Emacs 25, so if you go back to an older version, you'll need to recompile the .elc files.

Labels: emacs, howto, mac

Panorama to square images for Instagram #

Instagram recently added a feature to post up to 10 photos side by side. I saw someone cleverly used it to post a single panoramic photo. You split the panorama into square photos and then post them all at once to Instagram, which then arranges them side by side. Here's a quick & dirty script I wrote to split up my own panoramas into square photos. It uses a few Mac commands to get the size of the image, and ImageMagick to break up the images into smaller ones:

#!/bin/bash

if [ -z "$1" ]; then
    echo "Usage: $0 filename.jpg"
    exit
fi

filename="$1"

# Original image size
width=$(mdls -raw -name kMDItemPixelWidth "$filename")
height=$(mdls -raw -name kMDItemPixelHeight "$filename")

# Each of the square pieces will be 1/Nth the width
N=$[$width / $height]
if [ $N -gt 10 ]; then N=10; fi  # maximum 10 allowed by instagram

size=$[$width / N]
if [ $height -lt $size ]; then size=$height; fi

    
# Use the center of the x,y range
x_offset=$[$[$width - $size * N] / 2]
y_offset=$[$[$height - $size] / 2]


echo  "$filename : ${width}x${height}, $N parts"

if [ $x_offset -gt 0 ]; then
    echo "Wide image; truncating $[2 * $x_offset] pixels"
else
    x_offset=0
fi

if [ $y_offset -gt 0 ]; then
    echo "Tall image; truncating $[2 * $y_offset] pixels"
else
    y_offset=0
fi

for part in $(seq $[$N - 1] -1 0); do
    crop="${size}x${size}+$[${x_offset} + ${size}*${part}]+${y_offset}"
    echo "  part $part : $crop"
    convert "$filename" -crop "$crop" $part.jpg
done

I'm sure there are lots of things I could do better but I just wanted to get something working quickly.

Labels: howto

Emacs spaceline mode line #

Back in 2011 I posted my Emacs mode line configuration. At the time I was thinking it'd be nice to have a more modular way to define the mode line. A few weeks ago I rewrote my mode line configuration to use Spaceline, the mode line library used in Spacemacs. I made it match my tabbar:

...

Labels: emacs, howto

Emacs Org mode 9 #

Org mode 9 was just released, and it changed the syntax for export blocks. I need to change:

#+begin_html
…
#+end_html

to

#+begin_export html
…
#+end_export

The Org mode changes file includes some elisp to change this. However, I couldn't get it to work, and I also wanted to change all my files, not run this elisp on one file at a time. Here are the commands I ran:

ack -l --type-add=org:ext:org --org '#.begin_html' | xargs -n 1 perl -pi -e 's/#\+end_html/#+end_export/'
ack -l --type-add=org:ext:org --org '#.begin_html' | xargs -n 1 perl -pi -e 's/#\+begin_html/#+begin_export html/'

I only needed this for html, but you may need to extend this for other types you use. See what else you use with:

ack --type-add=org:ext:org --org '#.begin_(html|ascii|latex|odt|markdown|md|org|man|beamer|texinfo|groff|koma-letter)'

Labels: emacs, howto

My OS X Karabiner setup #

On Mac OS X, there's a fantastic tool called Karabiner (previously KeyRemap4MacBook) that lets you control how your keyboard and mouse work — things like keyboard repeat rate, mouse acceleration, arrow keys, function keys, control/escape/shift/option/fn, adding keys to enter unicode characters, and many other things. I use it primarily for remapping the function keys. On my laptop, I want F1,F2 to control brightness, F3,F4,F5,F6,F7,F8,F9 to be function keys, and F10,F11,F12 to control volume. Mac OS X offers in the System Preferences a choice of having them all be function keys or all be special functions. I want five to be special functions and seven to be function keys. Karabiner lets me do that.

Karabiner's standard settings dialog lets me swap just some of these keys. First, I told Mac OS X to make them all function keys. Then in Karabiner, I selected

• Change F1..F19 Key & Functional Key
  • Change Functional Key
    • Fn+Functional Keys to F1..F12
      • [X] Fn+Brightness Adjust to F1,F2
      • [X] Fn+Speaker Controls to F10,F11,F12
  • Change F1..F19 Key
    • F1..F12 to Functional Keys
      • [X] F1,F2 to Brightness Adjust
      • [X] F10,F11,F12 to Speaker Controls

Great! It now behaves how I want.

Except… I want more! Once I got an external keyboard, I wanted the function keys to behave differently. The external keyboard has its own volume keys, so I don't want them swapped there. I use it with an external display, so the brightness keys aren't useful to swap.

Karabiner offers an XML configuration file when the standard settings dialog isn't enough. Reading the XML file format documentation, I saw that I can restrict certain settings to only run in some apps or some devices. I came up with this equivalent for the above settings (and then later realized all the built-in settings have XML files like this one):

  <item>
    <name>Swap F1,F2 with Brightness keys</name>
    <identifier>amitp.brightness</identifier>
    <autogen>__KeyToKey__ KeyCode::F1, ConsumerKeyCode::BRIGHTNESS_DOWN</autogen>
    <autogen>__KeyToKey__ KeyCode::F2, ConsumerKeyCode::BRIGHTNESS_UP</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::BRIGHTNESS_DOWN, KeyCode::F1</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::BRIGHTNESS_UP, KeyCode::F2</autogen>
  </item>
  <item>
    <name>Swap F10,F11,F12 with Volume keys</name>
    <identifier>amitp.volume</identifier>
    <autogen>__KeyToKey__ KeyCode::F10, ConsumerKeyCode::VOLUME_MUTE</autogen>
    <autogen>__KeyToKey__ KeyCode::F11, ConsumerKeyCode::VOLUME_DOWN</autogen>
    <autogen>__KeyToKey__ KeyCode::F12, ConsumerKeyCode::VOLUME_UP</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::VOLUME_MUTE, KeyCode::F10</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::VOLUME_DOWN, KeyCode::F11</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::VOLUME_UP, KeyCode::F12</autogen>
  </item>

However, I want to swap these keys only on the laptop keyboard. I used Karabiner's Event Viewer helper application to figure out that my laptop keyboard's device id was 0x0259. I could then modify the settings to only apply to my laptop keyboard:

  <deviceproductdef>
    <productname>Apple_laptop</productname>
    <productid>0x0259</productid>
  </deviceproductdef>
  <item>
    <name>Swap F1,F2 with Brightness keys</name>
    <appendix>For laptop keyboard only</appendix>
    <identifier>amitp.brightness</identifier>
    <device_only>DeviceVendor::APPLE_COMPUTER, DeviceProduct::Apple_laptop</device_only>
    <autogen>__KeyToKey__ KeyCode::F1, ConsumerKeyCode::BRIGHTNESS_DOWN</autogen>
    <autogen>__KeyToKey__ KeyCode::F2, ConsumerKeyCode::BRIGHTNESS_UP</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::BRIGHTNESS_DOWN, KeyCode::F1</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::BRIGHTNESS_UP, KeyCode::F2</autogen>
  </item>
  <item>
    <name>Swap F10,F11,F12 with Volume keys</name>
    <identifier>amitp.volume</identifier>
    <appendix>For laptop keyboard only</appendix>
    <device_only>DeviceVendor::APPLE_COMPUTER, DeviceProduct::Apple_laptop</device_only>
    <autogen>__KeyToKey__ KeyCode::F10, ConsumerKeyCode::VOLUME_MUTE</autogen>
    <autogen>__KeyToKey__ KeyCode::F11, ConsumerKeyCode::VOLUME_DOWN</autogen>
    <autogen>__KeyToKey__ KeyCode::F12, ConsumerKeyCode::VOLUME_UP</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::VOLUME_MUTE, KeyCode::F10</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::VOLUME_DOWN, KeyCode::F11</autogen>
    <autogen>__KeyToKey__ ConsumerKeyCode::VOLUME_UP, KeyCode::F12</autogen>
  </item>

It works great!

I also solved another annoyance. My external keyboard always sends number keys from the number pad, and I'd rather send arrow keys. Karabiner can do that too. I had to find the device vendor and product id from the event viewer, and then I used these settings:

  <devicevendordef>
    <vendorname>KBTalking</vendorname>
    <vendorid>0x099a</vendorid>
  </devicevendordef>

  <deviceproductdef>
    <productname>Bluetooth_keyboard</productname>
    <productid>0x0100</productid>
  </deviceproductdef>
  
  <item>
    <name>Amit's Keypad</name>
    <appendix>Amit's keypad settings</appendix>
    <identifier>amitp.keypad</identifier>
    <device_only>DeviceVendor::KBTalking, DeviceProduct::Bluetooth_keyboard</device_only>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_5, KeyCode::F19</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_2, KeyCode::CURSOR_DOWN</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_4, KeyCode::CURSOR_LEFT</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_6, KeyCode::CURSOR_RIGHT</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_8, KeyCode::CURSOR_UP</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_1, KeyCode::END</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_3, KeyCode::PAGEDOWN</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_7, KeyCode::HOME</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_9, KeyCode::PAGEUP</autogen>
    <autogen>__KeyToKey__ KeyCode::KEYPAD_DOT, KeyCode::FORWARD_DELETE</autogen>
  </item>

These small things have made my computing more pleasant. There's so much more that Karabiner can do, but I haven't explored most of its capabilities.

P.S. If you find Karabiner useful, consider donating.

Labels: howto, mac

Emacs: find files anywhere, updated for 2016 #

Update: [2017-12-29] See the update for 2017

Back in 2014 I posted my emacs setup for finding files globally. It acts like switch-to-buffer, if all the files on my system were already open in buffers. I can type in a substring of any filename I'm likely to work on and switch to that buffer. Since them, helm has changed its interface, and I've updated my code, adding a few extra quality-of-life features.

I use the same hourly cron job to take inventory of all files I am likely to edit, putting recently modified files near the top, as they are most likely to be the best match. I save this list to ~/.global-file-list.txt.

What's changed is how I set up helm. In 2014 I was using helm-recentf and temporarily binding the recentf file list to my own list. This was a quick & dirty way to do it, and it worked, except when I opened a file, it didn't get put onto the recentf list, since I had let-bound that list to something else. Oops. I learned more helm (by reading the source code and John Kitchin's blog posts) and now construct my own helm source instead of abusing helm-recentf:

(defun amitp/helm-all-files ()
  "Global filename match, over all files I typically open"
  (interactive)
  (helm
   :sources '(amitp/helm-source-my-files helm-source-locate)
   :buffer "*helm all files*"))

Helm will look at this source to get the filename list:

(defvar amitp/helm-source-my-files
  (helm-build-sync-source "My files"
    :candidates #'amitp/helm-global-file-list
    :filtered-candidate-transformer #'amitp/helm-filter-my-files
    :keymap helm-generic-files-map
    :action 'helm-type-file-actions))

This source tells helm to call amitp/helm-global-file-list. This function returns the most recently modified files at the top, then the currently open files, then files in the current folder, then recently opened files, then the global file list. (I'm not happy with this order and am still experimenting.) Some filenames will be in more than one list so I eliminate duplicates.

(defun amitp/helm-global-file-list ()
  "Files to list in amitp/helm-all-files"
  ;; delete-dups much faster than cl-remove-duplicates
  (delete-dups
   (mapcar 'abbreviate-file-name
           (append
            (read-file-into-lines "~/.recent-file-list.txt")
            (amitp/buffer-file-names)
            (helm-skip-boring-files
             (directory-files default-directory t))
            recentf-list
            amitp/global-file-list))))

I also use a filter to remove some filenames from the list. Why? Normally I want to show only the “source” and not the “compiled” version of something. For example I want to show .el files but not .elc files. I want to show .c files but not .o files. I can hide these with helm-boring-file-regexp-list. However, for my web pages, some of them are directly written as .html files (*.html should be included) but others are written in Org mode or markdown or something else (*.html should be excluded). To decide whether a file ending in .html is source or not, I need to look at the other filenames in the list. I use a “filtered candidate transformer” (see the documentation for helm-source) to take the currently matching filenames and filter them further. If the filenames contain both $something.org and $something.html then I know that the html is not a source file, so I hide it.

(defun dominated-by-filename (regexp replacement filename candidates)
  "True if FILENAME with REGEXP replaced by REPLACEMENT is already in CANDIDATES"
  (let ((new-filename (replace-regexp-in-string regexp replacement filename)))
    (and (not (equal new-filename filename))
         (member new-filename candidates))))

(defun amitp/helm-filter-my-files (candidates _source)
  "Ignore a build target if a build source exists in the candidates"
  (cl-loop for filename in candidates
      unless
      (or
       (dominated-by-filename "\\.html$" ".org" filename candidates)
       (dominated-by-filename "\\.html$" ".md" filename candidates))
      ;; (I have more rules but you get the idea)
      collect filename))

I don't currently colorize the output. I've considered coloring by type (source code, prose, build file, etc.) and by origin (recentf, current folder, open buffer, global list) but neither of these seems particularly appealing. I'll continue to experiment.

The last change since 2014 is that my global file list doesn't have everything, and it is annoyingly missing any new files I've created in the past hour. In 2014 I set up C-l to switch to helm-locate so that if I was unable to find what I wanted, I could have it search more. However, I never remembered to use it.

I now have two ways to solve that problem. First, I augment my hourly cron job with a quick cron job that runs every minute, and reports back any files edited recently. I had started with any files added recently but realized if I changed it to files edited recently, I could put those files at the top of my list. I run this every minute:

mdfind -onlyin $HOME "kMDItemFSContentChangeDate > \$time.now(-7200)
   && kMDItemContentTypeTree = 'public.text'" >$HOME/.recent-file-list.txt

In the helm source, I read ~/.recent-file-list.txt (a very short list) and put those items ahead of others.

The second solution is to use helm-source-locate as a source in the main interface, even though the UI is incompatible (for regular results you can use space to separate words, but for locate you can't). It will show the locate results below the main results. For times when my global file list and my recent file list don't show anything, locate might find it.

(setq helm-locate-command "/Users/amitp/bin/locate %.0s %s")

I use Mac's mdfind (which is updated in real time) to find the base filename and then I filter that through grep to handle folders. For example, if I run locate foo/bar then I use mdfind to find files named bar, then in the results I grep for foo/bar. However if I run locate ar without a folder name then I should find bar as a substring match. The shell script isn't perfect but it's “good enough” for now.

#!/bin/bash
rawquery="$*"
suffix=$(basename -- "$rawquery")
pattern="${suffix}*"

if [ "$rawquery" = "$suffix" ]; then
    # If there's no folder then it should be a substring
    pattern="*${suffix}*"
    # NOTE: double wildcards are slower!
fi

mdfind -onlyin $HOME "kMDItemFSName = '$pattern'cd" \
    | sed -e "s:^$HOME:~:" \
    | fgrep -i -- "$rawquery" \
    | postprocess

So far I've not needed the locate results, so I might end up removing that part of the code. I think the minutely cron job might be all I need.

I'm much happier with my setup now compared to the original, but there's always more tweaking to do. It's Emacs after all!

Labels: emacs, howto

Annotating MP3 files with BPM #

I wanted to tag my music library with the BPM of each song. There are commercial products for doing this through iTunes, but I wanted to do it from the command line, because I also use my music library outside of iTunes, and I don't want to be tied to iTunes anyway.

The bpm-tools package can calculate bpm for a song. On Mac, it's easy to install with Homebrew. The base package only includes the analyzer; I wanted something to tag my MP3 files too, so I used the =-with-bpm-tag= option. It ends up installing dependencies: mad, sox, id3lib, id3v2, flac, libogg, libvorbis, libao, vorbis-tools.

brew install bpm --bpm-tag
cd Music/iTunes\ Music/
find . -name \*.mp3 -exec bpm-tag '{}' ';'

However, the id3v2 command (on Mac/Homebrew, anyway) doesn't work the way bpm-tag wants, and for some reason sed wasn't cooperating with me, so I changed one line of /usr/local/bin/bpm-tag from

BPM=`id3v2 -l "$FILE" | sed -n 's/^TBPM.*: \([0-9\.]\+\)/\1/p'`

to

BPM=`id3v2 -l "$FILE" | sed -n 's/^TBPM.*: //p'`

This made it skip files that already had a BPM tag.

It took a while to run, but now all my MP3 files have a BPM, and iTunes picks that up too. Yay!

Labels: howto, mac

Emacs: rainbow-identifiers, customized #

A few months ago I had read Evan Brook's post about using text editor colors for identifiers instead of keywords. I tried the two Emacs modes for this: rainbow-identifiers and color-identifiers-mode. I wrote up my thoughts back then, and that blog post led to some features being added to rainbow-identifiers. Pros and cons:

Neither is a clear win. Try both and see. I ended up using rainbow-identifiers. However, there were too many identifiers being colored, which meant there were too many colors, which meant I couldn't really distinguish things anymore. Here's what color-identifiers looks like:

var edges = this.graph.edges_from(this.curr);
for (var i = 0; i < edges.length; i++) {
    var next = edges[i];
    var edge_weight = this.graph.edge_weight(this.curr, next);
    if (edge_weight != Infinity) {
        this.neighbors.push(next);
        mark_changed(next);
        if (!this.visited[next]) {
            this.g[next] = this.g[this.curr] + edge_weight;
            this.open.push(next);
            this.parent[next] = this.curr;
            this.visited[next] = true;
        }
    }
}

And here's what rainbow-identifiers looks like:

var edges = this.graph.edges_from(this.curr);
for (var i = 0; i < edges.length; i++) {
    var next = edges[i];
    var edge_weight = this.graph.edge_weight(this.curr, next);
    if (edge_weight != Infinity) {
        this.neighbors.push(next);
        mark_changed(next);
        if (!this.visited[next]) {
            this.g[next] = this.g[this.curr] + edge_weight;
            this.open.push(next);
            this.parent[next] = this.curr;
            this.visited[next] = true;
        }
    }
}

You can see that color-identifiers picks out local variables and not all names. And rainbow-identifiers colors lots of things, but it misses the variable declaration, which is important to color. So I configured rainbow-identifiers to do this:

var edges = this.graph.edges_from(this.curr);
for (var i = 0; i < edges.length; i++) {
    var next = edges[i];
    var edge_weight = this.graph.edge_weight(this.curr, next);
    if (edge_weight != Infinity) {
        this.neighbors.push(next);
        mark_changed(next);
        if (!this.visited[next]) {
            this.g[next] = this.g[this.curr] + edge_weight;
            this.open.push(next);
            this.parent[next] = this.curr;
            this.visited[next] = true;
        }
    }
}

It's in between the default configuration of the two packages. I highlight declarations in bold, and in color. I highlight local variables. I highlight local fields (this.x). But I don't highlight all names. Here's my configuration:

;; Customized filter: don't mark *all* identifiers
(defun amitp/rainbow-identifiers-filter (beg end)
  "Only highlight standalone words or those following 'this.' or 'self.'"
  (let ((curr-char (char-after beg))
        (prev-char (char-before beg))
        (prev-self (buffer-substring-no-properties
                    (max (point-min) (- beg 5)) beg)))
    (and (not (member curr-char 
                    '(?0 ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8 ?9 ??)))
         (or (not (equal prev-char ?\.))
             (equal prev-self "self.")
             (equal prev-self "this.")))))

;; Filter: don't mark identifiers inside comments or strings
(setq rainbow-identifiers-faces-to-override
      '(font-lock-type-face
        font-lock-variable-name-face
        font-lock-function-name-face))

;; Set the filter
(add-hook 'rainbow-identifiers-filter-functions 'amitp/rainbow-identifiers-filter)

;; Use a wider set of colors
(setq rainbow-identifiers-choose-face-function
      'rainbow-identifiers-cie-l*a*b*-choose-face)
(setq rainbow-identifiers-cie-l*a*b*-lightness 45)
(setq rainbow-identifiers-cie-l*a*b*-saturation 45))

I also turned off colors for the font-lock faces, and made font-lock-variable-name-face bold. I still think it may be too many colors, so I may end up turning off rainbow-identifiers and trying some of the things Wilfred Hughes has suggested.

Labels: emacs, howto

Mac OS: index source code #

A few weeks ago I posted how to make Mac OS Spotlight index source code. Mac OS is already indexing my C, C++, ObjC, Java, Fortran, Shell, Python, Perl, Ruby, Pascal, Ada, Javascript, and HTML files, but it doesn't get the other languages and text files I use. Here's the followup post.

Problem to solve: I want to search the contents of *.css and other text files with Spotlight.

In my previous blog post, I went into the Spotlight configuration file and told it to index certain file types like dyn.ah62d4rv4ge80e8drru, which I had determined was *.css. This wasn't satisfying. Instead of telling Spotlight, I wanted a solution at the Mac OS level.

Labels: howto, mac

Mac OS: Spotlight and source code #

Mac OS X has a real time file indexing system called Spotlight. By itself, it's not terribly useful for me. What makes it useful is that it's accessible from the command line. I can search filenames:

locate mapgen2.as
mdfind -name mapgen2.as

The locate command relies on a database that's rebuilt nightly; mdfind is updated in real time. Spotlight also indexes the contents, which gives me something similar to recursive fgrep:

fgrep -R word *
mdfind -onlyin . word

It won't search for arbitrary regexps but for quickly narrowing down files to find certain words, it's useful.

Labels: howto, mac

Extracting Safari's Reading List #

I use Safari's Reading List to transfer pages from my phone to my computer. But I don't really want to read these in Safari on my desktop. I'd rather read them in Firefox, maybe through Pinboard, which has an API to write items and mark them as "to read". This blog post documents what I learned.

Update: [2014-06-07] Deleting items out of the reading list does not work properly. It's ok to read everything out of there but I haven't figured out how to properly modify it. So I guess I need to delete things manually after I read them out.

Update: [2014-09-18] With iOS 8, I'm switching from Reading List to something else, maybe Pinner, to make this process easier.

I searched the web for some resources, and learned that the reading list, as well as other Safari bookmarks, are stored in ~/Library/Safari/Bookmarks.plist. I can use plutil and PlistBuddy to read and edit that file.

Labels: howto, mac