This blog has my random thoughts; game-related posts go here.

Back in 2014 I posted my emacs setup for finding files globally. It acts like switch-to-buffer, if all the files on my system were already open in buffers. I can type in a substring of any filename I'm likely to work on and switch to that buffer. Since them, helm has changed its interface, and I've updated my code, adding a few extra quality-of-life features.

I use the same hourly cron job to take inventory of all files I am likely to edit, putting recently modified files near the top, as they are most likely to be the best match. I save this list to ~/.global-file-list.txt.

What's changed is how I set up helm. In 2014 I was using helm-recentf and temporarily binding the recentf file list to my own list. This was a quick & dirty way to do it, and it worked, except when I opened a file, it didn't get put onto the recentf list, since I had let-bound that list to something else. Oops. I learned more helm (by reading the source code and John Kitchin's blog posts) and now construct my own helm source instead of abusing helm-recentf:

(defun amitp/helm-all-files ()
  "Global filename match, over all files I typically open"
   :sources '(amitp/helm-source-my-files helm-source-locate)
   :buffer "*helm all files*"))

Helm will look at this source to get the filename list:

(defvar amitp/helm-source-my-files
  (helm-build-sync-source "My files"
    :candidates #'amitp/helm-global-file-list
    :filtered-candidate-transformer #'amitp/helm-filter-my-files
    :keymap helm-generic-files-map
    :action 'helm-type-file-actions))

This source tells helm to call amitp/helm-global-file-list. This function returns the most recently modified files at the top, then the currently open files, then files in the current folder, then recently opened files, then the global file list. (I'm not happy with this order and am still experimenting.) Some filenames will be in more than one list so I eliminate duplicates.

(defun amitp/helm-global-file-list ()
  "Files to list in amitp/helm-all-files"
  ;; delete-dups much faster than cl-remove-duplicates
   (mapcar 'abbreviate-file-name
            (read-file-into-lines "~/.recent-file-list.txt")
             (directory-files default-directory t))

I also use a filter to remove some filenames from the list. Why? Normally I want to show only the “source” and not the “compiled” version of something. For example I want to show .el files but not .elc files. I want to show .c files but not .o files. I can hide these with helm-boring-file-regexp-list. However, for my web pages, some of them are directly written as .html files (*.html should be included) but others are written in org-mode or markdown or something else (*.html should be excluded). To decide whether a file ending in .html is source or not, I need to look at the other filenames in the list. I use a “filtered candidate transformer” (see the documentation for helm-source) to take the currently matching filenames and filter them further. If the filenames contain both $ and $something.html then I know that the html is not a source file, so I hide it.

(defun dominated-by-filename (regexp replacement filename candidates)
  "True if FILENAME with REGEXP replaced by REPLACEMENT is already in CANDIDATES"
  (let ((new-filename (replace-regexp-in-string regexp replacement filename)))
    (and (not (equal new-filename filename))
         (member new-filename candidates))))

(defun amitp/helm-filter-my-files (candidates _source)
  "Ignore a build target if a build source exists in the candidates"
  (cl-loop for filename in candidates
       (dominated-by-filename "\\.html$" ".org" filename candidates)
       (dominated-by-filename "\\.html$" ".md" filename candidates))
      ;; (I have more rules but you get the idea)
      collect filename))

I don't currently colorize the output. I've considered coloring by type (source code, prose, build file, etc.) and by origin (recentf, current folder, open buffer, global list) but neither of these seems particularly appealing. I'll continue to experiment.

The last change since 2014 is that my global file list doesn't have everything, and it is annoyingly missing any new files I've created in the past hour. In 2014 I set up C-l to switch to helm-locate so that if I was unable to find what I wanted, I could have it search more. However, I never remembered to use it.

I now have two ways to solve that problem. First, I augment my hourly cron job with a quick cron job that runs every minute, and reports back any files edited recently. I had started with any files added recently but realized if I changed it to files edited recently, I could put those files at the top of my list. I run this every minute:

mdfind -onlyin $HOME "kMDItemFSContentChangeDate > \$
   && kMDItemContentTypeTree = 'public.text'" >$HOME/.recent-file-list.txt

In the helm source, I read ~/.recent-file-list.txt (a very short list) and put those items ahead of others.

The second solution is to use helm-source-locate as a source in the main interface, even though the UI is incompatible (for regular results you can use space to separate words, but for locate you can't). It will show the locate results below the main results. For times when my global file list and my recent file list don't show anything, locate might find it.

(setq helm-locate-command "/Users/amitp/bin/locate %.0s %s")

I use Mac's mdfind (which is updated in real time) to find the base filename and then I filter that through grep to handle folders. For example, if I run locate foo/bar then I use mdfind to find files named bar, then in the results I grep for foo/bar. However if I run locate ar without a folder name then I should find bar as a substring match. The shell script isn't perfect but it's “good enough” for now.

suffix=$(basename -- "$rawquery")

if [ "$rawquery" = "$suffix" ]; then
    # If there's no folder then it should be a substring
    # NOTE: double wildcards are slower!

mdfind -onlyin $HOME "kMDItemFSName = '$pattern'cd" \
    | sed -e "s:^$HOME:~:" \
    | fgrep -i -- "$rawquery" \
    | postprocess

So far I've not needed the locate results, so I might end up removing that part of the code. I think the minutely cron job might be all I need.

I'm much happier with my setup now compared to the original, but there's always more tweaking to do. It's Emacs after all!

Labels: ,

We have 3D printers that can take a description and print an object. Some 3D printers are capable of printing everything needed to make another 3D printer.

3d printer operation

This reminds me of programming language implementations.

An interpreter takes a description (a program) and prints some output. A meta-circular interpreter is an interpreter written in the same language that it inteprets.

program interpreter

With programming languages we also have compilers that translate a program into another program that does the same thing but better (smaller and faster).

program compiler

What's the equivalent for 3D printers? I think it'd be a 3D printer that prints another, more specialized printer. That specialized printer might work faster or produce higher quality output or work with materials that a general-purpose 3D printer can't work with.

3D printer that prints 3D printers

The analogy isn't perfect but I'd like to see 3D printers that print more specialized printers.

Labels: ,

A few months ago I had read Evan Brook's post about using text editor colors for identifiers instead of keywords. I tried the two Emacs modes for this: rainbow-identifiers and color-identifiers-mode. I wrote up my thoughts back then, and that blog post led to some features being added to rainbow-identifiers. Pros and cons:

colors are stable on reload/edit better than colors change on reload/edit
same name has the same color across buffers better than the same name has different colors across buffers
color assigned to every name worse than color assigned to important names
colors might be similar to each other worse than colors chosen to be distinguishable
universal configuration across modes better than separate configuration per mode

Neither is a clear win. Try both and see. I ended up using rainbow-identifiers. However, there were too many identifiers being colored, which meant there were too many colors, which meant I couldn't really distinguish things anymore. Here's what color-identifiers looks like:

var edges = this.graph.edges_from(this.curr);
for (var i = 0; i < edges.length; i++) {
    var next = edges[i];
    var edge_weight = this.graph.edge_weight(this.curr, next);
    if (edge_weight != Infinity) {
        if (!this.visited[next]) {
            this.g[next] = this.g[this.curr] + edge_weight;
            this.parent[next] = this.curr;
            this.visited[next] = true;

And here's what rainbow-identifiers looks like:

var edges = this.graph.edges_from(this.curr);
for (var i = 0; i < edges.length; i++) {
    var next = edges[i];
    var edge_weight = this.graph.edge_weight(this.curr, next);
    if (edge_weight != Infinity) {
        if (!this.visited[next]) {
            this.g[next] = this.g[this.curr] + edge_weight;
            this.parent[next] = this.curr;
            this.visited[next] = true;

You can see that color-identifiers picks out local variables and not all names. And rainbow-identifiers colors lots of things, but it misses the variable declaration, which is important to color. So I configured rainbow-identifiers to do this:

var edges = this.graph.edges_from(this.curr);
for (var i = 0; i < edges.length; i++) {
    var next = edges[i];
    var edge_weight = this.graph.edge_weight(this.curr, next);
    if (edge_weight != Infinity) {
        if (!this.visited[next]) {
            this.g[next] = this.g[this.curr] + edge_weight;
            this.parent[next] = this.curr;
            this.visited[next] = true;

It's in between the default configuration of the two packages. I highlight declarations in bold, and in color. I highlight local variables. I highlight local fields (this.x). But I don't highlight all names. Here's my configuration:

;; Customized filter: don't mark *all* identifiers
(defun amitp/rainbow-identifiers-filter (beg end)
  "Only highlight standalone words or those following 'this.' or 'self.'"
  (let ((curr-char (char-after beg))
        (prev-char (char-before beg))
        (prev-self (buffer-substring-no-properties
                    (max (point-min) (- beg 5)) beg)))
    (and (not (member curr-char 
                    '(?0 ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8 ?9 ??)))
         (or (not (equal prev-char ?\.))
             (equal prev-self "self.")
             (equal prev-self "this.")))))

;; Filter: don't mark identifiers inside comments or strings
(setq rainbow-identifiers-faces-to-override

;; Set the filter
(add-hook 'rainbow-identifiers-filter-functions 'amitp/rainbow-identifiers-filter)

;; Use a wider set of colors
(setq rainbow-identifiers-choose-face-function
(setq rainbow-identifiers-cie-l*a*b*-lightness 45)
(setq rainbow-identifiers-cie-l*a*b*-saturation 45))

I also turned off colors for the font-lock faces, and made font-lock-variable-name-face bold. I still think it may be too many colors, so I may end up turning off rainbow-identifiers and trying some of the things Wilfred Hughes has suggested.

Labels: ,

We don't collect enough weather data from the atmosphere (or ocean). We also would like internet service everywhere on the planet. Let's solve both problems at once:

A 3D grid of tiny flying drones that collect temperature, humidity, wind from every location and elevation, and also serve as a mesh network ISP that provide internet access to everywhere on the planet.

Ok, maybe it wouldn't really work. Oh well.


A few weeks ago I posted how to make Mac OS Spotlight index source code. Mac OS is already indexing my C, C++, ObjC, Java, Fortran, Shell, Python, Perl, Ruby, Pascal, Ada, Javascript, and HTML files, but it doesn't get the other languages and text files I use. Here's the followup post.

Problem to solve: I want to search the contents of *.css and other text files with Spotlight.

In my previous blog post, I went into the Spotlight configuration file and told it to index certain file types like dyn.ah62d4rv4ge80e8drru, which I had determined was *.css. This wasn't satisfying. Instead of telling Spotlight, I wanted a solution at the Mac OS level.

Labels: ,