Loading Emacs in 100ms under Nix

At some point in the past I made Emacs start in 200ms under Nix. It was OK, but 200ms is still a perceptible lag for me. So I did a bit of digging and figured out the reason. TL;DR: it's a combination of Nix and load paths. If you want a quick solution, place this to your ~/.early-init.el:

;;; -*- lexical-binding: t -*-

(let* ((emacs-lisp-path (seq-find (lambda (s) (string-suffix-p "lisp/emacs-lisp" s)) load-path))
       (url-path (seq-find (lambda (s) (string-suffix-p "lisp/url" s)) load-path))
       (lisp-path (directory-file-name (file-name-directory emacs-lisp-path)))
       (old-load-path (mapcar #'copy-sequence load-path)))
  (when emacs-lisp-path
    (delete emacs-lisp-path load-path)
    (delete lisp-path load-path)
    (delete url-path load-path)

    ;; Move builtin paths to beginning of load-path so during init directories
    ;; are not scanned needlessly for each `require`
    (setq load-path (append (list emacs-lisp-path lisp-path url-path) load-path))

    ;; After init is done, restore the load-path back. This may be important
    ;; for some packages that need newer versions of builtins from melpa. Such
    ;; as Magit and transient-mode. Hopefully, there are few such packages.
    (add-hook 'after-init-hook #'(lambda ()(setq load-path old-load-path)))
    )
  )

Explanation and root cause analysis

I have a pretty fast machine with an M.2 SSD, so IO latency should be good enough. I've also tuned my Emacs config to lazy-load as much as possible, and only eagerly load just a few packages that are enough to show the initial window. Everything else depends on the file that is being opened (like loading the LSP or tree-sitter grammars). So I knew that something fishy is going on.

To confirm this, I decided to run Emacs under strace like this:

strace -o trace.txt emacs -nw --eval="(kill-emacs)"

By looking at the output, it immediately became obvious that Emacs is doing a lot of openat syscalls. Just how many?

cat trace.txt | grep openat | grep elc | wc -l
6045

Well, 6045 is a lot of syscalls. I then did something around the lines of:

cat ~/scratch/trace.txt | grep openat | grep -E -o '/[^/]*el[^/]"' | tr -d '/"' | sort | uniq -c | sort -n

This command filters attempts to load Elisp modules and counts number of attempts for each. The output is quite big, but here's a snippet:

...
66 comp-cstr.elc
66 comp.elc
66 comp-run.elc
66 easy-mmode.elc
66 eieio-core.elc
66 eieio.elc
66 generate-lisp-file.elc
66 gv.elc
66 icons.elc
66 map.elc
66 package.elc
66 pcase.elc
66 rx.elc
66 subr-x.elc
66 warnings.elc
70 .emacs.elc
70 tmux-256color.elc
92 xterm.elc

As you can see, every module was attempted many times. This is because when Emacs does a (require 'some-package), it will sequentially go over load-path and try to find some-package there with a bunch of possible extensions (.el, .elc, .elc.gz, .so, etc.).

Manually examining load-path with M-x describe-variable shows that there are about 60 entries there, and "core" emacs modules are stuffed to the very end of the list. This is our problem right there: Emacs needs to load quite a few standard modules at startup time and the same paths are scanned over and over again.

Why this problem is worse in Nix

Nix wants to wrap programs in such a way that it usually takes charge of installing and plugging all dependencies. There is a good reason for doing that in terms of reproducibility, but occasionally some programs can be stubborn. Which happens to be Emacs in this case.

Nix derivation emacsWithPackages accepts a list of packages to get from MELPA, installs them to individual directories and prepends all those directory paths to load-path. In normal circumstances, this job is taken by the use-package module, which kicks in way later than the core modules have been loaded, and so the extension of load-path doesn't have such "detrimental" effect. However, since Nix cannot embed its extension of load-path into the middle of the user's init.el, it has to modify it in advance.

Workaround

The workaround in the beginning of this article is kind of a hack, and it may or may not work for you depending on the type of configuration you're having. The way it works is to temporarily move paths to the core modules to the beginning of load-path and reset the load-path back to its original state as soon as the init process is over.

The reason why we need to return load-path to its original state is due to some third-party modules depending on newer versions of core libraries that are distributed via github. One such case is Magit and transient-mode. But fortunately, such modules can be deferred to after the init is done.

After applying the snippet above, my load time has reduced to 100ms which is good enough for day-to-day use. I will try to further reduce it by creating a "pre-heated" Emacs memory dump, but that's a topic for another time.