How colors influence `ls`'s speed and doors open

Kilian from Standford’s Sherlock team presents a very interesting analysis of why ls might feel slow. He started his investigation after a user reports:

It all started from a support question, from a user reporting a usability problem with ls taking several minutes to list the contents of a 15,000+ entries directory …

While I have experienced slow file system interactions when working with a large amount of files in one directory before and even analysed it to cache evictions in the XFS’ kernel memory, I’ve have never thought about Kilian’s findings regarding ls’s colorful output’:

But those pretty colors come at a price: for each and every file it displays, ls needs to get information about a file’s type, its permissions, flags, extended attributes and the like, to choose the appropriate color to display.

For every file in the directory, ls makes calls to lstat(), getxattr(), and capget(). This makes 15.000+ * 3 syscalls which, well, takes quite some time.

The second part of the article show’s how you can affect the runtime by adapting the coloring options of ls by changing the environment variable LS_COLORS. This variable is usually set by the executable dir_colors (1) and the configuration file dir_colors (5) — see respective man pages for details.

In the end, the article give a peek on a UNIX file type called “Door” file which is amazing and just because of this, the article is worth a read. I’m using Unix for almost 25 years professionally, I’ve never heard of Door files … there’s always something old to learn.