A Vim errorformat for `firebase deploy`

Posted by Fulvio Casali at Jun 03, 2020 07:55 AM |
Filed under: , ,

Vim has a very useful feature called Quickfix. By way of a very cursory introduction, Quickfix is a specialized window that collects and parses the output from a command that you run. Every line in Quickfix acts as a hyperlink to a particular line and column of a given file. What makes this so useful is that you can use it with any conceivable command that potentially produces many "hits" and Quickfix lets you jump from one "hit" to the next and back at will.

A few examples:

  • grep - when you are looking for all occurrences of a particular string or regex over an entire codebase
  • Compilation - when you are building code and want a clean, uncluttered list of all the errors
  • Testing - when you run a test suite and want to see any test failures that occurred
  • Linting - similar to compilation, but this is such a frequent use case that I wanted to mention it on its own.

The Quickfix window gets populated by default when you run :vimgrep, :grep or :make. A lot of vim plugins tie into Quickfix, too. For example, the fugitive.vim plugin adds a :Ggrep command, which is a wrapper for git grep, so you can browse all matches from your current git repository right in the Quickfix window (though that is but a side benefit of all the capabilities this plugin offers).

The :make command is special, though, in that it is completely configurable. If you don’t customize it, :make will look for a Makefile in your working directory and run it just as you would expect. However, you can harness the power of :make by setting two variables: makeprg and errorformat.

The first variable, makeprg, simply tells :make what external program or script to execute, and of course you can configure your command arguments as dynamically as you want. The second variable, errorformat, is where all the parsing magic happens and is the topic of this post.

Out of the box, vim comes with support for a lot of different programming languages, compilers and frameworks, and if you start tapping into the plugin ecosystem you can build, compile, test or lint to your heart’s content and never even know about makeprg and errorformat. At least, that’s how it was for me over several years. Then, I finally hit a point where I got curious about how to bend the Quickfix window to my will.

I’ve been writing some code for Google Cloud Platform’s Firebase Functions, and for a while my development cycle looked like this:

  1. Edit my code
  2. firebase deploy
  3. Fix any errors
  4. If there were no errors during deployment, test my code in Firebase
  5. Repeat

I won’t get into how inefficient this cycle is, and what I’ve discovered to make it much tighter, because that would be a whole different story.
But for the sake of this story, suffice to say that steps 1, 2 and 3 were happening in vim. Actually, neovim, so that step 2 would happen in a terminal window right inside nvim itself.
Firebase deploy is the crux of the problem: it takes about 15 seconds to finish if it finds an error, and about 1 minute if it succeeds.

So, in order to automate and make this whole cycle a little more asynchronous, I first used Tim Pope’s vim-dispatch plugin, which introduces an asynchronous variant of :make, invoked with a capital M (:Make). Then, using an autocommand for the BufWritePost event and targeting just the file I was working on, I would start :Make! in the background every time I saved the file. Thus, I could keep working and when firebase deploy was done it would tell me at the bottom of the screen, whether successfully or with an error.

This was a nice step in the right direction, but up to this point I left the errorformat variable completely untouched. Thus, even if an error occurred during deployment, the Quickfix window remained empty and useless. I would still have to look at the output of :Make to see what error firebase deploy had reported and where it was.

I did not really search for a firebase compiler plugin for vim on the net. I figured, the time had come for me to finally learn how errorformat works. I have always had a soft spot for regular expressions, but it turns out that the structure of errorformat goes way beyond simple regexes. So it was still a bit challenging to figure out.

Here is a sample run of firebase deploy. I intentionally introduced a syntax error in my code so I could show you how the error is reported:

> time firebase deploy
⚠  functions: package.json indicates an outdated version of firebase-functions.
Please upgrade using npm install --save firebase-functions@latest in your functions directory.

=== Deploying to 'myproj-b00de'...

i  deploying functions
i  functions: ensuring required API cloudfunctions.googleapis.com is enabled...
✔  functions: required API cloudfunctions.googleapis.com is enabled
i  functions: preparing functions directory for uploading...

Error: Error occurred while parsing your function triggers.

/home/me/code/assistant/myproj/functions/data.js:4
class (DataManager {
      ^

SyntaxError: Unexpected token '('
    at wrapSafe (internal/modules/cjs/loader.js:1067:16)
    at Module._compile (internal/modules/cjs/loader.js:1115:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
    at Module.load (internal/modules/cjs/loader.js:1000:32)
    at Function.Module._load (internal/modules/cjs/loader.js:899:14)
    at Module.require (internal/modules/cjs/loader.js:1040:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/home/me/code/assistant/myproj/functions/functions/index.js:40:22)
    at Module._compile (internal/modules/cjs/loader.js:1151:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)

Having trouble? Try firebase [command] --help

real    0m14.499s
user    0m13.276s
sys     0m1.740s

As you can see from the above output, the error is in line 4 of the file data.js. The column in which the error was spotted is marked with a caret in the next line. Then you can see the description of the error (Unexpected token ‘(‘) and finally a traceback which we don’t really care about.

All of that information gets distilled in my Quickfix window to the following single line:

data.js|4 col 7 error|  SyntaxError: Unexpected token '('

And by hitting enter with the cursor on the line above in the Quickfix window, vim takes me straight to line 4, column 7 of data.js, where the error is.

And here is my firebase compiler in all its glory:

 > ~/.config/nvim/after/compiler/firebase.vim:
 ---------------------------------------------
 line numbers are not part of the file
 lines starting with " are comments,
 giving examples that would be matched by the pattern in the next line

 1| let current_compiler="firebase"                                                                                                                                                                                                                                        
 2| CompilerSet makeprg=firebase\ deploy

 3| "/home/me/code/assistant/myproj/functions/data.js:4       
 4| CompilerSet errorformat=%E%f:%l

 5| "      ^                                                                            
 6| CompilerSet errorformat+=%-C%p^

 7| "SyntaxError: Unexpected token '('                                                  
 8| CompilerSet errorformat+=%+Z%[%^:\ ]%#:\ %m

 9| "class (DataManager {                                                               
10| CompilerSet errorformat+=%-C%.%#

11| "=== Deploying to 'myproj-b00de'...
11| CompilerSet errorformat+=%-G%.%#

Line 1 allows me to just say :compiler firebase, which is a shortcut to setting the two variables makeprg and errorformat in a single step.
Line 2 sets makeprg to the command firebase deploy. The space has to be escaped with a backslash.

Lines 4-11 all build the errorformat in several successive steps. Notice the first line has an equal sign, but the subsequent ones use +=, to append to the previous value. This is not necessary - we could just assign all the patterns to errorformat, separating them with commas. But that would be much harder to read:

CompilerSet errorformat=%E%f:%l,%-C%p^,%+Z%[%^:\ ]%#:\ %m,%-C%.%#,%-G%.%#

Apart from legibility, another benefit of building errorformat incrementally is that we can adjust the priority given to each pattern by moving them up or down, to control whether a pattern matches before or after another one. Plus, for debugging purposes, we can just comment out any patterns.

An errorformat is a comma-delimited list of patterns. Each line in the output of makeprg is matched to every pattern in errorformat until one matches. The patterns use a %-based token notation similar to the scanf format string in C. You can also see a comment above each pattern, which shows an example string that would be matched.

%E%f:%l

If this patterns matches, set a multi-line flag (%E): match a file path, a colon, and a line number.

%-C%p^

Only consider this pattern if the multi-line flag is set (%-C), and if it matches, do not add it to the error message (the - in the %-C token): if there is caret preceded by white-space, ‘-’ or ‘.’ characters, count them and set that number as the column number;

%+Z%[%^:\ ]%#:\ %m

Only consider this pattern if the multi-line flag is set, and if it matches, clear the flag (%+Z): match the line if it starts with a string without white-space characters followed by a colon and a space. Save the remainder of the line as the error message (%m).

%-C%.%#

Ignore any other line if the multi-line flag is set (%-C).

%-G%.%#

Ignore all the informational lines before and after the error.


I found the following pages useful in exploring this topic:

Finally, I want to mention that I also found Tim Pope’s Projectionist plugin very useful to segregate these customizations to just the projects in which they were needed, instead of getting applied globally.

Filed under: , ,