Debugging Neovim with Neovim and nvim-dap

  Friday, February 17, 2023 » Neovim

Table of content

Recently a change got merged in Neovim that decoupled its TUI from the main process. A side effect of the change is that debugging it became a bit more troublesome. It now forks itself and you end up with two processes. Depending on what you want to debug you need to attach to that second process.

I thought this might be an interesting use-case for more advanced features of nvim-dap. This post explores using it to automate attaching to the second process.

Even if you’ll never debug Neovim you may find this interesting as it could give you some ideas on what you can do with a hackable debugger.

This post uses nvim-dap features of the upcoming 0.5 release. If you’re on 0.4 you’ll have to switch to the development branch. (If you are from the future, use 0.5+)

Setting up cpptools

For this to work we’ll need to use vscode-cpptools. It’s a debug adapter for gdb. I tried using codelldb as well, but it doesn’t generate process events, which are necessary for this to work. More about them later.

I have a little Ansible role that takes care of downloading and extracting it:

---
- name: Ensure cpptools folder exists
  ansible.builtin.file:
    path: ~/apps/cpptools
    state: directory
    mode: 0755

- name: Download cpptools
  ansible.builtin.unarchive:
    src: https://github.com/microsoft/vscode-cpptools/releases/download/1.13.9/cpptools-linux.vsix
    dest: ~/apps/cpptools/
    remote_src: true

- name: Set executable flag
  ansible.builtin.file:
    path: ~/apps/cpptools/extension/debugAdapters/bin/OpenDebugAD7
    mode: 0755

If you don’t fancy Ansible you can download cpptools from their Release page. The vsix files are ZIP archives in disguise, you can unzip them. You also have to make sure that gdb is in your $PATH.

Installing & Configuring nvim-dap

Next you’ll need to install nvim-dap and configure the cpptools adapter:

It’s a regular plugin, you can use your favourite plugin manager, or clone it into the packages path:

git clone \
    https://github.com/mfussenegger/nvim-dap.git \
    ~/.config/nvim/pack/plugins/start/nvim-dap

Next configure the adapter. You can place this in ~/.config/nvim/init.lua:

local dap = require("dap")
dap.adapters.cppdbg = {
  id = 'cppdbg',
  type = 'executable',
  command = os.getenv("HOME") .. '/apps/cpptools/extension/debugAdapters/bin/OpenDebugAD7',
}

You’ll have to adapt the command to point to wherever you’ve installed cpptools. If the binary is in your $PATH environment variable you can use the command name instead of a full absolute path. This little snippet tells nvim-dap how it can launch the debug adapter named cppdbg. You can read more about the adapter configuration in :help dap-adapter.

Next we need to add a configuration that tells nvim-dap and the adapter what process it should launch. In our case that’s Neovim itself. Configurations go into dap.configurations.<filetype> and are a list. A simple configuration for Neovim could look like this:

local dap = require("dap")
dap.configurations.c = {
  {
    name = "Neovim",

    -- ⬇️ References the `cppdbg` in the `dap.adapters.cppdbg` definition
    type = "cppdbg",

    -- ⬇️ Used to indicate that this should launch a process
    request = "launch",

    -- ⬇️ The program we want to debug, the path to the `nvim` binary in this case
    -- Make sure it's built with debug symbols
    program = "/full/path/to/neovim/build/bin/nvim",

    -- ⬇️ Requires `external_terminal` configuration
    externalConsole = true,
  },
}

Starting Neovim inside Neovim can get confusing, and given that it requires a real terminal it’s best to launch it in an external terminal. cpptools supports that using the externalConsole property. You’ll also need to tell nvim-dap what and how to launch an external terminal. Here is an example for alacritty:

local dap = require("dap")
dap.defaults.fallback.external_terminal = {
  command = '/usr/bin/alacritty';
  args = {'-e'};
}

You can read more about the use of terminals with nvim-dap in :help dap-terminal.

You need to change one more option if you’re on Linux. Linux by default doesn’t allow any process to inspect other processes. That’s a security feature. You wouldn’t want to have malware read the memory of your key agent. For debugging this security feature can get in the way. You need to (temporarily) disable it:

echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

This will allow gdb to inspect the processes you want to debug. The default setting would prevent that because gdb isn’t launching the process itself. Neovim/nvim-dap is doing that.

With this in place you should be able to launch Neovim in the Neovim source tree, open a source file like src/nvim/main.c and call :DapContinue to start the debug session. If you got everything right this should start another Neovim instance. (If you have more than one configuration entry you’ll get a prompt and need to select Neovim)

Compile Neovim before starting the debug session

The setup so far is fairly standard and covered by the nvim-dap documentation and Wiki. Now one improvement we can make is to have it compile Neovim whenever we start the debug session.

To do that we can wrap the configuration into a Lua metatable with a __call property:

local dap = require("dap")
dap.configurations.c = {
  setmetatable(
    -- ⬇️ This first table is the same as before. It's the actual "dap-configuration"
    {
      name = "Neovim",
      type = "cppdbg",
      request = "launch",
      program = "/full/path/to/neovim/build/bin/nvim",
      externalConsole = true,
    },
    {
      __call = function(config)
        -- Empty for now, we extend this in a moment
        return config
      end
    }
  ),
}

This does exactly the same as before, except that nvim-dap calls the function under __call when you run the configuration. One of the use-cases for this function is to generate extra properties for the configuration. It receives the initial configuration as argument and must return a (updated) configuration.

But we don’t use it like that here. Instead of extending the configuration with new properties we can use it as a hook to run arbitrary logic before starting the debug session.

To build Neovim we can call into make:

__call = function(config)
  vim.fn.system("CMAKE_BUILD_TYPE=RelWithDebInfo make")
  return config
end

Sub-process auto-attach

Now we’re getting into the really advanced territory that motivated this post. nvim-dap includes a listener system that allows to listen to any event or response generated by a debug adapter. When Neovim forks its process, cpptool and gdb generate a process event. This process event tells us the pid of the new process. We can use this to start a second debug session that attaches to the child process. We’re going to update the __call hook to add some event listeners. You can read more about the listener system under :help dap-extensions.

__call = function(config)
  vim.fn.system("CMAKE_BUILD_TYPE=RelWithDebInfo make")

  -- ⬇️ listeners are indexed by a key.
  -- This is like a namespace and must not conflict with what plugins
  -- like nvim-dap-ui or nvim-dap itself uses.
  -- It's best to not use anything starting with `dap`
  local key = "a-unique-arbitrary-key"

  -- ⬇️ `dap.listeners.<before | after>.<event_or_command>.<plugin_key>`
  -- We listen to the `initialize` response. It indicates a new session got initialized
  dap.listeners.after.initialize[key] = function(session)
    -- ⬇️ immediately clear the listener, we don't want to run this logic for additional sessions
    dap.listeners.after.initialize[key] = nil

    -- The first argument to a event or response is always the session
    -- A session contains a `on_close` table that allows us to register functions
    -- that get called when the session closes.
    -- We use this to ensure the listeners get cleaned up
    session.on_close[key] = function()
      for _, handler in pairs(dap.listeners.after) do
        handler[key] = nil
      end
    end
  end

  -- We listen to `event_process` to get the pid:
  dap.listeners.after.event_process[key] = function(_, body)
    -- ⬇️ immediately clear the listener, we don't want to run this logic for additional sessions
    dap.listeners.after.event_process[key] = nil

    local ppid = body.systemProcessId
    -- The pid is the parent pid, we need to attach to the child. This uses the `ps` tool to get it
    -- It takes a bit for the child to arrive. This uses the `vim.wait` function to wait up to a second
    -- to get the child pid.
    vim.wait(1000, function()
      return tonumber(vim.fn.system("ps -o pid= --ppid " .. tostring(ppid))) ~= nil
    end)
    local pid = tonumber(vim.fn.system("ps -o pid= --ppid " .. tostring(ppid)))

    -- If we found it, spawn another debug session that attaches to the pid.
    if pid then
      dap.run({
        name = "Neovim embedded",
        type = "cppdbg",
        request = "attach",
        processId = pid,
        -- ⬇️ Change paths as needed
        program = os.getenv("HOME") .. "/dev/neovim/neovim/build/bin/nvim",
        cwd = os.getenv("HOME") .. "/dev/neovim/neovim/",
        externalConsole = false,
      })
    end
  end
  return config
end

With this in place you should be able to launch the Neovim debug session to start Neovim in another terminal, and have it automatically attach to the sub-process. You can verify that by viewing the active debug sessions using something like this:

:lua local w = require('dap.ui.widgets'); w.sidebar(w.sessions, nil, '5 sp').open();

This should open a new split window listing the two active debug sessions.