No-Config Python debugging using Neovim

A little while ago Microsoft released No Config Debugging functionality in their vscode python extension.

This got me curious - wondering if it could be replicated in Neovim. Turns out it can be. This post shows how.

Shopping list ¶

On a high level what we’d like to have is:

• A way to set breakpoints within Neovim.
• A way to execute a Python script or application in the terminal and have it stop at the breakpoints.
• A way to inspect the state, step in, out, resume execution and so on.

For debugging support Visual Studio Code uses the Debug Adapter Protocol (DAP). This is similar to the Language Server Protocol (LSP) in that it aims to abstract the way development tools (your editor) communicate with debuggers or runtimes.

For Python there’s a debug-adapter implementation called debugpy. Once installed - e.g. on Archlinux via pacman -S python-debugpy - you can run your application in a shell/terminal like this:

python -m debugpy --listen localhost:5678 --wait-for-client myfile.py

This will create a service listening on port 5678 - waiting for some DAP client to connect before it executes myfile.py. For Neovim we’ve two popular DAP clients:

• nvim-dap
• vimspector

In this post we’re going to use nvim-dap.

nvim-dap - being a generic Debug Adapter Protocol client - needs some configuration to teach it about debugpy. You can learn more about the adapter configuration by reading :help dap-adapter. We’ll take a short-cut here and use nvim-dap-python, which takes care of the adapter setup. See its README for setup instructions.

The full machinery ends up looking something like this:

   ┌────────┐ DAP   ┌─────────────┐  ┌────────┐   ┌─────────┐
   │dev-tool│──────►│debug-adapter│─►│debugger│──►│ script  │
   │  nvim  │       │   debugpy   │  │ pydevd │   │myfile.py│
   │nvim-dap│       │             │  │        │   │         │
   └────────┘       └─────────────┘  └────────┘   └─────────┘

Usually you’d use nvim-dap to set some breakpoints and then start a debug session via :DapNew or require("dap").continue(). These commands prompt you for a application-debug-configuration declared in a .vscode/launch.json file or within a dap.configurations.<filetype> table that’s part of your init.lua. You can learn more about that by reading :help dap-configuration.

nvim-dap-python includes a few global configuration out of the box. Including one that lets us attach to a running debugpy service. It calls that configuration attach.

So, with debugpy, nvim-dap and nvim-dap-python we can run the following command in a shell:

python -m debugpy --listen localhost:5678 --wait-for-client myfile.py

And use :DapNew attach within a Python file opened in nvim and we’d have it run myfile.py in debug mode.

Except that’s not the goal of this post.

No-Config glue ¶

What we’d like instead is some kind of debugpy-run myfile.py somearg executable that automatically runs debugpy --listen with the provided file and argument and also automatically tells nvim-dap to connect to it.

Luckily for us we’ve all the ingredients we need to glue these components together:

• Neovim has a client-server architecture and comes with an RPC interface and each nvim instance by default listens on a unix domain socket. We can use this socket to send commands to a running instance.
• Any process run from within nvim using :terminal has a $NVIM environment variable set that points to the unix domain socket file for the current instance.
• Neovim integrates a Lua interpreter and includes nice APIs that allow us to spawn processes and to act as a RPC client to communicate with another Neovim process. We can use nvim -l to use its included Lua interpreter to execute a Lua script.

All these things put together allow us to create a debugpy-run Lua script. This allows us to have the following flow:

  ┌►nvim
  │myfile.py
  │  │
  │  │  :terminal
  │  └────────────► debugpy-run
  │                   nvim -l
  │                     │
  │                     │─► spawn debugpy --listen
  │    rpc connect      │    inherits stdin, stdout, stderr
  └─────────────────────┤
     exec_lua dap.run() │
                        ▼
                   vim.wait (block)

Here’s the code including some comments:

#!/usr/bin/env -S nvim -l
-- vim: ft=lua


-- We use the "luv" Lua bindings that vim includes to spawn the debugpy process
-- In older versions this was exposed as `vim.loop`, in newer version it is `vim.uv`
local uv = vim.uv or vim.loop

local opts = {
  -- `uv.spawn` lets us inherit fds if we provide them as integer here
  -- 0, 1, 2 are stdin, stdout, stderr
  -- This is done to ensure you see the output of the script you're debugging when using `debugpy-run`
  stdio = {0, 1, 2},
  detached = true,

  -- lua sets a global `arg` variable to the arguments of the script you run
  args = {"--listen", "localhost:1234", "--wait-for-client", arg[1]}
}

local handle, piderr
local function on_exit(code)
  print("exit " .. code)
  handle:close()
  handle = nil
end
handle, piderr = uv.spawn("debugpy", opts, on_exit)
if not handle then
  print(piderr)
  return
end

-- We get the path to the socket
local parent = assert(os.getenv("NVIM"), "debugpy-run only works if $NVIM is set")

-- And connect to it
local conn = vim.fn.sockconnect("pipe", parent, { rpc = true })

-- And start a new debug session via `dap.run()`
vim.fn.rpcrequest(conn, "nvim_exec_lua", [[require("dap").run(...)]], {{
  name = "debugpy-run", -- An arbitrary name

  -- This refers to the dap.adapters.debugpy definition created by nvim-dap-python
  type = "debugpy",

  -- This tells nvim-dap to connect to the debugpy instance listening on 1234
  -- See:
  -- - https://github.com/microsoft/debugpy/wiki/Command-Line-Reference
  -- - https://github.com/microsoft/debugpy/wiki/Debug-configuration-settings
  request = "attach",
  connect = {
    host = "127.0.0.1",
    port = 1234
  },
  purpose = {"debug-in-terminal"},
  pythonArgs = {"-Xfrozen_modules=off"},
}})

-- And block until the debugpy process stop
while handle do
  vim.wait(100, function() return false end)
end

Wrap up ¶

That’s it. Have a look at a demo.

I hope you learned something new. In my workflow I tend to use test cases as entry point or application specific configurations so I won’t be publishing this in a ready-to-use way. But feel free to use the code from this post and extend it as you see fit.

Note that you could use the same approach with other languages if there are debug adapters that support listening on TCP in a blocking mode.