The Debug Adapter Protocol is a REPL protocol in disguise

A couple months back I created nluarepl. It’s a REPL for the Neovim Lua interpreter with a little twist: It’s using the Debug Adapter Protocol. And before that, I worked on hprofdap. Also a kind of a REPL using DAP that lets you inspect Java heap dumps (.hprof files) using OQL.

As the name might imply, a REPL isn’t the main use case for the Debug Adapter Protocol (DAP). From the DAP page:

The idea behind the Debug Adapter Protocol (DAP) is to abstract the way how the debugging support of development tools communicates with debuggers or runtimes into a protocol.

But it works surprisingly well for a REPL interface to a language interpreter too.

Essentials ¶

The typical REPL shows you a prompt after which you can enter an expression. You then hit Enter to submit the expression, it gets evaluated and you’re presented with the result or an error.

The Debug Adapter protocol defines a evaluate command which - as the name implies - evaluates expressions.

The definition for the payload the client needs to send looks like this:

interface EvaluateArguments {
  /**
   * The expression to evaluate.
   */
  expression: string;

  // [...]
}

With a few more optional properties.

The (important bit) of the response format definition looks like this:

interface EvaluateResponse extends Response {
  body: {
    /**
     * The result of the evaluate request.
     */
    result: string;

    /**
     * The type of the evaluate result.
     * This attribute should only be returned by a debug adapter if the
     * corresponding capability `supportsVariableType` is true.
     */
    type?: string;

    /**
     * If `variablesReference` is > 0, the evaluate result is structured and its
     * children can be retrieved by passing `variablesReference` to the
     * `variables` request as long as execution remains suspended. See 'Lifetime
     * of Object References' in the Overview section for details.
     */
    variablesReference: number;

    // [...]
}

result is a string and there is optionally a type. The neat bit is the variablesReference. It’s used to model structured data - allowing to build a tree-like UI to drill down into the details of a data structure.

Here is a demo to see it in action:

To get the data - or expand an option as shown in the demo above, the client must call the variables command with the variablesReference as payload. The response has an array of variables, where a variable looks like this:

interface Variable {
  /**
   * The variable's name.
   */
  name: string;

  /**
   * The variable's value.
   * This can be a multi-line text, e.g. for a function the body of a function.
   * For structured variables (which do not have a simple value), it is
   * recommended to provide a one-line representation of the structured object.
   * This helps to identify the structured object in the collapsed state when
   * its children are not yet visible.
   * An empty string can be used if no value should be shown in the UI.
   */
  value: string;

  /**
   * The type of the variable's value. Typically shown in the UI when hovering
   * over the value.
   * This attribute should only be returned by a debug adapter if the
   * corresponding capability `supportsVariableType` is true.
   */
  type?: string;


  /**
   * If `variablesReference` is > 0, the variable is structured and its children
   * can be retrieved by passing `variablesReference` to the `variables` request
   * as long as execution remains suspended. See 'Lifetime of Object References'
   * in the Overview section for details.
   */
  variablesReference: number;

  // [...]
}

A Variable is pretty similar to the initial evaluate result, except that it has both name and value. It also again has a variablesReference property, which means that they can be arbitrarily deeply nested (and you can have cyclic references).

Extras ¶

This already covers most of the functionality of a typical REPL backend. One more feature that’s nice to have is completion, and the Debug Adapter Protocol also has a completions command for that. Click on the link if you’re interested - I won’t go into detail about that here.

Another untypical feature for a REPL that the Debug Adapter Protocol provides is finding the locations of a variable definition. That’s also implemented in nluarepl, although it only works for functions.

The Boilerplate ¶

You might be wondering if there is anything in the Debug Adapter Protocol one must implement that’s useless baggage if all you want is a REPL frontend or backend.

Yes, there are are a few things:

• There’s the RPC mechanism, which is close to JSON-RPC, but not quite.
• Breakpoint handling. You can send back a response that rejects all. (nluarepl implements log points - which is basically dynamic log statements you can create at runtime)
• Session initialization. Here you can send back the capabilities.
• launch/attach pseudo handling.
• Disconnect/terminate handling. Not much needed here - you can use these to clean up any state.

The typical flow is that a client starts a debug session with a initialize command. Then the debug adapter replies with its capabilities and a normal client follows up sending breakpoints. After that it typically follows up with a launch command, which in a normal scenario would launch the application you want to debug.

To give you an impression of what this entails, here’s a snippet of the nluarepl code to implement the “dummy” actions:

function Client:initialize(request)
  ---@type dap.Capabilities
  local capabilities = {
    supportsLogPoints = true,
    supportsConditionalBreakpoints = true,
    supportsCompletionsRequest = true,
    completionTriggerCharacters = {"."},
  }
  self:send_response(request, capabilities)
  self:send_event("initialized", {})
end


function Client:disconnect(request)
  debug.sethook()
  self:send_event("terminated", {})
  self:send_response(request, {})
end


function Client:terminate(request)
  debug.sethook()
  self:send_event("terminated", {})
  self:send_response(request, {})
end

function Client:launch(request)
  self:send_response(request, {})
end

But Why ¶

Final question: Why would you do that?

Partly because of lazyness. From a development perspective I didn’t want to have to implement another REPL UI. Going the DAP route let me focus on the evaluation parts. And from a user perspective - I also wanted to re-use UI elements from nvim-dap. I’m used to that interface and have keymaps setup. I didn’t want to have another slightly different interface with different keymaps or behavior.