Deep Dive into Ruff: PartI - Implementing First Version of Ruff in Python

Introduction

Inspired by a blog post highlighting the first version of ruff, I am motivated to learn Rust. I thought it would be a good idea to understand ruff, a Python linter that I use daily. This blog will focus on the implementation of ruff, with an emphasis on AST and parsing rather than the Rust language. In Part II, I will attempt to implement the same program in Rust.

Messages

Messages are classes that store specific types of lint errors. For example, using if with a tuple as an expression will always result in True. Therefore, there is an existing rule implemented in Flake8, known as rule F643, which is implemented as follows:

AST

Parsing is its own topic, but in Python this is easy because the standard library has implemented this for us. We can use ast directly. In rust, there is rustpython_parser.

We use AST to understand the structure of the code. For example, we can detect the rule F634 described above.

if_tuple = """
if (False, False,):
        pass
"""

import ast
print(ast.dump(ast.parse(if_tuple), indent=2))

Module(
  body=[
    If(
      test=Tuple(
        elts=[
          Constant(value=False),
          Constant(value=False)],
        ctx=Load()),
      body=[
        Pass()],
      orelse=[])],
  type_ignores=[])

  body=[
    If(
      test=Tuple(
        elts=[
          Constant(value=False),
          Constant(value=False)],
        ctx=Load()),

This is the basic idea of linter. It takes these steps: 1. Define a set of rules 2. Parse a file into AST 3. Walk through the AST, identify errors if it matches a rule. 4. Report the errors at the end