How Pyscn Analyzes Python with Go and tree-sitter

Pyscn is a code quality analyzer for Python built by DaisukeYoda at ludo-technologies. It finds dead code, code clones, coupling issues, and complexity hotspots. It's written in Go, uses tree-sitter for parsing, and the project reports processing over 100,000 lines per second.

I contribute to it (my pull requests, including the one that made dependency analysis scale), and a few of its design decisions are interesting enough to write up: how it detects dead code through control flow analysis, how it finds duplicated code across clone types 1-4 with LSH acceleration, and how it plugs into AI coding assistants via MCP.

Why Go + tree-sitter

Python has good AST modules (ast, astroid), but they're slow on large codebases, and an analyzer written in Python is bottlenecked by the interpreter. Go gives native performance, and tree-sitter provides a mature Python parser so nobody has to write a custom grammar.

tree-sitter is a parser generator used by editors like Neovim, Helix, and Zed. Its Go binding (go-tree-sitter) lets you parse Python source files into concrete syntax trees, then walk those trees at Go speed.

The catch is that tree-sitter produces CSTs (concrete syntax trees), so you get every token including whitespace and punctuation. That means more node types to handle. It also means you can reconstruct exact source locations, which is useful for reporting the lines of dead code.

Dead Code via Control Flow Graphs

Most dead code detectors look for unused functions or unreachable imports. Pyscn also builds a Control Flow Graph for each function and finds code that no execution path can reach.

CFG construction goes like this:

Parse the function body with tree-sitter
Create nodes for each statement or expression
Add edges for sequential flow and branching (if/elif/else, try/except, loops)
Mark exit points (return, raise, break, continue)
Walk from entry to find all reachable nodes
Report any node that wasn't reached

Exhaustive branches are the interesting case. If every branch of an if/elif/else chain ends with a return or raise, then code after the chain is unreachable:

def process(status):
    if status == "active":
        return handle_active()
    elif status == "pending":
        return handle_pending()
    else:
        raise ValueError(f"Unknown status: {status}")
 
    # This code is dead — all branches above exit the function
    log_completion()  # pyscn catches this

Simple detectors miss this because they don't track control flow through branches. Pyscn's CFG analysis handles nested branches, early returns inside loops, and try/except/finally blocks.

Clone Detection: Types 1-4

Pyscn classifies duplicated code into four types:

Type	Definition	Example
Type 1	Identical except whitespace/comments	Copy-pasted verbatim
Type 2	Identical structure, different identifiers/literals	Renamed variables
Type 3	Similar structure with small modifications	Added/removed a line
Type 4	Same logic, different syntax	List comp vs. for loop

Types 1 and 2 are detected by normalizing the AST (stripping identifiers, replacing literals with type markers) and hashing the result. Identical hashes mean identical structure.

Type 3 detection applies a similarity threshold to the normalized token sequences, and that gets expensive: naive pairwise comparison is O(n^2) in the number of code blocks.

Pyscn uses Locality-Sensitive Hashing (LSH) to cut that down. It hashes each block into multiple buckets using MinHash signatures, and blocks that land in the same bucket are likely similar. Only blocks that share a bucket get compared directly, which shrinks the comparison space a lot.

Type 4 is the hardest to catch, since the syntax differs. Pyscn handles it with semantic normalization: both [x for x in items if x > 0] and the equivalent for loop get normalized to a similar token sequence before comparison.

MCP Integration

Pyscn ships as a CLI tool and as an MCP (Model Context Protocol) server. The MCP server exposes the same analysis to AI coding assistants like Claude Code, Cursor, and ChatGPT.

So you can ask your assistant "find duplicate code in this directory" and it calls pyscn, gets structured analysis results back, and can reason about them and suggest refactors.

# Claude Code setup
claude mcp add pyscn-mcp uvx -- pyscn-mcp

The MCP server wraps the same Go binary the CLI uses, so there's one analysis engine and the results are identical either way. The Go microservice does the analysis, and a thin Python wrapper (pyscn on PyPI) handles distribution and MCP transport.

Performance

The 100K+ lines/sec figure comes from Go's concurrency model and tree-sitter's parsing speed. File parsing runs concurrently across goroutines, and since each file is independent the analysis parallelizes without much effort. tree-sitter parsers are generated C code called through Go's FFI, so the parsing runs at native speed.