Design an AI Coding Assistant (Copilot) — System Design Practice | AmanAI Lab

Sheet›Design an AI Coding Assistant (Copilot)

GitHubMSFTGoogleAnth

45:00

Hard

The Problem

Problem

Design an AI coding assistant (like GitHub Copilot) that gives inline code completions inside the editor for 5 million developers.

The product should:

Latency at the keystroke level, repo-aware context retrieval, model serving & caching, acceptance feedback loops, and privacy/security.

Scale & Constraints

Must Cover0/10

Hints (if stuck)

💡 The whole budget is ~300ms: debounce keystrokes, cancel in-flight requests, and keep the inline model small/quantised.

💡 Fill-in-the-middle (prefix + suffix) is the right formulation for inline completion, not plain left-to-right.

💡 Build context from the current file plus a handful of relevant files (recently opened, imports, or embedding retrieval over the repo).

💡 KV/prefix caching matters: the same file prefix is re-sent as the user types — reuse it.

💡 Run a fast secret/license filter on the output; never block the happy path on heavy moderation.

0 words · auto-saved

Problem

Design an AI coding assistant (like GitHub Copilot) that gives inline code completions inside the editor for 5 million developers.

The product should:

•Suggest completions as the developer types, with very low latency

•Use surrounding code + relevant files from the repo as context

•Support multi-line "ghost text" suggestions and a chat sidebar

•Never leak secrets and respect code-licensing concerns

What you'll be assessed on