This is a tool meant for exploring the codebase of Github Copilot (the client side, not the model itself).
It’s work in progress. See the Journal for details on what’s been done so far and my observations.
There’s a blog post that explains the high level findings, and provides necessary code pointers.
You can access a version of the tool here.
This explores version 1.57.7193 of the VSCode extension of Copilot.
The webpage shows 3 panels:
Module Names: I’ve also named modules – some of them manually, while most of them were automatically named using codex in a few-shot manner. Some names are not unique, and I’ve not dealt with that. IDs of every module are unique though.
Module Categories: I’ve similarly also categorized modules automatically (again using codex and a bit of manual labeling). The categories are not perfect, but they serve as a very good guideline for segregating modules.
Manual annotations: Some of the modules were quite interesting, so I manually annotated them (added comments and renamed variables) to make them easier to understand. These are now directly viewable in the middle panel. Links below.
If you want to start browsing, below are some interesting starting points.
M_prompt_extractor.getPrompt(), then you know to look for
prompt-extractormodule in the right panel.
Some interesting modules I’ve found so far (these notes were written before I wrote the blog post, so some comments might be outdated):
2388 (described below) tweaks the prompt a bit depending on different completion modes. Copilot has 3 modes of completion (in copilot panel flow): “OPEN_COPILOT”, “TODO_QUICK_FIX”, “UNKNOWN_FUNCTION_QUICK_FIX”
class vthere is actually the InlineCompletionItemProvider.
getGhostText.. Depends on lots of other copilot modules. Only two hops away from the main module. The core functionality appears to the following:
If you want to play with the code (different transformations, different visualizations, etc.), you can run the tool locally and modify the code.
node index.js– this processes the copilot extension code and produces modules by doing some automatic reverse engineering.
Steps 2 and 3 are optional. They’re only needed if you modify the extension code or the transformation code. The repo already contains the processed modules.
pip install manifest-ml.
codeviz/.openai-api-keystoring your OpenAI API key. You can keep this empty if you’re not going to predict names and categories, or going to use the
select-snippet -> right-click -> Describe this snippetfeature (which doesn’t work that well anyway).
python3 codeviz/app.pyto start the app.
localhost:5000in your browser.
The tool supports 3 types of annotations: (a) module names (b) module categories (c) module code.
You can change the first two annotations (rename/recategorize modules) by the UI. These annotations persist in your localstorage. If you want to export them, you can use browser console to do so. I know this isn’t ideal, but hey this is a hacky repo. I doubt anyone will reannotate stuff seriously.
For module code – if you find a particular module interesting and want to add your comments, you can do that and save the annotated module under
codeviz/data/manually_annotated_modules/<module_id>.js. Finally, run
codeviz/compress_annotated_codes.js which takes all the annotated modules and puts them in a single file that’s loaded in the browser…so that your annotations show up in the UI.
But seriously, this is all on a best-effort basis, I might’ve misunderstood a few things and so the conclusions might be slightly off. If you find something, do raise an issue or a PR.