Building an Ollama-Powered GitHub Copilot Extension for VS Code

In software development, intelligent code completion tools like GitHub Copilot have become indispensable for developers seeking to enhance productivity and streamline their coding workflows. However, GitHub Copilot is a proprietary service, limiting customization and local control. Enter Ollama—a powerful command-line tool that allows you to run LLaMA-based language models locally. In this article, we’ll guide you through building your own GitHub Copilot-like extension for Visual Studio Code (VS Code), powered by Ollama. This approach not only grants you greater control over your development environment but also ensures that your code completions remain entirely on your machine.

Table of Contents

1. Understanding the Components
   • Ollama
   • VS Code Extension APIs
   • Overall Flow
2. Installing and Configuring Ollama
3. Scaffolding a VS Code Extension
4. Integrating Ollama with Your Extension
   • CLI Approach
   • HTTP Server Approach
5. Implementing an Inline Completion Provider
6. Testing and Debugging
7. Packaging and Distribution
8. Enhancing Your Extension
9. Final Thoughts

1. Understanding the Components

Before diving into development, it’s essential to grasp the key components involved in building an Ollama-powered GitHub Copilot extension.

Ollama

• What is Ollama?
  Ollama is a command-line tool designed to run LLaMA-based language models locally on your machine, supporting platforms like macOS and Linux. It provides a flexible interface for managing and interacting with various models, including code-specific variants.
• Key Features:
  • Local Model Hosting: Run powerful language models without relying on external cloud services.
  • Model Management: Easily install, update, and switch between different models.
  • Flexible Interfaces: Interact via the command line or through an HTTP API by running the Ollama daemon.

VS Code Extension APIs

• Inline Completion Provider API:
  VS Code offers an Inline Completion Provider API that allows extensions to provide context-aware code completions directly within the editor, mimicking the behavior of GitHub Copilot.
• Completion Item Provider:
  Alternatively, the Completion Item Provider can be used for more traditional IntelliSense-like suggestions, though for a Copilot-like experience, the Inline Completion Provider is preferable.

Overall Flow

The extension will follow this sequence to provide code completions:

1. User Interaction: The developer types code in the VS Code editor.
2. Context Gathering: The extension captures the current file content or relevant context around the cursor.
3. Prompt Construction: This context is formulated into a prompt sent to Ollama.
4. Model Invocation: Ollama processes the prompt using the selected language model and returns a code continuation.
5. Display Suggestion: The extension displays the returned code suggestion inline, resembling Copilot’s functionality.

2. Installing and Configuring Ollama

To begin, ensure that Ollama is correctly installed and configured on your system.

• Install Ollama:
  Follow the official installation instructions for your operating system.
• Download a Suitable Model:
  Choose a model optimized for code completions. For instance:

ollama pull codellama-7b

or

ollama pull llama2-7b

• Verify the Installation:
  Test the model to ensure it’s functioning correctly:

ollama run codellama-7b --prompt "Write a function in Python that prints 'Hello world'"

You should receive a coherent Python function in response.

3. Scaffolding a VS Code Extension

Leverage VS Code’s Yeoman generator to create the foundational structure of your extension.

• Install Yeoman and the Code Generator:

npm install -g yo generator-code

• Generate the Extension Scaffold:

yo code

During the setup prompts:

• Choose TypeScript or JavaScript (TypeScript is recommended for type safety).
• Provide a name, description, and other metadata as prompted.

This process will create a new directory containing the basic files and structure for your VS Code extension.

4. Integrating Ollama with Your Extension

With the scaffold in place, the next step is to enable communication between your extension and Ollama. There are two primary methods to achieve this: the CLI approach and the HTTP server approach.

CLI Approach

This method involves spawning a child process that executes the ollama run command with the appropriate parameters.

Implementation Steps:

• Import Required Modules:

import * as cp from 'child_process';

• Create a Function to Invoke Ollama:

async function getCompletionFromOllama(prompt: string): Promise<string> {
  return new Promise((resolve, reject) => {
    const child = cp.spawn('ollama', ['run', 'codellama-7b', '--prompt', prompt]);

    let output = '';
    child.stdout.on('data', (data) => {
      output += data.toString();
    });

    child.stderr.on('data', (data) => {
      console.error(`[ollama error]: ${data}`);
    });

    child.on('close', (code) => {
      if (code === 0) {
        resolve(output.trim());
      } else {
        reject(new Error(`ollama process exited with code ${code}`));
      }
    });
  });
}

Pros:

• Simple to implement.
• No need to manage an additional server process.

Cons:

• Potential performance overhead due to spawning a new process for each request.
• Limited scalability for high-frequency requests.

HTTP Server Approach

Alternatively, you can run Ollama in server mode and communicate via HTTP requests, which is more efficient for multiple or rapid interactions.

Implementation Steps:

Start Ollama in Server Mode:

ollama serve

Use an HTTP Client in Your Extension:

import fetch from 'node-fetch';

async function getCompletionFromOllama(prompt: string): Promise<string> {
  const response = await fetch('http://localhost:11411/api/generate', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      model: 'codellama-7b',
      prompt: prompt
    })
  });

  const data = await response.json();
  return data?.response?.trim() ?? '';
}

Pros:

• Lower latency for repeated requests.
• Easier to manage and scale.

Cons:

• Requires keeping the Ollama server running continuously.
• Slightly more complex setup.

Recommendation:
For a smoother and more efficient experience, especially if you anticipate frequent code completions, the HTTP server approach is preferable.

5. Implementing an Inline Completion Provider

To emulate GitHub Copilot’s inline suggestions, you’ll need to register an Inline Completion Provider within your extension. This provider will handle generating and displaying code completions based on the user’s input.

Step-by-Step Implementation:

• Modify src/extension.ts:

import * as vscode from 'vscode';

export function activate(context: vscode.ExtensionContext) {
  const provider: vscode.InlineCompletionProvider = {
    async provideInlineCompletionItems(
      document: vscode.TextDocument,
      position: vscode.Position,
      context: vscode.InlineCompletionContext,
      token: vscode.CancellationToken
    ): Promise<vscode.InlineCompletionList> {
      // 1. Gather context from the document
      const textBeforeCursor = document.getText(new vscode.Range(new vscode.Position(0, 0), position));

      // 2. Build a prompt for Ollama
      const prompt = buildPrompt(textBeforeCursor);

      try {
        // 3. Call Ollama to get completion
        const completion = await getCompletionFromOllama(prompt);

        // 4. Convert the raw text into an InlineCompletionItem
        const item = new vscode.InlineCompletionItem(completion, position);

        // Return as a list
        return { items: [item] };
      } catch (error) {
        console.error('Error fetching completion from Ollama:', error);
        return { items: [] };
      }
    }
  };

  context.subscriptions.push(
    vscode.languages.registerInlineCompletionItemProvider(
      { pattern: '**' }, // Apply to all file types or specify a language
      provider
    )
  );
}

// Optional: Customize the prompt
function buildPrompt(currentCode: string): string {
  return `You are a coding assistant. Continue this code:\n\n${currentCode}\n`;
}

// Placeholder for your Ollama integration (CLI or HTTP)
async function getCompletionFromOllama(prompt: string): Promise<string> {
  // Implement the CLI or HTTP logic here
  return '...'; // Replace with actual implementation
}

export function deactivate() {}

• Handling Context Size:
  LLaMA-based models have limitations on the context length. To optimize, consider sending only the last N lines or a relevant snippet of code instead of the entire file content.
• Fine-Tuning Generation Parameters:
  Adjust parameters like temperature to control the creativity of the completions. Lower temperatures yield more deterministic results, while higher values introduce more variability.
• Managing Stop Sequences:
  Define stop tokens or sequences to prevent the model from generating unwanted continuations, ensuring that code completions are clean and syntactically correct.

Best Practices:

• Asynchronous Handling:
  Ensure that calls to Ollama are handled asynchronously to prevent blocking the editor’s main thread.
• Error Handling:
  Implement robust error handling to gracefully manage scenarios where Ollama fails to provide a completion.
• User Feedback:
  Optionally, provide visual indicators (like loading spinners) to inform users that a completion is being fetched.

6. Testing and Debugging

Thorough testing ensures that your extension functions as intended and provides meaningful code completions.

1. Run the Extension:
   • Press F5 in VS Code to launch a new Extension Host window with your extension loaded.
2. Open a Test File:
   • Create or open a file in a language supported by your extension (e.g., Python, JavaScript).
3. Trigger Completions:
   • Start typing code and observe if inline suggestions appear. They should resemble GitHub Copilot’s gray text suggestions.
4. Debugging Tips:
   • Console Logs:
     Insert console.log statements within your provideInlineCompletionItems method to inspect variables like prompt and completion.
   • Error Monitoring:
     Check the Developer Tools (Help > Toggle Developer Tools) for any runtime errors or logs emitted by your extension.
   • Ollama Verification:
     Ensure that Ollama is running correctly and that models are loaded without issues.
5. Performance Assessment:
   • Evaluate the latency between typing and receiving suggestions. Optimize the prompt construction and model invocation processes to minimize delays.

7. Packaging and Distribution

Once your extension is polished and thoroughly tested, you can package and distribute it for personal use or to share with the broader community.

• Install the VS Code Extension Manager (vsce):

npm install -g vsce

• Package Your Extension:

vsce package

This command generates a .vsix file, which is the packaged version of your extension.

• Publishing Options:
  • Private Distribution: Share the .vsix file directly with collaborators or install it locally in your VS Code instance.
  • Public Marketplace: If you wish to publish your extension to the Visual Studio Marketplace, follow the publishing guidelines.

Installation of a .vsix File:

• In VS Code, press Ctrl+Shift+P (or Cmd+Shift+P on macOS) to open the Command Palette.
• Type Extensions: Install from VSIX... and select your packaged .vsix file.

8. Enhancing Your Extension

After establishing the foundational functionality, consider implementing additional features to elevate your extension’s capabilities.

Multi-File Context

• Objective:
  Enhance the context provided to Ollama by considering multiple files within the workspace, especially for projects with interconnected modules.
• Implementation:
  Aggregate relevant code snippets from related files and incorporate them into the prompt. Ensure adherence to the model’s context length limitations.

Chat-Like Interface

• Objective:
  Offer a more interactive experience akin to ChatGPT, allowing for conversational code assistance.
• Implementation:
  Utilize VS Code’s WebView API to create a custom chat interface or employ vscode.window.createQuickPick() for simpler interactions.

Advanced Prompt Engineering

• Objective:
  Refine the prompts sent to Ollama to achieve more accurate and contextually appropriate code completions.
• Implementation:
  Experiment with different prompt structures, instructions, and context formulations. Consider adding role-based prompts, such as specifying the assistant’s expertise in a particular programming language or framework.

Model Experimentation

• Objective:
  Identify the most effective LLaMA or CodeLlama variants for your use case.
• Implementation:
  Test various models provided by Ollama, comparing their completion quality, speed, and resource utilization. Select or allow users to choose models based on their specific needs.

User Configuration Options

• Objective:
  Empower users to customize aspects of the extension, such as model selection, completion behavior, and UI preferences.
• Implementation:
  Define configuration settings in the extension’s package.json and provide a settings UI within VS Code for users to adjust parameters.

9. Final Thoughts

Creating a GitHub Copilot-like extension powered by Ollama offers a compelling alternative for developers seeking greater control, customization, and privacy in their coding assistants. By leveraging the robust capabilities of VS Code’s extension APIs and the flexibility of Ollama’s local language models, you can craft a tailored tool that aligns perfectly with your development workflow.

While GitHub Copilot remains a closed-source solution with its own set of features and integrations, building your own extension opens avenues for personalization and optimization that proprietary tools may not offer. Moreover, running models locally ensures that your code and data remain within your control, addressing potential privacy and security concerns.

Embarking on this project not only enhances your technical skills in extension development and language model integration but also contributes to the growing ecosystem of open-source development tools. Whether for personal use or to share with the community, an Ollama-powered Copilot extension stands as a testament to the power of combining local machine learning capabilities with versatile development environments.