Deploying DeepSeek Coder and Qwen Models Locally

# Deploying DeepSeek Coder and Qwen Models Locally

Introduction

DeepSeek Coder and Qwen are state-of-the-art open-source LLMs specialized for code generation and general reasoning. Deploying them locally enables private coding assistants, code completion, and automated refactoring without sending code to external APIs. This tutorial covers local deployment using Ollama and vLLM, API integration, and coding workflows.

Learning Objectives

By the end of this tutorial, you will be able to:

• Deploy DeepSeek Coder models locally with Ollama
• Deploy Qwen models locally with vLLM
• Integrate local LLMs with IDEs (VS Code, JetBrains)
• Use OpenAI-compatible API for custom applications
• Optimize inference for code generation tasks
• Compare DeepSeek Coder and Qwen capabilities

Technical Background

Model Comparison

| Model | Parameters | Specialization | Context Window | Best For | |-------|-----------|----------------|----------------|----------| | DeepSeek-Coder-1.3B | 1.3B | Code | 16K | Simple completion | | DeepSeek-Coder-6.7B | 6.7B | Code | 16K | General coding | | DeepSeek-Coder-33B | 33B | Code | 16K | Complex programming | | Qwen-7B | 7B | General | 32K | Chat, reasoning | | Qwen-72B | 72B | General | 32K | Complex tasks | | Qwen-1.5-72B | 72B | General | 32K | Advanced reasoning |

Local Code LLM Architecture

flowchart TB
    IDE[IDE / Editor] --> Extension[AI Extension]
    Extension -->|HTTP API| LocalLLM[Local LLM Server]
    LocalLLM -->|Model| Model[(Model Weights)]
    LocalLLM -->|GPU| GPU[GPU Acceleration]
    
    subgraph Local Server
    vLLM[vLLM Server]
    Ollama[Ollama Server]
    end
    
    LocalLLM --> vLLM & Ollama

Prerequisites

• Linux server (Ubuntu 22.04 LTS)
• NVIDIA GPU with 16+ GB VRAM
• 32+ GB RAM
• 200+ GB storage
• Docker and Docker Compose

Hardware and Software Requirements

| Component | Requirement | |-----------|-------------| | OS | Ubuntu 22.04 LTS | | GPU | NVIDIA RTX 4090 / A100 / H100 | | VRAM | 24 GB for 33B model | | RAM | 64 GB recommended | | Software | Docker, vLLM, Ollama, Python 3.10 |

Step-by-Step Implementation

Phase 1: Deploy DeepSeek Coder with Ollama

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull DeepSeek Coder models
ollama pull deepseek-coder:6.7b
ollama pull deepseek-coder:33b

# Run DeepSeek Coder
ollama run deepseek-coder:6.7b "Write a Python function to fibonacci"

# Test via API
curl http://localhost:11434/api/generate -d '{
  "model": "deepseek-coder:6.7b",
  "prompt": "Write a Python function to calculate fibonacci",
  "stream": false,
  "temperature": 0.1
}'

Phase 2: Deploy Qwen with vLLM

# Install vLLM
pip install vllm

# Start vLLM server with Qwen
python -m vllm.entrypoints.api_server \
  --model Qwen/Qwen-72B-Chat \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --port 8000

# Test Qwen via API
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "Qwen/Qwen-72B-Chat",
  "messages": [{"role": "user", "content": "Explain quantum computing"}],
  "temperature": 0.7
}'

Phase 3: Deploy with Docker Compose

# docker-compose.yml
version: '3.8'
services:
  deepseek-coder:
    image: ollama/ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama-data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: unless-stopped
    command: serve

  qwen-vllm:
    image: vllm/vllm-openai:latest
    ports:
      - "8000:8000"
    environment:
      - MODEL_NAME=Qwen/Qwen-72B-Chat
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 2
              capabilities: [gpu]
    restart: unless-stopped
    command: --model Qwen/Qwen-72B-Chat --tensor-parallel-size 2 --max-model-len 32768

volumes:
  ollama-data:

Phase 4: Integrate with VS Code

#### Continue Extension Configuration

// ~/.continue/config.json
{
  "models": [
    {
      "title": "DeepSeek Coder",
      "provider": "ollama",
      "model": "deepseek-coder:6.7b",
      "baseUrl": "http://localhost:11434"
    },
    {
      "title": "Qwen",
      "provider": "openai",
      "model": "Qwen/Qwen-72B-Chat",
      "baseUrl": "http://localhost:8000/v1",
      "apiKey": "not-required"
    }
  ]
}

Phase 5: Code Generation Workflows

#### Code Completion Example

# Example: Using local LLM for code generation
import requests

def generate_code(prompt, model="deepseek-coder:6.7b"):
    response = requests.post("http://localhost:11434/api/generate", json={
        "model": model,
        "prompt": prompt,
        "stream": False,
        "temperature": 0.1,
        "stop": ["```", "# End"]
    })
    return response.json()["response"]

# Generate a Python function
code = generate_code("Write a Python function to parse JSON with error handling")
print(code)

#### Code Review Example

curl http://localhost:11434/api/generate -d '{
  "model": "deepseek-coder:33b",
  "prompt": "Review this code for bugs and security issues:\n\n```python\nimport pickle\ndata = pickle.load(open(\"user_data.pkl\", \"rb\"))\n```",
  "stream": false,
  "temperature": 0.3
}'

Expected Output

• DeepSeek Coder running locally via Ollama
• Qwen running locally via vLLM
• OpenAI-compatible APIs for both models
• IDE integration for coding assistance
• Automated code generation and review workflows

Verification Procedures

| Check | Command | Expected Result | |-------|---------|-----------------| | Ollama models | `ollama list` | DeepSeek Coder models listed | | vLLM server | `curl localhost:8000/v1/models` | Qwen model listed | | Code generation | API call with code prompt | Valid code returned | | IDE integration | Continue extension connected | Completions appear in editor |

Troubleshooting Scenarios

| Issue | Diagnosis | Resolution | |-------|-----------|------------| | vLLM out of memory | Model too large for VRAM | Reduce tensor-parallel-size; use smaller model | | Ollama connection refused | Service not running | Start Ollama service | | Slow code generation | Running on CPU | Verify GPU is being used | | API timeout | Model loading | Wait for model to load; increase timeout |

Best Practices

• **Use temperature 0.1-0.2 for code:** Lower temperature for deterministic code
• **Use stop sequences:** Stop generation at code block boundaries
• **Provide context:** Include relevant files in prompts
• **Validate generated code:** Always test AI-generated code
• **Use appropriate model:** DeepSeek Coder for code, Qwen for general tasks

Security Considerations

• **Code privacy:** Local deployment keeps code private
• **No data leakage:** Code never sent to external APIs
• **Input validation:** Sanitize prompts before sending to model
• **Output validation:** Review and test all generated code
• **API authentication:** Secure local API endpoints

Conclusion

DeepSeek Coder and Qwen provide powerful local alternatives to cloud-based coding assistants. By deploying them with Ollama or vLLM, teams can build private AI coding infrastructure that integrates with IDEs and custom applications. Local deployment ensures code privacy while providing state-of-the-art code generation capabilities.

References

• DeepSeek Coder: https://github.com/deepseek-ai/DeepSeek-Coder
• Qwen: https://github.com/QwenLM/Qwen
• Ollama: https://ollama.com/library
• vLLM: https://github.com/vllm-project/vllm
• Continue: https://continue.dev/