# Copilot Instructions for Autogen-MCP-Server

## Project Overview
This project implements a hierarchical MCP (Model Context Protocol) server system using AutoGen multi-agent architecture. The system acts as a coordinator for multiple specialized MCP servers through configurable AI agents.

## Architecture
- **Main MCP Server**: Entry point for MCP-capable clients (e.g., AnythingLLM)
- **AutoGen Framework**: Multi-agent coordination system
- **Moderator Agent**: Orchestrates and routes requests to specialized agents
- **Specialized Agents**: Each agent handles specific domains (Azure DevOps, Database, etc.)
- **MCP Server Integration**: Each specialized agent connects to dedicated MCP servers
- **Configuration-Driven**: YAML/JSON configuration for agent setup and routing

## Key Components

### 1. MCP Server (Entry Point)
- Implements MCP protocol
- Receives requests from clients
- Forwards to AutoGen moderator
- Returns responses back to clients

### 2. Moderator Agent
- Analyzes incoming requests
- Determines appropriate specialized agent
- Coordinates multi-agent conversations
- Aggregates and formats responses

### 3. Specialized Agents
- Domain-specific expertise
- Connected to specialized MCP servers
- Configurable via YAML/JSON
- Pluggable architecture

### 4. Configuration System
- Agent definitions in YAML/JSON
- Routing rules based on keywords/patterns
- MCP server connections
- Model configurations
- Fallback strategies

## Development Guidelines

### Virtual Environment Management
- **ALWAYS activate the .venv before running any terminal commands**
- Use `source .venv/bin/activate` (Linux/macOS) or `.venv\Scripts\activate` (Windows)
- Never run Python commands or install packages without the virtual environment activated
- All pip installs, python executions, and package management must be done within the .venv

### Project Cleanliness
- **Keep the project structure clean and organized**
- After creating new files, verify they are actually needed and remove unnecessary ones
- Do not create duplicate files in subdirectories
- **NEVER create copies of existing files in subdirectories**
- Before creating a new file, check if a similar file already exists
- Remove temporary files, unused imports, and dead code regularly
- Maintain a minimal and focused project structure

### File Management Rules
- **Check for existing files before creating new ones**
- Use existing files and extend them rather than creating duplicates
- If a file needs to be moved, use proper refactoring instead of copying
- Remove any files that become obsolete after refactoring
- Keep related functionality in the same file when appropriate

### Code Structure
```
src/
├── mcp_server/          # Main MCP server implementation
├── agents/              # AutoGen agent implementations
│   ├── moderator.py     # Moderator agent
│   ├── base_agent.py    # Base class for specialized agents
│   └── specialized/     # Specialized agent implementations
├── config/              # Configuration management
├── utils/               # Utility functions
└── main.py             # Entry point
```

### Configuration Format
```yaml
moderator:
  name: "AgentModerator"
  model: "gpt-4"
  system_message: "..."

agents:
  - name: "AzureDevOpsAgent"
    specialization: "azure_devops"
    model: "gpt-4"
    mcp_server:
      url: "mcp://azure-devops-server"
      tools: ["get_work_items", "create_pull_request"]

routing_rules:
  - keywords: ["azure", "devops", "pipeline"]
    agent: "AzureDevOpsAgent"
```

### Key Technologies
- **Python 3.12+**
- **AutoGen**: Multi-agent framework
- **MCP SDK**: Model Context Protocol implementation
- **Pydantic**: Configuration validation
- **PyYAML**: Configuration parsing
- **AsyncIO**: Asynchronous operations

### Development Phases
1. **Phase 1**: Basic MCP server + single specialized agent
2. **Phase 2**: Moderator agent + routing logic
3. **Phase 3**: Configuration system + multiple agents
4. **Phase 4**: Error handling + monitoring
5. **Phase 5**: Plugin system + hot-reload

### Testing Strategy
- Unit tests for each component
- Integration tests for agent communication
- Configuration validation tests
- End-to-end tests with real MCP clients
- Performance tests for multi-agent scenarios

### Error Handling
- Graceful degradation when agents/servers unavailable
- Timeout handling for long-running operations
- Fallback strategies in configuration
- Comprehensive logging and monitoring

### Performance Considerations
- Async/await for non-blocking operations
- Connection pooling for MCP servers
- Request caching where appropriate
- Token usage optimization

## Implementation Notes

### Agent Factory Pattern
Use factory pattern to create agents from configuration:
```python
class AgentFactory:
    @staticmethod
    def create_agent(config: AgentConfig) -> BaseAgent:
        # Create agent based on configuration
```

### MCP Server Registry
Maintain registry of available MCP servers:
```python
class MCPServerRegistry:
    def register_server(self, name: str, server: MCPServer):
        # Register MCP server
    
    def get_server(self, name: str) -> MCPServer:
        # Get MCP server by name
```

### Configuration Hot-Reload
Implement configuration reload without restart:
```python
class ConfigManager:
    def reload_config(self):
        # Reload configuration and update agents
```

## Security Considerations
- Validate all configuration inputs
- Sanitize requests between agents
- Implement proper authentication for MCP servers
- Log all agent interactions for audit

## Monitoring & Observability
- Request tracing across agent boundaries
- Performance metrics collection
- Error rate monitoring
- Token usage tracking

## Future Enhancements
- Web UI for configuration management
- Agent performance analytics
- Dynamic agent scaling
- Advanced routing algorithms
- Integration with more MCP servers