Compare commits

..

3 Commits

Author SHA1 Message Date
5bd7384d7a Update Copilot instructions to emphasize FastMCP as the mandatory protocol for MCP server implementation 2025-07-06 23:28:18 +02:00
c8cdb3cc28 Enhance project structure and documentation
- Updated code structure in documentation for clarity.
- Added example configuration file for AutoGen-MCP-Server.
- Created detailed logs directory documentation.
- Expanded development requirements with additional tools.
- Updated core requirements with new dependencies.
- Added module docstrings for better code understanding.
- Introduced a web UI template for configuration management.
- Implemented integration and unit test structure.
2025-07-06 23:27:07 +02:00
a774772fe1 Add comprehensive documentation for Autogen-MCP-Server project 2025-07-06 23:20:16 +02:00
17 changed files with 702 additions and 1 deletions

199
.copilot-instructions.md Normal file
View File

@@ -0,0 +1,199 @@
# Copilot Instructions for Autogen-MCP-Server
## Project Overview
This project implements a hierarchical MCP (Model Context Protocol) server system using AutoGen multi-agent architecture. The system acts as a coordinator for multiple specialized MCP servers through configurable AI agents.
## Architecture
- **Main MCP Server**: Entry point for MCP-capable clients (e.g., AnythingLLM)
- **AutoGen Framework**: Multi-agent coordination system
- **Moderator Agent**: Orchestrates and routes requests to specialized agents
- **Specialized Agents**: Each agent handles specific domains (Azure DevOps, Database, etc.)
- **MCP Server Integration**: Each specialized agent connects to dedicated MCP servers
- **Configuration-Driven**: YAML/JSON configuration for agent setup and routing
- **FastMCP**: The main MCP server implementation MUST use FastMCP as the protocol and server library. No other MCP server implementation is allowed for the entry point.
## Key Components
### 1. MCP Server (Entry Point)
- Implements MCP protocol using **FastMCP** (mandatory)
- Receives requests from clients
- Forwards to AutoGen moderator
- Returns responses back to clients
### 2. Moderator Agent
- Analyzes incoming requests
- Determines appropriate specialized agent
- Coordinates multi-agent conversations
- Aggregates and formats responses
### 3. Specialized Agents
- Domain-specific expertise
- Connected to specialized MCP servers
- Configurable via YAML/JSON
- Pluggable architecture
### 4. Configuration System
- Agent definitions in YAML/JSON
- Routing rules based on keywords/patterns
- MCP server connections
- Model configurations
- Fallback strategies
## Development Guidelines
### Virtual Environment Management
- **ALWAYS activate the .venv before running any terminal commands**
- Use `source .venv/bin/activate` (Linux/macOS) or `.venv\Scripts\activate` (Windows)
- Never run Python commands or install packages without the virtual environment activated
- All pip installs, python executions, and package management must be done within the .venv
### Project Cleanliness
- **Keep the project structure clean and organized**
- After creating new files, verify they are actually needed and remove unnecessary ones
- Do not create duplicate files in subdirectories
- **NEVER create copies of existing files in subdirectories**
- Before creating a new file, check if a similar file already exists
- Remove temporary files, unused imports, and dead code regularly
- Maintain a minimal and focused project structure
### File Management Rules
- **Check for existing files before creating new ones**
- Use existing files and extend them rather than creating duplicates
- If a file needs to be moved, use proper refactoring instead of copying
- Remove any files that become obsolete after refactoring
- Keep related functionality in the same file when appropriate
### Code Structure
```
├── src/ # Source code
│ ├── mcp_server/ # Main MCP server implementation
│ ├── agents/ # AutoGen agent implementations
│ │ ├── moderator.py # Moderator agent
│ │ ├── base_agent.py # Base class for specialized agents
│ │ └── specialized/ # Specialized agent implementations
│ ├── config/ # Configuration management
│ ├── utils/ # Utility functions
│ ├── web_ui/ # Web UI for configuration management
│ │ ├── api/ # REST API endpoints
│ │ ├── static/ # CSS, JS, images
│ │ └── templates/ # HTML templates
│ └── main.py # Entry point
├── config/ # Configuration files
│ ├── example-config.yml # Example configuration
│ └── config.yml # Actual configuration (not in git)
├── tests/ # Test files
│ ├── unit/ # Unit tests
│ └── integration/ # Integration tests
├── logs/ # Log files (not in git)
├── docs/ # Documentation
├── requirements.txt # Python dependencies
├── requirements-dev.txt # Development dependencies
└── .venv/ # Virtual environment
```
### Configuration Format
```yaml
moderator:
name: "AgentModerator"
model: "gpt-4"
system_message: "..."
agents:
- name: "AzureDevOpsAgent"
specialization: "azure_devops"
model: "gpt-4"
mcp_server:
url: "mcp://azure-devops-server"
tools: ["get_work_items", "create_pull_request"]
routing_rules:
- keywords: ["azure", "devops", "pipeline"]
agent: "AzureDevOpsAgent"
```
### Key Technologies
- **Python 3.12+** (3.13+ preferred when available)
- **FastMCP**: Mandatory for the main MCP server implementation
- **AutoGen**: Multi-agent framework
- **MCP SDK**: Model Context Protocol implementation
- **FastAPI**: Modern web framework for Web UI
- **Pydantic**: Configuration validation
- **PyYAML**: Configuration parsing
- **AsyncIO**: Asynchronous operations
- **SQLAlchemy**: Database ORM (for configuration storage)
- **Uvicorn**: ASGI server
### Development Phases
1. **Phase 1**: Basic MCP server + single specialized agent
2. **Phase 2**: Moderator agent + routing logic
3. **Phase 3**: Configuration system + multiple agents
4. **Phase 4**: Error handling + monitoring
5. **Phase 5**: Plugin system + hot-reload
### Testing Strategy
- Unit tests for each component
- Integration tests for agent communication
- Configuration validation tests
- End-to-end tests with real MCP clients
- Performance tests for multi-agent scenarios
### Error Handling
- Graceful degradation when agents/servers unavailable
- Timeout handling for long-running operations
- Fallback strategies in configuration
- Comprehensive logging and monitoring
### Performance Considerations
- Async/await for non-blocking operations
- Connection pooling for MCP servers
- Request caching where appropriate
- Token usage optimization
## Implementation Notes
### Agent Factory Pattern
Use factory pattern to create agents from configuration:
```python
class AgentFactory:
@staticmethod
def create_agent(config: AgentConfig) -> BaseAgent:
# Create agent based on configuration
```
### MCP Server Registry
Maintain registry of available MCP servers:
```python
class MCPServerRegistry:
def register_server(self, name: str, server: MCPServer):
# Register MCP server
def get_server(self, name: str) -> MCPServer:
# Get MCP server by name
```
### Configuration Hot-Reload
Implement configuration reload without restart:
```python
class ConfigManager:
def reload_config(self):
# Reload configuration and update agents
```
## Security Considerations
- Validate all configuration inputs
- Sanitize requests between agents
- Implement proper authentication for MCP servers
- Log all agent interactions for audit
## Monitoring & Observability
- Request tracing across agent boundaries
- Performance metrics collection
- Error rate monitoring
- Token usage tracking
## Future Enhancements
- Web UI for configuration management
- Agent performance analytics
- Dynamic agent scaling
- Advanced routing algorithms
- Integration with more MCP servers

183
README.md
View File

@@ -1,3 +1,184 @@
# Autogen-MCP-Server
Ein MCP Server, der eine Gruppe von LLM und MCP Servern koppelt
Ein hierarchisches MCP-System mit AutoGen Multi-Agent-Architektur für die Koordination spezialisierter KI-Agenten.
## 🎯 Projektziel
Dieses Projekt implementiert einen intelligenten MCP (Model Context Protocol) Server, der als Einstiegspunkt für ein System aus spezialisierten KI-Agenten fungiert. Das System ermöglicht es, komplexe Anfragen automatisch an die am besten geeigneten Experten-Agenten zu routen und koordinierte Antworten zu liefern.
## 🏗️ Architektur
```
MCP Client (AnythingLLM)
Hauptserver (MCP Server)
Moderator-Agent (AutoGen)
Spezialisierte Agenten
Spezialisierte MCP Server
```
### Komponenten
- **🚪 Hauptserver**: MCP-konformer Einstiegspunkt für Clients
- **🎭 Moderator-Agent**: Intelligente Koordination und Routing
- **🔧 Spezialisierte Agenten**: Experten für spezifische Domänen (Azure DevOps, Datenbanken, etc.)
- **⚙️ MCP-Server**: Spezialisierte Backend-Services für jeden Agenten
- **📋 Konfigurationssystem**: YAML/JSON-basierte Agenten-Konfiguration
## 🌟 Hauptfeatures
### ✅ Intelligente Anfragerouting
- Automatische Erkennung der passenden Experten-Agenten
- Keyword-basierte und semantische Routing-Regeln
- Fallback-Strategien für unbekannte Anfragen
### ✅ Konfigurierbare Agenten
- YAML/JSON-Konfiguration für alle Agenten
- Plug-and-Play-Architektur für neue Domänen
- Hot-Reload für Konfigurationsänderungen
### ✅ Enterprise-Ready
- Umfassende Fehlerbehandlung
- Logging und Monitoring
- Skalierbare Architektur
- Sicherheitsfeatures
## 🚀 Anwendungsfall
**Beispiel-Workflow:**
1. Benutzer stellt Frage zu Azure DevOps in AnythingLLM
2. AnythingLLM sendet Anfrage an unseren MCP Server
3. Moderator-Agent analysiert die Anfrage
4. Azure DevOps-Spezialist wird aktiviert
5. Spezialist nutzt Azure DevOps MCP Server für Informationen
6. Antwort wird über die Kette zurück an den Benutzer geliefert
## 📝 Konfigurationsbeispiel
```yaml
# autogen-config.yml
moderator:
name: "AgentModerator"
model: "gpt-4"
system_message: "Du koordinierst spezialisierte Agenten..."
agents:
- name: "AzureDevOpsAgent"
specialization: "azure_devops"
model: "gpt-4"
mcp_server:
url: "mcp://azure-devops-server"
tools: ["get_work_items", "create_pull_request"]
- name: "DatabaseAgent"
specialization: "database"
model: "gpt-3.5-turbo"
mcp_server:
url: "mcp://database-server"
tools: ["query_db", "get_schema"]
routing_rules:
- keywords: ["azure", "devops", "pipeline", "build"]
agent: "AzureDevOpsAgent"
- keywords: ["database", "sql", "query", "table"]
agent: "DatabaseAgent"
```
## 🛠️ Technologie-Stack
- **Python 3.13+**
- **AutoGen**: Multi-Agent Framework
- **MCP SDK**: Model Context Protocol
- **Pydantic**: Konfigurationsvalidierung
- **PyYAML**: Konfigurationsparsing
- **AsyncIO**: Asynchrone Operationen
## 🎯 Entwicklungsphasen
### Phase 1: MVP (Minimum Viable Product)
- [x] Grundlegende MCP Server Implementierung
- [x] Einfacher Spezialist-Agent (Azure DevOps)
- [x] Basis-Konfigurationssystem
### Phase 2: Moderator-System
- [ ] Moderator-Agent Implementierung
- [ ] Intelligente Anfragerouting
- [ ] Multi-Agent-Koordination
### Phase 3: Erweiterte Konfiguration
- [ ] Vollständiges YAML/JSON-Konfigurationssystem
- [ ] Mehrere spezialisierte Agenten
- [ ] Hot-Reload-Funktionalität
### Phase 4: Enterprise-Features
- [ ] Umfassende Fehlerbehandlung
- [ ] Logging und Monitoring
- [ ] Performance-Optimierungen
### Phase 5: Erweiterungen
- [ ] Plugin-System für neue Agenten
- [ ] Web-UI für Konfiguration
- [ ] Analytics und Metriken
## 🔧 Installation & Setup
```bash
# Repository klonen
git clone https://github.com/KonnosPB/Autogen-MCP-Server.git
cd Autogen-MCP-Server
# Abhängigkeiten installieren
pip install -r requirements.txt
# Konfiguration anpassen
cp config/example-config.yml config/config.yml
# config.yml bearbeiten...
# Server starten
python src/main.py
```
## 📊 Vorteile
### Für Entwickler
- **Modulare Architektur**: Einfache Erweiterung um neue Domänen
- **Konfigurierbar**: Keine Code-Änderungen für neue Agenten
- **Testbar**: Isolierte Komponenten für einfache Tests
### Für Unternehmen
- **Skalierbar**: Neue Experten-Domänen ohne Neudeployment
- **Wartbar**: Zentrale Konfiguration
- **Ausfallsicher**: Fallback-Strategien und Fehlerbehandlung
### Für Benutzer
- **Intelligent**: Automatische Weiterleitung an die richtige Expertise
- **Schnell**: Direkte Verbindung zu spezialisierten Services
- **Umfassend**: Ein Interface für viele verschiedene Domänen
## 🤝 Beitragen
Wir freuen uns über Beiträge! Siehe [CONTRIBUTING.md](CONTRIBUTING.md) für Details.
## 📄 Lizenz
Dieses Projekt ist unter der MIT-Lizenz lizenziert - siehe [LICENSE](LICENSE) für Details.
## 🌍 Roadmap
- **Q1 2025**: MVP mit Azure DevOps Integration
- **Q2 2025**: Mehrere spezialisierte Agenten
- **Q3 2025**: Enterprise-Features und Monitoring
- **Q4 2025**: Plugin-System und Web-UI
## 📞 Support
- **Issues**: GitHub Issues für Bugs und Feature-Requests
- **Diskussionen**: GitHub Discussions für Fragen und Ideen
- **Dokumentation**: [Wiki](https://github.com/KonnosPB/Autogen-MCP-Server/wiki)
---
*Dieses Projekt kombiniert die Kraft von AutoGen Multi-Agent-Systemen mit der Flexibilität des Model Context Protocol für eine neue Generation intelligenter Assistenten.*

143
config/example-config.yml Normal file
View File

@@ -0,0 +1,143 @@
# Beispiel-Konfiguration für Autogen-MCP-Server
# Kopiere diese Datei zu config.yml und passe sie an
# Moderator Agent Konfiguration
moderator:
name: "AgentModerator"
model: "gpt-4"
system_message: |
Du bist ein intelligenter Moderator-Agent, der Anfragen an spezialisierte Agenten weiterleitet.
Analysiere die Anfrage und bestimme den besten Spezialisten für die Aufgabe.
Koordiniere die Kommunikation zwischen den Agenten und formatiere die finale Antwort.
# Modell-spezifische Konfiguration
model_config:
temperature: 0.7
max_tokens: 1000
timeout: 30
# Spezialisierte Agenten
agents:
- name: "AzureDevOpsAgent"
specialization: "azure_devops"
description: "Spezialist für Azure DevOps, Build-Pipelines und Deployment"
model: "gpt-4"
system_message: |
Du bist ein Azure DevOps Spezialist. Du hilfst bei:
- Work Items und Backlogs
- Build und Release Pipelines
- Git Repositories und Pull Requests
- Deployment Strategien
# MCP Server Verbindung
mcp_server:
url: "mcp://localhost:8001/azure-devops"
timeout: 60
tools:
- "get_work_items"
- "create_pull_request"
- "get_build_status"
- "trigger_pipeline"
model_config:
temperature: 0.3
max_tokens: 1500
- name: "DatabaseAgent"
specialization: "database"
description: "Spezialist für Datenbank-Operationen und SQL"
model: "gpt-3.5-turbo"
system_message: |
Du bist ein Datenbank-Spezialist. Du hilfst bei:
- SQL Queries und Optimierung
- Datenbank Design und Schema
- Performance-Analyse
- Backup und Recovery
mcp_server:
url: "mcp://localhost:8002/database"
timeout: 45
tools:
- "query_database"
- "get_schema"
- "analyze_performance"
- "backup_database"
model_config:
temperature: 0.2
max_tokens: 1200
# Routing-Regeln für automatische Agent-Auswahl
routing_rules:
- keywords: ["azure", "devops", "pipeline", "build", "deployment", "pull request", "work item"]
agent: "AzureDevOpsAgent"
confidence: 0.8
- keywords: ["database", "sql", "query", "schema", "table", "performance", "backup"]
agent: "DatabaseAgent"
confidence: 0.8
- keywords: ["datenbank", "abfrage", "tabelle", "schema"]
agent: "DatabaseAgent"
confidence: 0.7
# Fallback-Strategien
fallback:
default_agent: "moderator" # Wenn kein spezieller Agent gefunden wird
timeout_seconds: 120
retry_attempts: 3
# Wenn ein Agent nicht verfügbar ist
unavailable_strategy: "fallback_to_moderator"
# Fehlerbehandlung
error_responses:
agent_timeout: "Entschuldigung, der angefragte Service ist momentan nicht verfügbar."
mcp_server_error: "Es gab einen Fehler beim Zugriff auf die Backend-Services."
general_error: "Ein unerwarteter Fehler ist aufgetreten. Bitte versuchen Sie es später erneut."
# Logging-Konfiguration
logging:
level: "INFO"
format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
file: "logs/autogen-mcp-server.log"
max_file_size: "10MB"
backup_count: 5
# Spezielle Logger
loggers:
agents: "DEBUG"
mcp_server: "INFO"
config: "WARNING"
# Web UI Konfiguration (optional)
web_ui:
enabled: true
host: "localhost"
port: 8000
debug: false
# Sicherheit
secret_key: "your-secret-key-here" # Ändern Sie dies in Produktion!
# Authentifizierung (optional)
auth:
enabled: false
username: "admin"
password: "password" # Ändern Sie dies in Produktion!
# Performance-Einstellungen
performance:
# Maximale gleichzeitige Anfragen
max_concurrent_requests: 10
# Caching
cache:
enabled: true
ttl_seconds: 300
max_size: 1000
# Connection Pooling für MCP Server
mcp_connection_pool:
max_connections: 5
timeout: 30

27
logs/README.md Normal file
View File

@@ -0,0 +1,27 @@
# Logs Directory
This directory contains log files for the Autogen-MCP-Server.
## Log Files
- `autogen-mcp-server.log` - Main application log
- `agents.log` - Agent-specific logs
- `mcp-server.log` - MCP server communication logs
- `web-ui.log` - Web UI access and error logs
## Log Rotation
Log files are automatically rotated when they reach 10MB in size.
Up to 5 backup files are kept.
## Log Levels
- `DEBUG` - Detailed diagnostic information
- `INFO` - General information about system operation
- `WARNING` - Warning messages about potential issues
- `ERROR` - Error messages about failures
- `CRITICAL` - Critical error messages
## Configuration
Log configuration can be adjusted in `config/config.yml` under the `logging` section.

21
requirements-dev.txt Normal file
View File

@@ -0,0 +1,21 @@
# Development Requirements - zusätzlich zu requirements.txt
# Code Quality
pre-commit>=3.5.0
bandit>=1.7.5
# Testing
pytest-xdist>=3.3.0 # Parallel testing
pytest-mock>=3.12.0
httpx>=0.25.0 # For testing FastAPI endpoints
# Development Tools
ipython>=8.16.0
jupyter>=1.0.0
# Documentation
mkdocs-mermaid2-plugin>=1.1.0
# Type Checking
types-PyYAML>=6.0.0
types-requests>=2.31.0

47
requirements.txt Normal file
View File

@@ -0,0 +1,47 @@
# Requirements for Autogen-MCP-Server
# Core MCP and AutoGen
mcp>=1.0.0
autogen-agentchat>=0.2.0
# Configuration and Validation
pydantic>=2.5.0
pydantic-settings>=2.1.0
PyYAML>=6.0.1
# Async and Networking
aiohttp>=3.9.0
asyncio-mqtt>=0.16.0
# Web UI Framework (FastAPI chosen for modern async support)
fastapi>=0.104.0
uvicorn[standard]>=0.24.0
jinja2>=3.1.0
# Database (for configuration storage if needed)
sqlalchemy>=2.0.0
alembic>=1.12.0
# Logging and Monitoring
structlog>=23.2.0
prometheus-client>=0.19.0
# Security
python-jose[cryptography]>=3.3.0
passlib[bcrypt]>=1.7.4
# Development and Testing
pytest>=7.4.0
pytest-asyncio>=0.21.0
pytest-cov>=4.1.0
black>=23.9.0
ruff>=0.1.0
mypy>=1.7.0
# Documentation
mkdocs>=1.5.0
mkdocs-material>=9.4.0
# Optional: For advanced features
redis>=5.0.0 # For caching and session management
celery>=5.3.0 # For background tasks

3
src/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Autogen-MCP-Server: A hierarchical MCP system with AutoGen multi-agent architecture.
"""

3
src/agents/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
AutoGen agents implementation - Moderator and specialized agents.
"""

View File

@@ -0,0 +1,3 @@
"""
Specialized agents for different domains (Azure DevOps, Database, etc.).
"""

3
src/config/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Configuration management - YAML/JSON parsing and validation.
"""

View File

@@ -0,0 +1,3 @@
"""
MCP Server implementation - Entry point for MCP-capable clients.
"""

3
src/utils/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Utility functions - Logging, helpers, and shared functionality.
"""

3
src/web_ui/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Web UI for configuration management - Flask/FastAPI based interface.
"""

View File

@@ -0,0 +1,3 @@
"""
Web UI API endpoints - REST API for configuration management.
"""

View File

@@ -0,0 +1,53 @@
<!DOCTYPE html>
<html lang="de">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Autogen-MCP-Server - Konfiguration</title>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
margin: 0;
padding: 20px;
background-color: #f5f5f5;
}
.container {
max-width: 1200px;
margin: 0 auto;
background: white;
padding: 30px;
border-radius: 8px;
box-shadow: 0 2px 10px rgba(0,0,0,0.1);
}
h1 {
color: #333;
text-align: center;
margin-bottom: 30px;
}
.coming-soon {
text-align: center;
color: #666;
font-size: 18px;
padding: 40px;
}
</style>
</head>
<body>
<div class="container">
<h1>🚀 Autogen-MCP-Server Web UI</h1>
<div class="coming-soon">
<h2>🔧 Konfiguration Interface</h2>
<p>Die Web-basierte Konfigurationsoberfläche befindet sich in Entwicklung.</p>
<p>Features in Planung:</p>
<ul style="text-align: left; display: inline-block;">
<li>📝 Agent-Konfiguration bearbeiten</li>
<li>🔍 Live-Monitoring der Agenten</li>
<li>📊 Performance-Metriken</li>
<li>🔄 Hot-Reload von Konfigurationen</li>
<li>📋 Routing-Regeln verwalten</li>
<li>🔐 Sicherheitseinstellungen</li>
</ul>
</div>
</div>
</body>
</html>

View File

@@ -0,0 +1,3 @@
"""
Integration tests for component interactions.
"""

3
tests/unit/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""
Unit tests for individual components.
"""