🤖 Test Automator
A comprehensive, intelligent, and extensible testing automation platform powered by Large Language Models. Test Automator streamlines the process of generating, executing, and analyzing various types of software tests (unit, integration, end-to-end, API) for both web UI and backend logic.
✨ Features
🧠 LLM-Powered Test Generation
- Intelligent Analysis: Automatically analyzes your codebase to understand structure and dependencies
- Smart Test Cases: Generates comprehensive test scenarios including edge cases and error conditions
- Context-Aware: Understands your code patterns and generates idiomatic tests
- Multi-Language Support: Optimized for Python with extensible architecture
🔧 Comprehensive Test Types
🔬 Unit Testing
- Analyzes individual functions, methods, and classes
- Generates pytest test functions with proper fixtures
- Includes positive, negative, and edge case scenarios
- Handles both sync and async code patterns
🔗 Integration Testing
- Tests interactions between different modules and services
- Simulates real component interactions
- Uses appropriate mocking strategies
- Tests configuration and initialization flows
🌐 End-to-End (E2E) Testing
- Browser automation using Playwright and browser-use
- Simulates real user interactions
- Tests complete user workflows
- Captures screenshots and generates visual reports
🚀 API Testing
- Comprehensive HTTP endpoint testing
- Request/response validation
- Authentication and authorization testing
- Performance and timeout testing
📊 Intelligent Reporting
- LLM-Enhanced Analysis: AI-powered insights from test results
- Multi-Format Support: XML, JSON, and HTML report parsing
- Actionable Recommendations: Specific suggestions for improvement
- Risk Assessment: Identifies critical areas needing attention
🛠️ Installation
Prerequisites
- Python 3.11+
- Google API Key (for Gemini LLM)
- Claude Code or Cursor with MCP support
- Git
Quick Install
Windows (Native or WSL)
# Clone the repository
git clone https://github.com/your-repo/test-automator.git
cd test-automator
# Create virtual environment
python -m venv .venv
.venv\\Scripts\\activate # Windows
# or
source .venv/bin/activate # WSL/Linux
# Install dependencies
pip install -e .
# Install Playwright browsers
playwright install
Linux/macOS
# Clone and setup
git clone https://github.com/your-repo/test-automator.git
cd test-automator
# Create virtual environment
python3.11 -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install -e .
# Install Playwright browsers
playwright install
⚙️ Configuration
1. Get Google API Key
- Visit Google AI Studio
- Create a new API key
- Set environment variable:
# Windows set GOOGLE_API_KEY=your_api_key_here # Linux/macOS/WSL export GOOGLE_API_KEY=your_api_key_here
2. Configure Claude Code
Option A: CLI Configuration
claude mcp add test-automator "/path/to/test-automator/.venv/bin/test-automator" -e "GOOGLE_API_KEY=your_api_key"
Option B: Manual Configuration
Add to your Claude Code configuration (~/.claude.json):
{
"projects": {
"/your/project/path": {
"mcpServers": {
"test-automator": {
"type": "stdio",
"command": "/path/to/test-automator/.venv/bin/test-automator",
"env": {
"GOOGLE_API_KEY": "your_api_key_here"
}
}
}
}
}
}
Windows Path Examples
- Windows:
C:/path/to/test-automator/.venv/Scripts/test-automator.exe - WSL:
/home/username/test-automator/.venv/bin/test-automator
🚀 Usage
MCP Tools Available
generate_tests(code_path, test_type="all")
Generate intelligent tests for your codebase:
# Generate all test types
generate_tests("/path/to/your/code", "all")
# Generate specific test type
generate_tests("/path/to/your/api.py", "unit")
generate_tests("/path/to/your/project", "integration")
generate_tests("/path/to/your/webapp", "e2e")
generate_tests("/path/to/your/api", "api")
run_tests(test_type="all", target_path="tests/")
Execute generated tests:
# Run all tests
run_tests("all", "/path/to/tests")
# Run specific test type
run_tests("unit", "/path/to/tests")
analyze_test_report(report_path)
Get LLM-powered insights from test results:
analyze_test_report("/path/to/test_results.xml")
Example Workflow
# 1. Generate comprehensive tests
generate_tests("/home/user/my-project", "all")
# 2. Run the tests
run_tests("all", "/home/user/my-project/tests")
# 3. Analyze results
analyze_test_report("/home/user/my-project/tests/results/unit_results.xml")
🎯 Advanced Features
Large Codebase Support (100k-200k lines)
- Modular Analysis: Processes code in manageable chunks
- Incremental Testing: Generates tests incrementally for better performance
- Parallel Execution: Supports pytest-xdist for parallel test runs
- Smart Filtering: Focuses on testable units to avoid overwhelming LLM
Cross-Platform Compatibility
- Windows Native: Full support with proper path handling
- WSL Integration: Seamless Windows Subsystem for Linux support
- Linux/macOS: Native Unix support
- Event Loop Handling: Platform-specific async optimizations
Performance Optimizations
- Async Operations: Non-blocking test execution
- Batch Processing: Efficient handling of multiple test files
- Resource Management: Proper cleanup and memory management
- Timeout Handling: Configurable timeouts for different test types
📁 Project Structure
test-automator/
├── test_automator/
│ ├── __init__.py
│ ├── mcp_server.py # Main MCP server with tools
│ ├── test_generator.py # LLM-powered test generation
│ ├── test_runner.py # Cross-platform test execution
│ └── report_analyzer.py # AI-enhanced report analysis
├── pyproject.toml # Package configuration
└── README.md # This file
🔧 Troubleshooting
Common Issues
"GOOGLE_API_KEY not found"
# Set the environment variable
export GOOGLE_API_KEY="your_api_key_here"
# Or add to shell profile
echo 'export GOOGLE_API_KEY="your_api_key"' >> ~/.bashrc
"Playwright browsers not found"
# Install browsers
playwright install
# Install system dependencies (Linux)
playwright install-deps
Windows Permission Issues
# Ensure script execution is enabled
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
# Check virtual environment activation
.venv\\Scripts\\activate
WSL Display Issues (for E2E tests)
# Install X11 server for Windows
# Add to ~/.bashrc:
export DISPLAY=:0.0
🤝 Contributing
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
🙏 Acknowledgments
- Gemini LLM: Google's powerful language model for intelligent test generation
- Playwright: Cross-browser automation framework
- pytest: Robust Python testing framework
- MCP Protocol: Model Context Protocol for seamless AI integration
- Claude Code: AI-powered development environment