picoclaw/docs/design/provider-refactoring-tests.md

Provider Architecture Refactoring - Test Suite Summary

PRD: tasks/prd-provider-refactoring.md

This document summarizes the complete test suite designed for the Provider architecture refactoring.

Test File Structure

pkg/
├── config/
│   ├── model_config_test.go      # US-001, US-002: ModelConfig struct and GetModelConfig tests
│   └── migration_test.go         # US-003: Backward compatibility and migration tests
├── providers/
│   ├── registry_test.go          # US-006: Load balancing tests
│   ├── integration_test.go       # E2E integration tests
│   └── factory/
│       └── factory_test.go       # US-004, US-005: Provider factory tests

---

Test Case Checklist

1. pkg/config/model_config_test.go - Configuration Parsing Tests

Test Name	Purpose	PRD Reference
TestModelConfig_Parsing	Verify ModelConfig JSON parsing	US-001
TestModelConfig_ModelListInConfig	Verify model_list parsing in Config	US-001
TestModelConfig_Validation	Verify required field validation	US-001
TestConfig_GetModelConfig_Found	Verify GetModelConfig finds model	US-002
TestConfig_GetModelConfig_NotFound	Verify GetModelConfig returns error	US-002
TestConfig_GetModelConfig_EmptyModelList	Verify empty model_list handling	US-002
TestConfig_BackwardCompatibility_ProvidersToModelList	Verify old config conversion	US-003
TestConfig_DeprecationWarning	Verify deprecation warning	US-003
TestModelConfig_ProtocolExtraction	Verify protocol prefix extraction	US-004
TestConfig_ModelNameUniqueness	Verify model_name uniqueness	US-001

2. pkg/config/migration_test.go - Migration Tests

Test Name	Purpose	PRD Reference
TestConvertProvidersToModelList_OpenAI	OpenAI config conversion	US-003
TestConvertProvidersToModelList_Anthropic	Anthropic config conversion	US-003
TestConvertProvidersToModelList_MultipleProviders	Multiple provider conversion	US-003
TestConvertProvidersToModelList_EmptyProviders	Empty providers handling	US-003
TestConvertProvidersToModelList_GitHubCopilot	GitHub Copilot conversion	US-003
TestConvertProvidersToModelList_Antigravity	Antigravity conversion	US-003
TestGenerateModelName_*	Model name generation	US-003
TestHasProvidersConfig_*	Detect old config existence	US-003
TestValidateMigration_*	Migration validation	US-003
TestMigrateConfig_DryRun	Dry run migration	US-003
TestMigrateConfig_Actual	Actual migration	US-003

3. pkg/providers/registry_test.go - Load Balancing Tests

Test Name	Purpose	PRD Reference
TestModelRegistry_SingleConfig	Single config returns same result	US-006
TestModelRegistry_RoundRobinSelection	3-config round-robin selection	US-006
TestModelRegistry_RoundRobinTwoConfigs	2-config round-robin selection	US-006
TestModelRegistry_ConcurrentAccess	Concurrent access thread safety	US-006
TestModelRegistry_RaceDetection	Data race detection	US-006
TestModelRegistry_ModelNotFound	Model not found error	US-006
TestModelRegistry_EmptyRegistry	Empty registry handling	US-006
TestModelRegistry_MultipleModels	Multiple model registration	US-006
TestModelRegistry_MixedSingleAndMultiple	Single/multiple config mix	US-006
TestModelRegistry_CaseSensitiveModelNames	Case sensitivity	US-006

4. pkg/providers/factory/factory_test.go - Provider Factory Tests

Test Name	Purpose	PRD Reference
TestCreateProviderFromConfig_OpenAI	Create OpenAI provider	US-004
TestCreateProviderFromConfig_OpenAIDefault	Default openai protocol	US-004
TestCreateProviderFromConfig_Anthropic	Create Anthropic provider	US-004
TestCreateProviderFromConfig_Antigravity	Create Antigravity provider	US-004
TestCreateProviderFromConfig_ClaudeCLI	Create Claude CLI provider	US-004
TestCreateProviderFromConfig_CodexCLI	Create Codex CLI provider	US-004
TestCreateProviderFromConfig_GitHubCopilot	Create GitHub Copilot provider	US-004
TestCreateProviderFromConfig_UnknownProtocol	Unknown protocol error handling	US-004
TestCreateProviderFromConfig_MissingAPIKey	Missing API key error	US-004
TestExtractProtocol	Protocol prefix extraction	US-004
TestCreateProvider_UsesModelList	Create using model_list	US-005
TestCreateProvider_FallbackToProviders	Fallback to providers	US-005
TestCreateProvider_PriorityModelListOverProviders	model_list priority	US-005

5. pkg/providers/integration_test.go - E2E Integration Tests

Test Name	Purpose	PRD Reference
TestE2E_OpenAICompatibleProvider_NoCodeChange	Zero-code provider addition	Goal
TestE2E_LoadBalancing_RoundRobin	Load balancing actual effect	US-006
TestE2E_BackwardCompatibility_OldProvidersConfig	Old config compatibility	US-003
TestE2E_ErrorHandling_ModelNotFound	Model not found	FR-30
TestE2E_ErrorHandling_MissingAPIKey	Missing API key	FR-31
TestE2E_ErrorHandling_InvalidAPIBase	Invalid API base	FR-30
TestE2E_ToolCalls_OpenAICompatible	Tool call support	-
TestE2E_AntigravityProvider	Antigravity provider	US-004
TestE2E_ClaudeCLIProvider	Claude CLI provider	US-004

6. Performance Tests

Test Name	Purpose
BenchmarkCreateProviderFromConfig	Provider creation performance
BenchmarkGetModelConfig	Model lookup performance
BenchmarkGetModelConfigParallel	Concurrent lookup performance

---

Running Tests

# Run all tests
go test ./pkg/... -v

# Run with data race detection
go test ./pkg/... -race

# Run specific package tests
go test ./pkg/config -v
go test ./pkg/providers -v
go test ./pkg/providers/factory -v

# Run E2E tests
go test ./pkg/providers -run TestE2E -v

# Run performance tests
go test ./pkg/providers -bench=. -benchmem

---

PRD Acceptance Criteria Mapping

PRD Acceptance Criteria	Test Cases
US-001: Add ModelConfig struct	TestModelConfig_Parsing, TestModelConfig_Validation
US-001: model_name unique	TestConfig_ModelNameUniqueness
US-002: GetModelConfig method	TestConfig_GetModelConfig_*
US-003: Auto-convert providers	TestConvertProvidersToModelList_*
US-003: Deprecation warning	TestConfig_DeprecationWarning
US-003: Existing tests pass	(existing test files unchanged)
US-004: Protocol prefix factory	TestExtractProtocol, TestCreateProviderFromConfig_*
US-004: Default prefix openai	TestCreateProviderFromConfig_OpenAIDefault
US-005: CreateProvider uses factory	TestCreateProvider_*
US-006: Round-robin selection	TestModelRegistry_RoundRobin*
US-006: Thread-safe atomic	TestModelRegistry_RaceDetection

---

Recommended Implementation Order

1. Phase 1: Configuration Structure (US-001, US-002)

   • Implement ModelConfig struct
   • Implement GetModelConfig method
   • Run model_config_test.go

2. Phase 2: Protocol Factory (US-004)

   • Implement CreateProviderFromConfig
   • Implement ExtractProtocol
   • Run factory_test.go

3. Phase 3: Load Balancing (US-006)

   • Implement ModelRegistry
   • Implement round-robin selection
   • Run registry_test.go (with -race)

4. Phase 4: Backward Compatibility (US-003, US-005)

   • Implement ConvertProvidersToModelList
   • Refactor CreateProvider
   • Run migration_test.go
   • Verify existing tests pass

5. Phase 5: E2E Verification

   • Run integration_test.go
   • Manual testing with config.example.json