From d105a3ff5e07d20e357f98f6c7411df170acb0c2 Mon Sep 17 00:00:00 2001 From: agentson Date: Mon, 16 Feb 2026 21:44:59 +0900 Subject: [PATCH] =?UTF-8?q?docs:=20v2=20=EC=83=81=ED=83=9C=20=EB=B0=98?= =?UTF-8?q?=EC=98=81=20-=20=EC=A0=84=EC=B2=B4=20=EB=AC=B8=EC=84=9C=20?= =?UTF-8?q?=ED=98=84=ED=96=89=ED=99=94=20(#131)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - testing.md: 54 tests/4 files → 551 tests/25 files 반영, 전체 테스트 파일 설명 - architecture.md: v2 컴포넌트 추가 (Strategy, Context, Dashboard, Decision Logger 등), Playbook Mode 데이터 플로우, DB 스키마 5개 테이블, v2 환경변수 - commands.md: Dashboard 실행, Telegram 명령어 9종 레퍼런스 - CLAUDE.md: Project Structure 확장, 테스트 수 업데이트, --dashboard 플래그 - skills.md: DB 파일명 trades.db로 통일, Dashboard 명령어 추가 - requirements-log.md: 2026-02-16 문서 v2 동기화 요구사항 기록 Co-Authored-By: Claude Opus 4.6 --- CLAUDE.md | 22 +- docs/architecture.md | 460 ++++++++++++++++++++++++++++++--------- docs/commands.md | 52 ++++- docs/requirements-log.md | 25 +++ docs/skills.md | 12 +- docs/testing.md | 206 ++++++++++++------ 6 files changed, 606 insertions(+), 171 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 72c74c9..addfb0d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,6 +15,9 @@ pytest -v --cov=src # Run (paper trading) python -m src.main --mode=paper + +# Run with dashboard +python -m src.main --mode=paper --dashboard ``` ## Telegram Notifications (Optional) @@ -43,6 +46,10 @@ Get real-time alerts for trades, circuit breakers, and system events via Telegra - ℹ️ Market open/close notifications - 📝 System startup/shutdown status +### Interactive Commands + +With `TELEGRAM_COMMANDS_ENABLED=true` (default), the bot supports 9 bidirectional commands: `/help`, `/status`, `/positions`, `/report`, `/scenarios`, `/review`, `/dashboard`, `/stop`, `/resume`. + **Fail-safe**: Notifications never crash the trading system. Missing credentials or API errors are logged but trading continues normally. ## Smart Volatility Scanner (Optional) @@ -109,17 +116,23 @@ User requirements and feedback are tracked in [docs/requirements-log.md](docs/re ``` src/ ├── analysis/ # Technical analysis (RSI, volatility, smart scanner) +├── backup/ # Disaster recovery (scheduler, cloud storage, health) +├── brain/ # Gemini AI decision engine (prompt optimizer, context selector) ├── broker/ # KIS API client (domestic + overseas) -├── brain/ # Gemini AI decision engine +├── context/ # L1-L7 hierarchical memory system ├── core/ # Risk manager (READ-ONLY) -├── evolution/ # Self-improvement optimizer +├── dashboard/ # FastAPI read-only monitoring (8 API endpoints) +├── data/ # External data integration (news, market data, calendar) +├── evolution/ # Self-improvement (optimizer, daily review, scorecard) +├── logging/ # Decision logger (audit trail) ├── markets/ # Market schedules and timezone handling -├── notifications/ # Telegram real-time alerts +├── notifications/ # Telegram alerts + bidirectional commands (9 commands) +├── strategy/ # Pre-market planner, scenario engine, playbook store ├── db.py # SQLite trade logging ├── main.py # Trading loop orchestrator └── config.py # Settings (from .env) -tests/ # 343 tests across 14 files +tests/ # 551 tests across 25 files docs/ # Extended documentation ``` @@ -131,6 +144,7 @@ ruff check src/ tests/ # Lint mypy src/ --strict # Type check python -m src.main --mode=paper # Paper trading +python -m src.main --mode=paper --dashboard # With dashboard python -m src.main --mode=live # Live trading (⚠️ real money) # Gitea workflow (requires tea CLI) diff --git a/docs/architecture.md b/docs/architecture.md index e31ea9c..30d5d62 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,7 +2,9 @@ ## Overview -Self-evolving AI trading agent for global stock markets via KIS (Korea Investment & Securities) API. The main loop in `src/main.py` orchestrates four components across multiple markets with two trading modes: daily (batch API calls) or realtime (per-stock decisions). +Self-evolving AI trading agent for global stock markets via KIS (Korea Investment & Securities) API. The main loop in `src/main.py` orchestrates components across multiple markets with two trading modes: daily (batch API calls) or realtime (per-stock decisions). + +**v2 Proactive Playbook Architecture**: The system uses a "plan once, execute locally" approach. Pre-market, the AI generates a playbook of scenarios (one Gemini API call per market per day). During trading hours, a local scenario engine matches live market data against these pre-computed scenarios — no additional AI calls needed. This dramatically reduces API costs and latency. ## Trading Modes @@ -46,9 +48,11 @@ High-frequency trading with individual stock analysis: **KISBroker** (`kis_api.py`) — Async KIS API client for domestic Korean market - Automatic OAuth token refresh (valid for 24 hours) -- Leaky-bucket rate limiter (10 requests per second) +- Leaky-bucket rate limiter (configurable RPS, default 2.0) - POST body hash-key signing for order authentication - Custom SSL context with disabled hostname verification for VTS (virtual trading) endpoint due to known certificate mismatch +- `fetch_market_rankings()` — Fetch volume surge rankings from KIS API +- `get_daily_prices()` — Fetch OHLCV history for technical analysis **OverseasBroker** (`overseas.py`) — KIS overseas stock API wrapper @@ -63,10 +67,7 @@ High-frequency trading with individual stock analysis: - `is_market_open()` checks weekends, trading hours, lunch breaks - `get_open_markets()` returns currently active markets - `get_next_market_open()` finds next market to open and when - -**New API Methods** (added in v0.9.0): -- `fetch_market_rankings()` — Fetch volume surge rankings from KIS API -- `get_daily_prices()` — Fetch OHLCV history for technical analysis +- 10 global markets defined (KR, US_NASDAQ, US_NYSE, US_AMEX, JP, HK, CN_SHA, CN_SZA, VN_HNX, VN_HSX) ### 2. Analysis (`src/analysis/`) @@ -91,14 +92,9 @@ High-frequency trading with individual stock analysis: - **Fallback**: Uses static watchlist if ranking API unavailable - **Realtime mode only**: Daily mode uses batch processing for API efficiency -**Benefits:** -- Reduces Gemini API calls from 20-30 stocks to 1-3 qualified candidates -- Fast Python-based filtering before expensive AI judgment -- Logs selection context (RSI, volume_ratio, signal, score) for Evolution system +### 3. Brain (`src/brain/`) -### 3. Brain (`src/brain/gemini_client.py`) - -**GeminiClient** — AI decision engine powered by Google Gemini +**GeminiClient** (`gemini_client.py`) — AI decision engine powered by Google Gemini - Constructs structured prompts from market data - Parses JSON responses into `TradeDecision` objects (`action`, `confidence`, `rationale`) @@ -106,11 +102,20 @@ High-frequency trading with individual stock analysis: - Falls back to safe HOLD on any parse/API error - Handles markdown-wrapped JSON, malformed responses, invalid actions +**PromptOptimizer** (`prompt_optimizer.py`) — Token efficiency optimization + +- Reduces prompt size while preserving decision quality +- Caches optimized prompts + +**ContextSelector** (`context_selector.py`) — Relevant context selection for prompts + +- Selects appropriate context layers for current market conditions + ### 4. Risk Manager (`src/core/risk_manager.py`) **RiskManager** — Safety circuit breaker and order validation -⚠️ **READ-ONLY by policy** (see [`docs/agents.md`](./agents.md)) +> **READ-ONLY by policy** (see [`docs/agents.md`](./agents.md)) - **Circuit Breaker**: Halts all trading via `SystemExit` when daily P&L drops below -3.0% - Threshold may only be made stricter, never relaxed @@ -118,7 +123,79 @@ High-frequency trading with individual stock analysis: - **Fat-Finger Protection**: Rejects orders exceeding 30% of available cash - Must always be enforced, cannot be disabled -### 5. Notifications (`src/notifications/telegram_client.py`) +### 5. Strategy (`src/strategy/`) + +**Pre-Market Planner** (`pre_market_planner.py`) — AI playbook generation + +- Runs before market open (configurable `PRE_MARKET_MINUTES`, default 30) +- Generates scenario-based playbooks via single Gemini API call per market +- Handles timeout (`PLANNER_TIMEOUT_SECONDS`, default 60) with defensive playbook fallback +- Persists playbooks to database for audit trail + +**Scenario Engine** (`scenario_engine.py`) — Local scenario matching + +- Matches live market data against pre-computed playbook scenarios +- No AI calls during trading hours — pure Python matching logic +- Returns matched scenarios with confidence scores +- Configurable `MAX_SCENARIOS_PER_STOCK` (default 5) +- Periodic rescan at `RESCAN_INTERVAL_SECONDS` (default 300) + +**Playbook Store** (`playbook_store.py`) — Playbook persistence + +- SQLite-backed storage for daily playbooks +- Date and market-based retrieval +- Status tracking (generated, active, expired) + +**Models** (`models.py`) — Pydantic data models + +- Scenario, Playbook, MatchResult, and related type definitions + +### 6. Context System (`src/context/`) + +**Context Store** (`store.py`) — L1-L7 hierarchical memory + +- 7-layer context system (see [docs/context-tree.md](./context-tree.md)): + - L1: Tick-level (real-time price) + - L2: Intraday (session summary) + - L3: Daily (end-of-day) + - L4: Weekly (trend analysis) + - L5: Monthly (strategy review) + - L6: Daily Review (scorecard) + - L7: Evolution (long-term learning) +- Key-value storage with timeframe tagging +- SQLite persistence in `contexts` table + +**Context Scheduler** (`scheduler.py`) — Periodic aggregation + +- Scheduled summarization from lower to higher layers +- Configurable aggregation intervals + +**Context Summarizer** (`summarizer.py`) — Layer summarization + +- Aggregates lower-layer data into higher-layer summaries + +### 7. Dashboard (`src/dashboard/`) + +**FastAPI App** (`app.py`) — Read-only monitoring dashboard + +- Runs as daemon thread when enabled (`--dashboard` CLI flag or `DASHBOARD_ENABLED=true`) +- Configurable host/port (`DASHBOARD_HOST`, `DASHBOARD_PORT`, default `127.0.0.1:8080`) +- Serves static HTML frontend + +**8 API Endpoints:** + +| Endpoint | Method | Description | +|----------|--------|-------------| +| `/` | GET | Static HTML dashboard | +| `/api/status` | GET | Daily trading status by market | +| `/api/playbook/{date}` | GET | Playbook for specific date and market | +| `/api/scorecard/{date}` | GET | Daily scorecard from L6_DAILY context | +| `/api/performance` | GET | Trading performance metrics (by market + combined) | +| `/api/context/{layer}` | GET | Query context by layer (L1-L7) | +| `/api/decisions` | GET | Decision log entries with outcomes | +| `/api/scenarios/active` | GET | Today's matched scenarios | + +### 8. Notifications (`src/notifications/telegram_client.py`) **TelegramClient** — Real-time event notifications via Telegram Bot API @@ -126,7 +203,13 @@ High-frequency trading with individual stock analysis: - Non-blocking: failures are logged but never crash trading - Rate-limited: 1 message/second default to respect Telegram API limits - Auto-disabled when credentials missing -- Gracefully handles API errors, network timeouts, invalid tokens + +**TelegramCommandHandler** — Bidirectional command interface + +- Long polling from Telegram API (configurable `TELEGRAM_POLLING_INTERVAL`) +- 9 interactive commands: `/help`, `/status`, `/positions`, `/report`, `/scenarios`, `/review`, `/dashboard`, `/stop`, `/resume` +- Authorization filtering by `TELEGRAM_CHAT_ID` +- Enable/disable via `TELEGRAM_COMMANDS_ENABLED` (default: true) **Notification Types:** - Trade execution (BUY/SELL with confidence) @@ -134,12 +217,12 @@ High-frequency trading with individual stock analysis: - Fat-finger protection triggers (order rejection) - Market open/close events - System startup/shutdown status +- Playbook generation results +- Stop-loss monitoring alerts -**Setup:** See [src/notifications/README.md](../src/notifications/README.md) for bot creation and configuration. +### 9. Evolution (`src/evolution/`) -### 6. Evolution (`src/evolution/optimizer.py`) - -**StrategyOptimizer** — Self-improvement loop +**StrategyOptimizer** (`optimizer.py`) — Self-improvement loop - Analyzes high-confidence losing trades from SQLite - Asks Gemini to generate new `BaseStrategy` subclasses @@ -147,99 +230,196 @@ High-frequency trading with individual stock analysis: - Simulates PR creation for human review - Only activates strategies that pass all tests +**DailyReview** (`daily_review.py`) — End-of-day review + +- Generates comprehensive trade performance summary +- Stores results in L6_DAILY context layer +- Tracks win rate, P&L, confidence accuracy + +**DailyScorecard** (`scorecard.py`) — Performance scoring + +- Calculates daily metrics (trades, P&L, win rate, avg confidence) +- Enables trend tracking across days + +**Stop-Loss Monitoring** — Real-time position protection + +- Monitors positions against stop-loss levels from playbook scenarios +- Sends Telegram alerts when thresholds approached or breached + +### 10. Decision Logger (`src/logging/decision_logger.py`) + +**DecisionLogger** — Comprehensive audit trail + +- Logs every trading decision with full context snapshot +- Captures input data, rationale, confidence, and outcomes +- Supports outcome tracking (P&L, accuracy) for post-analysis +- Stored in `decision_logs` table with indexed queries +- Review workflow support (reviewed flag, review notes) + +### 11. Data Integration (`src/data/`) + +**External Data Sources** (optional): + +- `news_api.py` — News sentiment data +- `market_data.py` — Extended market data +- `economic_calendar.py` — Economic event calendar + +### 12. Backup (`src/backup/`) + +**Disaster Recovery** (see [docs/disaster_recovery.md](./disaster_recovery.md)): + +- `scheduler.py` — Automated backup scheduling +- `exporter.py` — Data export to various formats +- `cloud_storage.py` — S3-compatible cloud backup +- `health_monitor.py` — Backup integrity verification + ## Data Flow +### Playbook Mode (Daily — Primary v2 Flow) + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Pre-Market Phase (before market open) │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌──────────────────────────────────┐ + │ Pre-Market Planner │ + │ - 1 Gemini API call per market │ + │ - Generate scenario playbook │ + │ - Store in playbooks table │ + └──────────────────┬───────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Trading Hours (market open → close) │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌──────────────────────────────────┐ + │ Market Schedule Check │ + │ - Get open markets │ + │ - Filter by enabled markets │ + └──────────────────┬───────────────┘ + │ + ▼ + ┌──────────────────────────────────┐ + │ Scenario Engine (local) │ + │ - Match live data vs playbook │ + │ - No AI calls needed │ + │ - Return matched scenarios │ + └──────────────────┬───────────────┘ + │ + ▼ + ┌──────────────────────────────────┐ + │ Risk Manager: Validate Order │ + │ - Check circuit breaker │ + │ - Check fat-finger limit │ + └──────────────────┬───────────────┘ + │ + ▼ + ┌──────────────────────────────────┐ + │ Broker: Execute Order │ + │ - Domestic: send_order() │ + │ - Overseas: send_overseas_order()│ + └──────────────────┬───────────────┘ + │ + ▼ + ┌──────────────────────────────────┐ + │ Decision Logger + DB │ + │ - Full audit trail │ + │ - Context snapshot │ + │ - Telegram notification │ + └──────────────────┬───────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Post-Market Phase │ +└─────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌──────────────────────────────────┐ + │ Daily Review + Scorecard │ + │ - Performance summary │ + │ - Store in L6_DAILY context │ + │ - Evolution learning │ + └──────────────────────────────────┘ +``` + ### Realtime Mode (with Smart Scanner) ``` ┌─────────────────────────────────────────────────────────────┐ -│ Main Loop (60s cycle per market) │ +│ Main Loop (60s cycle per market) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────┐ - │ Market Schedule Check │ - │ - Get open markets │ - │ - Filter by enabled markets │ - │ - Wait if all closed │ - └──────────────────┬────────────────┘ + │ Market Schedule Check │ + │ - Get open markets │ + │ - Filter by enabled markets │ + │ - Wait if all closed │ + └──────────────────┬───────────────┘ │ ▼ ┌──────────────────────────────────┐ - │ Smart Scanner (Python-first) │ + │ Smart Scanner (Python-first) │ │ - Fetch volume rankings (KIS) │ │ - Get 20d price history per stock│ │ - Calculate RSI(14) + vol ratio │ │ - Filter: vol>2x AND RSI extreme │ │ - Return top 3 qualified stocks │ - └──────────────────┬────────────────┘ + └──────────────────┬───────────────┘ │ ▼ ┌──────────────────────────────────┐ - │ For Each Qualified Candidate │ - └──────────────────┬────────────────┘ + │ For Each Qualified Candidate │ + └──────────────────┬───────────────┘ │ ▼ ┌──────────────────────────────────┐ │ Broker: Fetch Market Data │ │ - Domestic: orderbook + balance │ │ - Overseas: price + balance │ - └──────────────────┬────────────────┘ + └──────────────────┬───────────────┘ │ ▼ ┌──────────────────────────────────┐ - │ Calculate P&L │ - │ pnl_pct = (eval - cost) / cost │ - └──────────────────┬────────────────┘ + │ Brain: Get Decision (AI) │ + │ - Build prompt with market data │ + │ - Call Gemini API │ + │ - Parse JSON response │ + │ - Return TradeDecision │ + └──────────────────┬───────────────┘ │ ▼ ┌──────────────────────────────────┐ - │ Brain: Get Decision (AI) │ - │ - Build prompt with market data │ - │ - Call Gemini API │ - │ - Parse JSON response │ - │ - Return TradeDecision │ - └──────────────────┬────────────────┘ + │ Risk Manager: Validate Order │ + │ - Check circuit breaker │ + │ - Check fat-finger limit │ + └──────────────────┬───────────────┘ │ ▼ ┌──────────────────────────────────┐ - │ Risk Manager: Validate Order │ - │ - Check circuit breaker │ - │ - Check fat-finger limit │ - │ - Raise if validation fails │ - └──────────────────┬────────────────┘ + │ Broker: Execute Order │ + │ - Domestic: send_order() │ + │ - Overseas: send_overseas_order()│ + └──────────────────┬───────────────┘ │ ▼ ┌──────────────────────────────────┐ - │ Broker: Execute Order │ - │ - Domestic: send_order() │ - │ - Overseas: send_overseas_order() │ - └──────────────────┬────────────────┘ - │ - ▼ - ┌──────────────────────────────────┐ - │ Notifications: Send Alert │ - │ - Trade execution notification │ - │ - Non-blocking (errors logged) │ - │ - Rate-limited to 1/sec │ - └──────────────────┬────────────────┘ - │ - ▼ - ┌──────────────────────────────────┐ - │ Database: Log Trade │ - │ - SQLite (data/trades.db) │ - │ - Track: action, confidence, │ - │ rationale, market, exchange │ - │ - NEW: selection_context (JSON) │ - │ - RSI, volume_ratio, signal │ - │ - For Evolution optimization │ - └───────────────────────────────────┘ + │ Decision Logger + Notifications │ + │ - Log trade to SQLite │ + │ - selection_context (JSON) │ + │ - Telegram notification │ + └──────────────────────────────────┘ ``` ## Database Schema -**SQLite** (`src/db.py`) +**SQLite** (`src/db.py`) — Database: `data/trades.db` +### trades ```sql CREATE TABLE trades ( id INTEGER PRIMARY KEY AUTOINCREMENT, @@ -251,25 +431,73 @@ CREATE TABLE trades ( quantity INTEGER, price REAL, pnl REAL DEFAULT 0.0, - market TEXT DEFAULT 'KR', -- KR | US_NASDAQ | JP | etc. - exchange_code TEXT DEFAULT 'KRX', -- KRX | NASD | NYSE | etc. - selection_context TEXT -- JSON: {rsi, volume_ratio, signal, score} + market TEXT DEFAULT 'KR', + exchange_code TEXT DEFAULT 'KRX', + selection_context TEXT, -- JSON: {rsi, volume_ratio, signal, score} + decision_id TEXT -- Links to decision_logs ); ``` -**Selection Context** (new in v0.9.0): Stores scanner selection criteria as JSON: -```json -{ - "rsi": 28.5, - "volume_ratio": 2.7, - "signal": "oversold", - "score": 85.2 -} +### contexts +```sql +CREATE TABLE contexts ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + layer TEXT NOT NULL, -- L1 through L7 + timeframe TEXT, + key TEXT NOT NULL, + value TEXT NOT NULL, -- JSON data + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL +); +-- Indices: idx_contexts_layer, idx_contexts_timeframe, idx_contexts_updated ``` -Enables Evolution system to analyze correlation between selection criteria and trade outcomes. +### decision_logs +```sql +CREATE TABLE decision_logs ( + decision_id TEXT PRIMARY KEY, + timestamp TEXT NOT NULL, + stock_code TEXT, + market TEXT, + exchange_code TEXT, + action TEXT, + confidence INTEGER, + rationale TEXT, + context_snapshot TEXT, -- JSON: full context at decision time + input_data TEXT, -- JSON: market data used + outcome_pnl REAL, + outcome_accuracy REAL, + reviewed INTEGER DEFAULT 0, + review_notes TEXT +); +-- Indices: idx_decision_logs_timestamp, idx_decision_logs_reviewed, idx_decision_logs_confidence +``` -Auto-migration: Adds `market`, `exchange_code`, and `selection_context` columns if missing for backward compatibility. +### playbooks +```sql +CREATE TABLE playbooks ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + date TEXT NOT NULL, + market TEXT NOT NULL, + status TEXT DEFAULT 'generated', + playbook_json TEXT NOT NULL, -- Full playbook with scenarios + generated_at TEXT NOT NULL, + token_count INTEGER, + scenario_count INTEGER, + match_count INTEGER DEFAULT 0 +); +-- Indices: idx_playbooks_date, idx_playbooks_market +``` + +### context_metadata +```sql +CREATE TABLE context_metadata ( + layer TEXT PRIMARY KEY, + description TEXT, + retention_days INTEGER, + aggregation_source TEXT +); +``` ## Configuration @@ -284,29 +512,62 @@ KIS_APP_SECRET=your_app_secret KIS_ACCOUNT_NO=XXXXXXXX-XX GEMINI_API_KEY=your_gemini_key -# Optional +# Optional — Trading Mode MODE=paper # paper | live -DB_PATH=data/trades.db -CONFIDENCE_THRESHOLD=80 -MAX_LOSS_PCT=3.0 -MAX_ORDER_PCT=30.0 -ENABLED_MARKETS=KR,US_NASDAQ # Comma-separated market codes - -# Trading Mode (API efficiency) TRADE_MODE=daily # daily | realtime DAILY_SESSIONS=4 # Sessions per day (daily mode only) SESSION_INTERVAL_HOURS=6 # Hours between sessions (daily mode only) -# Telegram Notifications (optional) +# Optional — Database +DB_PATH=data/trades.db + +# Optional — Risk +CONFIDENCE_THRESHOLD=80 +MAX_LOSS_PCT=3.0 +MAX_ORDER_PCT=30.0 + +# Optional — Markets +ENABLED_MARKETS=KR,US # Comma-separated market codes +RATE_LIMIT_RPS=2.0 # KIS API requests per second + +# Optional — Pre-Market Planner (v2) +PRE_MARKET_MINUTES=30 # Minutes before market open to generate playbook +MAX_SCENARIOS_PER_STOCK=5 # Max scenarios per stock in playbook +PLANNER_TIMEOUT_SECONDS=60 # Timeout for playbook generation +DEFENSIVE_PLAYBOOK_ON_FAILURE=true # Fallback on AI failure +RESCAN_INTERVAL_SECONDS=300 # Scenario rescan interval during trading + +# Optional — Smart Scanner (realtime mode only) +RSI_OVERSOLD_THRESHOLD=30 # 0-50, oversold threshold +RSI_MOMENTUM_THRESHOLD=70 # 50-100, momentum threshold +VOL_MULTIPLIER=2.0 # Minimum volume ratio (2.0 = 200%) +SCANNER_TOP_N=3 # Max qualified candidates per scan + +# Optional — Dashboard +DASHBOARD_ENABLED=false # Enable FastAPI dashboard +DASHBOARD_HOST=127.0.0.1 # Dashboard bind address +DASHBOARD_PORT=8080 # Dashboard port (1-65535) + +# Optional — Telegram TELEGRAM_BOT_TOKEN=1234567890:ABCdefGHIjklMNOpqrsTUVwxyz TELEGRAM_CHAT_ID=123456789 TELEGRAM_ENABLED=true +TELEGRAM_COMMANDS_ENABLED=true # Enable bidirectional commands +TELEGRAM_POLLING_INTERVAL=1.0 # Command polling interval (seconds) -# Smart Scanner (optional, realtime mode only) -RSI_OVERSOLD_THRESHOLD=30 # 0-50, oversold threshold -RSI_MOMENTUM_THRESHOLD=70 # 50-100, momentum threshold -VOL_MULTIPLIER=2.0 # Minimum volume ratio (2.0 = 200%) -SCANNER_TOP_N=3 # Max qualified candidates per scan +# Optional — Backup +BACKUP_ENABLED=false +BACKUP_DIR=data/backups +S3_ENDPOINT_URL=... +S3_ACCESS_KEY=... +S3_SECRET_KEY=... +S3_BUCKET_NAME=... +S3_REGION=... + +# Optional — External Data +NEWS_API_KEY=... +NEWS_API_PROVIDER=... +MARKET_DATA_API_KEY=... ``` Tests use in-memory SQLite (`DB_PATH=":memory:"`) and dummy credentials via `tests/conftest.py`. @@ -340,4 +601,9 @@ Tests use in-memory SQLite (`DB_PATH=":memory:"`) and dummy credentials via `tes - Invalid token → log error, trading unaffected - Rate limit exceeded → queued via rate limiter -**Guarantee**: Notification failures never interrupt trading operations. +### Playbook Generation Failure +- Timeout → fall back to defensive playbook (`DEFENSIVE_PLAYBOOK_ON_FAILURE`) +- API error → use previous day's playbook if available +- No playbook → skip pre-market phase, fall back to direct AI calls + +**Guarantee**: Notification and dashboard failures never interrupt trading operations. diff --git a/docs/commands.md b/docs/commands.md index 1053ac5..6f00b17 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -119,7 +119,7 @@ No decorator needed for async tests. # Install all dependencies (production + dev) pip install -e ".[dev]" -# Run full test suite with coverage +# Run full test suite with coverage (551 tests across 25 files) pytest -v --cov=src --cov-report=term-missing # Run a single test file @@ -137,11 +137,61 @@ mypy src/ --strict # Run the trading agent python -m src.main --mode=paper +# Run with dashboard enabled +python -m src.main --mode=paper --dashboard + # Docker docker compose up -d ouroboros # Run agent docker compose --profile test up test # Run tests in container ``` +## Dashboard + +The FastAPI dashboard provides read-only monitoring of the trading system. + +### Starting the Dashboard + +```bash +# Via CLI flag +python -m src.main --mode=paper --dashboard + +# Via environment variable +DASHBOARD_ENABLED=true python -m src.main --mode=paper +``` + +Dashboard runs as a daemon thread on `DASHBOARD_HOST:DASHBOARD_PORT` (default: `127.0.0.1:8080`). + +### API Endpoints + +| Endpoint | Description | +|----------|-------------| +| `GET /` | HTML dashboard UI | +| `GET /api/status` | Daily trading status by market | +| `GET /api/playbook/{date}` | Playbook for specific date (query: `market`) | +| `GET /api/scorecard/{date}` | Daily scorecard from L6_DAILY context | +| `GET /api/performance` | Performance metrics by market and combined | +| `GET /api/context/{layer}` | Context data by layer L1-L7 (query: `timeframe`) | +| `GET /api/decisions` | Decision log entries (query: `limit`, `market`) | +| `GET /api/scenarios/active` | Today's matched scenarios | + +## Telegram Commands + +When `TELEGRAM_COMMANDS_ENABLED=true` (default), the bot accepts these interactive commands: + +| Command | Description | +|---------|-------------| +| `/help` | List available commands | +| `/status` | Show trading status (mode, markets, P&L) | +| `/positions` | Display account summary (balance, cash, P&L) | +| `/report` | Daily summary metrics (trades, P&L, win rate) | +| `/scenarios` | Show today's playbook scenarios | +| `/review` | Display recent scorecards (L6_DAILY layer) | +| `/dashboard` | Show dashboard URL if enabled | +| `/stop` | Pause trading | +| `/resume` | Resume trading | + +Commands are only processed from the authorized `TELEGRAM_CHAT_ID`. + ## Environment Setup ```bash diff --git a/docs/requirements-log.md b/docs/requirements-log.md index 59a0eae..8522657 100644 --- a/docs/requirements-log.md +++ b/docs/requirements-log.md @@ -86,3 +86,28 @@ - Plan Consistency (필수), Safety & Constraints, Quality, Workflow 4개 카테고리 **이슈/PR:** #114 + +--- + +## 2026-02-16 + +### 문서 v2 동기화 (전체 문서 현행화) + +**배경:** +- v2 기능 구현 완료 후 문서가 실제 코드 상태와 크게 괴리 +- 문서에는 54 tests / 4 files로 기록되었으나 실제로는 551 tests / 25 files +- v2 핵심 기능(Playbook, Scenario Engine, Dashboard, Telegram Commands, Daily Review, Context System, Backup) 문서화 누락 + +**요구사항:** +1. `docs/testing.md` — 551 tests / 25 files 반영, 전체 테스트 파일 설명 +2. `docs/architecture.md` — v2 컴포넌트(Strategy, Context, Dashboard, Decision Logger 등) 추가, Playbook Mode 데이터 플로우, DB 스키마 5개 테이블, v2 환경변수 +3. `docs/commands.md` — Dashboard 실행 명령어, Telegram 명령어 9종 레퍼런스 +4. `CLAUDE.md` — Project Structure 트리 확장, 테스트 수 업데이트, `--dashboard` 플래그 +5. `docs/skills.md` — DB 파일명 `trades.db`로 통일, Dashboard 명령어 추가 +6. 기존에 유효한 트러블슈팅, 코드 예제 등은 유지 + +**구현 결과:** +- 6개 문서 파일 업데이트 +- 이전 시도(2개 커밋)는 기존 내용을 과도하게 삭제하여 폐기, main 기준으로 재작업 + +**이슈/PR:** #131, PR #134 diff --git a/docs/skills.md b/docs/skills.md index dc75ba4..96ea254 100644 --- a/docs/skills.md +++ b/docs/skills.md @@ -34,6 +34,12 @@ python -m src.main --mode=paper ``` Runs the agent in paper-trading mode (no real orders). +### Start Trading Agent with Dashboard +```bash +python -m src.main --mode=paper --dashboard +``` +Runs the agent with FastAPI dashboard on `127.0.0.1:8080` (configurable via `DASHBOARD_HOST`/`DASHBOARD_PORT`). + ### Start Trading Agent (Production) ```bash docker compose up -d ouroboros @@ -59,7 +65,7 @@ Analyze the last 30 days of trade logs and generate performance metrics. python -m src.evolution.optimizer --evolve ``` Triggers the evolution engine to: -1. Analyze `trade_logs.db` for failing patterns +1. Analyze `trades.db` for failing patterns 2. Ask Gemini to generate a new strategy 3. Run tests on the new strategy 4. Create a PR if tests pass @@ -91,12 +97,12 @@ curl http://localhost:8080/health ### View Trade Logs ```bash -sqlite3 data/trade_logs.db "SELECT * FROM trades ORDER BY timestamp DESC LIMIT 20;" +sqlite3 data/trades.db "SELECT * FROM trades ORDER BY timestamp DESC LIMIT 20;" ``` ### Export Trade History ```bash -sqlite3 -header -csv data/trade_logs.db "SELECT * FROM trades;" > trades_export.csv +sqlite3 -header -csv data/trades.db "SELECT * FROM trades;" > trades_export.csv ``` ## Safety Checklist (Pre-Deploy) diff --git a/docs/testing.md b/docs/testing.md index b65d35e..3ab75a5 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -2,51 +2,29 @@ ## Test Structure -**54 tests** across four files. `asyncio_mode = "auto"` in pyproject.toml — async tests need no special decorator. +**551 tests** across **25 files**. `asyncio_mode = "auto"` in pyproject.toml — async tests need no special decorator. The `settings` fixture in `conftest.py` provides safe defaults with test credentials and in-memory DB. ### Test Files -#### `tests/test_risk.py` (11 tests) -- Circuit breaker boundaries -- Fat-finger edge cases +#### Core Components + +##### `tests/test_risk.py` (14 tests) +- Circuit breaker boundaries and exact threshold triggers +- Fat-finger edge cases and percentage validation - P&L calculation edge cases - Order validation logic -**Example:** -```python -def test_circuit_breaker_exact_threshold(risk_manager): - """Circuit breaker should trip at exactly -3.0%.""" - with pytest.raises(CircuitBreakerTripped): - risk_manager.validate_order( - current_pnl_pct=-3.0, - order_amount=1000, - total_cash=10000 - ) -``` - -#### `tests/test_broker.py` (6 tests) +##### `tests/test_broker.py` (11 tests) - OAuth token lifecycle - Rate limiting enforcement - Hash key generation - Network error handling - SSL context configuration -**Example:** -```python -async def test_rate_limiter(broker): - """Rate limiter should delay requests to stay under 10 RPS.""" - start = time.monotonic() - for _ in range(15): # 15 requests - await broker._rate_limiter.acquire() - elapsed = time.monotonic() - start - assert elapsed >= 1.0 # Should take at least 1 second -``` - -#### `tests/test_brain.py` (18 tests) -- Valid JSON parsing -- Markdown-wrapped JSON handling +##### `tests/test_brain.py` (24 tests) +- Valid JSON parsing and markdown-wrapped JSON handling - Malformed JSON fallback - Missing fields handling - Invalid action validation @@ -54,33 +32,143 @@ async def test_rate_limiter(broker): - Empty response handling - Prompt construction for different markets -**Example:** -```python -async def test_confidence_below_threshold_forces_hold(brain): - """Decisions below confidence threshold should force HOLD.""" - decision = brain.parse_response('{"action":"BUY","confidence":70,"rationale":"test"}') - assert decision.action == "HOLD" - assert decision.confidence == 70 -``` - -#### `tests/test_market_schedule.py` (19 tests) +##### `tests/test_market_schedule.py` (24 tests) - Market open/close logic - Timezone handling (UTC, Asia/Seoul, America/New_York, etc.) - DST (Daylight Saving Time) transitions -- Weekend handling -- Lunch break logic +- Weekend handling and lunch break logic - Multiple market filtering - Next market open calculation -**Example:** -```python -def test_is_market_open_during_trading_hours(): - """Market should be open during regular trading hours.""" - # KRX: 9:00-15:30 KST, no lunch break - market = MARKETS["KR"] - trading_time = datetime(2026, 2, 3, 10, 0, tzinfo=ZoneInfo("Asia/Seoul")) # Monday 10:00 - assert is_market_open(market, trading_time) is True -``` +##### `tests/test_db.py` (3 tests) +- Database initialization and table creation +- Trade logging with all fields (market, exchange_code, decision_id) +- Query and retrieval operations + +##### `tests/test_main.py` (37 tests) +- Trading loop orchestration +- Market iteration and stock processing +- Dashboard integration (`--dashboard` flag) +- Telegram command handler wiring +- Error handling and graceful shutdown + +#### Strategy & Playbook (v2) + +##### `tests/test_pre_market_planner.py` (37 tests) +- Pre-market playbook generation +- Gemini API integration for scenario creation +- Timeout handling and defensive playbook fallback +- Multi-market playbook generation + +##### `tests/test_scenario_engine.py` (44 tests) +- Scenario matching against live market data +- Confidence scoring and threshold filtering +- Multiple scenario type handling +- Edge cases (no match, partial match, expired scenarios) + +##### `tests/test_playbook_store.py` (23 tests) +- Playbook persistence to SQLite +- Date-based retrieval and market filtering +- Playbook status management (generated, active, expired) +- JSON serialization/deserialization + +##### `tests/test_strategy_models.py` (33 tests) +- Pydantic model validation for scenarios, playbooks, decisions +- Field constraints and default values +- Serialization round-trips + +#### Analysis & Scanning + +##### `tests/test_volatility.py` (24 tests) +- ATR and RSI calculation accuracy +- Volume surge ratio computation +- Momentum scoring +- Breakout/breakdown pattern detection +- Market scanner watchlist management + +##### `tests/test_smart_scanner.py` (13 tests) +- Python-first filtering pipeline +- RSI and volume ratio filter logic +- Candidate scoring and ranking +- Fallback to static watchlist + +#### Context & Memory + +##### `tests/test_context.py` (18 tests) +- L1-L7 layer storage and retrieval +- Context key-value CRUD operations +- Timeframe-based queries +- Layer metadata management + +##### `tests/test_context_scheduler.py` (5 tests) +- Periodic context aggregation scheduling +- Layer summarization triggers + +#### Evolution & Review + +##### `tests/test_evolution.py` (24 tests) +- Strategy optimization loop +- High-confidence losing trade analysis +- Generated strategy validation + +##### `tests/test_daily_review.py` (10 tests) +- End-of-day review generation +- Trade performance summarization +- Context layer (L6_DAILY) integration + +##### `tests/test_scorecard.py` (3 tests) +- Daily scorecard metrics calculation +- Win rate, P&L, confidence tracking + +#### Notifications & Commands + +##### `tests/test_telegram.py` (25 tests) +- Message sending and formatting +- Rate limiting (leaky bucket) +- Error handling (network timeout, invalid token) +- Auto-disable on missing credentials +- Notification types (trade, circuit breaker, fat-finger, market events) + +##### `tests/test_telegram_commands.py` (31 tests) +- 9 command handlers (/help, /status, /positions, /report, /scenarios, /review, /dashboard, /stop, /resume) +- Long polling and command dispatch +- Authorization filtering by chat_id +- Command response formatting + +#### Dashboard + +##### `tests/test_dashboard.py` (14 tests) +- FastAPI endpoint responses (8 API routes) +- Status, playbook, scorecard, performance, context, decisions, scenarios +- Query parameter handling (market, date, limit) + +#### Performance & Quality + +##### `tests/test_token_efficiency.py` (34 tests) +- Gemini token usage optimization +- Prompt size reduction verification +- Cache effectiveness + +##### `tests/test_latency_control.py` (30 tests) +- API call latency measurement +- Rate limiter timing accuracy +- Async operation overhead + +##### `tests/test_decision_logger.py` (9 tests) +- Decision audit trail completeness +- Context snapshot capture +- Outcome tracking (P&L, accuracy) + +##### `tests/test_data_integration.py` (38 tests) +- External data source integration +- News API, market data, economic calendar +- Error handling for API failures + +##### `tests/test_backup.py` (23 tests) +- Backup scheduler and execution +- Cloud storage (S3) upload +- Health monitoring +- Data export functionality ## Coverage Requirements @@ -91,20 +179,6 @@ Check coverage: pytest -v --cov=src --cov-report=term-missing ``` -Expected output: -``` -Name Stmts Miss Cover Missing ------------------------------------------------------------ -src/brain/gemini_client.py 85 5 94% 165-169 -src/broker/kis_api.py 120 12 90% ... -src/core/risk_manager.py 35 2 94% ... -src/db.py 25 1 96% ... -src/main.py 150 80 47% (excluded from CI) -src/markets/schedule.py 95 3 97% ... ------------------------------------------------------------ -TOTAL 510 103 80% -``` - **Note:** `main.py` has lower coverage as it contains the main loop which is tested via integration/manual testing. ## Test Configuration