Compare commits
22 Commits
feature/is
...
feature/is
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dbf57b5068 | ||
| 7efc254ab5 | |||
|
|
2742628b78 | ||
| d60fd8947b | |||
|
|
694d73b212 | ||
|
|
b2b02b6f57 | ||
| 2dbe98615d | |||
|
|
34cf081c96 | ||
|
|
7bc4e88335 | ||
| 386e039ff6 | |||
|
|
13ba9e8081 | ||
|
|
5b52f593a8 | ||
| 2798558bf3 | |||
|
|
2331d80915 | ||
|
|
7d72669cb8 | ||
| 74a4784b7a | |||
|
|
dc70311aed | ||
|
|
e56819e9e2 | ||
| cfd5351b58 | |||
|
|
b206c23fc9 | ||
|
|
4d9f3e2cfc | ||
| a93a5c616b |
41
.gitea/ISSUE_TEMPLATE/runtime_verification.md
Normal file
41
.gitea/ISSUE_TEMPLATE/runtime_verification.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
name: Runtime Verification Incident
|
||||
about: 실운영/스테이징 동작 검증 중 발견된 이상 징후 등록
|
||||
title: "[RUNTIME-VERIFY][SCN-XXX] "
|
||||
labels: runtime, verification
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
- 현상:
|
||||
- 최초 관측 시각(UTC):
|
||||
|
||||
## Reproduction / Observation
|
||||
|
||||
- 실행 모드(`live`/`paper`):
|
||||
- 세션(`NXT`, `US_PRE`, `US_DAY`, `US_AFTER`, ...):
|
||||
- 실행 커맨드:
|
||||
- 로그 경로:
|
||||
|
||||
## Expected vs Actual
|
||||
|
||||
- Expected:
|
||||
- Actual:
|
||||
|
||||
## Requirement Mapping
|
||||
|
||||
- REQ:
|
||||
- TASK:
|
||||
- TEST:
|
||||
|
||||
## Temporary Mitigation
|
||||
|
||||
- 즉시 완화책:
|
||||
|
||||
## Close Criteria
|
||||
|
||||
- [ ] Dev 수정 반영
|
||||
- [ ] Verifier 재검증 PASS
|
||||
- [ ] Runtime Verifier 재관측 PASS
|
||||
- [ ] `NOT_OBSERVED = 0`
|
||||
|
||||
38
.gitea/PULL_REQUEST_TEMPLATE.md
Normal file
38
.gitea/PULL_REQUEST_TEMPLATE.md
Normal file
@@ -0,0 +1,38 @@
|
||||
## Linked Issue
|
||||
|
||||
- Closes #N
|
||||
|
||||
## Scope
|
||||
|
||||
- REQ: `REQ-...`
|
||||
- TASK: `TASK-...`
|
||||
- TEST: `TEST-...`
|
||||
|
||||
## Main -> Verifier Directive Contract
|
||||
|
||||
- Scope: 대상 요구사항/코드/로그 경로
|
||||
- Method: 실행 커맨드 + 관측 포인트
|
||||
- PASS criteria:
|
||||
- FAIL criteria:
|
||||
- NOT_OBSERVED criteria:
|
||||
- Evidence format: PR 코멘트 `Coverage Matrix`
|
||||
|
||||
## Verifier Coverage Matrix (Required)
|
||||
|
||||
| Item | Evidence | Status (PASS/FAIL/NOT_OBSERVED) |
|
||||
|---|---|---|
|
||||
| REQ-... | 링크/로그 | PASS |
|
||||
|
||||
`NOT_OBSERVED`가 1개라도 있으면 승인/머지 금지.
|
||||
|
||||
## Gitea Preflight
|
||||
|
||||
- [ ] `docs/commands.md`와 `docs/workflow.md` 트러블슈팅 선확인
|
||||
- [ ] `tea` 사용 (`gh` 미사용)
|
||||
|
||||
## Runtime Evidence
|
||||
|
||||
- 시스템 실제 구동 커맨드:
|
||||
- 모니터링 로그 경로:
|
||||
- 이상 징후/이슈 링크:
|
||||
|
||||
6
.github/workflows/ci.yml
vendored
6
.github/workflows/ci.yml
vendored
@@ -21,6 +21,12 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: pip install ".[dev]"
|
||||
|
||||
- name: Validate governance assets
|
||||
run: python3 scripts/validate_governance_assets.py
|
||||
|
||||
- name: Validate Ouroboros docs
|
||||
run: python3 scripts/validate_ouroboros_docs.py
|
||||
|
||||
- name: Lint
|
||||
run: ruff check src/ tests/
|
||||
|
||||
|
||||
@@ -12,6 +12,8 @@ It is distinct from `docs/requirements-log.md`, which records **project/product
|
||||
|
||||
1. **Workflow enforcement**
|
||||
- Follow `docs/workflow.md` for all changes.
|
||||
- Before any Gitea issue/PR/comment operation, read `docs/commands.md` and `docs/workflow.md` troubleshooting section.
|
||||
- Use `tea` for Gitea operations; do not use GitHub CLI (`gh`) in this repository workflow.
|
||||
- Create a Gitea issue before any code or documentation change.
|
||||
- Work on a feature branch `feature/issue-{N}-{short-description}` and open a PR.
|
||||
- Never commit directly to `main`.
|
||||
@@ -43,3 +45,8 @@ It is distinct from `docs/requirements-log.md`, which records **project/product
|
||||
- When work requires guidance, consult the relevant `docs/` policies first.
|
||||
- Any code change must be accompanied by relevant documentation updates.
|
||||
- Persist user constraints across sessions by recording them in this document.
|
||||
|
||||
### 2026-02-27
|
||||
|
||||
- All agents must pre-read `docs/commands.md` and `docs/workflow.md` troubleshooting before running Gitea issue/PR/comment commands.
|
||||
- `gh` CLI is prohibited for repository ticket/PR operations; use `tea` (or documented Gitea API fallback only).
|
||||
|
||||
@@ -4,6 +4,13 @@
|
||||
|
||||
**Critical: Learn from failures. Never repeat the same failed command without modification.**
|
||||
|
||||
## Repository VCS Rule (Mandatory)
|
||||
|
||||
- 이 저장소의 티켓/PR/코멘트 작업은 Gitea 기준으로 수행한다.
|
||||
- `gh`(GitHub CLI) 명령 사용은 금지한다.
|
||||
- 기본 도구는 `tea`이며, `tea` 미지원 케이스만 Gitea API를 fallback으로 사용한다.
|
||||
- 실행 전 `docs/workflow.md`의 `Gitea CLI Formatting Troubleshooting`을 반드시 확인한다.
|
||||
|
||||
### tea CLI (Gitea Command Line Tool)
|
||||
|
||||
#### ❌ TTY Error - Interactive Confirmation Fails
|
||||
@@ -140,6 +147,12 @@ python -m src.main --mode=paper
|
||||
# Run with dashboard enabled
|
||||
python -m src.main --mode=paper --dashboard
|
||||
|
||||
# Runtime verification monitor (NOT_OBSERVED detection)
|
||||
bash scripts/runtime_verify_monitor.sh
|
||||
|
||||
# Follow runtime verification log
|
||||
tail -f data/overnight/runtime_verify_*.log
|
||||
|
||||
# Docker
|
||||
docker compose up -d ouroboros # Run agent
|
||||
docker compose --profile test up test # Run tests in container
|
||||
|
||||
@@ -34,6 +34,12 @@ Main Agent 아이디에이션 책임:
|
||||
- DCP-03 구현 착수: Phase 2 종료 전 Main Agent 승인 필수
|
||||
- DCP-04 배포 승인: Phase 4 종료 후 Main Agent 최종 승인 필수
|
||||
|
||||
Main/Verifier 사고 재발 방지 규칙:
|
||||
- Main Agent는 검증 위임 시 `Directive Contract`를 충족하지 않으면 검증 착수 금지
|
||||
- Verifier Agent는 지시 누락/모호성 발견 시 즉시 `BLOCKED`를 선언하고 보완 요청
|
||||
- Verifier Agent는 `미관측(NOT_OBSERVED)` 항목을 PASS로 보고할 수 없다
|
||||
- Runtime 검증에서 요구 세션 증적이 없으면 "정상"이 아니라 `미검증 이상`으로 이슈화한다
|
||||
|
||||
## Phase Control Gates
|
||||
|
||||
### Phase 0: Scenario Intake and Scope Lock
|
||||
@@ -112,6 +118,8 @@ Exit criteria:
|
||||
|
||||
Control checks:
|
||||
- Verifier가 테스트 증적(로그/리포트/실행 커맨드) 첨부
|
||||
- Verifier가 `Coverage Matrix`(`REQ/TASK/TEST` x `PASS/FAIL/NOT_OBSERVED`) 첨부
|
||||
- `NOT_OBSERVED` 항목 수가 0인지 확인(0이 아니면 Gate 실패)
|
||||
- Runtime Verifier가 스테이징/실운영 모니터링 계획 승인
|
||||
- 산출물: 수용 승인 레코드
|
||||
|
||||
@@ -149,6 +157,9 @@ TPM 티켓 운영 규칙:
|
||||
- TPM은 합의된 변경을 이슈로 등록하고 우선순위(`P0/P1/P2`)를 지정한다.
|
||||
- PR 본문에는 TPM이 지정한 우선순위와 범위가 그대로 반영되어야 한다.
|
||||
- 우선순위 변경은 TPM 제안 + Main Agent 승인으로만 가능하다.
|
||||
- PM/TPM/Dev/Reviewer/Verifier/Runtime Verifier는 주요 의사결정 시점마다 PR 코멘트를 남겨 결정 근거를 추적 가능 상태로 유지한다.
|
||||
- PM/TPM/Dev/Reviewer/Verifier/Runtime Verifier는 이슈/PR/코멘트 조작 전에 `docs/commands.md`와 `docs/workflow.md`의 Gitea 트러블슈팅 섹션을 선참조해야 한다.
|
||||
- 저장소 협업에서 GitHub CLI(`gh`) 사용은 금지하며, Gitea 작업은 `tea`(필요 시 문서화된 API fallback)만 허용한다.
|
||||
|
||||
브랜치 운영 규칙:
|
||||
- TPM은 각 티켓에 대해 `ticket temp branch -> program feature branch` PR 경로를 지정한다.
|
||||
@@ -167,6 +178,8 @@ TPM 티켓 운영 규칙:
|
||||
- 시스템 실제 구동(스테이징/로컬 실운영 모드) 실행
|
||||
- 모니터링 체크리스트(핵심 경보/주문 경로/예외 로그) 수행
|
||||
- 결과를 티켓/PR 코멘트에 증적으로 첨부하지 않으면 완료로 간주하지 않음
|
||||
- 세션별 필수 관측 포인트(`NXT`, `US_PRE`, `US_DAY`, `US_AFTER` 등) 중 미관측 항목은 `NOT_OBSERVED`로 기록
|
||||
- `NOT_OBSERVED` 존재 시 승인 금지 + Runtime 이슈 발행
|
||||
|
||||
## Server Reflection Rule
|
||||
|
||||
|
||||
@@ -48,14 +48,20 @@ Updated: 2026-02-26
|
||||
병합 전 체크리스트:
|
||||
- 이슈 연결(`Closes #N`) 존재
|
||||
- PR 본문에 `REQ-*`, `TASK-*`, `TEST-*` 매핑 표 존재
|
||||
- Main -> Verifier Directive Contract(범위/방법/합격/실패/미관측/증적 형식) 기재
|
||||
- `src/core/risk_manager.py` 변경 없음
|
||||
- 주요 의사결정 체크포인트(DCP-01~04) 중 해당 단계 Main Agent 확인 기록 존재
|
||||
- 주요 의사결정(리뷰 지적/수정 합의/검증 승인)에 대한 에이전트 PR 코멘트 존재
|
||||
- 티켓 PR의 base가 `main`이 아닌 program feature branch인지 확인
|
||||
|
||||
자동 점검:
|
||||
- 문서 검증 스크립트 통과
|
||||
- 테스트 통과
|
||||
- 개발 완료 시 시스템 구동/모니터링 증적 코멘트 존재
|
||||
- 이슈/PR 조작 전에 `docs/commands.md` 및 `docs/workflow.md` 트러블슈팅 확인 코멘트 존재
|
||||
- `gh` CLI 미사용, `tea` 사용 증적 존재
|
||||
- Verifier `Coverage Matrix` 첨부(PASS/FAIL/NOT_OBSERVED)
|
||||
- `NOT_OBSERVED` 항목 0 확인(0이 아니면 머지 금지)
|
||||
|
||||
## 5) 감사 추적
|
||||
|
||||
|
||||
@@ -16,12 +16,27 @@
|
||||
|
||||
**Never commit directly to `main`.** This policy applies to all changes, no exceptions.
|
||||
|
||||
## Agent Gitea Preflight (Mandatory)
|
||||
|
||||
Gitea 이슈/PR/코멘트 작업 전에 모든 에이전트는 아래를 먼저 확인해야 한다.
|
||||
|
||||
1. `docs/commands.md`의 `tea CLI` 실패 사례/해결 패턴 확인
|
||||
2. 본 문서의 `Gitea CLI Formatting Troubleshooting` 확인
|
||||
3. 명령 실행 전 `gh`(GitHub CLI) 사용 금지 확인
|
||||
|
||||
강제 규칙:
|
||||
- 이 저장소 협업 명령은 `tea`를 기본으로 사용한다.
|
||||
- `gh issue`, `gh pr` 등 GitHub CLI 명령은 사용 금지다.
|
||||
- `tea` 실패 시 동일 명령 재시도 전에 원인/수정사항을 PR 코멘트에 남긴다.
|
||||
- 필요한 경우에만 Gitea API(`localhost:3000`)를 fallback으로 사용한다.
|
||||
|
||||
## Branch Strategy (Mandatory)
|
||||
|
||||
- Team operation default branch is the **program feature branch**, not `main`.
|
||||
- Ticket-level development happens only on **ticket temp branches** cut from the program feature branch.
|
||||
- Ticket PR merges into program feature branch are allowed after verifier approval.
|
||||
- Until final user sign-off, `main` merge is prohibited.
|
||||
- 각 에이전트는 주요 의사결정(리뷰 지적, 수정 방향, 검증 승인)마다 PR 코멘트를 적극 작성해 의사결정 과정을 남긴다.
|
||||
|
||||
## Gitea CLI Formatting Troubleshooting
|
||||
|
||||
@@ -136,6 +151,22 @@ task_tool(
|
||||
|
||||
Use `run_in_background=True` for independent tasks that don't block subsequent work.
|
||||
|
||||
### Main -> Verifier Directive Contract (Mandatory)
|
||||
|
||||
메인 에이전트가 검증 에이전트에 작업을 위임할 때, 아래 6개를 누락하면 지시가 무효다.
|
||||
|
||||
1. 검증 대상 범위: `REQ-*`, `TASK-*`, 코드/로그 경로
|
||||
2. 검증 방법: 실행 커맨드와 관측 포인트(예: 세션별 로그 키워드)
|
||||
3. 합격 기준: PASS 조건을 수치/문구로 명시
|
||||
4. 실패 기준: FAIL 조건을 수치/문구로 명시
|
||||
5. 미관측 기준: `NOT_OBSERVED` 조건과 즉시 에스컬레이션 규칙
|
||||
6. 증적 형식: PR 코멘트에 `Coverage Matrix` 표로 제출
|
||||
|
||||
`NOT_OBSERVED` 처리 규칙:
|
||||
- 요구사항 항목이 관측되지 않았으면 PASS로 간주 금지
|
||||
- `NOT_OBSERVED`는 운영상 `FAIL`과 동일하게 처리
|
||||
- `NOT_OBSERVED`가 하나라도 있으면 승인/머지 금지
|
||||
|
||||
## Code Review Checklist
|
||||
|
||||
**CRITICAL: Every PR review MUST verify plan-implementation consistency.**
|
||||
@@ -169,3 +200,7 @@ Before approving any PR, the reviewer (human or agent) must check ALL of the fol
|
||||
- [ ] PR references the Gitea issue number
|
||||
- [ ] Feature branch follows naming convention (`feature/issue-N-description`)
|
||||
- [ ] Commit messages are clear and descriptive
|
||||
- [ ] 이슈/PR 작업 전에 `docs/commands.md`와 본 문서 트러블슈팅 섹션을 확인했다
|
||||
- [ ] `gh` 명령을 사용하지 않고 `tea`(또는 허용된 Gitea API fallback)만 사용했다
|
||||
- [ ] Main -> Verifier 지시가 Directive Contract 6개 항목을 모두 포함한다
|
||||
- [ ] Verifier 결과에 `Coverage Matrix`(PASS/FAIL/NOT_OBSERVED)가 있고, `NOT_OBSERVED=0`이다
|
||||
|
||||
78
scripts/runtime_verify_monitor.sh
Executable file
78
scripts/runtime_verify_monitor.sh
Executable file
@@ -0,0 +1,78 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runtime verification monitor with NOT_OBSERVED detection.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
ROOT_DIR="${ROOT_DIR:-/home/agentson/repos/The-Ouroboros}"
|
||||
LOG_DIR="${LOG_DIR:-$ROOT_DIR/data/overnight}"
|
||||
INTERVAL_SEC="${INTERVAL_SEC:-60}"
|
||||
MAX_HOURS="${MAX_HOURS:-24}"
|
||||
|
||||
cd "$ROOT_DIR"
|
||||
|
||||
OUT_LOG="$LOG_DIR/runtime_verify_$(date +%Y%m%d_%H%M%S).log"
|
||||
END_TS=$(( $(date +%s) + MAX_HOURS*3600 ))
|
||||
|
||||
log() {
|
||||
printf '%s %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$1" | tee -a "$OUT_LOG" >/dev/null
|
||||
}
|
||||
|
||||
check_signal() {
|
||||
local name="$1"
|
||||
local pattern="$2"
|
||||
local run_log="$3"
|
||||
|
||||
if rg -q "$pattern" "$run_log"; then
|
||||
log "[COVERAGE] ${name}=PASS pattern=${pattern}"
|
||||
return 0
|
||||
fi
|
||||
log "[COVERAGE] ${name}=NOT_OBSERVED pattern=${pattern}"
|
||||
return 1
|
||||
}
|
||||
|
||||
log "[INFO] runtime verify monitor started interval=${INTERVAL_SEC}s max_hours=${MAX_HOURS}"
|
||||
|
||||
while true; do
|
||||
now=$(date +%s)
|
||||
if [ "$now" -ge "$END_TS" ]; then
|
||||
log "[INFO] monitor completed (time window reached)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
latest_run="$(ls -t "$LOG_DIR"/run_*.log 2>/dev/null | head -n1 || true)"
|
||||
if [ -z "$latest_run" ]; then
|
||||
log "[ANOMALY] no run log found"
|
||||
sleep "$INTERVAL_SEC"
|
||||
continue
|
||||
fi
|
||||
|
||||
# Basic liveness hints.
|
||||
app_pid="$(cat "$LOG_DIR/app.pid" 2>/dev/null || true)"
|
||||
wd_pid="$(cat "$LOG_DIR/watchdog.pid" 2>/dev/null || true)"
|
||||
app_alive=0
|
||||
wd_alive=0
|
||||
port_alive=0
|
||||
[ -n "$app_pid" ] && kill -0 "$app_pid" 2>/dev/null && app_alive=1
|
||||
[ -n "$wd_pid" ] && kill -0 "$wd_pid" 2>/dev/null && wd_alive=1
|
||||
ss -ltnp 2>/dev/null | rg -q ':8080' && port_alive=1
|
||||
log "[HEARTBEAT] run_log=$latest_run app_alive=$app_alive watchdog_alive=$wd_alive port8080=$port_alive"
|
||||
|
||||
# Coverage matrix rows (session paths and policy gate evidence).
|
||||
not_observed=0
|
||||
check_signal "LIVE_MODE" "Mode: live" "$latest_run" || not_observed=$((not_observed+1))
|
||||
check_signal "KR_LOOP" "Processing market: Korea Exchange" "$latest_run" || not_observed=$((not_observed+1))
|
||||
check_signal "NXT_PATH" "NXT_PRE|NXT_AFTER|session=NXT_" "$latest_run" || not_observed=$((not_observed+1))
|
||||
check_signal "US_PRE_PATH" "US_PRE|session=US_PRE" "$latest_run" || not_observed=$((not_observed+1))
|
||||
check_signal "US_DAY_PATH" "US_DAY|session=US_DAY|Processing market: .*NASDAQ|Processing market: .*NYSE|Processing market: .*AMEX" "$latest_run" || not_observed=$((not_observed+1))
|
||||
check_signal "US_AFTER_PATH" "US_AFTER|session=US_AFTER" "$latest_run" || not_observed=$((not_observed+1))
|
||||
check_signal "ORDER_POLICY_SESSION" "Order policy rejected .*\\[session=" "$latest_run" || not_observed=$((not_observed+1))
|
||||
|
||||
if [ "$not_observed" -gt 0 ]; then
|
||||
log "[ANOMALY] coverage_not_observed=$not_observed (treat as FAIL)"
|
||||
else
|
||||
log "[OK] coverage complete (NOT_OBSERVED=0)"
|
||||
fi
|
||||
|
||||
sleep "$INTERVAL_SEC"
|
||||
done
|
||||
|
||||
61
scripts/validate_governance_assets.py
Normal file
61
scripts/validate_governance_assets.py
Normal file
@@ -0,0 +1,61 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate persistent governance assets for agent workflow safety."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def must_contain(path: Path, required: list[str], errors: list[str]) -> None:
|
||||
if not path.exists():
|
||||
errors.append(f"missing file: {path}")
|
||||
return
|
||||
text = path.read_text(encoding="utf-8")
|
||||
for token in required:
|
||||
if token not in text:
|
||||
errors.append(f"{path}: missing required token -> {token}")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
errors: list[str] = []
|
||||
|
||||
pr_template = Path(".gitea/PULL_REQUEST_TEMPLATE.md")
|
||||
issue_template = Path(".gitea/ISSUE_TEMPLATE/runtime_verification.md")
|
||||
|
||||
must_contain(
|
||||
pr_template,
|
||||
[
|
||||
"Closes #N",
|
||||
"Main -> Verifier Directive Contract",
|
||||
"Coverage Matrix",
|
||||
"NOT_OBSERVED",
|
||||
"tea",
|
||||
"gh",
|
||||
],
|
||||
errors,
|
||||
)
|
||||
must_contain(
|
||||
issue_template,
|
||||
[
|
||||
"[RUNTIME-VERIFY][SCN-XXX]",
|
||||
"Requirement Mapping",
|
||||
"Close Criteria",
|
||||
"NOT_OBSERVED = 0",
|
||||
],
|
||||
errors,
|
||||
)
|
||||
|
||||
if errors:
|
||||
print("[FAIL] governance asset validation failed")
|
||||
for err in errors:
|
||||
print(f"- {err}")
|
||||
return 1
|
||||
|
||||
print("[OK] governance assets validated")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
52
src/analysis/backtest_cost_guard.py
Normal file
52
src/analysis/backtest_cost_guard.py
Normal file
@@ -0,0 +1,52 @@
|
||||
"""Backtest cost/slippage/failure validation guard."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import math
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BacktestCostModel:
|
||||
commission_bps: float | None = None
|
||||
slippage_bps_by_session: dict[str, float] | None = None
|
||||
failure_rate_by_session: dict[str, float] | None = None
|
||||
unfavorable_fill_required: bool = True
|
||||
|
||||
|
||||
def validate_backtest_cost_model(
|
||||
*,
|
||||
model: BacktestCostModel,
|
||||
required_sessions: list[str],
|
||||
) -> None:
|
||||
"""Raise ValueError when required cost assumptions are missing/invalid."""
|
||||
if (
|
||||
model.commission_bps is None
|
||||
or not math.isfinite(model.commission_bps)
|
||||
or model.commission_bps < 0
|
||||
):
|
||||
raise ValueError("commission_bps must be provided and >= 0")
|
||||
if not model.unfavorable_fill_required:
|
||||
raise ValueError("unfavorable_fill_required must be True")
|
||||
|
||||
slippage = model.slippage_bps_by_session or {}
|
||||
failure = model.failure_rate_by_session or {}
|
||||
|
||||
missing_slippage = [s for s in required_sessions if s not in slippage]
|
||||
if missing_slippage:
|
||||
raise ValueError(
|
||||
f"missing slippage_bps_by_session for sessions: {', '.join(missing_slippage)}"
|
||||
)
|
||||
|
||||
missing_failure = [s for s in required_sessions if s not in failure]
|
||||
if missing_failure:
|
||||
raise ValueError(
|
||||
f"missing failure_rate_by_session for sessions: {', '.join(missing_failure)}"
|
||||
)
|
||||
|
||||
for sess, bps in slippage.items():
|
||||
if not math.isfinite(bps) or bps < 0:
|
||||
raise ValueError(f"slippage bps must be >= 0 for session={sess}")
|
||||
for sess, rate in failure.items():
|
||||
if not math.isfinite(rate) or rate < 0 or rate > 1:
|
||||
raise ValueError(f"failure rate must be within [0,1] for session={sess}")
|
||||
103
src/analysis/backtest_execution_model.py
Normal file
103
src/analysis/backtest_execution_model.py
Normal file
@@ -0,0 +1,103 @@
|
||||
"""Conservative backtest execution model."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import math
|
||||
from random import Random
|
||||
from typing import Literal
|
||||
|
||||
|
||||
OrderSide = Literal["BUY", "SELL"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExecutionRequest:
|
||||
side: OrderSide
|
||||
session_id: str
|
||||
qty: int
|
||||
reference_price: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExecutionAssumptions:
|
||||
slippage_bps_by_session: dict[str, float]
|
||||
failure_rate_by_session: dict[str, float]
|
||||
partial_fill_rate_by_session: dict[str, float]
|
||||
partial_fill_min_ratio: float = 0.3
|
||||
partial_fill_max_ratio: float = 0.8
|
||||
seed: int = 0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExecutionResult:
|
||||
status: Literal["FILLED", "PARTIAL", "REJECTED"]
|
||||
filled_qty: int
|
||||
avg_price: float
|
||||
slippage_bps: float
|
||||
reason: str
|
||||
|
||||
|
||||
class BacktestExecutionModel:
|
||||
"""Execution simulator with conservative unfavorable fill assumptions."""
|
||||
|
||||
def __init__(self, assumptions: ExecutionAssumptions) -> None:
|
||||
self.assumptions = assumptions
|
||||
self._rng = Random(assumptions.seed)
|
||||
if assumptions.partial_fill_min_ratio <= 0 or assumptions.partial_fill_max_ratio > 1:
|
||||
raise ValueError("partial fill ratios must be within (0,1]")
|
||||
if assumptions.partial_fill_min_ratio > assumptions.partial_fill_max_ratio:
|
||||
raise ValueError("partial_fill_min_ratio must be <= partial_fill_max_ratio")
|
||||
for sess, bps in assumptions.slippage_bps_by_session.items():
|
||||
if not math.isfinite(bps) or bps < 0:
|
||||
raise ValueError(f"slippage_bps must be finite and >= 0 for session={sess}")
|
||||
for sess, rate in assumptions.failure_rate_by_session.items():
|
||||
if not math.isfinite(rate) or rate < 0 or rate > 1:
|
||||
raise ValueError(f"failure_rate must be in [0,1] for session={sess}")
|
||||
for sess, rate in assumptions.partial_fill_rate_by_session.items():
|
||||
if not math.isfinite(rate) or rate < 0 or rate > 1:
|
||||
raise ValueError(f"partial_fill_rate must be in [0,1] for session={sess}")
|
||||
|
||||
def simulate(self, request: ExecutionRequest) -> ExecutionResult:
|
||||
if request.qty <= 0:
|
||||
raise ValueError("qty must be positive")
|
||||
if request.reference_price <= 0:
|
||||
raise ValueError("reference_price must be positive")
|
||||
|
||||
slippage_bps = self.assumptions.slippage_bps_by_session.get(request.session_id, 0.0)
|
||||
failure_rate = self.assumptions.failure_rate_by_session.get(request.session_id, 0.0)
|
||||
partial_rate = self.assumptions.partial_fill_rate_by_session.get(request.session_id, 0.0)
|
||||
|
||||
if self._rng.random() < failure_rate:
|
||||
return ExecutionResult(
|
||||
status="REJECTED",
|
||||
filled_qty=0,
|
||||
avg_price=0.0,
|
||||
slippage_bps=slippage_bps,
|
||||
reason="execution_failure",
|
||||
)
|
||||
|
||||
slip_mult = 1.0 + (slippage_bps / 10000.0 if request.side == "BUY" else -slippage_bps / 10000.0)
|
||||
exec_price = request.reference_price * slip_mult
|
||||
|
||||
if self._rng.random() < partial_rate:
|
||||
ratio = self._rng.uniform(
|
||||
self.assumptions.partial_fill_min_ratio,
|
||||
self.assumptions.partial_fill_max_ratio,
|
||||
)
|
||||
filled = max(1, min(request.qty - 1, int(request.qty * ratio)))
|
||||
return ExecutionResult(
|
||||
status="PARTIAL",
|
||||
filled_qty=filled,
|
||||
avg_price=exec_price,
|
||||
slippage_bps=slippage_bps,
|
||||
reason="partial_fill",
|
||||
)
|
||||
|
||||
return ExecutionResult(
|
||||
status="FILLED",
|
||||
filled_qty=request.qty,
|
||||
avg_price=exec_price,
|
||||
slippage_bps=slippage_bps,
|
||||
reason="filled",
|
||||
)
|
||||
74
src/analysis/walk_forward_split.py
Normal file
74
src/analysis/walk_forward_split.py
Normal file
@@ -0,0 +1,74 @@
|
||||
"""Walk-forward splitter with purge/embargo controls."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WalkForwardFold:
|
||||
train_indices: list[int]
|
||||
test_indices: list[int]
|
||||
|
||||
@property
|
||||
def train_size(self) -> int:
|
||||
return len(self.train_indices)
|
||||
|
||||
@property
|
||||
def test_size(self) -> int:
|
||||
return len(self.test_indices)
|
||||
|
||||
|
||||
def generate_walk_forward_splits(
|
||||
*,
|
||||
n_samples: int,
|
||||
train_size: int,
|
||||
test_size: int,
|
||||
step_size: int | None = None,
|
||||
purge_size: int = 0,
|
||||
embargo_size: int = 0,
|
||||
min_train_size: int = 1,
|
||||
) -> list[WalkForwardFold]:
|
||||
"""Generate chronological folds with purge/embargo leakage controls."""
|
||||
if n_samples <= 0:
|
||||
raise ValueError("n_samples must be positive")
|
||||
if train_size <= 0 or test_size <= 0:
|
||||
raise ValueError("train_size and test_size must be positive")
|
||||
if purge_size < 0 or embargo_size < 0:
|
||||
raise ValueError("purge_size and embargo_size must be >= 0")
|
||||
if min_train_size <= 0:
|
||||
raise ValueError("min_train_size must be positive")
|
||||
|
||||
step = step_size if step_size is not None else test_size
|
||||
if step <= 0:
|
||||
raise ValueError("step_size must be positive")
|
||||
|
||||
folds: list[WalkForwardFold] = []
|
||||
prev_test_end: int | None = None
|
||||
test_start = train_size + purge_size
|
||||
|
||||
while test_start + test_size <= n_samples:
|
||||
test_end = test_start + test_size - 1
|
||||
train_end = test_start - purge_size - 1
|
||||
if train_end < 0:
|
||||
break
|
||||
|
||||
train_start = max(0, train_end - train_size + 1)
|
||||
train_indices = list(range(train_start, train_end + 1))
|
||||
|
||||
if prev_test_end is not None and embargo_size > 0:
|
||||
emb_from = prev_test_end + 1
|
||||
emb_to = prev_test_end + embargo_size
|
||||
train_indices = [i for i in train_indices if i < emb_from or i > emb_to]
|
||||
|
||||
if len(train_indices) >= min_train_size:
|
||||
folds.append(
|
||||
WalkForwardFold(
|
||||
train_indices=train_indices,
|
||||
test_indices=list(range(test_start, test_end + 1)),
|
||||
)
|
||||
)
|
||||
prev_test_end = test_end
|
||||
test_start += step
|
||||
|
||||
return folds
|
||||
@@ -59,6 +59,8 @@ class Settings(BaseSettings):
|
||||
# KIS VTS overseas balance API returns errors for most accounts.
|
||||
# This value is used as a fallback when the balance API returns 0 in paper mode.
|
||||
PAPER_OVERSEAS_CASH: float = Field(default=50000.0, ge=0.0)
|
||||
USD_BUFFER_MIN: float = Field(default=1000.0, ge=0.0)
|
||||
OVERNIGHT_EXCEPTION_ENABLED: bool = True
|
||||
|
||||
# Trading frequency mode (daily = batch API calls, realtime = per-stock calls)
|
||||
TRADE_MODE: str = Field(default="daily", pattern="^(daily|realtime)$")
|
||||
|
||||
72
src/db.py
72
src/db.py
@@ -31,8 +31,12 @@ def init_db(db_path: str) -> sqlite3.Connection:
|
||||
quantity INTEGER,
|
||||
price REAL,
|
||||
pnl REAL DEFAULT 0.0,
|
||||
strategy_pnl REAL DEFAULT 0.0,
|
||||
fx_pnl REAL DEFAULT 0.0,
|
||||
market TEXT DEFAULT 'KR',
|
||||
exchange_code TEXT DEFAULT 'KRX',
|
||||
session_id TEXT DEFAULT 'UNKNOWN',
|
||||
selection_context TEXT,
|
||||
decision_id TEXT,
|
||||
mode TEXT DEFAULT 'paper'
|
||||
)
|
||||
@@ -53,6 +57,32 @@ def init_db(db_path: str) -> sqlite3.Connection:
|
||||
conn.execute("ALTER TABLE trades ADD COLUMN decision_id TEXT")
|
||||
if "mode" not in columns:
|
||||
conn.execute("ALTER TABLE trades ADD COLUMN mode TEXT DEFAULT 'paper'")
|
||||
session_id_added = False
|
||||
if "session_id" not in columns:
|
||||
conn.execute("ALTER TABLE trades ADD COLUMN session_id TEXT DEFAULT 'UNKNOWN'")
|
||||
session_id_added = True
|
||||
if "strategy_pnl" not in columns:
|
||||
conn.execute("ALTER TABLE trades ADD COLUMN strategy_pnl REAL DEFAULT 0.0")
|
||||
if "fx_pnl" not in columns:
|
||||
conn.execute("ALTER TABLE trades ADD COLUMN fx_pnl REAL DEFAULT 0.0")
|
||||
# Backfill legacy rows where only pnl existed before split accounting columns.
|
||||
conn.execute(
|
||||
"""
|
||||
UPDATE trades
|
||||
SET strategy_pnl = pnl, fx_pnl = 0.0
|
||||
WHERE pnl != 0.0
|
||||
AND strategy_pnl = 0.0
|
||||
AND fx_pnl = 0.0
|
||||
"""
|
||||
)
|
||||
if session_id_added:
|
||||
conn.execute(
|
||||
"""
|
||||
UPDATE trades
|
||||
SET session_id = 'UNKNOWN'
|
||||
WHERE session_id IS NULL OR session_id = ''
|
||||
"""
|
||||
)
|
||||
|
||||
# Context tree tables for multi-layered memory management
|
||||
conn.execute(
|
||||
@@ -171,8 +201,11 @@ def log_trade(
|
||||
quantity: int = 0,
|
||||
price: float = 0.0,
|
||||
pnl: float = 0.0,
|
||||
strategy_pnl: float | None = None,
|
||||
fx_pnl: float | None = None,
|
||||
market: str = "KR",
|
||||
exchange_code: str = "KRX",
|
||||
session_id: str | None = None,
|
||||
selection_context: dict[str, any] | None = None,
|
||||
decision_id: str | None = None,
|
||||
mode: str = "paper",
|
||||
@@ -187,24 +220,37 @@ def log_trade(
|
||||
rationale: AI decision rationale
|
||||
quantity: Number of shares
|
||||
price: Trade price
|
||||
pnl: Profit/loss
|
||||
pnl: Total profit/loss (backward compatibility)
|
||||
strategy_pnl: Strategy PnL component
|
||||
fx_pnl: FX PnL component
|
||||
market: Market code
|
||||
exchange_code: Exchange code
|
||||
session_id: Session identifier (if omitted, auto-derived from market)
|
||||
selection_context: Scanner selection data (RSI, volume_ratio, signal, score)
|
||||
decision_id: Unique decision identifier for audit linking
|
||||
mode: Trading mode ('paper' or 'live') for data separation
|
||||
"""
|
||||
# Serialize selection context to JSON
|
||||
context_json = json.dumps(selection_context) if selection_context else None
|
||||
resolved_session_id = _resolve_session_id(market=market, session_id=session_id)
|
||||
if strategy_pnl is None and fx_pnl is None:
|
||||
strategy_pnl = pnl
|
||||
fx_pnl = 0.0
|
||||
elif strategy_pnl is None:
|
||||
strategy_pnl = pnl - float(fx_pnl or 0.0) if pnl != 0.0 else 0.0
|
||||
elif fx_pnl is None:
|
||||
fx_pnl = pnl - float(strategy_pnl) if pnl != 0.0 else 0.0
|
||||
if pnl == 0.0 and (strategy_pnl or fx_pnl):
|
||||
pnl = float(strategy_pnl) + float(fx_pnl)
|
||||
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO trades (
|
||||
timestamp, stock_code, action, confidence, rationale,
|
||||
quantity, price, pnl, market, exchange_code, selection_context, decision_id,
|
||||
mode
|
||||
quantity, price, pnl, strategy_pnl, fx_pnl,
|
||||
market, exchange_code, session_id, selection_context, decision_id, mode
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
datetime.now(UTC).isoformat(),
|
||||
@@ -215,8 +261,11 @@ def log_trade(
|
||||
quantity,
|
||||
price,
|
||||
pnl,
|
||||
strategy_pnl,
|
||||
fx_pnl,
|
||||
market,
|
||||
exchange_code,
|
||||
resolved_session_id,
|
||||
context_json,
|
||||
decision_id,
|
||||
mode,
|
||||
@@ -225,6 +274,21 @@ def log_trade(
|
||||
conn.commit()
|
||||
|
||||
|
||||
def _resolve_session_id(*, market: str, session_id: str | None) -> str:
|
||||
if session_id:
|
||||
return session_id
|
||||
try:
|
||||
from src.core.order_policy import classify_session_id
|
||||
from src.markets.schedule import MARKETS
|
||||
|
||||
market_info = MARKETS.get(market)
|
||||
if market_info is not None:
|
||||
return classify_session_id(market_info)
|
||||
except Exception:
|
||||
pass
|
||||
return "UNKNOWN"
|
||||
|
||||
|
||||
def get_latest_buy_trade(
|
||||
conn: sqlite3.Connection, stock_code: str, market: str
|
||||
) -> dict[str, Any] | None:
|
||||
|
||||
118
src/main.py
118
src/main.py
@@ -33,7 +33,11 @@ from src.core.blackout_manager import (
|
||||
parse_blackout_windows_kst,
|
||||
)
|
||||
from src.core.kill_switch import KillSwitchOrchestrator
|
||||
from src.core.order_policy import OrderPolicyRejected, validate_order_policy
|
||||
from src.core.order_policy import (
|
||||
OrderPolicyRejected,
|
||||
get_session_info,
|
||||
validate_order_policy,
|
||||
)
|
||||
from src.core.priority_queue import PriorityTaskQueue
|
||||
from src.core.risk_manager import CircuitBreakerTripped, FatFingerRejected, RiskManager
|
||||
from src.db import (
|
||||
@@ -63,6 +67,7 @@ BLACKOUT_ORDER_MANAGER = BlackoutOrderManager(
|
||||
windows=[],
|
||||
max_queue_size=500,
|
||||
)
|
||||
_SESSION_CLOSE_WINDOWS = {"NXT_AFTER", "US_AFTER"}
|
||||
|
||||
|
||||
def safe_float(value: str | float | None, default: float = 0.0) -> float:
|
||||
@@ -429,6 +434,41 @@ def _determine_order_quantity(
|
||||
return quantity
|
||||
|
||||
|
||||
def _should_block_overseas_buy_for_fx_buffer(
|
||||
*,
|
||||
market: MarketInfo,
|
||||
action: str,
|
||||
total_cash: float,
|
||||
order_amount: float,
|
||||
settings: Settings | None,
|
||||
) -> tuple[bool, float, float]:
|
||||
if (
|
||||
market.is_domestic
|
||||
or not market.code.startswith("US")
|
||||
or action != "BUY"
|
||||
or settings is None
|
||||
):
|
||||
return False, total_cash - order_amount, 0.0
|
||||
remaining = total_cash - order_amount
|
||||
required = settings.USD_BUFFER_MIN
|
||||
return remaining < required, remaining, required
|
||||
|
||||
|
||||
def _should_force_exit_for_overnight(
|
||||
*,
|
||||
market: MarketInfo,
|
||||
settings: Settings | None,
|
||||
) -> bool:
|
||||
session_id = get_session_info(market).session_id
|
||||
if session_id not in _SESSION_CLOSE_WINDOWS:
|
||||
return False
|
||||
if KILL_SWITCH.new_orders_blocked:
|
||||
return True
|
||||
if settings is None:
|
||||
return False
|
||||
return not settings.OVERNIGHT_EXCEPTION_ENABLED
|
||||
|
||||
|
||||
async def build_overseas_symbol_universe(
|
||||
db_conn: Any,
|
||||
overseas_broker: OverseasBroker,
|
||||
@@ -1194,6 +1234,23 @@ async def trading_cycle(
|
||||
loss_pct,
|
||||
take_profit_threshold,
|
||||
)
|
||||
if decision.action == "HOLD" and _should_force_exit_for_overnight(
|
||||
market=market,
|
||||
settings=settings,
|
||||
):
|
||||
decision = TradeDecision(
|
||||
action="SELL",
|
||||
confidence=max(decision.confidence, 85),
|
||||
rationale=(
|
||||
"Forced exit by overnight policy"
|
||||
" (session close window / kill switch priority)"
|
||||
),
|
||||
)
|
||||
logger.info(
|
||||
"Overnight policy override for %s (%s): HOLD -> SELL",
|
||||
stock_code,
|
||||
market.name,
|
||||
)
|
||||
logger.info(
|
||||
"Decision for %s (%s): %s (confidence=%d)",
|
||||
stock_code,
|
||||
@@ -1254,7 +1311,7 @@ async def trading_cycle(
|
||||
trade_price = current_price
|
||||
trade_pnl = 0.0
|
||||
if decision.action in ("BUY", "SELL"):
|
||||
if KILL_SWITCH.new_orders_blocked:
|
||||
if KILL_SWITCH.new_orders_blocked and decision.action == "BUY":
|
||||
logger.critical(
|
||||
"KillSwitch block active: skip %s order for %s (%s)",
|
||||
decision.action,
|
||||
@@ -1292,6 +1349,24 @@ async def trading_cycle(
|
||||
)
|
||||
return
|
||||
order_amount = current_price * quantity
|
||||
fx_blocked, remaining_cash, required_buffer = _should_block_overseas_buy_for_fx_buffer(
|
||||
market=market,
|
||||
action=decision.action,
|
||||
total_cash=total_cash,
|
||||
order_amount=order_amount,
|
||||
settings=settings,
|
||||
)
|
||||
if fx_blocked:
|
||||
logger.warning(
|
||||
"Skip BUY %s (%s): FX buffer guard (remaining=%.2f, required=%.2f, cash=%.2f, order=%.2f)",
|
||||
stock_code,
|
||||
market.name,
|
||||
remaining_cash,
|
||||
required_buffer,
|
||||
total_cash,
|
||||
order_amount,
|
||||
)
|
||||
return
|
||||
|
||||
# 4. Check BUY cooldown (set when a prior BUY failed due to insufficient balance)
|
||||
if decision.action == "BUY" and buy_cooldown is not None:
|
||||
@@ -2285,6 +2360,25 @@ async def run_daily_session(
|
||||
stock_code,
|
||||
market.name,
|
||||
)
|
||||
if decision.action == "HOLD":
|
||||
daily_open = get_open_position(db_conn, stock_code, market.code)
|
||||
if daily_open and _should_force_exit_for_overnight(
|
||||
market=market,
|
||||
settings=settings,
|
||||
):
|
||||
decision = TradeDecision(
|
||||
action="SELL",
|
||||
confidence=max(decision.confidence, 85),
|
||||
rationale=(
|
||||
"Forced exit by overnight policy"
|
||||
" (session close window / kill switch priority)"
|
||||
),
|
||||
)
|
||||
logger.info(
|
||||
"Daily overnight policy override for %s (%s): HOLD -> SELL",
|
||||
stock_code,
|
||||
market.name,
|
||||
)
|
||||
|
||||
# Log decision
|
||||
context_snapshot = {
|
||||
@@ -2325,7 +2419,7 @@ async def run_daily_session(
|
||||
trade_pnl = 0.0
|
||||
order_succeeded = True
|
||||
if decision.action in ("BUY", "SELL"):
|
||||
if KILL_SWITCH.new_orders_blocked:
|
||||
if KILL_SWITCH.new_orders_blocked and decision.action == "BUY":
|
||||
logger.critical(
|
||||
"KillSwitch block active: skip %s order for %s (%s)",
|
||||
decision.action,
|
||||
@@ -2360,6 +2454,24 @@ async def run_daily_session(
|
||||
)
|
||||
continue
|
||||
order_amount = stock_data["current_price"] * quantity
|
||||
fx_blocked, remaining_cash, required_buffer = _should_block_overseas_buy_for_fx_buffer(
|
||||
market=market,
|
||||
action=decision.action,
|
||||
total_cash=total_cash,
|
||||
order_amount=order_amount,
|
||||
settings=settings,
|
||||
)
|
||||
if fx_blocked:
|
||||
logger.warning(
|
||||
"Skip BUY %s (%s): FX buffer guard (remaining=%.2f, required=%.2f, cash=%.2f, order=%.2f)",
|
||||
stock_code,
|
||||
market.name,
|
||||
remaining_cash,
|
||||
required_buffer,
|
||||
total_cash,
|
||||
order_amount,
|
||||
)
|
||||
continue
|
||||
|
||||
# Check BUY cooldown (insufficient balance)
|
||||
if decision.action == "BUY":
|
||||
|
||||
83
tests/test_backtest_cost_guard.py
Normal file
83
tests/test_backtest_cost_guard.py
Normal file
@@ -0,0 +1,83 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from src.analysis.backtest_cost_guard import BacktestCostModel, validate_backtest_cost_model
|
||||
|
||||
|
||||
def test_valid_backtest_cost_model_passes() -> None:
|
||||
model = BacktestCostModel(
|
||||
commission_bps=5.0,
|
||||
slippage_bps_by_session={"KRX_REG": 10.0, "US_PRE": 50.0},
|
||||
failure_rate_by_session={"KRX_REG": 0.01, "US_PRE": 0.08},
|
||||
unfavorable_fill_required=True,
|
||||
)
|
||||
validate_backtest_cost_model(model=model, required_sessions=["KRX_REG", "US_PRE"])
|
||||
|
||||
|
||||
def test_missing_required_slippage_session_raises() -> None:
|
||||
model = BacktestCostModel(
|
||||
commission_bps=5.0,
|
||||
slippage_bps_by_session={"KRX_REG": 10.0},
|
||||
failure_rate_by_session={"KRX_REG": 0.01, "US_PRE": 0.08},
|
||||
unfavorable_fill_required=True,
|
||||
)
|
||||
with pytest.raises(ValueError, match="missing slippage_bps_by_session.*US_PRE"):
|
||||
validate_backtest_cost_model(model=model, required_sessions=["KRX_REG", "US_PRE"])
|
||||
|
||||
|
||||
def test_missing_required_failure_rate_session_raises() -> None:
|
||||
model = BacktestCostModel(
|
||||
commission_bps=5.0,
|
||||
slippage_bps_by_session={"KRX_REG": 10.0, "US_PRE": 50.0},
|
||||
failure_rate_by_session={"KRX_REG": 0.01},
|
||||
unfavorable_fill_required=True,
|
||||
)
|
||||
with pytest.raises(ValueError, match="missing failure_rate_by_session.*US_PRE"):
|
||||
validate_backtest_cost_model(model=model, required_sessions=["KRX_REG", "US_PRE"])
|
||||
|
||||
|
||||
def test_invalid_failure_rate_range_raises() -> None:
|
||||
model = BacktestCostModel(
|
||||
commission_bps=5.0,
|
||||
slippage_bps_by_session={"KRX_REG": 10.0},
|
||||
failure_rate_by_session={"KRX_REG": 1.2},
|
||||
unfavorable_fill_required=True,
|
||||
)
|
||||
with pytest.raises(ValueError, match="failure rate must be within"):
|
||||
validate_backtest_cost_model(model=model, required_sessions=["KRX_REG"])
|
||||
|
||||
|
||||
def test_unfavorable_fill_requirement_cannot_be_disabled() -> None:
|
||||
model = BacktestCostModel(
|
||||
commission_bps=5.0,
|
||||
slippage_bps_by_session={"KRX_REG": 10.0},
|
||||
failure_rate_by_session={"KRX_REG": 0.02},
|
||||
unfavorable_fill_required=False,
|
||||
)
|
||||
with pytest.raises(ValueError, match="unfavorable_fill_required must be True"):
|
||||
validate_backtest_cost_model(model=model, required_sessions=["KRX_REG"])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad_commission", [float("nan"), float("inf"), float("-inf")])
|
||||
def test_non_finite_commission_rejected(bad_commission: float) -> None:
|
||||
model = BacktestCostModel(
|
||||
commission_bps=bad_commission,
|
||||
slippage_bps_by_session={"KRX_REG": 10.0},
|
||||
failure_rate_by_session={"KRX_REG": 0.02},
|
||||
unfavorable_fill_required=True,
|
||||
)
|
||||
with pytest.raises(ValueError, match="commission_bps"):
|
||||
validate_backtest_cost_model(model=model, required_sessions=["KRX_REG"])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad_slippage", [float("nan"), float("inf"), float("-inf")])
|
||||
def test_non_finite_slippage_rejected(bad_slippage: float) -> None:
|
||||
model = BacktestCostModel(
|
||||
commission_bps=5.0,
|
||||
slippage_bps_by_session={"KRX_REG": bad_slippage},
|
||||
failure_rate_by_session={"KRX_REG": 0.02},
|
||||
unfavorable_fill_required=True,
|
||||
)
|
||||
with pytest.raises(ValueError, match="slippage bps"):
|
||||
validate_backtest_cost_model(model=model, required_sessions=["KRX_REG"])
|
||||
108
tests/test_backtest_execution_model.py
Normal file
108
tests/test_backtest_execution_model.py
Normal file
@@ -0,0 +1,108 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from src.analysis.backtest_execution_model import (
|
||||
BacktestExecutionModel,
|
||||
ExecutionAssumptions,
|
||||
ExecutionRequest,
|
||||
)
|
||||
|
||||
|
||||
def test_buy_uses_unfavorable_slippage_direction() -> None:
|
||||
model = BacktestExecutionModel(
|
||||
ExecutionAssumptions(
|
||||
slippage_bps_by_session={"US_PRE": 50.0},
|
||||
failure_rate_by_session={"US_PRE": 0.0},
|
||||
partial_fill_rate_by_session={"US_PRE": 0.0},
|
||||
seed=1,
|
||||
)
|
||||
)
|
||||
out = model.simulate(
|
||||
ExecutionRequest(side="BUY", session_id="US_PRE", qty=10, reference_price=100.0)
|
||||
)
|
||||
assert out.status == "FILLED"
|
||||
assert out.avg_price == pytest.approx(100.5)
|
||||
|
||||
|
||||
def test_sell_uses_unfavorable_slippage_direction() -> None:
|
||||
model = BacktestExecutionModel(
|
||||
ExecutionAssumptions(
|
||||
slippage_bps_by_session={"US_PRE": 50.0},
|
||||
failure_rate_by_session={"US_PRE": 0.0},
|
||||
partial_fill_rate_by_session={"US_PRE": 0.0},
|
||||
seed=1,
|
||||
)
|
||||
)
|
||||
out = model.simulate(
|
||||
ExecutionRequest(side="SELL", session_id="US_PRE", qty=10, reference_price=100.0)
|
||||
)
|
||||
assert out.status == "FILLED"
|
||||
assert out.avg_price == pytest.approx(99.5)
|
||||
|
||||
|
||||
def test_failure_rate_can_reject_order() -> None:
|
||||
model = BacktestExecutionModel(
|
||||
ExecutionAssumptions(
|
||||
slippage_bps_by_session={"KRX_REG": 10.0},
|
||||
failure_rate_by_session={"KRX_REG": 1.0},
|
||||
partial_fill_rate_by_session={"KRX_REG": 0.0},
|
||||
seed=42,
|
||||
)
|
||||
)
|
||||
out = model.simulate(
|
||||
ExecutionRequest(side="BUY", session_id="KRX_REG", qty=10, reference_price=100.0)
|
||||
)
|
||||
assert out.status == "REJECTED"
|
||||
assert out.filled_qty == 0
|
||||
|
||||
|
||||
def test_partial_fill_applies_when_rate_is_one() -> None:
|
||||
model = BacktestExecutionModel(
|
||||
ExecutionAssumptions(
|
||||
slippage_bps_by_session={"KRX_REG": 0.0},
|
||||
failure_rate_by_session={"KRX_REG": 0.0},
|
||||
partial_fill_rate_by_session={"KRX_REG": 1.0},
|
||||
partial_fill_min_ratio=0.4,
|
||||
partial_fill_max_ratio=0.4,
|
||||
seed=0,
|
||||
)
|
||||
)
|
||||
out = model.simulate(
|
||||
ExecutionRequest(side="BUY", session_id="KRX_REG", qty=10, reference_price=100.0)
|
||||
)
|
||||
assert out.status == "PARTIAL"
|
||||
assert out.filled_qty == 4
|
||||
assert out.avg_price == 100.0
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad_slip", [-1.0, float("nan"), float("inf")])
|
||||
def test_invalid_slippage_is_rejected(bad_slip: float) -> None:
|
||||
with pytest.raises(ValueError, match="slippage_bps"):
|
||||
BacktestExecutionModel(
|
||||
ExecutionAssumptions(
|
||||
slippage_bps_by_session={"US_PRE": bad_slip},
|
||||
failure_rate_by_session={"US_PRE": 0.0},
|
||||
partial_fill_rate_by_session={"US_PRE": 0.0},
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad_rate", [-0.1, 1.1, float("nan")])
|
||||
def test_invalid_failure_or_partial_rates_are_rejected(bad_rate: float) -> None:
|
||||
with pytest.raises(ValueError, match="failure_rate"):
|
||||
BacktestExecutionModel(
|
||||
ExecutionAssumptions(
|
||||
slippage_bps_by_session={"US_PRE": 10.0},
|
||||
failure_rate_by_session={"US_PRE": bad_rate},
|
||||
partial_fill_rate_by_session={"US_PRE": 0.0},
|
||||
)
|
||||
)
|
||||
with pytest.raises(ValueError, match="partial_fill_rate"):
|
||||
BacktestExecutionModel(
|
||||
ExecutionAssumptions(
|
||||
slippage_bps_by_session={"US_PRE": 10.0},
|
||||
failure_rate_by_session={"US_PRE": 0.0},
|
||||
partial_fill_rate_by_session={"US_PRE": bad_rate},
|
||||
)
|
||||
)
|
||||
136
tests/test_db.py
136
tests/test_db.py
@@ -155,6 +155,9 @@ def test_mode_column_exists_in_schema() -> None:
|
||||
cursor = conn.execute("PRAGMA table_info(trades)")
|
||||
columns = {row[1] for row in cursor.fetchall()}
|
||||
assert "mode" in columns
|
||||
assert "session_id" in columns
|
||||
assert "strategy_pnl" in columns
|
||||
assert "fx_pnl" in columns
|
||||
|
||||
|
||||
def test_mode_migration_adds_column_to_existing_db() -> None:
|
||||
@@ -182,6 +185,13 @@ def test_mode_migration_adds_column_to_existing_db() -> None:
|
||||
decision_id TEXT
|
||||
)"""
|
||||
)
|
||||
old_conn.execute(
|
||||
"""
|
||||
INSERT INTO trades (
|
||||
timestamp, stock_code, action, confidence, rationale, quantity, price, pnl
|
||||
) VALUES ('2026-01-01T00:00:00+00:00', 'AAPL', 'SELL', 90, 'legacy', 1, 100.0, 123.45)
|
||||
"""
|
||||
)
|
||||
old_conn.commit()
|
||||
old_conn.close()
|
||||
|
||||
@@ -190,6 +200,132 @@ def test_mode_migration_adds_column_to_existing_db() -> None:
|
||||
cursor = conn.execute("PRAGMA table_info(trades)")
|
||||
columns = {row[1] for row in cursor.fetchall()}
|
||||
assert "mode" in columns
|
||||
assert "session_id" in columns
|
||||
assert "strategy_pnl" in columns
|
||||
assert "fx_pnl" in columns
|
||||
migrated = conn.execute(
|
||||
"SELECT pnl, strategy_pnl, fx_pnl, session_id FROM trades WHERE stock_code='AAPL' LIMIT 1"
|
||||
).fetchone()
|
||||
assert migrated is not None
|
||||
assert migrated[0] == 123.45
|
||||
assert migrated[1] == 123.45
|
||||
assert migrated[2] == 0.0
|
||||
assert migrated[3] == "UNKNOWN"
|
||||
conn.close()
|
||||
finally:
|
||||
os.unlink(db_path)
|
||||
|
||||
|
||||
def test_log_trade_stores_strategy_and_fx_pnl_separately() -> None:
|
||||
conn = init_db(":memory:")
|
||||
log_trade(
|
||||
conn=conn,
|
||||
stock_code="AAPL",
|
||||
action="SELL",
|
||||
confidence=90,
|
||||
rationale="fx split",
|
||||
pnl=120.0,
|
||||
strategy_pnl=100.0,
|
||||
fx_pnl=20.0,
|
||||
market="US_NASDAQ",
|
||||
exchange_code="NASD",
|
||||
)
|
||||
row = conn.execute(
|
||||
"SELECT pnl, strategy_pnl, fx_pnl FROM trades ORDER BY id DESC LIMIT 1"
|
||||
).fetchone()
|
||||
assert row is not None
|
||||
assert row[0] == 120.0
|
||||
assert row[1] == 100.0
|
||||
assert row[2] == 20.0
|
||||
|
||||
|
||||
def test_log_trade_backward_compat_sets_strategy_pnl_from_pnl() -> None:
|
||||
conn = init_db(":memory:")
|
||||
log_trade(
|
||||
conn=conn,
|
||||
stock_code="005930",
|
||||
action="SELL",
|
||||
confidence=80,
|
||||
rationale="legacy",
|
||||
pnl=50.0,
|
||||
market="KR",
|
||||
exchange_code="KRX",
|
||||
)
|
||||
row = conn.execute(
|
||||
"SELECT pnl, strategy_pnl, fx_pnl FROM trades ORDER BY id DESC LIMIT 1"
|
||||
).fetchone()
|
||||
assert row is not None
|
||||
assert row[0] == 50.0
|
||||
assert row[1] == 50.0
|
||||
assert row[2] == 0.0
|
||||
|
||||
|
||||
def test_log_trade_partial_fx_input_does_not_infer_negative_strategy_pnl() -> None:
|
||||
conn = init_db(":memory:")
|
||||
log_trade(
|
||||
conn=conn,
|
||||
stock_code="AAPL",
|
||||
action="SELL",
|
||||
confidence=70,
|
||||
rationale="fx only",
|
||||
pnl=0.0,
|
||||
fx_pnl=10.0,
|
||||
market="US_NASDAQ",
|
||||
exchange_code="NASD",
|
||||
)
|
||||
row = conn.execute(
|
||||
"SELECT pnl, strategy_pnl, fx_pnl FROM trades ORDER BY id DESC LIMIT 1"
|
||||
).fetchone()
|
||||
assert row is not None
|
||||
assert row[0] == 10.0
|
||||
assert row[1] == 0.0
|
||||
assert row[2] == 10.0
|
||||
|
||||
|
||||
def test_log_trade_persists_explicit_session_id() -> None:
|
||||
conn = init_db(":memory:")
|
||||
log_trade(
|
||||
conn=conn,
|
||||
stock_code="AAPL",
|
||||
action="BUY",
|
||||
confidence=70,
|
||||
rationale="session test",
|
||||
market="US_NASDAQ",
|
||||
exchange_code="NASD",
|
||||
session_id="US_PRE",
|
||||
)
|
||||
row = conn.execute("SELECT session_id FROM trades ORDER BY id DESC LIMIT 1").fetchone()
|
||||
assert row is not None
|
||||
assert row[0] == "US_PRE"
|
||||
|
||||
|
||||
def test_log_trade_auto_derives_session_id_when_not_provided() -> None:
|
||||
conn = init_db(":memory:")
|
||||
log_trade(
|
||||
conn=conn,
|
||||
stock_code="005930",
|
||||
action="BUY",
|
||||
confidence=70,
|
||||
rationale="auto session",
|
||||
market="KR",
|
||||
exchange_code="KRX",
|
||||
)
|
||||
row = conn.execute("SELECT session_id FROM trades ORDER BY id DESC LIMIT 1").fetchone()
|
||||
assert row is not None
|
||||
assert row[0] != "UNKNOWN"
|
||||
|
||||
|
||||
def test_log_trade_unknown_market_falls_back_to_unknown_session() -> None:
|
||||
conn = init_db(":memory:")
|
||||
log_trade(
|
||||
conn=conn,
|
||||
stock_code="X",
|
||||
action="BUY",
|
||||
confidence=70,
|
||||
rationale="unknown market",
|
||||
market="MARS",
|
||||
exchange_code="MARS",
|
||||
)
|
||||
row = conn.execute("SELECT session_id FROM trades ORDER BY id DESC LIMIT 1").fetchone()
|
||||
assert row is not None
|
||||
assert row[0] == "UNKNOWN"
|
||||
|
||||
@@ -15,6 +15,8 @@ from src.evolution.scorecard import DailyScorecard
|
||||
from src.logging.decision_logger import DecisionLogger
|
||||
from src.main import (
|
||||
KILL_SWITCH,
|
||||
_should_force_exit_for_overnight,
|
||||
_should_block_overseas_buy_for_fx_buffer,
|
||||
_trigger_emergency_kill_switch,
|
||||
_apply_dashboard_flag,
|
||||
_determine_order_quantity,
|
||||
@@ -3690,6 +3692,81 @@ class TestOverseasBrokerIntegration:
|
||||
# DB도 브로커도 보유 없음 → BUY 주문이 실행되어야 함 (회귀 테스트)
|
||||
overseas_broker.send_overseas_order.assert_called_once()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_overseas_buy_blocked_by_usd_buffer_guard(self) -> None:
|
||||
"""Overseas BUY must be blocked when USD buffer would be breached."""
|
||||
db_conn = init_db(":memory:")
|
||||
|
||||
overseas_broker = MagicMock()
|
||||
overseas_broker.get_overseas_price = AsyncMock(
|
||||
return_value={"output": {"last": "182.50"}}
|
||||
)
|
||||
overseas_broker.get_overseas_balance = AsyncMock(
|
||||
return_value={
|
||||
"output1": [],
|
||||
"output2": [
|
||||
{
|
||||
"frcr_evlu_tota": "50000.00",
|
||||
"frcr_buy_amt_smtl": "0.00",
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
overseas_broker.get_overseas_buying_power = AsyncMock(
|
||||
return_value={"output": {"ovrs_ord_psbl_amt": "50000.00"}}
|
||||
)
|
||||
overseas_broker.send_overseas_order = AsyncMock(return_value={"msg1": "주문접수"})
|
||||
|
||||
engine = MagicMock(spec=ScenarioEngine)
|
||||
engine.evaluate = MagicMock(return_value=_make_buy_match("AAPL"))
|
||||
|
||||
market = MagicMock()
|
||||
market.name = "NASDAQ"
|
||||
market.code = "US_NASDAQ"
|
||||
market.exchange_code = "NASD"
|
||||
market.is_domestic = False
|
||||
|
||||
telegram = MagicMock()
|
||||
telegram.notify_trade_execution = AsyncMock()
|
||||
telegram.notify_fat_finger = AsyncMock()
|
||||
telegram.notify_circuit_breaker = AsyncMock()
|
||||
telegram.notify_scenario_matched = AsyncMock()
|
||||
|
||||
decision_logger = MagicMock()
|
||||
decision_logger.log_decision = MagicMock(return_value="decision-id")
|
||||
|
||||
settings = MagicMock()
|
||||
settings.POSITION_SIZING_ENABLED = False
|
||||
settings.CONFIDENCE_THRESHOLD = 80
|
||||
settings.USD_BUFFER_MIN = 49900.0
|
||||
settings.MODE = "paper"
|
||||
settings.PAPER_OVERSEAS_CASH = 50000.0
|
||||
|
||||
await trading_cycle(
|
||||
broker=MagicMock(),
|
||||
overseas_broker=overseas_broker,
|
||||
scenario_engine=engine,
|
||||
playbook=_make_playbook(market="US"),
|
||||
risk=MagicMock(),
|
||||
db_conn=db_conn,
|
||||
decision_logger=decision_logger,
|
||||
context_store=MagicMock(
|
||||
get_latest_timeframe=MagicMock(return_value=None),
|
||||
set_context=MagicMock(),
|
||||
),
|
||||
criticality_assessor=MagicMock(
|
||||
assess_market_conditions=MagicMock(return_value=MagicMock(value="NORMAL")),
|
||||
get_timeout=MagicMock(return_value=5.0),
|
||||
),
|
||||
telegram=telegram,
|
||||
market=market,
|
||||
stock_code="AAPL",
|
||||
scan_candidates={},
|
||||
settings=settings,
|
||||
)
|
||||
|
||||
overseas_broker.send_overseas_order.assert_not_called()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _retry_connection — unit tests (issue #209)
|
||||
@@ -3723,7 +3800,6 @@ class TestRetryConnection:
|
||||
with patch("src.main.asyncio.sleep") as mock_sleep:
|
||||
mock_sleep.return_value = None
|
||||
result = await _retry_connection(flaky, label="flaky")
|
||||
|
||||
assert result == "ok"
|
||||
assert call_count == 2
|
||||
mock_sleep.assert_called_once()
|
||||
@@ -3778,6 +3854,48 @@ class TestRetryConnection:
|
||||
assert call_count == 1 # No retry for non-ConnectionError
|
||||
|
||||
|
||||
def test_fx_buffer_guard_applies_only_to_us_and_respects_boundary() -> None:
|
||||
settings = MagicMock()
|
||||
settings.USD_BUFFER_MIN = 1000.0
|
||||
|
||||
us_market = MagicMock()
|
||||
us_market.is_domestic = False
|
||||
us_market.code = "US_NASDAQ"
|
||||
|
||||
blocked, remaining, required = _should_block_overseas_buy_for_fx_buffer(
|
||||
market=us_market,
|
||||
action="BUY",
|
||||
total_cash=5000.0,
|
||||
order_amount=4001.0,
|
||||
settings=settings,
|
||||
)
|
||||
assert blocked
|
||||
assert remaining == 999.0
|
||||
assert required == 1000.0
|
||||
|
||||
blocked_eq, _, _ = _should_block_overseas_buy_for_fx_buffer(
|
||||
market=us_market,
|
||||
action="BUY",
|
||||
total_cash=5000.0,
|
||||
order_amount=4000.0,
|
||||
settings=settings,
|
||||
)
|
||||
assert not blocked_eq
|
||||
|
||||
jp_market = MagicMock()
|
||||
jp_market.is_domestic = False
|
||||
jp_market.code = "JP"
|
||||
blocked_jp, _, required_jp = _should_block_overseas_buy_for_fx_buffer(
|
||||
market=jp_market,
|
||||
action="BUY",
|
||||
total_cash=5000.0,
|
||||
order_amount=4500.0,
|
||||
settings=settings,
|
||||
)
|
||||
assert not blocked_jp
|
||||
assert required_jp == 0.0
|
||||
|
||||
|
||||
# run_daily_session — daily CB baseline (daily_start_eval) tests (issue #207)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -5193,6 +5311,88 @@ async def test_order_policy_rejection_skips_order_execution() -> None:
|
||||
broker.send_order.assert_not_called()
|
||||
|
||||
|
||||
def test_overnight_policy_prioritizes_killswitch_over_exception() -> None:
|
||||
market = MagicMock()
|
||||
with patch("src.main.get_session_info", return_value=MagicMock(session_id="US_AFTER")):
|
||||
settings = MagicMock()
|
||||
settings.OVERNIGHT_EXCEPTION_ENABLED = True
|
||||
try:
|
||||
KILL_SWITCH.new_orders_blocked = True
|
||||
assert _should_force_exit_for_overnight(market=market, settings=settings)
|
||||
finally:
|
||||
KILL_SWITCH.clear_block()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_kill_switch_block_does_not_block_sell_reduction() -> None:
|
||||
"""KillSwitch should block BUY entries, but allow SELL risk reduction orders."""
|
||||
db_conn = init_db(":memory:")
|
||||
decision_logger = DecisionLogger(db_conn)
|
||||
|
||||
broker = MagicMock()
|
||||
broker.get_current_price = AsyncMock(return_value=(100.0, 0.5, 0.0))
|
||||
broker.get_balance = AsyncMock(
|
||||
return_value={
|
||||
"output1": [{"pdno": "005930", "ord_psbl_qty": "3"}],
|
||||
"output2": [
|
||||
{
|
||||
"tot_evlu_amt": "100000",
|
||||
"dnca_tot_amt": "50000",
|
||||
"pchs_amt_smtl_amt": "50000",
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
broker.send_order = AsyncMock(return_value={"msg1": "OK"})
|
||||
|
||||
market = MagicMock()
|
||||
market.name = "Korea"
|
||||
market.code = "KR"
|
||||
market.exchange_code = "KRX"
|
||||
market.is_domestic = True
|
||||
|
||||
telegram = MagicMock()
|
||||
telegram.notify_trade_execution = AsyncMock()
|
||||
telegram.notify_fat_finger = AsyncMock()
|
||||
telegram.notify_circuit_breaker = AsyncMock()
|
||||
telegram.notify_scenario_matched = AsyncMock()
|
||||
|
||||
settings = MagicMock()
|
||||
settings.POSITION_SIZING_ENABLED = False
|
||||
settings.CONFIDENCE_THRESHOLD = 80
|
||||
settings.OVERNIGHT_EXCEPTION_ENABLED = True
|
||||
settings.MODE = "paper"
|
||||
|
||||
try:
|
||||
KILL_SWITCH.new_orders_blocked = True
|
||||
await trading_cycle(
|
||||
broker=broker,
|
||||
overseas_broker=MagicMock(),
|
||||
scenario_engine=MagicMock(evaluate=MagicMock(return_value=_make_sell_match())),
|
||||
playbook=_make_playbook(),
|
||||
risk=MagicMock(),
|
||||
db_conn=db_conn,
|
||||
decision_logger=decision_logger,
|
||||
context_store=MagicMock(
|
||||
get_latest_timeframe=MagicMock(return_value=None),
|
||||
set_context=MagicMock(),
|
||||
),
|
||||
criticality_assessor=MagicMock(
|
||||
assess_market_conditions=MagicMock(return_value=MagicMock(value="NORMAL")),
|
||||
get_timeout=MagicMock(return_value=5.0),
|
||||
),
|
||||
telegram=telegram,
|
||||
market=market,
|
||||
stock_code="005930",
|
||||
scan_candidates={},
|
||||
settings=settings,
|
||||
)
|
||||
finally:
|
||||
KILL_SWITCH.clear_block()
|
||||
|
||||
broker.send_order.assert_called_once()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_blackout_queues_order_and_skips_submission() -> None:
|
||||
"""When blackout is active, order submission is replaced by queueing."""
|
||||
|
||||
92
tests/test_walk_forward_split.py
Normal file
92
tests/test_walk_forward_split.py
Normal file
@@ -0,0 +1,92 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from src.analysis.walk_forward_split import generate_walk_forward_splits
|
||||
|
||||
|
||||
def test_generates_sequential_folds() -> None:
|
||||
folds = generate_walk_forward_splits(
|
||||
n_samples=30,
|
||||
train_size=10,
|
||||
test_size=5,
|
||||
)
|
||||
assert len(folds) == 4
|
||||
assert folds[0].train_indices == list(range(0, 10))
|
||||
assert folds[0].test_indices == list(range(10, 15))
|
||||
assert folds[1].train_indices == list(range(5, 15))
|
||||
assert folds[1].test_indices == list(range(15, 20))
|
||||
|
||||
|
||||
def test_purge_removes_boundary_samples_before_test() -> None:
|
||||
folds = generate_walk_forward_splits(
|
||||
n_samples=25,
|
||||
train_size=8,
|
||||
test_size=4,
|
||||
purge_size=2,
|
||||
)
|
||||
first = folds[0]
|
||||
# test starts at 10, purge=2 => train end must be 7
|
||||
assert first.train_indices == list(range(0, 8))
|
||||
assert first.test_indices == list(range(10, 14))
|
||||
|
||||
|
||||
def test_embargo_excludes_post_test_samples_from_next_train() -> None:
|
||||
folds = generate_walk_forward_splits(
|
||||
n_samples=45,
|
||||
train_size=15,
|
||||
test_size=5,
|
||||
step_size=10,
|
||||
embargo_size=3,
|
||||
)
|
||||
assert len(folds) >= 2
|
||||
# Fold1 test: 15..19, next fold train window: 10..24.
|
||||
# embargo_size=3 should remove 20,21,22 from fold2 train.
|
||||
second_train = folds[1].train_indices
|
||||
assert 20 not in second_train
|
||||
assert 21 not in second_train
|
||||
assert 22 not in second_train
|
||||
assert 23 in second_train
|
||||
|
||||
|
||||
def test_respects_min_train_size_and_returns_empty_when_impossible() -> None:
|
||||
folds = generate_walk_forward_splits(
|
||||
n_samples=15,
|
||||
train_size=5,
|
||||
test_size=5,
|
||||
min_train_size=6,
|
||||
)
|
||||
assert folds == []
|
||||
|
||||
|
||||
def test_embargo_uses_last_accepted_fold_when_intermediate_fold_skips() -> None:
|
||||
folds = generate_walk_forward_splits(
|
||||
n_samples=30,
|
||||
train_size=5,
|
||||
test_size=3,
|
||||
step_size=5,
|
||||
embargo_size=1,
|
||||
min_train_size=5,
|
||||
)
|
||||
# 1st fold accepted, 2nd skipped by min_train_size, subsequent folds still generated.
|
||||
assert len(folds) == 3
|
||||
assert folds[0].test_indices == [5, 6, 7]
|
||||
assert folds[1].test_indices == [15, 16, 17]
|
||||
assert folds[2].test_indices == [25, 26, 27]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("n_samples", "train_size", "test_size"),
|
||||
[
|
||||
(0, 10, 2),
|
||||
(10, 0, 2),
|
||||
(10, 5, 0),
|
||||
],
|
||||
)
|
||||
def test_invalid_args_raise(n_samples: int, train_size: int, test_size: int) -> None:
|
||||
with pytest.raises(ValueError):
|
||||
generate_walk_forward_splits(
|
||||
n_samples=n_samples,
|
||||
train_size=train_size,
|
||||
test_size=test_size,
|
||||
)
|
||||
Reference in New Issue
Block a user