Add scanner diagnostics for zero-candidate trade stalls

- add morning report launcher\n- add overnight stop script\n- add watchdog health monitor script\n\nRefs #137

docs/issues/ISSUE-2026-02-17-no-trades-zero-candidates.md

@@ -0,0 +1,29 @@
+# Issue: Realtime 모드에서 거래가 지속적으로 0건
+
+## Summary
+`realtime` 실행 중 주문 단계까지 진입하지 못하고, 스캐너 단계에서 후보가 0건으로 반복 종료된다.
+
+## Observed
+- 로그에서 반복적으로 `Smart Scanner: No candidates ... — no trades` 출력
+- 해외 시장에서 `Overseas ranking endpoint unavailable (404)` 다수 발생
+- fallback 심볼 스캔도 `0 candidates`로 종료
+- `data/trade_logs.db` 기준 최근 구간에 `BUY/SELL` 없음
+
+## Impact
+- 매매 전략 품질과 무관하게 주문 경로가 실행되지 않아 실질 거래 불가
+- 장애 원인을 로그만으로 즉시 분해하기 어려움
+
+## Root-Cause Hypothesis
+- 스캐너 필터(가격/변동성) 단계에서 대부분 탈락
+- 해외 랭킹 API 불가 시 입력 유니버스가 빈 상태가 되어 후보 생성 실패
+- 기존 로그는 최종 결과(0 candidates)만 보여 원인별 분해가 어려움
+
+## Acceptance Criteria
+- 스캔 1회마다 탈락 사유가 구조화되어 로그에 남아야 함
+- 국내/해외(랭킹/폴백) 경로 모두 동일한 진단 지표를 제공해야 함
+- 운영자가 로그만 보고 `왜 0 candidates인지`를 즉시 판단 가능해야 함
+
+## Scope
+- 이번 이슈는 **진단 가능성 개선(Observability)** 에 한정
+- 후보 생성 전략 변경(기본 유니버스 강제 추가 등)은 별도 이슈로 분리
+

docs/prs/PR-2026-02-17-scanner-diagnostics-for-zero-candidates.md

@@ -0,0 +1,32 @@
+# PR: Smart Scanner 진단 로그 추가 (0 candidates 원인 분해)
+
+## Linked Issue
+- `docs/issues/ISSUE-2026-02-17-no-trades-zero-candidates.md`
+
+## What Changed
+- `src/analysis/smart_scanner.py`에 스캔 진단 카운터 추가
+- 국내 스캔 진단 로그 추가
+- 해외 랭킹 스캔 진단 로그 추가
+- 해외 fallback 심볼 스캔 진단 로그 추가
+
+## Diagnostics Keys
+- `total_rows`
+- `missing_code`
+- `invalid_price`
+- `low_volatility`
+- `connection_error` (해당 경로에서만)
+- `unexpected_error` (해당 경로에서만)
+- `qualified`
+
+## Expected Log Examples
+- `Domestic scan diagnostics: {...}`
+- `Overseas ranking scan diagnostics for US_NASDAQ: {...}`
+- `Overseas fallback scan diagnostics for US_NYSE: {...}`
+
+## Out of Scope
+- 해외 랭킹 404 시 기본 심볼 유니버스 강제 주입
+- 국내 경로 fallback 정책 변경
+
+## Validation
+- `.venv/bin/python -m py_compile src/analysis/smart_scanner.py`
+

scripts/morning_report.sh

@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+# Morning summary for overnight run logs.
+
+set -euo pipefail
+
+LOG_DIR="${LOG_DIR:-data/overnight}"
+
+if [ ! -d "$LOG_DIR" ]; then
+    echo "로그 디렉터리가 없습니다: $LOG_DIR"
+    exit 1
+fi
+
+latest_run="$(ls -1t "$LOG_DIR"/run_*.log 2>/dev/null | head -n 1 || true)"
+latest_watchdog="$(ls -1t "$LOG_DIR"/watchdog_*.log 2>/dev/null | head -n 1 || true)"
+
+if [ -z "$latest_run" ]; then
+    echo "run 로그가 없습니다: $LOG_DIR/run_*.log"
+    exit 1
+fi
+
+echo "Overnight report"
+echo "- run log: $latest_run"
+if [ -n "$latest_watchdog" ]; then
+    echo "- watchdog log: $latest_watchdog"
+fi
+
+start_line="$(head -n 1 "$latest_run" || true)"
+end_line="$(tail -n 1 "$latest_run" || true)"
+
+info_count="$(rg -c '"level": "INFO"' "$latest_run" || true)"
+warn_count="$(rg -c '"level": "WARNING"' "$latest_run" || true)"
+error_count="$(rg -c '"level": "ERROR"' "$latest_run" || true)"
+critical_count="$(rg -c '"level": "CRITICAL"' "$latest_run" || true)"
+traceback_count="$(rg -c 'Traceback' "$latest_run" || true)"
+
+echo "- start: ${start_line:-N/A}"
+echo "- end:   ${end_line:-N/A}"
+echo "- INFO: ${info_count:-0}"
+echo "- WARNING: ${warn_count:-0}"
+echo "- ERROR: ${error_count:-0}"
+echo "- CRITICAL: ${critical_count:-0}"
+echo "- Traceback: ${traceback_count:-0}"
+
+if [ -n "$latest_watchdog" ]; then
+    watchdog_errors="$(rg -c '\[ERROR\]' "$latest_watchdog" || true)"
+    echo "- watchdog ERROR: ${watchdog_errors:-0}"
+    echo ""
+    echo "최근 watchdog 로그:"
+    tail -n 5 "$latest_watchdog" || true
+fi
+
+echo ""
+echo "최근 앱 로그:"
+tail -n 20 "$latest_run" || true

scripts/run_overnight.sh

@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+# Start The Ouroboros overnight with logs and watchdog.
+
+set -euo pipefail
+
+LOG_DIR="${LOG_DIR:-data/overnight}"
+CHECK_INTERVAL="${CHECK_INTERVAL:-30}"
+TMUX_AUTO="${TMUX_AUTO:-true}"
+TMUX_ATTACH="${TMUX_ATTACH:-true}"
+TMUX_SESSION_PREFIX="${TMUX_SESSION_PREFIX:-ouroboros_overnight}"
+
+if [ -z "${APP_CMD:-}" ]; then
+    if [ -x ".venv/bin/python" ]; then
+        PYTHON_BIN=".venv/bin/python"
+    elif command -v python3 >/dev/null 2>&1; then
+        PYTHON_BIN="python3"
+    elif command -v python >/dev/null 2>&1; then
+        PYTHON_BIN="python"
+    else
+        echo ".venv/bin/python 또는 python3/python 실행 파일을 찾을 수 없습니다."
+        exit 1
+    fi
+
+    dashboard_port="${DASHBOARD_PORT:-8080}"
+
+    APP_CMD="DASHBOARD_PORT=$dashboard_port $PYTHON_BIN -m src.main --mode=paper --dashboard"
+fi
+
+mkdir -p "$LOG_DIR"
+
+timestamp="$(date +"%Y%m%d_%H%M%S")"
+RUN_LOG="$LOG_DIR/run_${timestamp}.log"
+WATCHDOG_LOG="$LOG_DIR/watchdog_${timestamp}.log"
+PID_FILE="$LOG_DIR/app.pid"
+WATCHDOG_PID_FILE="$LOG_DIR/watchdog.pid"
+
+if [ -f "$PID_FILE" ]; then
+    old_pid="$(cat "$PID_FILE" || true)"
+    if [ -n "$old_pid" ] && kill -0 "$old_pid" 2>/dev/null; then
+        echo "앱이 이미 실행 중입니다. pid=$old_pid"
+        exit 1
+    fi
+fi
+
+echo "[$(date -u +"%Y-%m-%dT%H:%M:%SZ")] starting: $APP_CMD" | tee -a "$RUN_LOG"
+nohup bash -lc "$APP_CMD" >>"$RUN_LOG" 2>&1 &
+app_pid=$!
+echo "$app_pid" > "$PID_FILE"
+
+echo "[$(date -u +"%Y-%m-%dT%H:%M:%SZ")] app pid=$app_pid" | tee -a "$RUN_LOG"
+
+nohup env PID_FILE="$PID_FILE" LOG_FILE="$WATCHDOG_LOG" CHECK_INTERVAL="$CHECK_INTERVAL" \
+    bash scripts/watchdog.sh >/dev/null 2>&1 &
+watchdog_pid=$!
+echo "$watchdog_pid" > "$WATCHDOG_PID_FILE"
+
+cat <<EOF
+시작 완료
+- app pid: $app_pid
+- watchdog pid: $watchdog_pid
+- app log: $RUN_LOG
+- watchdog log: $WATCHDOG_LOG
+
+실시간 확인:
+tail -f "$RUN_LOG"
+tail -f "$WATCHDOG_LOG"
+EOF
+
+if [ "$TMUX_AUTO" = "true" ]; then
+    if ! command -v tmux >/dev/null 2>&1; then
+        echo "tmux를 찾지 못해 자동 세션 생성은 건너뜁니다."
+        exit 0
+    fi
+
+    session_name="${TMUX_SESSION_PREFIX}_${timestamp}"
+    window_name="overnight"
+    tmux new-session -d -s "$session_name" -n "$window_name" "tail -f '$RUN_LOG'"
+    tmux split-window -t "${session_name}:${window_name}" -v "tail -f '$WATCHDOG_LOG'"
+    tmux select-layout -t "${session_name}:${window_name}" even-vertical
+
+    echo "tmux session 생성: $session_name"
+    echo "수동 접속: tmux attach -t $session_name"
+
+    if [ -z "${TMUX:-}" ] && [ "$TMUX_ATTACH" = "true" ]; then
+        tmux attach -t "$session_name"
+    fi
+fi

scripts/stop_overnight.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+# Stop The Ouroboros overnight app/watchdog/tmux session.
+
+set -euo pipefail
+
+LOG_DIR="${LOG_DIR:-data/overnight}"
+PID_FILE="$LOG_DIR/app.pid"
+WATCHDOG_PID_FILE="$LOG_DIR/watchdog.pid"
+TMUX_SESSION_PREFIX="${TMUX_SESSION_PREFIX:-ouroboros_overnight}"
+KILL_TIMEOUT="${KILL_TIMEOUT:-5}"
+
+stop_pid() {
+    local name="$1"
+    local pid="$2"
+
+    if [ -z "$pid" ]; then
+        echo "$name PID가 비어 있습니다."
+        return 1
+    fi
+
+    if ! kill -0 "$pid" 2>/dev/null; then
+        echo "$name 프로세스가 이미 종료됨 (pid=$pid)"
+        return 0
+    fi
+
+    kill "$pid" 2>/dev/null || true
+    for _ in $(seq 1 "$KILL_TIMEOUT"); do
+        if ! kill -0 "$pid" 2>/dev/null; then
+            echo "$name 종료됨 (pid=$pid)"
+            return 0
+        fi
+        sleep 1
+    done
+
+    kill -9 "$pid" 2>/dev/null || true
+    if ! kill -0 "$pid" 2>/dev/null; then
+        echo "$name 강제 종료됨 (pid=$pid)"
+        return 0
+    fi
+
+    echo "$name 종료 실패 (pid=$pid)"
+    return 1
+}
+
+status=0
+
+if [ -f "$WATCHDOG_PID_FILE" ]; then
+    watchdog_pid="$(cat "$WATCHDOG_PID_FILE" || true)"
+    stop_pid "watchdog" "$watchdog_pid" || status=1
+    rm -f "$WATCHDOG_PID_FILE"
+else
+    echo "watchdog pid 파일 없음: $WATCHDOG_PID_FILE"
+fi
+
+if [ -f "$PID_FILE" ]; then
+    app_pid="$(cat "$PID_FILE" || true)"
+    stop_pid "app" "$app_pid" || status=1
+    rm -f "$PID_FILE"
+else
+    echo "app pid 파일 없음: $PID_FILE"
+fi
+
+if command -v tmux >/dev/null 2>&1; then
+    sessions="$(tmux ls 2>/dev/null | awk -F: -v p="$TMUX_SESSION_PREFIX" '$1 ~ "^" p "_" {print $1}')"
+    if [ -n "$sessions" ]; then
+        while IFS= read -r s; do
+            [ -z "$s" ] && continue
+            tmux kill-session -t "$s" 2>/dev/null || true
+            echo "tmux 세션 종료: $s"
+        done <<< "$sessions"
+    else
+        echo "종료할 tmux 세션 없음 (prefix=${TMUX_SESSION_PREFIX}_)"
+    fi
+fi
+
+exit "$status"

scripts/watchdog.sh

@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+# Simple watchdog for The Ouroboros process.
+
+set -euo pipefail
+
+PID_FILE="${PID_FILE:-data/overnight/app.pid}"
+LOG_FILE="${LOG_FILE:-data/overnight/watchdog.log}"
+CHECK_INTERVAL="${CHECK_INTERVAL:-30}"
+STATUS_EVERY="${STATUS_EVERY:-10}"
+
+mkdir -p "$(dirname "$LOG_FILE")"
+
+log() {
+    printf '%s %s\n' "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" "$1" | tee -a "$LOG_FILE"
+}
+
+if [ ! -f "$PID_FILE" ]; then
+    log "[ERROR] pid file not found: $PID_FILE"
+    exit 1
+fi
+
+PID="$(cat "$PID_FILE")"
+if [ -z "$PID" ]; then
+    log "[ERROR] pid file is empty: $PID_FILE"
+    exit 1
+fi
+
+log "[INFO] watchdog started (pid=$PID, interval=${CHECK_INTERVAL}s)"
+
+count=0
+while true; do
+    if kill -0 "$PID" 2>/dev/null; then
+        count=$((count + 1))
+        if [ $((count % STATUS_EVERY)) -eq 0 ]; then
+            log "[INFO] process alive (pid=$PID)"
+        fi
+    else
+        log "[ERROR] process stopped (pid=$PID)"
+        exit 1
+    fi
+    sleep "$CHECK_INTERVAL"
+done

src/analysis/smart_scanner.py

@@ -128,6 +128,16 @@ class SmartVolatilityScanner:
         if not fluct_rows:
             return []
 
+        diagnostics: dict[str, int | float] = {
+            "total_rows": len(fluct_rows),
+            "missing_code": 0,
+            "invalid_price": 0,
+            "low_volatility": 0,
+            "connection_error": 0,
+            "unexpected_error": 0,
+            "qualified": 0,
+        }
+
         volume_rank_bonus: dict[str, float] = {}
         for idx, row in enumerate(volume_rows):
             code = _extract_stock_code(row)
@@ -139,6 +149,7 @@ class SmartVolatilityScanner:
         for stock in fluct_rows:
             stock_code = _extract_stock_code(stock)
             if not stock_code:
+                diagnostics["missing_code"] += 1
                 continue
 
             try:
@@ -168,7 +179,11 @@ class SmartVolatilityScanner:
                             volume_ratio = max(volume_ratio, volume / prev_day_volume)
 
                 volatility_pct = max(abs(change_rate), intraday_range_pct)
-                if price <= 0 or volatility_pct < 0.8:
+                if price <= 0:
+                    diagnostics["invalid_price"] += 1
+                    continue
+                if volatility_pct < 0.8:
+                    diagnostics["low_volatility"] += 1
                     continue
 
                 volatility_score = min(volatility_pct / 10.0, 1.0) * 85.0
@@ -189,14 +204,22 @@ class SmartVolatilityScanner:
                         score=score,
                     )
                 )
+                diagnostics["qualified"] += 1
 
             except ConnectionError as exc:
+                diagnostics["connection_error"] += 1
                 logger.warning("Failed to analyze %s: %s", stock_code, exc)
                 continue
             except Exception as exc:
+                diagnostics["unexpected_error"] += 1
                 logger.error("Unexpected error analyzing %s: %s", stock_code, exc)
                 continue
 
+        logger.info(
+            "Domestic scan diagnostics: %s (volatility_threshold=0.8, top_n=%d)",
+            diagnostics,
+            self.top_n,
+        )
         logger.info("Domestic ranking scan found %d candidates", len(candidates))
         candidates.sort(key=lambda c: c.score, reverse=True)
         return candidates[: self.top_n]
@@ -242,6 +265,14 @@ class SmartVolatilityScanner:
         if not fluct_rows:
             return []
 
+        diagnostics: dict[str, int | float] = {
+            "total_rows": len(fluct_rows),
+            "missing_code": 0,
+            "invalid_price": 0,
+            "low_volatility": 0,
+            "qualified": 0,
+        }
+
         volume_rank_bonus: dict[str, float] = {}
         try:
             volume_rows = await self.overseas_broker.fetch_overseas_rankings(
@@ -266,6 +297,7 @@ class SmartVolatilityScanner:
         for row in fluct_rows:
             stock_code = _extract_stock_code(row)
             if not stock_code:
+                diagnostics["missing_code"] += 1
                 continue
 
             price = _extract_last_price(row)
@@ -275,7 +307,11 @@ class SmartVolatilityScanner:
             volatility_pct = max(abs(change_rate), intraday_range_pct)
 
             # Volatility-first filter (not simple gainers/value ranking).
-            if price <= 0 or volatility_pct < 0.8:
+            if price <= 0:
+                diagnostics["invalid_price"] += 1
+                continue
+            if volatility_pct < 0.8:
+                diagnostics["low_volatility"] += 1
                 continue
 
             volatility_score = min(volatility_pct / 10.0, 1.0) * 85.0
@@ -295,7 +331,14 @@ class SmartVolatilityScanner:
                     score=score,
                 )
             )
+            diagnostics["qualified"] += 1
 
+        logger.info(
+            "Overseas ranking scan diagnostics for %s: %s (volatility_threshold=0.8, top_n=%d)",
+            market.code,
+            diagnostics,
+            self.top_n,
+        )
         if candidates:
             logger.info(
                 "Overseas ranking scan found %d candidates for %s",
@@ -315,6 +358,19 @@ class SmartVolatilityScanner:
             logger.info("Overseas scanner: no symbol universe for %s", market.name)
             return []
 
+        logger.info(
+            "Overseas scanner: scanning %d fallback symbols for %s",
+            len(symbols),
+            market.name,
+        )
+        diagnostics: dict[str, int | float] = {
+            "total_rows": len(symbols),
+            "invalid_price": 0,
+            "low_volatility": 0,
+            "connection_error": 0,
+            "unexpected_error": 0,
+            "qualified": 0,
+        }
         candidates: list[ScanCandidate] = []
         for stock_code in symbols:
             try:
@@ -328,7 +384,11 @@ class SmartVolatilityScanner:
                 intraday_range_pct = _extract_intraday_range_pct(output, price)
                 volatility_pct = max(abs(change_rate), intraday_range_pct)
 
-                if price <= 0 or volatility_pct < 0.8:
+                if price <= 0:
+                    diagnostics["invalid_price"] += 1
+                    continue
+                if volatility_pct < 0.8:
+                    diagnostics["low_volatility"] += 1
                     continue
 
                 score = min(volatility_pct / 10.0, 1.0) * 100.0
@@ -346,10 +406,24 @@ class SmartVolatilityScanner:
                         score=score,
                     )
                 )
+                diagnostics["qualified"] += 1
             except ConnectionError as exc:
+                diagnostics["connection_error"] += 1
                 logger.warning("Failed to analyze overseas %s: %s", stock_code, exc)
             except Exception as exc:
+                diagnostics["unexpected_error"] += 1
                 logger.error("Unexpected error analyzing overseas %s: %s", stock_code, exc)
+        logger.info(
+            "Overseas fallback scan diagnostics for %s: %s (volatility_threshold=0.8, top_n=%d)",
+            market.code,
+            diagnostics,
+            self.top_n,
+        )
+        logger.info(
+            "Overseas symbol fallback scan found %d candidates for %s",
+            len(candidates),
+            market.name,
+        )
         return candidates
 
     def get_stock_codes(self, candidates: list[ScanCandidate]) -> list[str]:

src/broker/kis_api.py

@@ -104,12 +104,14 @@ class KISBroker:
             time_since_last_attempt = now - self._last_refresh_attempt
             if time_since_last_attempt < self._refresh_cooldown:
                 remaining = self._refresh_cooldown - time_since_last_attempt
-                error_msg = (
+                # Do not fail fast here. If token is unavailable, upstream calls
-                    f"Token refresh on cooldown. "
+                # will all fail for up to a minute and scanning returns no trades.
-                    f"Retry in {remaining:.1f}s (KIS allows 1/minute)"
+                logger.warning(
+                    "Token refresh on cooldown. Waiting %.1fs before retry (KIS allows 1/minute)",
+                    remaining,
                 )
-                logger.warning(error_msg)
+                await asyncio.sleep(remaining)
-                raise ConnectionError(error_msg)
+                now = asyncio.get_event_loop().time()
 
             logger.info("Refreshing KIS access token")
             self._last_refresh_attempt = now

src/broker/overseas.py

@@ -82,14 +82,19 @@ class OverseasBroker:
         session = self._broker._get_session()
 
         if ranking_type == "volume":
-            tr_id = self._broker._settings.OVERSEAS_RANKING_VOLUME_TR_ID
+            configured_tr_id = self._broker._settings.OVERSEAS_RANKING_VOLUME_TR_ID
-            path = self._broker._settings.OVERSEAS_RANKING_VOLUME_PATH
+            configured_path = self._broker._settings.OVERSEAS_RANKING_VOLUME_PATH
+            default_tr_id = "HHDFS76200200"
+            default_path = "/uapi/overseas-price/v1/quotations/inquire-volume-rank"
         else:
-            tr_id = self._broker._settings.OVERSEAS_RANKING_FLUCT_TR_ID
+            configured_tr_id = self._broker._settings.OVERSEAS_RANKING_FLUCT_TR_ID
-            path = self._broker._settings.OVERSEAS_RANKING_FLUCT_PATH
+            configured_path = self._broker._settings.OVERSEAS_RANKING_FLUCT_PATH
+            default_tr_id = "HHDFS76200100"
+            default_path = "/uapi/overseas-price/v1/quotations/inquire-updown-rank"
 
-        headers = await self._broker._auth_headers(tr_id)
+        endpoint_specs: list[tuple[str, str]] = [(configured_tr_id, configured_path)]
-        url = f"{self._broker._base_url}{path}"
+        if (configured_tr_id, configured_path) != (default_tr_id, default_path):
+            endpoint_specs.append((default_tr_id, default_path))
 
         # Try common param variants used by KIS overseas quotation APIs.
         param_variants = [
@@ -100,24 +105,38 @@ class OverseasBroker:
         ]
 
         last_error: str | None = None
-        for params in param_variants:
+        saw_http_404 = False
-            try:
+        for tr_id, path in endpoint_specs:
-                async with session.get(url, headers=headers, params=params) as resp:
+            headers = await self._broker._auth_headers(tr_id)
-                    text = await resp.text()
+            url = f"{self._broker._base_url}{path}"
-                    if resp.status != 200:
+            for params in param_variants:
-                        last_error = f"HTTP {resp.status}: {text}"
+                try:
-                        continue
+                    async with session.get(url, headers=headers, params=params) as resp:
+                        text = await resp.text()
+                        if resp.status != 200:
+                            last_error = f"HTTP {resp.status}: {text}"
+                            if resp.status == 404:
+                                saw_http_404 = True
+                            continue
 
-                    data = await resp.json()
+                        data = await resp.json()
-                    rows = self._extract_ranking_rows(data)
+                        rows = self._extract_ranking_rows(data)
-                    if rows:
+                        if rows:
-                        return rows[:limit]
+                            return rows[:limit]
 
-                    # keep trying another param variant if response has no usable rows
+                        # keep trying another param variant if response has no usable rows
-                    last_error = f"empty output (keys={list(data.keys())})"
+                        last_error = f"empty output (keys={list(data.keys())})"
-            except (TimeoutError, aiohttp.ClientError) as exc:
+                except (TimeoutError, aiohttp.ClientError) as exc:
-                last_error = str(exc)
+                    last_error = str(exc)
-                continue
+                    continue
+
+        if saw_http_404:
+            logger.warning(
+                "Overseas ranking endpoint unavailable (404) for %s/%s; using symbol fallback scan",
+                exchange_code,
+                ranking_type,
+            )
+            return []
 
         raise ConnectionError(
             f"fetch_overseas_rankings failed for {exchange_code}/{ranking_type}: {last_error}"