diff --git a/README.md b/README.md index 017491f..bbd200b 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ ### 一个可本地运行的多模态 Agent 桌面应用 —— 真正能"动手"操作你电脑的 AI 助手 -基于 **LangGraph** 构建，具备 GUI 自动化、浏览器控制、终端执行、RAG 知识库、邮件收发、图像生成、语音对话、数据库操作、定时任务等 19 项工具，并提供 Electron 桌面端一键安装体验。 +基于 **LangGraph** 构建，具备 GUI 自动化、浏览器控制、终端执行、RAG 知识库、邮件收发、图像生成、语音对话、数据库操作、定时任务等 20 项工具，并提供 Electron 桌面端一键安装体验。 minor Agent 主界面 @@ -22,10 +22,11 @@ > 🎓 **项目初衷**：这是一个 LangChain + LangGraph 的学习项目。基于两者构建 Agent 工作流，结构清晰、上手容易、扩展方便，适合作为 Agent 开发的参考实现。 - 🖥️ **真能操作桌面**：截图 → 视觉定位 → 键鼠模拟，闭环完成 ERP 录入、表单填写、软件操控 -- 🧩 **19 项内置工具**：从终端命令到邮件收发，从 PPT 制作到图像生成，覆盖真实办公场景 +- 🧩 **20 项内置工具**：从终端命令到邮件收发，从 PPT 制作到图像生成，覆盖真实办公场景 - 🏠 **完全本地运行**：Electron 打包，数据不出本机；后端 FastAPI + 前端零延迟直读磁盘 - 🎨 **可深度定制**：主题 / 字体 / 字号 / 背景 / 工作空间 / 提示词分层架构，全部可调 -- 🔁 **工程级稳定性**：循环检测、上下文压缩、人工确认、轨迹回放，对抗 LLM 的"幻觉操作" +- 🔁 **工程级稳定性**：循环检测、上下文压缩、阻塞式人工确认、轨迹回放，对抗 LLM 的"幻觉操作" +- 💾 **存储双后端（接口隔离）**：会话与定时任务存储统一接口驱动，**JSON 可视化 / MySQL** 一键切换、不双写；新增存储方式只需实现接口，消费方无感知后端 > 💡 项目演进：`Web 在线应用` → `Electron 桌面应用`。源码同一套 `src/`，既可由 `uvicorn` 作 Web 服务启动，也可由 Electron 主进程拉起，**双模式共用**。 @@ -38,7 +39,7 @@ ## ✨ 功能亮点 -### 🛠️ 19 项工具能力 +### 🛠️ 20 项工具能力 | 工具 | 文件 | 能力 | |------|------|------| @@ -50,17 +51,18 @@ | 📄 **doc** | `tools/doc.py` | 创建/读取/编辑 docx / xlsx / pptx / pdf / txt | | 📂 **directory_listing** | `tools/directory_listing.py` | 递归列目录、文件元信息 | | 📨 **send_file** | `tools/send_file.py` | 以邮件附件形式发送本地文件 | -| 🔍 **web_search** | `tools/web_search.py` | 联网搜索（Tavily / Firecrawl） | +| 🔍 **web_search** | `tools/web_search.py` | 联网搜索（Tavily / Firecrawl / 火山引擎） | | 🗄️ **text2sql** | `tools/text2sql.py` | 自然语言/SQL 操作数据库：查表结构、查询、增删改 | | 📚 **rag** | `tools/rag.py` | ChromaDB 向量检索 + Reranker 重排序 | | 🎨 **image_gen** | `tools/image_gen.py` | 调用 Z-Image-Turbo 生成图片 | | ✅ **todo_list** | `tools/todo_list.py` | 多步任务规划与进度跟踪 | -| 🤝 **human_interaction** | `tools/human_interaction.py` | 主动询问用户、收集信息 | +| 🤝 **human_interaction** | `tools/human_interaction.py` | 阻塞式询问用户：收集信息 / 多选一，答案作为普通工具返回值 | | 🤖 **agent_call** | `tools/agent_call.py` | 委派子 Agent 处理子任务 | | 🧭 **skill_router** | `tools/skill_router.py` | 匹配并路由到预定义 Skill 流程 | | 📅 **date_query** | `tools/date_query.py` | 当前日期时间查询 | | ⏰ **cron_manager** | `tools/cron_manager.py` | 定时任务管理：查看/新增/修改/删除定时任务，支持 cron/间隔/一次性触发 | -| 🗣️ **streaming_client** | `tts/streaming_client.py` | 流式 TTS 语音合成客户端 | +| 📊 **hard_excel_read** | `tools/hard_excel_read.py` | 复杂 Excel 表结构读取：前 N 行原始数据 + 维度/合并区/数据区范围建议，供大模型解析结构 | +| 📥 **excel2sql** | `tools/excel2sql.py` | 按大模型给定的数据行列范围，将 Excel 数据区流式批量入库 MySQL（自动建库建表、表注释） | ### 🎯 区别于其他 Agent 项目的核心设计 @@ -70,7 +72,7 @@ | **视觉反馈** | 仅返回坐标文本 | **截图作为视觉闭环**注入上下文，多模态 LLM 看到真实屏幕状态 | | **坐标定位** | 逐元素请求 | **批量定位** `_batch_ground_elements`：N 个元素 1 次子模型调用 | | **循环失控** | 无防护，token 烧穿 | **两级循环检测**：连续 3 次同工具警告反思 → 连续 5 次强制终止 | -| **上下文膨胀** | 简单截断 | **图内分级压缩**：50% 窗口轻量压缩 / 75% 窗口全量压缩，每步 LLM 调用后检查 | +| **上下文膨胀** | 简单截断 | **图内分级压缩**：token 超 `窗口×COMPRESS_RATE` 触发，压缩历史工具调用为累积摘要，每步 LLM 调用后检查 | | **部署形态** | 仅 Web / 仅 API | **同一套源码双模式**：`uvicorn` Web 服务 ↔ Electron 桌面应用 | | **文件操作** | HTTP 上传下载 | 桌面端走 **IPC 直读磁盘**，零延迟；Web 端走 `/api/fs/*` | | **提示词** | 单层 system prompt | **四层提示词架构**：系统级 / 工具描述 / 运行时注入 / 内部子模型 | @@ -84,7 +86,7 @@ Agent 运行过程中的关键交互界面，支持文件从任意位置**拖拽 人机交互确认 任务规划 TodoList
-人机交互确认 — 敏感操作（终端执行、文件写入等）弹出确认浮窗，用户可同意、拒绝或修改指令 | 任务规划 TodoList — 多步任务自动拆解与进度跟踪，支持主 / 子 Agent 独立任务列表 +人机交互确认 — 敏感操作（终端执行、文件写入等）与信息收集以浮窗形式实时弹出，用户可同意、拒绝、跳过或补充信息；**拒绝也只是一次工具返回值**，Agent 自行调整方案继续，不会结束本轮 | 任务规划 TodoList — 多步任务自动拆解与进度跟踪，支持主 / 子 Agent 独立任务列表

@@ -136,7 +138,9 @@ Agent 运行过程中的关键交互界面，支持文件从任意位置**拖拽 │ FastAPI 后端 (web/server.py) │ │ 会话管理 · 多模态收发 · SSE 流式 · 文件服务 · 工具调用推送 │ └───────────────────────────────┬──────────────────────────────┘ - │ execute_agent() + │ turn_runner.start_turn() + │（每会话一个后台线程，HTTP 请求 + │ 挂载等待；断连不影响执行） ▼ ┌──────────────────────────────────────────────────────────────┐ │ Agent 编排层 (agents/agent_runtime.py) │ @@ -147,36 +151,45 @@ Agent 运行过程中的关键交互界面，支持文件从任意位置**拖拽 ┌──────────────────────────────────────────────────────────────┐ │ LangGraph ReAct 图 (core/graph.py) │ │ agent 节点 → tools 节点 → process_tool_artifact 节点 │ -│ 内含: 反思注入 · 压缩 · 循环检测 · 路由 · 人工确认 │ +│ 内含: 反思注入 · 压缩 · 循环检测 · 路由 │ +│ 人机交互: 阻塞式普通工具（core/human_request.py） │ └───────────────────────────────┬──────────────────────────────┘ │ 调用 ▼ ┌──────────────────────────────────────────────────────────────┐ -│ 19 项工具 (tools/*.py) · 5 项技能 (skills/) │ +│ 20 项工具 (tools/*.py) · 5 项技能 (skills/) │ └──────────────────────────────────────────────────────────────┘ ``` ### ReAct 循环（图内） ``` -START → [agent: call_model] ──有 tool_calls──▶ [tools: ToolNode] ──▶ [process_tool_artifact] - ▲ ◀────────────────────────┘ - │ 无 tool_calls - ▼ +START → [agent: call_model] ──有 tool_calls──▶ [tools: 并行工具节点] ──▶ [process_tool_artifact] ──▶ [compress] + ▲ ▲ │ │ + │ 无 tool_calls │ ◀────────────────────────────┘ + ▼ └── 子 Agent 结果 / 工具返回值注入 ◀── 并行工具节点（线程池） END ``` -- **`call_model`**（`core/nodes.py`）：注入反思提示 → LLM 调用 → 每步检查压缩 → 提取 thinking -- **`should_continue`**（`core/routing.py`）：含 tool_calls → 检查确认策略 / 循环检测 → 路由到 `tools` 或 `END` +- **`call_model`**（`core/nodes.py`）：注入反思提示 → LLM 调用 → 每步检查压缩（token 超阈值标记）→ 提取 thinking +- **`should_continue`**（`core/routing.py`）：含 tool_calls → 循环检测 → 路由到 `tools` 或 `END`（需人工确认的工具不在此暂停，而是在工具执行钳点阻塞征求用户意见，见「阻塞式人机交互」） - **`process_tool_artifact`**：gui 截图作为 synthetic HumanMessage 注入，实现视觉闭环 +- **`compress`**（`core/nodes.py`）：token 超 `窗口×COMPRESS_RATE` 时，将历史工具调用压缩为累积摘要（见「实现细节」） ### 多 Agent 模式 +多 Agent 并非独立图，而是**同一张 react 图内的工具级并行**：`tools` 节点由 `make_parallel_tool_node`（线程池）取代 ToolNode，一轮内的多个工具调用并行执行；其中 `agent_call` 工具会以主 Agent 当前消息快照启动**子 Agent 的独立图实例**执行子任务，结果以工具返回值注入主 Agent 上下文。子 Agent 内的人机交互同样为阻塞式——在子图线程中等待浮窗应答，完成后自然回到主图： + ``` -START → [supervisor] ──main完成──▶ [parallel_sub_agents] ──并行完成──▶ [supervisor] - │ │ - └─ 子Agent结果 tag_message 注入主Agent ◄┘ - 有 agent_call → 循环；无 → END +[agent] ──有 agent_call──▶ [tools: parallel_tool_node（线程池）] + │ + ├─ 普通工具：直接并行执行 + └─ agent_call：spawn 子 Agent Runtime → 子图 invoke + │（子图内 human_interaction 阻塞等待 + │ 浮窗应答，答案作为工具返回值） + └─ 结果作为 ToolMessage 注入主 Agent ◄┐ + ▲ │ + └──────────────── 有 agent_call → 继续循环；无 → END ──────────────────────┘ ``` --- @@ -193,27 +206,29 @@ Agent/ ├── src/ │ ├── agent/ # 🧠 Agent 核心 │ │ ├── agents/ -│ │ │ └── agent_runtime.py # 编排层入口：execute_agent / continue_after_human_action +│ │ │ └── agent_runtime.py # 编排层入口：execute_agent（动态 Agent 工厂） │ │ ├── core/ │ │ │ ├── graph.py # LangGraph 构建（单 Agent + 多 Agent） │ │ │ ├── nodes.py # ReAct 节点：call_model / process_tool_artifact / 压缩 │ │ │ ├── runtime.py # 图执行器：消息拼装、历史加载、轨迹落盘 │ │ │ ├── routing.py # should_continue 路由 + 强制终止 │ │ │ ├── state.py # Graph 状态定义（messages / agent_mode） -│ │ │ ├── loop_detector.py # 循环检测：重复工具 / TodoList 停滞，两级响应 +│ │ │ ├── loop_detector.py # 循环检测：重复工具调用，两级响应 +│ │ │ ├── human_request.py # ★ 人工请求注册表：交互类工具的通用阻塞通道（ask_human） +│ │ │ ├── turn_runner.py # ★ 后台线程轮次执行器（解耦 HTTP 与图执行生命周期） │ │ │ ├── llm.py # ChatQwen 多模态封装（图像/附件/非标准 tool_call） │ │ │ ├── tool_policy.py # 工具执行策略（直执 / 需确认） -│ │ │ ├── approvals.py # 待确认操作注册与 UI 元数据 +│ │ │ ├── storage.py # ★ 统一存储层：接口 + JSON/MySQL 双实现 + 工厂 + scope │ │ │ └── config_manager.py# 配置读写：agent/tool/env/gui/model/theme │ │ ├── memory/ │ │ │ └── system_prompt.py # ★ 四层提示词：MAIN / PLAN / REFLECTION / VISION... -│ │ ├── tools/ # 19 项工具（见上表） +│ │ ├── tools/ # 20 项工具（见上表） │ │ │ ├── ...（gui / browser_control / terminal_execute / email / doc / web_search / text2sql / rag / cron_manager 等） │ │ ├── cron/ # ⏰ 定时任务引擎 │ │ │ ├── models.py # 数据模型（CronTask / Trigger） -│ │ │ ├── storage.py # 持久化层（CronTaskRepository，JSON / MySQL） +│ │ │ ├── storage.py # 转发层（实现统一在 core/storage.py） │ │ │ ├── scheduler.py # 调度计算与触发扫描 -│ │ │ ├── runner.py # 任务执行器（子进程） +│ │ │ ├── runner.py # 任务执行器（子进程，切换存储 cron 作用域） │ │ │ └── live.py # 常驻调度循环 │ │ ├── skills/ # 5 项技能（skill.json + skill.md） │ │ │ ├── netease_mail_read/ @@ -224,8 +239,8 @@ Agent/ │ │ ├── tts/ │ │ │ └── streaming_client.py # 流式 TTS 客户端（SSE 收音频块） │ │ ├── history/ -│ │ │ ├── session_storage.py # 会话与消息持久化（JSON / MySQL 双后端） -│ │ │ └── tool_call_recorder.py # 每轮工具调用轨迹记录 (turn_{id}.json) +│ │ │ ├── session_storage.py # 转发层（实现统一在 core/storage.py） +│ │ │ └── tool_call_recorder.py # 每轮工具调用轨迹记录 (tool_{turn_id}.json) │ │ ├── utils/ # env_utils / agent_utils / image_utils / ppt_utils / tool_call_utils │ │ └── config/ # ★ 配置文件目录 │ │ ├── env_config.json # 运行时配置（首次向导生成，勿手动改） @@ -237,12 +252,12 @@ Agent/ │ │ └── workspace_config.json # 工作空间列表与当前选择 │ │ │ └── web/ # 🎨 前端 UI -│ ├── server.py # FastAPI 后端（50+ 路由） +│ ├── server.py # FastAPI 后端（75+ 路由） │ ├── ui_session.py # UI 会话状态管理 │ ├── index.html # 主页面（侧边栏/聊天/编辑/Agent面板/操作栏） │ ├── app.js / app.css # 入口 -│ ├── js/ # 28 个前端模块（见开发手册） -│ ├── css/ # 16 个样式模块 + variables.css 主题变量 +│ ├── js/ # 30 个前端模块（见开发手册） +│ ├── css/ # 18 个样式模块 + variables.css 主题变量 │ ├── packages/ # Monaco Editor 本地 npm 包 │ └── image/ # SVG 图标资源 │ @@ -361,22 +376,21 @@ cloudflared tunnel --url http://localhost:8765 |------|--------|------| | `models[]` | `SETUP_REQUIRED` | LLM 列表：`{id, name, model, api_key, base_url, timeout, max_retries}` | | `gui_model_id` | `""` | GUI 视觉定位专用模型（留空则回退主模型） | -| `WORKING_DIR` | `.` | 工作根目录（桌面端自动设为安装目录） | -| `WORKSPACE_DIR` | `workspace` | 默认工作空间子目录 | +| `WORKING_DIR` | 项目根目录 | 工作根目录（桌面端自动设为安装目录） | +| `WORKSPACE_DIR` | `""` | 工作空间目录（相对 WORKING_DIR），留空则使用 WORKING_DIR | | `USER_PYTHON_PATH` | `""` | 指定 Python 解释器路径 | | `LLM_CONTEXT_WINDOW` | `262144` | 上下文窗口大小（token） | -| `MINI_COMPRESS_RATE` | `0.4` | 轻量压缩阈值（占窗口 40% 触发） | -| `HARD_COMPRESS_RATE` | `0.7` | 全量压缩阈值（占窗口 70% 触发） | -| `CURRENT_TURNS` | `4` | 短期记忆保留的最近工具轮数 | +| `COMPRESS_RATE` | `0.6` | 上下文压缩阈值比例（0~1），token 用量超过 `窗口×该值` 时触发压缩 | +| `THINKING_LEVEL` | `"low"` | 深度思考档位：low / high / xhigh / max / ultra（xhigh 及以上启用深度思考） | | `IMG_SIZE` | `768` | 截图短边尺寸 | | `GROUNDING_WIDTH/HEIGHT` | `1000` | 视觉定位输入分辨率 | -| `LOOP_DETECT_TODOLIST_STALE_ROUNDS` | `20` | TodoList 停滞多少轮触发反思 | | `LOOP_DETECT_REPEATED_TOOL_WARN` | `3` | 连续同工具同参多少次警告 | | `LOOP_DETECT_REPEATED_TOOL_END` | `5` | 连续同工具同参多少次强制终止 | | `RAG_CHUNK_SIZE / OVERLAP` | `500 / 50` | RAG 分块大小与重叠 | +| `IMAGE_GEN_URL` | `http://localhost:8904` | 图像生成服务地址 | | `SEND_FILE_SIZE_LIMIT` | `30` | 发送文件大小上限（MB） | | `WEB_SEARCH_API_KEY` | `""` | 联网搜索 API Key | -| `WEB_SEARCH_ENGINE` | `"tavily"` | 搜索引擎选择（tavily / firecrawl） | +| `WEB_SEARCH_ENGINE` | `"tavily"` | 搜索引擎选择（tavily / firecrawl / volcengine） | | `STORAGE_BACKEND` | `"json"` | 存储后端（json / mysql） | | `MYSQL_HOST` / `MYSQL_PORT` / `MYSQL_USER` / `MYSQL_PASSWORD` / `MYSQL_DATABASE` | `""` | MySQL 连接参数（STORAGE_BACKEND=mysql 时生效） | | `TOOL_TIMEOUT` | `300` | 工具执行超时时间（秒） | @@ -433,21 +447,52 @@ cloudflared tunnel --url http://localhost:8765 ``` 每次 LLM 调用后读取 response.usage_metadata.total_tokens - ├─ > 50% 窗口 → 轻量压缩：保留本轮首个 agent 动作 - ├─ > 75% 窗口 → 全量压缩：仅保留当前 user 消息 - └─ 就地替换 state["messages"] = [SystemMessage(摘要) + 保留部分] + ├─ 未超阈值 → 不处理，继续下一轮 + ├─ 主 Agent 超阈值 → 标记压缩，由 compress 节点统一执行： + │ 将注入的历史工具调用与返回压缩为累积摘要（RemoveMessage 移除已覆盖消息） + └─ 子 Agent 超阈值 → 注入整理提示词，整理进度后直接返回主 Agent（子 Agent 不压缩） ``` -压缩发生在 `call_model` 内部（图内每步），而非图外一次性处理，**子 Agent 自动受益**。 +- **触发**：`call_model` 内每步检查 `total_tokens`，超过 `LLM_CONTEXT_WINDOW × COMPRESS_RATE`（默认 0.6）即标记压缩 +- **执行**：压缩发生在图内 `compress` 节点（`process_tool_artifact` 之后），只压缩注入的历史工具上下文，系统提示词与对话历史保留，摘要按压缩游标累积 +- **子 Agent**：不做压缩，超限时整理任务进度直接返回主 Agent，避免上下文继续膨胀 ### 3. 循环检测两级响应 | 级别 | 触发条件 | 响应 | |------|----------|------| -| ⚠️ 警告 | 连续 3 次同工具同参 / TodoList 停滞 20 轮 | 注入反思提示引导模型自我纠偏 | +| ⚠️ 警告 | 连续 3 次同工具同参 | 注入反思提示引导模型自我纠偏 | | 🛑 终止 | 连续 5 次同工具同参 | `should_force_end()` 返回 END，向前端输出错误报告 | -### 4. GUI 动作序列化（性能关键） +### 4. 阻塞式人机交互（交互类 / 展示类工具的统一模式） + +人机交互不再"暂停图 + 存快照 + 续跑"，而是**阻塞式普通工具**：工具在图上线程中等待前端浮窗应答，答案（同意 / 拒绝 / 跳过 / 补充信息）只是普通工具返回值，图自然继续——**拒绝不会结束整轮**，Agent 自行调整方案。 + +``` +LLM 发出 human_interaction / 需确认工具调用 + → tools 节点执行工具 + → human_request.ask_human() 阻塞等待（注册到请求表，随 live snapshot 推送前端） + → 前端浮窗出现，工具调用行显示"运行中" + → 用户点击 → POST /api/human-action → 写入答案并唤醒 + → 工具返回答案文本 → ToolMessage → 图继续（无快照重放、无续跑状态机） +``` + +| 设计点 | 传统做法（本项目旧版） | 阻塞式（当前） | +|--------|----------------------|----------------| +| 图执行 | 请求线程内同步 invoke | 后台线程（`core/turn_runner.py`），断连/关页不影响执行，双发请求挂同一轮 | +| 等待答案 | 图 END + 消息快照存 `PENDING_TOOL_APPROVALS` | 工具内 `threading.Event` 阻塞（`core/human_request.py`） | +| 决策处理 | `continue_after_human_action` 重放快照续跑 | 答案即返回值，无续跑路径 | +| 拒绝 | 结束整轮（等同手动暂停） | 仅返回值，Agent 调整后继续 | +| 工具列表 | 确认时清空重建、续跑时重放 | 自然展示：等待中 `running` → 应答后 `done` | +| 子 Agent 交互 | 异常冒泡（`SubAgentPendingError`）+ 恢复状态机 | 子图线程内直接阻塞，完成自然回主图 | +| 无人值守（cron） | 悬死靠超时兜底 | `ask_human` 检测 headless 立即返回默认答案 | + +**两类工具的扩展模式**（新增能力零框架改动）： + +- **交互类工具**（如 `human_interaction`）：工具内调一次 `ask_human(meta)` 阻塞，自己格式化返回值 +- **展示类工具**（如 `todo_list`）：纯文本返回、不维护全局 store，前端从通用 live 记录（工具调用的 args/结果）按工具名写一个小 adapter 渲染浮窗（如 `js/todo.js` 的 `updateTodoOverlayFromRecords`） + +### 5. GUI 动作序列化（性能关键） | 模式 | N 个动作的 LLM 调用次数 | |------|------------------------| @@ -456,7 +501,7 @@ cloudflared tunnel --url http://localhost:8765 通过 `MAIN_SYSTEM_PROMPT` + `REFLECTION_PROMPT` 双重强调"同页操作必须合并为一次 `gui` 调用"实现。 -### 5. 双模式源码共用 +### 6. 双模式源码共用 `src/web/` 同一套代码两种部署： @@ -469,6 +514,36 @@ cloudflared tunnel --url http://localhost:8765 前端通过 `window.electronAPI?.isElectron` 自动适配。 +### 7. 存储双后端（JSON 可视化 / MySQL，统一接口隔离） + +会话历史与定时任务存储统一在 **`agent/core/storage.py`**：接口（Protocol）驱动、双后端实现、**消费方零感知后端选择**（不判断 `STORAGE_BACKEND`）。 + +**接口与实现** +- `ConversationRecordRepository`（对话消息 / 工具调用 / 轮次元数据 / 会话生命周期，16 方法）与 `TaskConfigRepository`（cron 任务配置，6 方法）两个 Protocol +- 每后端一个实现类：`JsonRecordRepository` / `MysqlRecordRepository`（含会话与 cron 双作用域）、`JsonTaskConfigRepository` / `MysqlTaskConfigRepository` +- **新增存储后端只需实现上述接口并在工厂注册一行**；消费方新增功能/字段无需改动存储层（消息原样 JSON 存储、工具记录 meta 全量透传） + +**作用域（scope）** +- `session`（默认）：主进程普通会话，JSON 落盘 `history/sessions/{session_id}/`，MySQL 用 `sessions / session_messages / session_tool_calls` 三表 +- `cron`：定时任务，JSON 落盘 `history/cron/{task_id}/`，MySQL 用 `cron_messages / cron_tool_calls` 两表；cron 子进程切换存储作用域后**直写**记录，主进程经 cron 作用域仓库读取 + +| 后端 | 存储位置 | 说明 | +|------|----------|------| +| **json**（默认） | `history/sessions/{session_id}/` | 消息与工具调用以人类可读 JSON 文件落盘（`turn_{turn_id}.json` / `tool_{turn_id}.json` / `session_meta.json` / `_session_tokens.json`），可直接打开查看/修改，便于审计与调试 | +| **mysql** | session 三表 / cron 两表 | 消息与工具调用记录原样 JSON 入库、读取只走数据库；二进制产物（图片/附件等）两个后端均落盘 | + +- 由 `STORAGE_BACKEND` + `MYSQL_HOST / MYSQL_PORT / MYSQL_USER / MYSQL_PASSWORD / MYSQL_DATABASE` 配置，`agent/core/db.py` 提供统一连接与幂等建表 +- **不双写**：选哪种后端，读写就只走该后端；写入幂等（同一 `(session_id, turn_id)` 先 DELETE 再 INSERT）；DB 异常仅打印日志，不阻断主流程 +- 分支（`/api/sessions/branch`）与回滚等会话功能均基于存储接口实现，两个后端一致可用 + +**双后端行为差异（设计内）** + +| 场景 | json | mysql | +|------|------|-------| +| 轮次中实时工具记录 | 每次工具结果后**增量写盘** `tool_*.json`，进程崩溃后重启可回退恢复最近记录 | 不增量入库，由轮次结束 `save_tool_calls` 统一落库（无额外磁盘 IO，但崩溃时该轮记录丢失） | +| 服务重启后运行中轮次 | `get_live_tool_calls` 回退读取增量文件，可继续展示 | 内存记录丢失、数据库无该轮记录，实时面板为空 | +| cron 历史读取 | 直接读 `history/cron/` 下文件 | 子进程直写 cron 表，历史可正常读取 | + --- ## 🔌 LLM 推理服务 @@ -516,8 +591,10 @@ cloudflared tunnel --url http://localhost:8765 | `canvas-editor.js` | 画板编辑器（画笔/形状/文字/橡皮擦，支持缩放与图片修改） | | `audio.js` | 语音录制与播放 | | `i18n.js` | 国际化 | -| `token-bar.js` | Token 用量条 | -| `pending.js` / `pending-overlay.js` | 待确认操作 | +| `token-ring.js` | 上下文 Token 用量环形指示器（含压缩阈值刻度与三类占比） | +| `access-mode.js` | 工作空间访问模式切换（限制访问 / 权限审查 / 完全访问） | +| `think-level.js` | 思考模式档位选择（low / high / xhigh / max / ultra） | +| `pending.js` / `pending-overlay.js` | 人工请求浮窗与决策提交（数据源为 live snapshot 的 `human_requests`） | | `rollback.js` | 消息回滚 | | `workspace.js` | 工作空间切换 | | `dialog.js` | 对话框 | @@ -525,13 +602,13 @@ cloudflared tunnel --url http://localhost:8765 ### CSS 架构（`src/web/css/`） -`variables.css` 定义主题变量，其余按模块拆分：`base / layout / chat / composer / cron / dialog / settings / skills / todo / toolcalls / edit-mode / audio / animations / pending / pending-overlay / responsive`。 +`variables.css` 定义主题变量，其余按模块拆分：`base / layout / chat / composer / cron / dialog / settings / skills / todo / toolcalls / edit-mode / canvas-editor / audio / animations / pending / pending-overlay / toast / responsive`。 ### 添加自定义工具 1. 在 `src/agent/tools/` 新建 `my_tool.py`，继承 LangChain `BaseTool`，实现 `_run`，填写 `name` / `description` / `args_schema` -2. 在 `tool_config.json` 注册并配置启用状态与参数 -3. （可选）在 `tool_policy.py` 设置是否需要人工确认 +2. 在 `tool_config.json` 注册并配置启用状态与参数（权限设为 `confirm` 时，工具执行前会自动弹出确认浮窗，同意后执行、拒绝/跳过作为返回值继续） +3. 交互类工具（需要向用户提问）在 `_run` 中调用 `core/human_request.ask_human(meta)` 阻塞等待答案；展示类工具（如进度面板）返回纯文本即可，前端从 live 工具记录按工具名写 adapter 渲染浮窗 4. 重启后端，工具自动绑定到 LLM ### 添加自定义技能 @@ -557,7 +634,7 @@ Copy-Item "src\web\js\themes.js" "src\web\js\themes.js" -Force - **后端日志**：桌面端 `npm start` 会在终端打印 `[FastAPI]` 日志 - **端口占用**：`main.js` 的 `killPortProcess(8765)` 启动前自动清理 - **重置配置**：删除 `src/agent/config/env_config.json` 重触首次向导 -- **工具调用轨迹**：`history/tool_calling/{session_id}/turn_{id}.json` +- **工具调用轨迹**：`history/sessions/{session_id}/tool_{turn_id}.json` --- @@ -571,7 +648,7 @@ Copy-Item "src\web\js\themes.js" "src\web\js\themes.js" -Force | 🌐 **首次启动联网** | 加载 Google Fonts 需联网；离线环境字体选择器有 HTML 硬编码兜底 | | 🔑 **API Key 安全** | `env_config.json` 含明文凭证，**勿提交到公开仓库**（已在 `.gitignore` 排除 `env_config.json`，仅保留 `env_config_dist.json` 模板） | | 🎨 **图标** | 当前为 SVG，NSIS 打包后若显示默认图标，需准备 `.ico` 文件替换 `src/web/image/icon.ico` | -| 💾 **历史存储** | 会话历史与工具轨迹落盘在 `history/`，长期使用注意清理 | +| 💾 **历史存储** | 默认会话历史与工具轨迹以 JSON 落盘在 `history/`（可切换 MySQL 后端），长期使用注意清理 | | ⚡ **GUI 性能** | 同页多操作务必合并为一次 `gui` 调用，否则延迟显著（见「实现细节」） | --- @@ -580,7 +657,7 @@ Copy-Item "src\web\js\themes.js" "src\web\js\themes.js" -Force -- [minor](https://github.com/minorQAQ)：项目话事人，主导整体架构设计与核心实现（LangGraph 编排、19 项工具体系、多 Agent 模式、上下文分级压缩、循环检测、Electron 桌面端等），完成大部分的开发工作。 +- [minor](https://github.com/minorQAQ)：项目话事人，主导整体架构设计与核心实现（LangGraph 编排、20 项工具体系、多 Agent 模式、上下文分级压缩、循环检测、Electron 桌面端等），完成大部分的开发工作。 --- diff --git a/src/agent/agents/__init__.py b/src/agent/agents/__init__.py index 2df55c5..1c2dd8f 100644 --- a/src/agent/agents/__init__.py +++ b/src/agent/agents/__init__.py @@ -20,7 +20,6 @@ from agent.agents.agent_runtime import ( get_agent_runtime, reload_all_agent_runtimes, execute_agent, - continue_after_human_action, main_agent_runtime, AgentMode, ) @@ -36,6 +35,5 @@ __all__ = [ "get_all_agent_runtimes", "get_agent_runtime", "execute_agent", - "continue_after_human_action", "AgentMode", ] diff --git a/src/agent/agents/agent_runtime.py b/src/agent/agents/agent_runtime.py index 37bd89b..77fb687 100644 --- a/src/agent/agents/agent_runtime.py +++ b/src/agent/agents/agent_runtime.py @@ -3,7 +3,7 @@ 系统定位: 实现从配置文件全动态加载 Agent 实例。所有 Agent 的运行参数（图类型、 模型、工具集、系统提示词、最大迭代次数）均由 agent_config.json 驱动。 - 同时提供 Web/API 层的入口函数 execute_agent 和 continue_after_human_action。 + 同时提供 Web/API 层的入口函数 execute_agent。 使用方式: from agent.agents.agent_runtime import get_main_agent_runtime, reload_all_agent_runtimes, execute_agent @@ -15,7 +15,6 @@ from typing import Any, Literal from agent.core.config_manager import load_agent_configs, load_models, AgentConfig from agent.core.graph import build_agent_graph -from agent.core.runtime import continue_after_human_action as continue_runtime_after_human_action from agent.core.runtime import execute_agent as execute_runtime_agent from agent.core.runtime import AgentRuntime from agent.tools import get_tools @@ -265,24 +264,6 @@ def execute_agent( ) -def continue_after_human_action( - chat_history: list[dict[str, Any]], - session_id: str, - approval_id: str, - decision: Literal["approve", "reject", "edit"], - instruction: str | None = None, -) -> list[dict[str, Any]]: - """Web/API 层入口：用户对待确认项决策后继续执行 main_agent。""" - return continue_runtime_after_human_action( - runtime=main_agent_runtime, - chat_history=chat_history, - session_id=session_id, - approval_id=approval_id, - decision=decision, - instruction=instruction, - ) - - # ---------- 首次导入时从配置构建主 Agent 运行时并缓存引用 ---------- _built = build_all_agent_runtimes() main_agent_runtime = get_main_agent_runtime() diff --git a/src/agent/core/__init__.py b/src/agent/core/__init__.py index 65bc090..f464605 100644 --- a/src/agent/core/__init__.py +++ b/src/agent/core/__init__.py @@ -3,9 +3,10 @@ 包含: - state / graph: LangGraph 状态与图拓扑 - nodes / routing: 图节点与条件路由 - - runtime: 对外执行入口 execute_agent / continue_after_human_action + - runtime: 对外执行入口 execute_agent + - human_request: 人工请求注册表（交互类工具的通用阻塞通道） + - turn_runner: 后台线程轮次执行器 - llm: ChatQwen 多模态网关 - - approvals: HITL 待确认管理（含 human_interaction 参数规范化） - tool_policy: 工具执行安全策略 系统定位: diff --git a/src/agent/core/approvals.py b/src/agent/core/approvals.py deleted file mode 100644 index 7da5621..0000000 --- a/src/agent/core/approvals.py +++ /dev/null @@ -1,259 +0,0 @@ -"""待确认操作（HITL）：内存注册、UI 元数据构建、参数规范化与结果生成。 - -系统定位: - 维护进程内 ``PENDING_TOOL_APPROVALS`` 字典；在图因需确认工具而暂停时 - 保存上下文并构建前端 UI 元数据；在用户操作后生成 ToolMessage 供图继续执行。 - -内容分区: - - PENDING 存储与元数据构建（build_approval_meta / build_human_interaction_meta） - - UI 消息辅助（remove_pending_ui_message） - - human_interaction 工具的参数规范化与结果文案（normalize / result） - -注意: - 当前为内存存储，进程重启后待确认项丢失；生产环境可改为 Redis/DB。 - -可扩展性: - - 增加 TTL、session 级清理、持久化后端。 - - ``build_*_meta`` 可与前端组件协议版本化。 - - 新增 interaction_type 时同步扩展 normalize / result 分支。 -""" -from __future__ import annotations - -import json -import uuid -from typing import Any, Sequence - -from langchain_core.messages import BaseMessage -from langchain_core.messages import messages_to_dict - -from agent.tools.human_interaction import HumanInteraction -from agent.utils.tool_call_utils import json_safe - -# ============================================================ -# PENDING 存储 -# ============================================================ - -PENDING_TOOL_APPROVALS: dict[str, dict[str, Any]] = {} - - -# ============================================================ -# UI 元数据构建 -# ============================================================ - -def build_approval_meta(session_id: str, tool_call: dict[str, Any], messages: Sequence[BaseMessage], agent_name: str = "主Agent") -> dict[str, Any]: - """为需确认的普通工具调用构建 UI 元数据并注册 pending。 - - 输入: - session_id: 会话 ID。 - tool_call: normalize 后的工具调用 dict。 - messages: 当前图内消息序列（序列化后存入 pending）。 - agent_name: 发起该待确认项的 agent 名称（主 Agent 默认 "主Agent"）。 - - 输出: - ``{"pending_action": {...}}``，含 id、type=tool_call、args、options 等。 - """ - approval_id = uuid.uuid4().hex - tool_name = tool_call.get("name") or "unknown_tool" - args = json_safe(tool_call.get("args") or {}) - display_call = {"name": tool_name, "args": args} - instruction = json.dumps(display_call, ensure_ascii=False, indent=2) - PENDING_TOOL_APPROVALS[approval_id] = { - "session_id": session_id, - "type": "tool_call", - "tool_call": {"id": tool_call.get("id"), "name": tool_name, "args": args}, - "messages": messages_to_dict(list(messages)), - "agent_name": agent_name, - "is_sub_agent": agent_name not in ("", "主Agent", "__main__"), - } - return { - "pending_action": { - "id": approval_id, - "type": "tool_call", - "title": f"待确认执行工具：{tool_name}", - "tool_name": tool_name, - "args": display_call, - "instruction": instruction, - "options": ["approve", "reject"], - "agent_name": agent_name, - "is_sub_agent": agent_name not in ("", "主Agent", "__main__"), - "resolved": False, - } - } - - -def build_human_interaction_meta(session_id: str, tool_call: dict[str, Any], messages: Sequence[BaseMessage], agent_name: str = "主Agent") -> dict[str, Any]: - """为 human_interaction 工具构建 UI 元数据并注册 pending。 - - 输入: - session_id: 会话 ID。 - tool_call: 含 name=args 的工具调用。 - messages: 当前消息序列。 - agent_name: 发起该待确认项的 agent 名称（主 Agent 默认 "主Agent"）。 - - 输出: - ``{"pending_action": {...}}``，type=human_interaction，含 interaction_type 等。 - """ - approval_id = uuid.uuid4().hex - args = normalize_human_interaction_args(tool_call.get("args") or {}) - interaction_type = args.get("interaction_type") or "information" - default_titles = { - "information": "请补充信息", - "selection": "请选择一个选项", - } - PENDING_TOOL_APPROVALS[approval_id] = { - "session_id": session_id, - "type": "human_interaction", - "tool_call": {"id": tool_call.get("id"), "name": "human_interaction", "args": args}, - "messages": messages_to_dict(list(messages)), - "agent_name": agent_name, - "is_sub_agent": agent_name not in ("", "主Agent", "__main__"), - } - return { - "pending_action": { - "id": approval_id, - "type": "human_interaction", - "interaction_type": interaction_type, - "title": args.get("title") or default_titles.get(str(interaction_type), "需要人工处理"), - "prompt": args.get("prompt") or "请处理后继续。", - "tool_name": "human_interaction", - "args": { - "交互类型": interaction_type, - "说明": args.get("prompt") or "请处理后继续。", - "可选项": args.get("options") or [], - }, - "instruction": args.get("suggested_response") or "", - "options": args.get("options") or [], - "questions": args.get("questions") or None, - "agent_name": agent_name, - "is_sub_agent": agent_name not in ("", "主Agent", "__main__"), - "resolved": False, - } - } - - -def remove_pending_ui_message(chat_history: list[dict[str, Any]], approval_id: str) -> None: - """在 chat_history 中将指定 approval_id 的 pending_action 标记为已解决（原地修改）。""" - for msg in reversed(chat_history): - pending = ((msg.get("meta") or {}).get("pending_action") or {}) if isinstance(msg, dict) else {} - if pending.get("id") == approval_id: - pending["resolved"] = True - msg.setdefault("meta", {})["pending_action"] = pending - return - - -def get_pending_actions_for_session(session_id: str) -> list[dict[str, Any]]: - """获取指定会话所有未解决的 pending_action 列表（human_interaction + tool_call + sub_agent）。 - - 从 PENDING_TOOL_APPROVALS 中扫描，返回可供前端浮窗渲染的 pending_action 字典列表。 - 每项携带 agent_name / is_sub_agent，供前端按 agent 分组弹窗。 - """ - actions = [] - for approval_id, entry in PENDING_TOOL_APPROVALS.items(): - if entry.get("session_id") != session_id: - continue - is_sub = bool(entry.get("is_sub_agent")) - agent_name = entry.get("agent_name") or ("主Agent" if not is_sub else "子Agent") - entry_type = entry.get("type") - if entry_type == "human_interaction": - args = normalize_human_interaction_args((entry.get("tool_call") or {}).get("args") or {}) - interaction_type = args.get("interaction_type") or "information" - default_titles = { - "information": "请补充信息", - "selection": "请选择一个选项", - } - actions.append({ - "id": approval_id, - "type": "human_interaction", - "interaction_type": interaction_type, - "title": args.get("title") or default_titles.get(str(interaction_type), "需要人工处理"), - "prompt": args.get("prompt") or "请处理后继续。", - "instruction": args.get("suggested_response") or "", - "options": args.get("options") or [], - "questions": args.get("questions") or None, - "tool_name": "human_interaction", - "agent_name": agent_name, - "is_sub_agent": is_sub, - "resolved": False, - }) - elif entry_type == "tool_call": - tool_call = entry.get("tool_call") or {} - tool_name = tool_call.get("name") or "unknown_tool" - args = tool_call.get("args") or {} - display_call = {"name": tool_name, "args": args} - actions.append({ - "id": approval_id, - "type": "tool_call", - "title": f"待确认执行工具：{tool_name}", - "tool_name": tool_name, - "args": display_call, - "instruction": json.dumps(display_call, ensure_ascii=False, indent=2), - "options": ["approve", "skip"] if is_sub else ["approve", "reject", "skip"], - "agent_name": agent_name, - "is_sub_agent": is_sub, - # 工作空间访问策略：越界审批原因（无则为空串） - "policy_note": entry.get("policy_note") or "", - "resolved": False, - }) - return actions - - -# ============================================================ -# human_interaction 工具的参数规范化与结果文案 -# ============================================================ - -def normalize_human_interaction_args(args: dict[str, Any]) -> dict[str, Any]: - """规范化 human_interaction 工具参数为固定 schema。 - - 输入: - args: 模型传入的原始参数字典。 - - 输出: - 含 interaction_type、title、prompt、options、suggested_response、questions 的字典； - interaction_type 限定为 information | selection。 - """ - normalized = dict(args or {}) - normalized["interaction_type"] = str(normalized.get("interaction_type") or "information") - if normalized["interaction_type"] not in {"information", "selection"}: - normalized["interaction_type"] = "information" - normalized["title"] = str(normalized.get("title") or "需要人工处理") - normalized["prompt"] = str(normalized.get("prompt") or "请处理后继续。") - normalized["options"] = HumanInteraction.normalize_options(normalized.get("options")) - normalized["suggested_response"] = str(normalized.get("suggested_response") or "") - # 规范化 questions 字段 - questions = normalized.get("questions") - if isinstance(questions, str): - try: - questions = json.loads(questions) - except (json.JSONDecodeError, TypeError): - questions = None - if isinstance(questions, list): - normalized_questions = [] - for q in questions: - if isinstance(q, dict): - nq = { - "question": str(q.get("question", "")), - "options": HumanInteraction.normalize_options(q.get("options")), - } - normalized_questions.append(nq) - normalized["questions"] = normalized_questions if normalized_questions else None - else: - normalized["questions"] = None - return normalized - - -def human_interaction_result(pending: dict[str, Any], decision: str, instruction: str | None) -> str: - """根据用户决策生成写入 ToolMessage 的文本结果。 - - 输入: - pending: PENDING_TOOL_APPROVALS 中 type=human_interaction 的条目。 - decision: approve | reject | edit。 - instruction: 用户输入文本（补充信息、选项、修改说明等）。 - - 输出: - 供 LangGraph 继续推理的 ToolMessage 字符串内容。 - """ - tool_call = pending.get("tool_call") or {} - args = normalize_human_interaction_args(tool_call.get("args") or {}) - interaction_type = args.get("interaction_type") or "information" - options = args.get("options") or [] - return HumanInteraction.format_result(interaction_type, options, instruction or "") diff --git a/src/agent/core/human_request.py b/src/agent/core/human_request.py new file mode 100644 index 0000000..7d9c6c4 --- /dev/null +++ b/src/agent/core/human_request.py @@ -0,0 +1,165 @@ +"""人工请求注册表：交互类工具与确认钳点的通用阻塞通道。 + +系统定位: + 交互类工具的通用框架服务。工具只需调用 ``ask_human`` 阻塞等待答案并自行 + 格式化返回值；等待期间请求会随 live snapshot 推送到前端渲染浮窗， + 用户通过 ``POST /api/human-action`` 回答后阻塞解除、图自然继续—— + 同意/拒绝/补充信息都只是普通工具返回值，拒绝不会结束整轮任务。 + + 无人值守（AGENT_HEADLESS，如 cron）时 ``ask_human`` 立即返回默认答案， + 避免任务悬死；用户暂停（abort）时阻塞解除并抛 ``ABORTED_BY_USER``， + 由 runtime 统一按暂停处理。 + +可扩展性: + 新交互类工具 = 工具内调一次 ``ask_human(meta)``，框架零改动。 +""" +from __future__ import annotations + +import os +import threading +import time +import uuid +from contextvars import ContextVar +from typing import Any + +# ---------- 当前 agent 名称（用于前端浮窗按 agent 分组） ---------- +# 主 Agent 使用保留键 "__main__"；子 Agent 使用其 agent_name。 +# ContextVar + 全局 fallback 双轨：主线程/线程池子进程均可读到正确值； +# HTTP 请求线程（前端轮询）无上下文 → fallback 默认 "__main__"。 +_current_agent_name: ContextVar[str] = ContextVar("current_agent_name", default="") +_agent_name_fallback_lock = threading.Lock() +_agent_name_fallback = "__main__" + + +def set_current_agent_name(agent_name: str) -> None: + """设置当前线程的 agent 名称（主 Agent 传 "__main__"，子 Agent 传其名称）。""" + global _agent_name_fallback + _current_agent_name.set(agent_name or "") + with _agent_name_fallback_lock: + _agent_name_fallback = agent_name or "__main__" + + +def get_current_agent_name() -> str: + """获取当前 agent 名称；ContextVar 为空时回退到全局 fallback（默认 "__main__"）。""" + name = _current_agent_name.get() + if name: + return name + with _agent_name_fallback_lock: + return _agent_name_fallback or "__main__" + + +# ---------- 等待中的请求注册表 ---------- +# request_id -> {session_id, event, answer, meta, created_at} +_WAITING_REQUESTS: dict[str, dict[str, Any]] = {} +_requests_lock = threading.Lock() + +_ABORT_CHECK_INTERVAL = 0.2 # 等待循环中检查 abort 标志的间隔（秒） + +_DEFAULT_HEADLESS_ANSWER = "无人值守环境，无法进行人工交互，请基于现有信息自行决策。" + + +def _notify(session_id: str) -> None: + """推送一次 live snapshot，使前端浮窗数据源（human_requests）即时更新。""" + try: + from agent.history.tool_call_recorder import _notify_live + _notify_live(session_id) + except Exception: + pass + + +def ask_human(session_id: str, meta: dict[str, Any], timeout: float | None = None) -> dict[str, Any]: + """阻塞等待人工应答，返回 ``{"decision": str, "instruction": str}``。 + + 输入: + session_id: 会话 ID。 + meta: 浮窗渲染数据（id/type/title/prompt/options/questions/instruction/ + agent_name/is_sub_agent/policy_note 等），由调用方按前端协议构造。 + timeout: 等待超时秒数；None 表示无限等待。 + + 输出: + 用户决策字典。decision 取值与前端一致：approve | reject | edit | skip； + 无人值守时立即返回默认 reject；超时返回 decision="timeout"。 + + 异常: + 等待期间检测到 abort 标志时抛 RuntimeError("ABORTED_BY_USER")， + 由 runtime 统一按手动暂停处理。 + """ + # 无人值守（cron/headless）：不阻塞，立即返回默认答案 + if os.environ.get("AGENT_HEADLESS"): + return {"decision": "reject", "instruction": _DEFAULT_HEADLESS_ANSWER} + + request_id = uuid.uuid4().hex + entry: dict[str, Any] = { + "session_id": session_id, + "request_id": request_id, + "event": threading.Event(), + "answer": None, + "meta": dict(meta or {}), + "created_at": time.time(), + } + entry["meta"].setdefault("id", request_id) + with _requests_lock: + _WAITING_REQUESTS[request_id] = entry + _notify(session_id) + + try: + deadline = None if timeout is None else time.time() + timeout + while True: + # 每次醒来先检查 abort：暂停优先于答案 + try: + from agent.core.runtime import check_abort + if session_id and check_abort(session_id): + raise RuntimeError("ABORTED_BY_USER") + except RuntimeError: + raise + except Exception: + pass + if entry["event"].wait(timeout=_ABORT_CHECK_INTERVAL): + # 醒来后再检查一次 abort：暂停优先于答案，避免与 resolve 的竞态 + try: + from agent.core.runtime import check_abort + if session_id and check_abort(session_id): + raise RuntimeError("ABORTED_BY_USER") + except RuntimeError: + raise + except Exception: + pass + break + if deadline is not None and time.time() >= deadline: + return {"decision": "timeout", "instruction": "等待人工应答超时"} + return entry["answer"] or {"decision": "reject", "instruction": "未收到应答"} + finally: + with _requests_lock: + _WAITING_REQUESTS.pop(request_id, None) + _notify(session_id) + + +def answer_human_request(session_id: str, request_id: str, decision: str, instruction: str = "") -> bool: + """回答一个等待中的人工请求（由 POST /api/human-action 调用）。 + + 输出: + True 表示找到并已应答；False 表示请求不存在或不属于该会话（已过期）。 + """ + with _requests_lock: + entry = _WAITING_REQUESTS.get(request_id) + if entry is None or entry.get("session_id") != session_id: + return False + entry["answer"] = {"decision": decision, "instruction": instruction or ""} + entry["event"].set() + return True + + +def get_human_requests_for_session(session_id: str) -> list[dict[str, Any]]: + """获取指定会话所有等待中的人工请求（供 live snapshot 推送给前端渲染浮窗）。""" + with _requests_lock: + return [dict(e["meta"]) for e in _WAITING_REQUESTS.values() if e.get("session_id") == session_id] + + +def resolve_all_for_session(session_id: str, decision: str = "reject", instruction: str = "") -> int: + """一次性应答指定会话的所有等待请求（abort/清会话时调用），返回应答数量。""" + with _requests_lock: + entries = [e for e in _WAITING_REQUESTS.values() if e.get("session_id") == session_id] + for e in entries: + e["answer"] = {"decision": decision, "instruction": instruction or ""} + e["event"].set() + return len(entries) diff --git a/src/agent/core/loop_detector.py b/src/agent/core/loop_detector.py index ed08dd9..69bc098 100644 --- a/src/agent/core/loop_detector.py +++ b/src/agent/core/loop_detector.py @@ -1,8 +1,8 @@ """Agent 死循环检测模块。 系统定位: - 在 ReAct 循环的 agent 节点和路由节点之间，检测工具调用死循环、 - todolist 进度停滞等异常模式，提供分级响应： + 在 ReAct 循环的 agent 节点和路由节点之间，检测连续相同工具调用的异常模式， + 提供分级响应： - Level 1: 注入反思提示引导模型自我纠正 - Level 2: 强制终止图执行并向用户报告 @@ -16,10 +16,9 @@ from __future__ import annotations import hashlib import json -import threading -from typing import Any, Optional +from typing import Any -from langchain_core.messages import AIMessage, ToolMessage +from langchain_core.messages import AIMessage from agent.utils.tool_call_utils import normalize_tool_call, format_args_summary @@ -35,53 +34,10 @@ def get_loop_config() -> dict: except Exception: env = {} return { - "todolist_stale_rounds": int(env.get("LOOP_DETECT_TODOLIST_STALE_ROUNDS", 10)), "repeated_tool_warn": int(env.get("LOOP_DETECT_REPEATED_TOOL_WARN", 3)), "repeated_tool_end": int(env.get("LOOP_DETECT_REPEATED_TOOL_END", 5)), } -# ============================================================ -# 状态存储（按 session_id，线程安全） -# ============================================================ - -_lock = threading.Lock() -_session_state: dict[str, dict] = {} - - -def _get_session_state(session_id: str) -> dict: - """获取或创建 session 级循环检测状态。""" - if not session_id: - session_id = "_default" - with _lock: - if session_id not in _session_state: - _session_state[session_id] = { - "round_count": 0, - "last_todolist_done_count": -1, - "todolist_stale_rounds": 0, - } - return _session_state[session_id] - - -def reset_session(session_id: str) -> None: - """重置指定 session 的循环检测状态（每轮用户输入时调用）。""" - if not session_id: - session_id = "_default" - with _lock: - _session_state[session_id] = { - "round_count": 0, - "last_todolist_done_count": -1, - "todolist_stale_rounds": 0, - } - - -def clear_session(session_id: str) -> None: - """清除指定 session 的循环检测状态（会话结束时调用）。""" - if not session_id: - session_id = "_default" - with _lock: - _session_state.pop(session_id, None) - - # ============================================================ # 工具调用历史分析 # ============================================================ @@ -181,86 +137,6 @@ def detect_repeated_tool_calls( return consecutive, last_call.get("name", "unknown"), args_summary -# ============================================================ -# Todolist 进度停滞检测 -# ============================================================ - -def _get_current_todolist_done_count(session_id: str) -> int: - """获取当前 todolist 的已完成步骤数。""" - try: - from agent.tools.todo_list import get_todo_store_data - todo = get_todo_store_data(session_id) - if todo and todo.get("steps"): - return len(todo.get("done_steps", [])) - except Exception: - pass - return -1 - - -def check_and_record_todolist_progress( - messages: list, - session_id: str = "", -) -> tuple[bool, int, str]: - """检查 todolist 进度是否停滞，并更新内部状态。 - - 每轮 call_model 调用时调用一次。随着 round_count 递增， - 对比当前 done_count 与上次记录的 done_count，统计停滞轮数。 - - 输入: - messages: 当前消息列表。 - session_id: 会话 ID。 - - 输出: - (是否停滞超过阈值, 停滞轮数, todolist 状态摘要) - 需要注入反思提示时返回 True。 - """ - cfg = get_loop_config() - stale_threshold = max(cfg["todolist_stale_rounds"], 1) - - state = _get_session_state(session_id) - state["round_count"] = state.get("round_count", 0) + 1 - - current_done = _get_current_todolist_done_count(session_id) - - # 如果没有 todolist，不检查 - if current_done < 0: - state["last_todolist_done_count"] = -1 - state["todolist_stale_rounds"] = 0 - return False, 0, "" - - last_done = state.get("last_todolist_done_count", -1) - - if current_done != last_done: - # 有进展，重置停滞计数 - state["last_todolist_done_count"] = current_done - state["todolist_stale_rounds"] = 0 - return False, 0, "" - else: - # 无进展，递增停滞计数 - state["todolist_stale_rounds"] = state.get("todolist_stale_rounds", 0) + 1 - stale = state["todolist_stale_rounds"] - - # 构建 todolist 摘要 - try: - from agent.tools.todo_list import get_todo_store_data - todo = get_todo_store_data(session_id) - steps = todo.get("steps", []) if todo else [] - done_set = set(todo.get("done_steps", [])) if todo else set() - next_undone = "" - for i, s in enumerate(steps): - if i not in done_set: - next_undone = s[:50] - break - total = len(steps) - summary = f"总共 {total} 步，当前卡在步骤 {len(done_set)+1}/{total}：{next_undone}" - except Exception: - summary = f"已停滞 {stale} 轮" - - if stale >= stale_threshold: - return True, stale, summary - return False, stale, summary - - def should_force_end( messages: list, session_id: str = "", @@ -319,19 +195,6 @@ def build_loop_reflection_prompt( f"3. 如果确实无法推进，向用户说明当前困难并请求帮助" ) - # 检查 2: Todolist 进度停滞 - is_stale, stale_rounds, stale_summary = check_and_record_todolist_progress( - messages, session_id - ) - stale_threshold = max(cfg["todolist_stale_rounds"], 1) - - if is_stale: - parts.append( - f"【进度停滞警告】TodoList 已有 {stale_rounds} 轮没有进度" - + (f"（{stale_summary}）。" if stale_summary else "。") - + f"\n请反思当前卡点，考虑是否需要调整计划或换一种方式继续。" - ) - if not parts: return None diff --git a/src/agent/core/nodes.py b/src/agent/core/nodes.py index 4626cd2..874fbad 100644 --- a/src/agent/core/nodes.py +++ b/src/agent/core/nodes.py @@ -115,35 +115,6 @@ def _filter_short_term_memory(messages: list, trajectory_rounds: int) -> list: return cleaned -def _get_todolist_status() -> str: - """获取当前会话的 todolist 状态摘要。 - - 输出: - todolist 状态文本；无 todolist 时返回空字符串。 - """ - try: - from agent.history.tool_call_recorder import get_current_session - from agent.tools.todo_list import get_todo_store_data - sid = get_current_session() - if not sid: - return "" - todo = get_todo_store_data(sid) - if not todo or not todo.get("steps"): - return "" - steps = todo["steps"] - done = set(todo.get("done_steps", [])) - lines = [] - for i, s in enumerate(steps): - status = "[已完成]" if i in done else "[待完成]" - lines.append(f" {i+1}. {status} {s}") - total = len(steps) - done_count = len(done) - header = f"当前TodoList进度：{done_count}/{total} 已完成" - return header + "\n" + "\n".join(lines) - except Exception: - return "" - - def _build_reflection_prompt(agent_mode: str, is_first_call: bool, messages: list, session_id: str = "") -> str | None: """构建反思提示词，引导 Agent 在下一步执行前进行思考。 @@ -156,20 +127,13 @@ def _build_reflection_prompt(agent_mode: str, is_first_call: bool, messages: lis 输出: 反思提示文本，不需要反思时返回 None。 """ - todo_status = _get_todolist_status() - # 非 plan 模式首次调用，不需要反思 if is_first_call: return None - # 后续调用：引导模型先思考再行动 + # 后续调用：引导模型先思考再行动（TodoList 状态由工具返回值提供，无需额外注入） parts = [REFLECTION_PROMPT] - if todo_status: - parts.append(todo_status) - else: - parts.append("当前无 TodoList。") - # 循环检测：注入反循环反思提示 loop_warning = build_loop_reflection_prompt(messages, session_id) if loop_warning: @@ -526,12 +490,6 @@ def make_call_model_node( sid = get_current_session() if sid: record_tool_call_live(sid, tool_name, tool_args) - if tool_name == "todo_list": - try: - from agent.tools.todo_list import update_todo_store_from_tool_args - update_todo_store_from_tool_args(sid, tool_args) - except Exception: - pass except Exception: pass diff --git a/src/agent/core/routing.py b/src/agent/core/routing.py index 40fbd69..ecf8fa2 100644 --- a/src/agent/core/routing.py +++ b/src/agent/core/routing.py @@ -1,27 +1,22 @@ -"""LangGraph 条件路由：工具循环 vs 结束 vs 待确认暂停。 +"""LangGraph 条件路由：工具循环 vs 结束。 系统定位: - 连接 ``graph`` 的条件边与 ``runtime.execute_agent`` 的 - 后处理逻辑，在模型输出 tool_calls 时决定图走向或提取 UI 待确认元数据。 + 连接 ``graph`` 的条件边，在模型输出 tool_calls 时决定图走向。 + 人工确认不再由图级暂停承接：confirm 工具 / human_interaction 均为 + 阻塞式普通工具，在 ``tool_call_utils.invoke_tool_and_build_message`` + 钳点处阻塞等待用户决策，图自然继续。 可扩展性: - ``should_continue`` 可扩展为多目标路由（如专用 RAG 节点）。 - - ``first_pending_tool_meta`` 可支持多个并行待确认项的优先级队列。 """ from __future__ import annotations -from typing import Any, Literal, Sequence +from typing import Literal -from langchain_core.messages import BaseMessage from langgraph.graph import END -from agent.core.approvals import PENDING_TOOL_APPROVALS, build_approval_meta, build_human_interaction_meta -from agent.core.loop_detector import should_force_end as _loop_should_force_end from agent.core.state import AgentState -from agent.core.tool_policy import classify_tool_execution -from agent.core.workspace_policy import check_violation as _policy_check_violation -from agent.core.workspace_policy import decision as _policy_decision -from agent.utils.tool_call_utils import normalize_tool_call +from agent.core.loop_detector import should_force_end as _loop_should_force_end def should_continue(state: AgentState) -> Literal["tools", "__end__"]: @@ -31,12 +26,11 @@ def should_continue(state: AgentState) -> Literal["tools", "__end__"]: state: 含 messages 的 AgentState。 输出: - ``"tools"`` 进入 ToolNode；``"__end__"`` 结束图（含需确认工具场景、 + ``"tools"`` 进入 ToolNode；``"__end__"`` 结束图（无待执行工具调用、 循环检测强制终止）。 系统定位: ``graph`` 中 agent 节点的 conditional_edges 路由函数。 - 若任一 tool_call 策略为 confirm，则直接 END，由 runtime 展示确认 UI。 若检测到死循环（连续相同工具调用超阈值），则强制 END。 可扩展性: @@ -55,62 +49,5 @@ def should_continue(state: AgentState) -> Literal["tools", "__end__"]: force_end, loop_tool, loop_args = _loop_should_force_end(messages, _sid) if force_end: return END - - for raw_tool_call in last_msg.tool_calls: - tool_call = normalize_tool_call(raw_tool_call) - if classify_tool_execution(tool_call.get("name"), tool_call.get("args") or {}) == "confirm": - return END - # 权限审查模式：越界操作暂停图，交由前端人工审批 - if _policy_decision(tool_call.get("name"), tool_call.get("args") or {}) == "approve": - return END return "tools" return END - - -def first_pending_tool_meta(messages: Sequence[BaseMessage], session_id: str) -> dict[str, Any] | None: - """从图执行结果中提取第一个需 UI 处理的待确认项元数据。 - - 输入: - messages: 图 invoke 返回的完整消息列表。 - session_id: 当前会话 ID，写入 PENDING_TOOL_APPROVALS。 - - 输出: - 含 ``pending_action`` 的 meta 字典；无待确认项时返回 None。 - - 系统定位: - ``runtime.execute_agent`` 在图结束后调用，决定是否追加确认型 assistant 消息。 - - 可扩展性: - 可返回列表以支持批量确认；可按工具类型定制 meta 结构。 - """ - if not messages: - return None - last_msg = messages[-1] - if not hasattr(last_msg, "tool_calls") or not last_msg.tool_calls: - return None - # 当前 agent 名称（主 Agent 显示为 "主Agent"，子 Agent 为其名称） - try: - from agent.tools.todo_list import get_current_agent_name - _ag = get_current_agent_name() - agent_name = "主Agent" if _ag in ("", "__main__") else _ag - except Exception: - agent_name = "主Agent" - for raw_tool_call in last_msg.tool_calls: - tool_call = normalize_tool_call(raw_tool_call) - if tool_call.get("name") == "human_interaction": - return build_human_interaction_meta(session_id, tool_call, messages, agent_name=agent_name) - if classify_tool_execution(tool_call.get("name"), tool_call.get("args") or {}) == "confirm": - return build_approval_meta(session_id, tool_call, messages, agent_name=agent_name) - # 权限审查模式：越界操作注册审批（复用 tool_call 审批卡片），并携带越界原因 - if _policy_decision(tool_call.get("name"), tool_call.get("args") or {}) == "approve": - meta = build_approval_meta(session_id, tool_call, messages, agent_name=agent_name) - approval_id = (meta.get("pending_action") or {}).get("id") - if approval_id and approval_id in PENDING_TOOL_APPROVALS: - reason = (_policy_check_violation(tool_call.get("name"), tool_call.get("args") or {}) - or "检测到超出工作空间范围的操作") - PENDING_TOOL_APPROVALS[approval_id]["policy_note"] = reason - pa = meta["pending_action"] - pa["title"] = "待审批：检测到超出工作空间范围的操作" - pa["policy_note"] = reason - return meta - return None diff --git a/src/agent/core/runtime.py b/src/agent/core/runtime.py index e409aae..6d38f2c 100644 --- a/src/agent/core/runtime.py +++ b/src/agent/core/runtime.py @@ -1,37 +1,33 @@ -"""Agent 运行时：对话执行、历史持久化与人工确认续跑。 +"""Agent 运行时：对话执行与历史持久化。 系统定位: Web 层（即 ``web.ui_session``）与 LangGraph 之间的编排层。 ``agents/*`` 组装 ``AgentRuntime`` 实例，本模块提供统一执行 API。 主要流程: - execute_agent: 用户消息 -> summarizer 压缩 -> graph.invoke -> 响应/待确认 - continue_after_human_action: 用户确认/拒绝 -> 工具执行或图续跑 + execute_agent: 用户消息 -> summarizer 压缩 -> graph.invoke -> 响应 + + 人机交互为阻塞式普通工具（见 ``agent.core.human_request``）：图内工具调用 + 在线程中阻塞等待前端浮窗应答，同意/拒绝/补充信息都只是普通工具返回值， + 无需图暂停/续跑机制（不再有 continue_after_human_action）。 可扩展性： - ``AgentRuntime`` 可增加 checkpointer、tracing、多模态配置。 """ from __future__ import annotations -import json import threading from dataclasses import dataclass from typing import Any, Literal -from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage +from langchain_core.messages import AIMessage, HumanMessage, SystemMessage from langgraph.errors import GraphRecursionError -from langchain_core.messages import messages_from_dict -from agent.core.approvals import PENDING_TOOL_APPROVALS, remove_pending_ui_message, human_interaction_result from agent.history.tool_call_recorder import save_turn, start_live_turn, end_live_turn, save_aborted_turn, set_current_turn, set_current_session from agent.memory.system_prompt import get_plan_system_prompt, get_cron_system_prompt -from agent.core.loop_detector import reset_session as reset_loop_session, should_force_end as loop_should_force_end, build_force_end_message, clear_session as clear_loop_session -from agent.core.routing import first_pending_tool_meta -from agent.core.tool_policy import classify_tool_execution -from agent.tools.agent_call import SubAgentPendingError +from agent.core.loop_detector import should_force_end as loop_should_force_end, build_force_end_message from agent.utils.agent_utils import assistant_text, get_history -from agent.utils.tool_call_utils import normalize_tool_call, invoke_tool_and_build_message -from web.ui_session import rewrite_user_message_files_to_session_dir, session_dir, sync_turn_from_chat_history +from web.ui_session import rewrite_user_message_files_to_session_dir, sync_turn_from_chat_history AgentMode = Literal["agent", "plan", "cron"] @@ -104,28 +100,6 @@ def _inject_tool_history_context(session_id: str, messages: list) -> None: pass -def _rollback_chat_history_to_last_user(chat_history: list[dict[str, Any]], session_id: str) -> list[dict[str, Any]]: - """将 chat_history 回滚到上一条用户消息之前，并同步 turn 文件。""" - rolled_back = list(chat_history) - while rolled_back and rolled_back[-1].get("role") != "user": - rolled_back.pop() - if rolled_back: - rolled_back.pop() - if session_id: - sync_turn_from_chat_history(session_id, rolled_back, "") - return rolled_back - - -def _ensure_todo_cleared(session_id: str) -> None: - """安全清理指定会话的 todo_list 缓存。""" - if not session_id: - return - try: - from agent.tools.todo_list import clear_todo_store_session - clear_todo_store_session(session_id) - except Exception: - pass - # ---------- Abort 机制 ---------- _abort_flags: dict[str, threading.Event] = {} _abort_lock = threading.Lock() @@ -174,146 +148,6 @@ def _invoke_with_recursion_limit( except GraphRecursionError: print(f"[OOM] Agent '{runtime.name}' 达到最大迭代次数 {runtime.max_iterations}，输出 OOM。)") return {"messages": [AIMessage(content=_OOM_MESSAGE)], "_oom": True} - except SubAgentPendingError as e: - # 子 Agent 触发 HITL：冒泡到主图，返回主图消息快照 + sub_pending 元数据 - return { - "messages": e.main_messages or [], - "_sub_pending": e.pending_meta, - "_main_tool_call_id": e.main_tool_call_id, - } - - -def save_messages_to_history(session_id: str, user_msg: HumanMessage, assistant_msg: AIMessage) -> None: - """将本轮用户与助手消息写入 JSON 聊天历史（turn 文件）。 - - 输入: - session_id: 会话 ID；空则跳过。 - user_msg: 用户 HumanMessage；含 synthetic 标记时不写入。 - assistant_msg: 助手 AIMessage。 - - 输出: - 无。 - - 系统定位: - execute_agent 正常回复结束后持久化对话。 - - 可扩展性： - 可改为异步写入、双写向量库、按 agent_name 分表。 - """ - if not session_id: - return - history = get_history(session_id) - if not user_msg.additional_kwargs.get("synthetic"): - history.add_message(user_msg) - history.add_message(assistant_msg) - - -def _message_content_equals(left: Any, right: Any) -> bool: - """宽松比较消息 content，兼容纯文本与多模态列表。""" - if left == right: - return True - return assistant_text(left) == assistant_text(right) - - -def _pending_action_id(meta: dict[str, Any] | None) -> str | None: - """从 assistant meta 中提取 pending_action.id。""" - pending_action = (meta or {}).get("pending_action") or {} - return pending_action.get("id") - - -def _remember_pending_user_content(pending_meta: dict[str, Any], user_content: Any) -> None: - """将本轮用户输入缓存到 PENDING_TOOL_APPROVALS，供确认续跑时对齐历史。""" - approval_id = _pending_action_id(pending_meta) - if approval_id and approval_id in PENDING_TOOL_APPROVALS: - PENDING_TOOL_APPROVALS[approval_id]["user_content"] = user_content - - -def _persist_messages_from_pending_turn( - session_id: str, - messages: list[Any], - pending: dict[str, Any], -) -> None: - if not session_id or not messages: - return - # 找到本轮用户消息 - user_content = pending.get("user_content") - start_index = 0 - if user_content is not None: - for idx in range(len(messages) - 1, -1, -1): - msg = messages[idx] - if isinstance(msg, HumanMessage) and not msg.additional_kwargs.get("synthetic"): - if _message_content_equals(msg.content, user_content): - start_index = idx - break - # 只提取： - # - 用户消息（跳过 synthetic。） - # - 最终一条不包含 tool_calls 的助手消息 - user_msg = None - final_assistant = None - - for msg in messages[start_index:]: - if isinstance(msg, HumanMessage) and not msg.additional_kwargs.get("synthetic"): - user_msg = msg - if isinstance(msg, AIMessage) and not msg.tool_calls: - final_assistant = msg - - history = get_history(session_id) - if user_msg: - history.add_message(user_msg) - if final_assistant: - history.add_message(final_assistant) - - -def _invoke_after_manual_tool_result( - runtime: AgentRuntime, - messages: list[Any], - session_id: str, - *, - max_steps: int = 8, - agent_mode: str = "agent", -) -> tuple[list[Any], dict[str, Any] | None]: - """人工确认后继续推进图，并在遇到新的待确认项时暂停。""" - current_messages = messages - for _ in range(max_steps): - result = _invoke_with_recursion_limit(runtime, runtime.graph, {"messages": current_messages, "agent_mode": agent_mode}, {"configurable": {"session_id": session_id}}) - all_messages = result["messages"] - - # 循环检测 Level 2：检查是否因死循环被强制终止 - force_end, loop_tool, loop_args = loop_should_force_end(all_messages, session_id) - if force_end: - # 返回特殊标记：pending_meta 中携带 _loop_force_end 信息 - return all_messages, {"_loop_force_end": True, "tool_name": loop_tool, "args_summary": loop_args or ""} - - pending_meta = first_pending_tool_meta(all_messages, session_id) - if pending_meta: - return all_messages, pending_meta - - last_msg = all_messages[-1] if all_messages else None - if not isinstance(last_msg, AIMessage) or not last_msg.tool_calls: - return all_messages, None - - next_messages = [*all_messages] - progressed = False - for raw_tool_call in last_msg.tool_calls: - tool_call = normalize_tool_call(raw_tool_call) - tool_name = tool_call.get("name") - tool_args = tool_call.get("args") or {} - tool_call_id = tool_call.get("id") or f"manual_{session_id}_{len(next_messages)}" - if classify_tool_execution(tool_name, tool_args) != "direct": - continue - tool = runtime.tools_by_name.get(tool_name or "") - if not tool: - continue - tool_content, tool_artifact = invoke_tool_and_build_message(tool, tool_name, tool_args, tool_call_id) - msg_kwargs = {"content": tool_content, "tool_call_id": tool_call_id, "name": tool_name or "unknown_tool"} - if tool_artifact: - msg_kwargs["artifact"] = tool_artifact - next_messages.append(ToolMessage(**msg_kwargs)) - progressed = True - if not progressed: - return all_messages, None - current_messages = next_messages - return current_messages, None def _finalize_turn( @@ -325,7 +159,7 @@ def _finalize_turn( turn_id: str, agent_mode: str = "agent", ) -> list[dict[str, Any]]: - """处理图执行完成后的公共逻辑：检查 pending、保存 turn、构建 assistant 消息。""" + """处理图执行完成后的公共逻辑：检查循环强制终止、保存 turn、构建 assistant 消息。""" # 循环检测 Level 2 loop_force_end, loop_tool_name, loop_args_summary = loop_should_force_end(all_messages, session_id) if loop_force_end: @@ -335,26 +169,10 @@ def _finalize_turn( loop_msg = build_force_end_message(loop_tool_name, actual_count, loop_args_summary) chat_history.append({"role": "assistant", "content": loop_msg, "meta": {"turn_id": turn_id}}) if session_id: - clear_loop_session(session_id) - _ensure_todo_cleared(session_id) sync_turn_from_chat_history(session_id, chat_history, "") return chat_history - # 检查待确认项 - pending_meta = first_pending_tool_meta(all_messages, session_id) - if pending_meta: - # 不调用 end_live_turn：human_interaction 和其他工具一样走 live 记录通道， - # 前端持续轮询展示 running 状态，浮窗只是它的"前端 UI" - _remember_pending_user_content(pending_meta, user_content) - approval_id = _pending_action_id(pending_meta) - if approval_id and approval_id in PENDING_TOOL_APPROVALS: - PENDING_TOOL_APPROVALS[approval_id]["agent_mode"] = agent_mode - PENDING_TOOL_APPROVALS[approval_id]["turn_id"] = turn_id - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - - # 无待确认项：正常结束 + # 正常结束 end_live_turn(session_id) assistant_msg = all_messages[-1] if all_messages else AIMessage(content="") content = assistant_text(assistant_msg.content) or "（空回复）" @@ -367,9 +185,6 @@ def _finalize_turn( # 持久化 if session_id and turn_id: save_turn(session_id, turn_id, {"role": "user", "content": user_content}, all_messages) - _persist_messages_from_pending_turn(session_id, all_messages, {"user_content": user_content}) - _ensure_todo_cleared(session_id) - if session_id: sync_turn_from_chat_history(session_id, chat_history, "") return chat_history @@ -402,6 +217,7 @@ def _classify_error(e: Exception) -> str: # 通用兜底 return f"执行过程中出现错误，请稍后重试。如果问题持续，请联系开发者。\n错误详情: {msg[:200]}" + def execute_agent( runtime: AgentRuntime, chat_history: list[dict[str, Any]], @@ -417,10 +233,11 @@ def execute_agent( agent_mode: agent（默认）| plan（先规划再执行）。 输出: - 更新后的 chat_history；可能追加 assistant 消息、pending_action。 + 更新后的 chat_history；可能追加 assistant 消息。 系统定位: - 对外主入口，由 Web/API 层在每轮用户发送后调用。 + 对外主入口，由 Web/API 层（turn_runner）在每轮用户发送后调用。 + 人机交互在图上线程内阻塞等待，无待确认返回路径。 可扩展性: 可返回 generator 支持流式；可注入 middleware 做鉴权、限流。 @@ -473,10 +290,8 @@ def execute_agent( turn_id = start_live_turn(session_id) _set_cur_sess(session_id) clear_abort_flag(session_id) - # 重置循环检测状态（每轮用户输入后重新开始计数） - reset_loop_session(session_id) - # 主 Agent 上下文：TodoList 按 agent 隔离时，主 Agent 用保留键 "__main__" - from agent.tools.todo_list import set_current_agent_name + # 主 Agent 上下文 + from agent.core.human_request import set_current_agent_name set_current_agent_name("__main__") result = _invoke_with_recursion_limit(runtime, runtime.graph, {"messages": messages, "agent_mode": agent_mode}, graph_config) set_current_turn("") @@ -487,28 +302,11 @@ def execute_agent( clear_abort_flag(session_id) save_aborted_turn(session_id, turn_id) end_live_turn(session_id) - _ensure_todo_cleared(session_id) chat_history.append({"role": "assistant", "content": "已被用户手动暂停", "meta": {"paused": True}}) if session_id: sync_turn_from_chat_history(session_id, chat_history, "") return chat_history - # ===== 子 Agent HITL 冒泡：主图在 call_subagent 处暂停 ===== - sub_pending_meta = result.get("_sub_pending") - if sub_pending_meta: - sub_approval_id = _pending_action_id(sub_pending_meta) - if sub_approval_id and sub_approval_id in PENDING_TOOL_APPROVALS: - entry = PENDING_TOOL_APPROVALS[sub_approval_id] - entry["main_messages"] = list(result.get("messages") or []) - entry["main_tool_call_id"] = result.get("_main_tool_call_id") or f"manual_{sub_approval_id}" - entry["agent_mode"] = agent_mode - entry["turn_id"] = turn_id - entry["user_content"] = user_content - # 不调用 end_live_turn：turn 仍 live，前端轮询展示 running + 浮窗 - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - all_messages = result["messages"] chat_history = _finalize_turn(runtime, all_messages, chat_history, session_id, user_content, turn_id, agent_mode) return chat_history @@ -520,7 +318,6 @@ def execute_agent( clear_abort_flag(session_id) save_aborted_turn(session_id, turn_id) end_live_turn(session_id) - _ensure_todo_cleared(session_id) chat_history.append({"role": "assistant", "content": "已被用户手动暂停", "meta": {"paused": True}}) if session_id: sync_turn_from_chat_history(session_id, chat_history, "") @@ -546,408 +343,3 @@ def execute_agent( if session_id: sync_turn_from_chat_history(session_id, chat_history, "") return chat_history - - -def _handle_loop_force_end_in_pending(pending_meta: dict[str, Any], chat_history: list[dict[str, Any]], session_id: str) -> list[dict[str, Any]] | None: - """若 pending_meta 包含 _loop_force_end，写入终止消息并返回 chat_history；否则返回 None。""" - if not pending_meta.get("_loop_force_end"): - return None - loop_msg = build_force_end_message( - pending_meta.get("tool_name", ""), 0, - pending_meta.get("args_summary", ""), - ) - chat_history.append({"role": "assistant", "content": loop_msg, "meta": {}}) - if session_id: - clear_loop_session(session_id) - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - - -def _finalize_new_pending_after_human_action( - pending_meta: dict[str, Any], - pending: dict[str, Any], - chat_history: list[dict[str, Any]], - session_id: str, - agent_mode: str, -) -> list[dict[str, Any]]: - """记录新的待确认项并返回 chat_history。""" - _remember_pending_user_content(pending_meta, pending.get("user_content")) - approval_id2 = _pending_action_id(pending_meta) - if approval_id2 and approval_id2 in PENDING_TOOL_APPROVALS: - PENDING_TOOL_APPROVALS[approval_id2]["agent_mode"] = agent_mode - # 传递原始 turn_id 给嵌套的人机交互，确保所有记录写入同一个文件 - original_turn_id = pending.get("turn_id", "") - if original_turn_id: - PENDING_TOOL_APPROVALS[approval_id2]["turn_id"] = original_turn_id - # 不追加空消息 — pending_action 由 API 层单独返回 - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - - -def _resume_graph_after_tool_message( - runtime: AgentRuntime, - prior_messages: list[Any], - session_id: str, - agent_mode: str, - pending: dict[str, Any], - chat_history: list[dict[str, Any]], - turn_id: str, -) -> list[dict[str, Any]]: - """用已追加好 ToolMessage 的 prior_messages 续跑图，统一处理 pending/结束/持久化。 - - 用于主 Agent skip 续跑、子 Agent 完成后恢复主图。pending 提供 user_content/turn_id。 - """ - try: - all_messages, pending_meta = _invoke_after_manual_tool_result(runtime, prior_messages, session_id, agent_mode=agent_mode) - if pending_meta: - end_live_turn(session_id) - result = _handle_loop_force_end_in_pending(pending_meta, chat_history, session_id) - if result is not None: - return result - return _finalize_new_pending_after_human_action(pending_meta, pending, chat_history, session_id, agent_mode) - end_live_turn(session_id) - assistant_msg = all_messages[-1] if all_messages else AIMessage(content="") - content = assistant_text(assistant_msg.content) or "（空回复）" - chat_history.append({"role": "assistant", "content": content, "meta": {"turn_id": turn_id}}) - _persist_messages_from_pending_turn(session_id, all_messages, pending) - if session_id and turn_id: - save_turn(session_id, turn_id, {"role": "user", "content": pending.get("user_content") or ""}, all_messages) - _ensure_todo_cleared(session_id) - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - except Exception: - import traceback - traceback.print_exc() - end_live_turn(session_id) - content = "续跑图执行时遇到问题。" - chat_history.append({"role": "assistant", "content": content, "meta": {}}) - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - - -def _extract_sub_final_text(all_messages: list[Any]) -> str: - """从子 Agent 图消息中提取最后一条非工具调用的助手消息文本。""" - for msg in reversed(all_messages or []): - if isinstance(msg, AIMessage) and not getattr(msg, "tool_calls", None): - content = msg.content - if isinstance(content, str): - return content - if isinstance(content, list): - return "\n".join( - p.get("text", "") for p in content - if isinstance(p, dict) and p.get("type") == "text" - ) - return "子 Agent 已完成任务但未返回文字结果。" - - -def _invoke_sub_graph( - sub_runtime: AgentRuntime, - sub_messages: list[Any], - session_id: str, - agent_name: str, - main_messages: list[Any], - main_tool_call_id: str, - agent_mode: str, - user_content: Any, - turn_id: str, -) -> tuple[list[Any], dict[str, Any] | None]: - """恢复子 Agent 图执行（复用 _invoke_after_manual_tool_result）。 - - 若子图又产生 pending，已在 PENDING_TOOL_APPROVALS 中标记 is_sub_agent 并附加 - sub_runtime/sub_messages/main_messages/main_tool_call_id，供下一轮确认恢复。 - """ - from agent.tools.todo_list import set_current_agent_name - set_current_agent_name(agent_name) - # 子 Agent 续跑同样需抑制 record_*_live，避免其思考/工具调用 - # 混入主 Agent 的实时列表（与 agent_call.CallSubAgentTool._run 保持一致）。 - from agent.history.tool_call_recorder import sub_agent_context - try: - with sub_agent_context(): - sub_all_messages, sub_pending_meta = _invoke_after_manual_tool_result( - sub_runtime, sub_messages, session_id, agent_mode=agent_mode - ) - finally: - set_current_agent_name("__main__") - - if sub_pending_meta and not sub_pending_meta.get("_loop_force_end"): - approval_id = _pending_action_id(sub_pending_meta) - if approval_id and approval_id in PENDING_TOOL_APPROVALS: - entry = PENDING_TOOL_APPROVALS[approval_id] - entry["is_sub_agent"] = True - entry["agent_name"] = agent_name - entry["sub_runtime"] = sub_runtime - entry["sub_messages"] = list(sub_all_messages) # 更新为最新子图消息 - entry["main_messages"] = main_messages - entry["main_tool_call_id"] = main_tool_call_id - entry["agent_mode"] = agent_mode - entry["turn_id"] = turn_id - entry["user_content"] = user_content - return sub_all_messages, sub_pending_meta - - -def _resume_sub_agent( - runtime: AgentRuntime, - pending: dict[str, Any], - chat_history: list[dict[str, Any]], - session_id: str, - approval_id: str, - decision: str, - instruction: str | None, - agent_mode: str, -) -> list[dict[str, Any]]: - """子 Agent HITL 确认/跳过后恢复：先续子图，子图完成后再恢复主图。""" - sub_runtime = pending.get("sub_runtime") - sub_messages = list(pending.get("sub_messages") or []) - main_messages = list(pending.get("main_messages") or []) - main_tool_call_id = pending.get("main_tool_call_id") or f"manual_{approval_id}" - agent_name = pending.get("agent_name") or "子Agent" - original_turn_id = pending.get("turn_id", "") - sub_type = pending.get("type") - sub_tool_call = dict(pending.get("tool_call") or {}) - sub_tool_name = sub_tool_call.get("name") or "unknown_tool" - sub_call_id = sub_tool_call.get("id") or f"sub_{approval_id}" - - # 1) 构造子图续跑的 ToolMessage - if decision == "skip" and sub_type != "human_interaction": - sub_messages.append(ToolMessage( - content=f"用户已跳过此工具调用（{sub_tool_name}），未执行。", - tool_call_id=sub_call_id, name=sub_tool_name, - )) - elif sub_type == "human_interaction": - sub_messages.append(ToolMessage( - content=human_interaction_result(pending, "approve", instruction), - tool_call_id=sub_call_id, name="human_interaction", - )) - else: - sub_tool = sub_runtime.tools_by_name.get(sub_tool_name) if sub_runtime else None - if sub_tool: - tc, ta = invoke_tool_and_build_message(sub_tool, sub_tool_name, sub_tool_call.get("args") or {}, sub_call_id) - msg_kwargs = {"content": tc, "tool_call_id": sub_call_id, "name": sub_tool_name} - if ta: - msg_kwargs["artifact"] = ta - sub_messages.append(ToolMessage(**msg_kwargs)) - else: - sub_messages.append(ToolMessage( - content=f"工具 {sub_tool_name} 未找到", tool_call_id=sub_call_id, name=sub_tool_name, - )) - - # 2) 恢复子图 - if session_id: - set_current_session(session_id) - if original_turn_id: - from agent.history.tool_call_recorder import set_current_turn as _set_tc_turn - _set_tc_turn(original_turn_id) - sub_all_messages, sub_pending_meta = _invoke_sub_graph( - sub_runtime, sub_messages, session_id, agent_name, - main_messages, main_tool_call_id, agent_mode, - pending.get("user_content"), original_turn_id, - ) - - if sub_pending_meta: - # 子图又产生 pending（嵌套/连续 HITL）— _invoke_sub_graph 已注册新 pending（含 main 上下文） - result = _handle_loop_force_end_in_pending(sub_pending_meta, chat_history, session_id) - if result is not None: - return result - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - - # 3) 子图完成：提取最终文本，恢复主图 - sub_final_text = _extract_sub_final_text(sub_all_messages) - main_messages.append(ToolMessage(content=sub_final_text, tool_call_id=main_tool_call_id, name="call_subagent")) - from agent.tools.todo_list import set_current_agent_name - set_current_agent_name("__main__") - if session_id: - turn_id = start_live_turn(session_id) - else: - turn_id = original_turn_id or "" - main_pending = {"user_content": pending.get("user_content"), "turn_id": original_turn_id} - return _resume_graph_after_tool_message(runtime, main_messages, session_id, agent_mode, main_pending, chat_history, original_turn_id or turn_id) - - -def continue_after_human_action( - runtime: AgentRuntime, - chat_history: list[dict[str, Any]], - session_id: str, - approval_id: str, - decision: Literal["approve", "reject", "edit", "skip"], - instruction: str | None = None, -) -> list[dict[str, Any]]: - """用户对待确认项做出决策后，继续执行或结束该轮交互。 - - 输入: - runtime: Agent 运行时。 - chat_history: 当前聊天历史。 - session_id: 会话 ID。 - approval_id: pending_action.id。 - decision: approve | reject | edit | skip。 - - approve: 执行工具/确认续跑；skip: 不执行工具但继续图（ToolMessage 标记跳过）。 - - reject: 与手动暂停一致——暂停当前 Agent 运行、保留之前的运行记录、 - 前端追加"已被用户手动暂停"消息（对三种审批统一生效：权限不足 / - workspace 越界 / 人机交互）。 - - edit: human_interaction selection 提交文本。 - instruction: selection 时为用户输入文本。 - - 输出: - 更新后的 chat_history。 - - 系统定位: - UI 确认框回调入口；按 pending.type 与 is_sub_agent 分支： - human_interaction / tool_call / sub_agent。 - - 可扩展性： - 可增加 timeout 自动 reject；支持批量 approve 多个 tool_call。 - """ - chat_history = chat_history or [] - pending = PENDING_TOOL_APPROVALS.pop(approval_id, None) - if not pending: - chat_history.append({"role": "assistant", "content": "该待确认项已处理或已过期。", "meta": {}}) - return chat_history - - remove_pending_ui_message(chat_history, approval_id) - - agent_mode = pending.get("agent_mode", "agent") - - # ===== 拒绝：统一三种审批（权限不足 / workspace 越界 / 人机交互）===== - # 与手动暂停（/api/chat/abort）一致：暂停当前 Agent 运行、保留之前的运行记录， - # 前端收到"已被用户手动暂停"消息。不恢复图执行。 - if decision == "reject": - last_msg = chat_history[-1] if chat_history else None - if not (last_msg and last_msg.get("role") == "assistant" and last_msg.get("content") == "已被用户手动暂停"): - chat_history.append({"role": "assistant", "content": "已被用户手动暂停", "meta": {"paused": True}}) - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - - # ===== 子 Agent HITL 恢复（approve / skip，无 reject）===== - if pending.get("is_sub_agent"): - return _resume_sub_agent(runtime, pending, chat_history, session_id, approval_id, decision, instruction, agent_mode) - - # ===== 跳过：仅 tool_call 类型支持，human_interaction 不支持跳过 ===== - if decision == "skip" and pending.get("type") != "human_interaction": - tool_call = dict(pending.get("tool_call") or {}) - tool_name = tool_call.get("name") or "unknown_tool" - call_id = tool_call.get("id") or f"manual_{approval_id}" - skip_content = f"用户已跳过此工具调用（{tool_name}），未执行。" - prior_messages = messages_from_dict(pending.get("messages") or []) - prior_messages.append(ToolMessage(content=skip_content, tool_call_id=call_id, name=tool_name)) - original_turn_id = pending.get("turn_id", "") - if session_id: - set_current_session(session_id) - if original_turn_id: - from agent.history.tool_call_recorder import set_current_turn as _set_tc_turn - _set_tc_turn(original_turn_id) - turn_id = start_live_turn(session_id) - else: - turn_id = original_turn_id or "" - from agent.tools.todo_list import set_current_agent_name - set_current_agent_name("__main__") - return _resume_graph_after_tool_message(runtime, prior_messages, session_id, agent_mode, pending, chat_history, turn_id) - - if pending.get("type") == "human_interaction": - original_turn_id = pending.get("turn_id", "") - if session_id and original_turn_id: - set_current_session(session_id) - # 恢复原始 turn_id 到上下文，确保后续 live 记录写入正确的文件 - from agent.history.tool_call_recorder import record_tool_result_live, set_current_turn as _set_tc_turn - _set_tc_turn(original_turn_id) - tool_content = human_interaction_result(pending, decision, instruction) - record_tool_result_live(session_id, "human_interaction", tool_content) - else: - tool_content = human_interaction_result(pending, decision, instruction) if session_id else "" - try: - tool_call = dict(pending.get("tool_call") or {}) - prior_messages = messages_from_dict(pending.get("messages") or []) - call_id = tool_call.get("id") or f"human_{approval_id}" - prior_messages.append(ToolMessage(content=tool_content, tool_call_id=call_id, name="human_interaction")) - all_messages, pending_meta = _invoke_after_manual_tool_result(runtime, prior_messages, session_id, agent_mode=agent_mode) - - if pending_meta: - # 新一轮 human_interaction：不清理 live 记录，前端继续轮询展示 - result = _handle_loop_force_end_in_pending(pending_meta, chat_history, session_id) - if result is not None: - return result - return _finalize_new_pending_after_human_action(pending_meta, pending, chat_history, session_id, agent_mode) - - # 正常结束 - end_live_turn(session_id) - assistant_msg = all_messages[-1] if all_messages else AIMessage(content="") - content = assistant_text(assistant_msg.content) or "（空回复）" - chat_history.append({"role": "assistant", "content": content, "meta": {"turn_id": original_turn_id or f"human_{approval_id}"}}) - _persist_messages_from_pending_turn(session_id, all_messages, pending) - if session_id and original_turn_id: - save_turn(session_id, original_turn_id, {"role": "user", "content": pending.get("user_content") or ""}, all_messages) - _ensure_todo_cleared(session_id) - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - except Exception: - import traceback - traceback.print_exc() - end_live_turn(session_id) - content = "处理人工交互结果时遇到问题。" - chat_history.append({"role": "assistant", "content": content, "meta": {}}) - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - - tool_call = dict(pending.get("tool_call") or {}) - tool_name = tool_call.get("name") - tool_args = tool_call.get("args") or {} - tool = runtime.tools_by_name.get(tool_name) - if not tool: - chat_history.append({"role": "assistant", "content": f"找不到工具：{tool_name}", "meta": {}}) - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - - if session_id: - set_current_session(session_id) - turn_id = start_live_turn(session_id) - else: - turn_id = "" - try: - tool_content, tool_artifact = invoke_tool_and_build_message(tool, tool_name, tool_args, tool_call.get("id", "")) - prior_messages = messages_from_dict(pending.get("messages") or []) - call_id = tool_call.get("id") or f"manual_{approval_id}" - msg_kwargs = {"content": tool_content, "tool_call_id": call_id, "name": tool_name} - if tool_artifact: - msg_kwargs["artifact"] = tool_artifact - prior_messages.append(ToolMessage(**msg_kwargs)) - all_messages, pending_meta = _invoke_after_manual_tool_result(runtime, prior_messages, session_id, agent_mode=agent_mode) - - if pending_meta: - end_live_turn(session_id) - result = _handle_loop_force_end_in_pending(pending_meta, chat_history, session_id) - if result is not None: - return result - return _finalize_new_pending_after_human_action(pending_meta, pending, chat_history, session_id, agent_mode) - - # 正常结束 - end_live_turn(session_id) - assistant_msg = all_messages[-1] if all_messages else AIMessage(content="") - content = assistant_text(assistant_msg.content) or "（空回复）" - chat_history.append({"role": "assistant", "content": content, "meta": {"turn_id": turn_id}}) - _persist_messages_from_pending_turn(session_id, all_messages, pending) - if session_id and turn_id: - save_turn(session_id, turn_id, {"role": "user", "content": pending.get("user_content") or ""}, all_messages) - _ensure_todo_cleared(session_id) - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history - except Exception: - import traceback - traceback.print_exc() - end_live_turn(session_id) - content = "执行确认后的工具调用时遇到问题。" - prior_messages = messages_from_dict(pending.get("messages") or []) - call_id = tool_call.get("id") or f"manual_{approval_id}" - prior_messages.extend([ - ToolMessage(content=content, tool_call_id=call_id, name=tool_name or "unknown_tool"), - AIMessage(content=content), - ]) - chat_history.append({"role": "assistant", "content": content, "meta": {}}) - _persist_messages_from_pending_turn(session_id, prior_messages, pending) - if session_id: - sync_turn_from_chat_history(session_id, chat_history, "") - return chat_history diff --git a/src/agent/core/storage.py b/src/agent/core/storage.py new file mode 100644 index 0000000..f90b176 --- /dev/null +++ b/src/agent/core/storage.py @@ -0,0 +1,1508 @@ +"""统一存储抽象层：会话与定时任务的 JSON / MySQL 双后端存储。 + +系统定位: + 会话历史（对话消息、工具调用记录、token/压缩元数据）与定时任务（任务配置、 + 执行记录）共用的存储接口与后端实现。消费方（agent_utils / tool_call_recorder / + ui_session / server / cron 等）只依赖本模块暴露的接口与便捷函数， + **不感知后端选择**（不判断 STORAGE_BACKEND / is_mysql）。 + +接口隔离: + - ``ConversationRecordRepository``：对话记录读写接口（16 方法）。 + - ``TaskConfigRepository``：cron 任务配置读写接口（6 方法）。 + - 新增存储后端 = 实现上述 Protocol + 在 ``get_record_repository`` / + ``get_task_repository`` 工厂中注册一行。 + +作用域（scope）: + - ``session``（默认）：主进程普通会话，JSON 落盘于 ``SESSIONS_ROOT``， + MySQL 使用 ``sessions / session_messages / session_tool_calls`` 表。 + - ``cron``：定时任务，JSON 落盘于 ``CRON_ROOT``（history/cron）， + MySQL 使用 ``cron_messages / cron_tool_calls`` 表。 + 主进程读 cron 记录用 ``get_record_repository(scope="cron")``； + cron 子进程在 ``runner._redirect_storage_roots`` 中调用 + ``set_storage_scope("cron")``，使模块级便捷函数直接路由到 cron 作用域。 + +不双写: + 选哪种后端，读写就只走该后端。二进制产物文件（图片/附件等）始终落盘， + 与后端无关（由消费方负责，见 tool_call_recorder.save_turn）。 + +可扩展性: + - 消费方新增字段/功能无需改动本模块：工具调用记录的 meta 全量透传， + 消息与记录原样 JSON 入库/落盘。 + - 新增存储方式（如 postgres）只需实现两个 Protocol 并注册工厂。 +""" +from __future__ import annotations + +import json +import os +import shutil +import sys +import threading +from typing import Any, Protocol + +from agent.cron.models import CronTask, ExecutionLog +from agent.utils import agent_utils as _au +from agent.utils.agent_utils import HISTORY_ROOT, TURN_JSON_PREFIX +from agent.core.db import ( + mysql_conn, + ensure_tables, + get_storage_backend, +) + +# 定时任务根目录：history/cron +CRON_ROOT = os.path.join(HISTORY_ROOT, "cron") +os.makedirs(CRON_ROOT, exist_ok=True) + +# ---------- 作用域（cron 子进程切换为 "cron"） ---------- +_STORAGE_SCOPE = "session" + + +def set_storage_scope(scope: str) -> None: + """切换模块级便捷函数的存储作用域（"session" / "cron"）。 + + 供 cron 子进程（runner._redirect_storage_roots）调用，使后续所有 + 记录读写路由到 cron 作用域（CRON_ROOT 文件 / cron_* 表）。 + """ + global _STORAGE_SCOPE + _STORAGE_SCOPE = scope if scope == "cron" else "session" + + +def get_storage_scope() -> str: + """返回当前存储作用域。""" + return _STORAGE_SCOPE + + +# ---------- Protocol 定义 ---------- +class ConversationRecordRepository(Protocol): + """对话消息与工具调用记录的读写接口（session / cron 双作用域通用）。""" + + def save_turn(self, session_id: str, turn_id: str, messages: list[dict]) -> None: ... + def load_messages(self, session_id: str) -> list[dict]: ... + def load_turn_messages(self, session_id: str, turn_id: str) -> list[dict] | None: ... + def list_turn_ids(self, session_id: str) -> list[str]: ... + def save_tool_calls( + self, session_id: str, turn_id: str, tool_calls: list[dict], meta: dict | None = None + ) -> None: ... + def get_turn_record(self, session_id: str, turn_id: str) -> dict[str, Any] | None: ... + def get_latest_turn_record(self, session_id: str) -> dict[str, Any] | None: ... + def list_turn_records(self, session_id: str, limit: int = 5) -> list[dict[str, Any]]: ... + def persist_live_records( + self, session_id: str, turn_id: str, records: list[dict], started_at: float | None = None + ) -> None: ... + def load_session_extra(self, session_id: str) -> dict[str, Any]: ... + def save_session_extra(self, session_id: str, extra: dict[str, Any]) -> None: ... + def list_session_ids(self) -> list[str]: ... + def get_last_turn_id(self, session_id: str) -> str: ... + def delete_turns_after(self, session_id: str, keep_until_turn_id: str) -> int: ... + def delete_tool_records(self, session_id: str, keep_turn_ids: set[str] | None = None) -> None: ... + def delete_session(self, session_id: str) -> None: ... + + +class TaskConfigRepository(Protocol): + """定时任务配置与状态的读写接口。""" + + def list_tasks(self) -> list[CronTask]: ... + def get_task(self, task_id: str) -> CronTask | None: ... + def save_task(self, task: CronTask) -> None: ... + def delete_task(self, task_id: str) -> None: ... + def update_status( + self, + task_id: str, + status: str, + *, + error: str = "", + last_run_at: str = "", + next_run_at: str = "", + ) -> None: ... + def append_execution(self, task_id: str, exec_log: dict[str, Any]) -> None: ... + + +# ---------- MySQL 表 DDL（幂等建表，session / cron 互不影响） ---------- +_SESSION_TABLE_DDLS = [ + """ + CREATE TABLE IF NOT EXISTS sessions ( + session_id VARCHAR(64) PRIMARY KEY, + created_at VARCHAR(32) NOT NULL, + updated_at VARCHAR(32) NOT NULL, + last_turn_id VARCHAR(64) DEFAULT '' + ) CHARACTER SET utf8mb4 + """, + """ + CREATE TABLE IF NOT EXISTS session_messages ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + session_id VARCHAR(64) NOT NULL, + turn_id VARCHAR(64) NOT NULL, + seq INT NOT NULL DEFAULT 0, + message JSON NOT NULL, + INDEX idx_sess_turn (session_id, turn_id) + ) CHARACTER SET utf8mb4 + """, + """ + CREATE TABLE IF NOT EXISTS session_tool_calls ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + session_id VARCHAR(64) NOT NULL, + turn_id VARCHAR(64) NOT NULL, + tool_calls JSON NOT NULL, + meta JSON DEFAULT NULL, + INDEX idx_sess_turn (session_id, turn_id) + ) CHARACTER SET utf8mb4 + """, +] + +_CRON_TABLE_DDLS = [ + """ + CREATE TABLE IF NOT EXISTS cron_tasks ( + task_id VARCHAR(64) PRIMARY KEY, + name VARCHAR(255) NOT NULL, + prompt TEXT NOT NULL, + trigger_type ENUM('cron','interval','once') NOT NULL, + cron_expr VARCHAR(128) DEFAULT '', + interval_seconds INT DEFAULT 0, + run_at VARCHAR(32) DEFAULT '', + enabled TINYINT(1) DEFAULT 1, + timeout_seconds INT DEFAULT 300, + created_at VARCHAR(32) NOT NULL, + next_run_at VARCHAR(32) DEFAULT '', + last_run_at VARCHAR(32) DEFAULT '', + last_status VARCHAR(16) DEFAULT 'pending', + last_error TEXT, + executions JSON DEFAULT NULL + ) CHARACTER SET utf8mb4 + """, + """ + CREATE TABLE IF NOT EXISTS cron_messages ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + task_id VARCHAR(64) NOT NULL, + turn_id VARCHAR(64) NOT NULL, + role VARCHAR(16) NOT NULL, + content JSON NOT NULL, + meta JSON DEFAULT NULL, + INDEX idx_task_turn (task_id, turn_id) + ) CHARACTER SET utf8mb4 + """, + """ + CREATE TABLE IF NOT EXISTS cron_tool_calls ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + task_id VARCHAR(64) NOT NULL, + turn_id VARCHAR(64) NOT NULL, + tool_calls JSON NOT NULL, + INDEX idx_task_turn (task_id, turn_id) + ) CHARACTER SET utf8mb4 + """, +] + + +def _ensure_session_tables() -> None: + """幂等建 session 三表（委托共享 ensure_tables）。""" + ensure_tables(_SESSION_TABLE_DDLS) + + +def _ensure_cron_tables() -> None: + """幂等建 cron 三表（委托共享 ensure_tables）。""" + ensure_tables(_CRON_TABLE_DDLS) + + +# ---------- 附件路径解析（cron 作用域共用：JSON / MySQL） ---------- +def _resolve_msg_paths(msg: dict, task_id: str) -> dict: + """将消息 content 中的相对附件路径解析为绝对地址（相对 CRON_ROOT）。 + + cron 子进程写入 / 数据库存储的附件路径均为相对 CRON_ROOT 的 relpath， + 主进程序列化前还原为绝对路径，供文件读取。 + """ + content = msg.get("content") + if not isinstance(content, list): + return msg + for piece in content: + if not isinstance(piece, dict): + continue + for key in ("path", "audio_path"): + p = piece.get(key) + if isinstance(p, str) and p and not os.path.isabs(p): + piece[key] = os.path.normpath(os.path.join(CRON_ROOT, p)) + iu = piece.get("image_url") + if isinstance(iu, dict): + u = iu.get("url") + if isinstance(u, str) and u and not u.startswith("data:") and not os.path.isabs(u): + iu["url"] = os.path.normpath(os.path.join(CRON_ROOT, u)) + return msg + + +def _to_rel_paths(msg: dict, task_id: str) -> dict: + """_resolve_msg_paths 的逆操作：绝对路径 → 相对 CRON_ROOT 的 relpath（入库前调用）。""" + import copy + msg = copy.deepcopy(msg) + content = msg.get("content") + if not isinstance(content, list): + return msg + cron_root_abs = os.path.abspath(CRON_ROOT) + for piece in content: + if not isinstance(piece, dict): + continue + for key in ("path", "audio_path"): + p = piece.get(key) + if isinstance(p, str) and p and os.path.isabs(p): + try: + rel = os.path.relpath(p, cron_root_abs) + piece[key] = rel.replace("\\", "/") + except ValueError: + pass + iu = piece.get("image_url") + if isinstance(iu, dict): + u = iu.get("url") + if isinstance(u, str) and u and not u.startswith("data:") and os.path.isabs(u): + try: + rel = os.path.relpath(u, cron_root_abs) + iu["url"] = rel.replace("\\", "/") + except ValueError: + pass + return msg + + +# ---------- JSON 实现 ---------- +class JsonRecordRepository: + """基于 JSON 文件（人类可读、可直观查看）的记录仓库。 + + session 作用域根目录为 SESSIONS_ROOT（动态引用 agent_utils.SESSIONS_ROOT， + 兼容 cron 子进程的重定向）；cron 作用域根目录为 CRON_ROOT。 + 文件格式保持不变：turn_{tid}.json / tool_{tid}.json / session_meta.json / + _session_tokens.json。 + """ + + def __init__(self, scope: str = "session") -> None: + self.scope = scope if scope == "cron" else "session" + + def _root(self) -> str: + if self.scope == "cron": + return CRON_ROOT + return _au.SESSIONS_ROOT + + def _session_dir(self, session_id: str) -> str: + return os.path.join(self._root(), session_id) + + def _turn_path(self, session_id: str, turn_id: str) -> str: + return os.path.join(self._session_dir(session_id), f"{TURN_JSON_PREFIX}{turn_id}.json") + + def _tool_path(self, session_id: str, turn_id: str) -> str: + return os.path.join(self._session_dir(session_id), f"tool_{turn_id}.json") + + def _meta_path(self, session_id: str) -> str: + return os.path.join(self._session_dir(session_id), "session_meta.json") + + def _token_path(self, session_id: str) -> str: + return os.path.join(self._session_dir(session_id), "_session_tokens.json") + + # ---- 消息 ---- + def save_turn(self, session_id: str, turn_id: str, messages: list[dict]) -> None: + if not session_id or not turn_id: + return + sess_dir = self._session_dir(session_id) + os.makedirs(sess_dir, exist_ok=True) + with open(self._turn_path(session_id, turn_id), "w", encoding="utf-8") as f: + json.dump({"turn_id": turn_id, "messages": messages}, f, ensure_ascii=False, indent=2) + + def load_messages(self, session_id: str) -> list[dict]: + sess_dir = self._session_dir(session_id) + if not os.path.isdir(sess_dir): + return [] + all_msgs: list[dict] = [] + for tf in sorted( + f for f in os.listdir(sess_dir) + if f.startswith(TURN_JSON_PREFIX) and f.endswith(".json") + ): + tp = os.path.join(sess_dir, tf) + if not os.path.isfile(tp): + continue + try: + with open(tp, "r", encoding="utf-8") as f: + data = json.load(f) + for msg in data.get("messages", []): + if self.scope == "cron": + msg = _resolve_msg_paths(msg, session_id) + all_msgs.append(msg) + except (json.JSONDecodeError, OSError): + pass + return all_msgs + + def load_turn_messages(self, session_id: str, turn_id: str) -> list[dict] | None: + tp = self._turn_path(session_id, turn_id) + if not os.path.isfile(tp): + return None + try: + with open(tp, "r", encoding="utf-8") as f: + data = json.load(f) + return data.get("messages", []) or [] + except (json.JSONDecodeError, OSError): + return None + + def list_turn_ids(self, session_id: str) -> list[str]: + sess_dir = self._session_dir(session_id) + if not os.path.isdir(sess_dir): + return [] + return sorted( + f[len(TURN_JSON_PREFIX):-len(".json")] + for f in os.listdir(sess_dir) + if f.startswith(TURN_JSON_PREFIX) and f.endswith(".json") + ) + + # ---- 工具调用记录 ---- + def save_tool_calls( + self, session_id: str, turn_id: str, tool_calls: list[dict], meta: dict | None = None + ) -> None: + if not session_id or not turn_id: + return + sess_dir = self._session_dir(session_id) + os.makedirs(sess_dir, exist_ok=True) + record: dict[str, Any] = { + "session_id": session_id, + "turn_id": turn_id, + "tool_calls": tool_calls, + "timestamp": turn_id, + } + if meta: + record.update(meta) # meta 全量透传，消费方新增字段无需改动存储层 + with open(self._tool_path(session_id, turn_id), "w", encoding="utf-8") as f: + json.dump(record, f, ensure_ascii=False, indent=2) + + def get_turn_record(self, session_id: str, turn_id: str) -> dict[str, Any] | None: + tp = self._tool_path(session_id, turn_id) + if not os.path.isfile(tp): + return None + try: + with open(tp, "r", encoding="utf-8") as f: + return json.load(f) + except (json.JSONDecodeError, OSError): + return None + + def get_latest_turn_record(self, session_id: str) -> dict[str, Any] | None: + records = self.list_turn_records(session_id, limit=1) + return records[0] if records else None + + def list_turn_records(self, session_id: str, limit: int = 5) -> list[dict[str, Any]]: + sess_dir = self._session_dir(session_id) + if not os.path.isdir(sess_dir): + return [] + files = sorted( + f for f in os.listdir(sess_dir) + if f.startswith("tool_") and f.endswith(".json") + ) + records: list[dict[str, Any]] = [] + for fname in reversed(files): + tp = os.path.join(sess_dir, fname) + try: + with open(tp, "r", encoding="utf-8") as f: + records.append(json.load(f)) + except (json.JSONDecodeError, OSError): + continue + if limit and len(records) >= limit: + break + return records + + def persist_live_records( + self, session_id: str, turn_id: str, records: list[dict], started_at: float | None = None + ) -> None: + """实时工具记录增量写盘（可选行为；mysql 后端不实现）。""" + if not session_id or not turn_id: + return + sess_dir = self._session_dir(session_id) + os.makedirs(sess_dir, exist_ok=True) + record: dict[str, Any] = { + "session_id": session_id, + "turn_id": turn_id, + "tool_calls": records, + "timestamp": turn_id, + } + if started_at: + record["started_at"] = started_at + with open(self._tool_path(session_id, turn_id), "w", encoding="utf-8") as f: + json.dump(record, f, ensure_ascii=False, indent=2) + + # ---- 会话元数据（extra = {tokens, token_breakdown, agent_meta}） ---- + def load_session_extra(self, session_id: str) -> dict[str, Any]: + extra: dict[str, Any] = {} + # agent_meta ↔ session_meta.json + mp = self._meta_path(session_id) + if os.path.isfile(mp): + try: + with open(mp, "r", encoding="utf-8") as f: + extra["agent_meta"] = json.load(f) + except (json.JSONDecodeError, OSError): + pass + # tokens / token_breakdown ↔ _session_tokens.json + tf = self._token_path(session_id) + if os.path.isfile(tf): + try: + with open(tf, "r", encoding="utf-8") as f: + data = json.load(f) + if "tokens" in data: + extra["tokens"] = data.get("tokens", 0) + if "breakdown" in data and isinstance(data.get("breakdown"), dict): + extra["token_breakdown"] = data["breakdown"] + except (json.JSONDecodeError, OSError): + pass + return extra + + def save_session_extra(self, session_id: str, extra: dict[str, Any]) -> None: + if not session_id: + return + sess_dir = self._session_dir(session_id) + os.makedirs(sess_dir, exist_ok=True) + if "agent_meta" in extra: + with open(self._meta_path(session_id), "w", encoding="utf-8") as f: + json.dump(extra.get("agent_meta") or {}, f, ensure_ascii=False, indent=2) + if "tokens" in extra or "token_breakdown" in extra: + payload: dict[str, Any] = { + "session_id": session_id, + "tokens": extra.get("tokens", 0), + } + if "token_breakdown" in extra and isinstance(extra.get("token_breakdown"), dict): + payload["breakdown"] = extra["token_breakdown"] + with open(self._token_path(session_id), "w", encoding="utf-8") as f: + json.dump(payload, f, ensure_ascii=False, indent=2) + + # ---- 会话生命周期 ---- + def list_session_ids(self) -> list[str]: + root = self._root() + if not os.path.isdir(root): + return [] + entries: list[tuple[float, str]] = [] + for name in os.listdir(root): + dir_path = os.path.join(root, name) + if not os.path.isdir(dir_path): + continue + try: + has_turns = any( + f.startswith(TURN_JSON_PREFIX) and f.endswith(".json") + for f in os.listdir(dir_path) + ) + except OSError: + has_turns = False + if has_turns: + entries.append((os.path.getmtime(dir_path), name)) + entries.sort(key=lambda x: x[0], reverse=True) + return [name for _, name in entries] + + def get_last_turn_id(self, session_id: str) -> str: + sess_dir = self._session_dir(session_id) + if not os.path.isdir(sess_dir): + return "" + turn_files = sorted( + f for f in os.listdir(sess_dir) + if f.startswith(TURN_JSON_PREFIX) and f.endswith(".json") + ) + if not turn_files: + return "" + return turn_files[-1][len(TURN_JSON_PREFIX):-len(".json")] + + def delete_turns_after(self, session_id: str, keep_until_turn_id: str) -> int: + """删除 keep 之后的 turn 消息与工具调用记录（turn_*.json / tool_*.json）。""" + sess_dir = self._session_dir(session_id) + if not os.path.isdir(sess_dir): + return 0 + keep_turn_full = f"{TURN_JSON_PREFIX}{keep_until_turn_id}.json" + keep_tool_full = f"tool_{keep_until_turn_id}.json" + deleted = 0 + for fname in sorted(os.listdir(sess_dir)): + if fname.startswith(TURN_JSON_PREFIX) and fname.endswith(".json"): + if fname > keep_turn_full: + try: + os.remove(os.path.join(sess_dir, fname)) + deleted += 1 + except OSError: + pass + elif fname.startswith("tool_") and fname.endswith(".json"): + if fname > keep_tool_full: + try: + os.remove(os.path.join(sess_dir, fname)) + except OSError: + pass + return deleted + + def delete_tool_records(self, session_id: str, keep_turn_ids: set[str] | None = None) -> None: + sess_dir = self._session_dir(session_id) + if not os.path.isdir(sess_dir): + return + for fname in os.listdir(sess_dir): + if not (fname.startswith("tool_") and fname.endswith(".json")): + continue + if keep_turn_ids: + turn_id = fname[len("tool_"):-len(".json")] + if turn_id in keep_turn_ids: + continue + try: + os.remove(os.path.join(sess_dir, fname)) + except OSError: + pass + + def delete_session(self, session_id: str) -> None: + sess_dir = self._session_dir(session_id) + if os.path.isdir(sess_dir): + shutil.rmtree(sess_dir, ignore_errors=True) + + +# ---------- MySQL 实现 ---------- +class MysqlRecordRepository: + """基于 MySQL（pymysql）的记录仓库。 + + session 作用域表: sessions / session_messages / session_tool_calls； + cron 作用域表: cron_messages / cron_tool_calls。 + 消息与工具调用记录原样 JSON 入库；工具记录 meta 全量透传（不白名单）。 + """ + + def __init__(self, scope: str = "session") -> None: + self.scope = scope if scope == "cron" else "session" + if self.scope == "cron": + _ensure_cron_tables() + else: + _ensure_session_tables() + self._ensure_extra_column() + + # ---- 表名 / 连接 ---- + def _messages_table(self) -> str: + return "cron_messages" if self.scope == "cron" else "session_messages" + + def _tool_table(self) -> str: + return "cron_tool_calls" if self.scope == "cron" else "session_tool_calls" + + def _sessions_table(self) -> str | None: + return None if self.scope == "cron" else "sessions" + + def _sid_col(self) -> str: + return "task_id" if self.scope == "cron" else "session_id" + + @staticmethod + def _now_iso() -> str: + from datetime import datetime + return datetime.now().strftime("%Y-%m-%dT%H:%M:%S") + + def _ensure_extra_column(self) -> None: + """幂等添加 sessions.extra JSON 列（存储 tokens + agent_meta）。""" + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute("ALTER TABLE sessions ADD COLUMN extra JSON DEFAULT NULL") + conn.commit() + except Exception: + pass # 列已存在 → 忽略 + finally: + conn.close() + + @staticmethod + def _loads(raw: Any) -> Any: + """pymysql 对 JSON 列可能返回 dict 或 str，统一反序列化。""" + if isinstance(raw, str): + try: + return json.loads(raw) + except (json.JSONDecodeError, TypeError): + return {} + if isinstance(raw, (bytes, bytearray)): + try: + return json.loads(raw.decode("utf-8")) + except Exception: + return {} + return raw + + # ---- 消息 ---- + def save_turn(self, session_id: str, turn_id: str, messages: list[dict]) -> None: + if not session_id or not turn_id: + return + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute( + f"DELETE FROM {self._messages_table()} WHERE {sid_col}=%s AND turn_id=%s", + (session_id, turn_id), + ) + if self.scope == "cron": + # cron 表按 role/content/meta 分列存储，附件路径转相对 CRON_ROOT + messages = [_to_rel_paths(m, session_id) for m in messages or []] + for msg in messages: + content = msg.get("content") + meta = msg.get("meta") + cur.execute( + f"INSERT INTO {self._messages_table()} " + f"({sid_col}, turn_id, role, content, meta) VALUES (%s,%s,%s,%s,%s)", + ( + session_id, turn_id, msg.get("role", "user"), + json.dumps(content, ensure_ascii=False), + json.dumps(meta, ensure_ascii=False) if meta is not None else None, + ), + ) + else: + for seq, msg in enumerate(messages or []): + cur.execute( + f"INSERT INTO {self._messages_table()} " + f"(session_id, turn_id, seq, message) VALUES (%s,%s,%s,%s)", + (session_id, turn_id, seq, json.dumps(msg, ensure_ascii=False)), + ) + now = self._now_iso() + cur.execute( + """ + INSERT INTO sessions (session_id, created_at, updated_at, last_turn_id) + VALUES (%s,%s,%s,%s) + ON DUPLICATE KEY UPDATE updated_at=VALUES(updated_at), last_turn_id=VALUES(last_turn_id) + """, + (session_id, now, now, turn_id), + ) + conn.commit() + finally: + conn.close() + + def load_messages(self, session_id: str) -> list[dict]: + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + if self.scope == "cron": + cur.execute( + f"SELECT turn_id, role, content, meta FROM {self._messages_table()} " + f"WHERE {sid_col}=%s ORDER BY id", + (session_id,), + ) + rows = cur.fetchall() or [] + msgs: list[dict] = [] + for r in rows: + content = self._loads(r.get("content")) + meta = self._loads(r.get("meta")) + msg: dict[str, Any] = {"role": r.get("role", "user"), "content": content} + if isinstance(meta, dict) and meta: + msg["meta"] = meta + msgs.append(_resolve_msg_paths(msg, session_id)) + return msgs + cur.execute( + f"SELECT message FROM {self._messages_table()} " + f"WHERE session_id=%s ORDER BY turn_id ASC, seq ASC", + (session_id,), + ) + rows = cur.fetchall() or [] + return [self._loads(r["message"]) for r in rows] + finally: + conn.close() + + def load_turn_messages(self, session_id: str, turn_id: str) -> list[dict] | None: + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + if self.scope == "cron": + cur.execute( + f"SELECT turn_id, role, content, meta FROM {self._messages_table()} " + f"WHERE {sid_col}=%s AND turn_id=%s ORDER BY id", + (session_id, turn_id), + ) + rows = cur.fetchall() or [] + if not rows: + return None + msgs: list[dict] = [] + for r in rows: + content = self._loads(r.get("content")) + meta = self._loads(r.get("meta")) + msg: dict[str, Any] = {"role": r.get("role", "user"), "content": content} + if isinstance(meta, dict) and meta: + msg["meta"] = meta + msgs.append(_resolve_msg_paths(msg, session_id)) + return msgs + cur.execute( + f"SELECT message FROM {self._messages_table()} " + f"WHERE session_id=%s AND turn_id=%s ORDER BY seq ASC", + (session_id, turn_id), + ) + rows = cur.fetchall() or [] + if not rows: + return None + return [self._loads(r["message"]) for r in rows] + finally: + conn.close() + + def list_turn_ids(self, session_id: str) -> list[str]: + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute( + f"SELECT DISTINCT turn_id FROM {self._messages_table()} " + f"WHERE {sid_col}=%s ORDER BY turn_id", + (session_id,), + ) + rows = cur.fetchall() or [] + return [r["turn_id"] for r in rows] + finally: + conn.close() + + # ---- 工具调用记录 ---- + def save_tool_calls( + self, session_id: str, turn_id: str, tool_calls: list[dict], meta: dict | None = None + ) -> None: + if not session_id or not turn_id: + return + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute( + f"DELETE FROM {self._tool_table()} WHERE {sid_col}=%s AND turn_id=%s", + (session_id, turn_id), + ) + if self.scope == "cron": + # cron_tool_calls 无 meta 列（与既有表结构一致） + cur.execute( + f"INSERT INTO {self._tool_table()} ({sid_col}, turn_id, tool_calls) " + "VALUES (%s,%s,%s)", + (session_id, turn_id, json.dumps(tool_calls or [], ensure_ascii=False)), + ) + else: + cur.execute( + f"INSERT INTO {self._tool_table()} " + "(session_id, turn_id, tool_calls, meta) VALUES (%s,%s,%s,%s)", + ( + session_id, + turn_id, + json.dumps(tool_calls or [], ensure_ascii=False), + json.dumps(meta, ensure_ascii=False) if meta else None, + ), + ) + conn.commit() + finally: + conn.close() + + @classmethod + def _row_to_record(cls, row: dict[str, Any], session_id: str, turn_id: str, + meta_key: str | None = None) -> dict[str, Any]: + """把工具调用表行还原为与 tool_*.json 同构的 record dict。 + + meta 全量透传（不做字段白名单）：消费方新增 meta 字段无需改动存储层。 + """ + tool_calls = cls._loads(row.get("tool_calls")) or [] + record: dict[str, Any] = { + "session_id": session_id, + "turn_id": turn_id, + "tool_calls": tool_calls, + "timestamp": turn_id, + } + if meta_key and row.get(meta_key) is not None: + meta = cls._loads(row.get(meta_key)) + if isinstance(meta, dict): + record.update(meta) + return record + + def get_turn_record(self, session_id: str, turn_id: str) -> dict[str, Any] | None: + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + if self.scope == "cron": + cur.execute( + f"SELECT tool_calls FROM {self._tool_table()} " + f"WHERE {sid_col}=%s AND turn_id=%s ORDER BY id", + (session_id, turn_id), + ) + rows = cur.fetchall() or [] + out: list[dict] = [] + for r in rows: + tc = self._loads(r.get("tool_calls")) + if isinstance(tc, list): + out.extend(tc) + if not out: + return None + return {"session_id": session_id, "turn_id": turn_id, + "tool_calls": out, "timestamp": turn_id} + cur.execute( + f"SELECT tool_calls, meta FROM {self._tool_table()} " + f"WHERE session_id=%s AND turn_id=%s", + (session_id, turn_id), + ) + row = cur.fetchone() + if not row: + return None + return self._row_to_record(row, session_id, turn_id, meta_key="meta") + finally: + conn.close() + + def get_latest_turn_record(self, session_id: str) -> dict[str, Any] | None: + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + if self.scope == "cron": + cur.execute( + f"SELECT turn_id, tool_calls FROM {self._tool_table()} " + f"WHERE {sid_col}=%s ORDER BY turn_id DESC LIMIT 1", + (session_id,), + ) + row = cur.fetchone() + if not row: + return None + tc = self._loads(row.get("tool_calls")) + return {"session_id": session_id, "turn_id": row.get("turn_id", ""), + "tool_calls": tc if isinstance(tc, list) else [], + "timestamp": row.get("turn_id", "")} + cur.execute( + f"SELECT turn_id, tool_calls, meta FROM {self._tool_table()} " + f"WHERE session_id=%s ORDER BY turn_id DESC LIMIT 1", + (session_id,), + ) + row = cur.fetchone() + if not row: + return None + return self._row_to_record(row, session_id, row["turn_id"], meta_key="meta") + finally: + conn.close() + + def list_turn_records(self, session_id: str, limit: int = 5) -> list[dict[str, Any]]: + """按 turn_id 倒序返回最近 limit 条 turn 记录（与 tool_*.json 同构）。""" + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + if self.scope == "cron": + cur.execute( + f"SELECT turn_id, tool_calls FROM {self._tool_table()} " + f"WHERE {sid_col}=%s ORDER BY turn_id DESC LIMIT %s", + (session_id, limit), + ) + rows = cur.fetchall() or [] + out: list[dict] = [] + for r in rows: + tc = self._loads(r.get("tool_calls")) + out.append({"session_id": session_id, "turn_id": r.get("turn_id", ""), + "tool_calls": tc if isinstance(tc, list) else [], + "timestamp": r.get("turn_id", "")}) + return out + cur.execute( + f"SELECT turn_id, tool_calls, meta FROM {self._tool_table()} " + f"WHERE session_id=%s ORDER BY turn_id DESC LIMIT %s", + (session_id, limit), + ) + rows = cur.fetchall() or [] + return [self._row_to_record(r, session_id, r["turn_id"], meta_key="meta") for r in rows] + finally: + conn.close() + + def persist_live_records( + self, session_id: str, turn_id: str, records: list[dict], started_at: float | None = None + ) -> None: + """实时工具记录不增量入库（由 save_tool_calls 在 turn 结束时统一落库）。""" + + # ---- 会话元数据（extra） ---- + def load_session_extra(self, session_id: str) -> dict[str, Any]: + """读取 sessions.extra JSON 列（含 tokens + agent_meta）；cron 作用域无此表 → {}。""" + table = self._sessions_table() + if table is None: + return {} + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute("SELECT extra FROM sessions WHERE session_id=%s", (session_id,)) + row = cur.fetchone() + raw = (row or {}).get("extra") + return self._loads(raw) if raw is not None else {} + finally: + conn.close() + + def save_session_extra(self, session_id: str, extra: dict[str, Any]) -> None: + """写入 sessions.extra JSON 列（与既有 extra 增量合并，不覆盖未涉及的键）。 + + 与 json 实现按 key 拆分的语义一致：消费方既可 load-modify-save 全量写入， + 也可只传增量键。cron 作用域无此表 → no-op。 + """ + table = self._sessions_table() + if table is None or not session_id: + return + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute("SELECT extra FROM sessions WHERE session_id=%s", (session_id,)) + row = cur.fetchone() + existing = self._loads((row or {}).get("extra")) if row else {} + if not isinstance(existing, dict): + existing = {} + existing.update(extra or {}) + now = self._now_iso() + cur.execute( + """ + INSERT INTO sessions (session_id, created_at, updated_at, extra) + VALUES (%s,%s,%s,%s) + ON DUPLICATE KEY UPDATE updated_at=%s, extra=VALUES(extra) + """, + (session_id, now, now, json.dumps(existing, ensure_ascii=False), now), + ) + conn.commit() + finally: + conn.close() + + # ---- 会话生命周期 ---- + def list_session_ids(self) -> list[str]: + conn = mysql_conn() + try: + with conn.cursor() as cur: + if self.scope == "cron": + cur.execute( + "SELECT session_id FROM cron_messages " + "GROUP BY session_id ORDER BY MAX(id) DESC" + ) + else: + cur.execute("SELECT session_id FROM sessions ORDER BY updated_at DESC") + rows = cur.fetchall() or [] + return [r["session_id"] for r in rows] + finally: + conn.close() + + def get_last_turn_id(self, session_id: str) -> str: + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + if self.scope == "cron": + cur.execute( + f"SELECT MAX(turn_id) AS m FROM {self._messages_table()} " + f"WHERE {sid_col}=%s", + (session_id,), + ) + r = cur.fetchone() + return (r or {}).get("m") or "" + cur.execute( + "SELECT last_turn_id FROM sessions WHERE session_id=%s", + (session_id,), + ) + row = cur.fetchone() + if row and row.get("last_turn_id"): + return row["last_turn_id"] + cur.execute( + "SELECT MAX(turn_id) AS m FROM session_messages WHERE session_id=%s", + (session_id,), + ) + r2 = cur.fetchone() + return (r2 or {}).get("m") or "" + finally: + conn.close() + + def delete_turns_after(self, session_id: str, keep_until_turn_id: str) -> int: + if not session_id: + return 0 + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute( + f"DELETE FROM {self._messages_table()} " + f"WHERE {sid_col}=%s AND turn_id > %s", + (session_id, keep_until_turn_id), + ) + cur.execute( + f"DELETE FROM {self._tool_table()} " + f"WHERE {sid_col}=%s AND turn_id > %s", + (session_id, keep_until_turn_id), + ) + if self.scope == "session": + cur.execute( + "SELECT MAX(turn_id) AS m FROM session_messages WHERE session_id=%s", + (session_id,), + ) + r = cur.fetchone() + last = (r or {}).get("m") or "" + cur.execute( + "UPDATE sessions SET last_turn_id=%s, updated_at=%s WHERE session_id=%s", + (last, self._now_iso(), session_id), + ) + conn.commit() + finally: + conn.close() + return 0 + + def delete_tool_records(self, session_id: str, keep_turn_ids: set[str] | None = None) -> None: + if not session_id: + return + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + if keep_turn_ids: + if not keep_turn_ids: + return + placeholders = ",".join(["%s"] * len(keep_turn_ids)) + cur.execute( + f"DELETE FROM {self._tool_table()} " + f"WHERE {sid_col}=%s AND turn_id NOT IN ({placeholders})", + (session_id, *sorted(keep_turn_ids)), + ) + else: + cur.execute( + f"DELETE FROM {self._tool_table()} WHERE {sid_col}=%s", + (session_id,), + ) + conn.commit() + finally: + conn.close() + + def delete_session(self, session_id: str) -> None: + if not session_id: + return + sid_col = self._sid_col() + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute( + f"DELETE FROM {self._messages_table()} WHERE {sid_col}=%s", (session_id,) + ) + cur.execute( + f"DELETE FROM {self._tool_table()} WHERE {sid_col}=%s", (session_id,) + ) + if self.scope == "session": + cur.execute("DELETE FROM sessions WHERE session_id=%s", (session_id,)) + conn.commit() + finally: + conn.close() + + +# ---------- JSON 任务仓库 ---------- +class JsonTaskConfigRepository: + """基于 JSON 文件的任务仓库。 + + 存储路径: CRON_ROOT/{task_id}/cron_task.json + """ + + def _task_dir(self, task_id: str) -> str: + return os.path.join(CRON_ROOT, task_id) + + def _task_file(self, task_id: str) -> str: + return os.path.join(self._task_dir(task_id), "cron_task.json") + + def list_tasks(self) -> list[CronTask]: + if not os.path.isdir(CRON_ROOT): + return [] + tasks: list[CronTask] = [] + for name in os.listdir(CRON_ROOT): + tf = self._task_file(name) + if not os.path.isfile(tf): + continue + task = self._load(tf) + if task: + tasks.append(task) + tasks.sort(key=lambda t: t.created_at, reverse=True) + return tasks + + def get_task(self, task_id: str) -> CronTask | None: + return self._load(self._task_file(task_id)) + + def save_task(self, task: CronTask) -> None: + if not task.task_id: + return + d = self._task_dir(task.task_id) + os.makedirs(d, exist_ok=True) + with open(self._task_file(task.task_id), "w", encoding="utf-8") as f: + json.dump(task.to_dict(), f, ensure_ascii=False, indent=2) + + def delete_task(self, task_id: str) -> None: + if not task_id: + return + d = self._task_dir(task_id) + if os.path.isdir(d): + shutil.rmtree(d, ignore_errors=True) + + def update_status( + self, + task_id: str, + status: str, + *, + error: str = "", + last_run_at: str = "", + next_run_at: str = "", + ) -> None: + task = self.get_task(task_id) + if not task: + return + task.last_status = status # type: ignore[assignment] + if error: + task.last_error = error + elif status != "failed": + task.last_error = "" + if last_run_at: + task.last_run_at = last_run_at + if next_run_at is not None: + task.next_run_at = next_run_at + self.save_task(task) + + def append_execution(self, task_id: str, exec_log: dict[str, Any]) -> None: + task = self.get_task(task_id) + if not task: + return + task.executions.append(ExecutionLog.from_dict(exec_log)) + self.save_task(task) + + @staticmethod + def _load(path: str) -> CronTask | None: + if not os.path.isfile(path): + return None + try: + with open(path, "r", encoding="utf-8") as f: + return CronTask.from_dict(json.load(f)) + except (json.JSONDecodeError, OSError): + return None + + +# ---------- MySQL 任务仓库 ---------- +def _row_to_task(row: dict[str, Any]) -> CronTask: + """MySQL 行 → CronTask（构造与 to_dict 同构的 dict 再 from_dict）。""" + executions = row.get("executions") + if isinstance(executions, str): + try: + executions = json.loads(executions) + except (json.JSONDecodeError, TypeError): + executions = [] + elif isinstance(executions, (bytes, bytearray)): + try: + executions = json.loads(executions.decode("utf-8")) + except Exception: + executions = [] + elif not isinstance(executions, list): + executions = [] + d = { + "task_id": row.get("task_id", ""), + "name": row.get("name", ""), + "prompt": row.get("prompt", ""), + "trigger": { + "type": row.get("trigger_type", "cron"), + "cron": row.get("cron_expr", "") or "", + "interval_seconds": int(row.get("interval_seconds", 0) or 0), + "run_at": row.get("run_at", "") or "", + }, + "enabled": bool(row.get("enabled", 1)), + "timeout_seconds": int(row.get("timeout_seconds", 300) or 300), + "created_at": row.get("created_at", "") or "", + "next_run_at": row.get("next_run_at", "") or "", + "last_run_at": row.get("last_run_at", "") or "", + "last_status": row.get("last_status", "pending") or "pending", + "last_error": row.get("last_error", "") or "", + "executions": executions, + } + return CronTask.from_dict(d) + + +class MysqlTaskConfigRepository: + """基于 MySQL（pymysql）的任务仓库。 + + 表: cron_tasks。trigger 拆为 trigger_type/cron_expr/interval_seconds/run_at 列， + executions 存为 JSON 列。 + """ + + def __init__(self) -> None: + _ensure_cron_tables() + + def list_tasks(self) -> list[CronTask]: + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute("SELECT * FROM cron_tasks ORDER BY created_at DESC") + rows = cur.fetchall() or [] + return [_row_to_task(r) for r in rows] + finally: + conn.close() + + def get_task(self, task_id: str) -> CronTask | None: + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute("SELECT * FROM cron_tasks WHERE task_id=%s", (task_id,)) + row = cur.fetchone() + return _row_to_task(row) if row else None + finally: + conn.close() + + def save_task(self, task: CronTask) -> None: + if not task.task_id: + return + trig = task.trigger + execs_json = json.dumps([e.to_dict() for e in task.executions], ensure_ascii=False) + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute( + """ + INSERT INTO cron_tasks + (task_id, name, prompt, trigger_type, cron_expr, interval_seconds, run_at, + enabled, timeout_seconds, created_at, next_run_at, last_run_at, + last_status, last_error, executions) + VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s) + ON DUPLICATE KEY UPDATE + name=VALUES(name), prompt=VALUES(prompt), trigger_type=VALUES(trigger_type), + cron_expr=VALUES(cron_expr), interval_seconds=VALUES(interval_seconds), + run_at=VALUES(run_at), enabled=VALUES(enabled), + timeout_seconds=VALUES(timeout_seconds), next_run_at=VALUES(next_run_at), + last_run_at=VALUES(last_run_at), last_status=VALUES(last_status), + last_error=VALUES(last_error), executions=VALUES(executions) + """, + ( + task.task_id, task.name, task.prompt, trig.type, trig.cron, + trig.interval_seconds, trig.run_at, int(task.enabled), + task.timeout_seconds, task.created_at, task.next_run_at, + task.last_run_at, task.last_status, task.last_error, execs_json, + ), + ) + conn.commit() + finally: + conn.close() + + def delete_task(self, task_id: str) -> None: + if not task_id: + return + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute("DELETE FROM cron_tasks WHERE task_id=%s", (task_id,)) + cur.execute("DELETE FROM cron_messages WHERE task_id=%s", (task_id,)) + cur.execute("DELETE FROM cron_tool_calls WHERE task_id=%s", (task_id,)) + conn.commit() + finally: + conn.close() + + def update_status( + self, + task_id: str, + status: str, + *, + error: str = "", + last_run_at: str = "", + next_run_at: str = "", + ) -> None: + sets: list[str] = ["last_status=%s"] + vals: list[Any] = [status] + if error: + sets.append("last_error=%s") + vals.append(error) + elif status != "failed": + sets.append("last_error=%s") + vals.append("") + if last_run_at: + sets.append("last_run_at=%s") + vals.append(last_run_at) + sets.append("next_run_at=%s") + vals.append(next_run_at) + vals.append(task_id) + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute( + f"UPDATE cron_tasks SET {', '.join(sets)} WHERE task_id=%s", + vals, + ) + conn.commit() + finally: + conn.close() + + def append_execution(self, task_id: str, exec_log: dict[str, Any]) -> None: + conn = mysql_conn() + try: + with conn.cursor() as cur: + cur.execute("SELECT executions FROM cron_tasks WHERE task_id=%s", (task_id,)) + row = cur.fetchone() + execs: list = [] + if row and row.get("executions"): + raw = row["executions"] + if isinstance(raw, str): + try: + execs = json.loads(raw) + except (json.JSONDecodeError, TypeError): + execs = [] + elif isinstance(raw, list): + execs = raw + execs.append(exec_log) + cur.execute( + "UPDATE cron_tasks SET executions=%s WHERE task_id=%s", + (json.dumps(execs, ensure_ascii=False), task_id), + ) + conn.commit() + finally: + conn.close() + + +# ---------- 工厂 ---------- +_repo_cache: dict[str, Any] = {} + + +def get_record_repository(scope: str | None = None) -> ConversationRecordRepository: + """获取记录仓库实例（按 STORAGE_BACKEND + 作用域选择实现并缓存）。 + + 主进程读 cron 记录时显式传 scope="cron"；其余场合不传，使用模块级作用域。 + """ + scope = (scope or _STORAGE_SCOPE) if (scope or _STORAGE_SCOPE) in ("session", "cron") else "session" + backend = get_storage_backend() + key = f"{scope}:{backend}" + if key not in _repo_cache: + if backend == "mysql": + _repo_cache[key] = MysqlRecordRepository(scope) + else: + _repo_cache[key] = JsonRecordRepository(scope) + return _repo_cache[key] # type: ignore[return-value] + + +def get_task_repository() -> TaskConfigRepository: + """获取任务仓库实例（按配置选择 json / mysql）。""" + backend = get_storage_backend() + if backend == "mysql": + if "task_mysql" not in _repo_cache: + _repo_cache["task_mysql"] = MysqlTaskConfigRepository() + return _repo_cache["task_mysql"] # type: ignore[return-value] + if "task_json" not in _repo_cache: + _repo_cache["task_json"] = JsonTaskConfigRepository() + return _repo_cache["task_json"] # type: ignore[return-value] + + +def new_task_id() -> str: + """生成基于时间戳的任务 ID（与 session_id 同构风格）。""" + from datetime import datetime + base = datetime.now().strftime("%Y%m%d_%H%M%S") + existing = {t.task_id for t in get_task_repository().list_tasks()} + if base not in existing: + return base + n = 1 + while True: + cand = f"{base}_{n}" + if cand not in existing: + return cand + n += 1 + + +# ---------- 模块级便捷函数（消费方无感知后端选择；失败仅日志，不阻断主流程） ---------- +def _repo() -> ConversationRecordRepository: + return get_record_repository() + + +def save_turn(session_id: str, turn_id: str, messages: list[dict]) -> None: + """持久化一轮对话消息（后端按配置选择，不双写）。""" + try: + _repo().save_turn(session_id, turn_id, messages) + except Exception as e: + print(f"[storage] save_turn 失败: {e}", file=sys.stderr) + + +def load_messages(session_id: str) -> list[dict]: + """按时间顺序加载所有 turn 的消息，返回扁平列表。""" + try: + return _repo().load_messages(session_id) + except Exception as e: + print(f"[storage] load_messages 失败: {e}", file=sys.stderr) + return [] + + +def load_turn_messages(session_id: str, turn_id: str) -> list[dict] | None: + """加载指定 turn 的消息；不存在返回 None。""" + try: + return _repo().load_turn_messages(session_id, turn_id) + except Exception as e: + print(f"[storage] load_turn_messages 失败: {e}", file=sys.stderr) + return None + + +def list_turn_ids(session_id: str) -> list[str]: + """列出会话全部 turn_id（升序）。""" + try: + return _repo().list_turn_ids(session_id) + except Exception as e: + print(f"[storage] list_turn_ids 失败: {e}", file=sys.stderr) + return [] + + +def save_tool_calls( + session_id: str, turn_id: str, tool_calls: list[dict], meta: dict | None = None +) -> None: + """持久化一轮工具调用记录（meta 全量透传）。""" + try: + _repo().save_tool_calls(session_id, turn_id, tool_calls, meta) + except Exception as e: + print(f"[storage] save_tool_calls 失败: {e}", file=sys.stderr) + + +def get_turn_record(session_id: str, turn_id: str) -> dict[str, Any] | None: + """读取指定 turn 的工具调用记录（与 tool_*.json 同构）。""" + try: + return _repo().get_turn_record(session_id, turn_id) + except Exception as e: + print(f"[storage] get_turn_record 失败: {e}", file=sys.stderr) + return None + + +def get_latest_turn_record(session_id: str) -> dict[str, Any] | None: + """读取最近一个 turn 的工具调用记录。""" + try: + return _repo().get_latest_turn_record(session_id) + except Exception as e: + print(f"[storage] get_latest_turn_record 失败: {e}", file=sys.stderr) + return None + + +def list_turn_records(session_id: str, limit: int = 5) -> list[dict[str, Any]]: + """按 turn_id 倒序返回最近 limit 条 turn 记录。""" + try: + return _repo().list_turn_records(session_id, limit) + except Exception as e: + print(f"[storage] list_turn_records 失败: {e}", file=sys.stderr) + return [] + + +def persist_live_records( + session_id: str, turn_id: str, records: list[dict], started_at: float | None = None +) -> None: + """实时工具记录增量持久化（可选行为：json 写盘，mysql 不实现）。""" + try: + _repo().persist_live_records(session_id, turn_id, records, started_at) + except Exception: + pass # 持久化失败不应影响主流程 + + +def load_session_extra(session_id: str) -> dict[str, Any]: + """读取会话扩展元数据（tokens / token_breakdown / agent_meta）。""" + try: + return _repo().load_session_extra(session_id) or {} + except Exception as e: + print(f"[storage] load_session_extra 失败: {e}", file=sys.stderr) + return {} + + +def save_session_extra(session_id: str, extra: dict[str, Any]) -> None: + """写入会话扩展元数据（后端无关）。""" + try: + _repo().save_session_extra(session_id, extra) + except Exception as e: + print(f"[storage] save_session_extra 失败: {e}", file=sys.stderr) + + +def list_session_ids() -> list[str]: + """按最近活动倒序列出所有会话 ID。""" + try: + return _repo().list_session_ids() + except Exception as e: + print(f"[storage] list_session_ids 失败: {e}", file=sys.stderr) + return [] + + +def get_last_turn_id(session_id: str) -> str: + """获取最后一条 turn_id；无记录返回空串。""" + try: + return _repo().get_last_turn_id(session_id) or "" + except Exception as e: + print(f"[storage] get_last_turn_id 失败: {e}", file=sys.stderr) + return "" + + +def delete_turns_after(session_id: str, keep_until_turn_id: str) -> int: + """删除 keep_until_turn_id 之后的 turn 消息与工具调用记录；返回删除的 turn 数。""" + try: + return _repo().delete_turns_after(session_id, keep_until_turn_id) + except Exception as e: + print(f"[storage] delete_turns_after 失败: {e}", file=sys.stderr) + return 0 + + +def delete_tool_records(session_id: str, keep_turn_ids: set[str] | None = None) -> None: + """删除工具调用记录（keep_turn_ids 为 None 时全部删除）。""" + try: + _repo().delete_tool_records(session_id, keep_turn_ids) + except Exception as e: + print(f"[storage] delete_tool_records 失败: {e}", file=sys.stderr) + + +def delete_session(session_id: str) -> None: + """删除会话全部记录（json 后端同时删除会话目录）。""" + try: + _repo().delete_session(session_id) + except Exception as e: + print(f"[storage] delete_session 失败: {e}", file=sys.stderr) diff --git a/src/agent/core/turn_runner.py b/src/agent/core/turn_runner.py new file mode 100644 index 0000000..3795d43 --- /dev/null +++ b/src/agent/core/turn_runner.py @@ -0,0 +1,90 @@ +"""轮次执行器：后台线程驱动 execute_agent，解耦 HTTP 请求与图执行生命周期。 + +系统定位: + Web 层将每次对话轮次放入后台线程执行，HTTP 请求通过 ``start_turn`` / + ``wait_turn`` 挂载到同一轮次。断连/关标签页不影响执行——轮次照常跑完并 + 落盘，重连后可见；同一会话的重复请求共享同一轮，避免并发竞态。 + + 轮次内的人工交互由 ``human_request.ask_human`` 在图上线程中阻塞等待， + 前端通过 live snapshot 渲染浮窗，答案由 ``POST /api/human-action`` 应答。 + +可扩展性: + 可增加轮次超时、自动清理、多 runtime 支持。 +""" +from __future__ import annotations + +import threading +from typing import Any + +_turns: dict[str, dict[str, Any]] = {} +_turns_lock = threading.Lock() + + +def start_turn(session_id: str, chat_history: list[dict[str, Any]], agent_mode: str = "agent") -> str: + """为会话启动（或复用）一轮后台执行。 + + 输入: + session_id: 会话 ID。 + chat_history: 以末条 user 结尾的聊天历史（与 execute_agent 入参一致）。 + agent_mode: agent | plan。 + + 输出: + "running" | "done"——该会话是否已有轮次在运行。 + """ + with _turns_lock: + existing = _turns.get(session_id) + if existing and not existing["done"].is_set(): + return "running" + + handle: dict[str, Any] = { + "session_id": session_id, + "done": threading.Event(), + "result": None, + } + _turns[session_id] = handle + + def _worker(): + try: + from agent.agents.agent_runtime import execute_agent + new_hist = execute_agent(list(chat_history), session_id=session_id, agent_mode=agent_mode) + handle["result"] = {"chat_history": new_hist} + except Exception as e: + import traceback + traceback.print_exc() + handle["result"] = {"error": str(e)} + finally: + handle["done"].set() + + thread = threading.Thread(target=_worker, name=f"turn-{session_id}", daemon=True) + handle["thread"] = thread + thread.start() + return "running" + + +def wait_turn(session_id: str, timeout: float | None = None) -> dict[str, Any] | None: + """等待轮次结束并返回 execute_agent 的结果 dict。 + + 输出: + {"chat_history": list} | {"error": str}；无运行轮次或等待超时返回 None。 + timeout=None 表示无限等待（人工交互可能持续较长时间）。 + """ + with _turns_lock: + handle = _turns.get(session_id) + if not handle: + return None + if not handle["done"].wait(timeout=timeout): + return None + return handle["result"] + + +def get_turn_status(session_id: str) -> str: + """查询会话当前是否有轮次在运行。""" + with _turns_lock: + handle = _turns.get(session_id) + return "running" if handle and not handle["done"].is_set() else "done" + + +def drop_turn(session_id: str) -> None: + """丢弃会话的轮次句柄（会话删除时调用；运行中的轮次不受影响，由 abort 负责终止）。""" + with _turns_lock: + _turns.pop(session_id, None) diff --git a/src/agent/cron/runner.py b/src/agent/cron/runner.py index 9bdae5b..7fe099c 100644 --- a/src/agent/cron/runner.py +++ b/src/agent/cron/runner.py @@ -44,6 +44,9 @@ def _redirect_storage_roots(task_id: str) -> str: - agent.utils.agent_utils.SESSIONS_ROOT → turn 文件、session_dir、get_history - agent.history.tool_call_recorder.TOOL_CALLING_ROOT → tool_*.json、token - web.ui_session.SESSIONS_ROOT → turn relpath 计算 + 同时将统一存储（agent.core.storage）的作用域切换为 "cron"， + 使消息/工具调用记录直接读写 cron 作用域（CRON_ROOT 文件 / cron_* 表）， + 无需执行后的文件→DB 桥接。 返回重定向后的任务目录绝对路径。 """ @@ -57,6 +60,10 @@ def _redirect_storage_roots(task_id: str) -> str: import web.ui_session as ui ui.SESSIONS_ROOT = CRON_ROOT + # 统一存储：模块级便捷函数路由到 cron 作用域 + from agent.core import storage as _storage + _storage.set_storage_scope("cron") + task_dir = os.path.join(CRON_ROOT, task_id) os.makedirs(task_dir, exist_ok=True) return task_dir @@ -271,14 +278,7 @@ def main() -> int: except Exception: pass - # 6.5 MySQL 模式下把执行记录导入数据库（json 模式无操作，文件即存储） - try: - from agent.cron.storage import persist_turn_records - persist_turn_records(task_id, turn_id) - except Exception as e: - print(f"[cron.runner] 导入执行记录失败: {e}", file=sys.stderr) - - # 7. 回传最终状态 + # 7. 回传最终状态（执行记录已由统一存储按 scope=cron 直写，无需桥接导入） _post_finished(task_id, status, error_msg, turn_id) # 给转发线程一点时间发送最后的快照 diff --git a/src/agent/cron/storage.py b/src/agent/cron/storage.py index 6faf66e..7b13964 100644 --- a/src/agent/cron/storage.py +++ b/src/agent/cron/storage.py @@ -1,760 +1,23 @@ -"""定时任务存储抽象层。 +"""定时任务存储转发层（统一实现在 agent.core.storage）。 -系统定位: - 为 scheduler / API / cron_manager 工具提供统一的任务与记录读写接口。 - 支持两种后端：JSON 文件（history/cron/{task_id}/cron_task.json）与 - MySQL（pymysql，表 cron_tasks / cron_messages / cron_tool_calls）。 - -可扩展性: - - get_task_repository() / get_record_repository() 依据 env_config.json 的 - STORAGE_BACKEND（默认 "json"）选择实现，两种后端均完整支持增删改查。 - - 多模态附件地址在读取时解析为绝对地址（相对 CRON_ROOT）， - 保证数据库迁移到不同机器后仍可定位文件。 +本模块保持既有导入路径（scheduler / cron_manager / server / runner）， +实际实现与普通会话存储统一在 ``agent.core.storage``： + - ``get_record_repository()`` 固定返回 cron 作用域记录仓库 + （cron_messages/cron_tool_calls 表，或 history/cron 下的 JSON 文件）。 + - 新增存储后端只需在 ``agent.core.storage`` 实现 Protocol 并注册工厂。 """ -from __future__ import annotations - -import json -import os -import shutil -from pathlib import Path -from typing import Any, Protocol - -from agent.cron.models import CronTask, ExecutionLog, now_iso -from agent.utils.agent_utils import HISTORY_ROOT -from agent.core.db import ( - mysql_conn, - ensure_tables, - get_storage_backend, +from agent.core.storage import ( + CRON_ROOT, + TaskConfigRepository, + ConversationRecordRepository, + get_task_repository, + new_task_id, + set_storage_scope, + get_storage_scope, + get_record_repository as _get_core_record_repository, ) -# 定时任务根目录：history/cron -CRON_ROOT = os.path.join(HISTORY_ROOT, "cron") -os.makedirs(CRON_ROOT, exist_ok=True) - -# ---------- Protocol 定义 ---------- -class CronTaskRepository(Protocol): - """任务配置与状态的读写接口。""" - - def list_tasks(self) -> list[CronTask]: ... - def get_task(self, task_id: str) -> CronTask | None: ... - def save_task(self, task: CronTask) -> None: ... - def delete_task(self, task_id: str) -> None: ... - def update_status( - self, - task_id: str, - status: str, - *, - error: str = "", - last_run_at: str = "", - next_run_at: str = "", - ) -> None: ... - def append_execution(self, task_id: str, exec_log: dict[str, Any]) -> None: ... - - -class CronRecordRepository(Protocol): - """任务执行记录（消息与工具调用）的读取接口。""" - - def load_messages(self, task_id: str) -> list[dict]: ... - def list_turns(self, task_id: str) -> list[dict]: ... - def get_tool_calls(self, task_id: str, turn_id: str) -> list[dict]: ... - - -# ---------- JSON 实现 ---------- -class JsonCronTaskRepository: - """基于 JSON 文件的任务仓库。 - - 存储路径: CRON_ROOT/{task_id}/cron_task.json - """ - - def _task_dir(self, task_id: str) -> str: - return os.path.join(CRON_ROOT, task_id) - - def _task_file(self, task_id: str) -> str: - return os.path.join(self._task_dir(task_id), "cron_task.json") - - def list_tasks(self) -> list[CronTask]: - if not os.path.isdir(CRON_ROOT): - return [] - tasks: list[CronTask] = [] - for name in os.listdir(CRON_ROOT): - tf = self._task_file(name) - if not os.path.isfile(tf): - continue - task = self._load(tf) - if task: - tasks.append(task) - tasks.sort(key=lambda t: t.created_at, reverse=True) - return tasks - - def get_task(self, task_id: str) -> CronTask | None: - return self._load(self._task_file(task_id)) - - def save_task(self, task: CronTask) -> None: - if not task.task_id: - return - d = self._task_dir(task.task_id) - os.makedirs(d, exist_ok=True) - with open(self._task_file(task.task_id), "w", encoding="utf-8") as f: - json.dump(task.to_dict(), f, ensure_ascii=False, indent=2) - - def delete_task(self, task_id: str) -> None: - if not task_id: - return - d = self._task_dir(task_id) - if os.path.isdir(d): - shutil.rmtree(d, ignore_errors=True) - - def update_status( - self, - task_id: str, - status: str, - *, - error: str = "", - last_run_at: str = "", - next_run_at: str = "", - ) -> None: - task = self.get_task(task_id) - if not task: - return - task.last_status = status # type: ignore[assignment] - if error: - task.last_error = error - elif status != "failed": - # 成功类状态清空错误描述 - task.last_error = "" - if last_run_at: - task.last_run_at = last_run_at - if next_run_at is not None: - task.next_run_at = next_run_at - self.save_task(task) - - def append_execution(self, task_id: str, exec_log: dict[str, Any]) -> None: - task = self.get_task(task_id) - if not task: - return - task.executions.append(ExecutionLog.from_dict(exec_log)) - self.save_task(task) - - @staticmethod - def _load(path: str) -> CronTask | None: - if not os.path.isfile(path): - return None - try: - with open(path, "r", encoding="utf-8") as f: - return CronTask.from_dict(json.load(f)) - except (json.JSONDecodeError, OSError): - return None - - -class JsonCronRecordRepository: - """基于 JSON 文件的执行记录仓库。 - - 复用 session 同构的 turn_*.json / tool_*.json（由 runner 写入）。 - 读取时将附件 relpath 解析为绝对地址（相对 CRON_ROOT），保证可移植性。 - """ - - def _task_dir(self, task_id: str) -> str: - return os.path.join(CRON_ROOT, task_id) - - def load_messages(self, task_id: str) -> list[dict]: - """加载该任务所有 turn 的扁平消息列表，附件路径解析为绝对地址。""" - from agent.utils.agent_utils import TURN_JSON_PREFIX - - sess_dir = self._task_dir(task_id) - if not os.path.isdir(sess_dir): - return [] - all_msgs: list[dict] = [] - turn_files = sorted( - f for f in os.listdir(sess_dir) - if f.startswith(TURN_JSON_PREFIX) and f.endswith(".json") - ) - for tf in turn_files: - tp = os.path.join(sess_dir, tf) - if not os.path.isfile(tp): - continue - try: - with open(tp, "r", encoding="utf-8") as f: - data = json.load(f) - for msg in data.get("messages", []): - all_msgs.append(self._resolve_abs_paths(msg, task_id)) - except (json.JSONDecodeError, OSError): - pass - return all_msgs - - def list_turns(self, task_id: str) -> list[dict]: - """列出该任务所有 turn 的元信息（turn_id + messages 概要）。""" - from agent.utils.agent_utils import TURN_JSON_PREFIX - - sess_dir = self._task_dir(task_id) - if not os.path.isdir(sess_dir): - return [] - turns: list[dict] = [] - turn_files = sorted( - f for f in os.listdir(sess_dir) - if f.startswith(TURN_JSON_PREFIX) and f.endswith(".json") - ) - for tf in turn_files: - tp = os.path.join(sess_dir, tf) - if not os.path.isfile(tp): - continue - try: - with open(tp, "r", encoding="utf-8") as f: - data = json.load(f) - turns.append({ - "turn_id": data.get("turn_id", ""), - "messages_count": len(data.get("messages", [])), - }) - except (json.JSONDecodeError, OSError): - pass - return turns - - def get_tool_calls(self, task_id: str, turn_id: str) -> list[dict]: - """读取指定 turn 的工具调用记录。""" - tool_path = os.path.join(self._task_dir(task_id), f"tool_{turn_id}.json") - if not os.path.isfile(tool_path): - return [] - try: - with open(tool_path, "r", encoding="utf-8") as f: - data = json.load(f) - return data.get("tool_calls", []) or [] - except (json.JSONDecodeError, OSError): - return [] - - def _resolve_abs_paths(self, msg: dict, task_id: str) -> dict: - """附件路径解析委托给模块级 _resolve_msg_paths（JSON/MySQL 共用）。""" - return _resolve_msg_paths(msg, task_id) - - -# ---------- MySQL 实现（pymysql） ---------- -# 连接 / 配置 / 后端选择已抽取至 agent.core.db，此处保留别名供模块内部调用 -_mysql_conn = mysql_conn -_get_backend = get_storage_backend - -# cron 三表 DDL（由 ensure_tables 幂等建表） -_CRON_TABLE_DDLS = [ - """ - CREATE TABLE IF NOT EXISTS cron_tasks ( - task_id VARCHAR(64) PRIMARY KEY, - name VARCHAR(255) NOT NULL, - prompt TEXT NOT NULL, - trigger_type ENUM('cron','interval','once') NOT NULL, - cron_expr VARCHAR(128) DEFAULT '', - interval_seconds INT DEFAULT 0, - run_at VARCHAR(32) DEFAULT '', - enabled TINYINT(1) DEFAULT 1, - timeout_seconds INT DEFAULT 300, - created_at VARCHAR(32) NOT NULL, - next_run_at VARCHAR(32) DEFAULT '', - last_run_at VARCHAR(32) DEFAULT '', - last_status VARCHAR(16) DEFAULT 'pending', - last_error TEXT, - executions JSON DEFAULT NULL - ) CHARACTER SET utf8mb4 - """, - """ - CREATE TABLE IF NOT EXISTS cron_messages ( - id BIGINT AUTO_INCREMENT PRIMARY KEY, - task_id VARCHAR(64) NOT NULL, - turn_id VARCHAR(64) NOT NULL, - role VARCHAR(16) NOT NULL, - content JSON NOT NULL, - meta JSON DEFAULT NULL, - INDEX idx_task_turn (task_id, turn_id) - ) CHARACTER SET utf8mb4 - """, - """ - CREATE TABLE IF NOT EXISTS cron_tool_calls ( - id BIGINT AUTO_INCREMENT PRIMARY KEY, - task_id VARCHAR(64) NOT NULL, - turn_id VARCHAR(64) NOT NULL, - tool_calls JSON NOT NULL, - INDEX idx_task_turn (task_id, turn_id) - ) CHARACTER SET utf8mb4 - """, -] - - -def _ensure_mysql_tables() -> None: - """幂等建 cron 三表（委托共享 ensure_tables）。""" - ensure_tables(_CRON_TABLE_DDLS) - - -def _row_to_task(row: dict[str, Any]) -> CronTask: - """MySQL 行 → CronTask（构造与 to_dict 同构的 dict 再 from_dict）。""" - executions = row.get("executions") - if isinstance(executions, str): - try: - executions = json.loads(executions) - except (json.JSONDecodeError, TypeError): - executions = [] - elif isinstance(executions, (bytes, bytearray)): - try: - executions = json.loads(executions.decode("utf-8")) - except Exception: - executions = [] - elif not isinstance(executions, list): - executions = [] - d = { - "task_id": row.get("task_id", ""), - "name": row.get("name", ""), - "prompt": row.get("prompt", ""), - "trigger": { - "type": row.get("trigger_type", "cron"), - "cron": row.get("cron_expr", "") or "", - "interval_seconds": int(row.get("interval_seconds", 0) or 0), - "run_at": row.get("run_at", "") or "", - }, - "enabled": bool(row.get("enabled", 1)), - "timeout_seconds": int(row.get("timeout_seconds", 300) or 300), - "created_at": row.get("created_at", "") or "", - "next_run_at": row.get("next_run_at", "") or "", - "last_run_at": row.get("last_run_at", "") or "", - "last_status": row.get("last_status", "pending") or "pending", - "last_error": row.get("last_error", "") or "", - "executions": executions, - } - return CronTask.from_dict(d) - - -class MysqlCronTaskRepository: - """基于 MySQL（pymysql）的任务仓库。 - - 表: cron_tasks。trigger 拆为 trigger_type/cron_expr/interval_seconds/run_at 列， - executions 存为 JSON 列。 - """ - - def __init__(self) -> None: - _ensure_mysql_tables() - - def list_tasks(self) -> list[CronTask]: - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute("SELECT * FROM cron_tasks ORDER BY created_at DESC") - rows = cur.fetchall() or [] - return [_row_to_task(r) for r in rows] - finally: - conn.close() - - def get_task(self, task_id: str) -> CronTask | None: - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute("SELECT * FROM cron_tasks WHERE task_id=%s", (task_id,)) - row = cur.fetchone() - return _row_to_task(row) if row else None - finally: - conn.close() - - def save_task(self, task: CronTask) -> None: - if not task.task_id: - return - trig = task.trigger - execs_json = json.dumps([e.to_dict() for e in task.executions], ensure_ascii=False) - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - """ - INSERT INTO cron_tasks - (task_id, name, prompt, trigger_type, cron_expr, interval_seconds, run_at, - enabled, timeout_seconds, created_at, next_run_at, last_run_at, - last_status, last_error, executions) - VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s) - ON DUPLICATE KEY UPDATE - name=VALUES(name), prompt=VALUES(prompt), trigger_type=VALUES(trigger_type), - cron_expr=VALUES(cron_expr), interval_seconds=VALUES(interval_seconds), - run_at=VALUES(run_at), enabled=VALUES(enabled), - timeout_seconds=VALUES(timeout_seconds), next_run_at=VALUES(next_run_at), - last_run_at=VALUES(last_run_at), last_status=VALUES(last_status), - last_error=VALUES(last_error), executions=VALUES(executions) - """, - ( - task.task_id, task.name, task.prompt, trig.type, trig.cron, - trig.interval_seconds, trig.run_at, int(task.enabled), - task.timeout_seconds, task.created_at, task.next_run_at, - task.last_run_at, task.last_status, task.last_error, execs_json, - ), - ) - conn.commit() - finally: - conn.close() - - def delete_task(self, task_id: str) -> None: - if not task_id: - return - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute("DELETE FROM cron_tasks WHERE task_id=%s", (task_id,)) - cur.execute("DELETE FROM cron_messages WHERE task_id=%s", (task_id,)) - cur.execute("DELETE FROM cron_tool_calls WHERE task_id=%s", (task_id,)) - conn.commit() - finally: - conn.close() - - def update_status( - self, - task_id: str, - status: str, - *, - error: str = "", - last_run_at: str = "", - next_run_at: str = "", - ) -> None: - sets: list[str] = ["last_status=%s"] - vals: list[Any] = [status] - if error: - sets.append("last_error=%s") - vals.append(error) - elif status != "failed": - sets.append("last_error=%s") - vals.append("") - if last_run_at: - sets.append("last_run_at=%s") - vals.append(last_run_at) - # next_run_at 显式更新（空串也写，表示不再运行） - sets.append("next_run_at=%s") - vals.append(next_run_at) - vals.append(task_id) - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - f"UPDATE cron_tasks SET {', '.join(sets)} WHERE task_id=%s", - vals, - ) - conn.commit() - finally: - conn.close() - - def append_execution(self, task_id: str, exec_log: dict[str, Any]) -> None: - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute("SELECT executions FROM cron_tasks WHERE task_id=%s", (task_id,)) - row = cur.fetchone() - execs: list = [] - if row and row.get("executions"): - raw = row["executions"] - if isinstance(raw, str): - try: - execs = json.loads(raw) - except (json.JSONDecodeError, TypeError): - execs = [] - elif isinstance(raw, list): - execs = raw - execs.append(exec_log) - cur.execute( - "UPDATE cron_tasks SET executions=%s WHERE task_id=%s", - (json.dumps(execs, ensure_ascii=False), task_id), - ) - conn.commit() - finally: - conn.close() - - -class MysqlCronRecordRepository: - """基于 MySQL（pymysql）的执行记录仓库。 - - 表: cron_messages / cron_tool_calls。 - 附件路径统一存相对 CRON_ROOT 的 relpath，读取时按本机 CRON_ROOT 解析为绝对地址。 - 写入由 runner 执行后调用 persist_turn_records 触发（见模块底部）。 - """ - - def __init__(self) -> None: - _ensure_mysql_tables() - - @staticmethod - def _parse_json_col(val: Any) -> Any: - if isinstance(val, str): - try: - return json.loads(val) - except (json.JSONDecodeError, TypeError): - return None - if isinstance(val, (bytes, bytearray)): - try: - return json.loads(val.decode("utf-8")) - except Exception: - return None - return val - - def load_messages(self, task_id: str) -> list[dict]: - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT turn_id, role, content, meta FROM cron_messages " - "WHERE task_id=%s ORDER BY id", - (task_id,), - ) - rows = cur.fetchall() or [] - msgs: list[dict] = [] - for r in rows: - content = self._parse_json_col(r.get("content")) or "" - meta = self._parse_json_col(r.get("meta")) - msg: dict[str, Any] = {"role": r.get("role", "user"), "content": content} - if meta: - msg["meta"] = meta - msgs.append(_resolve_msg_paths(msg, task_id)) - return msgs - finally: - conn.close() - - def list_turns(self, task_id: str) -> list[dict]: - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT turn_id, COUNT(*) AS cnt FROM cron_messages " - "WHERE task_id=%s GROUP BY turn_id ORDER BY MIN(id)", - (task_id,), - ) - rows = cur.fetchall() or [] - return [{"turn_id": r.get("turn_id", ""), "messages_count": int(r.get("cnt", 0))} for r in rows] - finally: - conn.close() - - def get_tool_calls(self, task_id: str, turn_id: str) -> list[dict]: - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT tool_calls FROM cron_tool_calls " - "WHERE task_id=%s AND turn_id=%s ORDER BY id", - (task_id, turn_id), - ) - rows = cur.fetchall() or [] - out: list[dict] = [] - for r in rows: - tc = self._parse_json_col(r.get("tool_calls")) - if isinstance(tc, list): - out.extend(tc) - return out - finally: - conn.close() - - def save_turn(self, task_id: str, turn_id: str, messages: list[dict]) -> None: - """写入一个 turn 的消息（幂等：先删同 turn 旧记录）。""" - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "DELETE FROM cron_messages WHERE task_id=%s AND turn_id=%s", - (task_id, turn_id), - ) - for msg in messages: - content = msg.get("content") - meta = msg.get("meta") - cur.execute( - "INSERT INTO cron_messages (task_id, turn_id, role, content, meta) " - "VALUES (%s,%s,%s,%s,%s)", - ( - task_id, turn_id, msg.get("role", "user"), - json.dumps(content, ensure_ascii=False), - json.dumps(meta, ensure_ascii=False) if meta is not None else None, - ), - ) - conn.commit() - finally: - conn.close() - - def save_tool_calls(self, task_id: str, turn_id: str, tool_calls: list[dict]) -> None: - """写入一个 turn 的工具调用记录（幂等）。""" - conn = _mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "DELETE FROM cron_tool_calls WHERE task_id=%s AND turn_id=%s", - (task_id, turn_id), - ) - cur.execute( - "INSERT INTO cron_tool_calls (task_id, turn_id, tool_calls) VALUES (%s,%s,%s)", - (task_id, turn_id, json.dumps(tool_calls, ensure_ascii=False)), - ) - conn.commit() - finally: - conn.close() - - -# ---------- 附件路径解析（JSON / MySQL 共用） ---------- -def _resolve_msg_paths(msg: dict, task_id: str) -> dict: - """将消息 content 中的相对附件路径解析为绝对地址（相对 CRON_ROOT）。 - - runner 写入 / 数据库存储的附件路径均为相对 CRON_ROOT 的 relpath， - 此处返回前端/序列化前还原为绝对路径，供文件读取。 - """ - content = msg.get("content") - if not isinstance(content, list): - return msg - for piece in content: - if not isinstance(piece, dict): - continue - for key in ("path", "audio_path"): - p = piece.get(key) - if isinstance(p, str) and p and not os.path.isabs(p): - piece[key] = os.path.normpath(os.path.join(CRON_ROOT, p)) - iu = piece.get("image_url") - if isinstance(iu, dict): - u = iu.get("url") - if isinstance(u, str) and u and not u.startswith("data:") and not os.path.isabs(u): - iu["url"] = os.path.normpath(os.path.join(CRON_ROOT, u)) - return msg - - -def _to_rel_paths(msg: dict, task_id: str) -> dict: - """_resolve_msg_paths 的逆操作：绝对路径 → 相对 CRON_ROOT 的 relpath（入库前调用）。""" - import copy - msg = copy.deepcopy(msg) - content = msg.get("content") - if not isinstance(content, list): - return msg - cron_root_abs = os.path.abspath(CRON_ROOT) - for piece in content: - if not isinstance(piece, dict): - continue - for key in ("path", "audio_path"): - p = piece.get(key) - if isinstance(p, str) and p and os.path.isabs(p): - try: - rel = os.path.relpath(p, cron_root_abs) - piece[key] = rel.replace("\\", "/") - except ValueError: - pass - iu = piece.get("image_url") - if isinstance(iu, dict): - u = iu.get("url") - if isinstance(u, str) and u and not u.startswith("data:") and os.path.isabs(u): - try: - rel = os.path.relpath(u, cron_root_abs) - iu["url"] = rel.replace("\\", "/") - except ValueError: - pass - return msg - - -# ---------- 工厂 ---------- -_backend_cache: dict[str, Any] = {} - - -def get_task_repository() -> CronTaskRepository: - """获取任务仓库实例（按配置选择 json / mysql）。""" - backend = _get_backend() - if backend == "mysql": - if "task_mysql" not in _backend_cache: - _backend_cache["task_mysql"] = MysqlCronTaskRepository() - return _backend_cache["task_mysql"] # type: ignore[return-value] - if "task_json" not in _backend_cache: - _backend_cache["task_json"] = JsonCronTaskRepository() - return _backend_cache["task_json"] # type: ignore[return-value] - - -def get_record_repository() -> CronRecordRepository: - """获取执行记录仓库实例（按配置选择 json / mysql）。""" - backend = _get_backend() - if backend == "mysql": - if "record_mysql" not in _backend_cache: - _backend_cache["record_mysql"] = MysqlCronRecordRepository() - return _backend_cache["record_mysql"] # type: ignore[return-value] - if "record_json" not in _backend_cache: - _backend_cache["record_json"] = JsonCronRecordRepository() - return _backend_cache["record_json"] # type: ignore[return-value] - - -def new_task_id() -> str: - """生成基于时间戳的任务 ID（与 session_id 同构风格）。""" - from datetime import datetime - base = datetime.now().strftime("%Y%m%d_%H%M%S") - existing = {t.task_id for t in get_task_repository().list_tasks()} - if base not in existing: - return base - n = 1 - while True: - cand = f"{base}_{n}" - if cand not in existing: - return cand - n += 1 - - -def persist_turn_records(task_id: str, turn_id: str) -> None: - """将 runner 写入的 turn/tool 文件记录导入存储后端。 - - - JSON 后端：文件即存储，无需操作。 - - MySQL 后端：读取 history/cron/{task_id}/turn_*.json 与 tool_*.json， - 将附件路径转为相对 CRON_ROOT 的 relpath 后写入 cron_messages / cron_tool_calls。 - - 由 runner.main() 在执行完成后调用。失败仅打印日志，不影响任务状态回传。 - """ - if _get_backend() != "mysql" or not task_id or not turn_id: - return - try: - from agent.utils.agent_utils import TURN_JSON_PREFIX - - sess_dir = os.path.join(CRON_ROOT, task_id) - # 读取 turn 消息 - turn_path = os.path.join(sess_dir, f"{TURN_JSON_PREFIX}{turn_id}.json") - messages: list[dict] = [] - if os.path.isfile(turn_path): - with open(turn_path, "r", encoding="utf-8") as f: - data = json.load(f) - messages = data.get("messages", []) or [] - # 读取工具调用 - tool_path = os.path.join(sess_dir, f"tool_{turn_id}.json") - tool_calls: list[dict] = [] - if os.path.isfile(tool_path): - with open(tool_path, "r", encoding="utf-8") as f: - data = json.load(f) - tool_calls = data.get("tool_calls", []) or [] - # 附件路径转相对（数据库存 relpath，保证可移植性） - messages = [_to_rel_paths(m, task_id) for m in messages] - repo = get_record_repository() - if hasattr(repo, "save_turn"): - repo.save_turn(task_id, turn_id, messages) # type: ignore[attr-defined] - if hasattr(repo, "save_tool_calls"): - repo.save_tool_calls(task_id, turn_id, tool_calls) # type: ignore[attr-defined] - except Exception as e: - import sys - print(f"[cron.storage] persist_turn_records 失败: {e}", file=sys.stderr) - - -# ---------- MySQL 表结构参考（已由 _ensure_mysql_tables 幂等建表实现） ---------- -""" -CREATE TABLE cron_tasks ( - task_id VARCHAR(64) PRIMARY KEY, - name VARCHAR(255) NOT NULL, - prompt TEXT NOT NULL, - trigger_type ENUM('cron','interval','once') NOT NULL, - cron_expr VARCHAR(128) DEFAULT '', - interval_seconds INT DEFAULT 0, - run_at VARCHAR(32) DEFAULT '', - enabled TINYINT(1) DEFAULT 1, - timeout_seconds INT DEFAULT 300, - created_at VARCHAR(32) NOT NULL, - next_run_at VARCHAR(32) DEFAULT '', - last_run_at VARCHAR(32) DEFAULT '', - last_status VARCHAR(16) DEFAULT 'pending', - last_error TEXT, - executions JSON DEFAULT NULL -); - -CREATE TABLE cron_messages ( - id BIGINT AUTO_INCREMENT PRIMARY KEY, - task_id VARCHAR(64) NOT NULL, - turn_id VARCHAR(64) NOT NULL, - role VARCHAR(16) NOT NULL, - content JSON NOT NULL, - meta JSON DEFAULT NULL, - INDEX idx_task_turn (task_id, turn_id) -); - -CREATE TABLE cron_tool_calls ( - id BIGINT AUTO_INCREMENT PRIMARY KEY, - task_id VARCHAR(64) NOT NULL, - turn_id VARCHAR(64) NOT NULL, - tool_calls JSON NOT NULL, - INDEX idx_task_turn (task_id, turn_id) -); - --- 附件路径统一存储相对 CRON_ROOT 的 relpath，读取时按本机 CRON_ROOT 拼绝对路径。 -""" +def get_record_repository() -> ConversationRecordRepository: + """获取 cron 作用域的执行记录仓库实例（按 STORAGE_BACKEND 选择 json / mysql）。""" + return _get_core_record_repository(scope="cron") # type: ignore[return-value] diff --git a/src/agent/history/session_storage.py b/src/agent/history/session_storage.py index 3fe89f0..8503359 100644 --- a/src/agent/history/session_storage.py +++ b/src/agent/history/session_storage.py @@ -1,512 +1,34 @@ -"""普通会话历史的存储后端抽象（MySQL）。 +"""会话历史存储转发层（统一实现在 agent.core.storage）。 -系统定位: - 为 web/ui_session、agent_utils、tool_call_recorder 提供会话消息与工具调用记录的 - 数据库读写入口。与 cron 共用 STORAGE_BACKEND 开关与 MYSQL_* 连接配置 - （见 agent.core.db）。 - -设计要点（不双写：sql 就是 sql，json 就是 json）: - - 选 mysql 时：对话消息与工具调用记录**只入库**（不写 turn_*.json / tool_*.json）， - 读取只走数据库；二进制产物文件（图片/附件等）仍按 json 方式落盘到 session 文件夹。 - - 选 json 时：本模块所有操作退化为 no-op，读写仍走文件（现状不变）。 - - 消息与工具调用记录**原样**入库（不做路径转换）：DB 行为与文件读取完全一致， - 下游 _serialize_content / _tool_calls_from_turn_record 无需感知数据来源。 - - 写入幂等：save_turn / save_tool_calls 对同一 (session_id, turn_id) 先 DELETE 再 INSERT。 - - sessions 表记录会话的 created_at/updated_at/last_turn_id，供列表排序与 get_last_turn_id。 - - DB 失败仅打印日志，不阻断主流程（与 cron persist_turn_records 一致）。 - -可扩展性: - - 未来如需跨机迁移，可在此处增加 relpath 转换（参考 cron/storage._to_rel_paths）。 +本模块保持既有导入路径（``from agent.history import session_storage``）， +实际实现与 cron 存储统一在 ``agent.core.storage``： + - 消费方调用本模块的便捷函数即可，无需感知后端选择（json / mysql）。 + - 新增存储后端只需在 ``agent.core.storage`` 实现 Protocol 并注册工厂。 """ -from __future__ import annotations - -import json -import sys -from datetime import datetime -from typing import Any, Protocol - -from agent.core.db import ensure_tables, get_storage_backend, mysql_conn - - -def _now_iso() -> str: - return datetime.now().strftime("%Y-%m-%dT%H:%M:%S") - - -# ---------- 会话三表 DDL（由 ensure_tables 幂等建表） ---------- -_SESSION_TABLE_DDLS = [ - """ - CREATE TABLE IF NOT EXISTS sessions ( - session_id VARCHAR(64) PRIMARY KEY, - created_at VARCHAR(32) NOT NULL, - updated_at VARCHAR(32) NOT NULL, - last_turn_id VARCHAR(64) DEFAULT '' - ) CHARACTER SET utf8mb4 - """, - """ - CREATE TABLE IF NOT EXISTS session_messages ( - id BIGINT AUTO_INCREMENT PRIMARY KEY, - session_id VARCHAR(64) NOT NULL, - turn_id VARCHAR(64) NOT NULL, - seq INT NOT NULL DEFAULT 0, - message JSON NOT NULL, - INDEX idx_sess_turn (session_id, turn_id) - ) CHARACTER SET utf8mb4 - """, - """ - CREATE TABLE IF NOT EXISTS session_tool_calls ( - id BIGINT AUTO_INCREMENT PRIMARY KEY, - session_id VARCHAR(64) NOT NULL, - turn_id VARCHAR(64) NOT NULL, - tool_calls JSON NOT NULL, - meta JSON DEFAULT NULL, - INDEX idx_sess_turn (session_id, turn_id) - ) CHARACTER SET utf8mb4 - """, -] - - -def is_mysql() -> bool: - """当前会话存储后端是否为 mysql。""" - return get_storage_backend() == "mysql" - - -# ---------- Protocol ---------- -class SessionRecordRepository(Protocol): - """会话消息与工具调用记录的读写接口（mysql 后端实现）。""" - - def save_turn(self, session_id: str, turn_id: str, messages: list[dict]) -> None: ... - def save_tool_calls( - self, session_id: str, turn_id: str, tool_calls: list[dict], meta: dict | None = None - ) -> None: ... - def load_messages(self, session_id: str) -> list[dict]: ... - def list_session_ids(self) -> list[str]: ... - def get_turn_record(self, session_id: str, turn_id: str) -> dict[str, Any] | None: ... - def get_latest_turn_record(self, session_id: str) -> dict[str, Any] | None: ... - def list_turn_records(self, session_id: str, limit: int) -> list[dict[str, Any]]: ... - def get_session_extra(self, session_id: str) -> dict[str, Any]: ... - def save_session_extra(self, session_id: str, extra: dict[str, Any]) -> None: ... - def get_last_turn_id(self, session_id: str) -> str: ... - def delete_turns_after(self, session_id: str, keep_until_turn_id: str) -> None: ... - def delete_session(self, session_id: str) -> None: ... - - -# ---------- MySQL 实现 ---------- -class MysqlSessionRecordRepository: - """基于 MySQL（pymysql）的会话记录仓库。 - - 表: sessions / session_messages / session_tool_calls。 - 消息与工具调用记录原样以 JSON 列存储；meta 列存 started_at/ended_at/duration 等。 - """ - - def __init__(self) -> None: - ensure_tables(_SESSION_TABLE_DDLS) - # 向后兼容：为已有 sessions 表补充 extra 列 - self._ensure_extra_column() - - def _ensure_extra_column(self) -> None: - """幂等添加 sessions.extra JSON 列（存储 tokens + agent_meta）。""" - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "ALTER TABLE sessions ADD COLUMN extra JSON DEFAULT NULL" - ) - conn.commit() - except Exception: - # 列已存在 → 忽略 - pass - finally: - conn.close() - - def save_turn(self, session_id: str, turn_id: str, messages: list[dict]) -> None: - if not session_id or not turn_id: - return - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "DELETE FROM session_messages WHERE session_id=%s AND turn_id=%s", - (session_id, turn_id), - ) - for seq, msg in enumerate(messages or []): - cur.execute( - "INSERT INTO session_messages (session_id, turn_id, seq, message) " - "VALUES (%s,%s,%s,%s)", - (session_id, turn_id, seq, json.dumps(msg, ensure_ascii=False)), - ) - now = _now_iso() - cur.execute( - """ - INSERT INTO sessions (session_id, created_at, updated_at, last_turn_id) - VALUES (%s,%s,%s,%s) - ON DUPLICATE KEY UPDATE updated_at=VALUES(updated_at), last_turn_id=VALUES(last_turn_id) - """, - (session_id, now, now, turn_id), - ) - conn.commit() - finally: - conn.close() - - def save_tool_calls( - self, session_id: str, turn_id: str, tool_calls: list[dict], meta: dict | None = None - ) -> None: - if not session_id or not turn_id: - return - meta = meta or {} - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "DELETE FROM session_tool_calls WHERE session_id=%s AND turn_id=%s", - (session_id, turn_id), - ) - cur.execute( - "INSERT INTO session_tool_calls (session_id, turn_id, tool_calls, meta) " - "VALUES (%s,%s,%s,%s)", - ( - session_id, - turn_id, - json.dumps(tool_calls or [], ensure_ascii=False), - json.dumps(meta, ensure_ascii=False) if meta else None, - ), - ) - conn.commit() - finally: - conn.close() - - def load_messages(self, session_id: str) -> list[dict]: - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT message FROM session_messages WHERE session_id=%s " - "ORDER BY turn_id ASC, seq ASC", - (session_id,), - ) - rows = cur.fetchall() or [] - return [self._loads(r["message"]) for r in rows] - finally: - conn.close() - - def list_session_ids(self) -> list[str]: - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute("SELECT session_id FROM sessions ORDER BY updated_at DESC") - rows = cur.fetchall() or [] - return [r["session_id"] for r in rows] - finally: - conn.close() - - def get_turn_record(self, session_id: str, turn_id: str) -> dict[str, Any] | None: - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT tool_calls, meta FROM session_tool_calls " - "WHERE session_id=%s AND turn_id=%s", - (session_id, turn_id), - ) - row = cur.fetchone() - if not row: - return None - return self._row_to_record(row, session_id, turn_id) - finally: - conn.close() - - def get_latest_turn_record(self, session_id: str) -> dict[str, Any] | None: - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT turn_id, tool_calls, meta FROM session_tool_calls " - "WHERE session_id=%s ORDER BY turn_id DESC LIMIT 1", - (session_id,), - ) - row = cur.fetchone() - if not row: - return None - return self._row_to_record(row, session_id, row["turn_id"]) - finally: - conn.close() - - def list_turn_records(self, session_id: str, limit: int) -> list[dict[str, Any]]: - """按 turn_id 倒序返回最近 limit 条 turn 记录（与 tool_*.json 同构）。""" - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT turn_id, tool_calls, meta FROM session_tool_calls " - "WHERE session_id=%s ORDER BY turn_id DESC LIMIT %s", - (session_id, limit), - ) - rows = cur.fetchall() or [] - return [self._row_to_record(r, session_id, r["turn_id"]) for r in rows] - finally: - conn.close() - - def get_session_extra(self, session_id: str) -> dict[str, Any]: - """读取 sessions.extra JSON 列（含 tokens + agent_meta），不存在时返回 {}。""" - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT extra FROM sessions WHERE session_id=%s", - (session_id,), - ) - row = cur.fetchone() - raw = (row or {}).get("extra") - return self._loads(raw) if raw is not None else {} - finally: - conn.close() - - def save_session_extra(self, session_id: str, extra: dict[str, Any]) -> None: - """写入 sessions.extra JSON 列；session 行不存在时先创建占位行。""" - conn = mysql_conn() - try: - with conn.cursor() as cur: - now = _now_iso() - cur.execute( - """ - INSERT INTO sessions (session_id, created_at, updated_at, extra) - VALUES (%s,%s,%s,%s) - ON DUPLICATE KEY UPDATE updated_at=%s, extra=VALUES(extra) - """, - (session_id, now, now, json.dumps(extra, ensure_ascii=False), now), - ) - conn.commit() - finally: - conn.close() - - def get_last_turn_id(self, session_id: str) -> str: - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "SELECT last_turn_id FROM sessions WHERE session_id=%s", - (session_id,), - ) - row = cur.fetchone() - if row and row.get("last_turn_id"): - return row["last_turn_id"] - # 回退：从消息表取最大 turn_id - cur.execute( - "SELECT MAX(turn_id) AS m FROM session_messages WHERE session_id=%s", - (session_id,), - ) - r2 = cur.fetchone() - return (r2 or {}).get("m") or "" - finally: - conn.close() - - def delete_turns_after(self, session_id: str, keep_until_turn_id: str) -> None: - if not session_id: - return - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute( - "DELETE FROM session_messages WHERE session_id=%s AND turn_id > %s", - (session_id, keep_until_turn_id), - ) - cur.execute( - "DELETE FROM session_tool_calls WHERE session_id=%s AND turn_id > %s", - (session_id, keep_until_turn_id), - ) - # 更新 last_turn_id 为剩余最大 turn_id - cur.execute( - "SELECT MAX(turn_id) AS m FROM session_messages WHERE session_id=%s", - (session_id,), - ) - r = cur.fetchone() - last = (r or {}).get("m") or "" - cur.execute( - "UPDATE sessions SET last_turn_id=%s, updated_at=%s WHERE session_id=%s", - (last, _now_iso(), session_id), - ) - conn.commit() - finally: - conn.close() - - def delete_session(self, session_id: str) -> None: - if not session_id: - return - conn = mysql_conn() - try: - with conn.cursor() as cur: - cur.execute("DELETE FROM session_messages WHERE session_id=%s", (session_id,)) - cur.execute("DELETE FROM session_tool_calls WHERE session_id=%s", (session_id,)) - cur.execute("DELETE FROM sessions WHERE session_id=%s", (session_id,)) - conn.commit() - finally: - conn.close() - - @staticmethod - def _loads(raw: Any) -> Any: - """pymysql 对 JSON 列可能返回 dict 或 str，统一反序列化。""" - if isinstance(raw, str): - try: - return json.loads(raw) - except (json.JSONDecodeError, TypeError): - return {} - return raw - - @classmethod - def _row_to_record(cls, row: dict[str, Any], session_id: str, turn_id: str) -> dict[str, Any]: - """把 session_tool_calls 行还原为与 tool_*.json 同构的 record dict。""" - tool_calls = cls._loads(row.get("tool_calls")) or [] - meta = cls._loads(row.get("meta")) if row.get("meta") is not None else {} - if not isinstance(meta, dict): - meta = {} - record: dict[str, Any] = { - "session_id": session_id, - "turn_id": turn_id, - "tool_calls": tool_calls, - "timestamp": turn_id, - } - # 合并 meta 中的 started_at/ended_at/duration/aborted 等 - for k in ("started_at", "ended_at", "duration", "aborted"): - if k in meta: - record[k] = meta[k] - return record - - -# ---------- 工厂 ---------- -_session_repo_cache: dict[str, Any] = {} - - -def get_session_record_repository() -> MysqlSessionRecordRepository: - """获取会话记录仓库实例（仅在 mysql 后端调用，带缓存）。""" - if "session_mysql" not in _session_repo_cache: - _session_repo_cache["session_mysql"] = MysqlSessionRecordRepository() - return _session_repo_cache["session_mysql"] # type: ignore[return-value] - - -# ---------- 模块级便捷封装（内部自检 backend，json 后端 no-op / 返回 None） ---------- -def persist_messages(session_id: str, turn_id: str, messages: list[dict]) -> None: - """mysql 后端下把一轮消息写入 DB；json 后端 no-op。""" - if not is_mysql() or not session_id or not turn_id: - return - try: - get_session_record_repository().save_turn(session_id, turn_id, messages) - except Exception as e: - print(f"[session_storage] persist_messages 失败: {e}", file=sys.stderr) - - -def persist_tool_calls( - session_id: str, turn_id: str, tool_calls: list[dict], meta: dict | None = None -) -> None: - """mysql 后端下把一轮工具调用记录写入 DB；json 后端 no-op。""" - if not is_mysql() or not session_id or not turn_id: - return - try: - get_session_record_repository().save_tool_calls(session_id, turn_id, tool_calls, meta) - except Exception as e: - print(f"[session_storage] persist_tool_calls 失败: {e}", file=sys.stderr) - - -def load_messages(session_id: str) -> list[dict] | None: - """mysql 后端返回消息列表；json 后端返回 None（调用方回退文件逻辑）。""" - if not is_mysql(): - return None - try: - return get_session_record_repository().load_messages(session_id) - except Exception as e: - print(f"[session_storage] load_messages 失败: {e}", file=sys.stderr) - return None - - -def list_session_ids() -> list[str] | None: - """mysql 后端返回会话 ID 列表（updated_at 倒序）；json 后端返回 None。""" - if not is_mysql(): - return None - try: - return get_session_record_repository().list_session_ids() - except Exception as e: - print(f"[session_storage] list_session_ids 失败: {e}", file=sys.stderr) - return None - - -def get_turn_record(session_id: str, turn_id: str) -> dict[str, Any] | None: - """mysql 后端返回 turn 记录（与 tool_*.json 同构）；json 后端返回 None。""" - if not is_mysql(): - return None - try: - return get_session_record_repository().get_turn_record(session_id, turn_id) - except Exception as e: - print(f"[session_storage] get_turn_record 失败: {e}", file=sys.stderr) - return None - - -def get_latest_turn_record(session_id: str) -> dict[str, Any] | None: - """mysql 后端返回最近 turn 记录；json 后端返回 None。""" - if not is_mysql(): - return None - try: - return get_session_record_repository().get_latest_turn_record(session_id) - except Exception as e: - print(f"[session_storage] get_latest_turn_record 失败: {e}", file=sys.stderr) - return None - - -def list_turn_records(session_id: str, limit: int = 5) -> list[dict[str, Any]] | None: - """mysql 后端返回最近 limit 条 turn 记录（turn_id 倒序）；json 后端返回 None。""" - if not is_mysql(): - return None - try: - return get_session_record_repository().list_turn_records(session_id, limit) - except Exception as e: - print(f"[session_storage] list_turn_records 失败: {e}", file=sys.stderr) - return None - - -def load_session_extra(session_id: str) -> dict[str, Any] | None: - """mysql 后端返回 sessions.extra（tokens + agent_meta）；json 后端返回 None。""" - if not is_mysql(): - return None - try: - return get_session_record_repository().get_session_extra(session_id) - except Exception as e: - print(f"[session_storage] load_session_extra 失败: {e}", file=sys.stderr) - return None - - -def save_session_extra(session_id: str, extra: dict[str, Any]) -> None: - """mysql 后端下写入 sessions.extra；json 后端 no-op。""" - if not is_mysql(): - return - try: - get_session_record_repository().save_session_extra(session_id, extra) - except Exception as e: - print(f"[session_storage] save_session_extra 失败: {e}", file=sys.stderr) - - -def get_last_turn_id(session_id: str) -> str | None: - """mysql 后端返回最后 turn_id；json 后端返回 None。""" - if not is_mysql(): - return None - try: - return get_session_record_repository().get_last_turn_id(session_id) - except Exception as e: - print(f"[session_storage] get_last_turn_id 失败: {e}", file=sys.stderr) - return None - - -def delete_turns_after(session_id: str, keep_until_turn_id: str) -> None: - """mysql 后端下删除 keep_until_turn_id 之后的 turn 记录；json 后端 no-op。""" - if not is_mysql(): - return - try: - get_session_record_repository().delete_turns_after(session_id, keep_until_turn_id) - except Exception as e: - print(f"[session_storage] delete_turns_after 失败: {e}", file=sys.stderr) - - -def delete_session(session_id: str) -> None: - """mysql 后端下删除会话全部记录；json 后端 no-op。""" - if not is_mysql(): - return - try: - get_session_record_repository().delete_session(session_id) - except Exception as e: - print(f"[session_storage] delete_session 失败: {e}", file=sys.stderr) +from agent.core.storage import ( + # Protocol 与实现 + ConversationRecordRepository, + JsonRecordRepository, + MysqlRecordRepository, + # 便捷函数（后端无关） + save_turn, + load_messages, + load_turn_messages, + list_turn_ids, + save_tool_calls, + get_turn_record, + get_latest_turn_record, + list_turn_records, + persist_live_records, + load_session_extra, + save_session_extra, + list_session_ids, + get_last_turn_id, + delete_turns_after, + delete_tool_records, + delete_session, + # 作用域与工厂 + set_storage_scope, + get_storage_scope, + get_record_repository, +) diff --git a/src/agent/history/tool_call_recorder.py b/src/agent/history/tool_call_recorder.py index fb59150..6e169d6 100644 --- a/src/agent/history/tool_call_recorder.py +++ b/src/agent/history/tool_call_recorder.py @@ -104,53 +104,20 @@ _session_token_breakdowns: dict[str, dict] = {} # session_id → {messages, too _session_tokens_loaded: set[str] = set() # 已从磁盘加载过的 session_id -def _token_file(session_id: str) -> Path: - """session 的 token 持久化文件路径。""" - return TOOL_CALLING_ROOT / session_id / "_session_tokens.json" - - def _persist_token(session_id: str, tokens: int, breakdown: dict | None = None) -> None: - """将 token 写入存储（优先 MySQL，回退 JSON 文件）。""" - # mysql 后端：写入 sessions.extra.tokens - try: - from agent.history import session_storage - if session_storage.is_mysql(): - existing = session_storage.load_session_extra(session_id) or {} - existing["tokens"] = tokens - if breakdown: - existing["token_breakdown"] = breakdown - session_storage.save_session_extra(session_id, existing) - return - except Exception: - pass - # json 后端：写入文件 - tf = _token_file(session_id) - tf.parent.mkdir(parents=True, exist_ok=True) - payload = {"session_id": session_id, "tokens": tokens} + """将 token 写入存储（后端无关：mysql 存 sessions.extra，json 写 _session_tokens.json）。""" + from agent.history import session_storage + existing = session_storage.load_session_extra(session_id) or {} + existing["tokens"] = tokens if breakdown: - payload["breakdown"] = breakdown - tf.write_text(json.dumps(payload), encoding="utf-8") + existing["token_breakdown"] = breakdown + session_storage.save_session_extra(session_id, existing) def _load_token_from_disk(session_id: str) -> int: - """从磁盘/数据库读取 token（若不存在返回 0）。""" - # mysql 后端：从 sessions.extra 读取 - try: - from agent.history import session_storage - _db_extra = session_storage.load_session_extra(session_id) - if _db_extra is not None: - return int(_db_extra.get("tokens", 0)) - except Exception: - pass - # json 后端：从文件读取 - tf = _token_file(session_id) - if not tf.exists(): - return 0 - try: - data = json.loads(tf.read_text(encoding="utf-8")) - return int(data.get("tokens", 0)) - except Exception: - return 0 + """从存储读取 token（若不存在返回 0）。""" + from agent.history import session_storage + return int(session_storage.load_session_extra(session_id).get("tokens", 0) or 0) def set_session_tokens(session_id: str, total_tokens: int, breakdown: dict | None = None) -> None: @@ -185,29 +152,16 @@ def get_session_tokens(session_id: str) -> int: def get_session_token_breakdown(session_id: str) -> dict: """获取当前会话的三类 token 估算占比（消息/工具/系统提示词）。 - 内存优先，回退磁盘（JSON 文件的 breakdown / MySQL 的 token_breakdown）。 + 内存优先，回退存储（JSON 文件的 breakdown / MySQL 的 token_breakdown）。 """ with _session_tokens_lock: if session_id in _session_token_breakdowns: return dict(_session_token_breakdowns[session_id]) - try: - from agent.history import session_storage - if session_storage.is_mysql(): - extra = session_storage.load_session_extra(session_id) or {} - bd = extra.get("token_breakdown") - if isinstance(bd, dict): - return bd - except Exception: - pass - tf = _token_file(session_id) - if tf.exists(): - try: - data = json.loads(tf.read_text(encoding="utf-8")) - bd = data.get("breakdown") - if isinstance(bd, dict): - return bd - except Exception: - pass + from agent.history import session_storage + extra = session_storage.load_session_extra(session_id) or {} + bd = extra.get("token_breakdown") + if isinstance(bd, dict): + return bd return {"messages": 0.0, "tools": 0.0, "system": 0.0} @@ -216,21 +170,11 @@ def clear_session_tokens(session_id: str) -> None: with _session_tokens_lock: _session_tokens.pop(session_id, None) _session_token_breakdowns.pop(session_id, None) - # mysql 后端：设置 tokens=0 - try: - from agent.history import session_storage - if session_storage.is_mysql(): - existing = session_storage.load_session_extra(session_id) or {} - existing["tokens"] = 0 - existing.pop("token_breakdown", None) - session_storage.save_session_extra(session_id, existing) - return - except Exception: - pass - # json 后端：删除文件 - tf = _token_file(session_id) - if tf.exists(): - tf.unlink(missing_ok=True) + from agent.history import session_storage + existing = session_storage.load_session_extra(session_id) or {} + existing["tokens"] = 0 + existing.pop("token_breakdown", None) + session_storage.save_session_extra(session_id, existing) # ---------- 待附加的 thinking ---------- # 当 record_reflection_live 被调用时，thinking 暂存于此； @@ -376,33 +320,20 @@ def record_tool_result_live( def _safe_persist_live_records(session_id: str, records: list[dict[str, Any]]) -> None: - """将实时记录增量写入磁盘（仅 json 后端），防止进程崩溃丢失数据。 + """将实时记录增量写入存储，防止进程崩溃丢失数据。 系统定位: - mysql 后端不写 tool_*.json 文件，由 save_turn 统一入库；此处直接跳过。 + 委托统一存储的 persist_live_records（可选行为：json 后端写 tool_*.json， + mysql 后端不增量入库，由 save_turn 在轮次结束时统一落库）。 """ - try: - from agent.history import session_storage - if session_storage.is_mysql(): - return # mysql 后端：记录只入库，不写增量文件 - except Exception: - pass try: turn_id = get_current_turn() if not turn_id: return - session_dir = session_tool_dir(session_id) - _ensure_dir(session_dir) - record = { - "session_id": session_id, - "turn_id": turn_id, - "tool_calls": records, - "timestamp": turn_id, - "started_at": _turn_start_times.get(session_id), - } - json_path = session_dir / f"tool_{turn_id}.json" - with open(json_path, "w", encoding="utf-8") as f: - json.dump(record, f, ensure_ascii=False, indent=2) + from agent.history import session_storage + session_storage.persist_live_records( + session_id, turn_id, records, started_at=_turn_start_times.get(session_id) + ) except Exception: pass # 持久化失败不应影响主流程 @@ -571,12 +502,23 @@ def unsubscribe_live(session_id: str, q: _queue_module.Queue) -> None: def _build_live_snapshot(session_id: str) -> dict[str, Any]: - """构建当前实时状态的快照（不加锁，调用者需确保数据一致性或在锁外读取）。""" - return { + """构建当前实时状态的快照（不加锁，调用者需确保数据一致性或在锁外读取）。 + + 除工具调用/思考/轮次开始时间外，还附带当前等待中的人工请求 + （human_requests），前端据此渲染确认浮窗。 + """ + snapshot = { "tool_calls": get_live_tool_calls(session_id), "reflections": get_live_reflections(session_id), "started_at": get_turn_started_at(session_id), } + try: + # 惰性导入避免与 human_request(其模块级导入本模块) 循环依赖 + from agent.core.human_request import get_human_requests_for_session + snapshot["human_requests"] = get_human_requests_for_session(session_id) + except Exception: + snapshot["human_requests"] = [] + return snapshot def _notify_live(session_id: str) -> None: @@ -624,44 +566,15 @@ def save_aborted_turn(session_id: str, turn_id: str) -> None: ended_at = time.time() # 优先使用 _turn_start_times（轮次开始时刻），回退到 ended_at（确保 duration 不为负） started_at = _turn_start_times.get(session_id) or ended_at - record = { - "session_id": session_id, - "turn_id": turn_id, - "user_message": {}, - "tool_calls": normalized, - "assistant_response": "", - "timestamp": turn_id, + # 工具调用记录统一走存储（后端无关：mysql 只入库，json 只写文件，不双写） + from agent.history import session_storage + _meta = { "started_at": started_at, "ended_at": ended_at, "duration": max(0, ended_at - started_at), "aborted": True, } - # 使用 tool_ 前缀（与 save_turn / _safe_persist_live_records 一致）， - # 确保 get_tool_calls_for_turn 能正确读取 - json_path = session_dir / f"tool_{turn_id}.json" - # 判定后端：mysql 只入库，json 只写文件（不双写） - try: - from agent.history import session_storage - _is_mysql = session_storage.is_mysql() - except Exception: - _is_mysql = False - _meta = { - "started_at": started_at, - "ended_at": ended_at, - "duration": record.get("duration"), - "aborted": True, - } - if _is_mysql: - # mysql 后端：工具调用记录只入库 - try: - session_storage.persist_tool_calls(session_id, turn_id, normalized, meta=_meta) - except Exception as _e: - import sys as _sys - print(f"[tool_call_recorder] save_aborted_turn DB 写入失败: {_e}", file=_sys.stderr) - else: - # json 后端：工具调用记录只写文件 - with open(json_path, "w", encoding="utf-8") as f: - json.dump(record, f, ensure_ascii=False, indent=2) + session_storage.save_tool_calls(session_id, turn_id, normalized, meta=_meta) def get_live_tool_calls(session_id: str) -> list[dict[str, Any]]: @@ -944,30 +857,16 @@ def save_turn(session_id: str, turn_id: str, user_message: dict[str, Any], for fname, data_bytes in all_bytes_map.items(): (session_dir / fname).write_bytes(data_bytes) - # 判定后端：mysql 只入库，json 只写文件（不双写） - try: - from agent.history import session_storage - _is_mysql = session_storage.is_mysql() - except Exception: - _is_mysql = False - - # 读取已有的 live 记录（含正确的 thinking/args_summary） - json_path = session_dir / f"tool_{turn_id}.json" + # 读取已有的 live 记录（含正确的 thinking/args_summary）： + # 统一从存储读取（json 后端为实时增量写盘的文件，mysql 后端无增量记录 → None） started_at = _turn_start_times.pop(session_id, None) + from agent.history import session_storage + live_record = session_storage.get_turn_record(session_id, turn_id) live_calls: list[dict[str, Any]] = [] - if _is_mysql: - # mysql 不写增量文件，直接从内存 live 记录读取用于合并（保留准确时间戳/thinking） - with _live_lock: - live_calls = list(_live_records.get(session_id, [])) - elif json_path.is_file(): - try: - with open(json_path, "r", encoding="utf-8") as f: - existing = json.load(f) - live_calls = existing.get("tool_calls") or [] - if existing.get("started_at"): - started_at = existing["started_at"] - except Exception: - live_calls = [] + if live_record: + live_calls = live_record.get("tool_calls") or [] + if live_record.get("started_at"): + started_at = live_record["started_at"] # 合并：以 extracted 为基础，从 live 记录中补充正确的 thinking/args_summary tool_calls = _merge_extracted_with_live(extracted, live_calls) @@ -976,32 +875,14 @@ def save_turn(session_id: str, turn_id: str, user_message: dict[str, Any], _inject_sub_traces_to_record(tool_calls, session_id, turn_id) ended_at = time.time() - record = { - "session_id": session_id, - "turn_id": turn_id, - "tool_calls": tool_calls, - "timestamp": turn_id, + # 工具调用记录统一走存储（后端无关：mysql 只入库，json 只写文件，不双写） + _meta = { "started_at": started_at, "ended_at": ended_at, "duration": (ended_at - started_at) if started_at else None, } - if _is_mysql: - # mysql 后端：工具调用记录只入库 - try: - _meta = { - "started_at": started_at, - "ended_at": ended_at, - "duration": record.get("duration"), - } - session_storage.persist_tool_calls(session_id, turn_id, tool_calls, meta=_meta) - except Exception as _e: - import sys as _sys - print(f"[tool_call_recorder] save_turn DB 写入失败: {_e}", file=_sys.stderr) - else: - # json 后端：工具调用记录只写文件 - with open(json_path, "w", encoding="utf-8") as f: - json.dump(record, f, ensure_ascii=False, indent=2) - return str(json_path) + session_storage.save_tool_calls(session_id, turn_id, tool_calls, meta=_meta) + return None def _merge_extracted_with_live( @@ -1062,52 +943,19 @@ def _inject_sub_traces_to_record(tool_calls: list[dict[str, Any]], session_id: s def list_turns(session_id: str, limit: int = 5) -> list[dict[str, Any]]: """列出会话下所有 turn（按时间倒序，默认最近5轮）。 - + 输入: session_id: 会话 ID。 limit: 返回的最大轮数，默认 5。 """ - # mysql 后端：从数据库读取 - try: - from agent.history import session_storage - _db_records = session_storage.list_turn_records(session_id, limit) - if _db_records is not None: - return _db_records - except Exception as _e: - import sys as _sys - print(f"[tool_call_recorder] list_turns DB 读取失败: {_e}", file=_sys.stderr) - # json 后端：从文件读取 - session_dir = session_tool_dir(session_id) - if not session_dir.is_dir(): - return [] - turns = [] - for f in sorted(session_dir.glob("tool_*.json"), reverse=True): - try: - with open(f, "r", encoding="utf-8") as fh: - turns.append(json.load(fh)) - except (json.JSONDecodeError, OSError): - pass - return turns[:limit] + from agent.history import session_storage + return session_storage.list_turn_records(session_id, limit) def get_latest_turn(session_id: str) -> dict[str, Any] | None: - """获取最近一个 turn 的完整记录。 - - 系统定位: - mysql 后端从数据库读取；json 后端从文件读取（list_turns[0]）。 - """ - # mysql 后端：从数据库读取 - try: - from agent.history import session_storage - _db_rec = session_storage.get_latest_turn_record(session_id) - if _db_rec is not None: - return _db_rec - except Exception as _e: - import sys as _sys - print(f"[tool_call_recorder] get_latest_turn DB 读取失败: {_e}", file=_sys.stderr) - # json 后端：从文件读取 - turns = list_turns(session_id) - return turns[0] if turns else None + """获取最近一个 turn 的完整记录（后端无关）。""" + from agent.history import session_storage + return session_storage.get_latest_turn_record(session_id) def get_tool_calls_for_latest_turn(session_id: str) -> list[dict[str, Any]]: @@ -1119,32 +967,16 @@ def get_tool_calls_for_latest_turn(session_id: str) -> list[dict[str, Any]]: def _load_turn_record(session_id: str, turn_id: str) -> dict[str, Any] | None: - """读取指定 turn_id 的工具调用记录（mysql 走 DB，json 走文件）。 + """读取指定 turn_id 的工具调用记录（后端无关）。 系统定位: get_tool_calls_for_turn / get_turn_meta / get_thinking_for_turn 共用的 - 后端感知读取入口。返回与 tool_*.json 同构的 record dict。 + 读取入口。返回与 tool_*.json 同构的 record dict；不存在返回 None。 """ - # mysql 后端：从数据库读取（无行返回 None，不回退文件——旧会话未迁移） - try: - from agent.history import session_storage - _db_rec = session_storage.get_turn_record(session_id, turn_id) - if _db_rec is not None: - return _db_rec - except Exception as _e: - import sys as _sys - print(f"[tool_call_recorder] _load_turn_record DB 读取失败: {_e}", file=_sys.stderr) - # json 后端：从文件读取 if not turn_id: return None - json_path = session_tool_dir(session_id) / f"tool_{turn_id}.json" - if not json_path.is_file(): - return None - try: - with open(json_path, "r", encoding="utf-8") as f: - return json.load(f) - except (json.JSONDecodeError, OSError): - return None + from agent.history import session_storage + return session_storage.get_turn_record(session_id, turn_id) def get_tool_calls_for_turn(session_id: str, turn_id: str) -> list[dict[str, Any]]: diff --git a/src/agent/skills/__init__.py b/src/agent/skills/__init__.py index 6ae13bd..066449a 100644 --- a/src/agent/skills/__init__.py +++ b/src/agent/skills/__init__.py @@ -422,8 +422,10 @@ def parse_skill_zip(zip_bytes: bytes) -> dict | None: except KeyError: return None meta, body = _parse_frontmatter(md_text) - # 附件：一级子目录下除 skill.md/skill.json 之外的其余文件（不深入展开子目录内部） + # 附件：一级目录下除 skill.md/skill.json 之外的树（不深入展开， + # 子文件夹内的文件折叠为 "子文件夹/" 一级展示，与文件夹导入行为一致） attachments = [] + seen_first = set() for n in zf.namelist(): if n.endswith("/") or n == md_entry: continue @@ -431,13 +433,19 @@ def parse_skill_zip(zip_bytes: bytes) -> dict | None: if top_dir: if parts[0] != top_dir: continue - rel = "/".join(parts[1:]) + rel_parts = parts[1:] else: - rel = n + rel_parts = parts + if not rel_parts: + continue + rel = "/".join(rel_parts) if rel.lower() in ("skill.md", "skill.json"): continue - attachments.append(rel) - attachments.sort() + if len(rel_parts) == 1: + seen_first.add(rel) + else: + seen_first.add(rel_parts[0] + "/") + attachments = sorted(seen_first) return { "name": meta.get("name", ""), "description": meta.get("description", ""), diff --git a/src/agent/tools/agent_call.py b/src/agent/tools/agent_call.py index 4446cfe..c39d4a0 100644 --- a/src/agent/tools/agent_call.py +++ b/src/agent/tools/agent_call.py @@ -5,6 +5,9 @@ 调用子 Agent 执行子任务。支持一轮中发起多个 call_subagent 调用， 结合并行 ToolNode 实现子 Agent 并行执行。 + 子 Agent 内的人机交互为阻塞式普通工具：子图在线程中阻塞等待前端应答， + 完成后自然回到主图，无需 pending 冒泡/续跑机制。 + 工具签名: call_subagent(agent_name, task_description) -> str @@ -25,22 +28,6 @@ from agent.core.state import AgentState from agent.utils.tool_call_utils import invoke_tool_and_build_message -class SubAgentPendingError(Exception): - """子 Agent 执行中遇到 HITL/确认工具时抛出，冒泡到主图捕获。 - - 携带 pending_meta（含 approval_id）与 agent_name； - main_messages / main_tool_call_id 由 parallel_tool_node 捕获时填充， - 供主图在用户确认/跳过后恢复执行。 - """ - - def __init__(self, pending_meta: dict, agent_name: str = ""): - super().__init__(f"sub-agent '{agent_name}' pending human action") - self.pending_meta = pending_meta - self.agent_name = agent_name - self.main_messages = None # 由 parallel_tool_node 填充：主图消息快照 - self.main_tool_call_id = None # 由 parallel_tool_node 填充：call_subagent 的 tool_call_id - - class CallSubAgentInput(BaseModel): """call_subagent 工具的输入参数。""" agent_name: str = Field( @@ -163,8 +150,8 @@ class CallSubAgentTool(BaseTool): config = {"recursion_limit": runtime.max_iterations} - # 设置子 Agent 上下文：TodoList 按 agent 隔离时使用子 Agent 名称 - from agent.tools.todo_list import set_current_agent_name + # 设置子 Agent 上下文：agent 名称供前端浮窗按 agent 分组 + from agent.core.human_request import set_current_agent_name set_current_agent_name(agent_name) # 进入子 Agent 执行上下文：抑制其 record_*_live 调用， # 避免子 Agent 的思考/工具调用混入主 Agent 的实时列表。 @@ -178,17 +165,6 @@ class CallSubAgentTool(BaseTool): # 提取子 Agent 工具调用轨迹 → 存档供前端嵌套展开 _record_sub_trace(agent_name, all_messages) - # ===== 子 Agent HITL 冒泡检测（Part 2）===== - # 子图在 confirm/human_interaction 工具处会 END；检测是否产生 pending， - # 若有则向 PENDING_TOOL_APPROVALS 打 agent_name 标记并抛 SubAgentPendingError， - # 由主图捕获后弹窗（前端按 agent_name 分组显示）。 - from agent.history.tool_call_recorder import get_current_session - sub_session_id = get_current_session() or "" - sub_pending_meta = _detect_sub_pending(all_messages, sub_session_id, runtime, agent_name) - if sub_pending_meta is not None: - # 抛出前还原主 Agent 上下文（finally 会再次还原，此处冗余但安全） - raise SubAgentPendingError(sub_pending_meta, agent_name) - # 提取最后一条非工具调用的助手消息作为最终结果 from langchain_core.messages import AIMessage final_text = "" @@ -211,8 +187,6 @@ class CallSubAgentTool(BaseTool): return f"[子Agent: {agent_name}] 执行结果:\n{final_text}" - except SubAgentPendingError: - raise except Exception as e: return f"[子Agent: {agent_name}] 执行出错: {str(e)}" finally: @@ -226,34 +200,6 @@ class CallSubAgentTool(BaseTool): # ---------- 子 Agent 轨迹记录 ---------- -def _detect_sub_pending(all_messages: list, session_id: str, runtime: Any, agent_name: str) -> dict | None: - """检测子 Agent 图是否在 confirm/human_interaction 处暂停。 - - 若是，向 PENDING_TOOL_APPROVALS 对应条目打 agent_name/is_sub_agent 标记， - 并附加 sub_runtime/sub_messages（原始 BaseMessage 列表）供主图恢复时使用。 - 返回 pending_meta 或 None。 - """ - try: - from agent.core.routing import first_pending_tool_meta - from agent.core.approvals import PENDING_TOOL_APPROVALS - meta = first_pending_tool_meta(all_messages, session_id) - if not meta: - return None - pending_action = meta.get("pending_action") or {} - approval_id = pending_action.get("id") - if approval_id and approval_id in PENDING_TOOL_APPROVALS: - entry = PENDING_TOOL_APPROVALS[approval_id] - entry["agent_name"] = agent_name - entry["is_sub_agent"] = True - entry["sub_runtime"] = runtime - entry["sub_messages"] = list(all_messages) # 原始 BaseMessage 列表，供恢复 - return meta - except SubAgentPendingError: - raise - except Exception: - return None - - def _record_sub_trace(agent_name: str, messages: list) -> None: """记录子 Agent 的工具调用轨迹，供前端嵌套展开。 @@ -304,7 +250,6 @@ def make_parallel_tool_node(tools: list[Any], max_workers: int = 5): 委托 invoke_tool_and_build_message 统一处理返回类型。 返回 (tool_name, content, artifact)，artifact 为 None 表示无产物。 - SubAgentPendingError 不被吞掉，向上冒泡到 parallel_tool_node。 """ tool = tools_by_name.get(tool_name) if tool is None: @@ -312,8 +257,6 @@ def make_parallel_tool_node(tools: list[Any], max_workers: int = 5): try: content, artifact = invoke_tool_and_build_message(tool, tool_name, tool_args, tool_call_id) return tool_name, content, artifact - except SubAgentPendingError: - raise # 子 Agent HITL 冒泡，主图需暂停 except Exception as e: return tool_name, f"工具执行出错: {e}", None @@ -326,8 +269,6 @@ def make_parallel_tool_node(tools: list[Any], max_workers: int = 5): return {} tool_calls = last_msg.tool_calls - # 主图消息快照：若子 Agent 触发 HITL 冒泡，供主图恢复时使用 - main_messages_snapshot = list(state["messages"]) # 只有 1 个工具调用时直接串行执行，避免线程开销 if len(tool_calls) == 1: @@ -335,12 +276,7 @@ def make_parallel_tool_node(tools: list[Any], max_workers: int = 5): tc_name = tc.get("name") if isinstance(tc, dict) else getattr(tc, "name", "") tc_args = tc.get("args") if isinstance(tc, dict) else getattr(tc, "args", {}) tc_id = tc.get("id") if isinstance(tc, dict) else getattr(tc, "id", "") - try: - name, content, artifact = _execute_single(tc_name, tc_args, tc_id) - except SubAgentPendingError as e: - e.main_messages = main_messages_snapshot - e.main_tool_call_id = tc_id - raise + name, content, artifact = _execute_single(tc_name, tc_args, tc_id) msg_kwargs = {"content": str(content), "tool_call_id": tc_id, "name": name} if artifact: msg_kwargs["artifact"] = artifact @@ -369,10 +305,8 @@ def make_parallel_tool_node(tools: list[Any], max_workers: int = 5): if meta["id"] == tc_id: try: _, content, artifact = future.result() - except SubAgentPendingError as e: - e.main_messages = main_messages_snapshot - e.main_tool_call_id = tc_id - raise + except Exception: + content, artifact = f"工具执行出错: {tc_name}", None msg_kwargs = {"content": str(content), "tool_call_id": tc_id, "name": tc_name} if artifact: msg_kwargs["artifact"] = artifact @@ -381,13 +315,8 @@ def make_parallel_tool_node(tools: list[Any], max_workers: int = 5): break if not matched: # 降级：尝试单次执行 - try: - name, content, artifact = _execute_single(tc_name, {}, tc_id) - except SubAgentPendingError as e: - e.main_messages = main_messages_snapshot - e.main_tool_call_id = tc_id - raise - msg_kwargs = {"content": str(content), "tool_call_id": tc_id, "name": tc_name} + name, content, artifact = _execute_single(tc_name, {}, tc_id) + msg_kwargs = {"content": str(content), "tool_call_id": tc_id, "name": name} if artifact: msg_kwargs["artifact"] = artifact tool_messages.append(ToolMessage(**msg_kwargs)) diff --git a/src/agent/tools/human_interaction.py b/src/agent/tools/human_interaction.py index 9b74dcd..8036f0c 100644 --- a/src/agent/tools/human_interaction.py +++ b/src/agent/tools/human_interaction.py @@ -28,8 +28,48 @@ class HumanInteractionInput(BaseModel): ) +def normalize_human_interaction_args(args: dict) -> dict: + """规范化 human_interaction 工具参数为固定 schema。 + + 输入: + args: 模型传入的原始参数字典。 + + 输出: + 含 interaction_type、title、prompt、options、suggested_response、questions 的字典； + interaction_type 限定为 information | selection。 + """ + normalized = dict(args or {}) + normalized["interaction_type"] = str(normalized.get("interaction_type") or "information") + if normalized["interaction_type"] not in {"information", "selection"}: + normalized["interaction_type"] = "information" + normalized["title"] = str(normalized.get("title") or "需要人工处理") + normalized["prompt"] = str(normalized.get("prompt") or "请处理后继续。") + normalized["options"] = HumanInteraction.normalize_options(normalized.get("options")) + normalized["suggested_response"] = str(normalized.get("suggested_response") or "") + # 规范化 questions 字段 + questions = normalized.get("questions") + if isinstance(questions, str): + try: + questions = json.loads(questions) + except (json.JSONDecodeError, TypeError): + questions = None + if isinstance(questions, list): + normalized_questions = [] + for q in questions: + if isinstance(q, dict): + nq = { + "question": str(q.get("question", "")), + "options": HumanInteraction.normalize_options(q.get("options")), + } + normalized_questions.append(nq) + normalized["questions"] = normalized_questions if normalized_questions else None + else: + normalized["questions"] = None + return normalized + + class HumanInteraction(BaseTool): - """向人类请求补充信息或选择；实际 UI 确认由图级人工干预层承接。""" + """向人类请求补充信息或选择；调用时阻塞等待前端浮窗应答，答案作为普通工具返回值。""" args_schema: type[BaseModel] = HumanInteractionInput name: str = "human_interaction" @@ -62,9 +102,6 @@ class HumanInteraction(BaseTool): def format_result(interaction_type: str, options: list[str] | None = None, response: str = "") -> str: """根据交互类型与用户决策生成结果文本。 - 供 ``_run``（工具直接执行时的默认结果）与 ``approvals.human_interaction_result`` - （用户确认后的真实结果）共用，消除重复的分支逻辑。 - 输入: interaction_type: information | selection。 options: selection 的可选项（用于默认值）。 @@ -88,8 +125,45 @@ class HumanInteraction(BaseTool): pass selected = text or (options[0] if options else "") return f"用户选择：{selected}" if selected else "用户选择：" + if text.startswith("用户拒绝") or text.startswith("用户已跳过"): + return text return f"用户补充信息如下：{text}" if text else "用户没有补充信息" + @staticmethod + def build_request_meta(args: dict) -> dict: + """将规范化后的工具参数构造成前端浮窗渲染数据（与旧 pending_action 形状一致）。""" + args = normalize_human_interaction_args(args) + interaction_type = args["interaction_type"] + default_titles = { + "information": "请补充信息", + "selection": "请选择一个选项", + } + agent_name = "主Agent" + try: + from agent.core.human_request import get_current_agent_name + _ag = get_current_agent_name() + agent_name = "主Agent" if _ag in ("", "__main__") else _ag + except Exception: + pass + return { + "type": "human_interaction", + "interaction_type": interaction_type, + "title": args["title"] or default_titles.get(interaction_type, "需要人工处理"), + "prompt": args["prompt"] or "请处理后继续。", + "tool_name": "human_interaction", + "args": { + "交互类型": interaction_type, + "说明": args["prompt"] or "请处理后继续。", + "可选项": args["options"] or [], + }, + "instruction": args["suggested_response"] or "", + "options": args["options"] or [], + "questions": args["questions"] or None, + "agent_name": agent_name, + "is_sub_agent": agent_name != "主Agent", + "resolved": False, + } + def _run( self, interaction_type: str = "information", @@ -97,10 +171,28 @@ class HumanInteraction(BaseTool): prompt: str = "请处理后继续。", options: Optional[list[str] | str] = None, suggested_response: Optional[str] = None, + questions: Optional[list[dict]] = None, ) -> str: try: - normalized_options = self.normalize_options(options) - return self.format_result(interaction_type, normalized_options, suggested_response or "") + args = { + "interaction_type": interaction_type, + "title": title, + "prompt": prompt, + "options": options, + "suggested_response": suggested_response, + "questions": questions, + } + normalized = normalize_human_interaction_args(args) + from agent.core.human_request import ask_human + from agent.history.tool_call_recorder import get_current_session + session_id = get_current_session() or "" + answer = ask_human(session_id, self.build_request_meta(normalized)) + # 拒绝/跳过：原样返回用户决策文本，让 Agent 自行调整，不结束整轮 + if answer.get("decision") in ("reject", "skip"): + return f"用户{('拒绝' if answer['decision'] == 'reject' else '已跳过')}此次交互：{answer.get('instruction') or ''}".strip() + return self.format_result(interaction_type, normalized.get("options") or [], answer.get("instruction") or "") + except RuntimeError: + raise # ABORTED_BY_USER 冒泡到 runtime 统一按暂停处理 except Exception as e: return f"[ERROR] human_interaction 工具执行出错: {str(e)}" @@ -111,5 +203,6 @@ class HumanInteraction(BaseTool): prompt: str = "请处理后继续。", options: Optional[list[str] | str] = None, suggested_response: Optional[str] = None, + questions: Optional[list[dict]] = None, ) -> str: - return await asyncio.to_thread(self._run, interaction_type, title, prompt, options, suggested_response) + return await asyncio.to_thread(self._run, interaction_type, title, prompt, options, suggested_response, questions) diff --git a/src/agent/tools/skill_router.py b/src/agent/tools/skill_router.py index b073690..4f82616 100644 --- a/src/agent/tools/skill_router.py +++ b/src/agent/tools/skill_router.py @@ -90,9 +90,11 @@ class SkillRouter(BaseTool): meta = metas.get(name, {}) content = contents[name] # 将 {SKILL_DIR} 替换为实际 skills 目录路径 - content = content.replace("{SKILL_DIR}", skill_dir_str + "/" + name) + skill_path = skill_dir_str + "/" + name + content = content.replace("{SKILL_DIR}", skill_path) parts.append( f"## Skill: {name}\n" + f"Skill 文件夹: {skill_path}\n" f"描述: {meta.get('description', '')}\n\n" f"{content}\n" ) diff --git a/src/agent/tools/todo_list.py b/src/agent/tools/todo_list.py index 1632d1a..1efc850 100644 --- a/src/agent/tools/todo_list.py +++ b/src/agent/tools/todo_list.py @@ -1,123 +1,60 @@ from __future__ import annotations import json -import threading -from contextvars import ContextVar -from typing import Any, Optional, Literal +from typing import Any, Literal, Optional from langchain.tools import BaseTool from pydantic import BaseModel, Field, model_validator -# ---------- 当前 agent 上下文（用于 TodoList 按 agent 隔离）---------- -# 主 Agent 使用保留键 "__main__"；子 Agent 使用其 agent_name。 -# ContextVar + 全局 fallback 双轨：主线程/线程池子进程均可读到正确值； -# HTTP 请求线程（前端轮询）无上下文 → fallback 默认 "__main__" → 只见主 Agent。 -_current_agent_name: ContextVar[str] = ContextVar("current_agent_name", default="") -_agent_name_fallback_lock = threading.Lock() -_current_agent_name_fallback = "__main__" +def _iter_todo_records(records: list[dict[str, Any]] | None): + """遍历 live 工具记录中的 todo_list 调用（含子 Agent 嵌套在 call_subagent 内的记录）。""" + for rec in records or []: + if rec.get("name") == "todo_list": + yield rec + for sub in rec.get("sub_tool_calls") or []: + if sub.get("name") == "todo_list": + yield sub -def set_current_agent_name(agent_name: str) -> None: - """设置当前线程的 agent 名称（主 Agent 传 "__main__"，子 Agent 传其名称）。""" - global _current_agent_name_fallback - _current_agent_name.set(agent_name or "") - with _agent_name_fallback_lock: - _current_agent_name_fallback = agent_name or "__main__" +def _reconstruct_state(sid: str, steps_arg: Optional[list[str]], done_step_arg: int) -> tuple[list[str], set[int]]: + """从当前轮的 live 工具记录重建累计 todo 状态。 - -def get_current_agent_name() -> str: - """获取当前 agent 名称；ContextVar 为空时回退到全局 fallback（默认 "__main__"）。""" - name = _current_agent_name.get() - if name: - return name - with _agent_name_fallback_lock: - return _current_agent_name_fallback or "__main__" - - -# ---------- Todo List 实时推送共享存储 ---------- -# TodoList 工具本身和 web server 共享此 dict，无需跨模块 import server -# 二级键：外层 session_id，内层 agent_name（主 Agent 用 "__main__"） -_todo_store: dict[str, dict[str, dict]] = {} - - -def _resolve_agent(agent_name: str | None) -> str: - return agent_name or get_current_agent_name() or "__main__" - - -def get_todo_store_data(session_id: str, agent_name: str | None = None) -> dict | None: - """读取指定 session + agent 的 todo。 - - 前端轮询时不传 agent_name → 走 get_current_agent_name() → HTTP 线程回退到 "__main__"， - 因此前端只见主 Agent 的 todo；子 Agent 的 todo 隔离在内层键中不暴露。 + steps 取自最近一次传入完整步骤列表的调用；done_steps 累积全部 done_step。 + 本工具是纯函数、不维护任何全局 store——每次调用都从框架的 live 记录通道 + 读取自身调用历史，前端浮窗亦由同一通道（工具调用的 args）驱动更新。 """ - if not session_id: - return None - ag = _resolve_agent(agent_name) - return (_todo_store.get(session_id) or {}).get(ag) - - -def update_todo_store_session( - session_id: str, - steps: list[str] | None = None, - done_step: int = -1, - agent_name: str | None = None, -) -> dict | None: - if not session_id: - return None - ag = _resolve_agent(agent_name) - sess = _todo_store.setdefault(session_id, {}) - current = sess.get(ag) or {"steps": [], "done_steps": []} - stored_steps = list(current.get("steps") or []) - done_steps = list(current.get("done_steps") or []) - - if steps: - stored_steps = list(steps) - done_steps = [] - - if stored_steps and 1 <= done_step <= len(stored_steps): - for i in range(done_step): - if i not in done_steps: - done_steps.append(i) - done_steps.sort() - - todo_data = {"steps": stored_steps, "done_steps": done_steps} - sess[ag] = todo_data - return todo_data - - -def update_todo_store_from_tool_args(session_id: str, tool_args: dict[str, Any], agent_name: str | None = None) -> dict | None: - raw_steps = tool_args.get("steps") if isinstance(tool_args, dict) else None - if isinstance(raw_steps, str): + steps: list[str] = [] + done_steps: set[int] = set() + if sid: try: - raw_steps = json.loads(raw_steps) - except (json.JSONDecodeError, ValueError): - raw_steps = None - steps = raw_steps if isinstance(raw_steps, list) else None - - done_step = tool_args.get("done_step", -1) if isinstance(tool_args, dict) else -1 - try: - done_step = int(done_step) - except (TypeError, ValueError): - done_step = -1 - - return update_todo_store_session(session_id, steps, done_step, agent_name=agent_name) - - -def set_todo_store_data(session_id: str, todo_data: dict, agent_name: str | None = None) -> None: - if not session_id: - return - ag = _resolve_agent(agent_name) - sess = _todo_store.setdefault(session_id, {}) - sess[ag] = { - "steps": list(todo_data.get("steps") or []), - "done_steps": list(todo_data.get("done_steps") or []), - } - - -def clear_todo_store_session(session_id: str) -> None: - """清除指定 session 下所有 agent 的 todo（主 + 各子 Agent）。""" - _todo_store.pop(session_id, None) + from agent.history.tool_call_recorder import get_live_tool_calls + for rec in _iter_todo_records(get_live_tool_calls(sid)): + args = rec.get("args") or {} + raw_steps = args.get("steps") + if isinstance(raw_steps, str): + try: + raw_steps = json.loads(raw_steps) + except (json.JSONDecodeError, ValueError): + raw_steps = None + if isinstance(raw_steps, list) and raw_steps: + steps = [str(s) for s in raw_steps] + done_steps = set() + try: + n = int(args.get("done_step", -1)) + except (TypeError, ValueError): + n = -1 + if steps and n >= 1: + done_steps.update(range(min(n, len(steps)))) + except Exception: + pass + # 本次调用的参数覆盖/追加（live 记录不可用时的兜底） + if steps_arg: + steps = [str(s) for s in steps_arg] + done_steps = set() + if steps and done_step_arg >= 1: + done_steps.update(range(min(done_step_arg, len(steps)))) + return steps, done_steps class TodoListInput(BaseModel): @@ -149,6 +86,9 @@ class TodoList(BaseTool): Plan 模式下此工具为**必须调用**，Agent 模式下为可选。 首次调用时传入完整步骤列表，后续仅传 done_step 标记进度。 + + 调用时会随 live 工具记录推送状态，前端据此实时渲染 Todo 浮窗； + 返回值是纯文本，不携带任何结构化前缀。 """ args_schema: type[BaseModel] = TodoListInput @@ -159,15 +99,15 @@ class TodoList(BaseTool): "参数说明：\n" "- steps (list[str], 可选)：任务步骤列表。仅在首次创建计划时传入所有步骤。\n" " **步骤粒度要求**：同一页面/界面上的连续操作应合并为一条步骤，用\"并\"连接。例如：\n" - ' * 推荐：\"点击搜索框，输入XXX并点击搜索\"（合并了点击、输入、搜索）\n' - ' * 推荐：\"在结果中点击目标进入主页\"（合并了查找+点击）\n' - ' * 禁止：\"1.点击搜索框 2.输入文字 3.点击搜索\"（过于细碎）\n' + ' * 推荐："点击搜索框，输入XXX并点击搜索"（合并了点击、输入、搜索）\n' + ' * 推荐："在结果中点击目标进入主页"（合并了查找+点击）\n' + ' * 禁止："1.点击搜索框 2.输入文字 3.点击搜索"（过于细碎）\n' "- done_step (int, 可选)：当前已推进到的步骤编号（从1开始）。传入 n 则步骤 1~n 全部标记为已完成。不传或传 -1 则只查看当前状态。\n" "调用示例：\n" - '- 首次创建（B站搜索示例）：{\"steps\": [\"打开哔哩哔哩网站\", \"点击顶部搜索框，输入\'燕三嘤嘤嘤\'并点击搜索\", \"在搜索结果中找到燕三嘤嘤嘤并点击头像或名称进入他的主页\", \"在他的主页点开视频列表里的第一个视频\", \"验证视频是否正在播放\"], \"done_step\": -1}\n' - '- 已完成前2步：{\"done_step\": 2} （步骤1和2均标记为完成）\n' - '- 已完成前4步：{\"done_step\": 4}\n' - '- 查看状态：{\"done_step\": -1}\n' + '- 首次创建（B站搜索示例）：{"steps": ["打开哔哩哔哩网站", "点击顶部搜索框，输入\'燕三嘤嘤嘤\'并点击搜索", "在搜索结果中找到燕三嘤嘤嘤并点击头像或名称进入他的主页", "在他的主页点开视频列表里的第一个视频", "验证视频是否正在播放"], "done_step": -1}\n' + '- 已完成前2步：{"done_step": 2} （步骤1和2均标记为完成）\n' + '- 已完成前4步：{"done_step": 4}\n' + '- 查看状态：{"done_step": -1}\n' "任务全部完成后无需再次调用。" ) response_format: Literal["content", "content_and_artifact"] = "content" @@ -175,40 +115,20 @@ class TodoList(BaseTool): def _run(self, steps: list[str] | None = None, done_step: int = -1) -> str: try: steps = steps or [] - - # 从全局 store 读取当前会话 + 当前 agent 的状态（由 runtime 层在调用前设置 session/agent） - from agent.history.tool_call_recorder import get_current_session - sid = get_current_session() or "" - current = get_todo_store_data(sid) or {"steps": [], "done_steps": []} - stored_steps = list(current.get("steps") or []) - done_steps = list(current.get("done_steps") or []) - - # 如果传入了新的 steps，更新存储 - if steps: - stored_steps = steps - done_steps = [] - - # 标记步骤完成 - if stored_steps and 1 <= done_step <= len(stored_steps): - for i in range(done_step): - if i not in done_steps: - done_steps.append(i) - done_steps.sort() - - todo_data = {"steps": stored_steps, "done_steps": done_steps} - - # 实时推送到前端 todo 浮窗 - if sid: - set_todo_store_data(sid, todo_data) - + sid = "" + try: + from agent.history.tool_call_recorder import get_current_session + sid = get_current_session() or "" + except Exception: + pass + stored_steps, done_steps = _reconstruct_state(sid, steps, done_step) if not stored_steps: - return json.dumps({"__todo_list__": True, "data": todo_data}, ensure_ascii=False) + "\n[Todo List]\n(当前无任务)" - + return "[Todo List]\n(当前无任务)" lines = ["[Todo List]"] for i, s in enumerate(stored_steps): status = "[Done]" if i in done_steps else "[Pending]" lines.append(f"- [{i+1}/{len(stored_steps)}] {status}: {s}") - return json.dumps({"__todo_list__": True, "data": todo_data}, ensure_ascii=False) + "\n" + "\n".join(lines) + return "\n".join(lines) except Exception as e: return f"[ERROR] todo_list 工具执行出错: {str(e)}" diff --git a/src/agent/utils/agent_utils.py b/src/agent/utils/agent_utils.py index 1746c6a..e655a7f 100644 --- a/src/agent/utils/agent_utils.py +++ b/src/agent/utils/agent_utils.py @@ -147,48 +147,18 @@ def get_history(session_id: str) -> JsonChatMessageHistory: # ---------- session_meta 元数据管理 ---------- -def _session_meta_path(session_id: str) -> str: - return os.path.join(SESSIONS_ROOT, session_id, "session_meta.json") - - def _load_session_meta(session_id: str) -> dict: - """加载整个 session_meta 数据（优先 MySQL，回退 JSON 文件）。""" - # mysql 后端：从 sessions.extra.agent_meta 读取 - try: - from agent.history import session_storage - _db_extra = session_storage.load_session_extra(session_id) - if _db_extra is not None: - return _db_extra.get("agent_meta", {}) - except Exception: - pass - # json 后端：从文件读取 - mp = _session_meta_path(session_id) - if not os.path.isfile(mp): - return {} - try: - with open(mp, "r", encoding="utf-8") as f: - return json.load(f) - except (json.JSONDecodeError, OSError): - return {} + """加载整个 session_meta 数据（后端无关：mysql 存 sessions.extra.agent_meta，json 存文件）。""" + from agent.history import session_storage + return session_storage.load_session_extra(session_id).get("agent_meta", {}) def _save_session_meta(session_id: str, meta: dict) -> None: - """保存整个 session_meta 数据（优先 MySQL，回退 JSON 文件）。""" - # mysql 后端：写入 sessions.extra.agent_meta（不覆盖 tokens） - try: - from agent.history import session_storage - if session_storage.is_mysql(): - existing = session_storage.load_session_extra(session_id) or {} - existing["agent_meta"] = meta - session_storage.save_session_extra(session_id, existing) - return - except Exception: - pass - # json 后端：写入文件 - mp = _session_meta_path(session_id) - os.makedirs(os.path.dirname(mp), exist_ok=True) - with open(mp, "w", encoding="utf-8") as f: - json.dump(meta, f, ensure_ascii=False, indent=2) + """保存整个 session_meta 数据（后端无关，不覆盖 extra 中的 tokens）。""" + from agent.history import session_storage + existing = session_storage.load_session_extra(session_id) or {} + existing["agent_meta"] = meta + session_storage.save_session_extra(session_id, existing) def get_session_meta(session_id: str, agent_name: str) -> dict: @@ -264,137 +234,50 @@ import threading as _threading TURN_JSON_PREFIX = "turn_" -def _turn_json_path(session_id: str, turn_id: str) -> str: - """返回指定 turn 的对话 JSON 文件路径。""" - return os.path.join(SESSIONS_ROOT, session_id, f"{TURN_JSON_PREFIX}{turn_id}.json") - - def turn_files_dir(session_id: str, turn_id: str) -> str: """返回指定 turn 的附件目录路径。""" return os.path.join(SESSIONS_ROOT, session_id, turn_id, "files") def save_turn_messages(session_id: str, turn_id: str, messages: list[dict]) -> None: - """将一轮对话的消息持久化（mysql 后端只写 DB，json 后端只写文件）。 + """将一轮对话的消息持久化（后端无关：mysql 只入库，json 只写文件，不双写）。 系统定位: 消息持久化的唯一入口（sync_turn_from_chat_history 调用）。 - 不双写：sql 就是 sql，json 就是 json。附件二进制文件始终落盘（由调用方处理）， - 本函数仅负责对话消息记录的持久化。 + 附件二进制文件始终落盘（由调用方处理），本函数仅负责对话消息记录的持久化。 """ - # mysql 后端：只写数据库 - try: - from agent.history import session_storage - if session_storage.is_mysql(): - session_storage.persist_messages(session_id, turn_id, messages) - return - except Exception as _e: - import sys as _sys - print(f"[agent_utils] save_turn_messages 后端判定/DB 写入失败: {_e}", file=_sys.stderr) - return - # json 后端：只写文件 - sess_dir = os.path.join(SESSIONS_ROOT, session_id) - os.makedirs(sess_dir, exist_ok=True) - tp = _turn_json_path(session_id, turn_id) - with open(tp, "w", encoding="utf-8") as f: - json.dump({"turn_id": turn_id, "messages": messages}, f, ensure_ascii=False, indent=2) + from agent.history import session_storage + session_storage.save_turn(session_id, turn_id, messages) def load_all_turn_messages(session_id: str) -> list[dict]: - """按时间顺序加载所有 turn 的消息，返回扁平列表。 - - 系统定位: - mysql 后端从数据库读取（旧文件会话未迁移，返回空）；json 后端从文件读取。 - """ - # mysql 后端：从数据库读取 - try: - from agent.history import session_storage - _db_msgs = session_storage.load_messages(session_id) - if _db_msgs is not None: - return _db_msgs - except Exception as _e: - import sys as _sys - print(f"[agent_utils] load_all_turn_messages DB 读取失败: {_e}", file=_sys.stderr) - # json 后端：从文件读取 - sess_dir = os.path.join(SESSIONS_ROOT, session_id) - if not os.path.isdir(sess_dir): - return [] - all_msgs: list[dict] = [] - turn_files = sorted( - [f for f in os.listdir(sess_dir) if f.startswith(TURN_JSON_PREFIX) and f.endswith(".json")] - ) - for tf in turn_files: - tp = os.path.join(sess_dir, tf) - if os.path.isfile(tp): - try: - with open(tp, "r", encoding="utf-8") as f: - data = json.load(f) - all_msgs.extend(data.get("messages", [])) - except (json.JSONDecodeError, OSError): - pass - return all_msgs + """按时间顺序加载所有 turn 的消息，返回扁平列表（后端无关）。""" + from agent.history import session_storage + return session_storage.load_messages(session_id) def delete_turns_after(session_id: str, keep_until_turn_id: str) -> int: - """删除指定 turn_id 之后（按文件名排序）的所有 turn JSON 文件及对应目录。返回删除数量。 + """删除指定 turn_id 之后（按文件名排序）的所有 turn 记录并清理对应 artifact 目录。 - 注意：keep_until_turn_id 自身的 turn 文件会被保留（只删除严格大于它的）。 + 记录删除（消息 + 工具调用）由统一存储处理后端无关；artifact 目录（以 turn_id + 命名的目录，两个后端均落盘）在此清理。返回删除的 turn 数。 + + 注意：keep_until_turn_id 自身的 turn 记录会被保留（只删除严格大于它的）。 """ sess_dir = os.path.join(SESSIONS_ROOT, session_id) - if not os.path.isdir(sess_dir): - return 0 - deleted = 0 - # 必须带上 ".json" 后缀进行比较，否则 keep_until_turn_id 自身的文件 - # (如 "turn_20260727_113953.json") 会因字符串扩展而 > "turn_20260727_113953"， - # 导致保留点 turn 被误删。 - keep_full = f"{TURN_JSON_PREFIX}{keep_until_turn_id}.json" - for fname in sorted(os.listdir(sess_dir)): - if fname.startswith(TURN_JSON_PREFIX) and fname.endswith(".json"): - if fname > keep_full: - tp = os.path.join(sess_dir, fname) - if os.path.isfile(tp): - os.remove(tp) - deleted += 1 - # 同时删除对应的 turn 目录（artifact 目录） - for dname in sorted(os.listdir(sess_dir)): - dp = os.path.join(sess_dir, dname) - if os.path.isdir(dp) and dname > keep_until_turn_id: - shutil.rmtree(dp, ignore_errors=True) - # mysql 后端：同步删除数据库中 keep_until_turn_id 之后的记录 - try: - from agent.history import session_storage - session_storage.delete_turns_after(session_id, keep_until_turn_id) - except Exception as _e: - import sys as _sys - print(f"[agent_utils] delete_turns_after DB 同步失败: {_e}", file=_sys.stderr) - return deleted + if os.path.isdir(sess_dir): + for dname in sorted(os.listdir(sess_dir)): + dp = os.path.join(sess_dir, dname) + if os.path.isdir(dp) and dname > keep_until_turn_id: + shutil.rmtree(dp, ignore_errors=True) + from agent.history import session_storage + return session_storage.delete_turns_after(session_id, keep_until_turn_id) def get_last_turn_id(session_id: str) -> str | None: - """获取最后一个 turn_id。 - - 系统定位: - mysql 后端从 sessions.last_turn_id 读取；json 后端从文件名读取。 - """ - # mysql 后端：从数据库读取 - try: - from agent.history import session_storage - _db_tid = session_storage.get_last_turn_id(session_id) - if _db_tid is not None: - return _db_tid or None - except Exception as _e: - import sys as _sys - print(f"[agent_utils] get_last_turn_id DB 读取失败: {_e}", file=_sys.stderr) - # json 后端：从文件名读取 - sess_dir = os.path.join(SESSIONS_ROOT, session_id) - if not os.path.isdir(sess_dir): - return None - turn_files = sorted( - [f for f in os.listdir(sess_dir) if f.startswith(TURN_JSON_PREFIX) and f.endswith(".json")] - ) - if not turn_files: - return None - return turn_files[-1][len(TURN_JSON_PREFIX):-5] # strip "turn_" prefix and ".json" suffix + """获取最后一个 turn_id（后端无关）；无记录返回 None。""" + from agent.history import session_storage + return session_storage.get_last_turn_id(session_id) or None def session_dir(session_id: str) -> str: diff --git a/src/agent/utils/tool_call_utils.py b/src/agent/utils/tool_call_utils.py index c830443..cd0ccd4 100644 --- a/src/agent/utils/tool_call_utils.py +++ b/src/agent/utils/tool_call_utils.py @@ -1,8 +1,8 @@ """工具调用规范化、工具执行与 JSON 安全转换。 系统定位: - 位于 Agent 工具链与人工确认（HITL）之间的适配层，被 ``core/routing``、 - ``core/runtime``、``core/approvals``、``tools/agent_call`` 调用， + 位于 Agent 工具链与工具执行之间的适配层，被 ``core/runtime``、 + ``tools/agent_call`` 调用， 负责：归一化工具调用对象、安全执行工具并构造 ToolMessage、JSON 安全转换。 可扩展性: @@ -21,7 +21,7 @@ def _invoke_with_timeout(tool, tool_call_input: dict, timeout: float) -> Any: """在 daemon 线程中执行 tool.invoke，超时返回 None。 超时后工作线程仍在后台运行（Python 无法强杀线程），结果被丢弃。 - 工具内部抛出的异常会在主线程 re-raise（保留 SubAgentPendingError 冒泡）。 + 工具内部抛出的异常会在主线程 re-raise（保留 ABORTED_BY_USER 冒泡）。 输入: tool: LangChain BaseTool 实例。 @@ -36,7 +36,7 @@ def _invoke_with_timeout(tool, tool_call_input: dict, timeout: float) -> Any: def _worker(): try: container["result"] = tool.invoke(tool_call_input) - except BaseException as e: # noqa: BLE001 - 需透传 SubAgentPendingError 等所有异常 + except BaseException as e: # noqa: BLE001 - 需透传 ABORTED_BY_USER 等所有异常 container["error"] = e t = threading.Thread(target=_worker, daemon=True) @@ -68,7 +68,10 @@ def invoke_tool_and_build_message(tool, tool_name: str, tool_args: dict, tool_ca 系统定位: 由 ``core/runtime`` 和 ``tools/agent_call`` 在需要手动 invoke 工具时调用。 同时是工具执行前的唯一钳点：工作空间访问策略（限制访问 / 无人值守审批 - 退化）在此拦截越界操作，覆盖图内执行、人工确认恢复循环与审批后重放路径。 + 退化）在此拦截越界操作；权限为 confirm 的工具与审批模式下的越界操作 + 在此阻塞征求用户意见（拒绝/跳过也仅是返回值，图继续执行）。 + human_interaction 为阻塞工具，直连调用绕过超时包装，其内部 + ask_human 的 ABORTED_BY_USER 直接冒泡到 runtime。 """ from langchain_core.messages import ToolMessage @@ -81,6 +84,55 @@ def invoke_tool_and_build_message(tool, tool_name: str, tool_args: dict, tool_ca except Exception: pass # 策略模块异常不应阻断工具执行 + # 人工确认钳点：confirm 权限工具 / 审批模式越界操作，先阻塞征求用户同意 + if tool_name != "human_interaction": + try: + from agent.core.tool_policy import classify_tool_execution + from agent.core.workspace_policy import check_violation as _policy_check_violation + from agent.core.workspace_policy import decision as _policy_decision + from agent.core.human_request import ask_human, get_current_agent_name + from agent.history.tool_call_recorder import get_current_session + + _need_ask = classify_tool_execution(tool_name, tool_args) == "confirm" + _policy_note = "" + if _policy_decision(tool_name, tool_args) == "approve": + _need_ask = True + _policy_note = _policy_check_violation(tool_name, tool_args) or "检测到超出工作空间范围的操作" + + if _need_ask: + _agent = get_current_agent_name() + _agent_name = "主Agent" if _agent in ("", "__main__") else _agent + _safe_args = json_safe(tool_args) + _meta = { + "type": "tool_call", + "title": f"待确认执行工具：{tool_name}", + "tool_name": tool_name, + "args": {"name": tool_name, "args": _safe_args}, + "instruction": json.dumps({"name": tool_name, "args": _safe_args}, ensure_ascii=False, indent=2), + "options": ["approve", "reject", "skip"], + "policy_note": _policy_note, + "agent_name": _agent_name, + "is_sub_agent": _agent_name != "主Agent", + "resolved": False, + } + _sid = get_current_session() or "" + _answer = ask_human(_sid, _meta) + _decision = _answer.get("decision") + if _decision in ("reject", "skip"): + _verb = "拒绝" if _decision == "reject" else "已跳过" + _hint = (_answer.get("instruction") or "").strip() + _text = f"用户{_verb}此工具调用（{tool_name}）" + if _hint: + _text += f"：{_hint}" + return _text, None + if _decision == "timeout": + return f"等待用户确认超时，未执行工具（{tool_name}）。", None + # approve（或 edit）：继续执行 + except RuntimeError: + raise # ABORTED_BY_USER 冒泡 + except Exception: + pass # 确认通道异常不应阻断工具执行 + tool_call_input = { "name": tool_name, "args": tool_args, @@ -94,7 +146,12 @@ def invoke_tool_and_build_message(tool, tool_name: str, tool_args: dict, tool_ca timeout = get_tool_timeout(tool_name) or DEFAULT_TOOL_TIMEOUT except Exception: timeout = 300.0 - result = _invoke_with_timeout(tool, tool_call_input, timeout) + if tool_name == "human_interaction": + # 人机交互是阻塞工具：等待可能很长，绕过超时包装直连调用， + # 其内部 ask_human 的 ABORTED_BY_USER 直接冒泡到 runtime 按暂停处理 + result = tool.invoke(tool_call_input) + else: + result = _invoke_with_timeout(tool, tool_call_input, timeout) if result is None: return f"工具执行超时（{int(timeout)}s）", None if isinstance(result, ToolMessage): @@ -122,7 +179,7 @@ def json_safe(value: Any) -> Any: JSON 兼容的 dict / list / str / 基本类型；结构与原值对应。 系统定位: - 供 ``core/approvals.build_approval_meta`` 在展示待确认工具参数时使用， + 供 ``core/runtime`` 与 ``tools/agent_call`` 在展示待确认工具参数时使用， 避免 UI 层因不可序列化对象而报错。 可扩展性: diff --git a/src/web/app.js b/src/web/app.js index 7a5f94e..040d625 100644 --- a/src/web/app.js +++ b/src/web/app.js @@ -12,18 +12,15 @@ import { setRenderAttachmentChips as audioSetRenderChips } from './js/audio.js'; // ---- 聊天渲染 ---- import { renderMessages, renderMessage, renderAttachmentChips, scrollChatToBottom, - appendThinkingIndicator, showPendingActionInThinking, streamAssistantMessage, + appendThinkingIndicator, streamAssistantMessage, renderAudioBubble, addTextQuote, initChatQuoteMenu, } from './js/chat-render.js'; import { setRenderPersistentToolCalls } from './js/chat-render.js'; // ---- 工具调用 / think ---- -import { injectLiveToolCalls } from './js/toolcalls.js'; +import { injectLiveToolCalls, setLiveUiDeps } from './js/toolcalls.js'; import { setRenderAudioBubble as toolcallsSetAudioBubble } from './js/toolcalls.js'; -// ---- 人工确认 ---- -import { setRenderFnDeps as pendingSetDeps } from './js/pending.js'; - // ---- 人工确认浮窗 ---- import { updatePendingOverlay, clearPendingOverlay } from './js/pending-overlay.js'; @@ -36,7 +33,7 @@ import { sendChat, handlePauseClick } from './js/send.js'; import { setSendDeps } from './js/send.js'; // ---- Todo ---- -import { updateTodoOverlay, clearTodoOverlay } from './js/todo.js'; +import { updateTodoOverlayFromRecords, clearTodoOverlay } from './js/todo.js'; // ---- 布局 / 背景 ---- import { @@ -93,14 +90,10 @@ import { renderPersistentToolCalls } from './js/toolcalls.js'; setRenderPersistentToolCalls(renderPersistentToolCalls); toolcallsSetAudioBubble(renderAudioBubble); -// pending 需要 sessions/chat 的函数 -pendingSetDeps({ - renderSessions, - appendThinkingIndicator, - renderMessages, - streamAssistantMessage, - injectLiveToolCalls, - scrollChatToBottom, +// live snapshot 驱动人工确认浮窗与 Todo 浮窗（toolcalls 收到快照后回调） +setLiveUiDeps({ + updatePendingOverlay, + updateTodoOverlayFromRecords, }); // sessions 需要 renderMessages, renderAttachmentChips, clearTodoOverlay @@ -117,9 +110,7 @@ setSendDeps({ renderAttachmentChips, appendThinkingIndicator, injectLiveToolCalls, - updateTodoOverlay, clearTodoOverlay, - showPendingActionInThinking, streamAssistantMessage, onSendStart: () => setComposerCentered(false), }); diff --git a/src/web/css/settings.css b/src/web/css/settings.css index 81d3356..25944b4 100644 --- a/src/web/css/settings.css +++ b/src/web/css/settings.css @@ -990,7 +990,28 @@ background: var(--divider); } -/* ===== Skill 导入菜单 ===== */ +/* ===== Skill 导入按钮与菜单 ===== */ +.settings-toolbar-buttons { + display: flex; + align-items: center; + gap: 0.5rem; +} + +/* 导入按钮的定位锚点：下拉菜单相对该包裹元素定位 */ +.skill-import-wrap { + position: relative; + display: inline-flex; +} + +/* 导入按钮与"创建 Skill"同风格（btn-primary），仅配色略有不同 */ +.import-skill-btn { + background: linear-gradient(165deg, #64748b, #475569); + box-shadow: 0 4px 14px rgba(71, 85, 105, 0.35); +} +.import-skill-btn:hover:not(:disabled) { + background: linear-gradient(165deg, #94a3b8, #64748b); +} + .skill-import-menu { position: absolute; z-index: 90; @@ -1020,6 +1041,3 @@ .skill-import-menu button:hover { background: color-mix(in srgb, var(--accent) 12%, transparent); } -.settings-toolbar { - position: relative; -} diff --git a/src/web/index.html b/src/web/index.html index 4457c93..6b677f1 100644 --- a/src/web/index.html +++ b/src/web/index.html @@ -389,8 +389,12 @@