优化加载后删除节点的逻辑;和数据库查询性能

This commit is contained in:
strepsiades
2026-04-02 19:31:04 +08:00
parent 74df53889c
commit 543c3ff906
17 changed files with 571 additions and 32 deletions
+9
View File
@@ -37,6 +37,10 @@
| `ABNORMAL` | 数据损坏或核心绑定异常 | 否 |
| `DEPENDENCY_ERROR` | 依赖不可满足 | 否 |
bootstrap 特殊语义:
- 如果节点在持久化恢复后存在绑定,但 bootstrap datasource refresh 未返回最新数据,则节点进入 `S17` 语义并被 `bootstrap_blocked` 隔离。
- 如果节点仅残留在 persistence 中、没有绑定,且 datasource refresh 也未返回,则节点按 stale zombie 清理,不再进入 bind。
## 3. create / update 语义
### create
@@ -64,6 +68,11 @@
- 状态落地:`Decision.to_state` 对应 YAML `states[Sxx]`
- 不变量:每次落地后执行 `runtime.check_invariants()`
bootstrap 额外输入:
- `is_create_zombie`
- `bootstrap_source_present`
- `has_binding_record`
## 6. 关键一致性约束
- `UNCHECKED` 不允许带动作(`INV-1_UNCHECKED_ACTION_NONE`)。
+3
View File
@@ -21,6 +21,7 @@
- `S12C/S12U/S12D`:按动作拆分的执行失败状态
- `S13`:对侧创建等待态(`PEER_CREATING/NONE/PENDING`
- `S14`UPDATE 运行时降级为 `NONE` 并跳过
- `S17`:bootstrap 加载异常隔离(存在绑定,但 datasource refresh 未返回最新节点)
## 3. 事件到代码映射
@@ -51,6 +52,7 @@
## 5. context 字段(以 YAML 为准)
### bind 相关
- `bootstrap_source_present`
- `has_binding_record`
- `core_binding_result` / `peer_core_binding_result` (`NORMAL|WARNING|ABNORMAL`)
- `data_present`
@@ -75,6 +77,7 @@
## 6. 语义边界
- strategy 负责业务语义计算(如依赖是否满足、自动绑定结果、差异检测)。
- 状态机只负责迁移判定与不变量校验,不直接访问数据源。
- bootstrap 阶段的数据源刷新由 pipeline 完成,状态机只消费 `bootstrap_source_present / has_binding_record / is_create_zombie` 这些已计算上下文。
## 7. 不变量
- `INV-1_UNCHECKED_ACTION_NONE`
+20 -7
View File
@@ -13,23 +13,34 @@
### 2.1 加载时恢复
-`sync_state_machine/common/collection.py::load_from_persistence()` 执行。
- 语义:仅恢复持久化状态不在此阶段做状态归并
- 语义:仅恢复持久化状态,并记录本轮 `sync_log``persistence` 恢复来源;不在此阶段做状态归并。
- 恢复后保留 `data/origin_data`,等待 bootstrap 阶段的 datasource 刷新覆盖最新数据。
### 2.2 加载后 cleanup
- pipeline 直接调用 `run_phase2_cleanup()`
### 2.2 bootstrap datasource refresh
- pipeline 在持久化恢复后、`E01` 之前,按已恢复到内存的 `node_type` 做一次 datasource 刷新
- 实际实现:`sync_state_machine/pipeline/full_sync_pipeline.py::_bootstrap_refresh_latest_data()`
- 语义:
- 如果 datasource 命中节点,则用最新数据覆盖持久化快照,并清除 `_loaded_from_persistence` 标记。
- 如果 datasource 未命中节点,则节点保留持久化快照,供 `E01` 判断它是 stale zombie 还是 bootstrap 异常隔离对象。
### 2.3 加载后 cleanup
- pipeline 在 datasource refresh 完成后调用 `run_phase2_cleanup()`
- 实际实现:`sync_state_machine/sync_system/strategy_ops/reset_ops.py::run_phase2_cleanup()`
- 核心事件:`E01``S16 -> S00/S15`)。
- `S16` 表示“持久化加载输入态”:本轮从持久化恢复出的节点集合(可包含上轮遗留节点,也可包含上轮已稳定节点)
- `S16` 表示“bootstrap 输入态”:持久化恢复出的节点,叠加 datasource refresh 后仍待 `E01` 判定的节点集合
- 判定:
- `is_create_zombie=true` -> `S15`,节点删除(并尝试解绑);
- `is_create_zombie=false` -> `S00`节点归并为 `UNCHECKED/NONE/PENDING` 并清空 `error`
- `bootstrap_source_present=false && has_binding_record=false` -> `S15`说明该节点仅残留在 persistence 中且未绑定,按 stale zombie 清理;
- `bootstrap_source_present=false && has_binding_record=true` -> `S17`,说明该节点存在绑定,但 datasource refresh 未返回,进入 bootstrap 加载异常隔离;
- 其余情况 -> `S00`,节点归并为 `UNCHECKED/NONE/PENDING` 并清空 `error`
### 2.2.1 `enable_persistence` 与 bootstrap 关系
- `enable_persistence=true`:先 `load_from_persistence()`,再对加载出的节点执行 `E01``S16` 域内处理)。
- `enable_persistence=true`:先 `load_from_persistence()`,再做 bootstrap datasource refresh,最后对输入节点执行 `E01``S16` 域内处理)。
- `enable_persistence=false`:本轮没有持久化输入,`S16/E01` 不参与;随后 Stage1 新加载节点直接进入本轮 bind/create/update/sync 主流程。
### 2.3 加载异常隔离(S17
### 2.4 加载异常隔离(S17
- 持久化中若出现非法枚举值(`action/status/binding_status`),节点标记为 `bootstrap_blocked``S17` 语义)。
- 持久化恢复后若节点存在绑定,但 bootstrap datasource refresh 未返回该节点,也会进入 `S17` 语义并隔离。
- 这类节点保留 `error`,用于人工排查。
- 这类节点不会进入 `E01`,也不会参与 bind/create/update/sync 自动流程。
@@ -51,6 +62,8 @@
- 有绑定记录:根据本端/对端有效性进入 `S01/S03/S04`
- 无绑定记录:进入 `S02`
注意:如果节点在 bootstrap 阶段已被隔离为 `S17` 语义(`bootstrap_blocked=true`),默认不会进入 bind 工作集。
### 依赖检查
- `phase_dependency_check()` -> `check_dependencies_for_nodes()` 内触发 `E15`
- `data_present=true``dep_satisfied=true` -> 不迁移,节点保持在 `S02`,继续进入自动绑定判定。
+2
View File
@@ -488,6 +488,7 @@ class DataCollection:
context=context_value,
)
node.context["_loaded_from_persistence"] = True
node.append_log("加载来源: persistence 恢复节点")
except Exception:
import logging
logger = logging.getLogger(__name__)
@@ -623,6 +624,7 @@ class DataCollection:
context=context_value,
)
node.context["_loaded_from_persistence"] = True
node.append_log("加载来源: persistence 恢复节点")
self._nodes[node.node_id] = node
if node.data_id and node.data_id != "":
@@ -112,6 +112,9 @@ class SQLitePersistenceBackend(PersistenceBackend):
await self.conn.execute(
"CREATE INDEX IF NOT EXISTS idx_nodes_collection_scope_data_id ON nodes (collection_id, scope, data_id)"
)
await self.conn.execute(
"CREATE INDEX IF NOT EXISTS idx_nodes_scope_type_node_id ON nodes (scope, node_type, node_id)"
)
async def _ensure_bindings_schema(self) -> None:
assert self.conn is not None
@@ -17,6 +17,7 @@ meta:
# represented as separate invocations with different inputs.
context:
is_create_zombie: { type: bool, desc: "初始化时判定为 CREATE 僵尸:action=CREATE 且 (status=FAILED 或 data_id 为空)" }
bootstrap_source_present: { type: bool, desc: "bootstrap 阶段 datasource 刷新后,当前节点是否在最新数据源中命中" }
has_binding_record: { type: bool, desc: "binding_manager 里存在绑定记录 (remote_id/local_id 非空)" }
core_binding_result: { type: enum, values: [NORMAL, WARNING, ABNORMAL], desc: "SyncDecisionEngine.determine_core_status() 给本节点的结果(仅在 has_binding_record=true 时有效)" }
@@ -304,19 +305,30 @@ states:
status: SKIPPED
data_id_exist: true
S17:
binding_status: ABNORMAL
action: NONE
status: FAILED
data_id_exist: [false, true]
desc: |
bootstrap 加载异常隔离态
持久化恢复后存在绑定,但 datasource 刷新未命中最新数据
binding_status: ABNORMAL
action: NONE
status: FAILED
data_id_exist: false | true
pseudo_states:
S15:
desc: "节点被清理删除(zombie cleanup"
S16:
desc: "持久化加载输入态:本轮 E01 的输入节点集合(可包含上轮遗留与已稳定节点)"
S17:
desc: "加载异常隔离态:持久化枚举非法,保留错误供人工处理,不进入自动流程"
stages:
bootstrap:
desc: "启动/初始化(仅针对持久化加载出的节点):僵尸清理与归并到 S00"
desc: "启动/初始化:持久化恢复后先做 datasource 刷新,再按 E01 清理僵尸或归并/隔离"
entry_states: [S16]
exit_states: [S00, S15]
exit_states: [S00, S15, S17]
bind:
desc: "绑定判定:core matrix +(可选)依赖检查 +(可选)自动绑定;stage 仅约束边界,不要求内部顺序"
@@ -346,13 +358,23 @@ stages:
transitions:
E01:
stage: bootstrap
desc: "初始化:删CREATE僵尸;其余重置后进入起点"
desc: "初始化:先依据 datasource 刷新结果判定 source 是否存在,再做僵尸清理/隔离/归并"
from: [S16]
outcomes:
- when: { is_create_zombie: true }
to: S15
- when: { is_create_zombie: false }
- when: { is_create_zombie: false, bootstrap_source_present: false, has_binding_record: false }
to: S15
emit:
note: "持久化孤儿节点未命中最新数据源,按僵尸清理"
- when: { is_create_zombie: false, bootstrap_source_present: false, has_binding_record: true }
to: S17
emit:
note: "存在绑定但最新数据源未返回节点,进入加载异常隔离"
- when: { is_create_zombie: false, bootstrap_source_present: true }
to: S00
emit:
note: "bootstrap 已刷新到最新数据,归并到起点"
# --- Bind phase1 core ---
E10:
@@ -101,7 +101,7 @@ class ConstructionTaskApiHandler(BaseApiHandler):
else:
results.append(TaskResult.in_progress(node_id=node.node_id, task_id=push_id))
except Exception as e:
results.append(TaskResult.failed(node_id=node.node_id, error=str(e)))
results.append(TaskResult.failed(node_id=node.node_id, error=str(e), push_id=push_id))
return results
@@ -135,6 +135,7 @@ class ConstructionTaskApiHandler(BaseApiHandler):
node_updated = False
error = None
push_failure_id = None
schema_status_lines: List[str] = []
# 1. 基础信息更新 (ConstructionUpdate)
@@ -186,6 +187,7 @@ class ConstructionTaskApiHandler(BaseApiHandler):
except Exception as e:
error = str(e)
base_update_error = str(e)
push_failure_id = push_id
else:
base_update_error = "show_name changed but task_type != QT"
@@ -236,6 +238,7 @@ class ConstructionTaskApiHandler(BaseApiHandler):
except Exception as e:
error = str(e)
plan_update_error = str(e)
push_failure_id = push_id
schema_status_lines.append(
self.build_update_schema_status(
@@ -253,7 +256,7 @@ class ConstructionTaskApiHandler(BaseApiHandler):
# 汇总结果
if error:
results.append(TaskResult.failed(node_id=node.node_id, error=error))
results.append(TaskResult.failed(node_id=node.node_id, error=error, push_id=push_failure_id))
elif node_updated:
results.append(TaskResult.success(node_id=node.node_id))
else:
@@ -279,7 +282,7 @@ class ConstructionTaskApiHandler(BaseApiHandler):
await api_delete_construction_task(self.api_client, biz_id, push_id)
results.append(TaskResult.success(node_id=node.node_id))
except Exception as e:
results.append(TaskResult.failed(node_id=node.node_id, error=str(e)))
results.append(TaskResult.failed(node_id=node.node_id, error=str(e), push_id=push_id))
return results
@@ -130,7 +130,7 @@ class ConstructionTaskDetailApiHandler(BaseApiHandler):
else:
results.append(TaskResult.in_progress(node_id=node.node_id, task_id=push_id))
except Exception as e:
results.append(TaskResult.failed(node_id=node.node_id, error=str(e)))
results.append(TaskResult.failed(node_id=node.node_id, error=str(e), push_id=push_id))
return results
@@ -222,7 +222,7 @@ class ConstructionTaskDetailApiHandler(BaseApiHandler):
# 整组失败
error_msg = str(e)
for node, _ in group_items:
results.append(TaskResult.failed(node_id=node.node_id, error=error_msg))
results.append(TaskResult.failed(node_id=node.node_id, error=error_msg, push_id=push_id))
return results
@@ -240,7 +240,7 @@ class ConstructionTaskDetailApiHandler(BaseApiHandler):
await api_delete_construction_task_detail(self.api_client, biz_id, push_id)
results.append(TaskResult.success(node_id=node.node_id))
except Exception as e:
results.append(TaskResult.failed(node_id=node.node_id, error=str(e)))
results.append(TaskResult.failed(node_id=node.node_id, error=str(e), push_id=push_id))
return results
+16 -1
View File
@@ -276,8 +276,14 @@ def e01_bootstrap(
*,
node: "SyncNode",
is_create_zombie: bool,
bootstrap_source_present: bool = True,
has_binding_record: bool = False,
) -> Decision:
context = {"is_create_zombie": is_create_zombie}
context = {
"is_create_zombie": is_create_zombie,
"bootstrap_source_present": bootstrap_source_present,
"has_binding_record": has_binding_record,
}
decision = _dispatch_required(
runtime,
node=node,
@@ -289,6 +295,15 @@ def e01_bootstrap(
_apply_decision(runtime, node=node, decision=decision, fields={"binding_status", "action", "status"})
node.error = None
return decision
if decision.to_state == "S17":
_apply_decision(runtime, node=node, decision=decision, fields={"binding_status", "action", "status"})
reason_text = decision_note(decision) or "<no-reason>"
node.context["bootstrap_blocked"] = True
node.context["bootstrap_block_reason"] = "source_missing_after_persistence_refresh"
node.context["bootstrap_block_errors"] = [reason_text]
node.context["bootstrap_source_present"] = False
node.error = f"BOOTSTRAP_SOURCE_MISSING: {reason_text}"
return decision
if decision.to_state == "S15":
reason_text = decision_note(decision) or "<no-reason>"
node.append_log(
@@ -256,6 +256,12 @@ class FullSyncPipeline:
self._prepare_datasources()
self._logger.info("✅ Bootstrap: datasource prepared")
await self._bootstrap_refresh_latest_data()
self._logger.info("✅ Bootstrap: datasource refresh completed")
await self._run_bootstrap_reset_cleanup()
self._logger.info("✅ Bootstrap: reset cleanup completed")
await self._apply_project_scope_cleanup()
async def phase_bind(self) -> None:
@@ -359,7 +365,7 @@ class FullSyncPipeline:
await _safe_close("persistence", self.persistence.close())
async def _load_persistence_state(self) -> None:
"""加载持久化数据并清理僵尸节点"""
"""加载持久化数据到内存,等待 bootstrap datasource refresh 与 E01 cleanup。"""
if not self.enable_persistence:
return
@@ -376,10 +382,50 @@ class FullSyncPipeline:
for node_type in binding_node_types:
await self.binding_manager.load_from_persistence(node_type)
self._logger.info("🔄 Persistence loaded (state preserved; pending E01 normalization)")
# 加载后 reset 清理:清理 CREATE 失败的僵尸绑定
def _resolve_bootstrap_refresh_node_types(self) -> List[str]:
local_types = {node.node_type for node in self.local_collection.filter()}
remote_types = {node.node_type for node in self.remote_collection.filter()}
candidate_types = local_types | remote_types
if not candidate_types:
return []
ordered: List[str] = []
for node_type in [*self.load_order, *self.bootstrap_binding_node_types, *sorted(candidate_types)]:
if node_type in candidate_types and node_type not in ordered:
ordered.append(node_type)
return ordered
async def _bootstrap_refresh_latest_data(self) -> None:
if not self.enable_persistence:
return
node_types = self._resolve_bootstrap_refresh_node_types()
if not node_types:
self._logger.info("⏭️ Bootstrap datasource refresh skipped: no persisted node types")
return
for node_type in node_types:
self._logger.info("🔄 Bootstrap refresh node_type=%s from datasource", node_type)
await self._load_node_type_from_datasource(
self.local_datasource,
datasource_role="local",
node_type=node_type,
reason="bootstrap persistence refresh",
)
await self._load_node_type_from_datasource(
self.remote_datasource,
datasource_role="remote",
node_type=node_type,
reason="bootstrap persistence refresh",
)
async def _run_bootstrap_reset_cleanup(self) -> None:
if not self.enable_persistence:
return
if not self.strategies:
raise RuntimeError("no strategies configured: cannot run bootstrap event E01")
total_cleaned = await run_phase2_cleanup(
node_types=self.node_types,
local_collection=self.local_collection,
@@ -388,8 +434,7 @@ class FullSyncPipeline:
runtime=self.sm_runtime,
)
if total_cleaned > 0:
self._logger.info(f"🧹 Reset cleanup: {total_cleaned} CREATE-failed zombies cleaned")
self._logger.info("🔄 Post-load reset cleanup completed")
self._logger.info(f"🧹 Reset cleanup: {total_cleaned} bootstrap zombies cleaned")
async def _apply_preloaded_state(self) -> None:
if self._preloaded_local_nodes:
@@ -30,6 +30,25 @@ async def _resolve_local_id_for_unbind(
return None
async def _has_binding_record(
*,
node_type: str,
any_side_node_id: str,
binding_manager: "BindingManager",
) -> bool:
local_id = await _resolve_local_id_for_unbind(
node_type=node_type,
any_side_node_id=any_side_node_id,
binding_manager=binding_manager,
)
return local_id is not None
def _bootstrap_source_present(node) -> bool:
context = node.context if isinstance(node.context, dict) else {}
return not bool(context.get("_loaded_from_persistence"))
def _is_create_zombie(node) -> bool:
if node.action != SyncAction.CREATE:
return False
@@ -55,8 +74,27 @@ async def _cleanup_one_side(
)
continue
decision = e01_bootstrap(runtime, node=node, is_create_zombie=_is_create_zombie(node))
has_binding_record = await _has_binding_record(
node_type=node_type,
any_side_node_id=node.node_id,
binding_manager=binding_manager,
)
bootstrap_source_present = _bootstrap_source_present(node)
decision = e01_bootstrap(
runtime,
node=node,
is_create_zombie=_is_create_zombie(node),
bootstrap_source_present=bootstrap_source_present,
has_binding_record=has_binding_record,
)
if decision.to_state != "S15":
if decision.to_state == "S17":
node.context.pop("_loaded_from_persistence", None)
logger.warning(
f"[{node_type}] Bootstrap isolated node ({label}): "
f"node_id={node.node_id}, reason={node.error}"
)
continue
local_id_for_unbind = await _resolve_local_id_for_unbind(
@@ -74,9 +112,12 @@ async def _cleanup_one_side(
else:
bind_note = "no_binding"
reason = "status=FAILED" if node.status == SyncStatus.FAILED else "data_id empty"
if _is_create_zombie(node):
reason = "status=FAILED" if node.status == SyncStatus.FAILED else "data_id empty"
else:
reason = "persistence_only_source_missing_without_binding"
logger.info(
f"[{node_type}] Cleaning up CREATE zombie ({label}): "
f"[{node_type}] Cleaning up bootstrap zombie ({label}): "
f"node_id={node.node_id}, reason={reason}, {bind_note}"
)
@@ -118,7 +159,7 @@ async def run_phase2_cleanup(
total_cleaned += cleaned_count
if cleaned_count > 0:
logger.warning(
f"[{node_type}] Cleaned up CREATE-failed zombies: "
f"[{node_type}] Cleaned up bootstrap zombies: "
f"local_deleted={len(cleaned_local)}, remote_deleted={len(cleaned_remote)}"
)
@@ -128,6 +128,16 @@ async def test_mock_based_full_event_and_state_coverage() -> None:
d = e01_bootstrap(runtime, node=node_e01_del, is_create_zombie=True)
record(node_e01_del, d)
node_e01_isolated = _new_node("e01_isolated", action=SyncAction.NONE, status=SyncStatus.PENDING)
d = e01_bootstrap(
runtime,
node=node_e01_isolated,
is_create_zombie=False,
bootstrap_source_present=False,
has_binding_record=True,
)
record(node_e01_isolated, d)
# E10 merged outcomes
for idx, (has_binding, local_res, peer_res) in enumerate(
[
@@ -507,7 +517,8 @@ async def test_sync_log_cleared_on_initial_load(tmp_path: Path) -> None:
await reloaded.load_from_persistence()
loaded_node = reloaded.get("log-node")
assert loaded_node is not None
assert loaded_node.sync_log is None
assert loaded_node.sync_log is not None
assert loaded_node.sync_log.count("加载来源: persistence 恢复节点") == 1
in_memory_after_reload_report = REPORT_DIR / "in_memory_sync_log_after_reload.txt"
in_memory_after_reload_report.write_text(
@@ -566,6 +577,8 @@ async def test_load_preserves_state_then_e01_normalizes_to_s00(tmp_path: Path) -
assert loaded_node.status == SyncStatus.FAILED
assert loaded_node.binding_status == BindingStatus.WARNING
assert loaded_node.error == "last run failed"
assert loaded_node.sync_log is not None
assert "加载来源: persistence 恢复节点" in loaded_node.sync_log
# E01 后归并到 S00,并清空历史 error
runtime = StateMachineRuntime(StateMachineConfig.load(CFG_PATH))
@@ -579,6 +592,114 @@ async def test_load_preserves_state_then_e01_normalizes_to_s00(tmp_path: Path) -
await persistence.close()
@pytest.mark.asyncio
async def test_persistence_only_unbound_node_is_cleaned_after_bootstrap_refresh_miss(tmp_path: Path) -> None:
db = tmp_path / "bootstrap_source_miss_unbound.db"
persistence = SQLitePersistenceBackend(str(db))
await persistence.initialize()
collection = DataCollection("project-bootstrap-unbound", persistence=persistence, auto_persist=False)
node = ProjectSyncNode(
node_id="stale-unbound-node",
data_id="proj-stale-unbound",
data={
"id": "proj-stale-unbound",
"type": "project",
"name": "Stale Unbound Project",
"code": "P-ST-UNBOUND",
"status": "active",
"progress_type": "C",
"project_type": "onshore_wind",
},
action=SyncAction.NONE,
status=SyncStatus.PENDING,
binding_status=BindingStatus.WARNING,
)
await collection.add(node)
await collection.persist()
reloaded = DataCollection("project-bootstrap-unbound", persistence=persistence, auto_persist=False)
await reloaded.load_from_persistence()
loaded_node = reloaded.get("stale-unbound-node")
assert loaded_node is not None
assert loaded_node.context.get("_loaded_from_persistence") is True
runtime = StateMachineRuntime(StateMachineConfig.load(CFG_PATH))
binding_manager = BindingManager(persistence, auto_persist=False)
remote_collection = DataCollection("empty-remote-unbound", persistence=persistence, auto_persist=False)
cleaned = await run_phase2_cleanup(
node_types=["project"],
local_collection=reloaded,
remote_collection=remote_collection,
binding_manager=binding_manager,
runtime=runtime,
)
assert cleaned == 1
assert reloaded.get("stale-unbound-node") is None
await persistence.close()
@pytest.mark.asyncio
async def test_bound_node_is_isolated_after_bootstrap_refresh_miss(tmp_path: Path) -> None:
db = tmp_path / "bootstrap_source_miss_bound.db"
persistence = SQLitePersistenceBackend(str(db))
await persistence.initialize()
collection = DataCollection("project-bootstrap-bound", persistence=persistence, auto_persist=False)
node = ProjectSyncNode(
node_id="stale-bound-local-node",
data_id="proj-stale-bound",
data={
"id": "proj-stale-bound",
"type": "project",
"name": "Stale Bound Project",
"code": "P-ST-BOUND",
"status": "active",
"progress_type": "C",
"project_type": "onshore_wind",
},
action=SyncAction.NONE,
status=SyncStatus.PENDING,
binding_status=BindingStatus.WARNING,
)
await collection.add(node)
await collection.persist()
reloaded = DataCollection("project-bootstrap-bound", persistence=persistence, auto_persist=False)
await reloaded.load_from_persistence()
loaded_node = reloaded.get("stale-bound-local-node")
assert loaded_node is not None
assert loaded_node.context.get("_loaded_from_persistence") is True
runtime = StateMachineRuntime(StateMachineConfig.load(CFG_PATH))
binding_manager = BindingManager(persistence, auto_persist=False)
await binding_manager.bind("project", "stale-bound-local-node", "remote-missing-node")
remote_collection = DataCollection("empty-remote-bound", persistence=persistence, auto_persist=False)
cleaned = await run_phase2_cleanup(
node_types=["project"],
local_collection=reloaded,
remote_collection=remote_collection,
binding_manager=binding_manager,
runtime=runtime,
)
isolated = reloaded.get("stale-bound-local-node")
assert cleaned == 0
assert isolated is not None
assert reloaded.is_bootstrap_blocked(isolated)
assert isolated.binding_status == BindingStatus.ABNORMAL
assert isolated.action == SyncAction.NONE
assert isolated.status == SyncStatus.FAILED
assert "BOOTSTRAP_SOURCE_MISSING" in (isolated.error or "")
assert isolated.context.get("bootstrap_block_reason") == "source_missing_after_persistence_refresh"
await persistence.close()
@pytest.mark.asyncio
async def test_invalid_persisted_enum_is_bootstrap_blocked_and_skipped_in_reset(tmp_path: Path) -> None:
REPORT_DIR.mkdir(parents=True, exist_ok=True)
@@ -146,3 +146,39 @@ async def test_api_datasource_failed_update_does_not_use_unrelated_op_trace() ->
assert updated.sync_log is not None
assert "API链路[submit-failed] PUT /material/plan" not in updated.sync_log
assert "未找到匹配 biz_id 的请求记录" in updated.sync_log
@pytest.mark.asyncio
async def test_api_datasource_failed_create_uses_push_id_trace_when_data_id_empty() -> None:
collection = DataCollection("remote")
ds = ApiDataSource(base_url="http://example.com", uid="u", secret="s")
ds.set_collection(collection)
node = _build_node(node_id="n-create-fail", data_id="", name="X")
with node.allow_core_state_write("test"):
node.binding_status = BindingStatus.NORMAL
node.action = SyncAction.CREATE
node.status = SyncStatus.IN_PROGRESS
await collection.add(node)
ds.client._record_trace(
method="POST",
endpoint="/construction/create",
url="http://example.com/construction/create",
request_params=None,
request_payload={"push_id": "push-create-fail", "data": {"task_type": "jxfx"}},
response={"code": 400, "message": "施工类型错误"},
elapsed_ms=12,
)
await ds._handle_task_result(
cast(Any, _Handler()),
TaskResult.failed(node_id=node.node_id, error="Create failed: 施工类型错误", push_id="push-create-fail"),
{},
)
updated = collection.get(node.node_id)
assert updated is not None
assert updated.sync_log is not None
assert "API链路[submit-failed] POST /construction/create" in updated.sync_log
assert "response={\"code\":400,\"message\":\"施工类型错误\"}" in updated.sync_log
@@ -0,0 +1,54 @@
from __future__ import annotations
import pytest
from sync_state_machine.domain.construction.api_handler import ConstructionTaskApiHandler
from sync_state_machine.domain.construction.sync_node import ConstructionTaskSyncNode
@pytest.mark.asyncio
async def test_construction_create_failure_preserves_push_id(monkeypatch: pytest.MonkeyPatch) -> None:
handler = ConstructionTaskApiHandler()
handler.api_client = object()
async def _fake_create(client, data, push_id: str, *, steps=None):
del client, data, steps
raise Exception(f"API error for {push_id}")
monkeypatch.setattr(
"sync_state_machine.domain.construction.api_handler.api_create_construction_task",
_fake_create,
)
node = ConstructionTaskSyncNode(
node_id="n1",
data_id="",
data={
"id": "",
"project_id": "p1",
"contract_id": "c1",
"show_name": "架线工程",
"construction_section": "SC",
"status": -1,
"task_type": "jxfx",
"plan_start_date": "2024-06-18",
"plan_complete_date": "2024-12-05",
"actual_complete_date": "2025-12-29",
"complete_quantity": 2.0,
"complete_rate": 100.0,
"contract_quantity": 2.0,
"plan_node": [],
"statistic_date": "2025-12-29",
"is_crucial": False,
"delay_days": None,
},
)
node.context["project_id"] = "p1"
results = await handler.create_all([node])
assert len(results) == 1
result = results[0]
assert result.status.value == "failed"
assert result.push_id is not None
assert result.error == f"API error for {result.push_id}"
+52 -1
View File
@@ -52,7 +52,13 @@ def test_e01_bootstrap_accepts_all_combinations(action: SyncAction, status: Sync
original_error = node.error
is_zombie = _is_create_zombie(node)
decision = e01_bootstrap(runtime, node=node, is_create_zombie=is_zombie)
decision = e01_bootstrap(
runtime,
node=node,
is_create_zombie=is_zombie,
bootstrap_source_present=True,
has_binding_record=False,
)
if is_zombie:
assert decision.to_state == "S15"
@@ -66,3 +72,48 @@ def test_e01_bootstrap_accepts_all_combinations(action: SyncAction, status: Sync
assert node.action == SyncAction.NONE
assert node.status == SyncStatus.PENDING
assert node.error is None
def test_e01_bootstrap_cleans_unbound_persistence_only_node_on_source_miss() -> None:
runtime = StateMachineRuntime(StateMachineConfig.load(CFG_PATH))
node = _new_node(
node_id="e01-stale-unbound",
action=SyncAction.NONE,
status=SyncStatus.PENDING,
data_id="id-1",
)
decision = e01_bootstrap(
runtime,
node=node,
is_create_zombie=False,
bootstrap_source_present=False,
has_binding_record=False,
)
assert decision.to_state == "S15"
def test_e01_bootstrap_isolates_bound_node_on_source_miss() -> None:
runtime = StateMachineRuntime(StateMachineConfig.load(CFG_PATH))
node = _new_node(
node_id="e01-stale-bound",
action=SyncAction.NONE,
status=SyncStatus.PENDING,
data_id="id-1",
)
decision = e01_bootstrap(
runtime,
node=node,
is_create_zombie=False,
bootstrap_source_present=False,
has_binding_record=True,
)
assert decision.to_state == "S17"
assert node.binding_status == BindingStatus.ABNORMAL
assert node.action == SyncAction.NONE
assert node.status == SyncStatus.FAILED
assert node.context["bootstrap_blocked"] is True
assert "BOOTSTRAP_SOURCE_MISSING" in (node.error or "")
@@ -7,7 +7,10 @@ import pytest
from sync_state_machine.common.binding import BindingManager
from sync_state_machine.common.collection import DataCollection
from sync_state_machine.common.sqlite_backend import SQLitePersistenceBackend
from sync_state_machine.common.types import BindingStatus, SyncAction, SyncStatus
from sync_state_machine.datasource.jsonl import JsonlDataSource
from sync_state_machine.domain.project.sync_node import ProjectSyncNode
import sync_state_machine.pipeline.full_sync_pipeline as full_sync_pipeline_module
from sync_state_machine.pipeline.full_sync_pipeline import FullSyncPipeline, NodeTypeLoadError, WriteOutcome
@@ -30,6 +33,37 @@ class _FakeDataSource:
return None
class _BootstrapRefreshDataSource(_FakeDataSource):
def __init__(self, *, node_data: dict[str, str]) -> None:
self.node_data = dict(node_data)
async def load_all(self, order=None, data_type=None, raise_on_error=False) -> None:
del data_type, raise_on_error
if self.collection is None:
raise RuntimeError("collection not set")
if not order or tuple(order) != ("project",):
return
existing = self.collection.get_by_data_id("project", self.node_data["id"])
if existing is None:
node = ProjectSyncNode(
node_id="bootstrap-refreshed-project",
data_id=self.node_data["id"],
data=self.node_data,
)
node.append_log(f"加载来源: {self.__class__.__name__}.load 新增节点")
await self.collection.add(node)
return
existing.set_data(self.node_data)
existing.set_origin_data(self.node_data)
if existing.context.pop("_loaded_from_persistence", False):
existing.append_log(f"加载来源: persistence -> {self.__class__.__name__}.load 覆盖刷新")
else:
existing.append_log(f"加载来源: {self.__class__.__name__}.load 覆盖刷新")
await self.collection.update(existing)
class _StubStrategy:
def __init__(self, node_type: str):
self.node_type = node_type
@@ -348,4 +382,73 @@ async def test_run_persists_before_reraising_load_failure(monkeypatch: pytest.Mo
await pipeline.run()
assert calls == ["bootstrap", "sync", "persist"]
assert persistence.close_calls == 1
assert persistence.close_calls == 1
@pytest.mark.asyncio
async def test_phase_bootstrap_refreshes_persisted_nodes_before_e01_cleanup(tmp_path: Path) -> None:
db_path = tmp_path / "bootstrap-refresh.db"
persistence = SQLitePersistenceBackend(str(db_path))
await persistence.initialize()
seed_remote = DataCollection("remote", persistence=persistence, auto_persist=False)
stale_node = ProjectSyncNode(
node_id="persisted-project-node",
data_id="proj-bootstrap-1",
data={
"id": "proj-bootstrap-1",
"type": "project",
"name": "Persisted Project",
"code": "P-BOOT",
"status": "active",
"progress_type": "C",
"project_type": "onshore_wind",
},
action=SyncAction.NONE,
status=SyncStatus.PENDING,
binding_status=BindingStatus.WARNING,
)
await seed_remote.add(stale_node)
await seed_remote.persist()
local_collection = DataCollection("local", persistence=persistence, auto_persist=False)
remote_collection = DataCollection("remote", persistence=persistence, auto_persist=False)
binding_manager = BindingManager(persistence, auto_persist=False)
pipeline = FullSyncPipeline(
local_datasource=_FakeDataSource(),
remote_datasource=_BootstrapRefreshDataSource(
node_data={
"id": "proj-bootstrap-1",
"type": "project",
"name": "Refreshed Project",
"code": "P-BOOT",
"status": "active",
"progress_type": "C",
"project_type": "onshore_wind",
}
),
strategies=[_StubStrategy("project")],
persistence=persistence,
local_collection=local_collection,
remote_collection=remote_collection,
binding_manager=binding_manager,
node_types=["project"],
load_order=["project"],
sync_order=["project"],
enable_persistence=True,
)
await pipeline.phase_bootstrap()
refreshed = remote_collection.get("persisted-project-node")
assert refreshed is not None
assert refreshed.binding_status == BindingStatus.UNCHECKED
assert refreshed.action == SyncAction.NONE
assert refreshed.status == SyncStatus.PENDING
assert remote_collection.is_bootstrap_blocked(refreshed) is False
assert refreshed.sync_log is not None
assert "加载来源: persistence 恢复节点" in refreshed.sync_log
assert "加载来源: persistence -> _BootstrapRefreshDataSource.load 覆盖刷新" in refreshed.sync_log
await pipeline.close()
@@ -55,4 +55,22 @@ async def test_scope_partitioning_and_copy_between_scopes(tmp_path: Path) -> Non
await reloaded_global.load_from_persistence()
assert len(reloaded_global.filter(node_type="test_project")) == 1
await persistence.close()
@pytest.mark.asyncio
async def test_sqlite_backend_creates_restore_join_index(tmp_path: Path) -> None:
db_path = tmp_path / "scope_partitioning.db"
persistence = SQLitePersistenceBackend(str(db_path))
await persistence.initialize()
columns, rows = PersistenceBackend.query_records(
backend="sqlite",
db_path=str(db_path),
sql="PRAGMA index_list('nodes')",
)
assert columns == ["seq", "name", "unique", "origin", "partial"]
assert any(row["name"] == "idx_nodes_scope_type_node_id" for row in rows)
await persistence.close()