修改多个问题,优化代码结构

This commit is contained in:
strepsiades
2026-04-01 16:23:43 +08:00
parent 8f4727a772
commit 7c49df94fc
29 changed files with 968 additions and 504 deletions
+4 -4
View File
@@ -26,12 +26,12 @@
### strategy(决策组织层) ### strategy(决策组织层)
- 外部入口:`sync_state_machine/sync_system/strategy.py` - 外部入口:`sync_state_machine/sync_system/strategy.py`
- 具体实现下沉:`sync_state_machine/sync_system/strategy_ops/` - 具体实现下沉:`sync_state_machine/sync_system/strategy_ops/`
- `bind_ops.py`: `run_bind()` + `phase_core_state()/phase_dependency_check()/phase_auto_binding()` - `bind_ops.py`: bind 阶段静态函数与纯工具
- `create_ops.py`: `run_create()` - `create_ops.py`: create 阶段静态函数与纯工具
- `update_ops.py`: `run_update()` - `update_ops.py`: update 阶段静态函数与纯工具
- `delete_ops.py`: `run_delete()` - `delete_ops.py`: `run_delete()`
- `reset_ops.py`: `run_phase2_cleanup()` - `reset_ops.py`: `run_phase2_cleanup()`
- strategy 负责收集上下文并调用状态机事件函数,不直接硬编码状态值。 - strategy 负责 bind/create/update 的相位内部编排,并调用状态机事件函数,不直接硬编码状态值。
### datasource(执行与回写层) ### datasource(执行与回写层)
- 入口:`sync_state_machine/datasource/datasource.py` - 入口:`sync_state_machine/datasource/datasource.py`
+3 -3
View File
@@ -43,9 +43,9 @@
- `E40D_START/E40D`DELETE 执行入口与结果回写 - `E40D_START/E40D`DELETE 执行入口与结果回写
## 4. strategy 调用链(当前) ## 4. strategy 调用链(当前)
- bind 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.bind()` -> `sync_state_machine/sync_system/strategy_ops/bind_ops.py::run_bind()` - bind 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.bind()`
- create 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.create()` -> `sync_state_machine/sync_system/strategy_ops/create_ops.py::run_create()` - create 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.create()`
- update 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.update()` -> `sync_state_machine/sync_system/strategy_ops/update_ops.py::run_update()` - update 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.update()`
- reset 主入口:`sync_state_machine/pipeline/full_sync_pipeline.py::_load_persistence_state()` -> `sync_state_machine/sync_system/strategy_ops/reset_ops.py::run_phase2_cleanup()` - reset 主入口:`sync_state_machine/pipeline/full_sync_pipeline.py::_load_persistence_state()` -> `sync_state_machine/sync_system/strategy_ops/reset_ops.py::run_phase2_cleanup()`
## 5. context 字段(以 YAML 为准) ## 5. context 字段(以 YAML 为准)
+14 -4
View File
@@ -34,9 +34,18 @@
- 这类节点不会进入 `E01`,也不会参与 bind/create/update/sync 自动流程。 - 这类节点不会进入 `E01`,也不会参与 bind/create/update/sync 自动流程。
## 3. bind 流程(strategy_ops ## 3. bind 流程(strategy_ops
- 入口:`sync_state_machine/sync_system/strategy_ops/bind_ops.py::run_bind()` - 入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.bind()`
- 入口状态:`S00` - 入口状态:`S00`
### bind 编排步骤
- strategy 先通过 `bind_ops.collect_bind_entry_nodes()` 收集两侧 `S00` 节点,作为 bind 入口工作集。
- `phase_core_state()` 对入口工作集触发 `E10`,把“已有绑定记录”和“无绑定记录”的节点分流到 `S01/S03/S04/S02`
- `phase_dependency_check()` 对仍需继续处理的缺绑节点触发 `E15`,把缺数据、依赖不满足、语义错误的节点拦到 `S03/S05/S04`
- strategy 再通过 `bind_ops.collect_auto_bind_ready_nodes()` 收窄工作集,只保留当前 `S02 + binding_status=MISSING` 的节点进入自动绑定判定。
- strategy 在顶层显式构造 orphan create 控制参数,并传给 `bind_ops.plan_auto_bind_updates()` 生成自动绑定/自动创建决策。
- `apply_auto_bind_updates()` 对这些决策逐个触发 `E16`,把节点迁移到 `S01/S04/S05/S13`
- `refresh_bind_data_id()` 最后按绑定关系回填两侧节点的 `context.bind_data_id`
### 核心绑定判定 ### 核心绑定判定
- `phase_core_state()` 内触发 `E10` - `phase_core_state()` 内触发 `E10`
- 有绑定记录:根据本端/对端有效性进入 `S01/S03/S04` - 有绑定记录:根据本端/对端有效性进入 `S01/S03/S04`
@@ -44,9 +53,10 @@
### 依赖检查 ### 依赖检查
- `phase_dependency_check()` -> `check_dependencies_for_nodes()` 内触发 `E15` - `phase_dependency_check()` -> `check_dependencies_for_nodes()` 内触发 `E15`
- `data_present=true``dep_satisfied=true` -> 不迁移,节点保持在 `S02`,继续进入自动绑定判定。
- `data_present=false` -> `S03` - `data_present=false` -> `S03`
- `dep_satisfied=false` -> `S05` - `dep_satisfied=false` -> `S05`
- `depend_fields` 为空 -> `S04`(无法自动处理,阻断后续自动创建) - `depend_fields` 为空本身不会触发阻断;若节点数据存在,则等价于“当前依赖满足”,继续留在 `S02`
### 自动绑定 ### 自动绑定
- `phase_auto_binding()` 内触发 `E16` - `phase_auto_binding()` 内触发 `E16`
@@ -57,7 +67,7 @@
## 4. create/update 准备 ## 4. create/update 准备
### create ### create
- 入口:`sync_state_machine/sync_system/strategy_ops/create_ops.py::run_create()` - 入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.create()`
- 运行时事件:`E20``E21/E22` 为配置保留分支,当前 strategy 未接入自动执行) - 运行时事件:`E20``E21/E22` 为配置保留分支,当前 strategy 未接入自动执行)
- 运行时入口状态:`S13` - 运行时入口状态:`S13`
- 说明:`E16` 负责识别“需自动创建”并将源节点置为 `S13`;业务代码执行对侧 spawn+bind 后,再通过 `E20``S13` 提交到 `S01`(成功);若 spawn 失败,`E20` 不迁移并保留在 `S13` 等待人工/重试。 - 说明:`E16` 负责识别“需自动创建”并将源节点置为 `S13`;业务代码执行对侧 spawn+bind 后,再通过 `E20``S13` 提交到 `S01`(成功);若 spawn 失败,`E20` 不迁移并保留在 `S13` 等待人工/重试。
@@ -66,7 +76,7 @@
- 新创建目标节点在 `S06/S10C` 阶段允许 `data_id` 为空;`CREATE SUCCESS``S11C`)必须携带非空 `result_data_id`,否则事件直接报错。 - 新创建目标节点在 `S06/S10C` 阶段允许 `data_id` 为空;`CREATE SUCCESS``S11C`)必须携带非空 `result_data_id`,否则事件直接报错。
### update ### update
- 入口:`sync_state_machine/sync_system/strategy_ops/update_ops.py::run_update()` - 入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.update()`
- 事件:`E30` - 事件:`E30`
- 入口状态(source 节点):`S01` - 入口状态(source 节点):`S01`
- `target_has_data_id=false``update_id_resolve_ok=false` -> `S08` - `target_has_data_id=false``update_id_resolve_ok=false` -> `S08`
+8 -3
View File
@@ -11,9 +11,13 @@ if str(PROJECT_ROOT) not in sys.path:
sys.path.insert(0, str(PROJECT_ROOT)) sys.path.insert(0, str(PROJECT_ROOT))
from sync_state_machine.config import load_overrides_from_file from sync_state_machine.config import load_overrides_from_file
from sync_state_machine.logging import get_logger, init_app_logger
from sync_state_machine.pipeline import run_profile_from_file from sync_state_machine.pipeline import run_profile_from_file
logger = get_logger(__name__)
def _resolve_config_path(raw_path: str) -> Path: def _resolve_config_path(raw_path: str) -> Path:
path = Path(raw_path) path = Path(raw_path)
if not path.is_absolute(): if not path.is_absolute():
@@ -22,6 +26,7 @@ def _resolve_config_path(raw_path: str) -> Path:
async def _main() -> int: async def _main() -> int:
init_app_logger(replace_handlers=True)
parser = argparse.ArgumentParser(description="Run sync pipeline from config override file") parser = argparse.ArgumentParser(description="Run sync pipeline from config override file")
parser.add_argument( parser.add_argument(
"--config_path", "--config_path",
@@ -34,7 +39,7 @@ async def _main() -> int:
if not cfg_path.exists() or not cfg_path.is_file(): if not cfg_path.exists() or not cfg_path.is_file():
raise FileNotFoundError(f"Config file not found: {cfg_path}") raise FileNotFoundError(f"Config file not found: {cfg_path}")
print(f"🧩 Loaded config file: {cfg_path}") logger.info("🧩 Loaded config file: %s", cfg_path)
await run_profile_from_file( await run_profile_from_file(
project_root=PROJECT_ROOT, project_root=PROJECT_ROOT,
config_path=cfg_path, config_path=cfg_path,
@@ -47,10 +52,10 @@ def main() -> int:
try: try:
return asyncio.run(_main()) return asyncio.run(_main())
except KeyboardInterrupt: except KeyboardInterrupt:
print("\n⚠️ Interrupted by user") logger.warning("⚠️ Interrupted by user")
return 130 return 130
except Exception as exc: except Exception as exc:
print(f"\n❌ Pipeline failed: {type(exc).__name__}: {exc}") logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
return 1 return 1
finally: finally:
logging.shutdown() logging.shutdown()
+28 -4
View File
@@ -5,7 +5,7 @@
from typing import List, Optional from typing import List, Optional
from fastapi import HTTPException from fastapi import HTTPException
from pydantic import Field, BaseModel, field_validator from pydantic import Field, BaseModel, field_validator, model_validator
from ..common.push_system import BaseUpdateRequestSchema from ..common.push_system import BaseUpdateRequestSchema
from ..common.base import BaseResponseWithID from ..common.base import BaseResponseWithID
from ..common.datetime import get_current_date from ..common.datetime import get_current_date
@@ -78,7 +78,7 @@ class ProjectResponseBase(BaseResponseWithID):
# 拓展信息 # 拓展信息
dbtc_score_list: Optional[list] = Field(default_factory=list, description='达标投产评分列表') # 这里项目用的是list而不是List dbtc_score_list: Optional[list] = Field(default_factory=list, description='达标投产评分列表') # 这里项目用的是list而不是List
# : Optional[str] = Field(None, description="审核状态", examples=["pending"]) # approval_status: Optional[str] = Field(None, description="审核状态", examples=["pending"])
@@ -266,11 +266,35 @@ class ProjectProductionCapacity(BaseModel):
""" """
plan_production_date: DateStr | None = Field(None, description="集团公司计划投产日期", examples=["2024-08-01"]) plan_production_date: DateStr | None = Field(None, description="集团公司计划投产日期", examples=["2024-08-01"])
plan_full_production_date: DateStr | None = Field(None, description="集团公司计划全部投产日期", examples=["2024-09-01"]) plan_full_production_date: DateStr | None = Field(None, description="集团公司计划全部投产日期", examples=["2024-09-01"])
ic_plan_first_production_date: DateStr | None = Field(None, description="区域公司计划首批投产日期(只有新能源需要)",examples=["2024-08-01"])
ic_plan_full_production_date: DateStr | None = Field(None, description="区域公司计划全投日期(只有新能源需要)",examples=["2024-09-01"])
actual_production_date: DateStr | None = Field(None, description="实际投产日期(只有新能源需要)", examples=["2024-08-15"]) actual_production_date: DateStr | None = Field(None, description="实际投产日期(只有新能源需要)", examples=["2024-08-15"])
actual_full_production_date: DateStr | None = Field(None, description="实际全投日期(只有新能源需要)", examples=["2024-09-10"]) actual_full_production_date: DateStr | None = Field(None, description="实际全投日期(只有新能源需要)", examples=["2024-09-10"])
actual_production_list : list[_ActualProductionNode] | None = Field(None, description="实际投产节点列表(每次必须填写全部)(只有新能源需要)") actual_production_list : list[_ActualProductionNode] | None = Field(None, description="实际投产节点列表(每次必须填写全部)(只有新能源需要)")
@model_validator(mode="before")
@classmethod
def reject_ic_plan_dates(cls, data):
if not isinstance(data, dict):
return data
blocked_fields = [
field for field in ("ic_plan_first_production_date", "ic_plan_full_production_date")
if field in data
]
if blocked_fields:
raise HTTPException(
status_code=400,
detail=f"/production_capacity 接口不支持更新字段: {', '.join(blocked_fields)}"
)
return data
@property
def ic_plan_first_production_date(self) -> str:
return ""
@property
def ic_plan_full_production_date(self) -> str:
return ""
@field_validator("actual_production_date") @field_validator("actual_production_date")
def validate_actual_production_date(cls, v): def validate_actual_production_date(cls, v):
if v is None: if v is None:
@@ -47,8 +47,6 @@ async def main() -> int:
return 130 return 130
except Exception as exc: except Exception as exc:
logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc) logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
import traceback
traceback.print_exc()
return 1 return 1
@@ -47,8 +47,6 @@ async def main() -> int:
return 130 return 130
except Exception as exc: except Exception as exc:
logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc) logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
import traceback
traceback.print_exc()
return 1 return 1
+8 -3
View File
@@ -5,9 +5,13 @@ import asyncio
import logging import logging
from pathlib import Path from pathlib import Path
from .logging import get_logger, init_app_logger
from .pipeline import run_profile_from_file from .pipeline import run_profile_from_file
logger = get_logger(__name__)
def package_project_root() -> Path: def package_project_root() -> Path:
return Path(__file__).resolve().parents[1] return Path(__file__).resolve().parents[1]
@@ -20,6 +24,7 @@ def resolve_config_path(raw_path: str, *, project_root: Path) -> Path:
async def _main() -> int: async def _main() -> int:
init_app_logger(replace_handlers=True)
project_root = package_project_root() project_root = package_project_root()
parser = argparse.ArgumentParser(description="Run sync pipeline from config override file") parser = argparse.ArgumentParser(description="Run sync pipeline from config override file")
@@ -34,7 +39,7 @@ async def _main() -> int:
if not cfg_path.exists() or not cfg_path.is_file(): if not cfg_path.exists() or not cfg_path.is_file():
raise FileNotFoundError(f"Config file not found: {cfg_path}") raise FileNotFoundError(f"Config file not found: {cfg_path}")
print(f"🧩 Loaded config file: {cfg_path}") logger.info("🧩 Loaded config file: %s", cfg_path)
await run_profile_from_file( await run_profile_from_file(
project_root=project_root, project_root=project_root,
config_path=cfg_path, config_path=cfg_path,
@@ -47,10 +52,10 @@ def main() -> int:
try: try:
return asyncio.run(_main()) return asyncio.run(_main())
except KeyboardInterrupt: except KeyboardInterrupt:
print("\n⚠️ Interrupted by user") logger.warning("⚠️ Interrupted by user")
return 130 return 130
except Exception as exc: except Exception as exc:
print(f"\n❌ Pipeline failed: {type(exc).__name__}: {exc}") logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
return 1 return 1
finally: finally:
logging.shutdown() logging.shutdown()
+3 -1
View File
@@ -1,10 +1,12 @@
import json import json
import logging import logging
from typing import Optional, Dict, Any, List from typing import Optional, Dict, Any, List
from ..logging import get_logger
from .persistence import PersistenceBackend from .persistence import PersistenceBackend
from .types import BindingStatus, BindingRecord from .types import BindingStatus, BindingRecord
logger = logging.getLogger(__name__) logger = get_logger(__name__)
class BindingManager: class BindingManager:
+4 -3
View File
@@ -23,6 +23,7 @@ from datetime import datetime
import httpx import httpx
from schemas.common.push_system import PushIdsSchema from schemas.common.push_system import PushIdsSchema
from ...logging import get_logger
def _build_default_push_steps() -> List[Dict[str, str]]: def _build_default_push_steps() -> List[Dict[str, str]]:
@@ -68,7 +69,7 @@ class ApiClient:
self._trace_by_biz_id: Dict[str, deque[Dict[str, Any]]] = defaultdict(deque) self._trace_by_biz_id: Dict[str, deque[Dict[str, Any]]] = defaultdict(deque)
self._trace_by_op: Dict[str, deque[Dict[str, Any]]] = defaultdict(deque) self._trace_by_op: Dict[str, deque[Dict[str, Any]]] = defaultdict(deque)
self._load_trace_by_item: Dict[tuple[str, str], deque[Dict[str, Any]]] = defaultdict(deque) self._load_trace_by_item: Dict[tuple[str, str], deque[Dict[str, Any]]] = defaultdict(deque)
self._logger = logging.getLogger(__name__) self._logger = get_logger(__name__)
self._rate_limit_max_requests = int(rate_limit_max_requests) self._rate_limit_max_requests = int(rate_limit_max_requests)
self._rate_limit_window_seconds = float(rate_limit_window_seconds) self._rate_limit_window_seconds = float(rate_limit_window_seconds)
self._rate_limit_lock = asyncio.Lock() self._rate_limit_lock = asyncio.Lock()
@@ -138,8 +139,8 @@ class ApiClient:
if not self._debug: if not self._debug:
return return
self._log( self._log(
f"ApiClient request: method={method.upper()} endpoint={endpoint} elapsed={elapsed_seconds:.3f}s", f"🔎 ApiClient request: method={method.upper()} endpoint={endpoint} elapsed={elapsed_seconds:.3f}s",
level="DEBUG", level="INFO",
) )
def _log(self, message: str, level: str = "DEBUG"): def _log(self, message: str, level: str = "DEBUG"):
@@ -3,8 +3,10 @@ from __future__ import annotations
import logging import logging
from pathlib import Path from pathlib import Path
from ...logging import get_logger
logger = logging.getLogger(__name__)
logger = get_logger(__name__)
def prepare_remote_jsonl_dir(remote_dir: Path, *, project_root: Path) -> Path: def prepare_remote_jsonl_dir(remote_dir: Path, *, project_root: Path) -> Path:
@@ -1,9 +1,18 @@
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from ...common.sync_node import SyncNode
from ...sync_system.strategy import DefaultSyncStrategy from ...sync_system.strategy import DefaultSyncStrategy
from ...sync_system.config import StrategyConfig, OrphanAction, UpdateDirection from ...sync_system.config import StrategyConfig, OrphanAction, UpdateDirection
from ...sync_system.strategy_ops.compare_ops import build_data_id_normalization_map from ...sync_system.strategy_ops.bind_ops import (
from ...sync_system.strategy_ops.update_ops import emit_schema_diff_report, _build_schema_diff_validator apply_auto_bind_updates,
collect_auto_bind_ready_nodes,
collect_bind_entry_nodes,
plan_auto_bind_updates,
phase_core_state,
phase_dependency_check,
refresh_bind_data_id,
)
from ...sync_system.strategy_ops import BoundNodePair
from schemas.project_extensions.project_detail import ProjectDetailKey, ProjectDetailProject from schemas.project_extensions.project_detail import ProjectDetailKey, ProjectDetailProject
@@ -37,45 +46,108 @@ class ProjectDetailSyncStrategy(DefaultSyncStrategy[ProjectDetailProject]):
update_direction=UpdateDirection.PUSH, update_direction=UpdateDirection.PUSH,
) )
async def update(self): async def bind(self) -> None:
self.ensure_runtime() runtime = self.ensure_runtime()
validator = _build_schema_diff_validator(self) local_nodes, remote_nodes = collect_bind_entry_nodes(
self._active_schema_diff_validator = validator node_type=self.node_type,
data_id_map = await build_data_id_normalization_map( local_collection=self.local_collection,
remote_collection=self.remote_collection,
)
await phase_core_state(
node_type=self.node_type,
runtime=runtime,
binding_manager=self.binding_manager,
local_nodes=local_nodes,
remote_nodes=remote_nodes,
)
await phase_dependency_check(
node_type=self.node_type,
runtime=runtime,
depend_fields=dict(self.config.depend_fields),
local_nodes=local_nodes,
remote_nodes=remote_nodes,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
)
local_nodes, remote_nodes = collect_auto_bind_ready_nodes(
node_type=self.node_type,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
)
local_create_enabled = self.config.local_orphan_action == OrphanAction.CREATE_REMOTE
remote_create_enabled = self.config.remote_orphan_action == OrphanAction.CREATE_LOCAL
local_orphan_create_enabled_by_node = {
node.node_id: local_create_enabled for node in local_nodes
}
remote_orphan_create_enabled_by_node = {
node.node_id: remote_create_enabled for node in remote_nodes
}
if self.domain_option.pull_group_production_plans:
for node in local_nodes:
if self._extract_node_key(node) in self._GROUP_PRODUCTION_PLAN_KEYS:
local_orphan_create_enabled_by_node[node.node_id] = False
for node in remote_nodes:
if self._extract_node_key(node) in self._GROUP_PRODUCTION_PLAN_KEYS:
remote_orphan_create_enabled_by_node[node.node_id] = True
pending_updates = await plan_auto_bind_updates(
node_type=self.node_type,
config=self.config,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
binding_manager=self.binding_manager,
local_nodes=local_nodes,
remote_nodes=remote_nodes,
id_field_hints=dict(self.config.depend_fields),
local_orphan_create_enabled_by_node=local_orphan_create_enabled_by_node,
remote_orphan_create_enabled_by_node=remote_orphan_create_enabled_by_node,
)
await apply_auto_bind_updates(
node_type=self.node_type,
runtime=runtime,
binding_manager=self.binding_manager,
pending_auto_bind_updates=pending_updates,
)
await refresh_bind_data_id(
node_type=self.node_type, node_type=self.node_type,
local_collection=self.local_collection, local_collection=self.local_collection,
remote_collection=self.remote_collection, remote_collection=self.remote_collection,
binding_manager=self.binding_manager, binding_manager=self.binding_manager,
) )
async def process_update_pairs(
self,
pairs: list[BoundNodePair],
data_id_map: dict[str, str] | None = None,
) -> dict[str, SyncNode]:
pull_keys = self._GROUP_PRODUCTION_PLAN_KEYS if self.domain_option.pull_group_production_plans else set()
pull_pairs = [] pull_pairs = []
generic_pairs = [] generic_pairs = []
for pair in await self.collect_update_pairs(): for pair in pairs:
if self._should_pull_pair(pair.local_node, pair.remote_node): pair_key = self._extract_pair_key(pair.local_node, pair.remote_node)
if pair_key in pull_keys:
pull_pairs.append(pair) pull_pairs.append(pair)
else: else:
generic_pairs.append(pair) generic_pairs.append(pair)
updated_by_id = {} updated_by_id: dict[str, SyncNode] = {}
try: for pair in pull_pairs:
for pair in pull_pairs: updated_node = await self.prepare_update_for_direction(
updated_node = await self.prepare_update_for_direction( pair.local_node,
pair.local_node, pair.remote_node,
pair.remote_node, UpdateDirection.PULL,
UpdateDirection.PULL, data_id_map,
data_id_map, )
) if updated_node is not None:
if updated_node is not None: updated_by_id[updated_node.node_id] = updated_node
updated_by_id[updated_node.node_id] = updated_node
for pair in generic_pairs: for pair in generic_pairs:
for updated_node in await self.update_pair(pair.local_node, pair.remote_node, data_id_map): for updated_node in await self.update_pair(pair.local_node, pair.remote_node, data_id_map):
updated_by_id[updated_node.node_id] = updated_node updated_by_id[updated_node.node_id] = updated_node
finally:
self._active_schema_diff_validator = None
emit_schema_diff_report(self, validator) return updated_by_id
return list(updated_by_id.values())
@property @property
def domain_option(self) -> ProjectDetailDomainOption: def domain_option(self) -> ProjectDetailDomainOption:
@@ -129,6 +201,16 @@ class ProjectDetailSyncStrategy(DefaultSyncStrategy[ProjectDetailProject]):
return items return items
@staticmethod
def _extract_node_key(node: SyncNode) -> str | None:
data = node.get_data() or {}
key = data.get("key")
if isinstance(key, ProjectDetailKey):
return key.value
if isinstance(key, str):
return key
return None
@staticmethod @staticmethod
def _extract_pair_key(local_node, remote_node) -> str | None: def _extract_pair_key(local_node, remote_node) -> str | None:
local_data = local_node.get_data() or {} local_data = local_node.get_data() or {}
@@ -141,7 +223,3 @@ class ProjectDetailSyncStrategy(DefaultSyncStrategy[ProjectDetailProject]):
return key return key
return None return None
def _should_pull_pair(self, local_node, remote_node) -> bool:
if not self.domain_option.pull_group_production_plans:
return False
return self._extract_pair_key(local_node, remote_node) in self._GROUP_PRODUCTION_PLAN_KEYS
+1
View File
@@ -39,6 +39,7 @@ def clear_app_logger_handlers(logger: Optional[logging.Logger] = None) -> None:
handler.close() handler.close()
except Exception: except Exception:
pass pass
target.propagate = True
def init_app_logger( def init_app_logger(
+9 -3
View File
@@ -48,7 +48,6 @@ from ..config import (
from ..config.strategy_config import resolve_domain_option_config from ..config.strategy_config import resolve_domain_option_config
from ..sync_system.strategy import BaseSyncStrategy from ..sync_system.strategy import BaseSyncStrategy
from ..logging import configure_app_logging, ensure_app_logging, get_logger from ..logging import configure_app_logging, ensure_app_logging, get_logger
from .run_logger import init_pipeline_logger
from .full_sync_pipeline import FullSyncPipeline from .full_sync_pipeline import FullSyncPipeline
@@ -458,9 +457,16 @@ async def run_profile_from_file(
cfg_path = Path(config_path) cfg_path = Path(config_path)
raw = load_overrides_from_file(cfg_path, project_root) raw = load_overrides_from_file(cfg_path, project_root)
if is_multi_project_payload(raw): if is_multi_project_payload(raw):
from .multi_project_pipeline import MultiProjectPipeline from .multi_project_pipeline import MultiProjectPipeline, _resolve_embedded_config_path
multi_cfg = MultiProjectRunConfig.model_validate(raw) multi_cfg = MultiProjectRunConfig.model_validate(raw)
global_cfg_path = _resolve_embedded_config_path(
multi_cfg.global_config,
project_root=project_root,
profile_path=cfg_path,
)
global_config = build_config_from_file(project_root=project_root, file_path=global_cfg_path)
configure_app_logging(global_config.logging, replace_handlers=True)
pipeline = MultiProjectPipeline(project_root=project_root, config_path=cfg_path, config=multi_cfg) pipeline = MultiProjectPipeline(project_root=project_root, config_path=cfg_path, config=multi_cfg)
if print_summary: if print_summary:
logger.info("\n%s", "=" * 80) logger.info("\n%s", "=" * 80)
@@ -469,7 +475,7 @@ async def run_profile_from_file(
return await pipeline.run() return await pipeline.run()
config = build_config(project_root=project_root, overrides=raw) config = build_config(project_root=project_root, overrides=raw)
configure_app_logging(config.logging, replace_handlers=False) configure_app_logging(config.logging, replace_handlers=True)
mode = f"{config.local_datasource.type}->{config.remote_datasource.type}" mode = f"{config.local_datasource.type}->{config.remote_datasource.type}"
return await run_pipeline_from_config( return await run_pipeline_from_config(
config, config,
@@ -15,17 +15,19 @@ if TYPE_CHECKING:
from ..datasource.jsonl import JsonlDataSource from ..datasource.jsonl import JsonlDataSource
from ..common.collection import DataCollection from ..common.collection import DataCollection
from ..common.binding import BindingManager from ..common.binding import BindingManager
from ..logging import get_logger
from ..common.persistence import PersistenceBackend from ..common.persistence import PersistenceBackend
from ..sync_system.strategy import BaseSyncStrategy from ..sync_system.strategy import BaseSyncStrategy
from ..sync_system.config import OrphanAction, UpdateDirection from ..sync_system.config import OrphanAction, UpdateDirection
from ..engine import StateMachineConfig, StateMachineRuntime from ..engine import StateMachineConfig, StateMachineRuntime
from ..engine import e41_post_create_ready from ..engine import e41_post_create_ready
from ..sync_system.strategy_ops import finalize_peer_creating_sources, run_phase2_cleanup from ..sync_system.strategy_ops import finalize_peer_creating_sources, run_phase2_cleanup
from ..sync_system.strategy_ops.bind_ops import refresh_bind_data_id
from ..sync_system.strategy_ops.post_check_ops import evaluate_consistency_for_node_type from ..sync_system.strategy_ops.post_check_ops import evaluate_consistency_for_node_type
from .summary_report import print_pipeline_summary from .summary_report import print_pipeline_summary
logger = logging.getLogger(__name__) logger = get_logger(__name__)
class FullSyncPipeline: class FullSyncPipeline:
@@ -211,10 +213,24 @@ class FullSyncPipeline:
try: try:
created = await strategy.create() created = await strategy.create()
self._logger.info(f"✅ Strategy create: {node_type} (created={len(created)})") self._logger.info(f"✅ Strategy create: {node_type} (created={len(created)})")
created_local_node_ids = {
node.node_id for node in created if self.local_collection.get_by_node_id(node.node_id) is not None
}
created_remote_node_ids = {
node.node_id for node in created if self.remote_collection.get_by_node_id(node.node_id) is not None
}
create_written = await self.commit_creates(node_type) create_written = await self.commit_creates(node_type)
await self.normalize_create_success_to_update_entry(node_type) await self.normalize_create_success_to_update_entry(node_type)
if create_written: if create_written:
await self._reload_node_type(node_type, reason="create wrote data") await self._reload_node_type(node_type, reason="create wrote data")
await refresh_bind_data_id(
node_type=node_type,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
binding_manager=self.binding_manager,
local_node_ids=created_local_node_ids,
remote_node_ids=created_remote_node_ids,
)
except Exception as exc: except Exception as exc:
self.stats["failed"] += 1 self.stats["failed"] += 1
self._logger.error(f"❌ Strategy create failed but pipeline continues: {node_type} | {type(exc).__name__}: {exc}") self._logger.error(f"❌ Strategy create failed but pipeline continues: {node_type} | {type(exc).__name__}: {exc}")
@@ -12,10 +12,11 @@ from ..config import (
apply_config_overrides, apply_config_overrides,
build_config_from_file, build_config_from_file,
) )
from ..logging import get_logger
from .factory import create_pipeline_from_config from .factory import create_pipeline_from_config
logger = logging.getLogger(__name__) logger = get_logger(__name__)
class _SharedStateSnapshot(TypedDict): class _SharedStateSnapshot(TypedDict):
+129 -12
View File
@@ -1,35 +1,45 @@
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from typing import TypeVar, Generic, Type, Dict, List, Optional, Any from typing import TypeVar, Generic, Type, Dict, List, Optional, Any
from pydantic import BaseModel from pydantic import BaseModel
import logging
from pathlib import Path from pathlib import Path
from ..common.collection import DataCollection from ..common.collection import DataCollection
from ..common.binding import BindingManager from ..common.binding import BindingManager
from ..common.sync_node import SyncNode from ..common.sync_node import SyncNode
from ..logging import get_logger
from .config import StrategyConfig, OrphanAction, UpdateDirection from .config import StrategyConfig, OrphanAction, UpdateDirection
from ..config.strategy_config import resolve_domain_option_config from ..config.strategy_config import resolve_domain_option_config
from .strategy_ops import ( from .strategy_ops import (
run_bind,
run_create,
run_delete, run_delete,
) )
from .strategy_ops.bind_ops import (
apply_auto_bind_updates,
collect_auto_bind_ready_nodes,
collect_bind_entry_nodes,
plan_auto_bind_updates,
phase_core_state,
phase_dependency_check,
refresh_bind_data_id,
)
from .strategy_ops.create_ops import add_create_action
from .strategy_ops.update_ops import ( from .strategy_ops.update_ops import (
BoundNodePair,
_build_schema_diff_validator,
collect_bound_node_pairs, collect_bound_node_pairs,
default_get_node_update_payload, default_get_node_update_payload,
default_needs_update, default_needs_update,
default_update_pair, default_update_pair,
emit_schema_diff_report,
prepare_directional_update, prepare_directional_update,
run_update,
) )
from ..engine import ( from ..engine import (
StateMachineConfig, StateMachineConfig,
StateMachineRuntime, StateMachineRuntime,
) )
from .strategy_ops.compare_ops import normalized_data_for_compare from .strategy_ops.compare_ops import build_data_id_normalization_map, normalized_data_for_compare
T = TypeVar("T", bound=BaseModel) T = TypeVar("T", bound=BaseModel)
logger = logging.getLogger(__name__) logger = get_logger(__name__)
class BaseSyncStrategy(ABC, Generic[T]): class BaseSyncStrategy(ABC, Generic[T]):
""" """
@@ -69,7 +79,7 @@ class BaseSyncStrategy(ABC, Generic[T]):
pass pass
@abstractmethod @abstractmethod
async def bind(self, **kwargs): async def bind(self):
"""进行绑定逻辑判定,设置 node.binding_status 和 node.action""" """进行绑定逻辑判定,设置 node.binding_status 和 node.action"""
pass pass
@@ -191,17 +201,124 @@ class DefaultSyncStrategy(BaseSyncStrategy[T]):
) )
return normalized return normalized
async def bind(self, **kwargs): async def bind(self):
self.ensure_runtime() runtime = self.ensure_runtime()
await run_bind(self, **kwargs) bind_local_nodes, bind_remote_nodes = collect_bind_entry_nodes(
node_type=self.node_type,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
)
await phase_core_state(
node_type=self.node_type,
runtime=runtime,
binding_manager=self.binding_manager,
local_nodes=bind_local_nodes,
remote_nodes=bind_remote_nodes,
)
await phase_dependency_check(
node_type=self.node_type,
runtime=runtime,
depend_fields=dict(self.config.depend_fields),
local_nodes=bind_local_nodes,
remote_nodes=bind_remote_nodes,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
)
auto_bind_local_nodes, auto_bind_remote_nodes = collect_auto_bind_ready_nodes(
node_type=self.node_type,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
)
local_create_enabled = self.config.local_orphan_action == OrphanAction.CREATE_REMOTE
remote_create_enabled = self.config.remote_orphan_action == OrphanAction.CREATE_LOCAL
local_orphan_create_enabled_by_node = {
node.node_id: local_create_enabled for node in auto_bind_local_nodes
}
remote_orphan_create_enabled_by_node = {
node.node_id: remote_create_enabled for node in auto_bind_remote_nodes
}
pending_updates = await plan_auto_bind_updates(
node_type=self.node_type,
config=self.config,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
binding_manager=self.binding_manager,
local_nodes=auto_bind_local_nodes,
remote_nodes=auto_bind_remote_nodes,
id_field_hints=dict(self.config.depend_fields),
local_orphan_create_enabled_by_node=local_orphan_create_enabled_by_node,
remote_orphan_create_enabled_by_node=remote_orphan_create_enabled_by_node,
)
await apply_auto_bind_updates(
node_type=self.node_type,
runtime=runtime,
binding_manager=self.binding_manager,
pending_auto_bind_updates=pending_updates,
)
await refresh_bind_data_id(
node_type=self.node_type,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
binding_manager=self.binding_manager,
local_node_ids={
*(node.node_id for node in bind_local_nodes),
*(node.node_id for node in auto_bind_local_nodes),
},
remote_node_ids={
*(node.node_id for node in bind_remote_nodes),
*(node.node_id for node in auto_bind_remote_nodes),
},
)
async def create(self) -> List[SyncNode]: async def create(self) -> List[SyncNode]:
self.ensure_runtime() self.ensure_runtime()
return await run_create(self) created_nodes: List[SyncNode] = []
created_nodes.extend(await self.create_for_direction(source_is_local=True))
created_nodes.extend(await self.create_for_direction(source_is_local=False))
return created_nodes
async def create_for_direction(self, *, source_is_local: bool) -> List[SyncNode]:
from_collection = self.local_collection if source_is_local else self.remote_collection
to_collection = self.remote_collection if source_is_local else self.local_collection
return await add_create_action(
self,
from_collection=from_collection,
to_collection=to_collection,
source_is_local=source_is_local,
)
async def update(self) -> List[SyncNode]: async def update(self) -> List[SyncNode]:
self.ensure_runtime() self.ensure_runtime()
return await run_update(self) validator = _build_schema_diff_validator(self)
self._active_schema_diff_validator = validator
data_id_map = await self.build_update_data_id_map()
try:
pairs = await self.collect_update_pairs()
updated_by_id = await self.process_update_pairs(pairs, data_id_map)
finally:
self._active_schema_diff_validator = None
emit_schema_diff_report(self, validator)
return list(updated_by_id.values())
async def build_update_data_id_map(self) -> Dict[str, str]:
return await build_data_id_normalization_map(
node_type=self.node_type,
local_collection=self.local_collection,
remote_collection=self.remote_collection,
binding_manager=self.binding_manager,
)
async def process_update_pairs(
self,
pairs: List[BoundNodePair],
data_id_map: Optional[Dict[str, str]] = None,
) -> Dict[str, SyncNode]:
updated_by_id: Dict[str, SyncNode] = {}
for pair in pairs:
for updated_node in await self.update_pair(pair.local_node, pair.remote_node, data_id_map):
updated_by_id[updated_node.node_id] = updated_node
return updated_by_id
def get_node_update_payload(self, node: SyncNode) -> Dict[str, Any]: def get_node_update_payload(self, node: SyncNode) -> Dict[str, Any]:
return default_get_node_update_payload(node) return default_get_node_update_payload(node)
@@ -1,18 +1,16 @@
from .create_ops import add_create_action, run_create, finalize_peer_creating_sources from .create_ops import add_create_action, finalize_peer_creating_sources
from .update_ops import ( from .update_ops import (
BoundNodePair, BoundNodePair,
add_update_action, add_update_action,
collect_bound_node_pairs, collect_bound_node_pairs,
prepare_directional_update, prepare_directional_update,
run_update,
) )
from .delete_ops import run_delete from .delete_ops import run_delete
from .bind_ops import ( from .bind_ops import (
run_bind,
phase_core_state, phase_core_state,
phase_dependency_check, phase_dependency_check,
check_dependencies_for_nodes,
phase_auto_binding, phase_auto_binding,
check_dependencies_for_nodes,
check_dependency_satisfied, check_dependency_satisfied,
collect_dependency_errors, collect_dependency_errors,
) )
@@ -20,19 +18,16 @@ from .reset_ops import run_phase2_cleanup
__all__ = [ __all__ = [
"add_create_action", "add_create_action",
"run_create",
"finalize_peer_creating_sources", "finalize_peer_creating_sources",
"BoundNodePair", "BoundNodePair",
"add_update_action", "add_update_action",
"collect_bound_node_pairs", "collect_bound_node_pairs",
"prepare_directional_update", "prepare_directional_update",
"run_update",
"run_delete", "run_delete",
"run_bind",
"phase_core_state", "phase_core_state",
"phase_dependency_check", "phase_dependency_check",
"check_dependencies_for_nodes",
"phase_auto_binding", "phase_auto_binding",
"check_dependencies_for_nodes",
"check_dependency_satisfied", "check_dependency_satisfied",
"collect_dependency_errors", "collect_dependency_errors",
"run_phase2_cleanup", "run_phase2_cleanup",
@@ -1,23 +1,31 @@
from __future__ import annotations from __future__ import annotations
import logging from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Tuple from typing import TYPE_CHECKING, Any, Dict, List, Mapping, Optional, Set, Tuple
from ...common.types import BindingStatus from ...common.types import BindingStatus
from ...sync_system.config import OrphanAction from ...logging import get_logger
from ...sync_system.resolve_ids import IDResolver from ...sync_system.resolve_ids import IDResolver
from ...sync_system.utils import extract_biz_key_dict, dict_to_biz_key_tuple, resolve_id_fields_in_key_dict from ...sync_system.utils import extract_biz_key_dict, dict_to_biz_key_tuple, resolve_id_fields_in_key_dict
from ...engine import e10_bind_core, decision_note, e15_bind_dependency, e16_bind_auto from ...engine import e10_bind_core, decision_note, e15_bind_dependency, e16_bind_auto
if TYPE_CHECKING: if TYPE_CHECKING:
from ...common.binding import BindingManager
from ...common.collection import DataCollection from ...common.collection import DataCollection
from ...common.sync_node import SyncNode from ...common.sync_node import SyncNode
from ...config.strategy_config import StrategyConfig
from ...engine import StateMachineRuntime
logger = logging.getLogger(__name__) logger = get_logger(__name__)
def get_id_field_hints(strategy) -> Dict[str, str]: @dataclass(frozen=True)
return dict(strategy.config.depend_fields) if strategy.config.depend_fields else {} class PendingAutoBindUpdate:
node: "SyncNode"
outcome: Optional[str]
create_enabled: Optional[bool]
bind_remote_node_id: Optional[str] = None
detail: Optional[str] = None
def _stable_pick_bind_pair(local_candidates: List["SyncNode"], remote_candidates: List["SyncNode"]): def _stable_pick_bind_pair(local_candidates: List["SyncNode"], remote_candidates: List["SyncNode"]):
@@ -26,55 +34,118 @@ def _stable_pick_bind_pair(local_candidates: List["SyncNode"], remote_candidates
return local_node, remote_node return local_node, remote_node
async def run_bind(strategy, **kwargs): def collect_bind_entry_nodes(
logger.info(f"[{strategy.node_type}] Starting Binding Pipeline...") *,
node_type: str,
local_nodes: List["SyncNode"] = strategy.local_collection.filter_by_state_ids( local_collection: "DataCollection",
node_type=strategy.node_type, remote_collection: "DataCollection",
) -> tuple[List["SyncNode"], List["SyncNode"]]:
local_nodes = local_collection.filter_by_state_ids(
node_type=node_type,
state_ids=["S00"], state_ids=["S00"],
) )
remote_nodes: List["SyncNode"] = strategy.remote_collection.filter_by_state_ids( remote_nodes = remote_collection.filter_by_state_ids(
node_type=strategy.node_type, node_type=node_type,
state_ids=["S00"], state_ids=["S00"],
) )
return local_nodes, remote_nodes
await phase_core_state(strategy, local_nodes, remote_nodes)
await phase_dependency_check(strategy, local_nodes, remote_nodes)
await phase_auto_binding(strategy, local_nodes, remote_nodes)
await refresh_bind_data_id(strategy)
logger.info(f"[{strategy.node_type}] Binding Pipeline Completed.")
async def refresh_bind_data_id(strategy) -> None: def collect_auto_bind_ready_nodes(
binding_pairs = await strategy.binding_manager.get_all_bindings(strategy.node_type) *,
node_type: str,
local_collection: "DataCollection",
remote_collection: "DataCollection",
) -> tuple[List["SyncNode"], List["SyncNode"]]:
local_nodes = [
node
for node in local_collection.filter_by_state_ids(
node_type=node_type,
state_ids=["S02"],
)
if node.binding_status == BindingStatus.MISSING
]
remote_nodes = [
node
for node in remote_collection.filter_by_state_ids(
node_type=node_type,
state_ids=["S02"],
)
if node.binding_status == BindingStatus.MISSING
]
return local_nodes, remote_nodes
async def refresh_bind_data_id(
*,
node_type: str,
local_collection: "DataCollection",
remote_collection: "DataCollection",
binding_manager: "BindingManager",
local_node_ids: Optional[Set[str]] = None,
remote_node_ids: Optional[Set[str]] = None,
) -> None:
binding_pairs = await binding_manager.get_all_bindings(node_type)
local_to_remote = {local_id: remote_id for local_id, remote_id in binding_pairs if local_id} local_to_remote = {local_id: remote_id for local_id, remote_id in binding_pairs if local_id}
remote_to_local = {remote_id: local_id for local_id, remote_id in binding_pairs if local_id and remote_id} remote_to_local = {remote_id: local_id for local_id, remote_id in binding_pairs if local_id and remote_id}
for local_node in strategy.local_collection.filter(node_type=strategy.node_type): if local_node_ids is None:
target_local_node_ids = {node.node_id for node in local_collection.filter(node_type=node_type)}
else:
target_local_node_ids = {node_id for node_id in local_node_ids if node_id}
if remote_node_ids is None:
target_remote_node_ids = {node.node_id for node in remote_collection.filter(node_type=node_type)}
else:
target_remote_node_ids = {node_id for node_id in remote_node_ids if node_id}
for local_node_id in list(target_local_node_ids):
remote_node_id = local_to_remote.get(local_node_id)
if remote_node_id:
target_remote_node_ids.add(remote_node_id)
for remote_node_id in list(target_remote_node_ids):
local_node_id = remote_to_local.get(remote_node_id)
if local_node_id:
target_local_node_ids.add(local_node_id)
for local_node_id in target_local_node_ids:
local_node = local_collection.get_by_node_id(local_node_id)
if local_node is None:
continue
remote_node_id = local_to_remote.get(local_node.node_id) remote_node_id = local_to_remote.get(local_node.node_id)
remote_node = strategy.remote_collection.get_by_node_id(remote_node_id) if remote_node_id else None remote_node = remote_collection.get_by_node_id(remote_node_id) if remote_node_id else None
if remote_node and remote_node.data_id: if remote_node and remote_node.data_id:
local_node.context["bind_data_id"] = remote_node.data_id local_node.context["bind_data_id"] = remote_node.data_id
else: else:
local_node.context.pop("bind_data_id", None) local_node.context.pop("bind_data_id", None)
for remote_node in strategy.remote_collection.filter(node_type=strategy.node_type): for remote_node_id in target_remote_node_ids:
remote_node = remote_collection.get_by_node_id(remote_node_id)
if remote_node is None:
continue
local_node_id = remote_to_local.get(remote_node.node_id) local_node_id = remote_to_local.get(remote_node.node_id)
local_node = strategy.local_collection.get_by_node_id(local_node_id) if local_node_id else None local_node = local_collection.get_by_node_id(local_node_id) if local_node_id else None
if local_node and local_node.data_id: if local_node and local_node.data_id:
remote_node.context["bind_data_id"] = local_node.data_id remote_node.context["bind_data_id"] = local_node.data_id
else: else:
remote_node.context.pop("bind_data_id", None) remote_node.context.pop("bind_data_id", None)
async def phase_core_state(strategy, local_nodes: List["SyncNode"], remote_nodes: List["SyncNode"]): async def phase_core_state(
*,
node_type: str,
runtime: "StateMachineRuntime",
binding_manager: "BindingManager",
local_nodes: List["SyncNode"],
remote_nodes: List["SyncNode"],
) -> None:
remote_map = {n.node_id: n for n in remote_nodes} remote_map = {n.node_id: n for n in remote_nodes}
processed_local_ids: Set[str] = set() processed_local_ids: Set[str] = set()
processed_remote_ids: Set[str] = set() processed_remote_ids: Set[str] = set()
for node in local_nodes: for node in local_nodes:
remote_id = await strategy.binding_manager.get_remote_id(strategy.node_type, node.node_id) remote_id = await binding_manager.get_remote_id(node_type, node.node_id)
if remote_id: if remote_id:
remote_node = remote_map.get(remote_id) remote_node = remote_map.get(remote_id)
@@ -84,7 +155,7 @@ async def phase_core_state(strategy, local_nodes: List["SyncNode"], remote_nodes
local_valid = local_data is not None local_valid = local_data is not None
peer_valid = remote_data is not None peer_valid = remote_data is not None
local_decision = e10_bind_core( local_decision = e10_bind_core(
strategy.ensure_runtime(), runtime,
node=node, node=node,
has_binding_record=True, has_binding_record=True,
local_data=local_data, local_data=local_data,
@@ -94,14 +165,14 @@ async def phase_core_state(strategy, local_nodes: List["SyncNode"], remote_nodes
) )
local_reason = decision_note(local_decision) local_reason = decision_note(local_decision)
if local_decision.to_state in {"S03", "S04"}: if local_decision.to_state in {"S03", "S04"}:
logger.debug(f"[{strategy.node_type}] 核心绑定阻断: node={node.node_id}, reason={local_reason}") logger.debug(f"[{node_type}] 核心绑定阻断: node={node.node_id}, reason={local_reason}")
else: else:
logger.debug(f"[{strategy.node_type}] 核心绑定完成: node={node.node_id}, reason={local_reason}") logger.debug(f"[{node_type}] 核心绑定完成: node={node.node_id}, reason={local_reason}")
processed_local_ids.add(node.node_id) processed_local_ids.add(node.node_id)
if remote_node: if remote_node:
peer_decision = e10_bind_core( peer_decision = e10_bind_core(
strategy.ensure_runtime(), runtime,
node=remote_node, node=remote_node,
has_binding_record=True, has_binding_record=True,
local_data=remote_data, local_data=remote_data,
@@ -111,32 +182,22 @@ async def phase_core_state(strategy, local_nodes: List["SyncNode"], remote_nodes
) )
peer_reason = decision_note(peer_decision) peer_reason = decision_note(peer_decision)
if peer_decision.to_state in {"S03", "S04"}: if peer_decision.to_state in {"S03", "S04"}:
logger.debug( logger.debug(f"[{node_type}] 核心绑定阻断: node={remote_node.node_id}, reason={peer_reason}")
f"[{strategy.node_type}] 核心绑定阻断: node={remote_node.node_id}, reason={peer_reason}"
)
else: else:
logger.debug(f"[{strategy.node_type}] 核心绑定完成: node={remote_node.node_id}, reason={peer_reason}") logger.debug(f"[{node_type}] 核心绑定完成: node={remote_node.node_id}, reason={peer_reason}")
processed_remote_ids.add(remote_node.node_id) processed_remote_ids.add(remote_node.node_id)
for node in local_nodes: for node in local_nodes:
if node.node_id not in processed_local_ids: if node.node_id not in processed_local_ids:
decision = e10_bind_core( decision = e10_bind_core(runtime, node=node, has_binding_record=False)
strategy.ensure_runtime(),
node=node,
has_binding_record=False,
)
reason = decision_note(decision) reason = decision_note(decision)
logger.debug(f"[{strategy.node_type}] 核心绑定完成: node={node.node_id}, reason={reason}") logger.debug(f"[{node_type}] 核心绑定完成: node={node.node_id}, reason={reason}")
for r_node in remote_nodes: for remote_node in remote_nodes:
if r_node.node_id not in processed_remote_ids: if remote_node.node_id not in processed_remote_ids:
decision = e10_bind_core( decision = e10_bind_core(runtime, node=remote_node, has_binding_record=False)
strategy.ensure_runtime(),
node=r_node,
has_binding_record=False,
)
reason = decision_note(decision) reason = decision_note(decision)
logger.debug(f"[{strategy.node_type}] 核心绑定完成: node={r_node.node_id}, reason={reason}") logger.debug(f"[{node_type}] 核心绑定完成: node={remote_node.node_id}, reason={reason}")
def check_dependency_satisfied( def check_dependency_satisfied(
@@ -206,17 +267,45 @@ def collect_dependency_errors(
return errors return errors
async def phase_dependency_check(strategy, local_nodes: List["SyncNode"], remote_nodes: List["SyncNode"]): async def phase_dependency_check(
await check_dependencies_for_nodes(strategy, local_nodes, strategy.local_collection) *,
await check_dependencies_for_nodes(strategy, remote_nodes, strategy.remote_collection) node_type: str,
runtime: "StateMachineRuntime",
depend_fields: Mapping[str, str],
local_nodes: List["SyncNode"],
remote_nodes: List["SyncNode"],
local_collection: "DataCollection",
remote_collection: "DataCollection",
) -> None:
await check_dependencies_for_nodes(
node_type=node_type,
runtime=runtime,
depend_fields=depend_fields,
nodes=local_nodes,
collection=local_collection,
)
await check_dependencies_for_nodes(
node_type=node_type,
runtime=runtime,
depend_fields=depend_fields,
nodes=remote_nodes,
collection=remote_collection,
)
async def check_dependencies_for_nodes(strategy, nodes: List["SyncNode"], collection: "DataCollection"): async def check_dependencies_for_nodes(
*,
node_type: str,
runtime: "StateMachineRuntime",
depend_fields: Mapping[str, str],
nodes: List["SyncNode"],
collection: "DataCollection",
) -> None:
for node in nodes: for node in nodes:
if node.binding_status != BindingStatus.MISSING: if node.binding_status != BindingStatus.MISSING:
continue continue
has_depend_fields = bool(strategy.config.depend_fields) has_depend_fields = bool(depend_fields)
node_data = node.get_data() node_data = node.get_data()
data_present = bool(node_data) data_present = bool(node_data)
@@ -228,7 +317,7 @@ async def check_dependencies_for_nodes(strategy, nodes: List["SyncNode"], collec
semantic_errors: List[str] = [] semantic_errors: List[str] = []
if has_depend_fields and data_present: if has_depend_fields and data_present:
for field_name, dep_node_type in strategy.config.depend_fields.items(): for field_name, dep_node_type in depend_fields.items():
dep_data_values = _normalize_dependency_field_values(node_data.get(field_name)) dep_data_values = _normalize_dependency_field_values(node_data.get(field_name))
if not dep_data_values: if not dep_data_values:
continue continue
@@ -282,7 +371,7 @@ async def check_dependencies_for_nodes(strategy, nodes: List["SyncNode"], collec
) )
decision = e15_bind_dependency( decision = e15_bind_dependency(
strategy.ensure_runtime(), runtime,
node=node, node=node,
data_present=data_present, data_present=data_present,
has_depend_fields=has_depend_fields, has_depend_fields=has_depend_fields,
@@ -293,34 +382,77 @@ async def check_dependencies_for_nodes(strategy, nodes: List["SyncNode"], collec
reason = decision_note(decision) reason = decision_note(decision)
if decision.to_state in {"S03", "S04", "S05"}: if decision.to_state in {"S03", "S04", "S05"}:
detail_reason = node.error or reason detail_reason = node.error or reason
logger.debug(f"[{strategy.node_type}] 依赖检查阻断: node={node.node_id}, reason={detail_reason}") logger.debug(f"[{node_type}] 依赖检查阻断: node={node.node_id}, reason={detail_reason}")
async def phase_auto_binding(strategy, local_nodes: List["SyncNode"], remote_nodes: List["SyncNode"]): async def phase_auto_binding(
pending_auto_bind_updates: List[Dict[str, Any]] = [] *,
node_type: str,
runtime: "StateMachineRuntime",
config: "StrategyConfig",
local_collection: "DataCollection",
remote_collection: "DataCollection",
binding_manager: "BindingManager",
local_nodes: List["SyncNode"],
remote_nodes: List["SyncNode"],
id_field_hints: Mapping[str, str],
local_orphan_create_enabled_by_node: Mapping[str, bool],
remote_orphan_create_enabled_by_node: Mapping[str, bool],
) -> None:
pending_auto_bind_updates = await plan_auto_bind_updates(
node_type=node_type,
config=config,
local_collection=local_collection,
remote_collection=remote_collection,
binding_manager=binding_manager,
local_nodes=local_nodes,
remote_nodes=remote_nodes,
id_field_hints=id_field_hints,
local_orphan_create_enabled_by_node=local_orphan_create_enabled_by_node,
remote_orphan_create_enabled_by_node=remote_orphan_create_enabled_by_node,
)
await apply_auto_bind_updates(
node_type=node_type,
runtime=runtime,
binding_manager=binding_manager,
pending_auto_bind_updates=pending_auto_bind_updates,
)
async def plan_auto_bind_updates(
*,
node_type: str,
config: "StrategyConfig",
local_collection: "DataCollection",
remote_collection: "DataCollection",
binding_manager: "BindingManager",
local_nodes: List["SyncNode"],
remote_nodes: List["SyncNode"],
id_field_hints: Mapping[str, str],
local_orphan_create_enabled_by_node: Mapping[str, bool],
remote_orphan_create_enabled_by_node: Mapping[str, bool],
) -> List[PendingAutoBindUpdate]:
pending_auto_bind_updates: List[PendingAutoBindUpdate] = []
prebound_node_ids: Set[str] = set() prebound_node_ids: Set[str] = set()
local_s02_ids = { local_s02_ids = {
n.node_id n.node_id
for n in strategy.local_collection.filter_by_state_ids( for n in local_collection.filter_by_state_ids(
node_type=strategy.node_type, node_type=node_type,
state_ids=["S02"], state_ids=["S02"],
) )
} }
remote_s02_ids = { remote_s02_ids = {
n.node_id n.node_id
for n in strategy.remote_collection.filter_by_state_ids( for n in remote_collection.filter_by_state_ids(
node_type=strategy.node_type, node_type=node_type,
state_ids=["S02"], state_ids=["S02"],
) )
} }
local_create_enabled = strategy.config.local_orphan_action == OrphanAction.CREATE_REMOTE for prebind_pair in config.pre_bind_data_id:
remote_create_enabled = strategy.config.remote_orphan_action == OrphanAction.CREATE_LOCAL local_node = local_collection.get_by_data_id(node_type, prebind_pair.local_data_id)
remote_node = remote_collection.get_by_data_id(node_type, prebind_pair.remote_data_id)
for prebind_pair in strategy.config.pre_bind_data_id:
local_node = strategy.local_collection.get_by_data_id(strategy.node_type, prebind_pair.local_data_id)
remote_node = strategy.remote_collection.get_by_data_id(strategy.node_type, prebind_pair.remote_data_id)
if local_node is None or remote_node is None: if local_node is None or remote_node is None:
continue continue
@@ -329,325 +461,269 @@ async def phase_auto_binding(strategy, local_nodes: List["SyncNode"], remote_nod
if local_node.binding_status != BindingStatus.MISSING or remote_node.binding_status != BindingStatus.MISSING: if local_node.binding_status != BindingStatus.MISSING or remote_node.binding_status != BindingStatus.MISSING:
continue continue
existing_remote = await strategy.binding_manager.get_remote_id(strategy.node_type, local_node.node_id) existing_remote = await binding_manager.get_remote_id(node_type, local_node.node_id)
existing_local = await strategy.binding_manager.get_local_id(strategy.node_type, remote_node.node_id) existing_local = await binding_manager.get_local_id(node_type, remote_node.node_id)
if existing_remote or existing_local: if existing_remote or existing_local:
continue continue
pending_auto_bind_updates.append( pending_auto_bind_updates.append(
{ PendingAutoBindUpdate(
"node": local_node, node=local_node,
"outcome": "ONE_TO_ONE", outcome="ONE_TO_ONE",
"create_enabled": None, create_enabled=None,
"bind_remote_node_id": remote_node.node_id, bind_remote_node_id=remote_node.node_id,
} )
) )
pending_auto_bind_updates.append( pending_auto_bind_updates.append(
{ PendingAutoBindUpdate(node=remote_node, outcome="ONE_TO_ONE", create_enabled=None)
"node": remote_node,
"outcome": "ONE_TO_ONE",
"create_enabled": None,
"bind_remote_node_id": None,
}
) )
prebound_node_ids.add(local_node.node_id) prebound_node_ids.add(local_node.node_id)
prebound_node_ids.add(remote_node.node_id) prebound_node_ids.add(remote_node.node_id)
if not strategy.config.auto_bind or not strategy.config.auto_bind_fields: if not config.auto_bind or not config.auto_bind_fields:
for node in local_nodes: for node in local_nodes:
if node.node_id in prebound_node_ids: if node.node_id in prebound_node_ids:
continue continue
if node.binding_status == BindingStatus.MISSING: if node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append({"node": node, "outcome": None, "create_enabled": None}) pending_auto_bind_updates.append(PendingAutoBindUpdate(node=node, outcome=None, create_enabled=None))
for node in remote_nodes: for node in remote_nodes:
if node.node_id in prebound_node_ids: if node.node_id in prebound_node_ids:
continue continue
if node.binding_status == BindingStatus.MISSING: if node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append({"node": node, "outcome": None, "create_enabled": None}) pending_auto_bind_updates.append(PendingAutoBindUpdate(node=node, outcome=None, create_enabled=None))
else: return pending_auto_bind_updates
id_resolver = IDResolver(strategy.local_collection, strategy.remote_collection, strategy.binding_manager)
local_key_map: Dict[Tuple, List["SyncNode"]] = {} id_resolver = IDResolver(local_collection, remote_collection, binding_manager)
remote_key_map: Dict[Tuple, List["SyncNode"]] = {} local_key_map: Dict[Tuple, List["SyncNode"]] = {}
remote_key_map: Dict[Tuple, List["SyncNode"]] = {}
for node in local_nodes: async def _collect_side_key_map(
if node.node_id in prebound_node_ids: *,
continue nodes: List["SyncNode"],
if node.node_id not in local_s02_ids: s02_ids: Set[str],
key_map: Dict[Tuple, List["SyncNode"]],
source_is_local: bool,
) -> None:
for node in nodes:
if node.node_id in prebound_node_ids or node.node_id not in s02_ids:
continue continue
node_data = node.get_data() node_data = node.get_data()
if not node_data: if not node_data:
if node.binding_status == BindingStatus.MISSING: if node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append( pending_auto_bind_updates.append(
{ PendingAutoBindUpdate(node=node, outcome="DATA_MISSING", create_enabled=None)
"node": node,
"outcome": "DATA_MISSING",
"create_enabled": None,
"bind_remote_node_id": None,
}
) )
continue continue
missing_fields = [fn for fn in strategy.config.auto_bind_fields if fn not in node_data] missing_fields = [fn for fn in config.auto_bind_fields if fn not in node_data]
if missing_fields: if missing_fields:
if node.binding_status == BindingStatus.MISSING: if node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append( pending_auto_bind_updates.append(
{"node": node, "outcome": "KEY_MISSING", "create_enabled": None, "bind_remote_node_id": None} PendingAutoBindUpdate(node=node, outcome="KEY_MISSING", create_enabled=None)
) )
continue continue
key_dict = extract_biz_key_dict(node_data, strategy.config.auto_bind_fields) key_dict = extract_biz_key_dict(node_data, config.auto_bind_fields)
if not key_dict: if not key_dict:
continue continue
resolved_key_dict, resolve_error = await resolve_id_fields_in_key_dict( if source_is_local:
key_dict, key_dict, resolve_error = await resolve_id_fields_in_key_dict(
id_resolver, key_dict,
get_id_field_hints(strategy), id_resolver,
) dict(id_field_hints),
if not resolved_key_dict: )
if not key_dict:
if node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
PendingAutoBindUpdate(
node=node,
outcome="KEY_ID_RESOLVE_FAIL",
create_enabled=None,
detail=resolve_error,
)
)
continue
key = dict_to_biz_key_tuple(key_dict, config.auto_bind_fields)
key_map.setdefault(key, []).append(node)
await _collect_side_key_map(
nodes=local_nodes,
s02_ids=local_s02_ids,
key_map=local_key_map,
source_is_local=True,
)
await _collect_side_key_map(
nodes=remote_nodes,
s02_ids=remote_s02_ids,
key_map=remote_key_map,
source_is_local=False,
)
all_keys = set(local_key_map.keys()) | set(remote_key_map.keys())
for biz_key in all_keys:
local_candidates = local_key_map.get(biz_key, [])
remote_candidates = remote_key_map.get(biz_key, [])
excluded_local = set()
excluded_remote = set()
for l_node in local_candidates:
if l_node.binding_status != BindingStatus.NORMAL:
continue
remote_id = await binding_manager.get_remote_id(node_type, l_node.node_id)
if not remote_id:
continue
for r_node in remote_candidates:
if r_node.node_id == remote_id and r_node.binding_status == BindingStatus.NORMAL:
excluded_local.add(l_node.node_id)
excluded_remote.add(r_node.node_id)
break
l_remain = [n for n in local_candidates if n.node_id not in excluded_local]
r_remain = [n for n in remote_candidates if n.node_id not in excluded_remote]
if len(l_remain) == 1 and len(r_remain) == 1:
l_node = l_remain[0]
r_node = r_remain[0]
if l_node.binding_status == BindingStatus.MISSING and r_node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
PendingAutoBindUpdate(
node=l_node,
outcome="ONE_TO_ONE",
create_enabled=None,
bind_remote_node_id=r_node.node_id,
)
)
pending_auto_bind_updates.append(
PendingAutoBindUpdate(node=r_node, outcome="ONE_TO_ONE", create_enabled=None)
)
continue
if len(l_remain) >= 1 and len(r_remain) == 0:
for node in l_remain:
if node.binding_status == BindingStatus.MISSING: if node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append( pending_auto_bind_updates.append(
{ PendingAutoBindUpdate(
"node": node, node=node,
"outcome": "KEY_ID_RESOLVE_FAIL", outcome="ZERO",
"create_enabled": None, create_enabled=bool(local_orphan_create_enabled_by_node.get(node.node_id, False)),
"bind_remote_node_id": None, )
"detail": resolve_error,
}
) )
continue
key = dict_to_biz_key_tuple(resolved_key_dict, strategy.config.auto_bind_fields) if len(r_remain) >= 1 and len(l_remain) == 0:
local_key_map.setdefault(key, []).append(node) for node in r_remain:
if node.binding_status == BindingStatus.MISSING:
for r_node in remote_nodes:
if r_node.node_id in prebound_node_ids:
continue
if r_node.node_id not in remote_s02_ids:
continue
r_node_data = r_node.get_data()
if not r_node_data:
if r_node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append( pending_auto_bind_updates.append(
{ PendingAutoBindUpdate(
"node": r_node, node=node,
"outcome": "DATA_MISSING", outcome="ZERO",
"create_enabled": None, create_enabled=bool(remote_orphan_create_enabled_by_node.get(node.node_id, False)),
"bind_remote_node_id": None, )
}
) )
continue
missing_fields = [fn for fn in strategy.config.auto_bind_fields if fn not in r_node_data] many_local_one_remote = len(l_remain) > 1 and len(r_remain) == 1
if missing_fields: one_local_many_remote = len(l_remain) == 1 and len(r_remain) > 1
if r_node.binding_status == BindingStatus.MISSING: if config.allow_many_to_one_auto_bind and (many_local_one_remote or one_local_many_remote):
pending_auto_bind_updates.append( chosen_local, chosen_remote = _stable_pick_bind_pair(l_remain, r_remain)
{ if chosen_local.binding_status == BindingStatus.MISSING:
"node": r_node, pending_auto_bind_updates.append(
"outcome": "KEY_MISSING", PendingAutoBindUpdate(
"create_enabled": None, node=chosen_local,
"bind_remote_node_id": None, outcome="ONE_TO_ONE",
} create_enabled=None,
bind_remote_node_id=chosen_remote.node_id,
) )
continue )
if chosen_remote.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
PendingAutoBindUpdate(node=chosen_remote, outcome="ONE_TO_ONE", create_enabled=None)
)
key_dict = extract_biz_key_dict(r_node_data, strategy.config.auto_bind_fields) for node in l_remain:
if key_dict: if node.node_id == chosen_local.node_id or node.binding_status != BindingStatus.MISSING:
key = dict_to_biz_key_tuple(key_dict, strategy.config.auto_bind_fields)
remote_key_map.setdefault(key, []).append(r_node)
all_keys = set(local_key_map.keys()) | set(remote_key_map.keys())
for biz_key in all_keys:
local_candidates = local_key_map.get(biz_key, [])
remote_candidates = remote_key_map.get(biz_key, [])
excluded_local = set()
excluded_remote = set()
for l_node in local_candidates:
if l_node.binding_status != BindingStatus.NORMAL:
continue continue
pending_auto_bind_updates.append(
remote_id = await strategy.binding_manager.get_remote_id(strategy.node_type, l_node.node_id) PendingAutoBindUpdate(
if not remote_id: node=node,
outcome="ZERO",
create_enabled=bool(local_orphan_create_enabled_by_node.get(node.node_id, False)),
)
)
for node in r_remain:
if node.node_id == chosen_remote.node_id or node.binding_status != BindingStatus.MISSING:
continue continue
pending_auto_bind_updates.append(
for r_node in remote_candidates: PendingAutoBindUpdate(
if r_node.node_id == remote_id and r_node.binding_status == BindingStatus.NORMAL: node=node,
excluded_local.add(l_node.node_id) outcome="ZERO",
excluded_remote.add(r_node.node_id) create_enabled=bool(remote_orphan_create_enabled_by_node.get(node.node_id, False)),
break
l_remain = [n for n in local_candidates if n.node_id not in excluded_local]
r_remain = [n for n in remote_candidates if n.node_id not in excluded_remote]
if len(l_remain) == 1 and len(r_remain) == 1:
l_node = l_remain[0]
r_node = r_remain[0]
if l_node.binding_status == BindingStatus.MISSING and r_node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
{
"node": l_node,
"outcome": "ONE_TO_ONE",
"create_enabled": None,
"bind_remote_node_id": r_node.node_id,
}
) )
)
continue
if (len(l_remain) > 1 and len(r_remain) >= 1) or (len(l_remain) == 1 and len(r_remain) > 1):
for node in l_remain:
if node.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append( pending_auto_bind_updates.append(
{ PendingAutoBindUpdate(node=node, outcome="AMBIGUOUS", create_enabled=None)
"node": r_node, )
"outcome": "ONE_TO_ONE", if (len(r_remain) > 1 and len(l_remain) >= 1) or (len(r_remain) == 1 and len(l_remain) > 1):
"create_enabled": None, for node in r_remain:
"bind_remote_node_id": None, if node.binding_status == BindingStatus.MISSING:
} pending_auto_bind_updates.append(
PendingAutoBindUpdate(node=node, outcome="AMBIGUOUS", create_enabled=None)
) )
else:
if len(l_remain) >= 1 and len(r_remain) == 0:
for n in l_remain:
if n.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
{
"node": n,
"outcome": "ZERO",
"create_enabled": local_create_enabled,
"bind_remote_node_id": None,
}
)
if len(r_remain) >= 1 and len(l_remain) == 0: return pending_auto_bind_updates
for n in r_remain:
if n.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
{
"node": n,
"outcome": "ZERO",
"create_enabled": remote_create_enabled,
"bind_remote_node_id": None,
}
)
many_local_one_remote = len(l_remain) > 1 and len(r_remain) == 1
one_local_many_remote = len(l_remain) == 1 and len(r_remain) > 1
if strategy.config.allow_many_to_one_auto_bind and (many_local_one_remote or one_local_many_remote):
chosen_local, chosen_remote = _stable_pick_bind_pair(l_remain, r_remain)
if chosen_local.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
{
"node": chosen_local,
"outcome": "ONE_TO_ONE",
"create_enabled": None,
"bind_remote_node_id": chosen_remote.node_id,
}
)
if chosen_remote.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
{
"node": chosen_remote,
"outcome": "ONE_TO_ONE",
"create_enabled": None,
"bind_remote_node_id": None,
}
)
for n in l_remain:
if n.node_id == chosen_local.node_id or n.binding_status != BindingStatus.MISSING:
continue
pending_auto_bind_updates.append(
{
"node": n,
"outcome": "ZERO",
"create_enabled": local_create_enabled,
"bind_remote_node_id": None,
}
)
for n in r_remain:
if n.node_id == chosen_remote.node_id or n.binding_status != BindingStatus.MISSING:
continue
pending_auto_bind_updates.append(
{
"node": n,
"outcome": "ZERO",
"create_enabled": remote_create_enabled,
"bind_remote_node_id": None,
}
)
else:
if (len(l_remain) > 1 and len(r_remain) >= 1) or (len(l_remain) == 1 and len(r_remain) > 1):
for n in l_remain:
if n.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
{
"node": n,
"outcome": "AMBIGUOUS",
"create_enabled": None,
"bind_remote_node_id": None,
}
)
if (len(r_remain) > 1 and len(l_remain) >= 1) or (len(r_remain) == 1 and len(l_remain) > 1):
for n in r_remain:
if n.binding_status == BindingStatus.MISSING:
pending_auto_bind_updates.append(
{
"node": n,
"outcome": "AMBIGUOUS",
"create_enabled": None,
"bind_remote_node_id": None,
}
)
async def apply_auto_bind_updates(
*,
node_type: str,
runtime: "StateMachineRuntime",
binding_manager: "BindingManager",
pending_auto_bind_updates: List[PendingAutoBindUpdate],
) -> None:
for item in pending_auto_bind_updates: for item in pending_auto_bind_updates:
node = item["node"] auto_bind_enabled = item.outcome is not None
auto_bind_outcome = item["outcome"]
create_enabled = item["create_enabled"]
bind_remote_node_id = item.get("bind_remote_node_id")
detail = item.get("detail")
auto_bind_enabled = auto_bind_outcome is not None
decision = e16_bind_auto( decision = e16_bind_auto(
strategy.ensure_runtime(), runtime,
node=node, node=item.node,
auto_bind_enabled=auto_bind_enabled, auto_bind_enabled=auto_bind_enabled,
auto_bind_outcome=auto_bind_outcome, auto_bind_outcome=item.outcome,
create_enabled=create_enabled, create_enabled=item.create_enabled,
id_resolve_detail=detail, id_resolve_detail=item.detail,
) )
reason = decision_note(decision) reason = decision_note(decision)
if decision.to_state == "S01" and auto_bind_outcome == "ONE_TO_ONE": if decision.to_state == "S01" and item.outcome == "ONE_TO_ONE":
if bind_remote_node_id: if item.bind_remote_node_id:
await strategy.binding_manager.bind( await binding_manager.bind(node_type, item.node.node_id, item.bind_remote_node_id)
strategy.node_type, logger.debug(f"[{node_type}] 自动绑定成功: node={item.node.node_id}")
node.node_id,
bind_remote_node_id,
)
logger.debug(f"[{strategy.node_type}] 自动绑定成功: node={node.node_id}")
if decision.to_state in {"S04", "S05"}: if decision.to_state in {"S04", "S05"}:
if detail: if item.detail:
reason = f"{reason} | detail={detail}" reason = f"{reason} | detail={item.detail}"
node.error = reason item.node.error = reason
logger.debug(f"[{strategy.node_type}] 自动绑定阻断: node={node.node_id}, reason={reason}") logger.debug(f"[{node_type}] 自动绑定阻断: node={item.node.node_id}, reason={reason}")
outcome_counts: Dict[str, int] = {} outcome_counts: Dict[str, int] = {}
create_enabled_count = 0 create_enabled_count = 0
bound_pairs = 0 bound_pairs = 0
for item in pending_auto_bind_updates: for item in pending_auto_bind_updates:
outcome = str(item.get("outcome") or "NONE") outcome = str(item.outcome or "NONE")
outcome_counts[outcome] = outcome_counts.get(outcome, 0) + 1 outcome_counts[outcome] = outcome_counts.get(outcome, 0) + 1
if item.get("create_enabled"): if item.create_enabled:
create_enabled_count += 1 create_enabled_count += 1
if item.get("bind_remote_node_id"): if item.bind_remote_node_id:
bound_pairs += 1 bound_pairs += 1
if pending_auto_bind_updates: if pending_auto_bind_updates:
ordered = ", ".join(f"{key}={outcome_counts[key]}" for key in sorted(outcome_counts)) ordered = ", ".join(f"{key}={outcome_counts[key]}" for key in sorted(outcome_counts))
logger.info( logger.info(
"[%s] Auto-bind summary: candidates=%d, bind_pairs=%d, create_enabled=%d, outcomes={%s}", "[%s] Auto-bind summary: candidates=%d, bind_pairs=%d, create_enabled=%d, outcomes={%s}",
strategy.node_type, node_type,
len(pending_auto_bind_updates), len(pending_auto_bind_updates),
bound_pairs, bound_pairs,
create_enabled_count, create_enabled_count,
@@ -113,40 +113,6 @@ async def add_create_action(
return created_nodes return created_nodes
async def run_create(strategy) -> List["SyncNode"]:
created_nodes: List["SyncNode"] = []
if strategy.config.local_orphan_action == OrphanAction.CREATE_REMOTE:
local_nodes = await add_create_action(
strategy,
from_collection=strategy.local_collection,
to_collection=strategy.remote_collection,
source_is_local=True,
)
created_nodes.extend(local_nodes)
logger.info(f"[{strategy.node_type}] Created {len(local_nodes)} nodes in remote collection (Push)")
elif strategy.config.local_orphan_action == OrphanAction.DELETE_LOCAL:
raise NotImplementedError(
f"[{strategy.node_type}] local_orphan_action=DELETE_LOCAL is not supported in run_create; "
"use delete stage (delete_ops) explicitly."
)
if strategy.config.remote_orphan_action == OrphanAction.CREATE_LOCAL:
remote_nodes = await add_create_action(
strategy,
from_collection=strategy.remote_collection,
to_collection=strategy.local_collection,
source_is_local=False,
)
created_nodes.extend(remote_nodes)
logger.info(f"[{strategy.node_type}] Created {len(remote_nodes)} nodes in local collection (Pull)")
elif strategy.config.remote_orphan_action == OrphanAction.DELETE_REMOTE:
raise NotImplementedError(
f"[{strategy.node_type}] remote_orphan_action=DELETE_REMOTE is not supported in run_create; "
"use delete stage (delete_ops) explicitly."
)
return created_nodes
async def finalize_peer_creating_sources( async def finalize_peer_creating_sources(
@@ -4,6 +4,7 @@ import logging
from typing import TYPE_CHECKING, Any, Dict, List, Optional from typing import TYPE_CHECKING, Any, Dict, List, Optional
from ...common.types import BindingStatus, SyncAction, SyncStatus from ...common.types import BindingStatus, SyncAction, SyncStatus
from ...logging import get_logger
from ...engine import e01_bootstrap from ...engine import e01_bootstrap
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -11,7 +12,7 @@ if TYPE_CHECKING:
from ...common.collection import DataCollection from ...common.collection import DataCollection
from ...engine.dispatcher import StateMachineRuntime from ...engine.dispatcher import StateMachineRuntime
logger = logging.getLogger(__name__) logger = get_logger(__name__)
async def _resolve_local_id_for_unbind( async def _resolve_local_id_for_unbind(
@@ -6,6 +6,7 @@ import logging
from typing import TYPE_CHECKING, Any, Iterable, List, Optional, Sequence from typing import TYPE_CHECKING, Any, Iterable, List, Optional, Sequence
from ...common.types import SyncAction from ...common.types import SyncAction
from ...logging import get_logger
from ...sync_system.config import UpdateDirection from ...sync_system.config import UpdateDirection
from ...sync_system.resolve_ids import IDResolver from ...sync_system.resolve_ids import IDResolver
from ...engine import decision_note, e30_update_prepare from ...engine import decision_note, e30_update_prepare
@@ -20,7 +21,7 @@ if TYPE_CHECKING:
from ...common.collection import DataCollection from ...common.collection import DataCollection
from ...common.sync_node import SyncNode from ...common.sync_node import SyncNode
logger = logging.getLogger(__name__) logger = get_logger(__name__)
def _build_schema_diff_validator(strategy) -> SchemaDiffValidator: def _build_schema_diff_validator(strategy) -> SchemaDiffValidator:
@@ -396,23 +397,3 @@ async def add_update_action(
return updated_nodes return updated_nodes
async def run_update(strategy) -> List["SyncNode"]:
updated_by_id: dict[str, "SyncNode"] = {}
validator = _build_schema_diff_validator(strategy)
strategy._active_schema_diff_validator = validator
data_id_map = await build_data_id_normalization_map(
node_type=strategy.node_type,
local_collection=strategy.local_collection,
remote_collection=strategy.remote_collection,
binding_manager=strategy.binding_manager,
)
try:
for pair in await collect_bound_node_pairs(strategy):
nodes = await strategy.update_pair(pair.local_node, pair.remote_node, data_id_map)
for node in nodes:
updated_by_id[node.node_id] = node
finally:
strategy._active_schema_diff_validator = None
emit_schema_diff_report(strategy, validator)
return list(updated_by_id.values())
+3 -1
View File
@@ -6,7 +6,9 @@
from typing import List, Tuple, Optional, Any, Dict from typing import List, Tuple, Optional, Any, Dict
import logging import logging
logger = logging.getLogger(__name__) from ..logging import get_logger
logger = get_logger(__name__)
def is_id_field(field_name: str) -> bool: def is_id_field(field_name: str) -> bool:
@@ -225,6 +225,8 @@ async def test_full_pipeline_multinode_mock_covers_business_points(pipeline_bund
assert local_collection.get("lp_norm").context.get("bind_data_id") == remote_collection.get("rp_norm").data_id assert local_collection.get("lp_norm").context.get("bind_data_id") == remote_collection.get("rp_norm").data_id
assert remote_collection.get("rp_norm").context.get("bind_data_id") == local_collection.get("lp_norm").data_id assert remote_collection.get("rp_norm").context.get("bind_data_id") == local_collection.get("lp_norm").data_id
assert local_collection.get("lp_auto").context.get("bind_data_id") == remote_collection.get("rp_auto").data_id
assert remote_collection.get("rp_auto").context.get("bind_data_id") == local_collection.get("lp_auto").data_id
assert await binding_manager.get_remote_id("test_project", "lp_auto") == "rp_auto" assert await binding_manager.get_remote_id("test_project", "lp_auto") == "rp_auto"
@@ -242,6 +244,12 @@ async def test_full_pipeline_multinode_mock_covers_business_points(pipeline_bund
assert create_fail_nodes[0].status == SyncStatus.FAILED assert create_fail_nodes[0].status == SyncStatus.FAILED
assert create_async_nodes[0].status == SyncStatus.PENDING assert create_async_nodes[0].status == SyncStatus.PENDING
assert create_async_nodes[0].action == SyncAction.NONE assert create_async_nodes[0].action == SyncAction.NONE
assert local_collection.get("lp_create_ok").context.get("bind_data_id") == create_ok_nodes[0].data_id
assert create_ok_nodes[0].context.get("bind_data_id") == local_collection.get("lp_create_ok").data_id
assert local_collection.get("lp_create_async").context.get("bind_data_id") == create_async_nodes[0].data_id
assert create_async_nodes[0].context.get("bind_data_id") == local_collection.get("lp_create_async").data_id
assert "bind_data_id" not in local_collection.get("lp_create_fail").context
assert "bind_data_id" not in create_fail_nodes[0].context
assert remote_collection.get("rc_upd_skip").status == SyncStatus.SKIPPED assert remote_collection.get("rc_upd_skip").status == SyncStatus.SKIPPED
assert remote_collection.get("rc_upd_skip").action == SyncAction.NONE assert remote_collection.get("rc_upd_skip").action == SyncAction.NONE
@@ -100,9 +100,11 @@ async def test_core_binding_matrix_migrated(
await bm.bind("test_project", f"{name}_l", f"{name}_r") await bm.bind("test_project", f"{name}_l", f"{name}_r")
await phase_core_state( await phase_core_state(
strategy, node_type=strategy.node_type,
local.filter(node_type="test_project"), runtime=strategy.ensure_runtime(),
remote.filter(node_type="test_project"), binding_manager=bm,
local_nodes=local.filter(node_type="test_project"),
remote_nodes=remote.filter(node_type="test_project"),
) )
if expected_local is not None: if expected_local is not None:
@@ -123,9 +125,11 @@ async def test_dependency_resolution_migrated(setup_env):
await bm.bind("test_project", "proj_l", "proj_r") await bm.bind("test_project", "proj_l", "proj_r")
await phase_core_state( await phase_core_state(
project_strategy, node_type=project_strategy.node_type,
local.filter(node_type="test_project"), runtime=project_strategy.ensure_runtime(),
remote.filter(node_type="test_project"), binding_manager=bm,
local_nodes=local.filter(node_type="test_project"),
remote_nodes=remote.filter(node_type="test_project"),
) )
assert local.get("proj_l").binding_status == BindingStatus.NORMAL assert local.get("proj_l").binding_status == BindingStatus.NORMAL
@@ -159,8 +163,22 @@ async def test_dependency_resolution_migrated(setup_env):
local_contracts = local.filter(node_type="test_contract") local_contracts = local.filter(node_type="test_contract")
remote_contracts = remote.filter(node_type="test_contract") remote_contracts = remote.filter(node_type="test_contract")
await phase_core_state(contract_strategy, local_contracts, remote_contracts) await phase_core_state(
await phase_dependency_check(contract_strategy, local_contracts, remote_contracts) node_type=contract_strategy.node_type,
runtime=contract_strategy.ensure_runtime(),
binding_manager=bm,
local_nodes=local_contracts,
remote_nodes=remote_contracts,
)
await phase_dependency_check(
node_type=contract_strategy.node_type,
runtime=contract_strategy.ensure_runtime(),
depend_fields=dict(contract_strategy.config.depend_fields),
local_nodes=local_contracts,
remote_nodes=remote_contracts,
local_collection=local,
remote_collection=remote,
)
assert local.get("contract_ok").binding_status == BindingStatus.MISSING assert local.get("contract_ok").binding_status == BindingStatus.MISSING
assert local.get("contract_ok").depend_ids == ["proj_l"] assert local.get("contract_ok").depend_ids == ["proj_l"]
@@ -269,8 +287,22 @@ async def test_dependency_check_data_missing_goes_abnormal(setup_env):
) )
local_contracts = local.filter(node_type="test_contract") local_contracts = local.filter(node_type="test_contract")
await phase_core_state(strategy, local_contracts, []) await phase_core_state(
await phase_dependency_check(strategy, local_contracts, []) node_type=strategy.node_type,
runtime=strategy.ensure_runtime(),
binding_manager=bm,
local_nodes=local_contracts,
remote_nodes=[],
)
await phase_dependency_check(
node_type=strategy.node_type,
runtime=strategy.ensure_runtime(),
depend_fields=dict(strategy.config.depend_fields),
local_nodes=local_contracts,
remote_nodes=[],
local_collection=local,
remote_collection=remote,
)
assert local.get("contract_no_data").binding_status == BindingStatus.ABNORMAL assert local.get("contract_no_data").binding_status == BindingStatus.ABNORMAL
@@ -306,8 +338,31 @@ async def test_auto_binding_key_id_resolve_fail_migrated(setup_env):
await local.add(build_project_node("kid_l", "KID-L", name="KID", code="KID-1")) await local.add(build_project_node("kid_l", "KID-L", name="KID", code="KID-1"))
local_nodes = local.filter(node_type="test_project") local_nodes = local.filter(node_type="test_project")
await phase_core_state(strategy, local_nodes, []) await phase_core_state(
await phase_auto_binding(strategy, local_nodes, []) node_type=strategy.node_type,
runtime=strategy.ensure_runtime(),
binding_manager=bm,
local_nodes=local_nodes,
remote_nodes=[],
)
local_flags = {
node.node_id: strategy.config.local_orphan_action == OrphanAction.CREATE_REMOTE
for node in local_nodes
}
remote_flags = {}
await phase_auto_binding(
node_type=strategy.node_type,
runtime=strategy.ensure_runtime(),
config=strategy.config,
local_collection=local,
remote_collection=remote,
binding_manager=bm,
local_nodes=local_nodes,
remote_nodes=[],
id_field_hints=dict(strategy.config.depend_fields),
local_orphan_create_enabled_by_node=local_flags,
remote_orphan_create_enabled_by_node=remote_flags,
)
node = local.get("kid_l") node = local.get("kid_l")
assert node.binding_status == BindingStatus.DEPENDENCY_ERROR assert node.binding_status == BindingStatus.DEPENDENCY_ERROR
+2 -2
View File
@@ -110,7 +110,7 @@ async def test_api_client_logs_one_line_request_summary_in_debug_mode(caplog: py
fake_client = _FakeAsyncClient() fake_client = _FakeAsyncClient()
client._client = fake_client client._client = fake_client
with caplog.at_level(logging.DEBUG, logger=logger_name): with caplog.at_level(logging.INFO, logger=logger_name):
await client.request("GET", "/ping") await client.request("GET", "/ping")
assert "ApiClient request: method=GET endpoint=/ping elapsed=" in caplog.text assert "🔎 ApiClient request: method=GET endpoint=/ping elapsed=" in caplog.text
@@ -20,7 +20,7 @@ class _DS(BaseDataSource):
class _FailStrategy(DefaultSyncStrategy): class _FailStrategy(DefaultSyncStrategy):
schema = object # type: ignore schema = object # type: ignore
async def bind(self, **kwargs): async def bind(self):
raise RuntimeError("bind failed") raise RuntimeError("bind failed")
async def create(self): async def create(self):
@@ -39,7 +39,7 @@ class _OkStrategy(DefaultSyncStrategy):
self.create_calls = 0 self.create_calls = 0
self.update_calls = 0 self.update_calls = 0
async def bind(self, **kwargs): async def bind(self):
self.bound = True self.bound = True
async def create(self): async def create(self):
+96 -17
View File
@@ -1,26 +1,62 @@
from types import SimpleNamespace from types import SimpleNamespace
from typing import cast
from unittest.mock import AsyncMock, MagicMock, patch from unittest.mock import AsyncMock, MagicMock, patch
import pytest import pytest
import pytest_asyncio
import sync_state_machine.domain # noqa: F401
from sync_state_machine.common.binding import BindingManager
from sync_state_machine.common.collection import DataCollection
from sync_state_machine.common.sync_node import SyncNode
from sync_state_machine.common.sqlite_backend import SQLitePersistenceBackend
from sync_state_machine.common.types import BindingStatus
from sync_state_machine.domain.project_detail.sync_strategy import ProjectDetailSyncStrategy from sync_state_machine.domain.project_detail.sync_strategy import ProjectDetailSyncStrategy
from sync_state_machine.domain.project_detail.sync_node import ProjectDetailSyncNode
from sync_state_machine.sync_system.config import UpdateDirection from sync_state_machine.sync_system.config import UpdateDirection
from sync_state_machine.sync_system.strategy_ops import BoundNodePair from sync_state_machine.sync_system.strategy_ops import BoundNodePair
def _fake_node(*, node_id: str, key: str): def _fake_node(*, node_id: str, key: str):
data = {"id": node_id, "key": key} data = {"id": node_id, "key": key}
return SimpleNamespace( return cast(
node_id=node_id, SyncNode,
data_id=node_id, SimpleNamespace(
action=None, node_id=node_id,
context={}, data_id=node_id,
get_data=lambda: data, action=None,
context={},
get_data=lambda: data,
),
) )
def _build_project_detail_node(*, node_id: str, data_id: str, key: str) -> ProjectDetailSyncNode:
return ProjectDetailSyncNode(
node_id=node_id,
data_id=data_id,
data={
"id": data_id,
"project_id": "project-1",
"key": key,
"value": [{"year": 2026, "capacity": 4.0}],
},
)
@pytest_asyncio.fixture
async def project_detail_env(tmp_path):
persistence = SQLitePersistenceBackend(str(tmp_path / "project-detail.db"))
await persistence.initialize()
local = DataCollection("local", persistence=persistence, auto_persist=False)
remote = DataCollection("remote", persistence=persistence, auto_persist=False)
binding_manager = BindingManager(persistence, auto_persist=False)
yield local, remote, binding_manager, persistence
await persistence.close()
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_project_detail_update_partitions_group_production_plans_before_generic_logic(): async def test_project_detail_process_update_pairs_partitions_group_production_plans_before_generic_logic():
strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock()) strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock())
pull_pair = BoundNodePair( pull_pair = BoundNodePair(
@@ -35,13 +71,6 @@ async def test_project_detail_update_partitions_group_production_plans_before_ge
generic_node = SimpleNamespace(node_id="generic-update") generic_node = SimpleNamespace(node_id="generic-update")
with patch.object( with patch.object(
strategy,
"collect_update_pairs",
new=AsyncMock(return_value=[pull_pair, generic_pair]),
), patch(
"sync_state_machine.domain.project_detail.sync_strategy.build_data_id_normalization_map",
new=AsyncMock(return_value={}),
), patch.object(
strategy, strategy,
"prepare_update_for_direction", "prepare_update_for_direction",
new=AsyncMock(return_value=pull_node), new=AsyncMock(return_value=pull_node),
@@ -50,9 +79,12 @@ async def test_project_detail_update_partitions_group_production_plans_before_ge
"update_pair", "update_pair",
new=AsyncMock(return_value=[generic_node]), new=AsyncMock(return_value=[generic_node]),
) as mock_update_pair: ) as mock_update_pair:
updated_nodes = await strategy.update() updated_by_id = await strategy.process_update_pairs([pull_pair, generic_pair], {})
assert updated_nodes == [pull_node, generic_node] assert updated_by_id == {
"pull-update": pull_node,
"generic-update": generic_node,
}
mock_prepare.assert_awaited_once_with( mock_prepare.assert_awaited_once_with(
pull_pair.local_node, pull_pair.local_node,
pull_pair.remote_node, pull_pair.remote_node,
@@ -65,4 +97,51 @@ async def test_project_detail_update_partitions_group_production_plans_before_ge
def test_project_detail_domain_option_is_typed_schema() -> None: def test_project_detail_domain_option_is_typed_schema() -> None:
strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock()) strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock())
assert strategy.domain_option.pull_group_production_plans is True assert strategy.domain_option.pull_group_production_plans is True
@pytest.mark.asyncio
async def test_project_detail_bind_create_pulls_group_production_plan_from_remote(project_detail_env) -> None:
local, remote, binding_manager, _ = project_detail_env
strategy = ProjectDetailSyncStrategy("project_detail_data", local, remote, binding_manager)
strategy.config.depend_fields = {}
remote_node = _build_project_detail_node(
node_id="remote-pull",
data_id="detail-remote-pull",
key="ensure_production",
)
await remote.add(remote_node)
await strategy.bind()
created_nodes = await strategy.create()
assert len(created_nodes) == 1
created_node = created_nodes[0]
created_data = created_node.get_data()
assert created_node.node_type == "project_detail_data"
assert created_data is not None
assert created_data["key"] == "ensure_production"
assert remote_node.binding_status == BindingStatus.PEER_CREATING
assert await binding_manager.get_local_id("project_detail_data", remote_node.node_id) == created_node.node_id
@pytest.mark.asyncio
async def test_project_detail_bind_does_not_push_group_production_plan_from_local(project_detail_env) -> None:
local, remote, binding_manager, _ = project_detail_env
strategy = ProjectDetailSyncStrategy("project_detail_data", local, remote, binding_manager)
strategy.config.depend_fields = {}
local_node = _build_project_detail_node(
node_id="local-pull",
data_id="detail-local-pull",
key="ensure_production",
)
await local.add(local_node)
await strategy.bind()
created_nodes = await strategy.create()
assert created_nodes == []
assert remote.filter(node_type="project_detail_data") == []
assert await binding_manager.get_remote_id("project_detail_data", local_node.node_id) is None
+49 -12
View File
@@ -2,34 +2,32 @@ import pytest
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import AsyncMock, patch, MagicMock from unittest.mock import AsyncMock, patch, MagicMock
from sync_state_machine.domain.company.sync_strategy import CompanySyncStrategy from sync_state_machine.domain.company.sync_strategy import CompanySyncStrategy
from sync_state_machine.sync_system.strategy_ops.update_ops import BoundNodePair, build_merged_update_payload, run_update from sync_state_machine.sync_system.strategy_ops.update_ops import BoundNodePair, build_merged_update_payload
from sync_state_machine.domain.construction.sync_strategy import ConstructionTaskSyncStrategy from sync_state_machine.domain.construction.sync_strategy import ConstructionTaskSyncStrategy
from sync_state_machine.domain.project_detail.sync_strategy import ProjectDetailSyncStrategy from sync_state_machine.domain.project_detail.sync_strategy import ProjectDetailSyncStrategy
from sync_state_machine.sync_system.config import UpdateDirection
from schemas.project_extensions.project_detail import ProjectDetailKey from schemas.project_extensions.project_detail import ProjectDetailKey
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_run_update_delegates_to_strategy_pair_planner(): async def test_default_strategy_update_orchestrates_pairs_in_strategy_layer():
local_node = SimpleNamespace(node_id="local-1") local_node = SimpleNamespace(node_id="local-1")
remote_node = SimpleNamespace(node_id="remote-1") remote_node = SimpleNamespace(node_id="remote-1")
strategy = MagicMock() strategy = CompanySyncStrategy("company", MagicMock(), MagicMock(), MagicMock())
strategy.node_type = "TestNode" strategy.ensure_runtime = MagicMock()
strategy.local_collection = MagicMock()
strategy.remote_collection = MagicMock()
strategy.binding_manager = MagicMock()
strategy.reset_schema_diff_validator = MagicMock()
strategy.emit_schema_diff_report = MagicMock()
updated_node = SimpleNamespace(node_id="target-1") updated_node = SimpleNamespace(node_id="target-1")
strategy.update_pair = AsyncMock(return_value=[updated_node]) strategy.update_pair = AsyncMock(return_value=[updated_node])
with patch( with patch(
'sync_state_machine.sync_system.strategy_ops.update_ops.build_data_id_normalization_map', 'sync_state_machine.sync_system.strategy.build_data_id_normalization_map',
new=AsyncMock(return_value={"project": "remote-project"}), new=AsyncMock(return_value={"project": "remote-project"}),
), patch( ), patch(
'sync_state_machine.sync_system.strategy_ops.update_ops.collect_bound_node_pairs', 'sync_state_machine.sync_system.strategy.collect_bound_node_pairs',
new=AsyncMock(return_value=[BoundNodePair(local_node=local_node, remote_node=remote_node)]), new=AsyncMock(return_value=[BoundNodePair(local_node=local_node, remote_node=remote_node)]),
), patch(
'sync_state_machine.sync_system.strategy.emit_schema_diff_report',
): ):
updated = await run_update(strategy) updated = await strategy.update()
assert updated == [updated_node] assert updated == [updated_node]
@@ -155,3 +153,42 @@ def test_project_detail_compare_payload_normalizes_enum_key_values():
assert normalized["key"] == "plan_construction" assert normalized["key"] == "plan_construction"
assert normalized["value"] == [{"date": "2023-06-28", "capacity": 4.0}] assert normalized["value"] == [{"date": "2023-06-28", "capacity": 4.0}]
@pytest.mark.asyncio
async def test_project_detail_process_update_pairs_partitions_group_production_plans_before_generic_logic():
strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock())
pull_pair = BoundNodePair(
local_node=SimpleNamespace(node_id="local-pull", get_data=lambda: {"key": "ensure_production"}),
remote_node=SimpleNamespace(node_id="remote-pull", get_data=lambda: {"key": "ensure_production"}),
)
generic_pair = BoundNodePair(
local_node=SimpleNamespace(node_id="local-generic", get_data=lambda: {"key": "production"}),
remote_node=SimpleNamespace(node_id="remote-generic", get_data=lambda: {"key": "production"}),
)
pull_node = SimpleNamespace(node_id="pull-update")
generic_node = SimpleNamespace(node_id="generic-update")
with patch.object(
strategy,
"prepare_update_for_direction",
new=AsyncMock(return_value=pull_node),
) as mock_prepare, patch.object(
strategy,
"update_pair",
new=AsyncMock(return_value=[generic_node]),
) as mock_update_pair:
updated_by_id = await strategy.process_update_pairs([pull_pair, generic_pair], {})
assert updated_by_id == {
"pull-update": pull_node,
"generic-update": generic_node,
}
mock_prepare.assert_awaited_once_with(
pull_pair.local_node,
pull_pair.remote_node,
UpdateDirection.PULL,
{},
)
mock_update_pair.assert_awaited_once_with(generic_pair.local_node, generic_pair.remote_node, {})