修改多个问题,优化代码结构
This commit is contained in:
@@ -26,12 +26,12 @@
|
|||||||
### strategy(决策组织层)
|
### strategy(决策组织层)
|
||||||
- 外部入口:`sync_state_machine/sync_system/strategy.py`
|
- 外部入口:`sync_state_machine/sync_system/strategy.py`
|
||||||
- 具体实现下沉:`sync_state_machine/sync_system/strategy_ops/`
|
- 具体实现下沉:`sync_state_machine/sync_system/strategy_ops/`
|
||||||
- `bind_ops.py`: `run_bind()` + `phase_core_state()/phase_dependency_check()/phase_auto_binding()`
|
- `bind_ops.py`: bind 阶段静态函数与纯工具
|
||||||
- `create_ops.py`: `run_create()`
|
- `create_ops.py`: create 阶段静态函数与纯工具
|
||||||
- `update_ops.py`: `run_update()`
|
- `update_ops.py`: update 阶段静态函数与纯工具
|
||||||
- `delete_ops.py`: `run_delete()`
|
- `delete_ops.py`: `run_delete()`
|
||||||
- `reset_ops.py`: `run_phase2_cleanup()`
|
- `reset_ops.py`: `run_phase2_cleanup()`
|
||||||
- strategy 负责收集上下文并调用状态机事件函数,不直接硬编码状态值。
|
- strategy 负责 bind/create/update 的相位内部编排,并调用状态机事件函数,不直接硬编码状态值。
|
||||||
|
|
||||||
### datasource(执行与回写层)
|
### datasource(执行与回写层)
|
||||||
- 入口:`sync_state_machine/datasource/datasource.py`
|
- 入口:`sync_state_machine/datasource/datasource.py`
|
||||||
|
|||||||
@@ -43,9 +43,9 @@
|
|||||||
- `E40D_START/E40D`:DELETE 执行入口与结果回写
|
- `E40D_START/E40D`:DELETE 执行入口与结果回写
|
||||||
|
|
||||||
## 4. strategy 调用链(当前)
|
## 4. strategy 调用链(当前)
|
||||||
- bind 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.bind()` -> `sync_state_machine/sync_system/strategy_ops/bind_ops.py::run_bind()`
|
- bind 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.bind()`
|
||||||
- create 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.create()` -> `sync_state_machine/sync_system/strategy_ops/create_ops.py::run_create()`
|
- create 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.create()`
|
||||||
- update 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.update()` -> `sync_state_machine/sync_system/strategy_ops/update_ops.py::run_update()`
|
- update 主入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.update()`
|
||||||
- reset 主入口:`sync_state_machine/pipeline/full_sync_pipeline.py::_load_persistence_state()` -> `sync_state_machine/sync_system/strategy_ops/reset_ops.py::run_phase2_cleanup()`
|
- reset 主入口:`sync_state_machine/pipeline/full_sync_pipeline.py::_load_persistence_state()` -> `sync_state_machine/sync_system/strategy_ops/reset_ops.py::run_phase2_cleanup()`
|
||||||
|
|
||||||
## 5. context 字段(以 YAML 为准)
|
## 5. context 字段(以 YAML 为准)
|
||||||
|
|||||||
+14
-4
@@ -34,9 +34,18 @@
|
|||||||
- 这类节点不会进入 `E01`,也不会参与 bind/create/update/sync 自动流程。
|
- 这类节点不会进入 `E01`,也不会参与 bind/create/update/sync 自动流程。
|
||||||
|
|
||||||
## 3. bind 流程(strategy_ops)
|
## 3. bind 流程(strategy_ops)
|
||||||
- 入口:`sync_state_machine/sync_system/strategy_ops/bind_ops.py::run_bind()`
|
- 入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.bind()`
|
||||||
- 入口状态:`S00`
|
- 入口状态:`S00`
|
||||||
|
|
||||||
|
### bind 编排步骤
|
||||||
|
- strategy 先通过 `bind_ops.collect_bind_entry_nodes()` 收集两侧 `S00` 节点,作为 bind 入口工作集。
|
||||||
|
- `phase_core_state()` 对入口工作集触发 `E10`,把“已有绑定记录”和“无绑定记录”的节点分流到 `S01/S03/S04/S02`。
|
||||||
|
- `phase_dependency_check()` 对仍需继续处理的缺绑节点触发 `E15`,把缺数据、依赖不满足、语义错误的节点拦到 `S03/S05/S04`。
|
||||||
|
- strategy 再通过 `bind_ops.collect_auto_bind_ready_nodes()` 收窄工作集,只保留当前 `S02 + binding_status=MISSING` 的节点进入自动绑定判定。
|
||||||
|
- strategy 在顶层显式构造 orphan create 控制参数,并传给 `bind_ops.plan_auto_bind_updates()` 生成自动绑定/自动创建决策。
|
||||||
|
- `apply_auto_bind_updates()` 对这些决策逐个触发 `E16`,把节点迁移到 `S01/S04/S05/S13`。
|
||||||
|
- `refresh_bind_data_id()` 最后按绑定关系回填两侧节点的 `context.bind_data_id`。
|
||||||
|
|
||||||
### 核心绑定判定
|
### 核心绑定判定
|
||||||
- `phase_core_state()` 内触发 `E10`。
|
- `phase_core_state()` 内触发 `E10`。
|
||||||
- 有绑定记录:根据本端/对端有效性进入 `S01/S03/S04`。
|
- 有绑定记录:根据本端/对端有效性进入 `S01/S03/S04`。
|
||||||
@@ -44,9 +53,10 @@
|
|||||||
|
|
||||||
### 依赖检查
|
### 依赖检查
|
||||||
- `phase_dependency_check()` -> `check_dependencies_for_nodes()` 内触发 `E15`。
|
- `phase_dependency_check()` -> `check_dependencies_for_nodes()` 内触发 `E15`。
|
||||||
|
- `data_present=true` 且 `dep_satisfied=true` -> 不迁移,节点保持在 `S02`,继续进入自动绑定判定。
|
||||||
- `data_present=false` -> `S03`。
|
- `data_present=false` -> `S03`。
|
||||||
- `dep_satisfied=false` -> `S05`。
|
- `dep_satisfied=false` -> `S05`。
|
||||||
- `depend_fields` 为空 -> `S04`(无法自动处理,阻断后续自动创建)。
|
- `depend_fields` 为空本身不会触发阻断;若节点数据存在,则等价于“当前依赖满足”,继续留在 `S02`。
|
||||||
|
|
||||||
### 自动绑定
|
### 自动绑定
|
||||||
- `phase_auto_binding()` 内触发 `E16`。
|
- `phase_auto_binding()` 内触发 `E16`。
|
||||||
@@ -57,7 +67,7 @@
|
|||||||
## 4. create/update 准备
|
## 4. create/update 准备
|
||||||
|
|
||||||
### create
|
### create
|
||||||
- 入口:`sync_state_machine/sync_system/strategy_ops/create_ops.py::run_create()`
|
- 入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.create()`
|
||||||
- 运行时事件:`E20`(`E21/E22` 为配置保留分支,当前 strategy 未接入自动执行)
|
- 运行时事件:`E20`(`E21/E22` 为配置保留分支,当前 strategy 未接入自动执行)
|
||||||
- 运行时入口状态:`S13`
|
- 运行时入口状态:`S13`
|
||||||
- 说明:`E16` 负责识别“需自动创建”并将源节点置为 `S13`;业务代码执行对侧 spawn+bind 后,再通过 `E20` 将 `S13` 提交到 `S01`(成功);若 spawn 失败,`E20` 不迁移并保留在 `S13` 等待人工/重试。
|
- 说明:`E16` 负责识别“需自动创建”并将源节点置为 `S13`;业务代码执行对侧 spawn+bind 后,再通过 `E20` 将 `S13` 提交到 `S01`(成功);若 spawn 失败,`E20` 不迁移并保留在 `S13` 等待人工/重试。
|
||||||
@@ -66,7 +76,7 @@
|
|||||||
- 新创建目标节点在 `S06/S10C` 阶段允许 `data_id` 为空;`CREATE SUCCESS`(`S11C`)必须携带非空 `result_data_id`,否则事件直接报错。
|
- 新创建目标节点在 `S06/S10C` 阶段允许 `data_id` 为空;`CREATE SUCCESS`(`S11C`)必须携带非空 `result_data_id`,否则事件直接报错。
|
||||||
|
|
||||||
### update
|
### update
|
||||||
- 入口:`sync_state_machine/sync_system/strategy_ops/update_ops.py::run_update()`
|
- 入口:`sync_state_machine/sync_system/strategy.py::DefaultSyncStrategy.update()`
|
||||||
- 事件:`E30`
|
- 事件:`E30`
|
||||||
- 入口状态(source 节点):`S01`
|
- 入口状态(source 节点):`S01`
|
||||||
- `target_has_data_id=false` 或 `update_id_resolve_ok=false` -> `S08`
|
- `target_has_data_id=false` 或 `update_id_resolve_ok=false` -> `S08`
|
||||||
|
|||||||
@@ -11,9 +11,13 @@ if str(PROJECT_ROOT) not in sys.path:
|
|||||||
sys.path.insert(0, str(PROJECT_ROOT))
|
sys.path.insert(0, str(PROJECT_ROOT))
|
||||||
|
|
||||||
from sync_state_machine.config import load_overrides_from_file
|
from sync_state_machine.config import load_overrides_from_file
|
||||||
|
from sync_state_machine.logging import get_logger, init_app_logger
|
||||||
from sync_state_machine.pipeline import run_profile_from_file
|
from sync_state_machine.pipeline import run_profile_from_file
|
||||||
|
|
||||||
|
|
||||||
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def _resolve_config_path(raw_path: str) -> Path:
|
def _resolve_config_path(raw_path: str) -> Path:
|
||||||
path = Path(raw_path)
|
path = Path(raw_path)
|
||||||
if not path.is_absolute():
|
if not path.is_absolute():
|
||||||
@@ -22,6 +26,7 @@ def _resolve_config_path(raw_path: str) -> Path:
|
|||||||
|
|
||||||
|
|
||||||
async def _main() -> int:
|
async def _main() -> int:
|
||||||
|
init_app_logger(replace_handlers=True)
|
||||||
parser = argparse.ArgumentParser(description="Run sync pipeline from config override file")
|
parser = argparse.ArgumentParser(description="Run sync pipeline from config override file")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--config_path",
|
"--config_path",
|
||||||
@@ -34,7 +39,7 @@ async def _main() -> int:
|
|||||||
if not cfg_path.exists() or not cfg_path.is_file():
|
if not cfg_path.exists() or not cfg_path.is_file():
|
||||||
raise FileNotFoundError(f"Config file not found: {cfg_path}")
|
raise FileNotFoundError(f"Config file not found: {cfg_path}")
|
||||||
|
|
||||||
print(f"🧩 Loaded config file: {cfg_path}")
|
logger.info("🧩 Loaded config file: %s", cfg_path)
|
||||||
await run_profile_from_file(
|
await run_profile_from_file(
|
||||||
project_root=PROJECT_ROOT,
|
project_root=PROJECT_ROOT,
|
||||||
config_path=cfg_path,
|
config_path=cfg_path,
|
||||||
@@ -47,10 +52,10 @@ def main() -> int:
|
|||||||
try:
|
try:
|
||||||
return asyncio.run(_main())
|
return asyncio.run(_main())
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
print("\n⚠️ Interrupted by user")
|
logger.warning("⚠️ Interrupted by user")
|
||||||
return 130
|
return 130
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
print(f"\n❌ Pipeline failed: {type(exc).__name__}: {exc}")
|
logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
|
||||||
return 1
|
return 1
|
||||||
finally:
|
finally:
|
||||||
logging.shutdown()
|
logging.shutdown()
|
||||||
|
|||||||
@@ -5,7 +5,7 @@
|
|||||||
from typing import List, Optional
|
from typing import List, Optional
|
||||||
|
|
||||||
from fastapi import HTTPException
|
from fastapi import HTTPException
|
||||||
from pydantic import Field, BaseModel, field_validator
|
from pydantic import Field, BaseModel, field_validator, model_validator
|
||||||
from ..common.push_system import BaseUpdateRequestSchema
|
from ..common.push_system import BaseUpdateRequestSchema
|
||||||
from ..common.base import BaseResponseWithID
|
from ..common.base import BaseResponseWithID
|
||||||
from ..common.datetime import get_current_date
|
from ..common.datetime import get_current_date
|
||||||
@@ -78,7 +78,7 @@ class ProjectResponseBase(BaseResponseWithID):
|
|||||||
|
|
||||||
# 拓展信息
|
# 拓展信息
|
||||||
dbtc_score_list: Optional[list] = Field(default_factory=list, description='达标投产评分列表') # 这里项目用的是list而不是List
|
dbtc_score_list: Optional[list] = Field(default_factory=list, description='达标投产评分列表') # 这里项目用的是list而不是List
|
||||||
# : Optional[str] = Field(None, description="审核状态", examples=["pending"])
|
# approval_status: Optional[str] = Field(None, description="审核状态", examples=["pending"])
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -266,11 +266,35 @@ class ProjectProductionCapacity(BaseModel):
|
|||||||
"""
|
"""
|
||||||
plan_production_date: DateStr | None = Field(None, description="集团公司计划投产日期", examples=["2024-08-01"])
|
plan_production_date: DateStr | None = Field(None, description="集团公司计划投产日期", examples=["2024-08-01"])
|
||||||
plan_full_production_date: DateStr | None = Field(None, description="集团公司计划全部投产日期", examples=["2024-09-01"])
|
plan_full_production_date: DateStr | None = Field(None, description="集团公司计划全部投产日期", examples=["2024-09-01"])
|
||||||
ic_plan_first_production_date: DateStr | None = Field(None, description="区域公司计划首批投产日期(只有新能源需要)",examples=["2024-08-01"])
|
|
||||||
ic_plan_full_production_date: DateStr | None = Field(None, description="区域公司计划全投日期(只有新能源需要)",examples=["2024-09-01"])
|
|
||||||
actual_production_date: DateStr | None = Field(None, description="实际投产日期(只有新能源需要)", examples=["2024-08-15"])
|
actual_production_date: DateStr | None = Field(None, description="实际投产日期(只有新能源需要)", examples=["2024-08-15"])
|
||||||
actual_full_production_date: DateStr | None = Field(None, description="实际全投日期(只有新能源需要)", examples=["2024-09-10"])
|
actual_full_production_date: DateStr | None = Field(None, description="实际全投日期(只有新能源需要)", examples=["2024-09-10"])
|
||||||
actual_production_list : list[_ActualProductionNode] | None = Field(None, description="实际投产节点列表(每次必须填写全部)(只有新能源需要)")
|
actual_production_list : list[_ActualProductionNode] | None = Field(None, description="实际投产节点列表(每次必须填写全部)(只有新能源需要)")
|
||||||
|
|
||||||
|
@model_validator(mode="before")
|
||||||
|
@classmethod
|
||||||
|
def reject_ic_plan_dates(cls, data):
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
return data
|
||||||
|
|
||||||
|
blocked_fields = [
|
||||||
|
field for field in ("ic_plan_first_production_date", "ic_plan_full_production_date")
|
||||||
|
if field in data
|
||||||
|
]
|
||||||
|
if blocked_fields:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=400,
|
||||||
|
detail=f"/production_capacity 接口不支持更新字段: {', '.join(blocked_fields)}"
|
||||||
|
)
|
||||||
|
return data
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ic_plan_first_production_date(self) -> str:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ic_plan_full_production_date(self) -> str:
|
||||||
|
return ""
|
||||||
|
|
||||||
@field_validator("actual_production_date")
|
@field_validator("actual_production_date")
|
||||||
def validate_actual_production_date(cls, v):
|
def validate_actual_production_date(cls, v):
|
||||||
if v is None:
|
if v is None:
|
||||||
|
|||||||
@@ -47,8 +47,6 @@ async def main() -> int:
|
|||||||
return 130
|
return 130
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
|
logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
|
||||||
import traceback
|
|
||||||
traceback.print_exc()
|
|
||||||
return 1
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -47,8 +47,6 @@ async def main() -> int:
|
|||||||
return 130
|
return 130
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
|
logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
|
||||||
import traceback
|
|
||||||
traceback.print_exc()
|
|
||||||
return 1
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -5,9 +5,13 @@ import asyncio
|
|||||||
import logging
|
import logging
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .logging import get_logger, init_app_logger
|
||||||
from .pipeline import run_profile_from_file
|
from .pipeline import run_profile_from_file
|
||||||
|
|
||||||
|
|
||||||
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def package_project_root() -> Path:
|
def package_project_root() -> Path:
|
||||||
return Path(__file__).resolve().parents[1]
|
return Path(__file__).resolve().parents[1]
|
||||||
|
|
||||||
@@ -20,6 +24,7 @@ def resolve_config_path(raw_path: str, *, project_root: Path) -> Path:
|
|||||||
|
|
||||||
|
|
||||||
async def _main() -> int:
|
async def _main() -> int:
|
||||||
|
init_app_logger(replace_handlers=True)
|
||||||
project_root = package_project_root()
|
project_root = package_project_root()
|
||||||
|
|
||||||
parser = argparse.ArgumentParser(description="Run sync pipeline from config override file")
|
parser = argparse.ArgumentParser(description="Run sync pipeline from config override file")
|
||||||
@@ -34,7 +39,7 @@ async def _main() -> int:
|
|||||||
if not cfg_path.exists() or not cfg_path.is_file():
|
if not cfg_path.exists() or not cfg_path.is_file():
|
||||||
raise FileNotFoundError(f"Config file not found: {cfg_path}")
|
raise FileNotFoundError(f"Config file not found: {cfg_path}")
|
||||||
|
|
||||||
print(f"🧩 Loaded config file: {cfg_path}")
|
logger.info("🧩 Loaded config file: %s", cfg_path)
|
||||||
await run_profile_from_file(
|
await run_profile_from_file(
|
||||||
project_root=project_root,
|
project_root=project_root,
|
||||||
config_path=cfg_path,
|
config_path=cfg_path,
|
||||||
@@ -47,10 +52,10 @@ def main() -> int:
|
|||||||
try:
|
try:
|
||||||
return asyncio.run(_main())
|
return asyncio.run(_main())
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
print("\n⚠️ Interrupted by user")
|
logger.warning("⚠️ Interrupted by user")
|
||||||
return 130
|
return 130
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
print(f"\n❌ Pipeline failed: {type(exc).__name__}: {exc}")
|
logger.exception("❌ Pipeline failed: %s: %s", type(exc).__name__, exc)
|
||||||
return 1
|
return 1
|
||||||
finally:
|
finally:
|
||||||
logging.shutdown()
|
logging.shutdown()
|
||||||
|
|||||||
@@ -1,10 +1,12 @@
|
|||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
from typing import Optional, Dict, Any, List
|
from typing import Optional, Dict, Any, List
|
||||||
|
|
||||||
|
from ..logging import get_logger
|
||||||
from .persistence import PersistenceBackend
|
from .persistence import PersistenceBackend
|
||||||
from .types import BindingStatus, BindingRecord
|
from .types import BindingStatus, BindingRecord
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
class BindingManager:
|
class BindingManager:
|
||||||
|
|||||||
@@ -23,6 +23,7 @@ from datetime import datetime
|
|||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
from schemas.common.push_system import PushIdsSchema
|
from schemas.common.push_system import PushIdsSchema
|
||||||
|
from ...logging import get_logger
|
||||||
|
|
||||||
|
|
||||||
def _build_default_push_steps() -> List[Dict[str, str]]:
|
def _build_default_push_steps() -> List[Dict[str, str]]:
|
||||||
@@ -68,7 +69,7 @@ class ApiClient:
|
|||||||
self._trace_by_biz_id: Dict[str, deque[Dict[str, Any]]] = defaultdict(deque)
|
self._trace_by_biz_id: Dict[str, deque[Dict[str, Any]]] = defaultdict(deque)
|
||||||
self._trace_by_op: Dict[str, deque[Dict[str, Any]]] = defaultdict(deque)
|
self._trace_by_op: Dict[str, deque[Dict[str, Any]]] = defaultdict(deque)
|
||||||
self._load_trace_by_item: Dict[tuple[str, str], deque[Dict[str, Any]]] = defaultdict(deque)
|
self._load_trace_by_item: Dict[tuple[str, str], deque[Dict[str, Any]]] = defaultdict(deque)
|
||||||
self._logger = logging.getLogger(__name__)
|
self._logger = get_logger(__name__)
|
||||||
self._rate_limit_max_requests = int(rate_limit_max_requests)
|
self._rate_limit_max_requests = int(rate_limit_max_requests)
|
||||||
self._rate_limit_window_seconds = float(rate_limit_window_seconds)
|
self._rate_limit_window_seconds = float(rate_limit_window_seconds)
|
||||||
self._rate_limit_lock = asyncio.Lock()
|
self._rate_limit_lock = asyncio.Lock()
|
||||||
@@ -138,8 +139,8 @@ class ApiClient:
|
|||||||
if not self._debug:
|
if not self._debug:
|
||||||
return
|
return
|
||||||
self._log(
|
self._log(
|
||||||
f"ApiClient request: method={method.upper()} endpoint={endpoint} elapsed={elapsed_seconds:.3f}s",
|
f"🔎 ApiClient request: method={method.upper()} endpoint={endpoint} elapsed={elapsed_seconds:.3f}s",
|
||||||
level="DEBUG",
|
level="INFO",
|
||||||
)
|
)
|
||||||
|
|
||||||
def _log(self, message: str, level: str = "DEBUG"):
|
def _log(self, message: str, level: str = "DEBUG"):
|
||||||
|
|||||||
@@ -3,8 +3,10 @@ from __future__ import annotations
|
|||||||
import logging
|
import logging
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from ...logging import get_logger
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def prepare_remote_jsonl_dir(remote_dir: Path, *, project_root: Path) -> Path:
|
def prepare_remote_jsonl_dir(remote_dir: Path, *, project_root: Path) -> Path:
|
||||||
|
|||||||
@@ -1,9 +1,18 @@
|
|||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from ...common.sync_node import SyncNode
|
||||||
from ...sync_system.strategy import DefaultSyncStrategy
|
from ...sync_system.strategy import DefaultSyncStrategy
|
||||||
from ...sync_system.config import StrategyConfig, OrphanAction, UpdateDirection
|
from ...sync_system.config import StrategyConfig, OrphanAction, UpdateDirection
|
||||||
from ...sync_system.strategy_ops.compare_ops import build_data_id_normalization_map
|
from ...sync_system.strategy_ops.bind_ops import (
|
||||||
from ...sync_system.strategy_ops.update_ops import emit_schema_diff_report, _build_schema_diff_validator
|
apply_auto_bind_updates,
|
||||||
|
collect_auto_bind_ready_nodes,
|
||||||
|
collect_bind_entry_nodes,
|
||||||
|
plan_auto_bind_updates,
|
||||||
|
phase_core_state,
|
||||||
|
phase_dependency_check,
|
||||||
|
refresh_bind_data_id,
|
||||||
|
)
|
||||||
|
from ...sync_system.strategy_ops import BoundNodePair
|
||||||
from schemas.project_extensions.project_detail import ProjectDetailKey, ProjectDetailProject
|
from schemas.project_extensions.project_detail import ProjectDetailKey, ProjectDetailProject
|
||||||
|
|
||||||
|
|
||||||
@@ -37,45 +46,108 @@ class ProjectDetailSyncStrategy(DefaultSyncStrategy[ProjectDetailProject]):
|
|||||||
update_direction=UpdateDirection.PUSH,
|
update_direction=UpdateDirection.PUSH,
|
||||||
)
|
)
|
||||||
|
|
||||||
async def update(self):
|
async def bind(self) -> None:
|
||||||
self.ensure_runtime()
|
runtime = self.ensure_runtime()
|
||||||
validator = _build_schema_diff_validator(self)
|
local_nodes, remote_nodes = collect_bind_entry_nodes(
|
||||||
self._active_schema_diff_validator = validator
|
node_type=self.node_type,
|
||||||
data_id_map = await build_data_id_normalization_map(
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
)
|
||||||
|
await phase_core_state(
|
||||||
|
node_type=self.node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
local_nodes=local_nodes,
|
||||||
|
remote_nodes=remote_nodes,
|
||||||
|
)
|
||||||
|
await phase_dependency_check(
|
||||||
|
node_type=self.node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
depend_fields=dict(self.config.depend_fields),
|
||||||
|
local_nodes=local_nodes,
|
||||||
|
remote_nodes=remote_nodes,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
)
|
||||||
|
local_nodes, remote_nodes = collect_auto_bind_ready_nodes(
|
||||||
|
node_type=self.node_type,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
)
|
||||||
|
|
||||||
|
local_create_enabled = self.config.local_orphan_action == OrphanAction.CREATE_REMOTE
|
||||||
|
remote_create_enabled = self.config.remote_orphan_action == OrphanAction.CREATE_LOCAL
|
||||||
|
local_orphan_create_enabled_by_node = {
|
||||||
|
node.node_id: local_create_enabled for node in local_nodes
|
||||||
|
}
|
||||||
|
remote_orphan_create_enabled_by_node = {
|
||||||
|
node.node_id: remote_create_enabled for node in remote_nodes
|
||||||
|
}
|
||||||
|
|
||||||
|
if self.domain_option.pull_group_production_plans:
|
||||||
|
for node in local_nodes:
|
||||||
|
if self._extract_node_key(node) in self._GROUP_PRODUCTION_PLAN_KEYS:
|
||||||
|
local_orphan_create_enabled_by_node[node.node_id] = False
|
||||||
|
for node in remote_nodes:
|
||||||
|
if self._extract_node_key(node) in self._GROUP_PRODUCTION_PLAN_KEYS:
|
||||||
|
remote_orphan_create_enabled_by_node[node.node_id] = True
|
||||||
|
|
||||||
|
pending_updates = await plan_auto_bind_updates(
|
||||||
|
node_type=self.node_type,
|
||||||
|
config=self.config,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
local_nodes=local_nodes,
|
||||||
|
remote_nodes=remote_nodes,
|
||||||
|
id_field_hints=dict(self.config.depend_fields),
|
||||||
|
local_orphan_create_enabled_by_node=local_orphan_create_enabled_by_node,
|
||||||
|
remote_orphan_create_enabled_by_node=remote_orphan_create_enabled_by_node,
|
||||||
|
)
|
||||||
|
await apply_auto_bind_updates(
|
||||||
|
node_type=self.node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
pending_auto_bind_updates=pending_updates,
|
||||||
|
)
|
||||||
|
await refresh_bind_data_id(
|
||||||
node_type=self.node_type,
|
node_type=self.node_type,
|
||||||
local_collection=self.local_collection,
|
local_collection=self.local_collection,
|
||||||
remote_collection=self.remote_collection,
|
remote_collection=self.remote_collection,
|
||||||
binding_manager=self.binding_manager,
|
binding_manager=self.binding_manager,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
async def process_update_pairs(
|
||||||
|
self,
|
||||||
|
pairs: list[BoundNodePair],
|
||||||
|
data_id_map: dict[str, str] | None = None,
|
||||||
|
) -> dict[str, SyncNode]:
|
||||||
|
pull_keys = self._GROUP_PRODUCTION_PLAN_KEYS if self.domain_option.pull_group_production_plans else set()
|
||||||
pull_pairs = []
|
pull_pairs = []
|
||||||
generic_pairs = []
|
generic_pairs = []
|
||||||
for pair in await self.collect_update_pairs():
|
for pair in pairs:
|
||||||
if self._should_pull_pair(pair.local_node, pair.remote_node):
|
pair_key = self._extract_pair_key(pair.local_node, pair.remote_node)
|
||||||
|
if pair_key in pull_keys:
|
||||||
pull_pairs.append(pair)
|
pull_pairs.append(pair)
|
||||||
else:
|
else:
|
||||||
generic_pairs.append(pair)
|
generic_pairs.append(pair)
|
||||||
|
|
||||||
updated_by_id = {}
|
updated_by_id: dict[str, SyncNode] = {}
|
||||||
try:
|
for pair in pull_pairs:
|
||||||
for pair in pull_pairs:
|
updated_node = await self.prepare_update_for_direction(
|
||||||
updated_node = await self.prepare_update_for_direction(
|
pair.local_node,
|
||||||
pair.local_node,
|
pair.remote_node,
|
||||||
pair.remote_node,
|
UpdateDirection.PULL,
|
||||||
UpdateDirection.PULL,
|
data_id_map,
|
||||||
data_id_map,
|
)
|
||||||
)
|
if updated_node is not None:
|
||||||
if updated_node is not None:
|
updated_by_id[updated_node.node_id] = updated_node
|
||||||
updated_by_id[updated_node.node_id] = updated_node
|
|
||||||
|
|
||||||
for pair in generic_pairs:
|
for pair in generic_pairs:
|
||||||
for updated_node in await self.update_pair(pair.local_node, pair.remote_node, data_id_map):
|
for updated_node in await self.update_pair(pair.local_node, pair.remote_node, data_id_map):
|
||||||
updated_by_id[updated_node.node_id] = updated_node
|
updated_by_id[updated_node.node_id] = updated_node
|
||||||
finally:
|
|
||||||
self._active_schema_diff_validator = None
|
|
||||||
|
|
||||||
emit_schema_diff_report(self, validator)
|
return updated_by_id
|
||||||
return list(updated_by_id.values())
|
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def domain_option(self) -> ProjectDetailDomainOption:
|
def domain_option(self) -> ProjectDetailDomainOption:
|
||||||
@@ -129,6 +201,16 @@ class ProjectDetailSyncStrategy(DefaultSyncStrategy[ProjectDetailProject]):
|
|||||||
|
|
||||||
return items
|
return items
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_node_key(node: SyncNode) -> str | None:
|
||||||
|
data = node.get_data() or {}
|
||||||
|
key = data.get("key")
|
||||||
|
if isinstance(key, ProjectDetailKey):
|
||||||
|
return key.value
|
||||||
|
if isinstance(key, str):
|
||||||
|
return key
|
||||||
|
return None
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _extract_pair_key(local_node, remote_node) -> str | None:
|
def _extract_pair_key(local_node, remote_node) -> str | None:
|
||||||
local_data = local_node.get_data() or {}
|
local_data = local_node.get_data() or {}
|
||||||
@@ -141,7 +223,3 @@ class ProjectDetailSyncStrategy(DefaultSyncStrategy[ProjectDetailProject]):
|
|||||||
return key
|
return key
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def _should_pull_pair(self, local_node, remote_node) -> bool:
|
|
||||||
if not self.domain_option.pull_group_production_plans:
|
|
||||||
return False
|
|
||||||
return self._extract_pair_key(local_node, remote_node) in self._GROUP_PRODUCTION_PLAN_KEYS
|
|
||||||
|
|||||||
@@ -39,6 +39,7 @@ def clear_app_logger_handlers(logger: Optional[logging.Logger] = None) -> None:
|
|||||||
handler.close()
|
handler.close()
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
target.propagate = True
|
||||||
|
|
||||||
|
|
||||||
def init_app_logger(
|
def init_app_logger(
|
||||||
|
|||||||
@@ -48,7 +48,6 @@ from ..config import (
|
|||||||
from ..config.strategy_config import resolve_domain_option_config
|
from ..config.strategy_config import resolve_domain_option_config
|
||||||
from ..sync_system.strategy import BaseSyncStrategy
|
from ..sync_system.strategy import BaseSyncStrategy
|
||||||
from ..logging import configure_app_logging, ensure_app_logging, get_logger
|
from ..logging import configure_app_logging, ensure_app_logging, get_logger
|
||||||
from .run_logger import init_pipeline_logger
|
|
||||||
from .full_sync_pipeline import FullSyncPipeline
|
from .full_sync_pipeline import FullSyncPipeline
|
||||||
|
|
||||||
|
|
||||||
@@ -458,9 +457,16 @@ async def run_profile_from_file(
|
|||||||
cfg_path = Path(config_path)
|
cfg_path = Path(config_path)
|
||||||
raw = load_overrides_from_file(cfg_path, project_root)
|
raw = load_overrides_from_file(cfg_path, project_root)
|
||||||
if is_multi_project_payload(raw):
|
if is_multi_project_payload(raw):
|
||||||
from .multi_project_pipeline import MultiProjectPipeline
|
from .multi_project_pipeline import MultiProjectPipeline, _resolve_embedded_config_path
|
||||||
|
|
||||||
multi_cfg = MultiProjectRunConfig.model_validate(raw)
|
multi_cfg = MultiProjectRunConfig.model_validate(raw)
|
||||||
|
global_cfg_path = _resolve_embedded_config_path(
|
||||||
|
multi_cfg.global_config,
|
||||||
|
project_root=project_root,
|
||||||
|
profile_path=cfg_path,
|
||||||
|
)
|
||||||
|
global_config = build_config_from_file(project_root=project_root, file_path=global_cfg_path)
|
||||||
|
configure_app_logging(global_config.logging, replace_handlers=True)
|
||||||
pipeline = MultiProjectPipeline(project_root=project_root, config_path=cfg_path, config=multi_cfg)
|
pipeline = MultiProjectPipeline(project_root=project_root, config_path=cfg_path, config=multi_cfg)
|
||||||
if print_summary:
|
if print_summary:
|
||||||
logger.info("\n%s", "=" * 80)
|
logger.info("\n%s", "=" * 80)
|
||||||
@@ -469,7 +475,7 @@ async def run_profile_from_file(
|
|||||||
return await pipeline.run()
|
return await pipeline.run()
|
||||||
|
|
||||||
config = build_config(project_root=project_root, overrides=raw)
|
config = build_config(project_root=project_root, overrides=raw)
|
||||||
configure_app_logging(config.logging, replace_handlers=False)
|
configure_app_logging(config.logging, replace_handlers=True)
|
||||||
mode = f"{config.local_datasource.type}->{config.remote_datasource.type}"
|
mode = f"{config.local_datasource.type}->{config.remote_datasource.type}"
|
||||||
return await run_pipeline_from_config(
|
return await run_pipeline_from_config(
|
||||||
config,
|
config,
|
||||||
|
|||||||
@@ -15,17 +15,19 @@ if TYPE_CHECKING:
|
|||||||
from ..datasource.jsonl import JsonlDataSource
|
from ..datasource.jsonl import JsonlDataSource
|
||||||
from ..common.collection import DataCollection
|
from ..common.collection import DataCollection
|
||||||
from ..common.binding import BindingManager
|
from ..common.binding import BindingManager
|
||||||
|
from ..logging import get_logger
|
||||||
from ..common.persistence import PersistenceBackend
|
from ..common.persistence import PersistenceBackend
|
||||||
from ..sync_system.strategy import BaseSyncStrategy
|
from ..sync_system.strategy import BaseSyncStrategy
|
||||||
from ..sync_system.config import OrphanAction, UpdateDirection
|
from ..sync_system.config import OrphanAction, UpdateDirection
|
||||||
from ..engine import StateMachineConfig, StateMachineRuntime
|
from ..engine import StateMachineConfig, StateMachineRuntime
|
||||||
from ..engine import e41_post_create_ready
|
from ..engine import e41_post_create_ready
|
||||||
from ..sync_system.strategy_ops import finalize_peer_creating_sources, run_phase2_cleanup
|
from ..sync_system.strategy_ops import finalize_peer_creating_sources, run_phase2_cleanup
|
||||||
|
from ..sync_system.strategy_ops.bind_ops import refresh_bind_data_id
|
||||||
from ..sync_system.strategy_ops.post_check_ops import evaluate_consistency_for_node_type
|
from ..sync_system.strategy_ops.post_check_ops import evaluate_consistency_for_node_type
|
||||||
from .summary_report import print_pipeline_summary
|
from .summary_report import print_pipeline_summary
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
class FullSyncPipeline:
|
class FullSyncPipeline:
|
||||||
@@ -211,10 +213,24 @@ class FullSyncPipeline:
|
|||||||
try:
|
try:
|
||||||
created = await strategy.create()
|
created = await strategy.create()
|
||||||
self._logger.info(f"✅ Strategy create: {node_type} (created={len(created)})")
|
self._logger.info(f"✅ Strategy create: {node_type} (created={len(created)})")
|
||||||
|
created_local_node_ids = {
|
||||||
|
node.node_id for node in created if self.local_collection.get_by_node_id(node.node_id) is not None
|
||||||
|
}
|
||||||
|
created_remote_node_ids = {
|
||||||
|
node.node_id for node in created if self.remote_collection.get_by_node_id(node.node_id) is not None
|
||||||
|
}
|
||||||
create_written = await self.commit_creates(node_type)
|
create_written = await self.commit_creates(node_type)
|
||||||
await self.normalize_create_success_to_update_entry(node_type)
|
await self.normalize_create_success_to_update_entry(node_type)
|
||||||
if create_written:
|
if create_written:
|
||||||
await self._reload_node_type(node_type, reason="create wrote data")
|
await self._reload_node_type(node_type, reason="create wrote data")
|
||||||
|
await refresh_bind_data_id(
|
||||||
|
node_type=node_type,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
local_node_ids=created_local_node_ids,
|
||||||
|
remote_node_ids=created_remote_node_ids,
|
||||||
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
self.stats["failed"] += 1
|
self.stats["failed"] += 1
|
||||||
self._logger.error(f"❌ Strategy create failed but pipeline continues: {node_type} | {type(exc).__name__}: {exc}")
|
self._logger.error(f"❌ Strategy create failed but pipeline continues: {node_type} | {type(exc).__name__}: {exc}")
|
||||||
|
|||||||
@@ -12,10 +12,11 @@ from ..config import (
|
|||||||
apply_config_overrides,
|
apply_config_overrides,
|
||||||
build_config_from_file,
|
build_config_from_file,
|
||||||
)
|
)
|
||||||
|
from ..logging import get_logger
|
||||||
from .factory import create_pipeline_from_config
|
from .factory import create_pipeline_from_config
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
class _SharedStateSnapshot(TypedDict):
|
class _SharedStateSnapshot(TypedDict):
|
||||||
|
|||||||
@@ -1,35 +1,45 @@
|
|||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from typing import TypeVar, Generic, Type, Dict, List, Optional, Any
|
from typing import TypeVar, Generic, Type, Dict, List, Optional, Any
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
import logging
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ..common.collection import DataCollection
|
from ..common.collection import DataCollection
|
||||||
from ..common.binding import BindingManager
|
from ..common.binding import BindingManager
|
||||||
from ..common.sync_node import SyncNode
|
from ..common.sync_node import SyncNode
|
||||||
|
from ..logging import get_logger
|
||||||
from .config import StrategyConfig, OrphanAction, UpdateDirection
|
from .config import StrategyConfig, OrphanAction, UpdateDirection
|
||||||
from ..config.strategy_config import resolve_domain_option_config
|
from ..config.strategy_config import resolve_domain_option_config
|
||||||
from .strategy_ops import (
|
from .strategy_ops import (
|
||||||
run_bind,
|
|
||||||
run_create,
|
|
||||||
run_delete,
|
run_delete,
|
||||||
)
|
)
|
||||||
|
from .strategy_ops.bind_ops import (
|
||||||
|
apply_auto_bind_updates,
|
||||||
|
collect_auto_bind_ready_nodes,
|
||||||
|
collect_bind_entry_nodes,
|
||||||
|
plan_auto_bind_updates,
|
||||||
|
phase_core_state,
|
||||||
|
phase_dependency_check,
|
||||||
|
refresh_bind_data_id,
|
||||||
|
)
|
||||||
|
from .strategy_ops.create_ops import add_create_action
|
||||||
from .strategy_ops.update_ops import (
|
from .strategy_ops.update_ops import (
|
||||||
|
BoundNodePair,
|
||||||
|
_build_schema_diff_validator,
|
||||||
collect_bound_node_pairs,
|
collect_bound_node_pairs,
|
||||||
default_get_node_update_payload,
|
default_get_node_update_payload,
|
||||||
default_needs_update,
|
default_needs_update,
|
||||||
default_update_pair,
|
default_update_pair,
|
||||||
|
emit_schema_diff_report,
|
||||||
prepare_directional_update,
|
prepare_directional_update,
|
||||||
run_update,
|
|
||||||
)
|
)
|
||||||
from ..engine import (
|
from ..engine import (
|
||||||
StateMachineConfig,
|
StateMachineConfig,
|
||||||
StateMachineRuntime,
|
StateMachineRuntime,
|
||||||
)
|
)
|
||||||
from .strategy_ops.compare_ops import normalized_data_for_compare
|
from .strategy_ops.compare_ops import build_data_id_normalization_map, normalized_data_for_compare
|
||||||
|
|
||||||
T = TypeVar("T", bound=BaseModel)
|
T = TypeVar("T", bound=BaseModel)
|
||||||
logger = logging.getLogger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
class BaseSyncStrategy(ABC, Generic[T]):
|
class BaseSyncStrategy(ABC, Generic[T]):
|
||||||
"""
|
"""
|
||||||
@@ -69,7 +79,7 @@ class BaseSyncStrategy(ABC, Generic[T]):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
async def bind(self, **kwargs):
|
async def bind(self):
|
||||||
"""进行绑定逻辑判定,设置 node.binding_status 和 node.action"""
|
"""进行绑定逻辑判定,设置 node.binding_status 和 node.action"""
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@@ -191,17 +201,124 @@ class DefaultSyncStrategy(BaseSyncStrategy[T]):
|
|||||||
)
|
)
|
||||||
return normalized
|
return normalized
|
||||||
|
|
||||||
async def bind(self, **kwargs):
|
async def bind(self):
|
||||||
self.ensure_runtime()
|
runtime = self.ensure_runtime()
|
||||||
await run_bind(self, **kwargs)
|
bind_local_nodes, bind_remote_nodes = collect_bind_entry_nodes(
|
||||||
|
node_type=self.node_type,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
)
|
||||||
|
await phase_core_state(
|
||||||
|
node_type=self.node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
local_nodes=bind_local_nodes,
|
||||||
|
remote_nodes=bind_remote_nodes,
|
||||||
|
)
|
||||||
|
await phase_dependency_check(
|
||||||
|
node_type=self.node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
depend_fields=dict(self.config.depend_fields),
|
||||||
|
local_nodes=bind_local_nodes,
|
||||||
|
remote_nodes=bind_remote_nodes,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
)
|
||||||
|
auto_bind_local_nodes, auto_bind_remote_nodes = collect_auto_bind_ready_nodes(
|
||||||
|
node_type=self.node_type,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
)
|
||||||
|
local_create_enabled = self.config.local_orphan_action == OrphanAction.CREATE_REMOTE
|
||||||
|
remote_create_enabled = self.config.remote_orphan_action == OrphanAction.CREATE_LOCAL
|
||||||
|
local_orphan_create_enabled_by_node = {
|
||||||
|
node.node_id: local_create_enabled for node in auto_bind_local_nodes
|
||||||
|
}
|
||||||
|
remote_orphan_create_enabled_by_node = {
|
||||||
|
node.node_id: remote_create_enabled for node in auto_bind_remote_nodes
|
||||||
|
}
|
||||||
|
pending_updates = await plan_auto_bind_updates(
|
||||||
|
node_type=self.node_type,
|
||||||
|
config=self.config,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
local_nodes=auto_bind_local_nodes,
|
||||||
|
remote_nodes=auto_bind_remote_nodes,
|
||||||
|
id_field_hints=dict(self.config.depend_fields),
|
||||||
|
local_orphan_create_enabled_by_node=local_orphan_create_enabled_by_node,
|
||||||
|
remote_orphan_create_enabled_by_node=remote_orphan_create_enabled_by_node,
|
||||||
|
)
|
||||||
|
await apply_auto_bind_updates(
|
||||||
|
node_type=self.node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
pending_auto_bind_updates=pending_updates,
|
||||||
|
)
|
||||||
|
await refresh_bind_data_id(
|
||||||
|
node_type=self.node_type,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
local_node_ids={
|
||||||
|
*(node.node_id for node in bind_local_nodes),
|
||||||
|
*(node.node_id for node in auto_bind_local_nodes),
|
||||||
|
},
|
||||||
|
remote_node_ids={
|
||||||
|
*(node.node_id for node in bind_remote_nodes),
|
||||||
|
*(node.node_id for node in auto_bind_remote_nodes),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
async def create(self) -> List[SyncNode]:
|
async def create(self) -> List[SyncNode]:
|
||||||
self.ensure_runtime()
|
self.ensure_runtime()
|
||||||
return await run_create(self)
|
created_nodes: List[SyncNode] = []
|
||||||
|
created_nodes.extend(await self.create_for_direction(source_is_local=True))
|
||||||
|
created_nodes.extend(await self.create_for_direction(source_is_local=False))
|
||||||
|
return created_nodes
|
||||||
|
|
||||||
|
async def create_for_direction(self, *, source_is_local: bool) -> List[SyncNode]:
|
||||||
|
from_collection = self.local_collection if source_is_local else self.remote_collection
|
||||||
|
to_collection = self.remote_collection if source_is_local else self.local_collection
|
||||||
|
return await add_create_action(
|
||||||
|
self,
|
||||||
|
from_collection=from_collection,
|
||||||
|
to_collection=to_collection,
|
||||||
|
source_is_local=source_is_local,
|
||||||
|
)
|
||||||
|
|
||||||
async def update(self) -> List[SyncNode]:
|
async def update(self) -> List[SyncNode]:
|
||||||
self.ensure_runtime()
|
self.ensure_runtime()
|
||||||
return await run_update(self)
|
validator = _build_schema_diff_validator(self)
|
||||||
|
self._active_schema_diff_validator = validator
|
||||||
|
data_id_map = await self.build_update_data_id_map()
|
||||||
|
try:
|
||||||
|
pairs = await self.collect_update_pairs()
|
||||||
|
updated_by_id = await self.process_update_pairs(pairs, data_id_map)
|
||||||
|
finally:
|
||||||
|
self._active_schema_diff_validator = None
|
||||||
|
|
||||||
|
emit_schema_diff_report(self, validator)
|
||||||
|
return list(updated_by_id.values())
|
||||||
|
|
||||||
|
async def build_update_data_id_map(self) -> Dict[str, str]:
|
||||||
|
return await build_data_id_normalization_map(
|
||||||
|
node_type=self.node_type,
|
||||||
|
local_collection=self.local_collection,
|
||||||
|
remote_collection=self.remote_collection,
|
||||||
|
binding_manager=self.binding_manager,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def process_update_pairs(
|
||||||
|
self,
|
||||||
|
pairs: List[BoundNodePair],
|
||||||
|
data_id_map: Optional[Dict[str, str]] = None,
|
||||||
|
) -> Dict[str, SyncNode]:
|
||||||
|
updated_by_id: Dict[str, SyncNode] = {}
|
||||||
|
for pair in pairs:
|
||||||
|
for updated_node in await self.update_pair(pair.local_node, pair.remote_node, data_id_map):
|
||||||
|
updated_by_id[updated_node.node_id] = updated_node
|
||||||
|
return updated_by_id
|
||||||
|
|
||||||
def get_node_update_payload(self, node: SyncNode) -> Dict[str, Any]:
|
def get_node_update_payload(self, node: SyncNode) -> Dict[str, Any]:
|
||||||
return default_get_node_update_payload(node)
|
return default_get_node_update_payload(node)
|
||||||
|
|||||||
@@ -1,18 +1,16 @@
|
|||||||
from .create_ops import add_create_action, run_create, finalize_peer_creating_sources
|
from .create_ops import add_create_action, finalize_peer_creating_sources
|
||||||
from .update_ops import (
|
from .update_ops import (
|
||||||
BoundNodePair,
|
BoundNodePair,
|
||||||
add_update_action,
|
add_update_action,
|
||||||
collect_bound_node_pairs,
|
collect_bound_node_pairs,
|
||||||
prepare_directional_update,
|
prepare_directional_update,
|
||||||
run_update,
|
|
||||||
)
|
)
|
||||||
from .delete_ops import run_delete
|
from .delete_ops import run_delete
|
||||||
from .bind_ops import (
|
from .bind_ops import (
|
||||||
run_bind,
|
|
||||||
phase_core_state,
|
phase_core_state,
|
||||||
phase_dependency_check,
|
phase_dependency_check,
|
||||||
check_dependencies_for_nodes,
|
|
||||||
phase_auto_binding,
|
phase_auto_binding,
|
||||||
|
check_dependencies_for_nodes,
|
||||||
check_dependency_satisfied,
|
check_dependency_satisfied,
|
||||||
collect_dependency_errors,
|
collect_dependency_errors,
|
||||||
)
|
)
|
||||||
@@ -20,19 +18,16 @@ from .reset_ops import run_phase2_cleanup
|
|||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"add_create_action",
|
"add_create_action",
|
||||||
"run_create",
|
|
||||||
"finalize_peer_creating_sources",
|
"finalize_peer_creating_sources",
|
||||||
"BoundNodePair",
|
"BoundNodePair",
|
||||||
"add_update_action",
|
"add_update_action",
|
||||||
"collect_bound_node_pairs",
|
"collect_bound_node_pairs",
|
||||||
"prepare_directional_update",
|
"prepare_directional_update",
|
||||||
"run_update",
|
|
||||||
"run_delete",
|
"run_delete",
|
||||||
"run_bind",
|
|
||||||
"phase_core_state",
|
"phase_core_state",
|
||||||
"phase_dependency_check",
|
"phase_dependency_check",
|
||||||
"check_dependencies_for_nodes",
|
|
||||||
"phase_auto_binding",
|
"phase_auto_binding",
|
||||||
|
"check_dependencies_for_nodes",
|
||||||
"check_dependency_satisfied",
|
"check_dependency_satisfied",
|
||||||
"collect_dependency_errors",
|
"collect_dependency_errors",
|
||||||
"run_phase2_cleanup",
|
"run_phase2_cleanup",
|
||||||
|
|||||||
@@ -1,23 +1,31 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
from dataclasses import dataclass
|
||||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Tuple
|
from typing import TYPE_CHECKING, Any, Dict, List, Mapping, Optional, Set, Tuple
|
||||||
|
|
||||||
from ...common.types import BindingStatus
|
from ...common.types import BindingStatus
|
||||||
from ...sync_system.config import OrphanAction
|
from ...logging import get_logger
|
||||||
from ...sync_system.resolve_ids import IDResolver
|
from ...sync_system.resolve_ids import IDResolver
|
||||||
from ...sync_system.utils import extract_biz_key_dict, dict_to_biz_key_tuple, resolve_id_fields_in_key_dict
|
from ...sync_system.utils import extract_biz_key_dict, dict_to_biz_key_tuple, resolve_id_fields_in_key_dict
|
||||||
from ...engine import e10_bind_core, decision_note, e15_bind_dependency, e16_bind_auto
|
from ...engine import e10_bind_core, decision_note, e15_bind_dependency, e16_bind_auto
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
|
from ...common.binding import BindingManager
|
||||||
from ...common.collection import DataCollection
|
from ...common.collection import DataCollection
|
||||||
from ...common.sync_node import SyncNode
|
from ...common.sync_node import SyncNode
|
||||||
|
from ...config.strategy_config import StrategyConfig
|
||||||
|
from ...engine import StateMachineRuntime
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def get_id_field_hints(strategy) -> Dict[str, str]:
|
@dataclass(frozen=True)
|
||||||
return dict(strategy.config.depend_fields) if strategy.config.depend_fields else {}
|
class PendingAutoBindUpdate:
|
||||||
|
node: "SyncNode"
|
||||||
|
outcome: Optional[str]
|
||||||
|
create_enabled: Optional[bool]
|
||||||
|
bind_remote_node_id: Optional[str] = None
|
||||||
|
detail: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
def _stable_pick_bind_pair(local_candidates: List["SyncNode"], remote_candidates: List["SyncNode"]):
|
def _stable_pick_bind_pair(local_candidates: List["SyncNode"], remote_candidates: List["SyncNode"]):
|
||||||
@@ -26,55 +34,118 @@ def _stable_pick_bind_pair(local_candidates: List["SyncNode"], remote_candidates
|
|||||||
return local_node, remote_node
|
return local_node, remote_node
|
||||||
|
|
||||||
|
|
||||||
async def run_bind(strategy, **kwargs):
|
def collect_bind_entry_nodes(
|
||||||
logger.info(f"[{strategy.node_type}] Starting Binding Pipeline...")
|
*,
|
||||||
|
node_type: str,
|
||||||
local_nodes: List["SyncNode"] = strategy.local_collection.filter_by_state_ids(
|
local_collection: "DataCollection",
|
||||||
node_type=strategy.node_type,
|
remote_collection: "DataCollection",
|
||||||
|
) -> tuple[List["SyncNode"], List["SyncNode"]]:
|
||||||
|
local_nodes = local_collection.filter_by_state_ids(
|
||||||
|
node_type=node_type,
|
||||||
state_ids=["S00"],
|
state_ids=["S00"],
|
||||||
)
|
)
|
||||||
remote_nodes: List["SyncNode"] = strategy.remote_collection.filter_by_state_ids(
|
remote_nodes = remote_collection.filter_by_state_ids(
|
||||||
node_type=strategy.node_type,
|
node_type=node_type,
|
||||||
state_ids=["S00"],
|
state_ids=["S00"],
|
||||||
)
|
)
|
||||||
|
return local_nodes, remote_nodes
|
||||||
await phase_core_state(strategy, local_nodes, remote_nodes)
|
|
||||||
await phase_dependency_check(strategy, local_nodes, remote_nodes)
|
|
||||||
await phase_auto_binding(strategy, local_nodes, remote_nodes)
|
|
||||||
await refresh_bind_data_id(strategy)
|
|
||||||
|
|
||||||
logger.info(f"[{strategy.node_type}] Binding Pipeline Completed.")
|
|
||||||
|
|
||||||
|
|
||||||
async def refresh_bind_data_id(strategy) -> None:
|
def collect_auto_bind_ready_nodes(
|
||||||
binding_pairs = await strategy.binding_manager.get_all_bindings(strategy.node_type)
|
*,
|
||||||
|
node_type: str,
|
||||||
|
local_collection: "DataCollection",
|
||||||
|
remote_collection: "DataCollection",
|
||||||
|
) -> tuple[List["SyncNode"], List["SyncNode"]]:
|
||||||
|
local_nodes = [
|
||||||
|
node
|
||||||
|
for node in local_collection.filter_by_state_ids(
|
||||||
|
node_type=node_type,
|
||||||
|
state_ids=["S02"],
|
||||||
|
)
|
||||||
|
if node.binding_status == BindingStatus.MISSING
|
||||||
|
]
|
||||||
|
remote_nodes = [
|
||||||
|
node
|
||||||
|
for node in remote_collection.filter_by_state_ids(
|
||||||
|
node_type=node_type,
|
||||||
|
state_ids=["S02"],
|
||||||
|
)
|
||||||
|
if node.binding_status == BindingStatus.MISSING
|
||||||
|
]
|
||||||
|
return local_nodes, remote_nodes
|
||||||
|
|
||||||
|
|
||||||
|
async def refresh_bind_data_id(
|
||||||
|
*,
|
||||||
|
node_type: str,
|
||||||
|
local_collection: "DataCollection",
|
||||||
|
remote_collection: "DataCollection",
|
||||||
|
binding_manager: "BindingManager",
|
||||||
|
local_node_ids: Optional[Set[str]] = None,
|
||||||
|
remote_node_ids: Optional[Set[str]] = None,
|
||||||
|
) -> None:
|
||||||
|
binding_pairs = await binding_manager.get_all_bindings(node_type)
|
||||||
local_to_remote = {local_id: remote_id for local_id, remote_id in binding_pairs if local_id}
|
local_to_remote = {local_id: remote_id for local_id, remote_id in binding_pairs if local_id}
|
||||||
remote_to_local = {remote_id: local_id for local_id, remote_id in binding_pairs if local_id and remote_id}
|
remote_to_local = {remote_id: local_id for local_id, remote_id in binding_pairs if local_id and remote_id}
|
||||||
|
|
||||||
for local_node in strategy.local_collection.filter(node_type=strategy.node_type):
|
if local_node_ids is None:
|
||||||
|
target_local_node_ids = {node.node_id for node in local_collection.filter(node_type=node_type)}
|
||||||
|
else:
|
||||||
|
target_local_node_ids = {node_id for node_id in local_node_ids if node_id}
|
||||||
|
|
||||||
|
if remote_node_ids is None:
|
||||||
|
target_remote_node_ids = {node.node_id for node in remote_collection.filter(node_type=node_type)}
|
||||||
|
else:
|
||||||
|
target_remote_node_ids = {node_id for node_id in remote_node_ids if node_id}
|
||||||
|
|
||||||
|
for local_node_id in list(target_local_node_ids):
|
||||||
|
remote_node_id = local_to_remote.get(local_node_id)
|
||||||
|
if remote_node_id:
|
||||||
|
target_remote_node_ids.add(remote_node_id)
|
||||||
|
|
||||||
|
for remote_node_id in list(target_remote_node_ids):
|
||||||
|
local_node_id = remote_to_local.get(remote_node_id)
|
||||||
|
if local_node_id:
|
||||||
|
target_local_node_ids.add(local_node_id)
|
||||||
|
|
||||||
|
for local_node_id in target_local_node_ids:
|
||||||
|
local_node = local_collection.get_by_node_id(local_node_id)
|
||||||
|
if local_node is None:
|
||||||
|
continue
|
||||||
remote_node_id = local_to_remote.get(local_node.node_id)
|
remote_node_id = local_to_remote.get(local_node.node_id)
|
||||||
remote_node = strategy.remote_collection.get_by_node_id(remote_node_id) if remote_node_id else None
|
remote_node = remote_collection.get_by_node_id(remote_node_id) if remote_node_id else None
|
||||||
if remote_node and remote_node.data_id:
|
if remote_node and remote_node.data_id:
|
||||||
local_node.context["bind_data_id"] = remote_node.data_id
|
local_node.context["bind_data_id"] = remote_node.data_id
|
||||||
else:
|
else:
|
||||||
local_node.context.pop("bind_data_id", None)
|
local_node.context.pop("bind_data_id", None)
|
||||||
|
|
||||||
for remote_node in strategy.remote_collection.filter(node_type=strategy.node_type):
|
for remote_node_id in target_remote_node_ids:
|
||||||
|
remote_node = remote_collection.get_by_node_id(remote_node_id)
|
||||||
|
if remote_node is None:
|
||||||
|
continue
|
||||||
local_node_id = remote_to_local.get(remote_node.node_id)
|
local_node_id = remote_to_local.get(remote_node.node_id)
|
||||||
local_node = strategy.local_collection.get_by_node_id(local_node_id) if local_node_id else None
|
local_node = local_collection.get_by_node_id(local_node_id) if local_node_id else None
|
||||||
if local_node and local_node.data_id:
|
if local_node and local_node.data_id:
|
||||||
remote_node.context["bind_data_id"] = local_node.data_id
|
remote_node.context["bind_data_id"] = local_node.data_id
|
||||||
else:
|
else:
|
||||||
remote_node.context.pop("bind_data_id", None)
|
remote_node.context.pop("bind_data_id", None)
|
||||||
|
|
||||||
|
|
||||||
async def phase_core_state(strategy, local_nodes: List["SyncNode"], remote_nodes: List["SyncNode"]):
|
async def phase_core_state(
|
||||||
|
*,
|
||||||
|
node_type: str,
|
||||||
|
runtime: "StateMachineRuntime",
|
||||||
|
binding_manager: "BindingManager",
|
||||||
|
local_nodes: List["SyncNode"],
|
||||||
|
remote_nodes: List["SyncNode"],
|
||||||
|
) -> None:
|
||||||
remote_map = {n.node_id: n for n in remote_nodes}
|
remote_map = {n.node_id: n for n in remote_nodes}
|
||||||
processed_local_ids: Set[str] = set()
|
processed_local_ids: Set[str] = set()
|
||||||
processed_remote_ids: Set[str] = set()
|
processed_remote_ids: Set[str] = set()
|
||||||
|
|
||||||
for node in local_nodes:
|
for node in local_nodes:
|
||||||
remote_id = await strategy.binding_manager.get_remote_id(strategy.node_type, node.node_id)
|
remote_id = await binding_manager.get_remote_id(node_type, node.node_id)
|
||||||
|
|
||||||
if remote_id:
|
if remote_id:
|
||||||
remote_node = remote_map.get(remote_id)
|
remote_node = remote_map.get(remote_id)
|
||||||
@@ -84,7 +155,7 @@ async def phase_core_state(strategy, local_nodes: List["SyncNode"], remote_nodes
|
|||||||
local_valid = local_data is not None
|
local_valid = local_data is not None
|
||||||
peer_valid = remote_data is not None
|
peer_valid = remote_data is not None
|
||||||
local_decision = e10_bind_core(
|
local_decision = e10_bind_core(
|
||||||
strategy.ensure_runtime(),
|
runtime,
|
||||||
node=node,
|
node=node,
|
||||||
has_binding_record=True,
|
has_binding_record=True,
|
||||||
local_data=local_data,
|
local_data=local_data,
|
||||||
@@ -94,14 +165,14 @@ async def phase_core_state(strategy, local_nodes: List["SyncNode"], remote_nodes
|
|||||||
)
|
)
|
||||||
local_reason = decision_note(local_decision)
|
local_reason = decision_note(local_decision)
|
||||||
if local_decision.to_state in {"S03", "S04"}:
|
if local_decision.to_state in {"S03", "S04"}:
|
||||||
logger.debug(f"[{strategy.node_type}] 核心绑定阻断: node={node.node_id}, reason={local_reason}")
|
logger.debug(f"[{node_type}] 核心绑定阻断: node={node.node_id}, reason={local_reason}")
|
||||||
else:
|
else:
|
||||||
logger.debug(f"[{strategy.node_type}] 核心绑定完成: node={node.node_id}, reason={local_reason}")
|
logger.debug(f"[{node_type}] 核心绑定完成: node={node.node_id}, reason={local_reason}")
|
||||||
processed_local_ids.add(node.node_id)
|
processed_local_ids.add(node.node_id)
|
||||||
|
|
||||||
if remote_node:
|
if remote_node:
|
||||||
peer_decision = e10_bind_core(
|
peer_decision = e10_bind_core(
|
||||||
strategy.ensure_runtime(),
|
runtime,
|
||||||
node=remote_node,
|
node=remote_node,
|
||||||
has_binding_record=True,
|
has_binding_record=True,
|
||||||
local_data=remote_data,
|
local_data=remote_data,
|
||||||
@@ -111,32 +182,22 @@ async def phase_core_state(strategy, local_nodes: List["SyncNode"], remote_nodes
|
|||||||
)
|
)
|
||||||
peer_reason = decision_note(peer_decision)
|
peer_reason = decision_note(peer_decision)
|
||||||
if peer_decision.to_state in {"S03", "S04"}:
|
if peer_decision.to_state in {"S03", "S04"}:
|
||||||
logger.debug(
|
logger.debug(f"[{node_type}] 核心绑定阻断: node={remote_node.node_id}, reason={peer_reason}")
|
||||||
f"[{strategy.node_type}] 核心绑定阻断: node={remote_node.node_id}, reason={peer_reason}"
|
|
||||||
)
|
|
||||||
else:
|
else:
|
||||||
logger.debug(f"[{strategy.node_type}] 核心绑定完成: node={remote_node.node_id}, reason={peer_reason}")
|
logger.debug(f"[{node_type}] 核心绑定完成: node={remote_node.node_id}, reason={peer_reason}")
|
||||||
processed_remote_ids.add(remote_node.node_id)
|
processed_remote_ids.add(remote_node.node_id)
|
||||||
|
|
||||||
for node in local_nodes:
|
for node in local_nodes:
|
||||||
if node.node_id not in processed_local_ids:
|
if node.node_id not in processed_local_ids:
|
||||||
decision = e10_bind_core(
|
decision = e10_bind_core(runtime, node=node, has_binding_record=False)
|
||||||
strategy.ensure_runtime(),
|
|
||||||
node=node,
|
|
||||||
has_binding_record=False,
|
|
||||||
)
|
|
||||||
reason = decision_note(decision)
|
reason = decision_note(decision)
|
||||||
logger.debug(f"[{strategy.node_type}] 核心绑定完成: node={node.node_id}, reason={reason}")
|
logger.debug(f"[{node_type}] 核心绑定完成: node={node.node_id}, reason={reason}")
|
||||||
|
|
||||||
for r_node in remote_nodes:
|
for remote_node in remote_nodes:
|
||||||
if r_node.node_id not in processed_remote_ids:
|
if remote_node.node_id not in processed_remote_ids:
|
||||||
decision = e10_bind_core(
|
decision = e10_bind_core(runtime, node=remote_node, has_binding_record=False)
|
||||||
strategy.ensure_runtime(),
|
|
||||||
node=r_node,
|
|
||||||
has_binding_record=False,
|
|
||||||
)
|
|
||||||
reason = decision_note(decision)
|
reason = decision_note(decision)
|
||||||
logger.debug(f"[{strategy.node_type}] 核心绑定完成: node={r_node.node_id}, reason={reason}")
|
logger.debug(f"[{node_type}] 核心绑定完成: node={remote_node.node_id}, reason={reason}")
|
||||||
|
|
||||||
|
|
||||||
def check_dependency_satisfied(
|
def check_dependency_satisfied(
|
||||||
@@ -206,17 +267,45 @@ def collect_dependency_errors(
|
|||||||
return errors
|
return errors
|
||||||
|
|
||||||
|
|
||||||
async def phase_dependency_check(strategy, local_nodes: List["SyncNode"], remote_nodes: List["SyncNode"]):
|
async def phase_dependency_check(
|
||||||
await check_dependencies_for_nodes(strategy, local_nodes, strategy.local_collection)
|
*,
|
||||||
await check_dependencies_for_nodes(strategy, remote_nodes, strategy.remote_collection)
|
node_type: str,
|
||||||
|
runtime: "StateMachineRuntime",
|
||||||
|
depend_fields: Mapping[str, str],
|
||||||
|
local_nodes: List["SyncNode"],
|
||||||
|
remote_nodes: List["SyncNode"],
|
||||||
|
local_collection: "DataCollection",
|
||||||
|
remote_collection: "DataCollection",
|
||||||
|
) -> None:
|
||||||
|
await check_dependencies_for_nodes(
|
||||||
|
node_type=node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
depend_fields=depend_fields,
|
||||||
|
nodes=local_nodes,
|
||||||
|
collection=local_collection,
|
||||||
|
)
|
||||||
|
await check_dependencies_for_nodes(
|
||||||
|
node_type=node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
depend_fields=depend_fields,
|
||||||
|
nodes=remote_nodes,
|
||||||
|
collection=remote_collection,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
async def check_dependencies_for_nodes(strategy, nodes: List["SyncNode"], collection: "DataCollection"):
|
async def check_dependencies_for_nodes(
|
||||||
|
*,
|
||||||
|
node_type: str,
|
||||||
|
runtime: "StateMachineRuntime",
|
||||||
|
depend_fields: Mapping[str, str],
|
||||||
|
nodes: List["SyncNode"],
|
||||||
|
collection: "DataCollection",
|
||||||
|
) -> None:
|
||||||
for node in nodes:
|
for node in nodes:
|
||||||
if node.binding_status != BindingStatus.MISSING:
|
if node.binding_status != BindingStatus.MISSING:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
has_depend_fields = bool(strategy.config.depend_fields)
|
has_depend_fields = bool(depend_fields)
|
||||||
node_data = node.get_data()
|
node_data = node.get_data()
|
||||||
data_present = bool(node_data)
|
data_present = bool(node_data)
|
||||||
|
|
||||||
@@ -228,7 +317,7 @@ async def check_dependencies_for_nodes(strategy, nodes: List["SyncNode"], collec
|
|||||||
semantic_errors: List[str] = []
|
semantic_errors: List[str] = []
|
||||||
|
|
||||||
if has_depend_fields and data_present:
|
if has_depend_fields and data_present:
|
||||||
for field_name, dep_node_type in strategy.config.depend_fields.items():
|
for field_name, dep_node_type in depend_fields.items():
|
||||||
dep_data_values = _normalize_dependency_field_values(node_data.get(field_name))
|
dep_data_values = _normalize_dependency_field_values(node_data.get(field_name))
|
||||||
if not dep_data_values:
|
if not dep_data_values:
|
||||||
continue
|
continue
|
||||||
@@ -282,7 +371,7 @@ async def check_dependencies_for_nodes(strategy, nodes: List["SyncNode"], collec
|
|||||||
)
|
)
|
||||||
|
|
||||||
decision = e15_bind_dependency(
|
decision = e15_bind_dependency(
|
||||||
strategy.ensure_runtime(),
|
runtime,
|
||||||
node=node,
|
node=node,
|
||||||
data_present=data_present,
|
data_present=data_present,
|
||||||
has_depend_fields=has_depend_fields,
|
has_depend_fields=has_depend_fields,
|
||||||
@@ -293,34 +382,77 @@ async def check_dependencies_for_nodes(strategy, nodes: List["SyncNode"], collec
|
|||||||
reason = decision_note(decision)
|
reason = decision_note(decision)
|
||||||
if decision.to_state in {"S03", "S04", "S05"}:
|
if decision.to_state in {"S03", "S04", "S05"}:
|
||||||
detail_reason = node.error or reason
|
detail_reason = node.error or reason
|
||||||
logger.debug(f"[{strategy.node_type}] 依赖检查阻断: node={node.node_id}, reason={detail_reason}")
|
logger.debug(f"[{node_type}] 依赖检查阻断: node={node.node_id}, reason={detail_reason}")
|
||||||
|
|
||||||
|
|
||||||
async def phase_auto_binding(strategy, local_nodes: List["SyncNode"], remote_nodes: List["SyncNode"]):
|
async def phase_auto_binding(
|
||||||
pending_auto_bind_updates: List[Dict[str, Any]] = []
|
*,
|
||||||
|
node_type: str,
|
||||||
|
runtime: "StateMachineRuntime",
|
||||||
|
config: "StrategyConfig",
|
||||||
|
local_collection: "DataCollection",
|
||||||
|
remote_collection: "DataCollection",
|
||||||
|
binding_manager: "BindingManager",
|
||||||
|
local_nodes: List["SyncNode"],
|
||||||
|
remote_nodes: List["SyncNode"],
|
||||||
|
id_field_hints: Mapping[str, str],
|
||||||
|
local_orphan_create_enabled_by_node: Mapping[str, bool],
|
||||||
|
remote_orphan_create_enabled_by_node: Mapping[str, bool],
|
||||||
|
) -> None:
|
||||||
|
pending_auto_bind_updates = await plan_auto_bind_updates(
|
||||||
|
node_type=node_type,
|
||||||
|
config=config,
|
||||||
|
local_collection=local_collection,
|
||||||
|
remote_collection=remote_collection,
|
||||||
|
binding_manager=binding_manager,
|
||||||
|
local_nodes=local_nodes,
|
||||||
|
remote_nodes=remote_nodes,
|
||||||
|
id_field_hints=id_field_hints,
|
||||||
|
local_orphan_create_enabled_by_node=local_orphan_create_enabled_by_node,
|
||||||
|
remote_orphan_create_enabled_by_node=remote_orphan_create_enabled_by_node,
|
||||||
|
)
|
||||||
|
await apply_auto_bind_updates(
|
||||||
|
node_type=node_type,
|
||||||
|
runtime=runtime,
|
||||||
|
binding_manager=binding_manager,
|
||||||
|
pending_auto_bind_updates=pending_auto_bind_updates,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def plan_auto_bind_updates(
|
||||||
|
*,
|
||||||
|
node_type: str,
|
||||||
|
config: "StrategyConfig",
|
||||||
|
local_collection: "DataCollection",
|
||||||
|
remote_collection: "DataCollection",
|
||||||
|
binding_manager: "BindingManager",
|
||||||
|
local_nodes: List["SyncNode"],
|
||||||
|
remote_nodes: List["SyncNode"],
|
||||||
|
id_field_hints: Mapping[str, str],
|
||||||
|
local_orphan_create_enabled_by_node: Mapping[str, bool],
|
||||||
|
remote_orphan_create_enabled_by_node: Mapping[str, bool],
|
||||||
|
) -> List[PendingAutoBindUpdate]:
|
||||||
|
pending_auto_bind_updates: List[PendingAutoBindUpdate] = []
|
||||||
prebound_node_ids: Set[str] = set()
|
prebound_node_ids: Set[str] = set()
|
||||||
|
|
||||||
local_s02_ids = {
|
local_s02_ids = {
|
||||||
n.node_id
|
n.node_id
|
||||||
for n in strategy.local_collection.filter_by_state_ids(
|
for n in local_collection.filter_by_state_ids(
|
||||||
node_type=strategy.node_type,
|
node_type=node_type,
|
||||||
state_ids=["S02"],
|
state_ids=["S02"],
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
remote_s02_ids = {
|
remote_s02_ids = {
|
||||||
n.node_id
|
n.node_id
|
||||||
for n in strategy.remote_collection.filter_by_state_ids(
|
for n in remote_collection.filter_by_state_ids(
|
||||||
node_type=strategy.node_type,
|
node_type=node_type,
|
||||||
state_ids=["S02"],
|
state_ids=["S02"],
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
local_create_enabled = strategy.config.local_orphan_action == OrphanAction.CREATE_REMOTE
|
for prebind_pair in config.pre_bind_data_id:
|
||||||
remote_create_enabled = strategy.config.remote_orphan_action == OrphanAction.CREATE_LOCAL
|
local_node = local_collection.get_by_data_id(node_type, prebind_pair.local_data_id)
|
||||||
|
remote_node = remote_collection.get_by_data_id(node_type, prebind_pair.remote_data_id)
|
||||||
for prebind_pair in strategy.config.pre_bind_data_id:
|
|
||||||
local_node = strategy.local_collection.get_by_data_id(strategy.node_type, prebind_pair.local_data_id)
|
|
||||||
remote_node = strategy.remote_collection.get_by_data_id(strategy.node_type, prebind_pair.remote_data_id)
|
|
||||||
|
|
||||||
if local_node is None or remote_node is None:
|
if local_node is None or remote_node is None:
|
||||||
continue
|
continue
|
||||||
@@ -329,325 +461,269 @@ async def phase_auto_binding(strategy, local_nodes: List["SyncNode"], remote_nod
|
|||||||
if local_node.binding_status != BindingStatus.MISSING or remote_node.binding_status != BindingStatus.MISSING:
|
if local_node.binding_status != BindingStatus.MISSING or remote_node.binding_status != BindingStatus.MISSING:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
existing_remote = await strategy.binding_manager.get_remote_id(strategy.node_type, local_node.node_id)
|
existing_remote = await binding_manager.get_remote_id(node_type, local_node.node_id)
|
||||||
existing_local = await strategy.binding_manager.get_local_id(strategy.node_type, remote_node.node_id)
|
existing_local = await binding_manager.get_local_id(node_type, remote_node.node_id)
|
||||||
if existing_remote or existing_local:
|
if existing_remote or existing_local:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
pending_auto_bind_updates.append(
|
pending_auto_bind_updates.append(
|
||||||
{
|
PendingAutoBindUpdate(
|
||||||
"node": local_node,
|
node=local_node,
|
||||||
"outcome": "ONE_TO_ONE",
|
outcome="ONE_TO_ONE",
|
||||||
"create_enabled": None,
|
create_enabled=None,
|
||||||
"bind_remote_node_id": remote_node.node_id,
|
bind_remote_node_id=remote_node.node_id,
|
||||||
}
|
)
|
||||||
)
|
)
|
||||||
pending_auto_bind_updates.append(
|
pending_auto_bind_updates.append(
|
||||||
{
|
PendingAutoBindUpdate(node=remote_node, outcome="ONE_TO_ONE", create_enabled=None)
|
||||||
"node": remote_node,
|
|
||||||
"outcome": "ONE_TO_ONE",
|
|
||||||
"create_enabled": None,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
)
|
||||||
prebound_node_ids.add(local_node.node_id)
|
prebound_node_ids.add(local_node.node_id)
|
||||||
prebound_node_ids.add(remote_node.node_id)
|
prebound_node_ids.add(remote_node.node_id)
|
||||||
|
|
||||||
if not strategy.config.auto_bind or not strategy.config.auto_bind_fields:
|
if not config.auto_bind or not config.auto_bind_fields:
|
||||||
for node in local_nodes:
|
for node in local_nodes:
|
||||||
if node.node_id in prebound_node_ids:
|
if node.node_id in prebound_node_ids:
|
||||||
continue
|
continue
|
||||||
if node.binding_status == BindingStatus.MISSING:
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
pending_auto_bind_updates.append({"node": node, "outcome": None, "create_enabled": None})
|
pending_auto_bind_updates.append(PendingAutoBindUpdate(node=node, outcome=None, create_enabled=None))
|
||||||
for node in remote_nodes:
|
for node in remote_nodes:
|
||||||
if node.node_id in prebound_node_ids:
|
if node.node_id in prebound_node_ids:
|
||||||
continue
|
continue
|
||||||
if node.binding_status == BindingStatus.MISSING:
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
pending_auto_bind_updates.append({"node": node, "outcome": None, "create_enabled": None})
|
pending_auto_bind_updates.append(PendingAutoBindUpdate(node=node, outcome=None, create_enabled=None))
|
||||||
else:
|
return pending_auto_bind_updates
|
||||||
id_resolver = IDResolver(strategy.local_collection, strategy.remote_collection, strategy.binding_manager)
|
|
||||||
|
|
||||||
local_key_map: Dict[Tuple, List["SyncNode"]] = {}
|
id_resolver = IDResolver(local_collection, remote_collection, binding_manager)
|
||||||
remote_key_map: Dict[Tuple, List["SyncNode"]] = {}
|
local_key_map: Dict[Tuple, List["SyncNode"]] = {}
|
||||||
|
remote_key_map: Dict[Tuple, List["SyncNode"]] = {}
|
||||||
|
|
||||||
for node in local_nodes:
|
async def _collect_side_key_map(
|
||||||
if node.node_id in prebound_node_ids:
|
*,
|
||||||
continue
|
nodes: List["SyncNode"],
|
||||||
if node.node_id not in local_s02_ids:
|
s02_ids: Set[str],
|
||||||
|
key_map: Dict[Tuple, List["SyncNode"]],
|
||||||
|
source_is_local: bool,
|
||||||
|
) -> None:
|
||||||
|
for node in nodes:
|
||||||
|
if node.node_id in prebound_node_ids or node.node_id not in s02_ids:
|
||||||
continue
|
continue
|
||||||
node_data = node.get_data()
|
node_data = node.get_data()
|
||||||
if not node_data:
|
if not node_data:
|
||||||
if node.binding_status == BindingStatus.MISSING:
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
pending_auto_bind_updates.append(
|
pending_auto_bind_updates.append(
|
||||||
{
|
PendingAutoBindUpdate(node=node, outcome="DATA_MISSING", create_enabled=None)
|
||||||
"node": node,
|
|
||||||
"outcome": "DATA_MISSING",
|
|
||||||
"create_enabled": None,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
missing_fields = [fn for fn in strategy.config.auto_bind_fields if fn not in node_data]
|
missing_fields = [fn for fn in config.auto_bind_fields if fn not in node_data]
|
||||||
if missing_fields:
|
if missing_fields:
|
||||||
if node.binding_status == BindingStatus.MISSING:
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
pending_auto_bind_updates.append(
|
pending_auto_bind_updates.append(
|
||||||
{"node": node, "outcome": "KEY_MISSING", "create_enabled": None, "bind_remote_node_id": None}
|
PendingAutoBindUpdate(node=node, outcome="KEY_MISSING", create_enabled=None)
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
key_dict = extract_biz_key_dict(node_data, strategy.config.auto_bind_fields)
|
key_dict = extract_biz_key_dict(node_data, config.auto_bind_fields)
|
||||||
if not key_dict:
|
if not key_dict:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
resolved_key_dict, resolve_error = await resolve_id_fields_in_key_dict(
|
if source_is_local:
|
||||||
key_dict,
|
key_dict, resolve_error = await resolve_id_fields_in_key_dict(
|
||||||
id_resolver,
|
key_dict,
|
||||||
get_id_field_hints(strategy),
|
id_resolver,
|
||||||
)
|
dict(id_field_hints),
|
||||||
if not resolved_key_dict:
|
)
|
||||||
|
if not key_dict:
|
||||||
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
|
pending_auto_bind_updates.append(
|
||||||
|
PendingAutoBindUpdate(
|
||||||
|
node=node,
|
||||||
|
outcome="KEY_ID_RESOLVE_FAIL",
|
||||||
|
create_enabled=None,
|
||||||
|
detail=resolve_error,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
|
key = dict_to_biz_key_tuple(key_dict, config.auto_bind_fields)
|
||||||
|
key_map.setdefault(key, []).append(node)
|
||||||
|
|
||||||
|
await _collect_side_key_map(
|
||||||
|
nodes=local_nodes,
|
||||||
|
s02_ids=local_s02_ids,
|
||||||
|
key_map=local_key_map,
|
||||||
|
source_is_local=True,
|
||||||
|
)
|
||||||
|
await _collect_side_key_map(
|
||||||
|
nodes=remote_nodes,
|
||||||
|
s02_ids=remote_s02_ids,
|
||||||
|
key_map=remote_key_map,
|
||||||
|
source_is_local=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
all_keys = set(local_key_map.keys()) | set(remote_key_map.keys())
|
||||||
|
for biz_key in all_keys:
|
||||||
|
local_candidates = local_key_map.get(biz_key, [])
|
||||||
|
remote_candidates = remote_key_map.get(biz_key, [])
|
||||||
|
|
||||||
|
excluded_local = set()
|
||||||
|
excluded_remote = set()
|
||||||
|
for l_node in local_candidates:
|
||||||
|
if l_node.binding_status != BindingStatus.NORMAL:
|
||||||
|
continue
|
||||||
|
|
||||||
|
remote_id = await binding_manager.get_remote_id(node_type, l_node.node_id)
|
||||||
|
if not remote_id:
|
||||||
|
continue
|
||||||
|
|
||||||
|
for r_node in remote_candidates:
|
||||||
|
if r_node.node_id == remote_id and r_node.binding_status == BindingStatus.NORMAL:
|
||||||
|
excluded_local.add(l_node.node_id)
|
||||||
|
excluded_remote.add(r_node.node_id)
|
||||||
|
break
|
||||||
|
|
||||||
|
l_remain = [n for n in local_candidates if n.node_id not in excluded_local]
|
||||||
|
r_remain = [n for n in remote_candidates if n.node_id not in excluded_remote]
|
||||||
|
|
||||||
|
if len(l_remain) == 1 and len(r_remain) == 1:
|
||||||
|
l_node = l_remain[0]
|
||||||
|
r_node = r_remain[0]
|
||||||
|
if l_node.binding_status == BindingStatus.MISSING and r_node.binding_status == BindingStatus.MISSING:
|
||||||
|
pending_auto_bind_updates.append(
|
||||||
|
PendingAutoBindUpdate(
|
||||||
|
node=l_node,
|
||||||
|
outcome="ONE_TO_ONE",
|
||||||
|
create_enabled=None,
|
||||||
|
bind_remote_node_id=r_node.node_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
pending_auto_bind_updates.append(
|
||||||
|
PendingAutoBindUpdate(node=r_node, outcome="ONE_TO_ONE", create_enabled=None)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if len(l_remain) >= 1 and len(r_remain) == 0:
|
||||||
|
for node in l_remain:
|
||||||
if node.binding_status == BindingStatus.MISSING:
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
pending_auto_bind_updates.append(
|
pending_auto_bind_updates.append(
|
||||||
{
|
PendingAutoBindUpdate(
|
||||||
"node": node,
|
node=node,
|
||||||
"outcome": "KEY_ID_RESOLVE_FAIL",
|
outcome="ZERO",
|
||||||
"create_enabled": None,
|
create_enabled=bool(local_orphan_create_enabled_by_node.get(node.node_id, False)),
|
||||||
"bind_remote_node_id": None,
|
)
|
||||||
"detail": resolve_error,
|
|
||||||
}
|
|
||||||
)
|
)
|
||||||
continue
|
|
||||||
|
|
||||||
key = dict_to_biz_key_tuple(resolved_key_dict, strategy.config.auto_bind_fields)
|
if len(r_remain) >= 1 and len(l_remain) == 0:
|
||||||
local_key_map.setdefault(key, []).append(node)
|
for node in r_remain:
|
||||||
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
for r_node in remote_nodes:
|
|
||||||
if r_node.node_id in prebound_node_ids:
|
|
||||||
continue
|
|
||||||
if r_node.node_id not in remote_s02_ids:
|
|
||||||
continue
|
|
||||||
r_node_data = r_node.get_data()
|
|
||||||
if not r_node_data:
|
|
||||||
if r_node.binding_status == BindingStatus.MISSING:
|
|
||||||
pending_auto_bind_updates.append(
|
pending_auto_bind_updates.append(
|
||||||
{
|
PendingAutoBindUpdate(
|
||||||
"node": r_node,
|
node=node,
|
||||||
"outcome": "DATA_MISSING",
|
outcome="ZERO",
|
||||||
"create_enabled": None,
|
create_enabled=bool(remote_orphan_create_enabled_by_node.get(node.node_id, False)),
|
||||||
"bind_remote_node_id": None,
|
)
|
||||||
}
|
|
||||||
)
|
)
|
||||||
continue
|
|
||||||
|
|
||||||
missing_fields = [fn for fn in strategy.config.auto_bind_fields if fn not in r_node_data]
|
many_local_one_remote = len(l_remain) > 1 and len(r_remain) == 1
|
||||||
if missing_fields:
|
one_local_many_remote = len(l_remain) == 1 and len(r_remain) > 1
|
||||||
if r_node.binding_status == BindingStatus.MISSING:
|
if config.allow_many_to_one_auto_bind and (many_local_one_remote or one_local_many_remote):
|
||||||
pending_auto_bind_updates.append(
|
chosen_local, chosen_remote = _stable_pick_bind_pair(l_remain, r_remain)
|
||||||
{
|
if chosen_local.binding_status == BindingStatus.MISSING:
|
||||||
"node": r_node,
|
pending_auto_bind_updates.append(
|
||||||
"outcome": "KEY_MISSING",
|
PendingAutoBindUpdate(
|
||||||
"create_enabled": None,
|
node=chosen_local,
|
||||||
"bind_remote_node_id": None,
|
outcome="ONE_TO_ONE",
|
||||||
}
|
create_enabled=None,
|
||||||
|
bind_remote_node_id=chosen_remote.node_id,
|
||||||
)
|
)
|
||||||
continue
|
)
|
||||||
|
if chosen_remote.binding_status == BindingStatus.MISSING:
|
||||||
|
pending_auto_bind_updates.append(
|
||||||
|
PendingAutoBindUpdate(node=chosen_remote, outcome="ONE_TO_ONE", create_enabled=None)
|
||||||
|
)
|
||||||
|
|
||||||
key_dict = extract_biz_key_dict(r_node_data, strategy.config.auto_bind_fields)
|
for node in l_remain:
|
||||||
if key_dict:
|
if node.node_id == chosen_local.node_id or node.binding_status != BindingStatus.MISSING:
|
||||||
key = dict_to_biz_key_tuple(key_dict, strategy.config.auto_bind_fields)
|
|
||||||
remote_key_map.setdefault(key, []).append(r_node)
|
|
||||||
|
|
||||||
all_keys = set(local_key_map.keys()) | set(remote_key_map.keys())
|
|
||||||
|
|
||||||
for biz_key in all_keys:
|
|
||||||
local_candidates = local_key_map.get(biz_key, [])
|
|
||||||
remote_candidates = remote_key_map.get(biz_key, [])
|
|
||||||
|
|
||||||
excluded_local = set()
|
|
||||||
excluded_remote = set()
|
|
||||||
|
|
||||||
for l_node in local_candidates:
|
|
||||||
if l_node.binding_status != BindingStatus.NORMAL:
|
|
||||||
continue
|
continue
|
||||||
|
pending_auto_bind_updates.append(
|
||||||
remote_id = await strategy.binding_manager.get_remote_id(strategy.node_type, l_node.node_id)
|
PendingAutoBindUpdate(
|
||||||
if not remote_id:
|
node=node,
|
||||||
|
outcome="ZERO",
|
||||||
|
create_enabled=bool(local_orphan_create_enabled_by_node.get(node.node_id, False)),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
for node in r_remain:
|
||||||
|
if node.node_id == chosen_remote.node_id or node.binding_status != BindingStatus.MISSING:
|
||||||
continue
|
continue
|
||||||
|
pending_auto_bind_updates.append(
|
||||||
for r_node in remote_candidates:
|
PendingAutoBindUpdate(
|
||||||
if r_node.node_id == remote_id and r_node.binding_status == BindingStatus.NORMAL:
|
node=node,
|
||||||
excluded_local.add(l_node.node_id)
|
outcome="ZERO",
|
||||||
excluded_remote.add(r_node.node_id)
|
create_enabled=bool(remote_orphan_create_enabled_by_node.get(node.node_id, False)),
|
||||||
break
|
|
||||||
|
|
||||||
l_remain = [n for n in local_candidates if n.node_id not in excluded_local]
|
|
||||||
r_remain = [n for n in remote_candidates if n.node_id not in excluded_remote]
|
|
||||||
|
|
||||||
if len(l_remain) == 1 and len(r_remain) == 1:
|
|
||||||
l_node = l_remain[0]
|
|
||||||
r_node = r_remain[0]
|
|
||||||
|
|
||||||
if l_node.binding_status == BindingStatus.MISSING and r_node.binding_status == BindingStatus.MISSING:
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": l_node,
|
|
||||||
"outcome": "ONE_TO_ONE",
|
|
||||||
"create_enabled": None,
|
|
||||||
"bind_remote_node_id": r_node.node_id,
|
|
||||||
}
|
|
||||||
)
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if (len(l_remain) > 1 and len(r_remain) >= 1) or (len(l_remain) == 1 and len(r_remain) > 1):
|
||||||
|
for node in l_remain:
|
||||||
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
pending_auto_bind_updates.append(
|
pending_auto_bind_updates.append(
|
||||||
{
|
PendingAutoBindUpdate(node=node, outcome="AMBIGUOUS", create_enabled=None)
|
||||||
"node": r_node,
|
)
|
||||||
"outcome": "ONE_TO_ONE",
|
if (len(r_remain) > 1 and len(l_remain) >= 1) or (len(r_remain) == 1 and len(l_remain) > 1):
|
||||||
"create_enabled": None,
|
for node in r_remain:
|
||||||
"bind_remote_node_id": None,
|
if node.binding_status == BindingStatus.MISSING:
|
||||||
}
|
pending_auto_bind_updates.append(
|
||||||
|
PendingAutoBindUpdate(node=node, outcome="AMBIGUOUS", create_enabled=None)
|
||||||
)
|
)
|
||||||
else:
|
|
||||||
if len(l_remain) >= 1 and len(r_remain) == 0:
|
|
||||||
for n in l_remain:
|
|
||||||
if n.binding_status == BindingStatus.MISSING:
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": n,
|
|
||||||
"outcome": "ZERO",
|
|
||||||
"create_enabled": local_create_enabled,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
if len(r_remain) >= 1 and len(l_remain) == 0:
|
return pending_auto_bind_updates
|
||||||
for n in r_remain:
|
|
||||||
if n.binding_status == BindingStatus.MISSING:
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": n,
|
|
||||||
"outcome": "ZERO",
|
|
||||||
"create_enabled": remote_create_enabled,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
many_local_one_remote = len(l_remain) > 1 and len(r_remain) == 1
|
|
||||||
one_local_many_remote = len(l_remain) == 1 and len(r_remain) > 1
|
|
||||||
|
|
||||||
if strategy.config.allow_many_to_one_auto_bind and (many_local_one_remote or one_local_many_remote):
|
|
||||||
chosen_local, chosen_remote = _stable_pick_bind_pair(l_remain, r_remain)
|
|
||||||
|
|
||||||
if chosen_local.binding_status == BindingStatus.MISSING:
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": chosen_local,
|
|
||||||
"outcome": "ONE_TO_ONE",
|
|
||||||
"create_enabled": None,
|
|
||||||
"bind_remote_node_id": chosen_remote.node_id,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
if chosen_remote.binding_status == BindingStatus.MISSING:
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": chosen_remote,
|
|
||||||
"outcome": "ONE_TO_ONE",
|
|
||||||
"create_enabled": None,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
for n in l_remain:
|
|
||||||
if n.node_id == chosen_local.node_id or n.binding_status != BindingStatus.MISSING:
|
|
||||||
continue
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": n,
|
|
||||||
"outcome": "ZERO",
|
|
||||||
"create_enabled": local_create_enabled,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
for n in r_remain:
|
|
||||||
if n.node_id == chosen_remote.node_id or n.binding_status != BindingStatus.MISSING:
|
|
||||||
continue
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": n,
|
|
||||||
"outcome": "ZERO",
|
|
||||||
"create_enabled": remote_create_enabled,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
if (len(l_remain) > 1 and len(r_remain) >= 1) or (len(l_remain) == 1 and len(r_remain) > 1):
|
|
||||||
for n in l_remain:
|
|
||||||
if n.binding_status == BindingStatus.MISSING:
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": n,
|
|
||||||
"outcome": "AMBIGUOUS",
|
|
||||||
"create_enabled": None,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
if (len(r_remain) > 1 and len(l_remain) >= 1) or (len(r_remain) == 1 and len(l_remain) > 1):
|
|
||||||
for n in r_remain:
|
|
||||||
if n.binding_status == BindingStatus.MISSING:
|
|
||||||
pending_auto_bind_updates.append(
|
|
||||||
{
|
|
||||||
"node": n,
|
|
||||||
"outcome": "AMBIGUOUS",
|
|
||||||
"create_enabled": None,
|
|
||||||
"bind_remote_node_id": None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
|
async def apply_auto_bind_updates(
|
||||||
|
*,
|
||||||
|
node_type: str,
|
||||||
|
runtime: "StateMachineRuntime",
|
||||||
|
binding_manager: "BindingManager",
|
||||||
|
pending_auto_bind_updates: List[PendingAutoBindUpdate],
|
||||||
|
) -> None:
|
||||||
for item in pending_auto_bind_updates:
|
for item in pending_auto_bind_updates:
|
||||||
node = item["node"]
|
auto_bind_enabled = item.outcome is not None
|
||||||
auto_bind_outcome = item["outcome"]
|
|
||||||
create_enabled = item["create_enabled"]
|
|
||||||
bind_remote_node_id = item.get("bind_remote_node_id")
|
|
||||||
detail = item.get("detail")
|
|
||||||
auto_bind_enabled = auto_bind_outcome is not None
|
|
||||||
decision = e16_bind_auto(
|
decision = e16_bind_auto(
|
||||||
strategy.ensure_runtime(),
|
runtime,
|
||||||
node=node,
|
node=item.node,
|
||||||
auto_bind_enabled=auto_bind_enabled,
|
auto_bind_enabled=auto_bind_enabled,
|
||||||
auto_bind_outcome=auto_bind_outcome,
|
auto_bind_outcome=item.outcome,
|
||||||
create_enabled=create_enabled,
|
create_enabled=item.create_enabled,
|
||||||
id_resolve_detail=detail,
|
id_resolve_detail=item.detail,
|
||||||
)
|
)
|
||||||
reason = decision_note(decision)
|
reason = decision_note(decision)
|
||||||
if decision.to_state == "S01" and auto_bind_outcome == "ONE_TO_ONE":
|
if decision.to_state == "S01" and item.outcome == "ONE_TO_ONE":
|
||||||
if bind_remote_node_id:
|
if item.bind_remote_node_id:
|
||||||
await strategy.binding_manager.bind(
|
await binding_manager.bind(node_type, item.node.node_id, item.bind_remote_node_id)
|
||||||
strategy.node_type,
|
logger.debug(f"[{node_type}] 自动绑定成功: node={item.node.node_id}")
|
||||||
node.node_id,
|
|
||||||
bind_remote_node_id,
|
|
||||||
)
|
|
||||||
logger.debug(f"[{strategy.node_type}] 自动绑定成功: node={node.node_id}")
|
|
||||||
if decision.to_state in {"S04", "S05"}:
|
if decision.to_state in {"S04", "S05"}:
|
||||||
if detail:
|
if item.detail:
|
||||||
reason = f"{reason} | detail={detail}"
|
reason = f"{reason} | detail={item.detail}"
|
||||||
node.error = reason
|
item.node.error = reason
|
||||||
logger.debug(f"[{strategy.node_type}] 自动绑定阻断: node={node.node_id}, reason={reason}")
|
logger.debug(f"[{node_type}] 自动绑定阻断: node={item.node.node_id}, reason={reason}")
|
||||||
|
|
||||||
outcome_counts: Dict[str, int] = {}
|
outcome_counts: Dict[str, int] = {}
|
||||||
create_enabled_count = 0
|
create_enabled_count = 0
|
||||||
bound_pairs = 0
|
bound_pairs = 0
|
||||||
for item in pending_auto_bind_updates:
|
for item in pending_auto_bind_updates:
|
||||||
outcome = str(item.get("outcome") or "NONE")
|
outcome = str(item.outcome or "NONE")
|
||||||
outcome_counts[outcome] = outcome_counts.get(outcome, 0) + 1
|
outcome_counts[outcome] = outcome_counts.get(outcome, 0) + 1
|
||||||
if item.get("create_enabled"):
|
if item.create_enabled:
|
||||||
create_enabled_count += 1
|
create_enabled_count += 1
|
||||||
if item.get("bind_remote_node_id"):
|
if item.bind_remote_node_id:
|
||||||
bound_pairs += 1
|
bound_pairs += 1
|
||||||
|
|
||||||
if pending_auto_bind_updates:
|
if pending_auto_bind_updates:
|
||||||
ordered = ", ".join(f"{key}={outcome_counts[key]}" for key in sorted(outcome_counts))
|
ordered = ", ".join(f"{key}={outcome_counts[key]}" for key in sorted(outcome_counts))
|
||||||
logger.info(
|
logger.info(
|
||||||
"[%s] Auto-bind summary: candidates=%d, bind_pairs=%d, create_enabled=%d, outcomes={%s}",
|
"[%s] Auto-bind summary: candidates=%d, bind_pairs=%d, create_enabled=%d, outcomes={%s}",
|
||||||
strategy.node_type,
|
node_type,
|
||||||
len(pending_auto_bind_updates),
|
len(pending_auto_bind_updates),
|
||||||
bound_pairs,
|
bound_pairs,
|
||||||
create_enabled_count,
|
create_enabled_count,
|
||||||
|
|||||||
@@ -113,40 +113,6 @@ async def add_create_action(
|
|||||||
return created_nodes
|
return created_nodes
|
||||||
|
|
||||||
|
|
||||||
async def run_create(strategy) -> List["SyncNode"]:
|
|
||||||
created_nodes: List["SyncNode"] = []
|
|
||||||
|
|
||||||
if strategy.config.local_orphan_action == OrphanAction.CREATE_REMOTE:
|
|
||||||
local_nodes = await add_create_action(
|
|
||||||
strategy,
|
|
||||||
from_collection=strategy.local_collection,
|
|
||||||
to_collection=strategy.remote_collection,
|
|
||||||
source_is_local=True,
|
|
||||||
)
|
|
||||||
created_nodes.extend(local_nodes)
|
|
||||||
logger.info(f"[{strategy.node_type}] Created {len(local_nodes)} nodes in remote collection (Push)")
|
|
||||||
elif strategy.config.local_orphan_action == OrphanAction.DELETE_LOCAL:
|
|
||||||
raise NotImplementedError(
|
|
||||||
f"[{strategy.node_type}] local_orphan_action=DELETE_LOCAL is not supported in run_create; "
|
|
||||||
"use delete stage (delete_ops) explicitly."
|
|
||||||
)
|
|
||||||
|
|
||||||
if strategy.config.remote_orphan_action == OrphanAction.CREATE_LOCAL:
|
|
||||||
remote_nodes = await add_create_action(
|
|
||||||
strategy,
|
|
||||||
from_collection=strategy.remote_collection,
|
|
||||||
to_collection=strategy.local_collection,
|
|
||||||
source_is_local=False,
|
|
||||||
)
|
|
||||||
created_nodes.extend(remote_nodes)
|
|
||||||
logger.info(f"[{strategy.node_type}] Created {len(remote_nodes)} nodes in local collection (Pull)")
|
|
||||||
elif strategy.config.remote_orphan_action == OrphanAction.DELETE_REMOTE:
|
|
||||||
raise NotImplementedError(
|
|
||||||
f"[{strategy.node_type}] remote_orphan_action=DELETE_REMOTE is not supported in run_create; "
|
|
||||||
"use delete stage (delete_ops) explicitly."
|
|
||||||
)
|
|
||||||
|
|
||||||
return created_nodes
|
|
||||||
|
|
||||||
|
|
||||||
async def finalize_peer_creating_sources(
|
async def finalize_peer_creating_sources(
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import logging
|
|||||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional
|
from typing import TYPE_CHECKING, Any, Dict, List, Optional
|
||||||
|
|
||||||
from ...common.types import BindingStatus, SyncAction, SyncStatus
|
from ...common.types import BindingStatus, SyncAction, SyncStatus
|
||||||
|
from ...logging import get_logger
|
||||||
from ...engine import e01_bootstrap
|
from ...engine import e01_bootstrap
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@@ -11,7 +12,7 @@ if TYPE_CHECKING:
|
|||||||
from ...common.collection import DataCollection
|
from ...common.collection import DataCollection
|
||||||
from ...engine.dispatcher import StateMachineRuntime
|
from ...engine.dispatcher import StateMachineRuntime
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
async def _resolve_local_id_for_unbind(
|
async def _resolve_local_id_for_unbind(
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import logging
|
|||||||
from typing import TYPE_CHECKING, Any, Iterable, List, Optional, Sequence
|
from typing import TYPE_CHECKING, Any, Iterable, List, Optional, Sequence
|
||||||
|
|
||||||
from ...common.types import SyncAction
|
from ...common.types import SyncAction
|
||||||
|
from ...logging import get_logger
|
||||||
from ...sync_system.config import UpdateDirection
|
from ...sync_system.config import UpdateDirection
|
||||||
from ...sync_system.resolve_ids import IDResolver
|
from ...sync_system.resolve_ids import IDResolver
|
||||||
from ...engine import decision_note, e30_update_prepare
|
from ...engine import decision_note, e30_update_prepare
|
||||||
@@ -20,7 +21,7 @@ if TYPE_CHECKING:
|
|||||||
from ...common.collection import DataCollection
|
from ...common.collection import DataCollection
|
||||||
from ...common.sync_node import SyncNode
|
from ...common.sync_node import SyncNode
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def _build_schema_diff_validator(strategy) -> SchemaDiffValidator:
|
def _build_schema_diff_validator(strategy) -> SchemaDiffValidator:
|
||||||
@@ -396,23 +397,3 @@ async def add_update_action(
|
|||||||
return updated_nodes
|
return updated_nodes
|
||||||
|
|
||||||
|
|
||||||
async def run_update(strategy) -> List["SyncNode"]:
|
|
||||||
updated_by_id: dict[str, "SyncNode"] = {}
|
|
||||||
validator = _build_schema_diff_validator(strategy)
|
|
||||||
strategy._active_schema_diff_validator = validator
|
|
||||||
data_id_map = await build_data_id_normalization_map(
|
|
||||||
node_type=strategy.node_type,
|
|
||||||
local_collection=strategy.local_collection,
|
|
||||||
remote_collection=strategy.remote_collection,
|
|
||||||
binding_manager=strategy.binding_manager,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
for pair in await collect_bound_node_pairs(strategy):
|
|
||||||
nodes = await strategy.update_pair(pair.local_node, pair.remote_node, data_id_map)
|
|
||||||
for node in nodes:
|
|
||||||
updated_by_id[node.node_id] = node
|
|
||||||
finally:
|
|
||||||
strategy._active_schema_diff_validator = None
|
|
||||||
|
|
||||||
emit_schema_diff_report(strategy, validator)
|
|
||||||
return list(updated_by_id.values())
|
|
||||||
|
|||||||
@@ -6,7 +6,9 @@
|
|||||||
from typing import List, Tuple, Optional, Any, Dict
|
from typing import List, Tuple, Optional, Any, Dict
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
from ..logging import get_logger
|
||||||
|
|
||||||
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def is_id_field(field_name: str) -> bool:
|
def is_id_field(field_name: str) -> bool:
|
||||||
|
|||||||
@@ -225,6 +225,8 @@ async def test_full_pipeline_multinode_mock_covers_business_points(pipeline_bund
|
|||||||
|
|
||||||
assert local_collection.get("lp_norm").context.get("bind_data_id") == remote_collection.get("rp_norm").data_id
|
assert local_collection.get("lp_norm").context.get("bind_data_id") == remote_collection.get("rp_norm").data_id
|
||||||
assert remote_collection.get("rp_norm").context.get("bind_data_id") == local_collection.get("lp_norm").data_id
|
assert remote_collection.get("rp_norm").context.get("bind_data_id") == local_collection.get("lp_norm").data_id
|
||||||
|
assert local_collection.get("lp_auto").context.get("bind_data_id") == remote_collection.get("rp_auto").data_id
|
||||||
|
assert remote_collection.get("rp_auto").context.get("bind_data_id") == local_collection.get("lp_auto").data_id
|
||||||
|
|
||||||
assert await binding_manager.get_remote_id("test_project", "lp_auto") == "rp_auto"
|
assert await binding_manager.get_remote_id("test_project", "lp_auto") == "rp_auto"
|
||||||
|
|
||||||
@@ -242,6 +244,12 @@ async def test_full_pipeline_multinode_mock_covers_business_points(pipeline_bund
|
|||||||
assert create_fail_nodes[0].status == SyncStatus.FAILED
|
assert create_fail_nodes[0].status == SyncStatus.FAILED
|
||||||
assert create_async_nodes[0].status == SyncStatus.PENDING
|
assert create_async_nodes[0].status == SyncStatus.PENDING
|
||||||
assert create_async_nodes[0].action == SyncAction.NONE
|
assert create_async_nodes[0].action == SyncAction.NONE
|
||||||
|
assert local_collection.get("lp_create_ok").context.get("bind_data_id") == create_ok_nodes[0].data_id
|
||||||
|
assert create_ok_nodes[0].context.get("bind_data_id") == local_collection.get("lp_create_ok").data_id
|
||||||
|
assert local_collection.get("lp_create_async").context.get("bind_data_id") == create_async_nodes[0].data_id
|
||||||
|
assert create_async_nodes[0].context.get("bind_data_id") == local_collection.get("lp_create_async").data_id
|
||||||
|
assert "bind_data_id" not in local_collection.get("lp_create_fail").context
|
||||||
|
assert "bind_data_id" not in create_fail_nodes[0].context
|
||||||
|
|
||||||
assert remote_collection.get("rc_upd_skip").status == SyncStatus.SKIPPED
|
assert remote_collection.get("rc_upd_skip").status == SyncStatus.SKIPPED
|
||||||
assert remote_collection.get("rc_upd_skip").action == SyncAction.NONE
|
assert remote_collection.get("rc_upd_skip").action == SyncAction.NONE
|
||||||
|
|||||||
@@ -100,9 +100,11 @@ async def test_core_binding_matrix_migrated(
|
|||||||
await bm.bind("test_project", f"{name}_l", f"{name}_r")
|
await bm.bind("test_project", f"{name}_l", f"{name}_r")
|
||||||
|
|
||||||
await phase_core_state(
|
await phase_core_state(
|
||||||
strategy,
|
node_type=strategy.node_type,
|
||||||
local.filter(node_type="test_project"),
|
runtime=strategy.ensure_runtime(),
|
||||||
remote.filter(node_type="test_project"),
|
binding_manager=bm,
|
||||||
|
local_nodes=local.filter(node_type="test_project"),
|
||||||
|
remote_nodes=remote.filter(node_type="test_project"),
|
||||||
)
|
)
|
||||||
|
|
||||||
if expected_local is not None:
|
if expected_local is not None:
|
||||||
@@ -123,9 +125,11 @@ async def test_dependency_resolution_migrated(setup_env):
|
|||||||
await bm.bind("test_project", "proj_l", "proj_r")
|
await bm.bind("test_project", "proj_l", "proj_r")
|
||||||
|
|
||||||
await phase_core_state(
|
await phase_core_state(
|
||||||
project_strategy,
|
node_type=project_strategy.node_type,
|
||||||
local.filter(node_type="test_project"),
|
runtime=project_strategy.ensure_runtime(),
|
||||||
remote.filter(node_type="test_project"),
|
binding_manager=bm,
|
||||||
|
local_nodes=local.filter(node_type="test_project"),
|
||||||
|
remote_nodes=remote.filter(node_type="test_project"),
|
||||||
)
|
)
|
||||||
assert local.get("proj_l").binding_status == BindingStatus.NORMAL
|
assert local.get("proj_l").binding_status == BindingStatus.NORMAL
|
||||||
|
|
||||||
@@ -159,8 +163,22 @@ async def test_dependency_resolution_migrated(setup_env):
|
|||||||
|
|
||||||
local_contracts = local.filter(node_type="test_contract")
|
local_contracts = local.filter(node_type="test_contract")
|
||||||
remote_contracts = remote.filter(node_type="test_contract")
|
remote_contracts = remote.filter(node_type="test_contract")
|
||||||
await phase_core_state(contract_strategy, local_contracts, remote_contracts)
|
await phase_core_state(
|
||||||
await phase_dependency_check(contract_strategy, local_contracts, remote_contracts)
|
node_type=contract_strategy.node_type,
|
||||||
|
runtime=contract_strategy.ensure_runtime(),
|
||||||
|
binding_manager=bm,
|
||||||
|
local_nodes=local_contracts,
|
||||||
|
remote_nodes=remote_contracts,
|
||||||
|
)
|
||||||
|
await phase_dependency_check(
|
||||||
|
node_type=contract_strategy.node_type,
|
||||||
|
runtime=contract_strategy.ensure_runtime(),
|
||||||
|
depend_fields=dict(contract_strategy.config.depend_fields),
|
||||||
|
local_nodes=local_contracts,
|
||||||
|
remote_nodes=remote_contracts,
|
||||||
|
local_collection=local,
|
||||||
|
remote_collection=remote,
|
||||||
|
)
|
||||||
|
|
||||||
assert local.get("contract_ok").binding_status == BindingStatus.MISSING
|
assert local.get("contract_ok").binding_status == BindingStatus.MISSING
|
||||||
assert local.get("contract_ok").depend_ids == ["proj_l"]
|
assert local.get("contract_ok").depend_ids == ["proj_l"]
|
||||||
@@ -269,8 +287,22 @@ async def test_dependency_check_data_missing_goes_abnormal(setup_env):
|
|||||||
)
|
)
|
||||||
|
|
||||||
local_contracts = local.filter(node_type="test_contract")
|
local_contracts = local.filter(node_type="test_contract")
|
||||||
await phase_core_state(strategy, local_contracts, [])
|
await phase_core_state(
|
||||||
await phase_dependency_check(strategy, local_contracts, [])
|
node_type=strategy.node_type,
|
||||||
|
runtime=strategy.ensure_runtime(),
|
||||||
|
binding_manager=bm,
|
||||||
|
local_nodes=local_contracts,
|
||||||
|
remote_nodes=[],
|
||||||
|
)
|
||||||
|
await phase_dependency_check(
|
||||||
|
node_type=strategy.node_type,
|
||||||
|
runtime=strategy.ensure_runtime(),
|
||||||
|
depend_fields=dict(strategy.config.depend_fields),
|
||||||
|
local_nodes=local_contracts,
|
||||||
|
remote_nodes=[],
|
||||||
|
local_collection=local,
|
||||||
|
remote_collection=remote,
|
||||||
|
)
|
||||||
|
|
||||||
assert local.get("contract_no_data").binding_status == BindingStatus.ABNORMAL
|
assert local.get("contract_no_data").binding_status == BindingStatus.ABNORMAL
|
||||||
|
|
||||||
@@ -306,8 +338,31 @@ async def test_auto_binding_key_id_resolve_fail_migrated(setup_env):
|
|||||||
await local.add(build_project_node("kid_l", "KID-L", name="KID", code="KID-1"))
|
await local.add(build_project_node("kid_l", "KID-L", name="KID", code="KID-1"))
|
||||||
|
|
||||||
local_nodes = local.filter(node_type="test_project")
|
local_nodes = local.filter(node_type="test_project")
|
||||||
await phase_core_state(strategy, local_nodes, [])
|
await phase_core_state(
|
||||||
await phase_auto_binding(strategy, local_nodes, [])
|
node_type=strategy.node_type,
|
||||||
|
runtime=strategy.ensure_runtime(),
|
||||||
|
binding_manager=bm,
|
||||||
|
local_nodes=local_nodes,
|
||||||
|
remote_nodes=[],
|
||||||
|
)
|
||||||
|
local_flags = {
|
||||||
|
node.node_id: strategy.config.local_orphan_action == OrphanAction.CREATE_REMOTE
|
||||||
|
for node in local_nodes
|
||||||
|
}
|
||||||
|
remote_flags = {}
|
||||||
|
await phase_auto_binding(
|
||||||
|
node_type=strategy.node_type,
|
||||||
|
runtime=strategy.ensure_runtime(),
|
||||||
|
config=strategy.config,
|
||||||
|
local_collection=local,
|
||||||
|
remote_collection=remote,
|
||||||
|
binding_manager=bm,
|
||||||
|
local_nodes=local_nodes,
|
||||||
|
remote_nodes=[],
|
||||||
|
id_field_hints=dict(strategy.config.depend_fields),
|
||||||
|
local_orphan_create_enabled_by_node=local_flags,
|
||||||
|
remote_orphan_create_enabled_by_node=remote_flags,
|
||||||
|
)
|
||||||
|
|
||||||
node = local.get("kid_l")
|
node = local.get("kid_l")
|
||||||
assert node.binding_status == BindingStatus.DEPENDENCY_ERROR
|
assert node.binding_status == BindingStatus.DEPENDENCY_ERROR
|
||||||
|
|||||||
@@ -110,7 +110,7 @@ async def test_api_client_logs_one_line_request_summary_in_debug_mode(caplog: py
|
|||||||
fake_client = _FakeAsyncClient()
|
fake_client = _FakeAsyncClient()
|
||||||
client._client = fake_client
|
client._client = fake_client
|
||||||
|
|
||||||
with caplog.at_level(logging.DEBUG, logger=logger_name):
|
with caplog.at_level(logging.INFO, logger=logger_name):
|
||||||
await client.request("GET", "/ping")
|
await client.request("GET", "/ping")
|
||||||
|
|
||||||
assert "ApiClient request: method=GET endpoint=/ping elapsed=" in caplog.text
|
assert "🔎 ApiClient request: method=GET endpoint=/ping elapsed=" in caplog.text
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ class _DS(BaseDataSource):
|
|||||||
class _FailStrategy(DefaultSyncStrategy):
|
class _FailStrategy(DefaultSyncStrategy):
|
||||||
schema = object # type: ignore
|
schema = object # type: ignore
|
||||||
|
|
||||||
async def bind(self, **kwargs):
|
async def bind(self):
|
||||||
raise RuntimeError("bind failed")
|
raise RuntimeError("bind failed")
|
||||||
|
|
||||||
async def create(self):
|
async def create(self):
|
||||||
@@ -39,7 +39,7 @@ class _OkStrategy(DefaultSyncStrategy):
|
|||||||
self.create_calls = 0
|
self.create_calls = 0
|
||||||
self.update_calls = 0
|
self.update_calls = 0
|
||||||
|
|
||||||
async def bind(self, **kwargs):
|
async def bind(self):
|
||||||
self.bound = True
|
self.bound = True
|
||||||
|
|
||||||
async def create(self):
|
async def create(self):
|
||||||
|
|||||||
@@ -1,26 +1,62 @@
|
|||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
|
from typing import cast
|
||||||
from unittest.mock import AsyncMock, MagicMock, patch
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
import pytest_asyncio
|
||||||
|
|
||||||
|
import sync_state_machine.domain # noqa: F401
|
||||||
|
from sync_state_machine.common.binding import BindingManager
|
||||||
|
from sync_state_machine.common.collection import DataCollection
|
||||||
|
from sync_state_machine.common.sync_node import SyncNode
|
||||||
|
from sync_state_machine.common.sqlite_backend import SQLitePersistenceBackend
|
||||||
|
from sync_state_machine.common.types import BindingStatus
|
||||||
from sync_state_machine.domain.project_detail.sync_strategy import ProjectDetailSyncStrategy
|
from sync_state_machine.domain.project_detail.sync_strategy import ProjectDetailSyncStrategy
|
||||||
|
from sync_state_machine.domain.project_detail.sync_node import ProjectDetailSyncNode
|
||||||
from sync_state_machine.sync_system.config import UpdateDirection
|
from sync_state_machine.sync_system.config import UpdateDirection
|
||||||
from sync_state_machine.sync_system.strategy_ops import BoundNodePair
|
from sync_state_machine.sync_system.strategy_ops import BoundNodePair
|
||||||
|
|
||||||
|
|
||||||
def _fake_node(*, node_id: str, key: str):
|
def _fake_node(*, node_id: str, key: str):
|
||||||
data = {"id": node_id, "key": key}
|
data = {"id": node_id, "key": key}
|
||||||
return SimpleNamespace(
|
return cast(
|
||||||
node_id=node_id,
|
SyncNode,
|
||||||
data_id=node_id,
|
SimpleNamespace(
|
||||||
action=None,
|
node_id=node_id,
|
||||||
context={},
|
data_id=node_id,
|
||||||
get_data=lambda: data,
|
action=None,
|
||||||
|
context={},
|
||||||
|
get_data=lambda: data,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_project_detail_node(*, node_id: str, data_id: str, key: str) -> ProjectDetailSyncNode:
|
||||||
|
return ProjectDetailSyncNode(
|
||||||
|
node_id=node_id,
|
||||||
|
data_id=data_id,
|
||||||
|
data={
|
||||||
|
"id": data_id,
|
||||||
|
"project_id": "project-1",
|
||||||
|
"key": key,
|
||||||
|
"value": [{"year": 2026, "capacity": 4.0}],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture
|
||||||
|
async def project_detail_env(tmp_path):
|
||||||
|
persistence = SQLitePersistenceBackend(str(tmp_path / "project-detail.db"))
|
||||||
|
await persistence.initialize()
|
||||||
|
local = DataCollection("local", persistence=persistence, auto_persist=False)
|
||||||
|
remote = DataCollection("remote", persistence=persistence, auto_persist=False)
|
||||||
|
binding_manager = BindingManager(persistence, auto_persist=False)
|
||||||
|
yield local, remote, binding_manager, persistence
|
||||||
|
await persistence.close()
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_project_detail_update_partitions_group_production_plans_before_generic_logic():
|
async def test_project_detail_process_update_pairs_partitions_group_production_plans_before_generic_logic():
|
||||||
strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock())
|
strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock())
|
||||||
|
|
||||||
pull_pair = BoundNodePair(
|
pull_pair = BoundNodePair(
|
||||||
@@ -35,13 +71,6 @@ async def test_project_detail_update_partitions_group_production_plans_before_ge
|
|||||||
generic_node = SimpleNamespace(node_id="generic-update")
|
generic_node = SimpleNamespace(node_id="generic-update")
|
||||||
|
|
||||||
with patch.object(
|
with patch.object(
|
||||||
strategy,
|
|
||||||
"collect_update_pairs",
|
|
||||||
new=AsyncMock(return_value=[pull_pair, generic_pair]),
|
|
||||||
), patch(
|
|
||||||
"sync_state_machine.domain.project_detail.sync_strategy.build_data_id_normalization_map",
|
|
||||||
new=AsyncMock(return_value={}),
|
|
||||||
), patch.object(
|
|
||||||
strategy,
|
strategy,
|
||||||
"prepare_update_for_direction",
|
"prepare_update_for_direction",
|
||||||
new=AsyncMock(return_value=pull_node),
|
new=AsyncMock(return_value=pull_node),
|
||||||
@@ -50,9 +79,12 @@ async def test_project_detail_update_partitions_group_production_plans_before_ge
|
|||||||
"update_pair",
|
"update_pair",
|
||||||
new=AsyncMock(return_value=[generic_node]),
|
new=AsyncMock(return_value=[generic_node]),
|
||||||
) as mock_update_pair:
|
) as mock_update_pair:
|
||||||
updated_nodes = await strategy.update()
|
updated_by_id = await strategy.process_update_pairs([pull_pair, generic_pair], {})
|
||||||
|
|
||||||
assert updated_nodes == [pull_node, generic_node]
|
assert updated_by_id == {
|
||||||
|
"pull-update": pull_node,
|
||||||
|
"generic-update": generic_node,
|
||||||
|
}
|
||||||
mock_prepare.assert_awaited_once_with(
|
mock_prepare.assert_awaited_once_with(
|
||||||
pull_pair.local_node,
|
pull_pair.local_node,
|
||||||
pull_pair.remote_node,
|
pull_pair.remote_node,
|
||||||
@@ -65,4 +97,51 @@ async def test_project_detail_update_partitions_group_production_plans_before_ge
|
|||||||
def test_project_detail_domain_option_is_typed_schema() -> None:
|
def test_project_detail_domain_option_is_typed_schema() -> None:
|
||||||
strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock())
|
strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock())
|
||||||
|
|
||||||
assert strategy.domain_option.pull_group_production_plans is True
|
assert strategy.domain_option.pull_group_production_plans is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_project_detail_bind_create_pulls_group_production_plan_from_remote(project_detail_env) -> None:
|
||||||
|
local, remote, binding_manager, _ = project_detail_env
|
||||||
|
strategy = ProjectDetailSyncStrategy("project_detail_data", local, remote, binding_manager)
|
||||||
|
strategy.config.depend_fields = {}
|
||||||
|
|
||||||
|
remote_node = _build_project_detail_node(
|
||||||
|
node_id="remote-pull",
|
||||||
|
data_id="detail-remote-pull",
|
||||||
|
key="ensure_production",
|
||||||
|
)
|
||||||
|
await remote.add(remote_node)
|
||||||
|
|
||||||
|
await strategy.bind()
|
||||||
|
created_nodes = await strategy.create()
|
||||||
|
|
||||||
|
assert len(created_nodes) == 1
|
||||||
|
created_node = created_nodes[0]
|
||||||
|
created_data = created_node.get_data()
|
||||||
|
assert created_node.node_type == "project_detail_data"
|
||||||
|
assert created_data is not None
|
||||||
|
assert created_data["key"] == "ensure_production"
|
||||||
|
assert remote_node.binding_status == BindingStatus.PEER_CREATING
|
||||||
|
assert await binding_manager.get_local_id("project_detail_data", remote_node.node_id) == created_node.node_id
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_project_detail_bind_does_not_push_group_production_plan_from_local(project_detail_env) -> None:
|
||||||
|
local, remote, binding_manager, _ = project_detail_env
|
||||||
|
strategy = ProjectDetailSyncStrategy("project_detail_data", local, remote, binding_manager)
|
||||||
|
strategy.config.depend_fields = {}
|
||||||
|
|
||||||
|
local_node = _build_project_detail_node(
|
||||||
|
node_id="local-pull",
|
||||||
|
data_id="detail-local-pull",
|
||||||
|
key="ensure_production",
|
||||||
|
)
|
||||||
|
await local.add(local_node)
|
||||||
|
|
||||||
|
await strategy.bind()
|
||||||
|
created_nodes = await strategy.create()
|
||||||
|
|
||||||
|
assert created_nodes == []
|
||||||
|
assert remote.filter(node_type="project_detail_data") == []
|
||||||
|
assert await binding_manager.get_remote_id("project_detail_data", local_node.node_id) is None
|
||||||
@@ -2,34 +2,32 @@ import pytest
|
|||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
from unittest.mock import AsyncMock, patch, MagicMock
|
from unittest.mock import AsyncMock, patch, MagicMock
|
||||||
from sync_state_machine.domain.company.sync_strategy import CompanySyncStrategy
|
from sync_state_machine.domain.company.sync_strategy import CompanySyncStrategy
|
||||||
from sync_state_machine.sync_system.strategy_ops.update_ops import BoundNodePair, build_merged_update_payload, run_update
|
from sync_state_machine.sync_system.strategy_ops.update_ops import BoundNodePair, build_merged_update_payload
|
||||||
from sync_state_machine.domain.construction.sync_strategy import ConstructionTaskSyncStrategy
|
from sync_state_machine.domain.construction.sync_strategy import ConstructionTaskSyncStrategy
|
||||||
from sync_state_machine.domain.project_detail.sync_strategy import ProjectDetailSyncStrategy
|
from sync_state_machine.domain.project_detail.sync_strategy import ProjectDetailSyncStrategy
|
||||||
|
from sync_state_machine.sync_system.config import UpdateDirection
|
||||||
from schemas.project_extensions.project_detail import ProjectDetailKey
|
from schemas.project_extensions.project_detail import ProjectDetailKey
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_run_update_delegates_to_strategy_pair_planner():
|
async def test_default_strategy_update_orchestrates_pairs_in_strategy_layer():
|
||||||
local_node = SimpleNamespace(node_id="local-1")
|
local_node = SimpleNamespace(node_id="local-1")
|
||||||
remote_node = SimpleNamespace(node_id="remote-1")
|
remote_node = SimpleNamespace(node_id="remote-1")
|
||||||
|
|
||||||
strategy = MagicMock()
|
strategy = CompanySyncStrategy("company", MagicMock(), MagicMock(), MagicMock())
|
||||||
strategy.node_type = "TestNode"
|
strategy.ensure_runtime = MagicMock()
|
||||||
strategy.local_collection = MagicMock()
|
|
||||||
strategy.remote_collection = MagicMock()
|
|
||||||
strategy.binding_manager = MagicMock()
|
|
||||||
strategy.reset_schema_diff_validator = MagicMock()
|
|
||||||
strategy.emit_schema_diff_report = MagicMock()
|
|
||||||
updated_node = SimpleNamespace(node_id="target-1")
|
updated_node = SimpleNamespace(node_id="target-1")
|
||||||
strategy.update_pair = AsyncMock(return_value=[updated_node])
|
strategy.update_pair = AsyncMock(return_value=[updated_node])
|
||||||
|
|
||||||
with patch(
|
with patch(
|
||||||
'sync_state_machine.sync_system.strategy_ops.update_ops.build_data_id_normalization_map',
|
'sync_state_machine.sync_system.strategy.build_data_id_normalization_map',
|
||||||
new=AsyncMock(return_value={"project": "remote-project"}),
|
new=AsyncMock(return_value={"project": "remote-project"}),
|
||||||
), patch(
|
), patch(
|
||||||
'sync_state_machine.sync_system.strategy_ops.update_ops.collect_bound_node_pairs',
|
'sync_state_machine.sync_system.strategy.collect_bound_node_pairs',
|
||||||
new=AsyncMock(return_value=[BoundNodePair(local_node=local_node, remote_node=remote_node)]),
|
new=AsyncMock(return_value=[BoundNodePair(local_node=local_node, remote_node=remote_node)]),
|
||||||
|
), patch(
|
||||||
|
'sync_state_machine.sync_system.strategy.emit_schema_diff_report',
|
||||||
):
|
):
|
||||||
updated = await run_update(strategy)
|
updated = await strategy.update()
|
||||||
|
|
||||||
assert updated == [updated_node]
|
assert updated == [updated_node]
|
||||||
|
|
||||||
@@ -155,3 +153,42 @@ def test_project_detail_compare_payload_normalizes_enum_key_values():
|
|||||||
|
|
||||||
assert normalized["key"] == "plan_construction"
|
assert normalized["key"] == "plan_construction"
|
||||||
assert normalized["value"] == [{"date": "2023-06-28", "capacity": 4.0}]
|
assert normalized["value"] == [{"date": "2023-06-28", "capacity": 4.0}]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_project_detail_process_update_pairs_partitions_group_production_plans_before_generic_logic():
|
||||||
|
strategy = ProjectDetailSyncStrategy("project_detail_data", MagicMock(), MagicMock(), MagicMock())
|
||||||
|
|
||||||
|
pull_pair = BoundNodePair(
|
||||||
|
local_node=SimpleNamespace(node_id="local-pull", get_data=lambda: {"key": "ensure_production"}),
|
||||||
|
remote_node=SimpleNamespace(node_id="remote-pull", get_data=lambda: {"key": "ensure_production"}),
|
||||||
|
)
|
||||||
|
generic_pair = BoundNodePair(
|
||||||
|
local_node=SimpleNamespace(node_id="local-generic", get_data=lambda: {"key": "production"}),
|
||||||
|
remote_node=SimpleNamespace(node_id="remote-generic", get_data=lambda: {"key": "production"}),
|
||||||
|
)
|
||||||
|
pull_node = SimpleNamespace(node_id="pull-update")
|
||||||
|
generic_node = SimpleNamespace(node_id="generic-update")
|
||||||
|
|
||||||
|
with patch.object(
|
||||||
|
strategy,
|
||||||
|
"prepare_update_for_direction",
|
||||||
|
new=AsyncMock(return_value=pull_node),
|
||||||
|
) as mock_prepare, patch.object(
|
||||||
|
strategy,
|
||||||
|
"update_pair",
|
||||||
|
new=AsyncMock(return_value=[generic_node]),
|
||||||
|
) as mock_update_pair:
|
||||||
|
updated_by_id = await strategy.process_update_pairs([pull_pair, generic_pair], {})
|
||||||
|
|
||||||
|
assert updated_by_id == {
|
||||||
|
"pull-update": pull_node,
|
||||||
|
"generic-update": generic_node,
|
||||||
|
}
|
||||||
|
mock_prepare.assert_awaited_once_with(
|
||||||
|
pull_pair.local_node,
|
||||||
|
pull_pair.remote_node,
|
||||||
|
UpdateDirection.PULL,
|
||||||
|
{},
|
||||||
|
)
|
||||||
|
mock_update_pair.assert_awaited_once_with(generic_pair.local_node, generic_pair.remote_node, {})
|
||||||
|
|||||||
Reference in New Issue
Block a user