diff --git a/.agents/docs/2026-10-16-xlings-provider-port-and-shared-store-design.md b/.agents/docs/2026-10-16-xlings-provider-port-and-shared-store-design.md new file mode 100644 index 000000000..f37c9b7db --- /dev/null +++ b/.agents/docs/2026-10-16-xlings-provider-port-and-shared-store-design.md @@ -0,0 +1,607 @@ +--- +subject: xlings +status: active +--- + +# mcpp 对 xlings 的依赖：Provider 端口、归属与位置、上游复用、系统级安装、payload 不可变、文件系统能力与长路径（#809） + +- 日期：2026-10-16。状态：修订 3，待 review。 +- 生态联动：xlings 侧的设计 `openxlings/xlings:.agents/docs/2026-10-16-luban-inside-experience-and-ecosystem-design.md` + （下称 **xlings 设计**）：终端中继、Luban 系统内的路径与 index、系统自检与测试矩阵、aarch64、#652、#672，以及本文的 X1–X7。 + +**修订 3 的变化：** +- Luban 系统**内部**的 mcpp（core、agent-workspace 带 mcpp 与 gcc 16.1.0）：edition 带 `/etc/mcpp/config.toml`， + `upstream = ["/xlings"]`（系统自己的 home），默认工具链为系统已带的 gcc，系统里第一次构建不再下载工具链（§9.5）。 + 由 xlings 设计的 L7 检查与本文 §15 的 e2e 共同验证。 +- 依赖 xlings#652 的修复：安装失败时 mcpp 收到原因（今天为空的 `E_INTERNAL`）。 +- 发布顺序与 xlings 设计 §16 对齐：xim-pkgindex → xlings → mcpp → core 的下一版本带新 mcpp。 +- 基线：`main` `dd3ab3aa`（mcpp 2026.10.11.1 之后；内置 xlings 2026.10.10.2）。 +- 目标版本：mcpp **2026.10.16.1**。**一个 PR 完成**，不分阶段，不留临时方案。 +- 前置：一次 xlings 发布，内容为 §14 的 X1–X7（通用基础设施，不含 mcpp 专属概念）。mcpp 的 PR 把内置 xlings 钉到这次发布。 +- 证据：本地 Linux x86_64；openxlings/xlings `cf348cd`（2026.10.15.4）源码；mcpp#809 正文； + probe PR #810 两轮运行（[38780724019](https://github.com/mcpp-community/mcpp/actions/runs/38780724019)、 + [38781807313](https://github.com/mcpp-community/mcpp/actions/runs/38781807313)），见 §2。 + +**修订 2 的变化：** +- O1–O3 由 probe 实测关闭（§2）。结论：xlings 2026.10.15.4 安装的 gcc、binutils、llvm 不经 mcpp 修改即可运行； + 借来的工具链（链接或 junction）构建的产物，加载器与 rpath 是 mcpp 钉的运行库，上游一个字节都没被写。 + 原 X6（"工具链装完无需外部修改即可运行"）已由 2026.10.15.4 满足，从 xlings 需求中移除。 +- 新发现 F3：mcpp 自己往 payload 目录写"已安装"标记（`.mcpp_ok`、`.mcpp-fixup.json`）。没有它，链接条目被当作未安装、 + 重装覆盖；而只读上游写不进去。完整性改由 xlings 回答（§7.1、X4）。 +- 新发现 F4：`fallback/xpkg_copy` 在安装失败时从 `~/.xlings` 复制 payload，是重复安装的一个直接来源，删除。 +- 新增 §9 系统级安装（O5）：多用户共用的 mcpp 系统 home，管理员写、用户只读、用户侧无需 root。 +- 新增上游信任边界（§8.2）：上游必须属于当前用户或 root（Windows：当前用户、Administrators、SYSTEM）。 +- 新增借用的"每个 home 的配置"要求：借来的 payload 在借用方 home 中照样配置 subos 条目与 `abi` 链接（X3）。 +- 新增路径预算检查（§11）：Windows 未开长路径且 MCPP_HOME 过深时，在配置加载时报告；probe 实测 182 字符的 MCPP_HOME + 在开关关闭时 xlings 克隆 index 与建暂存文件失败，且报为"internal error"（X6 要求 xlings 如实报告）。 +- O4（dedupe）、O6（项目 index 放项目作用域）、D1（`mcpp.provider`）已确认。 + +## 0. 结论 + +1. **一个端口。** `mcpp.provider` 用 mcpp 的语言定义包提供者：index、包引用、payload、运行库、固定工具、事件。 + `mcpp.xlings` 降为它唯一的适配器；xlings 的命令、协议与布局只在适配器中出现，lint 保证这条边界。 +2. **归属决定位置。** mcpp 的 index（mcpplibs、全局与项目 `[indices]`，git、artifact、本地路径、私有）及其 payload 归 mcpp： + `~/.mcpp/indices/`、`~/.mcpp/xpkgs`，项目级在 `/.mcpp/`。xim 的 index 与 payload 归 xlings， + 位于适配器私有的 xlings home `~/.mcpp/registry`。 +3. **payload 不可变。** mcpp 不再写任何 payload：删除 `post_install` 的 patchelf 与 cfg 重写、`.mcpp_ok` 与 + `.mcpp-fixup.json` 标记、从 `~/.xlings` 复制 payload 的回退。安装是否完整由 xlings 回答。 +4. **复用是 store 的一条安装策略。** `[xlings] upstream` 给出只读上游（默认空，即自包含）。身份一致的 payload + 以链接条目（Windows junction）发布，整闭包一起，上游永不被写，失效时原路径自愈。通用层看不到这一点。 +5. **系统级安装。** 管理员以 `mcpp --system` 维护一个系统 home（Linux 默认 `/var/lib/mcpp`），用户各自的 `~/.mcpp` + 把它作为上游；用户侧所有操作无需 root。Luban OS 以系统级配置开启。 +6. **xlings 的来源由握手裁决。** `bundled`、`system`、路径三种来源都检查版本下限、协议、能力与 home 格式，不满足退回内置。 +7. **#809。** 内置 xlings 升级；mcpp 自己 `scaffold_fs` 的同类缺陷（probe 实测 `mcpp new` 在 NFSv3 上失败）以文件系统能力表修复。 +8. **长路径。** mcpp.exe 与 xlings.exe 声明 `longPathAware`；新目标键 `windows_long_paths`；配置加载时做路径预算检查； + doctor 报告系统开关。 +9. **index 刷新改为暂存再交换。** 删除 `index_snapshot` 的备份回滚。 + +## 1. 问题与证据 + +### 1.1 #809 与 mcpp 自己的同类缺陷 + +| 编号 | 观察 | 原因 | 状态 | +|---|---|---|---| +| F1 | NFSv3 上 `mcpp toolchain install gcc@16.1.0`：`cannot publish …/usr/include/asm: Invalid argument`（probe 复现） | xlings 用 `renameat2(RENAME_NOREPLACE)` 发布，NFSv3 返回 `EINVAL` | xlings 2026.10.15.4 修复（openxlings/xlings#671）；probe 中 mcpp 2026.10.11.1 改用它后，NFS 上安装、构建、运行都通过 | +| F2 | NFSv3 上 `mcpp new demo`：`atomic no-replace rename '…/.mcpp-new-…' -> '…/demo' failed: Invalid argument`（probe 实测） | `modules/platform/src/scaffold_fs.cppm:72` 直接 `renameat2(RENAME_NOREPLACE)`，无回退 | 本 PR 修复（§10） | + +### 1.2 两套重复安装 + +本机：`~/.xlings/data/xpkgs` 60G，`~/.mcpp/registry/data/xpkgs` 14G，其中 `-x-/` 相同的 **10.1G** +（gcc 638M、android-ndk 2.3G、emsdk 1.5G、libllvm 124M、glibc 122M 等）。mcpp store 的命名空间：xim 85、compat 46、 +freedesktop 8、mcpplibs 11 等。 + +### 1.3 布局知识散落在适配器之外 + +| 知识 | 位置 | +|---|---| +| store 路径与 `-x-/` 命名 | `src/toolchain/lifecycle.cppm:108,455,484,1136`；`src/doctor.cppm:243,270,291,529,539,657,1648`；`src/runtime/binding.cppm:253,289`；`src/build/prepare/toolchain.cpp:298-303,420`；`src/build/prepare/toolchain_decision.cpp:137`；`src/pm/package_fetcher.cppm:883,1057,1306`；`src/toolchain/probe.cppm:195,414`；`src/toolchain/msvc.cppm:1174,1448`；`src/cli/cmd_self.cppm:71`；`src/toolchain/post_install.cppm:772`；`modules/platform/src/env.cppm:237` | +| 从编译器路径向上找 store，并 canonical 化 | `src/build/hermetic.cppm:127,138`；`xpkgs_from_compiler` 的调用方 `src/toolchain/probe.cppm:420`、`src/toolchain/msvc.cppm:1171,1365`、`src/build/execute.cppm:3525`、`src/fallback/probe_sysroot.cppm:56,84` | +| 读用户自己的 xlings home | `src/fallback/xpkg_copy.cppm:40-41`（F4）；`active_home_xpkgs`/`find_home_tool` 的调用方 `src/toolchain/probe.cppm:424,489`、`src/toolchain/post_install.cppm:343` | +| `.xlings.json` 与 subos 目录 | `src/config.cppm:198,236,869`；`src/build/runtime_validation.cppm:344`；`src/build/execute.cppm:1836`；`src/build/prepare/graph_load.cpp:1077` | +| index 内部结构 | `src/xlings/xlings.cppm:723`（读 `pkgs/<首字母>/.lua`）；`src/config.cppm:198`（探测 `xim-index-repos/`） | + +38 个文件 `import mcpp.xlings`。进程调用已经集中在 `src/xlings/xlings.cppm`（12 处 `build_command_prefix`）。 +xlings ≥ 2026.9.27.1 的 `install_targets` 事件对每个请求返回 `payload_dir` 与 `revision`（`src/xlings/xlings.cppm:186-235`）， +下限 2026.10.10.2 高于它，但大多数调用方仍走路径语法。 + +### 1.4 mcpp 写 payload + +| 编号 | 写入 | 位置 | +|---|---|---| +| W1 | gcc、binutils 全部 ELF 的 PT_INTERP 与 RUNPATH | `gcc_post_install_fixup`，`src/toolchain/post_install.cppm:453-486`（理由见 `:68-72` 的 TODO：xim 的 `elfpatch.auto` 曾扫到 0 个文件） | +| W2 | llvm `lib/` 的 RUNPATH | `llvm_post_install_fixup`，`:493-518` | +| W3 | 重写 payload `bin/*.cfg` | `fixup_clang_cfg`，`:276-379`（只为人直接运行 `clang++`；mcpp 构建用 `--no-default-config`） | +| W4（F3） | `.mcpp-fixup.json` | `:648` | +| W5（F3） | `.mcpp_ok` | `src/pm/package_fetcher.cppm:1112-1211`、`src/config.cppm:945,957`、`src/build/prepare/graph_load.cpp:729,807`；判据 `src/fallback/install_integrity.cppm` | +| W6（F4） | 从 `~/.xlings` 复制整个 payload 进 registry | `src/fallback/xpkg_copy.cppm`，调用方 `src/pm/package_fetcher.cppm:1209` | + +同一个 `xim:gcc@16.1.0`，本机 `~/.xlings`（xlings 安装）的 `bin/gcc` 绑定 `xim-x-glibc/2.39`，`~/.mcpp/registry`（mcpp 修改后） +绑定 `xim-x-glibc/2.44.3`：被修改的 payload 不可能被复用。#273 的路径限制是因为修改 payload 才需要的。 + +### 1.5 index 的位置不由 mcpp 决定 + +`src/pm/index_snapshot.cppm` 头注释：`index_repos` 条目"没有版本，也没有目的地"，`xlings update` 原地改写，mcpp 只能备份、失败再回滚。 + +### 1.6 现有 `[xlings]` 配置是两个没有检查的半模式 + +`src/config.cppm:377-381`、`:765-767`、`:879-895`：`binary = "system"` 不检查版本；`home = ""` 不检查归属， +指向 `~/.xlings` 时 mcpp 会改写用户 home 的 index、mirror 与运行库钉版本。 + +### 1.7 xlings 2026.10.10.2 → 2026.10.15.4 中与 mcpp 相关的变化 + +| 变化 | 来源 | 对 mcpp | +|---|---|---| +| 文件系统能力探测、`publish_no_replace` | #671 | 修复 F1 | +| 每个环境一个默认运行库：payload 经 `/abi/` 绑定 libc 与 C++ 运行库，`.xlings.json` `runtime` 钉版本 | #667 C | W1、W2 不再需要（§2.1） | +| 接口协议 1.8 → 1.9 | #659、#667 | 适配器握手的受支持范围 | +| home 格式 layout 2 | 2026.10.4 起 | 握手要检查（§12） | +| system layer 与 borrow | #667 | §8 的基础；现状只支持模式为 `multi` 的 `/xlings`，借的是登记而不是本地条目，来源删除即报错 | + +### 1.8 长路径的现状 + +| 平台 | 现状 | +|---|---| +| Windows | 引擎经手的路径走 `extended_length`（`modules/platform/src/fs.cppm:57-75`）；对象路径已用 `__pkg`/`__ext/`（`src/build/plan.cppm:1972` 起）；mcpp.exe 只声明 `activeCodePage`；**mcpp 与 xlings 都没有 `longPathAware`**；xlings 的 `GetModuleFileNameW` 用 `MAX_PATH` 缓冲；工作目录在任何写法下受 MAX_PATH 限制 | +| Linux | socket 路径 108 字节（xlings 有检查）；5.1 以前内核 shebang 127 字节 | + +## 2. probe 实测（#810） + +所有脚本只打印不断言。xlings 2026.10.15.4 建"上游 home"；mcpp 2026.10.11.1（发布版）建 MCPP_HOME； +"借用"以把 mcpp registry 中的条目替换为指向上游 payload 的符号链接（Windows junction）模拟。 + +### 2.1 O1：xlings 2026.10.15.4 安装的工具链不经 mcpp 修改即可运行（关闭） + +| 平台 | 观察 | +|---|---| +| Linux x86_64、aarch64 | gcc、g++、cc1plus、binutils `ld`、clang++ 的 PT_INTERP 都是 `/abi/linux--glibc/ld-linux-*.so.*`，RUNPATH 含 `/abi/linux--glibc`；`abi.json` 记录 `glibc@2.44.3`、`gcc-runtime@15.1.0`。llvm 的 `libc++.so.1`、`libunwind.so.1` 的 RUNPATH 同样经 `abi` | +| Linux | clang++ 直接编译运行 hello 成功；产物绑定 `xim-x-glibc/2.44.3`（来自 xlings 生成的 cfg）。g++ 不带 sysroot 直接编译失败（`features.h` 找不到）：头文件经 subos 的 sysroot 提供，与 mcpp 无关（mcpp 总是传 `--sysroot`） | +| macOS arm64 | clang++ 只依赖 `/usr/lib` 的系统库，rpath 为 `@loader_path/../lib`；cfg 只有 `--sysroot=` 与 `-fuse-ld=lld`；直接编译运行成功 | + +xlings 生成的 gcc specs 与 clang cfg 中，产物的加载器与 rpath 写的是具体 glibc payload 路径；mcpp 构建不用它们 +（`write_clean_link_specs`、`--no-default-config`），不受影响。 + +### 2.2 O2：借来的工具链构建 mcpp 的产物（关闭） + +上游 store 先 `chown -R root`、`chmod -R a-w`（同时作为 O5 的只读系统层）。第 1 轮不预置标记，第 2 轮在上游 payload 中预置 +`.mcpp_ok`（模拟"端口回答已安装"）。 + +| 平台与工具链 | 第 1 轮（无标记） | 第 2 轮（有标记） | +|---|---|---| +| Linux x86_64 gcc（变体 A 只借 gcc，变体 B 再借 binutils） | mcpp 判为未安装，xlings 重装，把链接换成实体目录，再执行 patchelf | 链接保留；`skip gcc fixup: … resolves outside this home`；构建运行成功 | +| Linux x86_64、aarch64 llvm | 同上 | 链接保留；构建运行成功 | +| macOS llvm | 同上 | 链接保留；构建运行成功；`clang++ -print-resource-dir` 经链接解析到上游真实路径 | +| Windows llvm（junction） | 同上；上游 369 个文件、字节数不变（重装没有穿过 junction 删除上游） | junction 保留；构建运行成功；资源目录解析到上游真实路径；上游不变 | + +所有借用构建的产物：PT_INTERP 为 `/registry/data/xpkgs/xim-x-glibc/2.44.3/lib64/ld-linux-*.so.*`， +RUNPATH 为本地 store 路径（mcpp 钉的运行库）；上游目录树摘要前后一致。aarch64 上 mcpp 的 `gcc@16.1.0` 解析为 +`xim:musl-gcc`，与上游所装不是同一个包，gcc 变体不适用。 + +结论： +1. W1–W3 不再需要。工具作为工具运行在它所在 home 的运行库上，产物由 mcpp 每次构建给出的参数决定。 +2. W4、W5 是借用的障碍：完整性必须由 xlings 回答（X4），mcpp 不写标记。 +3. 链接条目被替换时 xlings 只删除链接本身（四个平台都如此）；X3 把"删除条目永不穿过链接"写成要求，并由测试保证。 + +### 2.3 O3：Windows（关闭） + +| 项 | 观察 | +|---|---| +| junction | 见 §2.2；`msvc@system` 与借来的 llvm 组合正常 | +| runner 的 `LongPathsEnabled` | 1 | +| 开关为 1，MCPP_HOME 157 字符 | 安装、构建、运行成功；最长路径 267 | +| 开关为 0，MCPP_HOME 182 字符 | `toolchain install llvm` 失败：xlings 克隆 index（`all index repo clone URLs failed`）与 `Failed to create staging file`，报为 "internal error … likely a bug" | + +结论：深路径在开关关闭时首先在 xlings 失败；mcpp 需要在配置加载时做路径预算检查（§11），xlings 需要如实报告路径过长（X6）。 + +### 2.4 F1、F2（NFSv3） + +见 §1.1。`renameat2(RENAME_NOREPLACE)` 在该挂载上返回 `EINVAL`。 + +## 3. 原则 + +| 编号 | 原则 | +|---|---| +| P1 | **依赖方向。** Luban OS → mcpp → xlings。xlings 只提供通用能力；是否复用、借哪些、放在哪里，是 mcpp 的配置与策略。 | +| P2 | **依赖能力，不依赖布局。** 通用层不拼 xlings 的路径、不读 xlings 的文件；payload 在哪里、是否完整由端口回答。 | +| P3 | **归属决定位置。** mcpp 的 index 与 payload 由 mcpp 决定放在 `~/.mcpp` 何处；xim 的东西属于 xlings。 | +| P4 | **payload 不可变。** payload 能否运行是它的主人（xlings 与 xim-pkgindex）的责任，发布前完成；mcpp 不写 payload，对工具链的定制只以每次构建的参数和 mcpp 自己目录中的生成文件表达。 | +| P5 | **store 路径不 canonical 化。** 本地 store 中的路径是 payload 的稳定身份，链接背后是谁与使用者无关。 | +| P6 | **上游只读。** mcpp 驱动的 xlings 只写 mcpp 的 home，从不写上游；删除条目永不穿过链接。 | +| P7 | **用户侧无需 root。** 任何用户操作都不需要提升权限；系统 home 只由管理员显式写入。 | + +## 4. Provider 端口 + +### 4.1 分层 + +``` +mcpp 通用层：build / toolchain / pm / runtime / doctor / pack / cli + │ import mcpp.provider （只用 mcpp 的概念） + ▼ +mcpp.provider 端口：类型与接口 +mcpp.provider.fake 假实现：内存 index、临时目录 store（单元测试） +mcpp.provider.xlings 适配器（今天的 src/xlings/ 迁入 src/provider/xlings/）：命令、NDJSON、握手、布局 + │ +xlings 运行时：bundled / system / 路径，经握手选出（§12） +``` + +### 4.2 接口 + +```cpp +export module mcpp.provider; +export namespace mcpp::provider { + +struct ScopeRef { enum class Kind { Global, Project } kind; std::filesystem::path root; }; + +struct IndexDecl { + std::string name; // 命名空间 + IndexSource source; // git url | artifact bases | 本地路径 + std::filesystem::path dir; // index 数据目录（本地路径 index 即其本身） + std::filesystem::path store; // 这个 index 的 payload 放在哪里 +}; + +struct PackageRef { std::string ns, name, versionReq; }; + +struct Payload { + PackageRef resolved; std::string version; int revision; + std::filesystem::path dir; // 本地 store 中的路径，不 canonical 化（P5） + enum class Origin { Local, Borrowed } origin; +}; + +struct Runtime { std::string family, id; std::filesystem::path loader, libDir, sysroot; }; + +struct ProviderInfo { std::string version, protocol, source; std::vector capabilities; int homeLayout; }; + +class Provider { +public: + ProviderInfo info() const; + std::expected sync_index(const IndexDecl&, const std::filesystem::path& staging); + std::expected plan(ScopeRef, std::span); + std::expected, ProviderError> install(ScopeRef, const Plan&, EventSink&); + std::optional find(ScopeRef, const PackageRef&) const; // 只查询；只返回完整的 payload + std::expected remove(ScopeRef, const PackageRef&); + std::expected runtime(ScopeRef) const; + std::expected tool(PinnedTool); // ninja / patchelf / nasm +}; + +Provider& provider(); // 进程内一个，由配置与握手构造 +} +``` + +- "编译器旁边的 binutils"、"linux-headers 在哪"改为 `find(scope, xim:binutils@…)`；`xpkgs_from_compiler`、 + `find_sibling_*`、`active_home_xpkgs`、路径语法回退全部删除。 +- 删除 payload 只经 `remove`（适配器交给 xlings），mcpp 不对 store 条目调用 `remove_all`。 +- 事件类型是端口的类型；NDJSON 解析在适配器内。 + +### 4.3 边界由 lint 保证 + +`.github/tools/check_provider_boundary.py`（接入 CI）：`src/provider/xlings/**` 之外， +不得 `import mcpp.xlings*`；不得出现 `"xpkgs"`、`"-x-"`、`xim-x-`、`".xlings.json"`、`"subos"`、`"xim-index-repos"`、 +`.mcpp_ok`；不得对 `Payload::dir` 调用 `canonical`/`weakly_canonical`/`remove_all`。 + +### 4.4 契约测试 + +`tests/provider/`：适配器对内置钉版本与最新发布 xlings 各跑一遍（CI 已用最新发布版构建，接在同一处）。 + +## 5. 归属与位置 + +### 5.1 布局 + +``` +~/.mcpp/ + config.toml + indices// mcpp 的 index 数据：mcpplibs、全局 [indices] 的 git/artifact index + xpkgs/ mcpp 的 store：mcpp index 所服务命名空间的 payload + registry/ 适配器私有：mcpp 的 xlings home（xim index、subos、abi、xim store） + data/xpkgs/ xim 的 payload；复用时其中可有链接条目 + build-cache/ cache/ provisioned/ … 不变 + +/.mcpp/ + indices// 项目 [indices] 的 git index + xpkgs/ 项目 index 所服务命名空间的 payload（项目作用域，D4） +``` + +- 本地路径 index 原地使用。私有 index 与公开 index 没有区别，只是来源不同；凭据由 git 与 xlings 的传输层按现有方式处理。 +- 一个 index 服务哪些命名空间由 index 自己声明（X2）。 + +### 5.2 配置 + +```toml +[indices.myteam] +url = "https://git.example.com/team/index.git" +# dir / store 省略时取 ~/.mcpp/indices/myteam 与 ~/.mcpp/xpkgs +``` + +`mcpplibs` 是内置默认，可不写。项目 `mcpp.toml` 的 `[indices]` 不接受 `dir`、`store`。 + +### 5.3 存量 home + +| 存量 | 处理 | +|---|---| +| `registry/data/mcpplibs` 等 mcpp index 数据 | 首次运行移动到 `~/.mcpp/indices/`（同文件系统 rename，跨文件系统复制后删除） | +| `registry/data/xpkgs` 下 mcpp 命名空间的 payload | 不移动（已有产物与 compat 共享库的 RUNPATH 指向它们）。适配器 `find` 先查新 store 再查旧位置，`COMPAT(internal:legacy-registry-store)`；新安装进新 store；`mcpp cache clean --legacy` 列出并可删除新 store 已有的旧条目 | +| 已被 mcpp 修改过的 xim payload | 保持原状，仍可运行，不再被修改 | +| payload 中的 `.mcpp_ok`、`.mcpp-fixup.json` | 不再读也不再写；`mcpp cache clean --legacy` 可清除 | + +## 6. index 刷新：暂存再交换 + +1. `sync_index(decl, staging)`：xlings 把 index 拉到 `.staging-`（X1）。 +2. mcpp 在暂存目录上检查 `index.toml` 契约（`min_mcpp`）。 +3. 通过则用 `exchange_paths`/`rename_no_replace`（§10）原子交换到 `dir`；不通过则删除暂存目录，旧 index 保持可用。 + +`index_snapshot.cppm` 的备份与回滚删除；离线优先策略（`index_refresh.cppm`）不变。 + +## 7. payload 不可变 + +### 7.1 W1–W6 的去向 + +| 写入 | 处理 | +|---|---| +| W1、W2 | 删除。工具链经 `/abi/` 绑定运行库（§2.1） | +| W3 | 删除。llvm.lua 的安装 hook 生成 cfg；mcpp 构建用 `--no-default-config` | +| W4 | 删除，随 `ensure_post_install_fixup` 与 `kFixupRev` 一起 | +| W5 | 删除写入。完整性由端口 `find`/`install` 的结果回答（X4：xlings 报告 payload 完整与否，含 #533 那类"报告成功但未安装"的检测）。mcpp 保留只读校验 `payload_is_substantive`，不满足时报错而不是写标记 | +| W6 | 删除 `fallback/xpkg_copy`。从 `~/.xlings` 获益的正确方式是上游（§8） | + +随之删除：`patchelf_walk`、`gcc_post_install_fixup`、`llvm_post_install_fixup`、`fixup_clang_cfg`、`ensure_post_install_fixup`、 +`escapes_containment`/`containment_root`（#273 的限制失去对象）、`InstallStash` 中针对 payload 目录的改名（由 xlings 的原子发布替代）。 +`patchelf` 仍是固定工具：`mcpp pack` 与 `loader_contract` 只对 **mcpp 自己的产物** 使用它。 + +### 7.2 工具链运行在哪个运行库上 + +适配器在 bootstrap 时把 mcpp 选定的 `RuntimeBinding`（如 `glibc@2.44.3`）写为 registry home 的运行库钉版本 +（xlings `config --runtime`，经接口调用）。自包含模式下工具链因此运行在 mcpp 选定的 glibc 上；借来的工具链运行在上游 home 的 +运行库上（§2.2 证明这对产物没有影响）。 + +### 7.3 产物不变 + +产物的加载器、rpath、specs、sysroot 由 `linkmodel` 与 `write_clean_link_specs` 每次构建给出，引用本地 store 路径（P5）。 + +## 8. 上游复用 + +### 8.1 配置与优先级 + +```toml +[xlings] +upstream = [] # 默认：自包含 +# upstream = ["~/.xlings"] # 普通机器 +``` + +读取顺序：用户 `~/.mcpp/config.toml` → mcpp 系统级配置（Linux/macOS `/etc/mcpp/config.toml`，Windows +`%ProgramData%\mcpp\config.toml`）→ 空。条目可以是一个 xlings home（只提供 xim），也可以是一个 mcpp home +（§9 的系统 home：其 registry 提供 xim，其 store 提供 mcpp 的命名空间），由目录中的标记判别。 + +### 8.2 信任边界 + +上游必须属于当前用户或 root（Windows：当前用户、Administrators、SYSTEM），且对当前用户不可写或属于当前用户本人。 +不满足时在配置加载时报错：借用另一个普通用户的 home 等于执行对方可以随时替换的程序。 + +### 8.3 语义（X3，mcpp 只给声明） + +适配器把 `upstream` 写为 registry home 的上游声明：xlings home 限定命名空间 `["xim"]`；mcpp 系统 home 的 registry 限定 `["xim"]`， +其 store 限定为 mcpp index 所服务的命名空间。安装一个 payload 时： + +| 情况 | 结果 | +|---|---| +| 本地 store 已有完整的 | 用本地 | +| 某个上游有同一身份（命名空间、名字、版本、revision）且完整，运行库闭包也可借 | 在本地 store 发布链接条目（Linux/macOS 符号链接，Windows junction），整闭包一起；并在借用方 home 中照常配置（subos 条目、`abi` 链接） | +| 否则 | 下载、安装到本地 | + +- 上游永不被写；删除或替换本地条目只删除链接本身（§2.2 已观察，X3 写成要求）。 +- 上游的文件系统不支持符号链接时不借。 +- mcpp 的 index 只从 mcpp 的 home 借（自己的系统 home），永不从 xlings home 借，即使用户的 `~/.xlings` 也注册了 mcpplibs。 + +### 8.4 失效与自愈 + +上游删除了被借的 payload 后，本地条目悬空；`find`/`install` 视为未安装，xlings 在同一路径上安装实体，已构建产物引用的路径不变。 +离线时（`--offline`）无法自愈，报错并说明哪个上游条目失效。 + +### 8.5 存量重复（O4，已确认） + +`mcpp self store dedupe [--apply]`：列出本地 xim payload 中与上游身份一致的条目，`--apply` 时以链接条目逐条原子替换 +（`exchange_paths`）。默认只列出。 + +## 9. 系统级安装（O5） + +### 9.1 部署形态 + +| 形态 | mcpp 入口 | 用户的 MCPP_HOME | 系统 home | 写入者 | +|---|---|---|---|---| +| 自包含（默认） | 任意位置 | `~/.mcpp` 或 `MCPP_HOME` | — | 用户 | +| 用户 + xlings 上游 | 同上 | `~/.mcpp` | — | 用户；上游为 `~/.xlings` | +| 系统 | 包管理器或 `sudo xlings install --system mcpp` | 每个用户的 `~/.mcpp` | `/var/lib/mcpp`（macOS `/Library/Application Support/mcpp`，Windows `%ProgramData%\mcpp`；`/etc/mcpp/config.toml` 的 `[system] home` 可改） | 管理员 `mcpp --system …`；用户只读 | +| Luban OS | `/xlings` 中的 `xim:mcpp` | 每个用户的 `~/.mcpp` | 同上；系统 home 自己的上游为 `/xlings` | 同上 | + +系统 home 的布局与用户 home 相同（§5.1），另有标记 `.mcpp-home`（`mode = "system"`）。它不含 `build-cache`。 + +### 9.2 写：管理员 + +- `--system` 是作用域标志，只用于维护命令：`self init`、`toolchain install|remove|default`、`index update`、 + `self store gc|dedupe`、`self doctor`。`build`、`run`、`test`、`new`、`add` 等带 `--system` 时拒绝： + 以 root 构建会把 root 所有的文件写进用户的项目。 +- 判断依据是对系统 home 的写权限，不是 uid：管理员也可以把系统 home 交给一个组维护。没有写权限时报错并提示 `sudo`。 +- 写入时 umask 为 `022`，保证所有用户可读；写入持有系统 home 的排他锁。 +- 在 `sudo` 下忽略调用者的 `HOME` 与 `MCPP_HOME`，系统 home 只取自系统级配置。 +- 系统级配置可以给出默认工具链（`[toolchain] default = "gcc@16.1.0"`），用户配置可覆盖。 + +### 9.3 读：用户（无需 root） + +- 系统级配置给出 `[xlings] upstream = ["/var/lib/mcpp", "/xlings"]`（Luban）或 `["/var/lib/mcpp"]`。 +- 用户有自己的 index 数据（很小，刷新无需 root）；系统 home 中的 payload 只在身份一致时被借。用户的 index 解析出更新的版本或 + revision 时装进自己的 `~/.mcpp`。 +- `build-cache`（含 std 模块 BMI）每个用户一份，不跨用户共享：跨用户的构建缓存是投毒面。 +- `mcpp self update` 在系统形态下拒绝，提示由包管理器更新。 + +### 9.4 Luban OS + +Luban 的 edition 提供 `/etc/mcpp/config.toml`（X7），并在系统镜像制作时以 `mcpp --system toolchain install` 预置默认工具链。 +系统 home 的 registry 以 `/xlings` 为上游，所以 xim 的 payload 在机器上仍只有一份。 + +### 9.5 Luban 系统内部的 mcpp + +上面是 Luban **宿主**上的多用户形态。一个 Luban 系统（`luban new`）**内部**也带 mcpp（core、agent-workspace）： + +- 系统的 `/xlings` 是它自己的 home 与 store，进入后在 `/` 下可见（xlings 设计 §5.1）；在系统内 root 所有，满足 §8.2。 +- edition 的 factory `/etc` 带 `mcpp/config.toml`：`[xlings] upstream = ["/xlings"]`，默认工具链写明目标： + `gcc@16.1.0` 加 `-linux-gnu`（edition 已带的 glibc 版 `xim:gcc`）。probe 实测 mcpp 在 aarch64 上把裸写的 + `gcc@16.1.0` 解析为 `xim:musl-gcc`（§2.2），不写目标会与系统已带的 gcc 身份不一致而再次下载。系统里的 `~/.mcpp` 借用系统的 gcc、glibc，第一次 `mcpp build` 不下载工具链。 +- 系统里的普通用户（desktop 默认以普通用户登录）：xlings 设计修订 2 §8 给普通用户一个叠加在 `/xlings` 之上的用户层 home； + mcpp 的上游仍是 `/xlings`（root 所有，满足 §8.2），用户的 `~/.mcpp` 只写自己的目录，无需 root。 +- 系统换代（升级 gcc）后，被借条目随上游的身份变化：旧身份的链接悬空即按 §8.4 自愈，新版本按 mcpp 的工具链选择决定。 + +## 10. 文件系统能力（#809 同类） + +`modules/platform` 增加与 xlings #671 同构的能力表： + +- `platform::fs_caps(dir)`：每个文件系统每进程探测一次（no-replace rename、exchange、硬链接、符号链接、大小写、类型名），按设备缓存。 +- `platform::rename_no_replace`、`exchange_paths` 按能力选择策略：原生 → 普通文件以硬链接发布 → 目录 rename 到不存在的名字。 +- `scaffold_fs.cppm` 改用它（修复 F2）；§6 的交换、§8.5 的替换也用它。 +- `doctor` 在 MCPP_HOME 的文件系统不完全原生时说明 mcpp 在那里如何写入。 + +## 11. 长路径 + +| 项 | 处理 | +|---|---| +| mcpp.exe | manifest 增加 `longPathAware`（与 `activeCodePage` 同一份） | +| 目标键 | `windows_long_paths = true \| false`，与 `windows_code_page` 同构：宿主工具默认 `true`；用户程序默认不写（程序能否处理长路径由作者声明）；库目标拒绝 | +| xlings.exe | X6 | +| 引擎路径 | 继续经 `extended_length`；store、index、交换操作同样经过它 | +| 路径预算 | 配置加载时：Windows 上 `LongPathsEnabled = 0`，且 MCPP_HOME 长度加上 registry 内已知最深后缀（以契约测试中测得的值为常量）超过 259 时，degraded：impact"安装可能因路径过长失败"，并给出开启开关或缩短 MCPP_HOME 的办法 | +| 系统开关 | `doctor` 报告 `LongPathsEnabled`；mcpp 不替用户修改注册表 | + +## 12. xlings 的来源与握手 + +`[xlings] binary = "bundled" | "system" | <路径>`（已有键）。握手（X5）读取版本、协议、能力清单与支持的 home layout： + +| 检查 | 不满足时 | +|---|---| +| 版本 ≥ `kXlingsVersion` | 退回内置，note | +| 协议在适配器支持的范围内 | 同上 | +| 能力包含 `install_packages` 与 X1–X5 | 同上 | +| 能写 registry home 当前的 layout | 同上；内置也不能写时报错，指出 home 由更新的 xlings 写过 | + +`[xlings] home` 仍是 registry 的位置；指向不是 mcpp 建立的 xlings home 时在配置加载时拒绝，提示改用 `upstream`。 + +## 13. 诊断分级 + +按 `src/diag.cppm`：做得比要求少，必须报错或 `degraded`（带 impact）。 + +| 情形 | 等级 | +|---|---| +| `[xlings] home` 指向非 mcpp 的 home | 配置加载时报错 | +| 上游不满足信任边界（§8.2） | 配置加载时报错 | +| `--system` 用于构建类命令 | 报错 | +| `--system` 而无写权限 | 报错，提示 `sudo` | +| 选定的 xlings 不满足握手，内置可用 | note | +| 内置 xlings 也不能写 registry 的 layout | 报错 | +| 上游不存在或不可读 | degraded：impact"这次不复用"；`--strict` 下为错误 | +| 上游 payload 身份不一致 | verbose（正常结果） | +| 链接条目悬空、原路径重装 | note；离线时报错 | +| 上游文件系统不支持符号链接 | degraded：impact"不复用" | +| Windows 路径预算超出且开关关闭 | degraded（§11） | +| MCPP_HOME 的文件系统不完全原生；`LongPathsEnabled` 未开 | doctor 信息行 | + +## 14. xlings 侧需求（前置发布） + +| 编号 | 能力 | 说明 | +|---|---|---| +| X1 | index 数据目录由调用方指定；拉取到指定目录 | `index_repos` 条目增加 `dir`；接口能力 `sync_index` 带 `dest` | +| X2 | 按 index 指定 store | `index_repos` 条目增加 `store`；index 声明自己服务的命名空间 | +| X3 | home 的只读上游 | `.xlings.json` `upstreams: [{home, namespaces}]`；来源可为任意模式的 home；身份（含 revision）与完整性校验；闭包；本地链接条目（Windows junction）；借用方 home 照常配置 subos 条目与 `abi` 链接；删除或替换条目只删链接；悬空自愈；从不写上游。推广现有 system layer 的 borrow | +| X4 | 完整性与查询 | 接口能力返回 `payload_dir`、`revision`、`origin`、`complete`，不触发安装；`install_targets` 同样给出 `complete`；"报告成功但未安装"（mcpp#533 一类）由 xlings 判为不完整 | +| X5 | 握手 | 接口描述返回版本、协议、能力清单、支持的 home layout | +| X6 | Windows 长路径 | xlings.exe `longPathAware`；`GetModuleFileNameW` 可增长缓冲；路径过长报为路径过长（今天报 "internal error … likely a bug"） | +| X7 | Luban OS | 宿主：系统级配置 `/etc/mcpp/config.toml` 与镜像制作时预置 mcpp 系统 home（§9.4）；系统内部：edition 的 factory `/etc/mcpp/config.toml` 以 `/xlings` 为上游（§9.5）。由 xim-pkgindex 的 edition 新版本提供 | + +另依赖 xlings#652：安装失败的原因进入 `InstallStatus.message`，适配器把它交给 mcpp 的诊断。 + +发布顺序（与 xlings 设计 §16 一致）：xim-pkgindex PR（edition、terminfo 等）→ xlings PR（含 X1–X6 与 #652）→ xlings 发布 → +mcpp PR 钉该版本 → mcpp 2026.10.16.1 → xim-pkgindex bump mcpp，core 的下一版本带它。 + +## 15. 测试与 CI + +| 层 | 内容 | +|---|---| +| 单元 | `mcpp.provider.fake` 驱动 toolchain、pm、runtime 的测试；`fs_caps` 各策略以声明的能力驱动；配置优先级；握手判定；信任边界判定；路径预算 | +| 契约 | `tests/provider/`：内置钉版本与最新发布 xlings 各一遍；测得 registry 内最深后缀，作为路径预算常量的依据 | +| e2e 自包含 | 全新 MCPP_HOME 安装 gcc、llvm：payload 中没有任何 mcpp 写入的文件；工具链二进制的 PT_INTERP 经 registry 的 `abi`；构建运行 | +| e2e 复用（Linux、macOS、Windows） | 上游 home（root 所有、只读）装好工具链；`upstream=[它]`：断言未下载、本地为链接条目、构建运行、产物加载器在本地 store；删除上游 payload 后原路径自愈；上游摘要前后一致 | +| e2e 系统级（Linux） | `sudo mcpp --system self init` 与 `toolchain install`；两个普通用户（`useradd`）各自构建运行，系统 home 摘要不变；`mcpp --system build` 被拒绝；无写权限的 `--system` 被拒绝 | +| e2e NFS | NFSv3 上的 MCPP_HOME 与项目目录：`toolchain install gcc`、`mcpp new`、构建运行（§2.4 的复现转为回归） | +| e2e Windows 深路径 | 开关为 0 时路径预算的 degraded 出现；开关为 1 时 157 字符 MCPP_HOME 构建成功；mcpp.exe manifest 含 `longPathAware` | +| e2e 迁移 | 2026.10.11.1 建立的 home：index 移动、旧 store 兼容读、已构建产物可运行、旧标记被忽略 | +| lint | `check_provider_boundary.py` | + +新增 e2e 在上一个发布版上必须失败（复用、系统级、NFS 的 `mcpp new`、manifest）。合并前 dispatch mcpp-plugins CI。 + +## 16. 文档与规范 + +- 新文档 `docs/25-home-indices-and-store.md`（与 `docs/zh/`）：`~/.mcpp` 布局、归属、复用、系统级安装、信任边界、文件系统要求、Windows 长路径。 +- `docs/04-mcpp-toml.md`：`windows_long_paths`；`[indices]` 位置规则。 +- config.toml 文档节：`[xlings] upstream`、握手、`[indices.] dir/store`、`[system] home`。 +- `docs/09-commands-by-scenario.md`：`--system`、`self store dedupe`。 +- `docs/specs/toolchain-management.md`：payload 不可变；工具链运行库由 registry home 钉版本；修订号与变更记录。 +- `docs/specs/package-identity.md`：payload 身份含 revision；store 位置由 index 决定；修订号与变更记录。 +- `docs/91-toolchain-internals.md`：删除 post-install 修改的描述。 +- CHANGELOG 2026.10.16.1。 + +## 17. 提交顺序（一个 PR 内） + +1. `platform`：`fs_caps`、`rename_no_replace`、`exchange_paths`；`scaffold_fs` 改用它们。 +2. 内置 xlings 钉到 X 发布；`kXlingsVersion` 说明增加第八条；`.github` 钉版本随之更新。 +3. `mcpp.provider` 端口、`fake`、适配器迁入 `src/provider/xlings/`；握手；lint。 +4. 按领域迁移调用方并删除布局知识：toolchain → build/prepare 与 hermetic → runtime → pm → doctor 与 cli → fallback。 +5. index 位置与暂存交换；存量迁移与兼容读。 +6. payload 不可变：删除 W1–W6；registry 运行库钉版本。 +7. 上游复用：配置、信任边界、`self store dedupe`、doctor。 +8. 系统级安装：`--system`、系统级配置、`.mcpp-home` 标记。 +9. 长路径：`windows_long_paths`、mcpp.exe manifest、路径预算、doctor。 +10. 测试与 CI lane。 +11. 文档、规范、CHANGELOG、版本号 2026.10.16.1。 + +## 18. 决策 + +| 编号 | 决策 | 状态 | +|---|---|---| +| D1 | 端口模块 `mcpp.provider`；`mcpp.xlings` 降为适配器 | 已确认，已实现 | +| D2 | 归属按 index 划分 | 已实现 | +| D3 | mcpp 的 index 与 store 在 `~/.mcpp/indices`、`~/.mcpp/xpkgs`；`registry/` 为适配器私有 | 已实现 | +| D4 | 项目 index 的数据与 payload 在 `/.mcpp/` | 已确认（O6），已实现 | +| D5 | index 刷新改为暂存再交换 | 已实现 | +| D6 | mcpp 不写 payload：删除 W1–W6 与 #273 限制 | 已实现（依据 §2.1、§2.2） | +| D7 | 工具链运行库由 registry home 的运行库钉版本决定 | 已实现（依据 §2.1） | +| D8 | 复用：`[xlings] upstream`，默认空，用户配置 > 系统级配置 > 空；链接条目；上游只读；悬空自愈 | 已实现 | +| D9 | store 路径不 canonical 化；兄弟包与删除经端口 | 已实现 | +| D10 | `[xlings] home` 仍是 registry 位置，拒绝非 mcpp 的 home | 已实现 | +| D11 | 所有 xlings 来源经握手，不满足回退内置 | 已实现 | +| D12 | mcpp 平台层的文件系统能力表与 xlings 同构 | 已实现 | +| D13 | `windows_long_paths`；mcpp 自身与宿主工具默认开；路径预算检查 | 已实现 | +| D14 | `mcpp self store dedupe`，默认只列出 | 已确认（O4），已实现 | +| D15 | 系统级安装：`--system` 只用于维护命令；以写权限判断；用户侧无需 root；构建缓存不跨用户 | 已实现 | +| D16 | 上游信任边界：当前用户或 root 所有 | 已实现 | +| D17 | 一个 PR，mcpp 2026.10.16.1；前置 xlings 发布 X1–X7 | 已实现 | + +## 19. 开放问题 + +| 编号 | 问题 | 建议 | +|---|---|---| +| O7 | 系统 home 的默认位置：Linux `/var/lib/mcpp`（FHS 的可变状态）还是 `/opt/mcpp` | `/var/lib/mcpp`；Luban 可在系统级配置中改 | +| O8 | `--system` 是否也允许 `index update` 以外的 index 管理（`index add`）：系统级私有 index | 允许，写入系统级配置的 `[indices]`，用户可见但不可改 | + +## 20. 自我 review + +| 角度 | 发现与处理 | +|---|---| +| 架构 | 修订 1 只删除 patchelf 与 cfg，漏了 `.mcpp_ok`、`.mcpp-fixup.json` 与 `xpkg_copy` 三处写入或复制（probe 第 1 轮暴露）；已补为 W4–W6 | +| 借用的完整性 | 修订 1 的借用只发布链接，没有说明借用方 home 的 subos 条目与 `abi` 链接；mcpp 的 sysroot 是 registry 的 `subos/default`，缺了它头文件与 crt 会缺失；已写入 X3 | +| 安全 | 修订 1 没有信任边界：上游可以是任意目录；已加 §8.2。系统级安装不共享构建缓存（§9.3） | +| 删除 | `std::filesystem::remove_all` 作用于 junction 的行为依赖实现；规定删除只经端口交给 xlings，lint 禁止对 `Payload::dir` 调用 `remove_all`；§2.2 观察到 xlings 只删链接，X3 写为要求并由 e2e 断言 | +| 并发 | 两个进程同时借同一条目：发布走 `rename_no_replace`，后者看到已存在即使用；自愈与借用同一路径同样如此 | +| 离线 | 悬空条目在离线时无法自愈，报错而不是静默降级（§8.4） | +| 跨平台 | junction 不能指向网络共享：Windows 上上游在网络路径时不借（degraded）；macOS 上 clang 经真实路径找资源目录，身份一致保证与上游 payload 内部自洽（§2.2） | +| 兼容 | 存量 home 的旧位置、旧标记、已修改的 payload 都保持可用（§5.3）；默认仍是自包含，已安装的 mcpp 行为不变 | +| 长路径 | 修订 1 只有 manifest；probe 显示开关关闭时首先失败在 xlings，增加路径预算检查与 X6 的如实报告 | +| 范围 | 内容多，但都是"mcpp 对 xlings 的依赖"一个主题的组成部分；按 §17 的顺序在一个 PR 内完成 | +| 生态（修订 3） | 修订 2 只考虑了 Luban 宿主，没有考虑 Luban 系统内部同时带 gcc 与 mcpp 而 mcpp 再下载一份的情形；已补 §9.5 与 X7 | +| 生态（修订 3） | mcpp 在系统里的失败诊断依赖 xlings#652；已列为依赖 | +| 生态（修订 3） | aarch64 上 mcpp 的 `gcc@16.1.0` 指 musl 版，core 带的是 glibc 版；edition 的 mcpp 配置写明目标（§9.5） | +| 发布（修订 3） | edition 是 mcpp 新能力的使用者，必须在 mcpp 发布之后才 bump 进 core；顺序已与 xlings 设计对齐 | + +## 21. 发布 + +release PR `release/2026.10.16.1`；CI 全绿后自我 review，dispatch mcpp-plugins CI，合并后 `release.yml`。 +检查每个归档内置的 xlings 版本等于 X 发布的钉版本；clean-room 中分别验证自包含、`upstream=["~/.xlings"]`、系统级三种形态。 +probe PR #810 保留到发布完成后关闭。 + +## 22. 实现记录 + +| 项 | 实现 | +|---|---| +| 归属与位置 | `[index.repos]` 与 mcpplibs 的条目带 `dir`、`store`（`/indices/`、`/xpkgs`；项目在 `/.mcpp`）；`own_indices` 在每次加载配置时把旧位置的数据移动一次；同名的旧副本不再被读取 | +| 刷新 | 适配器对每个自有索引调用 `sync_index`（目的地为 `.staging-`），按 `min_mcpp` 判定后 `exchange_paths`；xlings 侧 `update` 只在自有索引缺失时拉取它，之后由属主刷新（xlings `repo.cpp`） | +| 兼容读 | 提供者说不出、或在索引的 store 中没有时，按版本文法在 home 的各 store 中查找（`COMPAT(internal:legacy-registry-store)`） | +| 握手 | 非 bundled 来源在加载配置时经 `xlings interface --version` 判定，不满足退回 bundled 并给出 note；bundled 由钉版本保证 | +| 上游 | `[xlings] upstream`（用户配置，否则系统级配置）；信任检查在 `platform::untrusted_because`；mcpp home 提供两条上游（registry 的 `xim`、`xpkgs` 的全部命名空间） | +| 系统级 | `--system` 在 CLI 预处理阶段校验命令、写权限，设置 `MCPP_HOME`、umask 022、排他锁与 `.mcpp-home`；系统级配置路径可由 `MCPP_SYSTEM_CONFIG` 指定（测试用） | +| 长路径 | `windows_long_paths`（target 与 `[test]`）；host 工具默认 true；mcpp.exe 清单声明；路径预算常量 140（gcc 16.1.0 libstdc++ 头文件在 `` 下的最深路径，实测） | +| GCC 16 | 端口的虚函数不返回 `std::optional` 与 `std::expected`：两者都使导入方读不出模块 | +| 测试 | 单元：适配器的自有索引、迁移、store 查找；platform 的信任与目录链接；清单键与应用程序清单。e2e：914 上游、915 系统 home、916 存量 home、917 Windows 长路径；CI：NFSv3 上的 home 与项目；桩 xlings 回答握手 | + diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 08730041e..de9396ca4 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -331 records. +332 records. ## By subject @@ -121,10 +121,15 @@ Records that declare one. Everything else is listed by date below. - [The engine gaps left open after the SDK batch](2026-09-12-engine-gaps-after-the-sdk-batch.md) — landed - [Six open issues: what each one actually is, and what would answer it](2026-09-11-six-open-issues-analysis.md) — active +### xlings + +- [mcpp 对 xlings 的依赖：Provider 端口、归属与位置、上游复用、系统级安装、payload 不可变、文件系统能力与长路径（#809）](2026-10-16-xlings-provider-port-and-shared-store-design.md) — active + ## By date ### 2026-10 +- [mcpp 对 xlings 的依赖：Provider 端口、归属与位置、上游复用、系统级安装、payload 不可变、文件系统能力与长路径（#809）](2026-10-16-xlings-provider-port-and-shared-store-design.md) — active - [构建依赖的一条管线：配置与产物两轴、包的实例、库接口、兼容系统与工作空间收尾（#797）](2026-10-11-build-dependencies-one-pipeline-design.md) — active - [一处来源，一种含义：工作空间的传递语义、生成输出、模块角色与首次运行（#786、#785、#778、#790）](2026-10-10-one-source-one-meaning-design.md) — landed - [LLVM 23.1.3 Part 2：Linux aarch64 默认工具链与 glibc 生态闭环](2026-10-08-llvm-2313-linux-aarch64-ecosystem-part2-design.md) — active diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index b3b4a70ab..f1a5fdd5a 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -21,7 +21,7 @@ inputs: xlings-version: description: xlings release to bootstrap from required: false - # Must equal `pinned::kXlingsVersion` in src/xlings/xlings.cppm — enforced by + # Must equal `pinned::kXlingsVersion` in src/provider/xlings/xlings.cppm — enforced by # .github/tools/check_version_pins.sh, not by this comment. (The previous # comment here listed the files to keep in lock-step, and that list was # already incomplete; that is why the check exists.) @@ -33,7 +33,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.10.10.2' + default: '2026.10.16.1' outputs: sandbox-key: diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index 1ff84674f..b8104b4ec 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.10.10.2' + default: '2026.10.16.1' image: description: > The runner label the job runs on (macos-15, xcode-27). It is part of the diff --git a/.github/tools/check_provider_boundary.py b/.github/tools/check_provider_boundary.py new file mode 100644 index 000000000..86408b170 --- /dev/null +++ b/.github/tools/check_provider_boundary.py @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 +"""The provider boundary (.agents/docs/2026-10-16-xlings-provider-port-and-shared-store-design.md §4.3). + +mcpp speaks the port's words (mcpp.provider): what xlings' commands, its +protocol, its home layout and its store naming are is the adapter's +(src/provider/xlings/), and only the composition root (src/provider/select.cppm) +names the adapter. Outside them, in src/: + + - no `import mcpp.provider.xlings*`; + - none of the layout's spellings in code: "xpkgs", "-x-", "xim-x-", + ".xlings.json", "subos", "xim-index-repos", ".mcpp_ok"; + - a payload path is used as the provider answered it: no canonical / + weakly_canonical / remove_all applied to one. + +Comments are prose and may name anything; string literals and identifiers +in code may not. An exception names its reason on the line: +`// provider-boundary-ok: `. + +Exit 0 clean, 1 with the findings listed. +""" +from __future__ import annotations + +import argparse +import pathlib +import re +import sys + +LITERALS = ['"xpkgs"', '/xpkgs', 'xpkgs/', '-x-', 'xim-x-', '.xlings.json', '"subos"', '/subos', 'subos/', + 'xim-index-repos', '.mcpp_ok', '.mcpp-fixup.json'] +IMPORT = re.compile(r'^\s*(export\s+)?import\s+mcpp\.provider\.xlings\b') +PAYLOAD_FS = re.compile(r'\b(canonical|weakly_canonical|remove_all)\s*\([^)]*[pP]ayload') + + +def strip_comments(text: str) -> list[str]: + """Each line with // and /* */ comments blanked, string literals kept.""" + out, in_block = [], False + for line in text.split("\n"): + res, i, in_str = [], 0, None + while i < len(line): + c = line[i] + if in_block: + if line.startswith("*/", i): + in_block = False + i += 2 + continue + i += 1 + continue + if in_str: + res.append(c) + if c == "\\" and i + 1 < len(line): + res.append(line[i + 1]) + i += 2 + continue + if c == in_str: + in_str = None + i += 1 + continue + if line.startswith("//", i): + break + if line.startswith("/*", i): + in_block = True + i += 2 + continue + if c in "\"'": + in_str = c + res.append(c) + i += 1 + out.append("".join(res)) + return out + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + ap.add_argument("--root", type=pathlib.Path, default=pathlib.Path(__file__).resolve().parents[2]) + ROOT = ap.parse_args().root.resolve() + SRC = ROOT / "src" + ADAPTER = SRC / "provider" / "xlings" + ROOT_MODULE = SRC / "provider" / "select.cppm" + findings = [] + for path in sorted(SRC.rglob("*")): + if path.suffix not in (".cppm", ".cpp", ".hpp", ".h") or not path.is_file(): + continue + if ADAPTER in path.parents: + continue + raw = path.read_text(encoding="utf-8", errors="replace").split("\n") + code = strip_comments("\n".join(raw)) + for n, (line, original) in enumerate(zip(code, raw), 1): + if "provider-boundary-ok:" in original: + continue + rel = path.relative_to(ROOT) + if IMPORT.match(line) and path != ROOT_MODULE: + findings.append(f"{rel}:{n}: imports the adapter; import mcpp.provider.select") + for lit in LITERALS: + if lit in line: + findings.append(f"{rel}:{n}: names the provider's layout ({lit}): {original.strip()}") + break + if PAYLOAD_FS.search(line): + findings.append(f"{rel}:{n}: canonicalises or removes a payload: {original.strip()}") + if findings: + print("provider boundary: " + str(len(findings)) + " finding(s)") + for f in findings: + print(" " + f) + return 1 + print("provider boundary: clean") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/tools/check_version_pins.sh b/.github/tools/check_version_pins.sh index d0059b279..aa6e3ced6 100755 --- a/.github/tools/check_version_pins.sh +++ b/.github/tools/check_version_pins.sh @@ -5,13 +5,13 @@ # Two invariants that used to live only in a comment: # # 1. Every xlings version pinned anywhere in .github/ equals -# `pinned::kXlingsVersion` in src/xlings/xlings.cppm — which is the version +# `pinned::kXlingsVersion` in src/provider/xlings/xlings.cppm — which is the version # `mcpp self env` reports and the one release.yml bundles into the # tarball as /registry/bin/xlings. # # 2. mcpp's own version is identical in all four places that carry it. # -# Why this exists: src/xlings/xlings.cppm used to say "keep in lock-step with the +# Why this exists: src/provider/xlings/xlings.cppm used to say "keep in lock-step with the # XLINGS_VERSION pins in release.yml / cross-build-test.yml / ci-linux-e2e.yml" # and that list was ALREADY incomplete — it omitted both composite actions, # which sat on 0.4.30 while everything else moved to 0.4.69. CI's sandbox @@ -38,12 +38,12 @@ strip_comments() { sed 's/#.*//'; } # ── 1. xlings pins ──────────────────────────────────────────────────────── -XLINGS_EXPECTED=$(grep -oE 'kXlingsVersion[[:space:]]*=[[:space:]]*"[^"]+"' src/xlings/xlings.cppm \ +XLINGS_EXPECTED=$(grep -oE 'kXlingsVersion[[:space:]]*=[[:space:]]*"[^"]+"' src/provider/xlings/xlings.cppm \ | grep -oE '"[^"]+"' | tr -d '"' | head -1) [ -n "$XLINGS_EXPECTED" ] || { - echo "FAIL: could not read kXlingsVersion from src/xlings/xlings.cppm" >&2; exit 1; } + echo "FAIL: could not read kXlingsVersion from src/provider/xlings/xlings.cppm" >&2; exit 1; } -note "expected xlings pin: $XLINGS_EXPECTED (src/xlings/xlings.cppm)" +note "expected xlings pin: $XLINGS_EXPECTED (src/provider/xlings/xlings.cppm)" # Anchored patterns only — a bare "version-looking number on a line mentioning # xlings" would also match `xlings install llvm@20.1.7`, which pins LLVM, not diff --git a/.github/tools/run_published_arm64_cn.sh b/.github/tools/run_published_arm64_cn.sh index 6f66bb5dc..2114900f8 100755 --- a/.github/tools/run_published_arm64_cn.sh +++ b/.github/tools/run_published_arm64_cn.sh @@ -12,7 +12,7 @@ base=${XLINGS_HOME:?cold isolated home required} mkdir -p "$base/bootstrap" "$base/probes" "$base/reports" cd "$base" # Derive the client version from the repository's existing single pin. -xl_version=$(python3 - "$repo/src/xlings/xlings.cppm" <<'PY' +xl_version=$(python3 - "$repo/src/provider/xlings/xlings.cppm" <<'PY' import pathlib,re,sys s=pathlib.Path(sys.argv[1]).read_text() m=re.search(r'kXlingsVersion\s*=\s*"([0-9.]+)"',s) diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index 18cc6b4ec..60034c2e6 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.10.10.2' + XLINGS_VERSION: '2026.10.16.1' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-bootstrap.yml b/.github/workflows/ci-bootstrap.yml index b85fffe29..ca0c32fc3 100644 --- a/.github/workflows/ci-bootstrap.yml +++ b/.github/workflows/ci-bootstrap.yml @@ -41,7 +41,7 @@ jobs: os: [ubuntu-24.04, macos-15, windows-2025] env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.10.2' + XLINGS_VERSION: '2026.10.16.1' steps: - uses: actions/checkout@v4 @@ -50,7 +50,7 @@ jobs: shell: bash run: | set -euo pipefail - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.10.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.16.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" echo "$HOME/.xlings/bin" >> "$GITHUB_PATH" @@ -124,7 +124,7 @@ jobs: shell: bash run: | set -euo pipefail - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.10.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.16.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install the newest release and build the tree with it (mcpp build --strict) diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index c952e61a1..21305449c 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.10.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.16.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -315,7 +315,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.10.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.16.1 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -464,7 +464,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.10.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.16.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index f038ef47c..c26d92102 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -107,6 +107,55 @@ jobs: if-no-files-found: warn retention-days: 7 + # An MCPP_HOME and a project on NFSv3 (docs/25 §6, #809): the filesystem + # whose rename refuses RENAME_NOREPLACE with EINVAL, where `mcpp new` once + # failed. mcpp measures what the filesystem does and publishes by it. + nfs: + name: e2e nfs (NFSv3 home and project) + runs-on: ubuntu-24.04 + timeout-minutes: 30 + steps: + - uses: actions/checkout@v4 + + - uses: ./.github/actions/bootstrap-mcpp + + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 + + - name: An NFSv3 export of this machine, mounted + run: | + set -euo pipefail + sudo apt-get update -q + sudo apt-get install -y --no-install-recommends nfs-kernel-server nfs-common + sudo mkdir -p /srv/nfs /mnt/nfs + sudo chown "$(id -u):$(id -g)" /srv/nfs + echo "/srv/nfs 127.0.0.1(rw,sync,no_subtree_check,no_root_squash)" | sudo tee /etc/exports + sudo exportfs -ra + sudo systemctl restart nfs-kernel-server + sudo mount -t nfs -o vers=3,nolock 127.0.0.1:/srv/nfs /mnt/nfs + test "$(stat -f -c %T /mnt/nfs)" = nfs + + - name: A toolchain, a new project, its build and run, all on NFSv3 + run: | + set -euo pipefail + export MCPP_HOME=/mnt/nfs/mcpp-home + "$MCPP" toolchain install gcc 16.1.0 + cd /mnt/nfs + "$MCPP" new hello + cd hello + "$MCPP" build + "$MCPP" run > "$RUNNER_TEMP/run.log" + grep -q 'Hello from hello' "$RUNNER_TEMP/run.log" + + - name: Doctor names the home's filesystem + run: | + set -euo pipefail + export MCPP_HOME=/mnt/nfs/mcpp-home + # doctor's exit status reports the machine's health, not this line. + "$MCPP" self doctor > "$RUNNER_TEMP/doctor.log" 2>&1 || true + grep -q '/mnt/nfs/mcpp-home on nfs' "$RUNNER_TEMP/doctor.log" + # ────────────────────────────────────────────────────────────────── # Bare metal: the one chain the sharded suite above cannot be trusted # to exercise. @@ -365,7 +414,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.10.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.16.1 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d6aca258d..3722d4a9c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -143,11 +143,15 @@ jobs: - name: Reason tokens agree with the engine, and with the mirror run: bash .github/tools/check_reason_tokens.sh + - name: Only the adapter and the composition root name the provider + run: python3 .github/tools/check_provider_boundary.py + - name: The tools of the CI stages pass their fixtures run: | python3 tests/scripts/test_classify_changes.py python3 tests/scripts/test_check_e2e_coverage.py python3 tests/scripts/test_check_default_toolchain_docs.py + python3 tests/scripts/test_check_provider_boundary.py python3 tests/scripts/test_native_candidate_admission.py python3 -m unittest discover -s tests/scripts -p test_toolchain_env.py diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 6a0884c24..b229a97d4 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -115,7 +115,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - # Must equal `pinned::kXlingsVersion` (src/xlings/xlings.cppm) and the + # Must equal `pinned::kXlingsVersion` (src/provider/xlings/xlings.cppm) and the # xlings the release bundles — enforced by # .github/tools/check_version_pins.sh. # @@ -131,7 +131,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.10.10.2' + XLINGS_VERSION: '2026.10.16.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -323,7 +323,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.10.2' + XLINGS_VERSION: '2026.10.16.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 89cfb181c..08d463adb 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -104,7 +104,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.10.10.2' + XLINGS_VERSION: '2026.10.16.1' run: | # BY VERSION, NOT BY PRESENCE. A restored ~/.xlings holds the # xlings of an earlier release, and this binary is the one the @@ -329,7 +329,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.10.2' + XLINGS_VERSION: '2026.10.16.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -400,7 +400,7 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.10.10.2-linux-aarch64.tar.gz" + XLA="xlings-2026.10.16.1-linux-aarch64.tar.gz" # NOT fetch_release.sh: this asset is OPTIONAL and the `if` is the # point — an arch with no prebuilt xlings must fall through quietly, # while the helper retries a 404 five times before giving up. The one @@ -409,9 +409,9 @@ jobs: # cover it. if curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors \ --connect-timeout 20 --max-time 600 -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.10.10.2/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.10.16.1/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.10.10.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.10.16.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -489,7 +489,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.10.2' + XLINGS_VERSION: '2026.10.16.1' run: | # By version, not by presence (see the Linux job). have=$("$HOME/.xlings/subos/default/bin/xlings" --version 2>/dev/null | awk '{print $2}') @@ -676,7 +676,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.10.2' + XLINGS_VERSION: '2026.10.16.1' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/CHANGELOG.md b/CHANGELOG.md index 764840eea..fc4ba7f16 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,59 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [2026.10.16.1] - 2026-10-16 + +### Added + +- Keep the data and payloads of each index mcpp serves in mcpp's home: + `/indices/` and `/xpkgs` (a project's in + `/.mcpp`). The provider's own indexes stay under `registry/`. A home + from an earlier mcpp has its index data moved once; payloads already + installed stay where they are and are still found (docs/25 §1). +- Reuse another home's packages: `[xlings] upstream = ["~/.xlings"]` in the + user's `config.toml`, else in the system configuration + (`/etc/mcpp/config.toml`). A payload of the same identity is linked, never + copied or written; an upstream must belong to the user or to the system and + be unwritable by the user unless it is the user's own (docs/25 §3). +- `mcpp self store dedupe [--apply]`: list the payloads an upstream has the + same of, and replace each by a link. +- `--system` maintains the machine's system home (`/var/lib/mcpp`, `[system] + home` of the system configuration) with `self init|doctor|store`, + `toolchain install|remove|default` and `index update`; any other command is + refused, and so is a caller without write access (docs/25 §4). +- `windows_long_paths` (target and `[test]` keys): `longPathAware` in a PE + executable's application manifest; the default of a host tool. `mcpp.exe` + declares it. With Windows long paths off and a long `MCPP_HOME`, the path + budget is reported as a degradation. +- `mcpp self doctor` states the home: whether it is the system home, its + upstreams, what its filesystem does, and Windows' long-path setting. +- Ask every xlings source what it is (`xlings interface --version`); one that + cannot serve this mcpp gives way to the bundled xlings, with a note. + +### Changed + +- Payloads are written by the provider alone. mcpp no longer patches a + toolchain's ELF headers, rewrites its `specs` or clang `.cfg`, writes + `.mcpp_ok` or `.mcpp-fixup.json` into it, or copies payloads from + `~/.xlings`. A toolchain runs on the runtime the provider bound it to through + the registry home's runtime pin, which mcpp sets to the runtime it selected. +- Refresh an owned index into a staging directory, check its `min_mcpp` there, + and exchange it with the copy in use only when this mcpp can read it. The + backup-and-rollback of earlier versions is removed. +- Every use of xlings goes through one port (`mcpp.provider`); only the + adapter under `src/provider/xlings/` names xlings' commands, protocol and + store layout, which `.github/tools/check_provider_boundary.py` checks. +- Publish scaffolds and index swaps by what the filesystem holding them does + (NFSv3, SMB, FAT and FUSE included), measured once per filesystem (#809). +- Bundle xlings 2026.10.16.1, the first that places an index's data and store + where its client names them, links from upstreams, and answers the + handshake (openxlings/xlings#676). + +### Fixed + +- Resolve the openkal index and runtime revisions once per run (#804). +- Accept space-separated `toolchain remove` specs (#808). + ## [2026.10.11.1] - 2026-10-11 ### Added diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 88a2413dd..a3817342b 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -436,6 +436,7 @@ required_features = ["gui"] # only built when feature `gui` is | `windows_subsystem` *(2026.9.12.2+)* | The PE subsystem of an executable: `"console"` (the default) or `"windows"`, a GUI program that starts without a console. Reaches this target's link and no other, and renders nothing on a target that is not PE. See the section above. | | `windows_entry` *(2026.9.12.2+)* | The entry function the program defines: `"main"` (the default), `"wmain"`, `"WinMain"` or `"wWinMain"`. See the section above. | | `windows_code_page` *(2026.9.26.1+)* | The ANSI code page of an executable on Windows: `"utf-8"`, embedded as an application manifest that Windows 10 version 1903 and later honour, or `"legacy"`, the system's code page. A program target that writes neither runs in the system's code page, except a program built as a host tool (§2.14), which defaults to `"utf-8"`. Refused on a library target; renders nothing on a target that is not PE. See *Paths and text encoding* in §2.3. | +| `windows_long_paths` *(2026.10.16.1+)* | `true` embeds `longPathAware` in the executable's application manifest: the process may use paths longer than 260 characters where Windows enables them (`LongPathsEnabled`). `false` embeds nothing. A program target that writes neither embeds nothing, except a program built as a host tool (§2.14), which defaults to `true`. Refused on a library target; renders nothing on a target that is not PE. | | `linkage` *(2026.9.15.2+)* | A library target's **default** link form, `"static"` or `"shared"`: the form a consumer that writes no `linkage` receives. Unlike `kind = "shared"` it is not a constraint, so a consumer's explicit statement is honoured. Refused beside `kind = "shared"` and on a program target. See [`dependency_linkage`](#dependency_linkage--static-or-shared-is-the-consumers-decision). | > **Scope (important):** `defines` / `cxxflags` / `cflags` on a target apply **only to that @@ -1882,6 +1883,7 @@ discover = ["tests/**/*.cpp"] # the default |---|---|---| | `discover` | array of globs | every file a glob matches is one test program; a glob beginning with `!` removes the files it matches; `[]` discovers none | | `windows_code_page` | `"utf-8"` or `"legacy"` | the ANSI code page of every test program on Windows, in the meaning of the target key of the same name; the default is `"legacy"`, the system's code page *(2026.10.5.2+)* | +| `windows_long_paths` | `true` or `false` | whether every test program on Windows is long-path aware, in the meaning of the target key of the same name; the default is `false` *(2026.10.16.1+)* | The globs use the vocabulary of `[build] sources`. A test's name is its path relative to the fixed directory of the first glob that matched it, without the diff --git a/docs/09-commands-by-scenario.md b/docs/09-commands-by-scenario.md index a0058963e..9b23e7dd7 100644 --- a/docs/09-commands-by-scenario.md +++ b/docs/09-commands-by-scenario.md @@ -510,6 +510,26 @@ included. `mcpp self config --mirror CN|GLOBAL` selects the download mirror; mcpp and xlings hold this setting separately, so selecting it for one does not select it for the other. +## Reusing another home, and a machine's system home + +With `[xlings] upstream` set ([25](25-home-indices-and-store.md) §3), installs +link an upstream's payloads instead of downloading them. A home that installed +copies before can trade them for links: + +```bash +mcpp self store dedupe # list what an upstream has the same of +mcpp self store dedupe --apply # replace each by a link, atomically +``` + +An administrator maintains the machine's system home with `--system`, which +only the maintenance commands take ([25](25-home-indices-and-store.md) §4): + +```bash +sudo mcpp --system self init +sudo mcpp --system toolchain install gcc 16.1.0 +sudo mcpp --system index update +``` + ## `[hooks]` — Project Build Lifecycle Commands (experimental) > **Experimental.** A hook cannot currently decide whether a build succeeded. diff --git a/docs/11-publishing-a-library.md b/docs/11-publishing-a-library.md index f1ebca598..bc2bd8d1a 100644 --- a/docs/11-publishing-a-library.md +++ b/docs/11-publishing-a-library.md @@ -126,11 +126,10 @@ While the chain above is still in flight, seed the registry by hand so consumers can compile against the library before it is published: ```bash -REG=~/.mcpp/registry/data/xpkgs/-x-/0.0.48 +REG=~/.mcpp/xpkgs/-x-/0.0.48 # the store of mcpp's indices (25 §1) mkdir -p "$REG" git -C /path/to/library archive --format=tar --prefix=-0.0.48/ HEAD \ | tar -x -C "$REG" -touch "$REG/.mcpp_ok" # the marker that says "resolved" cp ../0.0.47/.xpkg.lua "$REG/.xpkg.lua" # add a 0.0.48 entry to it ``` diff --git a/docs/25-home-indices-and-store.md b/docs/25-home-indices-and-store.md new file mode 100644 index 000000000..39c9cabdb --- /dev/null +++ b/docs/25-home-indices-and-store.md @@ -0,0 +1,160 @@ +# 25 — The Home, Its Indices and Its Store + +**Reader:** someone who decides where mcpp keeps what it installs: a developer +with an existing `~/.xlings`, an administrator of a shared machine, a Luban +system. + +**The question this chapter answers:** what is in an mcpp home, which program +owns each part, and how a home reuses packages another home already has. + +**Not here:** choosing a toolchain, which is [20 — Toolchain +Management](20-toolchains.md), and a project's own build environment, which is +[23 — The Project Environment](23-the-project-environment.md). + +mcpp installs toolchains and packages through xlings, its package provider. A +payload — an installed package version — is written by the provider alone and +is never modified afterwards: mcpp does not patch, mark or copy it. Whatever a +build needs that a payload does not hold (a loader, a run path, a sysroot) is +stated on the build's own command lines. + +## 1. The layout + +``` +~/.mcpp/ (MCPP_HOME) + config.toml + indices// the data of each index mcpp serves (mcpplibs, [index.repos.]) + xpkgs/ the payloads those indices serve + registry/ the provider's own home: its index (xim), its runtime environments, its store + data/xpkgs/ toolchains and tools; an entry may be a link to an upstream's payload + build-cache/ cache/ provisioned/ … + +/.mcpp/ + indices// the data of the project's own [indices] + xpkgs/ the payloads they serve +``` + +Ownership follows the index. An index mcpp serves keeps its data and its +payloads in mcpp's home; the provider's own indexes (`xim` and the ones it adds) +stay under `registry/`. A local path index (`path = "…"`) is read where it is. + +A home created by an earlier mcpp keeps working. Its index data moves to +`indices/` once, on the first run; payloads already installed under +`registry/data/xpkgs` stay where they are and are still found, and what was +built against them keeps running. The markers an earlier mcpp wrote into +payloads (`.mcpp_ok`, `.mcpp-fixup.json`) are no longer read. + +## 2. Refreshing an index + +An index mcpp owns is fetched into a staging directory beside it, checked there +against the floor it declares (`index.toml` `min_mcpp`), and exchanged with the +copy in use only when this mcpp can read it. A refreshed index that needs a +newer mcpp leaves the previous copy in place and is reported once, after the +command's own output. + +## 3. Reusing another home: upstreams + +```toml +# ~/.mcpp/config.toml +[xlings] +upstream = ["~/.xlings"] +``` + +An upstream is a home whose payloads this home installs from **by link**: a +payload of the same identity — namespace, name, version and the recipe's +revision — that is complete there is published in this home's store as a +symbolic link (a junction on Windows). Nothing is downloaded and the upstream is +never written to. Removing the package here removes the link only. When the +upstream's payload disappears, the next install puts a payload of its own at the +same path, so what was built against that path keeps working; with `--offline` +that install cannot happen and the command fails. + +| upstream | provides | +|---|---| +| an xlings home (`~/.xlings`, `/xlings`) | the provider's own packages (`xim`) | +| an mcpp home (another `~/.mcpp`, the system home) | its registry's `xim` packages, and the packages of mcpp's indices in its `xpkgs` | + +mcpp's own packages are taken only from an mcpp home, never from an xlings home. + +**Trust.** An upstream belongs to the current user or to the system (root; on +Windows also Administrators and SYSTEM), and when it is not the user's own the +user cannot write it. A home that fails this is refused when the configuration +is read: using another person's home means running programs that person can +replace. An upstream that does not exist, or is neither kind of home, is +reported as a degradation (no reuse this run). + +**Where it is read.** `upstream` in the user's `config.toml`; when absent, the +system-level configuration (`/etc/mcpp/config.toml`; Windows +`%ProgramData%\mcpp\config.toml`); when absent too, none — the default, a +self-contained home. + +**A home that already has copies.** `mcpp self store dedupe` lists the payloads +this home installed itself that an upstream has the same of; +`mcpp self store dedupe --apply` replaces each with a link, atomically. + +## 4. A machine's system home + +On a machine shared by several users an administrator maintains one home for +all of them: + +```bash +sudo mcpp --system self init +sudo mcpp --system toolchain install gcc 16.1.0 +``` + +| | | +|---|---| +| location | `[system] home` of the system configuration; else `/var/lib/mcpp` (macOS `/Library/Application Support/mcpp`, Windows `%ProgramData%\mcpp`) | +| written by | `mcpp --system` with one of `self init`, `self doctor`, `self store`, `toolchain install`, `toolchain remove`, `toolchain default`, `index update` | +| refused | `--system` with any other command (a build as root would write root's files into a user's project); `--system` without write access to the system home (the message names `sudo`) | +| written as | files readable by every user (umask 022), one writer at a time, the caller's `HOME` and `MCPP_HOME` ignored | +| marked by | `.mcpp-home` with `mode = "system"` | + +Users read it without root. The system configuration names it as their +upstream, and may name a default toolchain: + +```toml +# /etc/mcpp/config.toml +[xlings] +upstream = ["/var/lib/mcpp"] + +[toolchain] +default = "gcc@16.1.0" +default_target = "x86_64-linux-gnu" +``` + +Each user keeps a home of their own: their index data, their build cache (never +shared between users) and links into the system home's payloads. An index of +the user's that resolves a newer version installs it into the user's home. + +On Luban OS the system configuration names `/xlings` as well, so a payload the +machine already has exists on it once. + +## 5. The provider program + +`[xlings] binary` chooses the xlings mcpp runs: `"bundled"` (the one mcpp +ships, the default), `"system"` (the one on `PATH`) or a path. Whichever it is, +mcpp asks it what it is (`xlings interface --version`): its version, its +protocol and the features mcpp relies on. One that cannot serve this mcpp gives +way to the bundled one, with a note naming why. + +`[xlings] home` moves the registry; it names a home mcpp made. A home mcpp did +not make (an `~/.xlings`) is refused there: reusing its packages is what +`upstream` is for (§3). + +## 6. Filesystems and long paths + +A home may be on any filesystem. mcpp measures what the one holding its home +can do — an atomic rename that refuses an existing name, an atomic exchange, +hard links, symbolic links — and publishes by what it measured. +`mcpp self doctor` shows the answer, and whether reuse from an upstream is +possible there (it needs symbolic links). + +On Windows a path is limited to 260 characters unless the system enables long +paths (`LongPathsEnabled` = 1). `mcpp.exe` declares itself long-path aware. When +long paths are off and `MCPP_HOME` is long enough that a toolchain's deepest +file would pass the limit, mcpp reports a degradation naming the two remedies: +enable long paths, or use a shorter `MCPP_HOME`. mcpp does not change the +setting; `mcpp self doctor` reports it. + +A program's own manifest states whether it is long-path aware: +`windows_long_paths` ([04](04-mcpp-toml.md)). diff --git a/docs/91-toolchain-internals.md b/docs/91-toolchain-internals.md index b00dd0ace..98963a122 100644 --- a/docs/91-toolchain-internals.md +++ b/docs/91-toolchain-internals.md @@ -25,15 +25,14 @@ resolve runtime binding ← which libc the artifact will load (§2.1) ▼ resolve toolchain payload ← project/default/install paths share one pipeline ▼ -ensure_post_install_fixup() ← exact glibc@version, marker-gated; never readdir-first +ensure_declared_runtime() ← exact glibc@version installed; the payload is never written ▼ detect / probe ← triple, sysroot, payload paths (glibc, linux-headers) ▼ ToolchainLinkModel (single resolver for the C-library axis) ├──► flags.cppm (main build compile/link flags) ├──► stdmod.cppm (`import std;` BMI precompile) - ├──► build_program (build.mcpp host compiles) - └──► cfg regeneration (the human-facing clang++.cfg) + └──► build_program (build.mcpp host compiles) ▼ hermetic link check (`-###` dry-run) ← checks sandbox CRT/loader resolution ▼ @@ -53,9 +52,9 @@ Two principles run through everything: path still works. 2. **Path knowledge has one owner per layer.** What used to be four divergent copies of "how to link against the payload glibc" is now one resolver - (`linkmodel`); what used to be per-entry-path fixup behavior is now one - pipeline. Divergence between copies is where an entire class of bugs came - from (issue #195). + (`linkmodel`); what used to be per-entry-path fixup behavior is gone, and a + payload is the provider's alone (§4). Divergence between copies is where an + entire class of bugs came from (issue #195). ## 2. Toolchain resolution @@ -309,69 +308,37 @@ shouldn't carry a mechanism whose sole reader is one downstream tool. If an installed-state metadata DB ever appears, it must be designed with xlings itself as its first consumer; mcpp can then re-add a reader. -## 4. The unified post-install fixup pipeline (`src/toolchain/post_install.cppm`) +## 4. Payloads are never written (`src/toolchain/post_install.cppm`) -Sandbox payloads are prebuilt ELF trees whose baked `PT_INTERP`/`RUNPATH` are -unknowable at packaging time and must be aligned to the *local* sandbox. -`ensure_post_install_fixup(cfg, payloadRoot, pkg)` is the **single entry** for -that alignment, called from all three entry paths (explicit install, default -auto-install, manifest auto-install). +A payload is written by the provider (xlings) when it installs it, and by no one +afterwards. mcpp reads payloads; it does not patch their ELF headers, rewrite +their `specs` or clang `.cfg`, write marker files into them, or copy them from +another home. The checks in `.github/tools/check_provider_boundary.py` keep code +outside the provider adapter from naming the store's layout or removing a +payload. -> This pipeline used to rewrite GCC's `specs` file as well, so that produced -> binaries got a loader and an rpath. That is no longer done, and §5 is why. +**What a toolchain runs on.** A prebuilt gcc or llvm on Linux is an ELF tree +whose `PT_INTERP` and `RUNPATH` must name a C runtime. The provider binds them +when it installs the toolchain, through its home's runtime pin: mcpp sets the +pin of its registry home to the runtime it selected (`glibc@2.44.3`) before the +first toolchain install (`xlings config --runtime`, once), and the provider +links the toolchain to that runtime's loader and libraries. A toolchain borrowed +from an upstream home runs on that home's runtime; what it produces is the same, +because the artifact's loader, run paths and sysroot come from the build's own +command lines (§5). -> Historical note: before 0.0.83 each path remembered — or forgot — its own -> subset. The manifest path ran *nothing*, which is how a freshly -> auto-installed llvm kept a stale, environment-dependent cfg (issue #195), -> and how gcc once shipped a sandbox that couldn't find `stdlib.h`. "Which -> command used to install it" must never decide "whether the toolchain -> works". +**What mcpp still does here** (`post_install.cppm`, all read-only): -**Trigger semantics — ask every build, act once:** - -``` -every build → ensure() → read /.mcpp-fixup.json - marker == {schema, kind, rev, glibcLib}? → return (ms-level) - mismatch → run the fixup for this kind, write marker -``` - -The marker is a *content-fingerprinted cache*, not an event flag: it encodes -the fixup revision and the glibc payload it was aligned against. The -"act" branch therefore fires exactly once per -`(payload × fixup-rev × glibc-fingerprint)` — first use, plus the two -re-convergence events that genuinely require rewriting (a fixup-logic -upgrade via `kFixupRev`, or the glibc payload changing underneath). mcpp -asks on every build because the events that invalidate a payload (xlings -swapping glibc, a payload inherited from another home) happen outside -mcpp's sight — trust-but-verify is the only reliable semantic. - -**Per-kind actions:** - -| kind | actions | +| function | answer | |---|---| -| `gcc` (glibc) | patchelf walk over the gcc payload **and the shared binutils payload** (PT_INTERP → sandbox loader, RUNPATH → glibc+gcc lib dirs) — so that *gcc itself* runs. Nothing is written into `specs` | -| `llvm` | patchelf walk over `lib/` only (runtime `.so` RUNPATH; `bin/` is left alone to preserve xlings-set RUNPATHs); deterministic cfg regeneration (§5) | -| `musl-gcc` | nothing — self-contained sysroot, static world | - -**Safety invariants** (each earned by a real incident): - -- **Never patch in place.** patchelf operates on a copy which is then - atomically `rename()`d in: the payload can contain libraries the *current - process* (a self-hosted, dynamically linked mcpp) or a concurrent build - has mmapped, and rewriting a live mapping's backing file corrupts the - running process (observed: exit-time SIGSEGV in `_dl_fini`). `rename` gives - new content a fresh inode; live processes keep the old one. -- **Ownership guard.** Payloads that resolve outside this home's registry - (symlink-inherited from another `MCPP_HOME`) are never patched — their - owner already converged them, and patching through the symlink would brick - the owner's toolchain. -- Extending content-awareness to the patchelf walk (compare - `--print-interpreter`/`--print-rpath` before writing, so an already-aligned - payload converges with **zero writes**) is a known follow-up. -- The long-term direction is for the *installer* (xlings) to own all - writes — at install time and when a payload enters a new home — leaving - mcpp read-only + verification. The pipeline here is the compatibility - layer until then, and the self-healing mechanism for drift either way. +| `runtime_payload_kind` | whether a toolchain runs on a C runtime payload: `gcc`, `llvm` on Linux; nothing for musl-gcc, a system compiler or a PE toolchain | +| `select_glibc_payload_lib` | the library directory of exactly `glibc@` (never another installed version) | +| `detect_baked_loader`, `baked_runtime_binding` | the loader and runtime a payload was bound to, read from its ELF headers | +| `write_clean_link_specs` | the per-build `-specs=` file in the build directory (§5) | + +A home written by an earlier mcpp keeps its patched payloads and the markers it +wrote (`.mcpp_ok`, `.mcpp-fixup.json`); they still run, and the markers are no +longer read. ## 5. The compiler is a capability, not a configuration @@ -408,14 +375,9 @@ the artifact, not the specs file — what a user ships is what matters. ### 5.1 The clang cfg `bin/clang++.cfg` exists so that direct invocations of the bundled -`clang++` (outside mcpp) get a working, hermetic compiler configuration. The fixup pipeline **regenerates** it -deterministically from the link model — same payload ⇒ byte-identical cfg on -every machine and install path — rather than line-patching whatever an -install produced. On Linux that means CRT discovery (`-B`), payload loader + -rpath, lld/compiler-rt/libunwind, and bundled libc++ for the C++ drivers; on -macOS it keeps the historical shape (`--sysroot=` + payload libc++ -headers — the C++ *runtime link* stays with the platform's -`needs_explicit_libcxx` handling in the main build). +`clang++` (outside mcpp) get a working, hermetic compiler configuration. The +llvm recipe's install hook writes it, for the home it installs into; mcpp does +not read it (`--no-default-config`) and does not write it. ## 6. The hermetic link check (`src/build/hermetic.cppm`) @@ -480,21 +442,18 @@ before/after Ninja, and perform zero ELF parses. An unexpected relink drops to the full path before success or execution. `mcpp self doctor` reports the same stored verdict rather than re-probing a potentially different current host. -Post-install alignment follows the same identity rule: `glibc@2.44` resolves -only `/xim-x-glibc/2.44/{lib64,lib}`. A missing/stale exact payload is an -error; another installed version is never a fallback. - -The exact payload is installed by mcpp before the first fixup that patches -against it (`ensure_declared_runtime`, mcpp#660). The step runs only when the -binding locates no payload, only on Linux, only for the `glibc` provider, and -only for a toolchain whose fixup consumes a C runtime (`gcc` and `llvm` -payloads), so a build with a system, musl or PE toolchain downloads nothing. It -installs `xim:glibc@` through xlings and re-resolves the -binding. Before this step existed, the payload was present only when xlings had -installed it as a dependency of a toolchain; a toolchain restored from a CI -cache is not reinstalled, so a cache holding another glibc revision left the -declared payload absent. When the install is impossible, for example offline, -the fixup error names the coordinate `xim:glibc@` to provide. +The runtime follows the same identity rule: `glibc@2.44` resolves only +`/xim-x-glibc/2.44/{lib64,lib}`. A missing exact payload is installed, +never replaced by another installed version. + +mcpp installs it before the first toolchain that runs on it is installed or +used (`ensure_declared_runtime`, mcpp#660): only when the binding locates no +payload, only on Linux, only for the `glibc` provider, and only for a toolchain +`runtime_payload_kind` names, so a build with a system, musl or PE toolchain +downloads nothing. It pins the registry home's runtime to the declared one, +installs `xim:glibc@` through the provider and re-resolves the +binding. When the install is impossible, for example offline, the error names +the coordinate `xim:glibc@` to provide. ### 6.2 The permitted locations of a runtime search path (`runtime_env_contract.cppm`) @@ -542,10 +501,9 @@ loader coupling, so a host binary that stumbles onto them is at worst confused. the C++ driver). Legacy spellings, if any, go in `compat.cppm` only. 3. **Capabilities** (`src/toolchain/provider.cppm`): stdlib identity, BMI traits, and feature switches consumed by `flags.cppm`. -4. **Fixup kind** (`post_install.cppm`): decide what post-install alignment - the payload needs — gcc-like (patchelf + specs), llvm-like (lib patchelf + - cfg), or none (self-contained). Wire it into - `ensure_post_install_fixup`'s dispatch. +4. **Runtime kind** (`post_install.cppm`): whether the payload runs on a C + runtime payload (`runtime_payload_kind`); the provider binds it to the + pinned runtime at install, and mcpp writes nothing into it. 5. **e2e**: a hermetic-link test in the spirit of `86_llvm_hermetic_link.sh`, and coverage in the no-host-toolchain CI job. @@ -560,8 +518,8 @@ The machinery is already arch-parameterized; the work is data: itself) — the aarch64-linux-musl cross target is the working precedent (`[target.aarch64-linux-musl]`, cross frontend resolution via the spec's `targetTriple`); -3. nothing else: `-B`/`-L`/loader emission, the fixup pipeline, and the - hermetic check are all name-agnostic. +3. nothing else: `-B`/`-L`/loader emission and the hermetic check are + name-agnostic, and the provider binds the toolchain to its runtime. ### 7.3 Embedded and bare-metal toolchains @@ -573,8 +531,8 @@ Three of this section's earlier predictions held: - **No dynamic linker.** `loader` stays empty, which every renderer already permitted; the deployment story is flashing rather than ELF interp. -- **The target side needs no fixup.** Host-run compiler binaries still want - PT_INTERP/RUNPATH alignment, identical to today's gcc kind. +- **The target side needs no fixup.** Host-run compiler binaries run on the + runtime the provider bound them to at install, as a hosted gcc does (§4). - **MCU flags, linker-script handling and a run story genuinely needed new design.** All three landed, and above this document's layer as predicted: ISA flags come from a one-row-per-target table in @@ -672,9 +630,9 @@ answer them. | spec → xim package, frontends | `src/toolchain/registry.cppm` | | detect/probe (triple, sysroot, payloads) | `src/toolchain/detect.cppm`, `probe.cppm` | | link model + loader resolution | `modules/toolchain-model/src/linkmodel.cppm` | -| unified fixup pipeline (patchelf/specs/cfg, marker) | `src/toolchain/post_install.cppm` | +| read-only payload facts (runtime kind, glibc lib dir, baked loader, link specs) | `src/toolchain/post_install.cppm` | | install/lifecycle entry | `src/toolchain/lifecycle.cppm`; auto-install entries in `src/build/prepare.cppm` | -| root runtime selection/binding | `src/xlings/runtime_selection.cppm`, `src/runtime/binding.cppm`, `src/xlings/subos_info.cppm` | +| root runtime selection/binding | `src/provider/runtime_selection.cppm`, `src/runtime/binding.cppm`, `src/provider/xlings/subos_info.cppm` | | generic runtime contract + LinkIntent | `modules/manifest/src/types.cppm`, `src/build/plan.cppm`, `src/build/flags.cppm` | | stored resolution explanation | `src/build/prepare.cppm`, `src/build/runtime_validation.cppm`, `src/doctor.cppm` | | flag assembly (main build) | `src/build/flags.cppm` | diff --git a/docs/README.md b/docs/README.md index f0e6d3ae9..5de8f3798 100644 --- a/docs/README.md +++ b/docs/README.md @@ -82,6 +82,7 @@ Within a part the order is a reading order, not an alphabet. - [22 — The Target Side](22-target-side.md) - [23 — The Project Environment](23-the-project-environment.md) - [24 — Cross-Compilation Over openkal](24-openkal-cross.md) +- [25 — The Home, Its Indices and Its Store](25-home-indices-and-store.md) ### 3x — Extending mcpp and its ecosystem diff --git a/docs/res/mcpp.exe.manifest b/docs/res/mcpp.exe.manifest index bddb2d96e..d6be854d6 100644 --- a/docs/res/mcpp.exe.manifest +++ b/docs/res/mcpp.exe.manifest @@ -3,6 +3,7 @@ UTF-8 + true diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 052b1d6d5..0ee48aec4 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -668,6 +668,7 @@ mcpp 2026.10.10.1,#785、#786;第 14 至 16 条与第 13 条的依赖位置为 m 3. 写进这类 DLL 所链接的包的 flag 中的 `/GL`,无论 LTO 是否生效,都在规划期被拒绝。 4. `mcpp pack` 产出的静态库**禁止**携带 LTO 中间表示;其对象不以 LTO 编译,构建报告一次。 5. `[test] windows_code_page` 为所发现的每个测试程序陈述 Windows 代码页,取值与 target 键相同。 +6. `windows_long_paths`(target 键与 `[test]` 键)陈述 PE 可执行文件是否声明 `longPathAware`;作为 host 工具构建的程序默认 `true`,其余默认不声明;写在库目标上被拒绝。 **状态:已实现(mcpp 2026.10.5.2,#770)。** diff --git a/docs/specs/package-identity.md b/docs/specs/package-identity.md index 64bb1a603..a03bc7546 100644 --- a/docs/specs/package-identity.md +++ b/docs/specs/package-identity.md @@ -5,8 +5,8 @@ | **规范编号** | SPEC-001 | | **标题** | 包身份(`package.namespace` / `package.name`)、`[dependencies]` 选择器与匹配机制 | | **状态** | **评审中(Review)** —— 已实现 | -| **版本** | 1.5 | -| **最后修改** | 2026-09-27 | +| **版本** | 1.6 | +| **最后修改** | 2026-10-16 | | **最低实现版本** | 描述符身份:mcpp **0.0.106**;精确 selector:mcpp **2026.8.10.1**(xlings >= 0.4.69) | | **作者/维护** | mcpp-community | | **相关设计文档** | `.agents/docs/2026-06-20-package-resolution-architecture.md` §4
`.agents/docs/2026-06-26-identity-first-resolution-no-filename.md`
`.agents/docs/2026-07-25-issue278-descriptor-name-form-canonicalization-design.md`
`.agents/docs/2026-07-25-name-namespace-bidirectional-verification-report.md`
`.agents/docs/2026-07-25-name-namespace-canonical-implementation-spec.md` | @@ -509,6 +509,16 @@ V2 规范。版本回答「装的是哪一个上游版本」,修订号回答「 - 修订号大于 0 的运行时载荷进入运行时契约(`revision=`),因此修订号变化使依赖该载荷的 产物重新链接;修订号为 0 时契约的文本不变。 +### 10.3 载荷的身份与位置(2026.10.16.1+) + +1. 载荷的身份是 `(namespace, name, version, revision)`。两个 home 中身份相同且完整的载荷可以互换:一个 + home **可以**以链接条目使用上游 home 中身份相同的载荷(见 [docs/25](../25-home-indices-and-store.md) §3); + 身份不同(含修订号不同)的载荷**禁止**互相代替。 +2. 载荷所在的 store 由服务它的索引决定:mcpp 所服务的索引的载荷在 `/xpkgs`(项目索引在 + `/.mcpp/xpkgs`),提供者自己的索引的载荷在提供者 home 的 store 中。 +3. 载荷的完整性由提供者回答(安装记录存在、无失败标记、修订号与索引条目一致、有内容)。mcpp **禁止**写入载荷, + 包括标记文件;一个载荷的路径按提供者的回答使用,**禁止**被规范化(链接条目的路径即其身份)。 + --- ## 11. 变更记录 @@ -516,6 +526,7 @@ V2 规范。版本回答「装的是哪一个上游版本」,修订号回答「 | 版本 | 日期 | 变更 | |---|---|---| | 1.5 | 2026-09-27 | 新增 §10.1:版本位的回答与 xlings 一致(安装记录优先,其次按 xlings 版本文法选择;已记录而缺失的载荷不算已安装),mcpp#712、#716;新增 §10.2:打包修订 `revision` 的含义与 mcpp 侧的三处使用(mcpp 2026.9.27.1,xlings 2026.9.27.1) | +| 1.6 | 2026-10-16 | 新增 §10.3:载荷的身份含修订号,身份相同的载荷可以经链接条目在 home 之间共用;载荷所在的 store 由服务它的索引决定;完整性由提供者回答,mcpp 不写入载荷(mcpp 2026.10.16.1,xlings 2026.10.16.1) | | 1.4 | 2026-09-14 | 新增 §5.4:`path` 与 `git` 依赖的身份取自其清单,键规范化到另一身份时采用清单声明并告警,同一来源上的两个身份在扫描前拒绝(mcpp 2026.9.14.2) | | 1.3 | 2026-09-07 | 新增 §10:同一条身份规则扩展到 xlings 工具地址(mcpp 2026.9.6.6)。此前工具侧有两套定义,同一个包被两处以不同版本声明时两份都装、只用一份 | | 1.2 | 2026-08-09 | selector 收敛为唯一精确 PackageId:裸名只表示默认 mcpplibs,dotted 以最后一段为 name;移除 compat/空 namespace 隐式候选,加入 lock 保持与一个 release train 的双 selector 迁移 warning | diff --git a/docs/specs/toolchain-management.md b/docs/specs/toolchain-management.md index a9d5f4558..e2c23fca4 100644 --- a/docs/specs/toolchain-management.md +++ b/docs/specs/toolchain-management.md @@ -4,8 +4,8 @@ |---|---| | 规范编号 | SPEC-006 | | 标题 | 工具链管理:身份、来源、选择与载荷契约 | -| 状态 | 草案 v0.7 | -| 最后修改 | 2026-10-09 | +| 状态 | 草案 v0.8 | +| 最后修改 | 2026-10-16 | | 对应实现 | 逐条标注;标为「已实现」的条款对应 mcpp >= 2026.9.24.1。标为「未实现」的条款计划与下一批 LLVM 工具链一同落地,届时按实测修订本规范 | | 相关设计文档 | `.agents/docs/2026-09-24-toolchain-selection-and-payload-trust-design.md`、`.agents/docs/2026-09-24-685-687-msvc-stl-and-toolchain-payloads.md`、`.agents/docs/2026-09-28-ecosystem-design-and-optimisation-plan.md`、`.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md` | | 相关 issue | mcpp#685、mcpp#687、mcpp#718、mcpp#755 | @@ -232,15 +232,14 @@ gcc 载荷的 `lib/gcc/<三元组>/<版本>/include-fixed/` **禁止**含带 fix 当前:gcc 13.3.0、15.1.0 与 11.5.0 的 x86_64-linux-gnu 发布归档含这类文件(mcpp#687)。索引中的 gcc 配方在安装时删除它们(openxlings/xim-pkgindex#870 之后生效); 在此之前安装的载荷由 `mcpp self doctor` 报告(§6.4)。故标为部分实现:安装后的载荷满足本条,发布归档不满足。 -### 4.3 安装时的改写 部分实现 +### 4.3 安装后不改写 已实现 | 要求 | 状态 | |---|---| -| 安装后的改写**必须**限于一张列出的清单:ELF 的 `PT_INTERP` 与 `RUNPATH`、clang 的 `.cfg` | 部分实现:改写集中在单一入口,清单未写成规范 | -| 改写**必须**由标记文件 `.mcpp-fixup.json` 记录 | 已实现 | -| 改写**必须**经副本与原子重命名完成 | 部分实现:mcpp 的安装后修正管线如此;xlings 安装时所做的改写未核对 | -| 清单之外的文件**必须**与发布归档一致 | 未实现:没有检查 | -| gcc 的 `specs` **禁止**在安装时改写 | 已实现 | +| 载荷**必须**只由提供者在安装时写入;此后**禁止**任何程序改写它,包括 mcpp(ELF 头、`specs`、clang 的 `.cfg`、标记文件) | 已实现(mcpp 2026.10.16.1;`check_provider_boundary.py`) | +| 工具链**必须**运行在提供者安装时为其绑定的运行库上;mcpp 经提供者 home 的运行库钉版本选定该运行库 | 已实现 | +| 产物的加载器、运行路径与 sysroot **必须**来自构建自己的命令行,不依赖载荷中的配置 | 已实现(gcc:`-specs=` 文件;clang:`--no-default-config`) | +| 早期 mcpp 改写过的载荷**可以**继续使用;其标记文件不再被读取 | 已实现 | ### 4.4 完整性 部分实现 @@ -330,3 +329,4 @@ xim-pkgindex 的准入脚本 `verify-toolchain.sh` 对一个载荷归档做一 | v0.4 | 2026-09-28 | 随 mcpp 2026.9.28.2:新增 §3.7.1,程序旁的文件由一个解析器决定;MSVC C++ 运行时是一个带版本的集合;契约决定种类;声明的运行时文件与 toolset 的版本比较;读不出的版本不作决定;action 的 `PATH` 首位是 toolset 的运行时目录(2026-09-28 设计 WS1)。 | | v0.6 | 2026-10-02 | §7 移入 SPEC-009(工具链的支持与维护):移动默认版本或 C 库绑定的顺序、门与撤销由 SPEC-009 §10 规定,本节改为引用。 | | v0.7 | 2026-10-09 | 随 mcpp 2026.10.8.1:新增 §3.8,使用受管 C 库的构建不隐式搜索宿主的系统头文件;需要时由项目显式写出。 | +| v0.8 | 2026-10-16 | 随 mcpp 2026.10.16.1:§4.3 改为「安装后不改写」,载荷只由提供者写入;工具链运行在提供者按 home 的运行库钉版本绑定的运行库上。 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 8e0a29b96..4605b5eac 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -422,6 +422,7 @@ required_features = ["gui"] # only built when feature `gui` is | `windows_subsystem` *（2026.9.12.2+）* | 可执行文件的 PE subsystem：`"console"`（默认）或 `"windows"`，一个启动时没有控制台的 GUI 程序。只到达这个目标的链接，别处不受影响，在非 PE 的目标上不渲染任何东西。见上一节。 | | `windows_entry` *（2026.9.12.2+）* | 程序定义的入口函数：`"main"`（默认）、`"wmain"`、`"WinMain"` 或 `"wWinMain"`。见上一节。 | | `windows_code_page` *（2026.9.26.1+）* | 可执行文件在 Windows 上的 ANSI 代码页：`"utf-8"`，以 Windows 10 1903 及以后版本会遵从的应用程序清单嵌入；或 `"legacy"`，即系统代码页。两者都不写的程序目标运行在系统代码页里，作为 host 工具构建的程序（§2.14）除外，它默认为 `"utf-8"`。写在库目标上会被拒绝；在非 PE 目标上不产生任何东西。见 §2.3 的*路径与文本编码*。 | +| `windows_long_paths` *（2026.10.16.1+）* | `true` 在可执行文件的应用程序清单中嵌入 `longPathAware`：在 Windows 启用长路径（`LongPathsEnabled`）时，进程可以使用超过 260 个字符的路径。`false` 不嵌入。两者都不写的程序目标不嵌入，作为 host 工具构建的程序（§2.14）除外，它默认为 `true`。写在库目标上会被拒绝；在非 PE 目标上不产生任何东西。 | | `linkage` *（2026.9.15.2+）* | 一个库目标的**默认**链接形态，`"static"` 或 `"shared"`：不写 `linkage` 的消费者得到的形态。与 `kind = "shared"` 不同，它不是约束，因此消费者的显式陈述会被遵从。与 `kind = "shared"` 同写，或写在程序目标上，都会被拒绝。见[`dependency_linkage`](#dependency_linkage--静态还是动态由消费者决定)。 | > **范围（重要）：** 目标上的 `defines` / `cxxflags` / `cflags` **只** @@ -1760,6 +1761,7 @@ discover = ["tests/**/*.cpp"] # the default |---|---|---| | `discover` | glob 数组 | 每个被某条 glob 匹配到的文件都是一个测试程序；以 `!` 开头的 glob 会移除它匹配到的文件；`[]` 不发现任何测试 | | `windows_code_page` | `"utf-8"` 或 `"legacy"` | Windows 上每个测试程序的 ANSI 代码页，含义与同名的 target 键相同；默认 `"legacy"`，即系统代码页 *(2026.10.5.2+)* | +| `windows_long_paths` | `true` 或 `false` | Windows 上每个测试程序是否支持长路径，含义与同名的 target 键相同；默认 `false` *(2026.10.16.1+)* | 这些 glob 使用与 `[build] sources` 相同的词汇。一个测试的名字，是它 相对第一条匹配到它的 glob 所在的固定目录的路径，去掉扩展名。两个 diff --git a/docs/zh/09-commands-by-scenario.md b/docs/zh/09-commands-by-scenario.md index c01f92b3b..c66437ce1 100644 --- a/docs/zh/09-commands-by-scenario.md +++ b/docs/zh/09-commands-by-scenario.md @@ -408,6 +408,24 @@ warning: pre-v1 cache at '/home/speak/.mcpp/bmi' occupies 167.5 MiB and is no lo `mcpp self config --mirror CN|GLOBAL` 选择下载镜像；mcpp 与 xlings 各自 持有这个设置，为其中一个选定，不会为另一个选定。 +## 复用另一个 home，以及机器的系统 home + +设置 `[xlings] upstream`（[25](25-home-indices-and-store.md) §3）后，安装以链接方式使用上游的 payload，而不是下载。 +此前已安装副本的 home 可以把它们换成链接： + +```bash +mcpp self store dedupe # list what an upstream has the same of +mcpp self store dedupe --apply # replace each by a link, atomically +``` + +管理员用 `--system` 维护机器的系统 home，只有维护类命令接受它（[25](25-home-indices-and-store.md) §4）： + +```bash +sudo mcpp --system self init +sudo mcpp --system toolchain install gcc 16.1.0 +sudo mcpp --system index update +``` + ## `[hooks]` —— 项目构建生命周期命令（实验性） > **实验性。** Hook 目前**不能**决定一次构建是否成功。每一次 Hook 失败 diff --git a/docs/zh/11-publishing-a-library.md b/docs/zh/11-publishing-a-library.md index b75d9f493..c12a89565 100644 --- a/docs/zh/11-publishing-a-library.md +++ b/docs/zh/11-publishing-a-library.md @@ -116,11 +116,10 @@ mcpp build # must download and compile 0.0.48 在上面这条链还没走完时，可以手工向 registry 播种，让消费方在库发布之前就能编译： ```bash -REG=~/.mcpp/registry/data/xpkgs/-x-/0.0.48 +REG=~/.mcpp/xpkgs/-x-/0.0.48 # the store of mcpp's indices (25 §1) mkdir -p "$REG" git -C /path/to/library archive --format=tar --prefix=-0.0.48/ HEAD \ | tar -x -C "$REG" -touch "$REG/.mcpp_ok" # the marker that says "resolved" cp ../0.0.47/.xpkg.lua "$REG/.xpkg.lua" # add a 0.0.48 entry to it ``` diff --git a/docs/zh/25-home-indices-and-store.md b/docs/zh/25-home-indices-and-store.md new file mode 100644 index 000000000..95de6b60e --- /dev/null +++ b/docs/zh/25-home-indices-and-store.md @@ -0,0 +1,123 @@ +# 25 —— home、索引与 store + +**读者：** 决定 mcpp 把安装内容放在哪里的人：已有 `~/.xlings` 的开发者、多人共用机器的管理员、Luban 系统。 + +**本章回答的那一个问题：** mcpp home 中有什么，各部分归哪个程序所有，一个 home 如何复用另一个 home 已有的包。 + +**不在这里：** 选择工具链见 [20 —— 工具链管理](20-toolchains.md)；项目自己的构建环境见 +[23 —— 项目环境](23-the-project-environment.md)。 + +mcpp 经由其包提供者 xlings 安装工具链与包。payload（一个包的一个已安装版本）只由提供者写入，此后不再被修改：mcpp +不修补、不标记、不复制它。构建需要而 payload 中没有的东西（加载器、运行路径、sysroot）由构建自己的命令行给出。 + +## 1. 布局 + +``` +~/.mcpp/ (MCPP_HOME) + config.toml + indices// mcpp 所服务的每个索引的数据（mcpplibs、[index.repos.]） + xpkgs/ 这些索引所服务的 payload + registry/ 提供者自己的 home：其索引（xim）、运行环境与 store + data/xpkgs/ 工具链与工具；条目可以是指向上游 payload 的链接 + build-cache/ cache/ provisioned/ … + +/.mcpp/ + indices// 项目自己的 [indices] 的数据 + xpkgs/ 它们所服务的 payload +``` + +归属按索引划分。mcpp 所服务的索引，其数据与 payload 在 mcpp 的 home 中；提供者自己的索引（`xim` 及其附带的索引）留在 +`registry/` 下。本地路径索引（`path = "…"`）在原处读取。 + +早期 mcpp 建立的 home 继续可用。其索引数据在第一次运行时移动到 `indices/`，只移动一次；已安装在 +`registry/data/xpkgs` 下的 payload 留在原处且仍能找到，基于它们构建的产物继续可以运行。早期 mcpp 写进 payload +的标记（`.mcpp_ok`、`.mcpp-fixup.json`）不再被读取。 + +## 2. 刷新索引 + +mcpp 所有的索引被拉取到它旁边的暂存目录，在那里按其声明的下限（`index.toml` 的 `min_mcpp`）检查，只有当前 mcpp +能读取时才与正在使用的副本交换。需要更新的 mcpp 的索引保留之前的副本，并在命令自身输出之后报告一次。 + +## 3. 复用另一个 home：上游 + +```toml +# ~/.mcpp/config.toml +[xlings] +upstream = ["~/.xlings"] +``` + +上游是本 home **以链接方式**从中安装的 home：上游中身份一致（命名空间、名字、版本与配方修订号）且完整的 payload， +以符号链接（Windows 上为 junction）的形式发布到本 home 的 store。不下载任何内容，上游永不被写入。在本 home +删除该包只删除链接。上游的 payload 消失后，下一次安装在同一路径上安装本 home 自己的 payload，基于该路径构建的产物继续可用； +`--offline` 下无法安装，命令失败。 + +| 上游 | 提供 | +|---|---| +| xlings home（`~/.xlings`、`/xlings`） | 提供者自己的包（`xim`） | +| mcpp home（另一个 `~/.mcpp`、系统 home） | 其 registry 中的 `xim` 包，以及其 `xpkgs` 中 mcpp 索引的包 | + +mcpp 自己的包只从 mcpp home 借用，从不从 xlings home 借用。 + +**信任。** 上游属于当前用户或系统（root；Windows 上还包括 Administrators 与 SYSTEM），不属于当前用户时，当前用户不能写入它。 +不满足的 home 在读取配置时被拒绝：使用另一个人的 home，等于运行那个人随时可以替换的程序。不存在的上游，或两种 home +都不是的目录，报告为降级（本次不复用）。 + +**读取位置。** 用户 `config.toml` 中的 `upstream`；没有时，系统级配置（`/etc/mcpp/config.toml`；Windows 上为 +`%ProgramData%\mcpp\config.toml`）；仍没有时为空，即默认的自包含 home。 + +**已有副本的 home。** `mcpp self store dedupe` 列出本 home 自己安装、且上游有相同副本的 payload； +`mcpp self store dedupe --apply` 将其逐个原子地替换为链接。 + +## 4. 机器的系统 home + +在多个用户共用的机器上，管理员为所有用户维护一个 home： + +```bash +sudo mcpp --system self init +sudo mcpp --system toolchain install gcc 16.1.0 +``` + +| | | +|---|---| +| 位置 | 系统级配置的 `[system] home`；否则为 `/var/lib/mcpp`（macOS 为 `/Library/Application Support/mcpp`，Windows 为 `%ProgramData%\mcpp`） | +| 写入方式 | `mcpp --system` 加 `self init`、`self doctor`、`self store`、`toolchain install`、`toolchain remove`、`toolchain default`、`index update` 之一 | +| 拒绝 | `--system` 与其他任何命令（以 root 构建会把 root 所有的文件写进用户的项目）；对系统 home 没有写权限的 `--system`（消息会提示 `sudo`） | +| 写入约定 | 所有用户可读（umask 022），同一时间只有一个写入者，忽略调用者的 `HOME` 与 `MCPP_HOME` | +| 标记 | `.mcpp-home`，其中 `mode = "system"` | + +用户无需 root 即可读取。系统级配置把它写为用户的上游，并可以给出默认工具链： + +```toml +# /etc/mcpp/config.toml +[xlings] +upstream = ["/var/lib/mcpp"] + +[toolchain] +default = "gcc@16.1.0" +default_target = "x86_64-linux-gnu" +``` + +每个用户保留自己的 home：自己的索引数据、自己的构建缓存（不在用户之间共享）以及指向系统 home 中 payload +的链接。用户的索引解析出更新的版本时，安装到用户自己的 home。 + +在 Luban OS 上，系统级配置同时写出 `/xlings`，因此机器上已有的 payload 只存在一份。 + +## 5. 提供者程序 + +`[xlings] binary` 选择 mcpp 运行的 xlings：`"bundled"`（mcpp 随附的，默认）、`"system"`（`PATH` 上的）或一个路径。 +无论哪一个，mcpp 都会询问它是什么（`xlings interface --version`）：版本、协议以及 mcpp 依赖的特性。不能服务当前 mcpp +的，让位于随附的 xlings，并以一条 note 说明原因。 + +`[xlings] home` 移动 registry，它指向的必须是 mcpp 建立的 home。不是 mcpp 建立的 home（例如 `~/.xlings`）在这里被拒绝： +复用它的包应使用 `upstream`（§3）。 + +## 6. 文件系统与长路径 + +home 可以在任何文件系统上。mcpp 测量其所在文件系统能做什么（拒绝覆盖已有名字的原子改名、原子交换、硬链接、符号链接）， +并按测量结果发布。`mcpp self doctor` 显示结果，以及那里能否从上游复用（需要符号链接）。 + +Windows 上路径长度限制为 260 个字符，除非系统启用长路径（`LongPathsEnabled` = 1）。`mcpp.exe` 声明自己支持长路径。 +长路径关闭且 `MCPP_HOME` 长到工具链最深的文件会超过限制时，mcpp 报告降级并给出两种办法：启用长路径，或使用更短的 +`MCPP_HOME`。mcpp 不修改该设置；`mcpp self doctor` 报告它。 + +程序自己的清单陈述它是否支持长路径：`windows_long_paths`（[04](04-mcpp-toml.md)）。 diff --git a/docs/zh/91-toolchain-internals.md b/docs/zh/91-toolchain-internals.md index 5657c5ef5..4e47bb24d 100644 --- a/docs/zh/91-toolchain-internals.md +++ b/docs/zh/91-toolchain-internals.md @@ -23,15 +23,14 @@ resolve runtime binding ← which libc the artifact will load (§2.1) ▼ resolve toolchain payload ← project/default/install paths share one pipeline ▼ -ensure_post_install_fixup() ← exact glibc@version, marker-gated; never readdir-first +ensure_declared_runtime() ← exact glibc@version installed; the payload is never written ▼ detect / probe ← triple, sysroot, payload paths (glibc, linux-headers) ▼ ToolchainLinkModel (single resolver for the C-library axis) ├──► flags.cppm (main build compile/link flags) ├──► stdmod.cppm (`import std;` BMI precompile) - ├──► build_program (build.mcpp host compiles) - └──► cfg regeneration (the human-facing clang++.cfg) + └──► build_program (build.mcpp host compiles) ▼ hermetic link check (`-###` dry-run) ← checks sandbox CRT/loader resolution ▼ @@ -49,7 +48,7 @@ link → internal ELF physics check ← validates the artifact and resolved c 正常工作。 2. **每一层的路径知识只有一个属主。** 过去「如何链接 payload glibc」有四份 彼此漂移的副本，现在收敛为一个解析器（`linkmodel`）；过去按入口路径各自 - 维护的 fixup 行为，现在是一条管线。副本之间的漂移正是一整类 bug 的来源 + 维护的 fixup 行为已经删除，payload 只归提供者所有（§4）。副本之间的漂移正是一整类 bug 的来源 （issue #195）。 ## 2. 工具链解析 @@ -286,63 +285,28 @@ CLibMode::None nothing usable — host defaults apply; the hermetic 库」的需求，它必须以 xlings 自身作为第一消费者来设计，届时 mcpp 可以再 加回一个读取端。 -## 4. 统一的 post-install fixup 管线（`src/toolchain/post_install.cppm`） +## 4. payload 从不被写入（`src/toolchain/post_install.cppm`） -沙箱 payload 是预先构建好的 ELF 树，其中烙入的 `PT_INTERP`/`RUNPATH` 在 -打包期无从得知，必须对齐到**本机**沙箱。 -`ensure_post_install_fixup(cfg, payloadRoot, pkg)` 是完成这次对齐的 -**唯一入口**，由三条入口路径共同调用：显式 install、默认 auto-install、 -manifest auto-install。 +payload 由提供者（xlings）在安装时写入，此后没有任何程序写它。mcpp 只读取 payload：不修补其 ELF 头，不改写其 `specs` +或 clang 的 `.cfg`，不往其中写标记文件，也不从另一个 home 复制它。`.github/tools/check_provider_boundary.py` +保证提供者适配器之外的代码不写出 store 的布局，也不删除 payload。 -> 这条管线过去也会重写 GCC 的 `specs` 文件，以便产物获得 loader 与 -> rpath。现在不再这样做，原因见 §5。 +**工具链运行在什么之上。** Linux 上预构建的 gcc 或 llvm 是一棵 ELF 树，其 `PT_INTERP` 与 `RUNPATH` 必须指向一份 C +运行库。提供者在安装工具链时经由其 home 的运行库钉版本完成绑定：mcpp 在第一次安装工具链之前，把 registry home 的钉版本设为它选定的运行库 +（`glibc@2.44.3`，`xlings config --runtime`，只设一次），提供者把工具链链接到该运行库的加载器与库。从上游 home +借来的工具链运行在那个 home 的运行库上；它的产物不变，因为产物的加载器、运行路径与 sysroot 来自构建自己的命令行（§5）。 -> 历史注记：0.0.83 之前，各条路径各自记得 —— 或者忘记 —— 自己那一部分。 -> manifest 路径**什么都不跑**，这正是一个刚刚自动安装的 llvm 会保留一份 -> 陈旧、随装机环境漂移的 cfg 的原因（issue #195），也是 gcc 曾经产出一个 -> 找不到 `stdlib.h` 的沙箱的原因。「用哪条命令装的」绝不能决定「这条工具 -> 链好不好用」。 +**mcpp 在这里仍做的事**（`post_install.cppm`，全部只读）： -**触发语义 —— 每次构建都询问，只执行一次：** - -``` -every build → ensure() → read /.mcpp-fixup.json - marker == {schema, kind, rev, glibcLib}? → return (ms-level) - mismatch → run the fixup for this kind, write marker -``` - -marker 是一份**内容指纹化的缓存**，不是事件标志：它编码了 fixup 的版本， -以及它对齐时所针对的那个 glibc payload。因此「执行」这一分支在每一个 -`(payload × fixup-rev × glibc 指纹)` 组合上恰好触发一次 —— 首次使用，外加 -两类确实需要重写的重新收敛事件（经 `kFixupRev` 升级的 fixup 逻辑，或者 -底层的 glibc payload 发生变化）。mcpp 之所以每次构建都要问一遍，是因为 -使 payload 失效的那些事件（xlings 换掉 glibc、payload 从另一个 home 继承 -而来）发生在 mcpp 的视野之外 —— trust-but-verify 是唯一可靠的语义。 - -**按 kind 划分的动作：** - -| kind | actions | +| 函数 | 回答 | |---|---| -| `gcc`（glibc） | 对 gcc payload **以及共享的 binutils payload** 做一次 patchelf 遍历（PT_INTERP → 沙箱 loader，RUNPATH → glibc + gcc 的 lib 目录）—— 目的是让 *gcc 自己* 能跑起来。不会往 `specs` 里写入任何东西 | -| `llvm` | 只对 `lib/` 做 patchelf 遍历（运行期 `.so` 的 RUNPATH；`bin/` 不动，以保留 xlings 设置的 RUNPATH）；确定性地重新生成 cfg（见 §5） | -| `musl-gcc` | 无操作 —— 自包含 sysroot，静态世界 | - -**安全不变量**（每一条都由一次真实事故换来）： - -- **绝不就地打补丁。** patchelf 作用于一份副本，随后原子地 `rename()` - 换入：payload 中可能有库被**当前进程**（一个自托管、动态链接的 mcpp） - 或某个并发构建 mmap 着，就地重写一个活映射的 backing file 会损坏正在 - 运行的进程（实测现象：退出时 `_dl_fini` 中的 SIGSEGV）。`rename` 让新 - 内容获得一个新 inode，活跃进程仍持有旧的那一个。 -- **所有权护栏。** 解析到本 home registry 之外的 payload（通过 symlink - 从另一个 `MCPP_HOME` 继承而来）绝不会被打补丁 —— 它们的属主已经完成 - 过收敛，隔着 symlink 打补丁会毁掉属主自己的工具链。 -- 把内容感知能力扩展到 patchelf 遍历本身（写入前先比对 - `--print-interpreter`/`--print-rpath`，使一个已经对齐的 payload 能以 - **零写入**收敛）是一项已知的后续工作。 -- 长期方向是让**安装器**（xlings）拥有全部写入 —— 无论是在安装时，还是 - 在 payload 进入一个新 home 时 —— 让 mcpp 退化为只读加校验。在那之前， - 这里的管线是兼容层，也是应对双向漂移的自愈机制。 +| `runtime_payload_kind` | 工具链是否运行在 C 运行库 payload 上：Linux 上的 `gcc`、`llvm`；musl-gcc、系统编译器与 PE 工具链为否 | +| `select_glibc_payload_lib` | 恰好 `glibc@<版本>` 的库目录（绝不取另一个已安装版本） | +| `detect_baked_loader`、`baked_runtime_binding` | payload 绑定的加载器与运行库，读自其 ELF 头 | +| `write_clean_link_specs` | 构建目录中每次构建的 `-specs=` 文件（§5） | + +早期 mcpp 写过的 home 保留其被修补过的 payload 与写入的标记（`.mcpp_ok`、`.mcpp-fixup.json`）；它们仍可运行， +标记不再被读取。 ## 5. 编译器是一种能力，不是一份配置 @@ -377,14 +341,8 @@ payload 自身的主张被覆盖，而 payload 本身不受触碰。这带来两 ### 5.1 clang 的 cfg `bin/clang++.cfg` 的存在，是为了让直接调用打包内 `clang++`（不经由 -mcpp）的场景，也能得到一份可用、hermetic 的编译器配置。fixup 管线从 -链接模型**确定性地重新生成**它 —— 同一个 payload 在任何机器、任何安装 -路径上都产出字节相同的 cfg —— 而不是对某次安装产出的文件做行级修补。在 -Linux 上，它包含 CRT 发现（`-B`）、payload loader 加 rpath、 -lld/compiler-rt/libunwind，以及供 C++ 驱动使用的 bundled libc++；在 -macOS 上，它保留历史形态（`--sysroot=` 加 payload 的 libc++ 头 -文件 —— C++ **运行时链接**仍由主构建中平台专属的 -`needs_explicit_libcxx` 处理）。 +mcpp）的场景，也能得到一份可用、hermetic 的编译器配置。llvm 配方的安装 hook +为它所安装进的 home 写出这个文件；mcpp 不读取它（`--no-default-config`），也不写它。 ## 6. hermetic 链接检查（`src/build/hermetic.cppm`） @@ -445,20 +403,14 @@ ELF/glibc 规则。 执行。`mcpp self doctor` 报告的是同一份存档判定，而不是针对可能已经 变化的当前宿主重新探测一次。 -装后对齐遵循同一条身份规则：`glibc@2.44` 只解析 -`/xim-x-glibc/2.44/{lib64,lib}`。精确 payload 缺失或陈旧是一个 -错误；另一个已安装的版本绝不会被当作回退。 - -精确的 payload 由 mcpp 在第一次需要针对它打补丁的 fixup 之前安装 -（`ensure_declared_runtime`，mcpp#660）。这一步只在以下条件同时成立时 -执行：binding 找不到 payload、宿主是 Linux、provider 是 `glibc`，且该 -工具链的 fixup 需要消费一份 C 运行时（`gcc` 与 `llvm` 的 payload）—— -因此使用系统、musl 或 PE 工具链的构建不会下载任何东西。它通过 xlings -安装 `xim:glibc@<声明的版本>`，然后重新解析 binding。在这一步存在之 -前，payload 只有在 xlings 把它当作某个工具链的依赖装上时才会出现；从 -CI 缓存恢复的工具链不会被重新安装，因此当缓存中是另一个 glibc 修订版 -时，声明的 payload 就会缺席。当安装不可能完成时（例如处于离线状态）， -fixup 的报错会写出需要预先提供的坐标 `xim:glibc@<版本>`。 +运行库遵循同一条身份规则：`glibc@2.44` 只解析 +`/xim-x-glibc/2.44/{lib64,lib}`。精确 payload 缺失时被安装，绝不以另一个已安装的版本代替。 + +mcpp 在第一个运行于它之上的工具链被安装或使用之前安装它（`ensure_declared_runtime`，mcpp#660）：只在 binding +找不到 payload、宿主是 Linux、provider 是 `glibc`，且 `runtime_payload_kind` 指出该工具链时执行，因此使用系统、musl +或 PE 工具链的构建不会下载任何东西。它把 registry home 的运行库钉到声明的版本，经提供者安装 +`xim:glibc@<声明的版本>`，然后重新解析 binding。安装不可能完成时（例如离线），报错写出需要预先提供的坐标 +`xim:glibc@<版本>`。 ### 6.2 运行时搜索路径的允许位置（`runtime_env_contract.cppm`） @@ -506,10 +458,8 @@ loader，任何环境变量都无法覆盖它，因此一个 `popen()`/`system() 驱动）。若有旧拼法，一律只放进 `compat.cppm`。 3. **能力**（`src/toolchain/provider.cppm`）：stdlib 身份、BMI 特性， 以及 `flags.cppm` 消费的 feature 开关。 -4. **fixup kind**（`post_install.cppm`）：确定这个 payload 需要哪种 - 装后对齐 —— gcc 式（patchelf + specs）、llvm 式（lib patchelf + - cfg），或者不需要（自包含）。把它接入 `ensure_post_install_fixup` - 的分发逻辑。 +4. **运行库类别**（`post_install.cppm`）：payload 是否运行在 C 运行库 payload 上 + （`runtime_payload_kind`）；提供者在安装时把它绑定到钉住的运行库，mcpp 不往其中写入任何东西。 5. **e2e**：按 `86_llvm_hermetic_link.sh` 的思路写一个 hermetic 链接 测试，并纳入无宿主工具链的 CI job。 @@ -523,8 +473,8 @@ loader，任何环境变量都无法覆盖它，因此一个 `popen()`/`system() 2. 为该架构发布 payload 资产（glibc、linux-headers、工具链本体）—— aarch64-linux-musl 交叉目标是现成的先例（`[target.aarch64-linux-musl]`， 交叉前端经由 spec 的 `targetTriple` 解析）； -3. 其余什么都不用做：`-B`/`-L`/loader 的发出、fixup 管线，以及 - hermetic 检查全都与具体名字无关。 +3. 其余什么都不用做：`-B`/`-L`/loader 的发出与 hermetic 检查都与具体名字无关， + 提供者把工具链绑定到它的运行库。 ### 7.3 嵌入式与裸机工具链 @@ -536,8 +486,8 @@ loader，任何环境变量都无法覆盖它，因此一个 `popen()`/`system() - **没有动态链接器。** `loader` 保持为空，每个渲染器本来就允许这一点； 部署方式是烧录，而不是 ELF interp。 -- **目标侧不需要 fixup。** 宿主上运行的编译器二进制仍然需要 - PT_INTERP/RUNPATH 对齐，与今天的 gcc kind 完全相同。 +- **目标侧不需要 fixup。** 宿主上运行的编译器二进制运行在提供者安装时为其绑定的运行库上， + 与 hosted gcc 相同（§4）。 - **MCU flag、linker script 处理与运行方式确实需要全新设计。** 三者均 已落地，而且都如预测那样位于本文档所述层次之上：ISA flag 来自 `src/freestanding/target.cppm` 中每个目标一行的表，linker script 经 @@ -625,9 +575,9 @@ dialect、产出 MSVC-ABI 的对象、生成 PE 镜像。问错轴的后果是 | spec → xim 包、前端 | `src/toolchain/registry.cppm` | | detect/probe（triple、sysroot、payload） | `src/toolchain/detect.cppm`、`probe.cppm` | | 链接模型 + loader 解析 | `modules/toolchain-model/src/linkmodel.cppm` | -| 统一 fixup 管线（patchelf/specs/cfg、marker） | `src/toolchain/post_install.cppm` | +| payload 的只读事实（运行库类别、glibc 库目录、烙入的加载器、链接 specs） | `src/toolchain/post_install.cppm` | | install/lifecycle 入口 | `src/toolchain/lifecycle.cppm`；auto-install 入口在 `src/build/prepare.cppm` | -| root runtime 选择 / binding | `src/xlings/runtime_selection.cppm`、`src/runtime/binding.cppm`、`src/xlings/subos_info.cppm` | +| root runtime 选择 / binding | `src/provider/runtime_selection.cppm`、`src/runtime/binding.cppm`、`src/provider/xlings/subos_info.cppm` | | 通用 runtime contract + LinkIntent | `modules/manifest/src/types.cppm`、`src/build/plan.cppm`、`src/build/flags.cppm` | | 已存储的 resolution 解释 | `src/build/prepare.cppm`、`src/build/runtime_validation.cppm`、`src/doctor.cppm` | | flag 组装（主构建） | `src/build/flags.cppm` | diff --git a/docs/zh/README.md b/docs/zh/README.md index 2504af6a7..74cf6f5be 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -78,6 +78,7 @@ - [22 —— 目标侧](22-target-side.md) - [23 —— 项目环境](23-the-project-environment.md) - [24 —— 基于 openkal 的交叉构建](24-openkal-cross.md) +- [25 —— home、索引与 store](25-home-indices-and-store.md) ### 3x —— 扩展 mcpp 与它的生态 diff --git a/mcpp.toml b/mcpp.toml index 707abb724..0006b4c74 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.10.11.1" +version = "2026.10.16.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 538c4f6b1..b5fe00296 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -345,7 +345,7 @@ inline XlingsEntry parse_address(std::string_view address) { } // The identity an `[xlings.overrides]` key names: `:`, the namespace -// defaulted to `xim` -- the identity mcpp.xlings.address_set gives a package, +// defaulted to `xim` -- the identity mcpp.provider.address_set gives a package, // restated here because the manifest reader cannot import it. inline std::string override_package_key(std::string_view key) { const auto e = parse_address(key); @@ -1991,6 +1991,12 @@ std::expected parse_document(const t::Document& documen tname, v, list))); t.windowsCodePage = v; } + if (auto it = tt.find("windows_long_paths"); it != tt.end()) { + if (!it->second.is_bool()) + return std::unexpected(error(origin, std::format( + "targets.{}.windows_long_paths must be true or false", tname))); + t.windowsLongPaths = it->second.as_bool(); + } // An executable's property. A library has no subsystem, and a GUI // subsystem on anything a test runner executes is the defect #618 // describes, so both are refused naming the key. `app` is accepted @@ -1999,14 +2005,15 @@ std::expected parse_document(const t::Document& documen // `is_program()` is "is this the program", which is what the PE // subsystem attaches to; `TestBinary` stays refused on purpose. if ((!t.windowsSubsystem.empty() || !t.windowsEntry.empty() - || !t.windowsCodePage.empty()) + || !t.windowsCodePage.empty() || t.windowsLongPaths) && !t.is_program()) return std::unexpected(error(origin, std::format( "targets.{}.{} applies to an executable (`kind = \"bin\"` or " "`\"app\"`), and this target is not one", tname, !t.windowsSubsystem.empty() ? "windows_subsystem" : !t.windowsEntry.empty() ? "windows_entry" - : "windows_code_page"))); + : !t.windowsCodePage.empty() ? "windows_code_page" + : "windows_long_paths"))); // Guard: -std=... belongs to [package].standard, not per-target flags // (same rule as [build].cxxflags). Reject early with a clear message. for (auto const& flag : t.cxxflags) { @@ -2025,7 +2032,7 @@ std::expected parse_document(const t::Document& documen static constexpr std::string_view kKnownTargetKeys[] = { "kind", "linkage", "main", "soname", "exports", "windows_auto_export", "cflags", "cxxflags", "defines", "required_features", - "windows_entry", "windows_subsystem", "windows_code_page", + "windows_entry", "windows_subsystem", "windows_code_page", "windows_long_paths", }; for (auto& [key, _] : tt) { bool known = false; @@ -3310,10 +3317,17 @@ std::expected parse_document(const t::Document& documen m.testWindowsCodePage = v; continue; } + if (key == "windows_long_paths") { + if (!value.is_bool()) + return std::unexpected(error(origin, + "[test] windows_long_paths must be true or false")); + m.testWindowsLongPaths = value.as_bool(); + continue; + } if (key != "discover") { m.schemaWarnings.push_back(std::format( "[test] has unsupported key '{}' (ignored). Supported keys: " - "discover, windows_code_page.", key)); + "discover, windows_code_page, windows_long_paths.", key)); continue; } bool ok = value.is_array(); diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 26f070098..bf8f79798 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -236,6 +236,14 @@ struct Target { // mcpp's UTF-8 paths on its command line). Inert on every object format // that is not PE. std::string windowsCodePage; + // `windows_long_paths` (design 2026-10-16 §11): true embeds + // `longPathAware` in this PE executable's application manifest, so the + // process may use paths past MAX_PATH where the system allows it + // (LongPathsEnabled); false embeds none. Unset = the default of the build + // role: true for a target built as a host tool, nothing for an ordinary + // one (whether a program handles long paths is its author's statement). + // Inert on every object format that is not PE. + std::optional windowsLongPaths; // Where `kind` was stated, as the manifest line that states it: // `[targets.fw] kind = "shared"`, or, when a row states it, // `[target.'cfg(os = "android")'.targets.fw] kind = "shared"`. Read by @@ -2051,6 +2059,8 @@ struct Manifest { // program on Windows, in the vocabulary of the target key of the same // name. Empty is `legacy`, the system's code page. std::string testWindowsCodePage; + // `[test] windows_long_paths`: the same for every discovered test program. + std::optional testWindowsLongPaths; // Set once a workspace root's own package has received `[workspace.package]` // and `[workspace.build]` (`mcpp::project::inherit_as_root_package`). The // manifest is read on two paths before it is planned, and an appended diff --git a/modules/platform/src/fs_caps.cppm b/modules/platform/src/fs_caps.cppm new file mode 100644 index 000000000..82953dc67 --- /dev/null +++ b/modules/platform/src/fs_caps.cppm @@ -0,0 +1,378 @@ +// What a filesystem does, measured where it is, not assumed from the +// platform (mcpp-community/mcpp#809; the same table as xlings' +// `platform::fs_caps`, openxlings/xlings#671). An MCPP_HOME and a project +// land on whatever the person has: ext4, APFS, NTFS -- and NFSv3, SMB, FAT, a +// FUSE mount, WSL's drvfs. NFSv3 answers renameat2(RENAME_NOREPLACE) with +// EINVAL: `mcpp new` failed there, and so would every publication that asked +// the flag. Each primitive below asks this table, once per filesystem, and +// takes the strategy that filesystem allows; none guesses from an errno. + +module; + +#if defined(_WIN32) +# ifndef NOMINMAX +# define NOMINMAX +# endif +# include +# include +# include +// GetNamedSecurityInfoW and the token calls live in advapi32. +# pragma comment(lib, "advapi32") +#elif defined(__linux__) +# include +# include +# include +# include +# include +# include +#elif defined(__APPLE__) +# include +# include +# include +# include +# include +# include +#else +# include +# include +#endif + +export module mcpp.platform.fs_caps; + +import std; + +export namespace mcpp::platform { + +struct FsCaps { + std::string type; // "ext4", "nfs", "apfs", "NTFS", ... ("" unknown) + bool rename_no_replace { true }; // a rename that refuses an existing name, atomically + bool rename_exchange { true }; // two entries swapped atomically + bool hard_links { true }; + bool symlinks { true }; + bool case_sensitive { true }; + bool measured { false }; // false: the probe could not run; the platform's native answers + + // Whether every operation mcpp asks for is the filesystem's own. + bool native() const { return rename_no_replace && rename_exchange && hard_links && symlinks; } +}; + +// The filesystem holding `dir` (an existing directory), measured once per +// filesystem per process: a scratch entry in `dir`, removed after. +FsCaps fs_caps(const std::filesystem::path& dir); + +// Measured now, not cached (tests, `mcpp self doctor`). +FsCaps probe_fs_caps(const std::filesystem::path& dir); + +// Publish `from` (a scratch on the same filesystem) under `to`, never over an +// existing entry, by what `caps` says the filesystem does: +// a native rename that refuses replacement; +// else a hard link for a file and the same link again for a symbolic link +// (each refused as atomically when the name exists), the scratch name then +// removed; +// else, for a directory, a rename onto a name that is not there -- the one +// window, a name made there in between, is the caller's own under its lock. +std::expected publish_no_replace(const std::filesystem::path& from, + const std::filesystem::path& to, + const FsCaps& caps); + +// The same, with the caps of `to`'s directory. +std::expected rename_no_replace(const std::filesystem::path& from, + const std::filesystem::path& to); + +// `a` and `b` (both present, on one filesystem) trade places: atomically where +// the filesystem exchanges; else `b` is moved aside, `a` renamed onto `b`'s +// name and the old `b` put at `a` -- a reader of `b` meets the name missing +// for that moment, never a half of either. +std::expected exchange_paths(const std::filesystem::path& a, + const std::filesystem::path& b); + +// Whether a directory another program wrote may be read and run from: it +// belongs to this user or to the system (root; on Windows also Administrators +// and SYSTEM), and when it is not this user's, this user cannot write it. +// Empty: trusted; else why not. Using another person's home is running +// what that person can replace at any time. +std::string untrusted_because(const std::filesystem::path& dir); + +// `link` made a link to the directory `target`: a symbolic link; on Windows, +// where one needs a privilege a user rarely has, a junction. +bool create_directory_link(const std::filesystem::path& target, const std::filesystem::path& link); + +} // namespace mcpp::platform + +namespace mcpp::platform::native { + +#if defined(_WIN32) +// MoveFileExW without MOVEFILE_REPLACE_EXISTING refuses an existing name on +// every Windows filesystem (NTFS, ReFS, FAT, SMB). +int rename_no_replace(const std::filesystem::path& from, const std::filesystem::path& to) { + return ::MoveFileExW(from.wstring().c_str(), to.wstring().c_str(), MOVEFILE_WRITE_THROUGH) + ? 0 : static_cast(::GetLastError()); +} +// Windows has no atomic exchange of two names. +int exchange(const std::filesystem::path&, const std::filesystem::path&) { return ERROR_NOT_SUPPORTED; } +bool unsupported(int error) { return error == ERROR_NOT_SUPPORTED || error == ERROR_INVALID_FUNCTION; } +std::error_category const& category() { return std::system_category(); } +std::uint64_t device_of(const std::filesystem::path& dir) { + wchar_t root[MAX_PATH] {}; + DWORD serial = 0; + if (!::GetVolumePathNameW(dir.wstring().c_str(), root, MAX_PATH)) return 0; + if (!::GetVolumeInformationW(root, nullptr, 0, &serial, nullptr, nullptr, nullptr, 0)) return 0; + return serial; +} +std::string owner_problem(const std::filesystem::path& dir) { + PSID owner = nullptr; + PSECURITY_DESCRIPTOR sd = nullptr; + if (::GetNamedSecurityInfoW(dir.wstring().c_str(), SE_FILE_OBJECT, OWNER_SECURITY_INFORMATION, + &owner, nullptr, nullptr, nullptr, &sd) != ERROR_SUCCESS) + return "its owner cannot be read"; + bool trusted = ::IsWellKnownSid(owner, WinLocalSystemSid) || ::IsWellKnownSid(owner, WinBuiltinAdministratorsSid); + HANDLE token = nullptr; + if (!trusted && ::OpenProcessToken(::GetCurrentProcess(), TOKEN_QUERY, &token)) { + DWORD size = 0; + ::GetTokenInformation(token, TokenUser, nullptr, 0, &size); + std::vector buffer(size); + if (::GetTokenInformation(token, TokenUser, buffer.data(), size, &size)) + trusted = ::EqualSid(owner, reinterpret_cast(buffer.data())->User.Sid); + ::CloseHandle(token); + } + ::LocalFree(sd); + return trusted ? std::string{} : "it belongs to another user"; +} +std::string filesystem_type(const std::filesystem::path& dir) { + wchar_t root[MAX_PATH] {}, name[MAX_PATH] {}; + if (!::GetVolumePathNameW(dir.wstring().c_str(), root, MAX_PATH)) return {}; + if (!::GetVolumeInformationW(root, nullptr, 0, nullptr, nullptr, nullptr, name, MAX_PATH)) return {}; + return std::filesystem::path(name).string(); +} +#elif defined(__linux__) +int rename_no_replace(const std::filesystem::path& from, const std::filesystem::path& to) { + constexpr unsigned kNoReplace = 1; // renameat2 RENAME_NOREPLACE + return ::syscall(SYS_renameat2, AT_FDCWD, from.c_str(), AT_FDCWD, to.c_str(), kNoReplace) == 0 ? 0 : errno; +} +int exchange(const std::filesystem::path& a, const std::filesystem::path& b) { + constexpr unsigned kExchange = 2; // renameat2 RENAME_EXCHANGE + return ::syscall(SYS_renameat2, AT_FDCWD, a.c_str(), AT_FDCWD, b.c_str(), kExchange) == 0 ? 0 : errno; +} +bool unsupported(int error) { return error == EINVAL || error == ENOSYS || error == EOPNOTSUPP; } +std::error_category const& category() { return std::generic_category(); } +std::string owner_problem(const std::filesystem::path& dir) { + struct stat st {}; + if (::stat(dir.c_str(), &st) != 0) return "it cannot be read"; + if (st.st_uid == ::getuid()) return {}; + if (st.st_uid != 0) return "it belongs to another user"; + if (::access(dir.c_str(), W_OK) == 0) return "it belongs to root and this user can write it"; + return {}; +} + +std::uint64_t device_of(const std::filesystem::path& dir) { + struct stat st {}; + return ::stat(dir.c_str(), &st) == 0 ? static_cast(st.st_dev) : 0; +} +std::string filesystem_type(const std::filesystem::path& dir) { + struct statfs st {}; + if (::statfs(dir.c_str(), &st) != 0) return {}; + switch (static_cast(st.f_type)) { + case 0xEF53: return "ext4"; + case 0x58465342: return "xfs"; + case 0x9123683E: return "btrfs"; + case 0x01021994: return "tmpfs"; + case 0x794C7630: return "overlay"; + case 0x6969: return "nfs"; + case 0xFF534D42: return "cifs"; + case 0xFE534D42: return "smb2"; + case 0x4D44: return "vfat"; + case 0x2011BAB0: return "exfat"; + case 0x7366746E: return "ntfs3"; + case 0x5346544E: return "ntfs"; + case 0x65735546: return "fuse"; + case 0x01021997: return "9p"; + case 0x2FC12FC1: return "zfs"; + case 0x00C36400: return "ceph"; + default: return std::format("0x{:x}", static_cast(st.f_type)); + } +} +#elif defined(__APPLE__) +int rename_no_replace(const std::filesystem::path& from, const std::filesystem::path& to) { + return ::renamex_np(from.c_str(), to.c_str(), RENAME_EXCL) == 0 ? 0 : errno; +} +int exchange(const std::filesystem::path& a, const std::filesystem::path& b) { + return ::renamex_np(a.c_str(), b.c_str(), RENAME_SWAP) == 0 ? 0 : errno; +} +bool unsupported(int error) { return error == EINVAL || error == ENOTSUP || error == EOPNOTSUPP || error == ENOSYS; } +std::error_category const& category() { return std::generic_category(); } +std::string owner_problem(const std::filesystem::path& dir) { + struct stat st {}; + if (::stat(dir.c_str(), &st) != 0) return "it cannot be read"; + if (st.st_uid == ::getuid()) return {}; + if (st.st_uid != 0) return "it belongs to another user"; + if (::access(dir.c_str(), W_OK) == 0) return "it belongs to root and this user can write it"; + return {}; +} + +std::uint64_t device_of(const std::filesystem::path& dir) { + struct stat st {}; + return ::stat(dir.c_str(), &st) == 0 ? static_cast(st.st_dev) : 0; +} +std::string filesystem_type(const std::filesystem::path& dir) { + struct statfs st {}; + return ::statfs(dir.c_str(), &st) == 0 ? std::string(st.f_fstypename) : std::string{}; +} +#else +// No native primitive: the strategies below that need none. +std::string owner_problem(const std::filesystem::path&) { return {}; } +int rename_no_replace(const std::filesystem::path&, const std::filesystem::path&) { return ENOSYS; } +int exchange(const std::filesystem::path&, const std::filesystem::path&) { return ENOSYS; } +bool unsupported(int error) { return error == ENOSYS || error == EINVAL; } +std::error_category const& category() { return std::generic_category(); } +std::uint64_t device_of(const std::filesystem::path& dir) { + struct stat st {}; + return ::stat(dir.c_str(), &st) == 0 ? static_cast(st.st_dev) : 0; +} +std::string filesystem_type(const std::filesystem::path&) { return {}; } +#endif + +} // namespace mcpp::platform::native + +namespace mcpp::platform { + +// Not `fs`: mcpp::platform::fs is the namespace of mcpp.platform.fs, and an +// alias of the same name beside it is ill-formed (GCC 16 crashes reading the +// two modules into one translation unit). +namespace stdfs = std::filesystem; + +FsCaps probe_fs_caps(const stdfs::path& dir) { + FsCaps caps; + caps.type = native::filesystem_type(dir); + std::error_code ec; + std::random_device random; + const auto scratch = dir / std::format(".mcpp-fsprobe-{:08x}{:08x}", random(), random()); + if (!stdfs::create_directory(scratch, ec) || ec) return caps; // cannot write here: native answers + const auto touch = [&](const stdfs::path& p) { std::ofstream(p).put('x'); return stdfs::exists(p, ec); }; + if (touch(scratch / "a") && touch(scratch / "c")) { + caps.measured = true; + if (const int e = native::rename_no_replace(scratch / "a", scratch / "b"); e != 0) { + caps.rename_no_replace = !native::unsupported(e); + stdfs::rename(scratch / "a", scratch / "b", ec); + } + if (const int e = native::exchange(scratch / "b", scratch / "c"); e != 0) + caps.rename_exchange = !native::unsupported(e); + ec.clear(); + stdfs::create_hard_link(scratch / "b", scratch / "d", ec); + caps.hard_links = !ec; + ec.clear(); + stdfs::create_symlink("b", scratch / "e", ec); + caps.symlinks = !ec; + ec.clear(); + if (touch(scratch / "Case")) caps.case_sensitive = !stdfs::exists(scratch / "cASE", ec); + } + stdfs::remove_all(scratch, ec); + return caps; +} + +FsCaps fs_caps(const stdfs::path& dir) { + static std::mutex lock; + static std::map known; + const auto device = native::device_of(dir); + { + std::scoped_lock guard(lock); + if (device != 0) + if (const auto it = known.find(device); it != known.end()) return it->second; + } + auto caps = probe_fs_caps(dir); + // An unmeasured answer is not kept: the next directory of that filesystem + // may be writable. + if (device != 0 && caps.measured) { + std::scoped_lock guard(lock); + known.emplace(device, caps); + } + return caps; +} + +std::expected publish_no_replace(const stdfs::path& from, const stdfs::path& to, const FsCaps& caps) { + const auto failed = [&](std::error_code error) { + return std::unexpected(std::format("cannot publish {} as {}: {}", from.string(), to.string(), error.message())); + }; + if (caps.rename_no_replace) { + const int e = native::rename_no_replace(from, to); + if (e == 0) return {}; + if (!native::unsupported(e)) return failed(std::error_code(e, native::category())); + // Measured elsewhere on this filesystem, refused here (a mount under + // the directory probed): the strategies below. + } + std::error_code ec; + const auto status = stdfs::symlink_status(from, ec); + if (ec) return failed(ec); + if (status.type() == stdfs::file_type::symlink) { + // A link made again with the same target refuses a name that is + // there, as the rename flag would. (A hard link to a symbolic link is + // the link on Linux but its target on macOS.) + stdfs::copy_symlink(from, to, ec); + if (ec) return failed(ec); + stdfs::remove(from, ec); + return {}; + } + if (status.type() != stdfs::file_type::directory && caps.hard_links) { + stdfs::create_hard_link(from, to, ec); + if (ec) return failed(ec); + stdfs::remove(from, ec); + return {}; + } + const auto there = stdfs::symlink_status(to, ec); + if (ec && ec != std::errc::no_such_file_or_directory) return failed(ec); + if (stdfs::exists(there)) return failed(std::make_error_code(std::errc::file_exists)); + stdfs::rename(from, to, ec); + if (ec) return failed(ec); + return {}; +} + +std::expected rename_no_replace(const stdfs::path& from, const stdfs::path& to) { + auto parent = to.parent_path(); + if (parent.empty()) parent = "."; + return publish_no_replace(from, to, fs_caps(parent)); +} + +bool create_directory_link(const stdfs::path& target, const stdfs::path& link) { + std::error_code ec; + stdfs::create_directory_symlink(target, link, ec); + if (!ec) return true; +#if defined(_WIN32) + const auto cmd = std::format("cmd /c mklink /J \"{}\" \"{}\" >nul 2>&1", link.string(), target.string()); + return std::system(cmd.c_str()) == 0; +#else + return false; +#endif +} + +std::string untrusted_because(const stdfs::path& dir) { + std::error_code ec; + if (!stdfs::is_directory(dir, ec)) return "it is not a directory"; + return native::owner_problem(dir); +} + +std::expected exchange_paths(const stdfs::path& a, const stdfs::path& b) { + auto parent = b.parent_path(); + if (parent.empty()) parent = "."; + if (fs_caps(parent).rename_exchange) { + const int e = native::exchange(a, b); + if (e == 0) return {}; + if (!native::unsupported(e)) + return std::unexpected(std::format("cannot exchange {} and {}: {}", a.string(), b.string(), + std::error_code(e, native::category()).message())); + } + std::error_code ec; + std::random_device random; + const auto aside = parent / std::format(".{}.mcpp-aside-{:08x}", b.filename().string(), random()); + stdfs::rename(b, aside, ec); + if (ec) return std::unexpected(std::format("cannot move {} aside: {}", b.string(), ec.message())); + stdfs::rename(a, b, ec); + if (ec) { + std::error_code back; + stdfs::rename(aside, b, back); + return std::unexpected(std::format("cannot move {} to {}: {}", a.string(), b.string(), ec.message())); + } + stdfs::rename(aside, a, ec); + if (ec) return std::unexpected(std::format("cannot move the old {} to {}: {}", b.string(), a.string(), ec.message())); + return {}; +} + +} // namespace mcpp::platform diff --git a/modules/platform/src/platform.cppm b/modules/platform/src/platform.cppm index 168d210ba..7be8fc197 100644 --- a/modules/platform/src/platform.cppm +++ b/modules/platform/src/platform.cppm @@ -55,6 +55,7 @@ export import mcpp.platform.common; export import mcpp.platform.shell; export import mcpp.platform.process; export import mcpp.platform.fs; +export import mcpp.platform.fs_caps; export import mcpp.platform.env; export import mcpp.platform.macos; export import mcpp.platform.linux; diff --git a/modules/platform/src/process.cppm b/modules/platform/src/process.cppm index c0266c480..862dacec4 100644 --- a/modules/platform/src/process.cppm +++ b/modules/platform/src/process.cppm @@ -36,6 +36,7 @@ module; #include // O_RDONLY, O_WRONLY for capture_stdout's /dev/null #include // waitpid #include // posix_spawnp, posix_spawn_file_actions_* (incl. addchdir_np) +#include // umask // The deadline runners' headers (signal.h, errno, poll.h, fcntl.h, time.h) // left with them: bounded runs now live in mcpp.platform.unix.bounded_process // and mcpp.platform.windows.bounded_process, and this file only dispatches. @@ -58,6 +59,16 @@ import mcpp.platform.windows.bounded_process; export namespace mcpp::platform::process { +// Every file this process creates from here on is readable by everyone and +// writable by its owner only (umask 022): what a home shared by every user of +// a machine is written with. Nothing on Windows, where the ACL inherited from +// the directory decides. +inline void set_umask_022() { +#if !defined(_WIN32) + ::umask(022); +#endif +} + struct RunResult { int exit_code = 0; std::string output; diff --git a/modules/platform/src/scaffold_fs.cppm b/modules/platform/src/scaffold_fs.cppm index 2603bb750..dc731890b 100644 --- a/modules/platform/src/scaffold_fs.cppm +++ b/modules/platform/src/scaffold_fs.cppm @@ -29,6 +29,7 @@ module; export module mcpp.platform.scaffold_fs; import std; +import mcpp.platform.fs_caps; namespace mcpp::platform::detail { @@ -60,47 +61,12 @@ export namespace mcpp::platform { std::expected atomic_rename_directory_no_replace( const std::filesystem::path& source, const std::filesystem::path& target) { -#if defined(_WIN32) - if (::MoveFileExW(source.c_str(), target.c_str(), - MOVEFILE_WRITE_THROUGH) != 0) { - return {}; - } - const auto error = ::GetLastError(); - return std::unexpected(std::format( - "atomic no-replace rename '{}' -> '{}' failed: {}", - source.string(), target.string(), detail::win32_message(error))); -#elif defined(__linux__) && defined(SYS_renameat2) - if (::syscall(SYS_renameat2, AT_FDCWD, source.c_str(), AT_FDCWD, - target.c_str(), RENAME_NOREPLACE) == 0) { - return {}; - } - const int error = errno; - return std::unexpected(std::format( - "atomic no-replace rename '{}' -> '{}' failed: {}", - source.string(), target.string(), detail::errno_message(error))); -#elif defined(__APPLE__) - if (::renamex_np(source.c_str(), target.c_str(), RENAME_EXCL) == 0) { - return {}; - } - const int error = errno; - return std::unexpected(std::format( - "atomic no-replace rename '{}' -> '{}' failed: {}", - source.string(), target.string(), detail::errno_message(error))); -#else - // The fallback remains no-clobber in ordinary use, but platforms without - // a native exclusive rename cannot close the existence-check race. All - // release platforms use one of the branches above. - std::error_code ec; - if (std::filesystem::exists(target, ec) || ec) { - return std::unexpected(std::format( - "atomic scaffold target '{}' already exists", target.string())); - } - std::filesystem::rename(source, target, ec); - if (!ec) return {}; - return std::unexpected(std::format( - "scaffold rename '{}' -> '{}' failed: {}", - source.string(), target.string(), ec.message())); -#endif + // By what the target's filesystem does (fs_caps, #809): renameat2's + // RENAME_NOREPLACE is EINVAL on NFSv3, where `mcpp new` failed. + if (auto published = rename_no_replace(source, target); !published) + return std::unexpected(std::format("atomic no-replace rename '{}' -> '{}' failed: {}", + source.string(), target.string(), published.error())); + return {}; } // Make a newly written regular file durable before its containing staging diff --git a/modules/platform/src/windows/windows.cppm b/modules/platform/src/windows/windows.cppm index 9a059bbd3..85b028392 100644 --- a/modules/platform/src/windows/windows.cppm +++ b/modules/platform/src/windows/windows.cppm @@ -53,6 +53,12 @@ std::optional machine_registry_path(std::wstring_view sub // `ProgramFiles(x86)`), or nullopt. The system drive is not always `C:`. std::optional env_directory(const char* name); +// Whether the system lets a program that declares itself `longPathAware` use +// paths past MAX_PATH (260): HKLM\SYSTEM\CurrentControlSet\Control\FileSystem +// `LongPathsEnabled` = 1. false when it is 0 or absent, and on every other +// platform, which has no such limit. +bool long_paths_enabled(); + } // namespace mcpp::platform::windows // ─── Implementation ────────────────────────────────────────────────────── @@ -94,6 +100,17 @@ std::optional machine_registry_path(std::wstring_view sub #endif } +bool long_paths_enabled() { +#if defined(_WIN32) + DWORD value = 0, bytes = sizeof(value); + return RegGetValueW(HKEY_LOCAL_MACHINE, L"SYSTEM\\CurrentControlSet\\Control\\FileSystem", + L"LongPathsEnabled", RRF_RT_REG_DWORD, nullptr, &value, &bytes) == ERROR_SUCCESS + && value == 1; +#else + return false; +#endif +} + std::optional env_directory(const char* name) { if (const char* v = std::getenv(name); v && *v) return std::filesystem::path(v); return std::nullopt; diff --git a/modules/platform/tests/test_fs_caps.cpp b/modules/platform/tests/test_fs_caps.cpp new file mode 100644 index 000000000..725a2cf68 --- /dev/null +++ b/modules/platform/tests/test_fs_caps.cpp @@ -0,0 +1,127 @@ +#include +#if !defined(_WIN32) +#include +#include +#endif + +import std; +import mcpp.platform.fs_caps; + +// What a filesystem does, and the strategy each answer selects (#809). The +// strategies are driven by stated capabilities, so every one of them runs on +// whatever filesystem the test machine has -- the NFSv3 run is the CI lane's. + +namespace fs = std::filesystem; +namespace pf = mcpp::platform; + +namespace { +struct Scratch { + fs::path dir; + Scratch() { + std::random_device random; + dir = fs::temp_directory_path() / std::format("mcpp-fscaps-{:08x}", random()); + fs::create_directories(dir); + } + ~Scratch() { + std::error_code ec; + fs::remove_all(dir, ec); + } +}; +void write(const fs::path& p, std::string_view text) { std::ofstream(p) << text; } +std::string read(const fs::path& p) { + std::ifstream in(p); + return {std::istreambuf_iterator(in), {}}; +} +} // namespace + +TEST(PlatformFsCaps, TheProbeMeasuresAWritableDirectoryAndLeavesNothing) { + Scratch s; + const auto caps = pf::probe_fs_caps(s.dir); + EXPECT_TRUE(caps.measured); + EXPECT_TRUE(fs::is_empty(s.dir)) << "the probe removes its scratch"; + // A directory that cannot be written: the platform's native answers. + const auto none = pf::probe_fs_caps(s.dir / "absent"); + EXPECT_FALSE(none.measured); + EXPECT_TRUE(none.rename_no_replace); +} + +TEST(PlatformFsCaps, EveryStrategyPublishesAndNoneReplacesAName) { + Scratch s; + for (int strategy = 0; strategy < 3; ++strategy) { + pf::FsCaps caps; + caps.rename_no_replace = strategy == 0; + caps.hard_links = strategy <= 1; + const auto dir = s.dir / std::to_string(strategy); + fs::create_directories(dir); + // A file. + write(dir / "scratch", "new"); + ASSERT_TRUE(pf::publish_no_replace(dir / "scratch", dir / "file", caps)) << strategy; + EXPECT_EQ(read(dir / "file"), "new"); + EXPECT_FALSE(fs::exists(dir / "scratch")); + write(dir / "scratch", "newer"); + EXPECT_FALSE(pf::publish_no_replace(dir / "scratch", dir / "file", caps)) << strategy; + EXPECT_EQ(read(dir / "file"), "new") << "an existing name is never replaced"; + // A directory. + fs::create_directory(dir / "staged"); + write(dir / "staged" / "a", "a"); + ASSERT_TRUE(pf::publish_no_replace(dir / "staged", dir / "tree", caps)) << strategy; + EXPECT_EQ(read(dir / "tree" / "a"), "a"); + fs::create_directory(dir / "staged"); + EXPECT_FALSE(pf::publish_no_replace(dir / "staged", dir / "tree", caps)); + // A symbolic link (where the filesystem has them). + std::error_code ec; + fs::create_symlink("file", dir / "link-scratch", ec); + if (!ec) { + ASSERT_TRUE(pf::publish_no_replace(dir / "link-scratch", dir / "link", caps)) << strategy; + EXPECT_EQ(fs::read_symlink(dir / "link"), fs::path("file")); + } + } +} + +TEST(PlatformFsCaps, AnExchangeTradesTwoTreesWhateverTheFilesystemDoes) { + Scratch s; + fs::create_directories(s.dir / "next"); + fs::create_directories(s.dir / "live"); + write(s.dir / "next" / "rev", "2"); + write(s.dir / "live" / "rev", "1"); + ASSERT_TRUE(pf::exchange_paths(s.dir / "next", s.dir / "live")); + EXPECT_EQ(read(s.dir / "live" / "rev"), "2"); + EXPECT_EQ(read(s.dir / "next" / "rev"), "1"); + std::size_t entries = 0; + for ([[maybe_unused]] const auto& e : fs::directory_iterator(s.dir)) ++entries; + EXPECT_EQ(entries, 2u) << "nothing set aside is left"; +} + +// Whom an upstream home may belong to (design 2026-10-16 §8.2): this user, or +// the system when this user cannot change it. +TEST(PlatformTrust, ThisUsersDirectoryIsTrusted) { + Scratch s; + EXPECT_EQ(pf::untrusted_because(s.dir), ""); + EXPECT_NE(pf::untrusted_because(s.dir / "absent"), ""); +} + +#if !defined(_WIN32) +TEST(PlatformTrust, ASystemDirectoryThisUserCanWriteIsNot) { + if (::getuid() == 0) GTEST_SKIP() << "root may write anything"; + const auto rootOwned = [](const char* p) { + struct stat st {}; + return ::stat(p, &st) == 0 && st.st_uid == 0; + }; + // /tmp: root's, and writable by everyone. + if (rootOwned("/tmp") && ::access("/tmp", W_OK) == 0) EXPECT_NE(pf::untrusted_because("/tmp"), ""); + // /usr: root's, and not this user's to write. + if (rootOwned("/usr") && ::access("/usr", W_OK) != 0) EXPECT_EQ(pf::untrusted_because("/usr"), ""); +} +#endif + +TEST(PlatformTrust, ADirectoryLinkReachesItsTarget) { + Scratch s; + fs::create_directories(s.dir / "target"); + write(s.dir / "target" / "f", "x"); + ASSERT_TRUE(pf::create_directory_link(s.dir / "target", s.dir / "link")); + EXPECT_EQ(read(s.dir / "link" / "f"), "x"); + // Removing the link removes the link only. + std::error_code ec; + fs::remove(s.dir / "link", ec); + EXPECT_TRUE(fs::exists(s.dir / "target" / "f")); +} diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 7dd9bb7bf..05241280b 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -14,7 +14,7 @@ // it from fingerprint made index_contract depend on xlings — which made it // impossible for xlings to depend on index_contract, which is exactly where the // index-refresh guard has to live (mcpp.pm.index_snapshot, and see -// mcpp::xlings::update_index). The cycle was the layering telling us the +// mcpp::provider::xlings::update_index). The cycle was the layering telling us the // constant was in the wrong place. // // A leaf module with no imports beyond `std` can be used by anyone. Keep it @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.10.11.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.10.16.1"; } // namespace mcpp diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 1e4910303..6c811364c 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -347,7 +347,7 @@ struct BuildProgramEnv { // project on two machines would not. A declared `[xlings].subos` is a // directory that belongs to the project and travels with it. // - // WHO DECIDES IS NOT DECIDED HERE. `mcpp::xlings::runtime` is the sole + // WHO DECIDES IS NOT DECIDED HERE. `mcpp::provider::runtime` is the sole // project runtime-selection policy and `RuntimeBinding::subosDir` is its // resolved answer; this field carries that answer to the child. Deriving // it a second time — from the manifest, from `[xlings] deps`, from the diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 91b61c73e..97bc94dc2 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -40,8 +40,8 @@ import mcpp.modgraph.scanner; import mcpp.toolchain.post_install; import mcpp.toolchain.stdmod; import mcpp.version; // MCPP_VERSION — the engine a build record names -import mcpp.xlings; -import mcpp.xlings.subos_info; +import mcpp.provider; +import mcpp.provider.select; import mcpp.runtime.binding; import mcpp.log; import mcpp.platform; @@ -1833,9 +1833,9 @@ admit_recorded_build(const BuildCacheEntry& e, const ReplayAsk& ask) { if (ec) return std::unexpected("the time of build.ninja cannot be read"); const auto runtimeTime = std::filesystem::last_write_time( - e.runtimeBinding->subosDir / ".xlings.json", ec); + mcpp::provider::layout().description(e.runtimeBinding->subosDir), ec); if (ec || runtimeTime > ninjaTime) - return std::unexpected("the runtime's .xlings.json is newer than build.ninja"); + return std::unexpected("the runtime environment's description is newer than build.ninja"); const auto manifestTime = std::filesystem::last_write_time(ask.manifest, ec); if (ec || manifestTime > ninjaTime) @@ -3518,14 +3518,12 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, invocationNotRunReason = listed.error(); } - // Prepend the sandbox's subos/default/bin to the CHILD PATH so test + // Prepend the toolchain's environment's bin to the CHILD PATH so test // binaries that shell out to bootstrapped tools (patchelf, ninja) find // them — applied to the child only, not via a leaky shell prefix. if constexpr (!mcpp::platform::is_windows) { - if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(ctx->tc.binaryPath)) { - // xpkgs is /data/xpkgs → registry = xpkgs/../.. - auto registryDir = xpkgs->parent_path().parent_path(); - auto sandboxBin = registryDir / "subos" / "default" / "bin"; + if (auto environment = mcpp::provider::environment_of(ctx->tc.binaryPath)) { + auto sandboxBin = *environment / "bin"; if (std::filesystem::exists(sandboxBin)) { std::array extra{sandboxBin}; auto pathVal = mcpp::platform::env::prepend_path_list("PATH", extra); diff --git a/src/build/hermetic.cppm b/src/build/hermetic.cppm index 4f21abdab..9c37fb809 100644 --- a/src/build/hermetic.cppm +++ b/src/build/hermetic.cppm @@ -26,6 +26,8 @@ export module mcpp.build.hermetic; import std; import mcpp.log; import mcpp.platform; +import mcpp.provider; +import mcpp.provider.select; import mcpp.toolchain.fingerprint; import mcpp.toolchain.model; import mcpp.toolchain.triple; @@ -120,24 +122,21 @@ std::expected verify_hermetic_link( { if constexpr (!mcpp::platform::is_linux) return {}; - // Sandbox toolchains only: the compiler must live under an xpkgs - // registry. A host/system compiler is the user's explicit opt-in to the - // host world. - auto binStr = tc.binaryPath.string(); - auto xpkgsPos = binStr.find("xpkgs"); - if (xpkgsPos == std::string::npos) return {}; + // Provider toolchains only: the compiler must live in a store. A + // host/system compiler is the user's explicit opt-in to the host world. + auto store = mcpp::provider::store_of(tc.binaryPath); + if (!store) return {}; std::vector allowed; - allowed.emplace_back(binStr.substr(0, xpkgsPos + 5)); // the xpkgs root - // Payloads may be symlink-inherited from another MCPP_HOME (the e2e - // isolation pattern): mcpp passes the symlink-view paths on the command - // line, but the driver reports its own resource dir (clang_rt.crt*) - // through the CANONICAL path. Allow that registry too. + allowed.push_back(*store); + // A toolchain borrowed from an upstream is a link entry (design + // 2026-10-16 §8): mcpp passes the link's paths on the command line, but + // the driver reports its own resource dir (clang_rt.crt*) through the + // path the link names. That store is allowed too. { std::error_code ec; - auto canon = std::filesystem::weakly_canonical(tc.binaryPath, ec).string(); - auto pos = ec ? std::string::npos : canon.find("xpkgs"); - if (pos != std::string::npos) - allowed.emplace_back(canon.substr(0, pos + 5)); + auto real = std::filesystem::weakly_canonical(tc.binaryPath, ec); + if (!ec) + if (auto upstream = mcpp::provider::store_of(real)) allowed.push_back(*upstream); } if (!tc.sysroot.empty()) allowed.push_back(tc.sysroot); diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index fecf5ab0c..5c942c2c4 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -45,7 +45,8 @@ import mcpp.toolchain.provider; import mcpp.toolchain.registry; import mcpp.toolchain.triple; // shared_soname_flag decides by TARGET, not host import mcpp.toolchain.msvc_coroutines; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.platform; import mcpp.ui; import mcpp.log; @@ -4872,7 +4873,7 @@ std::expected NinjaBackend::build(const BuildPlan& plan std::string ninja_program_for(const mcpp::toolchain::Toolchain& tc) { auto ninja_name = std::string("ninja") + std::string(mcpp::platform::exe_suffix); - if (auto nb = mcpp::xlings::paths::find_sibling_binary(tc.binaryPath, "ninja", ninja_name)) + if (auto nb = mcpp::provider::beside_binary(tc.binaryPath, "ninja", ninja_name)) return nb->string(); return "ninja"; } diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 9883d30df..ef1ef5ab5 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -24,7 +24,8 @@ import mcpp.platform; import mcpp.runtime.binding; import mcpp.platform.runtime_env_contract; import mcpp.platform.runtime_search; -import mcpp.xlings.subos_info; +import mcpp.provider; +import mcpp.provider.select; export namespace mcpp::build { @@ -1366,7 +1367,7 @@ std::vector runtime_search_closure( void merge_runtime_binding_contract( BuildPlan& plan, const mcpp::platform::runtime::RuntimeBinding& binding) { - auto package_id = [](const mcpp::xlings::subos::PackageIdentity& value) { + auto package_id = [](const mcpp::provider::runtime_info::PackageIdentity& value) { return mcpp::manifest::PackageId{ .namespace_ = value.namespace_, .name = value.name, diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index ecd39680e..b4a802901 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -91,8 +91,9 @@ import mcpp.build.graph_shape; // #407: the graph says which mode wrote it import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build -import mcpp.xlings; -import mcpp.xlings.runtime_selection; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.runtime_selection; import mcpp.runtime.binding; import mcpp.toolchain.post_install; import mcpp.platform; @@ -471,7 +472,7 @@ export struct BuildContext { mcpp::manifest::Manifest manifest; mcpp::toolchain::Toolchain tc; mcpp::toolchain::Fingerprint fp; - mcpp::xlings::runtime::RuntimeSelection runtimeSelection; + mcpp::provider::runtime::RuntimeSelection runtimeSelection; mcpp::platform::runtime::RuntimeBinding runtimeBinding; std::filesystem::path projectRoot; // THE SOURCE TREES THIS BUILD READ THAT ARE NOT UNDER `projectRoot`. @@ -822,7 +823,7 @@ export struct BuildOverrides { std::filesystem::path project_index_root; // Nested source/tool builds inherit the consumer root's local development // OS. A dependency's own [xlings].subos is never consulted or propagated. - std::shared_ptr + std::shared_ptr inherited_runtime_selection; std::shared_ptr inherited_runtime_binding; diff --git a/src/build/prepare/config.cpp b/src/build/prepare/config.cpp index f5a19777b..73045c15f 100644 --- a/src/build/prepare/config.cpp +++ b/src/build/prepare/config.cpp @@ -12,7 +12,9 @@ import mcpp.build.prepare_inputs; import std; import mcpp.targetside; import mcpp.diag; -import mcpp.xlings.address_set; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.address_set; import mcpp.build.version_floor; import mcpp.platform.axis; import mcpp.manifest; @@ -33,7 +35,6 @@ import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its mani import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build -import mcpp.xlings; import mcpp.platform; import mcpp.ui; import mcpp.project; @@ -169,9 +170,9 @@ void merge_conditional_xlings(mcpp::manifest::Manifest& m, // to be `parse_address(a).target` — the bare name, so `xim:cuda` and a // hypothetical `scode:cuda` collided, and the graph split a few thousand // lines below compared whole address strings instead. See - // mcpp.xlings.address_set for what the two definitions cost. + // mcpp.provider.address_set for what the two definitions cost. auto package_of = [](std::string_view address) { - return mcpp::xlings::addrset::package_key(address); + return mcpp::provider::addrset::package_key(address); }; for (auto const& a : cc.xlings.deps) { const auto pkg = package_of(a); diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 5099878d3..3f66c3fdc 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -45,8 +45,9 @@ import mcpp.build.build_program; import mcpp.build.tool_store; // #355 host tools: store layout + key + overrides import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build -import mcpp.xlings; -import mcpp.xlings.runtime_selection; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.runtime_selection; import mcpp.runtime.binding; import mcpp.toolchain.post_install; import mcpp.platform; @@ -1507,7 +1508,7 @@ step6_build_tool(PrepareState& state, HostToolCtx& ctx) { : std::make_shared(dep); } sub.inherited_runtime_selection = std::make_shared< - const mcpp::xlings::runtime::RuntimeSelection>( + const mcpp::provider::runtime::RuntimeSelection>( state.runtimeSelection); sub.inherited_runtime_binding = std::make_shared< const mcpp::platform::runtime::RuntimeBinding>( diff --git a/src/build/prepare/fetch.cpp b/src/build/prepare/fetch.cpp index 748d6aa06..33bf57a7e 100644 --- a/src/build/prepare/fetch.cpp +++ b/src/build/prepare/fetch.cpp @@ -31,7 +31,8 @@ import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.config; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.platform; import mcpp.fetcher; import mcpp.fetcher.progress; @@ -296,8 +297,7 @@ provision_xlings_addresses(const mcpp::config::GlobalConfig& cfg, std::vector missing; if (have == want) for (auto const& d : declaredDeps) - if (!mcpp::xlings::paths::xpkg_payload( - xlEnv, mcpp::xlings::paths::parse_xpkg_ref(d))) + if (!mcpp::provider::payload(*mcpp::provider::open(xlEnv), mcpp::provider::parse_ref(d))) missing.push_back(d); if (!missing.empty()) { std::string list; @@ -439,8 +439,7 @@ provision_xlings_addresses(const mcpp::config::GlobalConfig& cfg, args["yes"] = true; mcpp::fetcher::InstallProgressHandler progress; - auto r = mcpp::xlings::call( - xlEnv, "install_packages", args.dump(), &progress); + auto r = mcpp::provider::open(xlEnv)->call("install_packages", args.dump(), &progress); // `if (!r)` IS NOT THE FAILURE TEST, AND TESTING ONLY // IT MADE THIS PATH REPORT SUCCESS FOR EVERY FAILURE XLINGS // CAN REPORT. @@ -499,8 +498,7 @@ provision_xlings_addresses(const mcpp::config::GlobalConfig& cfg, // gets xlings' answer rather than a re-derivation of it. for (auto const& e : r->dataEvents) if (e.dataKind == "install_targets") - mcpp::xlings::paths::record_resolutions(xlEnv, - mcpp::xlings::paths::parse_install_targets(e.payloadJson)); + mcpp::provider::open(xlEnv)->remember(e); // Written only on success, for the same reason the check // above exists: a stamp is a record that the effect // happened, and recording an effect that did not is worse diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index ca6942bbe..f58dd5cd7 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -11,7 +11,9 @@ import mcpp.diag; import mcpp.log; import mcpp.build.engine_package; import mcpp.build.refusal; -import mcpp.xlings.address_set; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.address_set; import mcpp.build.version_floor; import mcpp.home; import mcpp.platform.axis; @@ -43,7 +45,6 @@ import mcpp.build.directives; // directive table: mark / fold_private_tail import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.config; -import mcpp.xlings; import mcpp.toolchain.post_install; import mcpp.platform; import mcpp.pm.resolver; diff --git a/src/build/prepare/graph_load.cpp b/src/build/prepare/graph_load.cpp index 81e0b977d..a6505140b 100644 --- a/src/build/prepare/graph_load.cpp +++ b/src/build/prepare/graph_load.cpp @@ -9,7 +9,9 @@ import mcpp.build.prepare_inputs; import std; import mcpp.diag; import mcpp.build.refusal; -import mcpp.xlings.address_set; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.address_set; import mcpp.build.version_floor; import mcpp.platform.axis; import mcpp.log; @@ -29,7 +31,6 @@ import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.config; -import mcpp.xlings; import mcpp.platform; import mcpp.fetcher; import mcpp.fetcher.progress; @@ -45,7 +46,6 @@ import mcpp.pm.dependency_selector; import mcpp.pm.lock_io; import mcpp.ui; import mcpp.log; -import mcpp.fallback.install_integrity; import mcpp.project; namespace mcpp::build { @@ -82,11 +82,11 @@ static void step4a_define_split_and_identity_closures(PrepareState& state) { // from a rule package were two packages: both installed, gigabytes each, // and `xpkg_dir` answered one of them. The unification below adjudicates // (the declaration nearer the artifact wins) and validates (the winner must - // satisfy every requirement that lost) — see mcpp.xlings.address_set. + // satisfy every requirement that lost) — see mcpp.provider.address_set. state.graph_xlings_split = [&]() -> std::expected< std::pair, std::vector>, std::string> { - namespace addrset = mcpp::xlings::addrset; + namespace addrset = mcpp::provider::addrset; auto describe = [&](std::size_t i) { auto const& pkg = state.packages[i].manifest.package; return pkg.namespace_.empty() ? pkg.name @@ -646,8 +646,7 @@ static void step4a_define_candidate_selection_closures(PrepareState& state) { // in. That is a download the run needs, not a wrong selector. if (mcpp::platform::env::offline_mode()) { if (auto cfgO = state.get_cfg(true); - cfgO && !mcpp::xlings::default_index_status( - mcpp::config::make_xlings_env(**cfgO), 0).present) { + cfgO && !mcpp::provider::open(mcpp::config::make_xlings_env(**cfgO))->default_index_status(0).present) { hint += "\n offline: the package index has never been fetched; " "run `mcpp index update` without --offline"; refusal::record(refusal::Code::OfflineDownloadRequired); @@ -723,12 +722,6 @@ step4a_find_raw_installed(PrepareState& state, LoadVersionDepCtx& ctx) { return fetcher.install_path(ctx.ns, ctx.shortName, ctx.version); } -// The body of the `markInstalled` closure, called once below after a fresh -// install completes. -static void step4a_mark_installed(const std::filesystem::path& p) { - mcpp::fallback::mark_install_complete(p); -} - static std::expected step4a_load_version_dep_locate(PrepareState& state, LoadVersionDepCtx& ctx) { // ─── Routing: check if this dep's namespace maps to a custom index ── @@ -790,7 +783,7 @@ step4a_load_version_dep_locate(PrepareState& state, LoadVersionDepCtx& ctx) { return 0; }(); auto revisionIsCurrent = [&](const std::filesystem::path& p) { - const auto installed = mcpp::xlings::paths::installed_revision(p); + const auto installed = mcpp::provider::revision_of(p); if (!installed || *installed == recipeRevision) return true; mcpp::log::verbose("fetcher", std::format( "{}@{}: installed revision {}, descriptor revision {}; reinstalling", @@ -798,22 +791,19 @@ step4a_load_version_dep_locate(PrepareState& state, LoadVersionDepCtx& ctx) { return false; }; + // Installed is what the payload holds: the files its descriptor names + // (and the provider's revision record, above). mcpp writes no marker + // into a payload and removes none (design 2026-10-16 §7.1): one that + // is not complete goes back to the provider, which replaces it. auto findCompleteInstalled = [&]() -> std::optional { auto p = step4a_find_raw_installed(state, ctx); if (!p) return std::nullopt; if (!revisionIsCurrent(*p)) return std::nullopt; - if (mcpp::fallback::is_install_complete(*p)) return p; - if (installedLayoutMatchesIndex(*p)) { - mcpp::fallback::mark_install_complete(*p); - return p; - } - mcpp::fallback::clean_incomplete_install(*p); + if (installedLayoutMatchesIndex(*p)) return p; return std::nullopt; }; - // For custom indices, try project-level xlings data roots first. - // Existing directories without the mcpp completion marker are treated - // as stale/incomplete on this active resolve path and reinstalled. + // For custom indices, try project-level data roots first. ctx.installed = findCompleteInstalled(); return {}; @@ -948,12 +938,12 @@ step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { std::string capturedChildError; // xlings' own error lines, after its structured summary (#614). auto append_xlings_stderr = [](std::string& into, - const mcpp::xlings::CallResult& r) { + const mcpp::provider::CallResult& r) { if (r.exitCode == 0) return; for (auto const& line : r.stderrTail) into += (into.empty() ? "" : "\n ") + std::string("xlings: ") + line; }; - auto install_one = [&](std::string target) -> std::expected { + auto install_one = [&](std::string target) -> std::expected { if (ctx.useProjectEnv) { // Project/custom-index deps install into the project-local // xlings data root (so a package's install hook can find @@ -969,8 +959,7 @@ step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { auto argsJson = std::format( R"({{"targets":["{}"],"yes":true}})", target); mcpp::fetcher::InstallProgressHandler progress; - auto r = mcpp::xlings::call( - projEnv, "install_packages", argsJson, &progress); + auto r = mcpp::provider::open(projEnv)->call("install_packages", argsJson, &progress); capturedChildError = progress.captured_error(); if (!r) return std::unexpected(mcpp::pm::CallError{r.error()}); append_xlings_stderr(capturedChildError, *r); @@ -1069,12 +1058,11 @@ step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { // global xlings home), any child error text we captured, plus // a hint about the known ≥2-repo xlings resolution gap. The // real fix lives in openxlings/xlings; this only surfaces WHY. - auto xlingsJson = (ctx.useProjectEnv - ? ((state.overrides.project_index_root.empty() ? state.workRoot - : state.overrides.project_index_root) - / ".mcpp") - : (*cfg)->xlingsHome()) - / ".xlings.json"; + const auto projectRoot = state.overrides.project_index_root.empty() + ? state.workRoot : state.overrides.project_index_root; + auto xlingsJson = mcpp::provider::open(ctx.useProjectEnv + ? mcpp::config::make_project_xlings_env(**cfg, projectRoot) + : mcpp::config::make_xlings_env(**cfg))->config_file(); auto indexRepos = mcpp::pm::read_seeded_index_repos(xlingsJson); std::string childErr = capturedChildError; if (attempted.size() > 1 && !firstError.empty()) { @@ -1101,7 +1089,6 @@ step4a_load_version_dep_fetch(PrepareState& state, LoadVersionDepCtx& ctx) { ctx.installed = step4a_find_raw_installed(state, ctx); if (!ctx.installed) return std::unexpected(std::format( "package '{}@{}' install path missing after fetch", depName, version)); - step4a_mark_installed(*ctx.installed); } return {}; diff --git a/src/build/prepare/host_graph.cpp b/src/build/prepare/host_graph.cpp index 7c1599bd4..7f8af1c03 100644 --- a/src/build/prepare/host_graph.cpp +++ b/src/build/prepare/host_graph.cpp @@ -710,7 +710,7 @@ build_group(PrepareState& state, const fs::path& dir, const std::vector ? std::string("root → build programs") : state.overrides.tool_chain + " → build programs"; sub.inherited_runtime_selection = - std::make_shared(state.runtimeSelection); + std::make_shared(state.runtimeSelection); sub.inherited_runtime_binding = std::make_shared(state.runtimeBindingSnapshot); sub.resolution_pins = resolution_pins(state); diff --git a/src/build/prepare/local_toolchain.cpp b/src/build/prepare/local_toolchain.cpp index 7b8c025b8..0ce1a163d 100644 --- a/src/build/prepare/local_toolchain.cpp +++ b/src/build/prepare/local_toolchain.cpp @@ -266,7 +266,8 @@ step2_use_local_toolchain(PrepareState& state, const mcpp::toolchain::ToolchainS pkg.ximName = family; pkg.family = family == "llvm" ? mcpp::toolchain::Family::Llvm : mcpp::toolchain::Family::Gcc; pkg.needsGccPostInstallFixup = family == "gcc"; - state.provide_runtime_payload(pkg); + if (auto provided = state.provide_runtime_payload(pkg); !provided) + return std::unexpected(provided.error()); } return {}; } diff --git a/src/build/prepare/manifest.cpp b/src/build/prepare/manifest.cpp index 6a22653d9..38e7af873 100644 --- a/src/build/prepare/manifest.cpp +++ b/src/build/prepare/manifest.cpp @@ -30,8 +30,9 @@ import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its mani import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build -import mcpp.xlings; -import mcpp.xlings.runtime_selection; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.runtime_selection; import mcpp.platform; import mcpp.pm.resolver; import mcpp.pm.index_spec; @@ -402,7 +403,7 @@ std::expected check_engine_floors(const PrepareState& state, ? m.package.name : m.package.namespace_ + "." + m.package.name; return std::unexpected(std::format( "package '{}' ({}) requires mcpp >= {}; this is mcpp {}.\n" - " hint: pin \"mcpp\": \"{}\" (or newer) in .xlings.json and run " + " hint: pin \"mcpp\": \"{}\" (or newer) in .xlings.json and run " // provider-boundary-ok: the project file a person edits to pin mcpp "`xlings install`, or run `xlings install mcpp@{}`", who, where, m.package.mcppFloor, mcpp::MCPP_VERSION, m.package.mcppFloor, m.package.mcppFloor)); @@ -592,7 +593,7 @@ std::expected phase0_manifest_and_workspace(PrepareState& sta } else { std::optional> wsRef; if (state.wsManifest) wsRef = std::cref(*state.wsManifest); - auto selected = mcpp::xlings::runtime::select_runtime( + auto selected = mcpp::provider::runtime::select_runtime( *state.m, wsRef, *state.root, state.runtimeWorkspaceRoot); if (!selected) return std::unexpected(selected.error()); state.runtimeSelection = std::move(*selected); diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 90cfcf07f..1fa6eee7e 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -54,7 +54,8 @@ import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.lockfile; import mcpp.config; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.runtime.binding; import mcpp.platform.runtime_search; import mcpp.toolchain.post_install; @@ -300,8 +301,8 @@ static std::expected step13_runner_and_xlings(PrepareState& s if (auto cfg = state.get_cfg(true)) { auto xlEnv = mcpp::config::make_xlings_env(**cfg); for (auto const& spec : xlingsSpecs) { - auto ref = mcpp::xlings::paths::parse_xpkg_ref(spec); - if (auto dir = mcpp::xlings::paths::xpkg_payload(xlEnv, ref)) { + auto ref = mcpp::provider::parse_ref(spec); + if (auto dir = mcpp::provider::payload(*mcpp::provider::open(xlEnv), ref)) { ctx.xlingsPayloads.push_back(*dir); // `bin/`, then the payload root. The measurement that // added the second entry is recorded with the rule, in @@ -1348,12 +1349,12 @@ static std::expected step13_assembly_units(PrepareState& stat if (!cfgNasm) return std::unexpected(cfgNasm.error()); std::optional nasmBin = - mcpp::xlings::find_usable_nasm(mcpp::config::make_xlings_env(**cfgNasm)); + mcpp::provider::usable_nasm(*mcpp::provider::open(mcpp::config::make_xlings_env(**cfgNasm))); if (!nasmBin) { mcpp::fetcher::Fetcher nasmFetcher(**cfgNasm); mcpp::fetcher::InstallProgressHandler nasmProgress; auto nasmTarget = std::format("xim:nasm@{}", - mcpp::xlings::pinned::kNasmVersion); + mcpp::provider::kNasmVersion); auto payload = nasmFetcher.resolve_xpkg_path( nasmTarget, /*autoInstall=*/true, &nasmProgress); if (!payload) { @@ -1361,8 +1362,7 @@ static std::expected step13_assembly_units(PrepareState& stat "NASM sources (.asm) present but nasm provisioning " "failed: {}", payload.error().message)); } - nasmBin = mcpp::xlings::find_sandbox_nasm( - mcpp::config::make_xlings_env(**cfgNasm)); + nasmBin = mcpp::provider::placed_nasm(*mcpp::provider::open(mcpp::config::make_xlings_env(**cfgNasm))); } if (!nasmBin) { return std::unexpected(std::string( @@ -1376,8 +1376,7 @@ static std::expected step13_assembly_units(PrepareState& stat // that already has an assembler. Saying nothing would leave two // machines assembling the same source with different tools and // no line in either build recording which. - if (mcpp::xlings::nasm_is_from_host( - mcpp::config::make_xlings_env(**cfgNasm), *nasmBin)) { + if (mcpp::provider::open(mcpp::config::make_xlings_env(**cfgNasm))->nasm_is_from_host(*nasmBin)) { mcpp::diag::degraded("build/nasm-from-host", std::format( "the assembler for this build is the host's ('{}'), not " "the one this engine pins", nasmBin->string()), diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index ca68e2736..ee6942de4 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -51,7 +51,8 @@ import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.lockfile; import mcpp.config; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.runtime.binding; import mcpp.platform.runtime_search; import mcpp.toolchain.post_install; diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index 86dbe07ee..50a919f6f 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -44,7 +44,8 @@ import mcpp.build.graph_shape; // #407: the graph says which mode wrote it import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.toolchain.post_install; import mcpp.platform; import mcpp.platform.macos; @@ -488,8 +489,7 @@ step11_package_std_module_source(PrepareState& state) { // A flag naming a path is relative to the package that named it, // for the same reason the module source is. auto candidate = pkg.root / f; - flags += " " + mcpp::xlings::shq( - std::filesystem::exists(candidate) ? candidate.string() : f); + flags += " " + mcpp::platform::shell::quote(std::filesystem::exists(candidate) ? candidate.string() : f); } // AND THE HEADERS THIS PACKAGE ITSELF IS BUILT AGAINST. // @@ -525,9 +525,9 @@ step11_package_std_module_source(PrepareState& state) { // reach both or the `std` BMI describes a different world than the // units importing it — which is mcpp#514 exactly. for (auto& d : state.targetSideUsage.includeDirs) - flags += " -isystem " + mcpp::xlings::shq(d.string()); + flags += " -isystem " + mcpp::platform::shell::quote(d.string()); for (auto& d : state.targetSideUsage.includeDirsAfter) - flags += " -idirafter " + mcpp::xlings::shq(d.string()); + flags += " -idirafter " + mcpp::platform::shell::quote(d.string()); // And the definitions, for the same reason as the directories: a C // library's headers show a different library depending on which feature // macros are set, and the ones this package is built with are the ones @@ -571,18 +571,18 @@ step11_package_std_module_source(PrepareState& state) { cAbiTargetIsMacos, state.m->buildConfig.macosDeploymentTarget); for (auto& t : mcpp::toolchain::host_compile_tokens( *state.tc, hopt, mcpp::toolchain::no_escape)) { - const auto q = " " + mcpp::xlings::shq(t); + const auto q = " " + mcpp::platform::shell::quote(t); if (flags.find(q) == std::string::npos) flags += q; } for (auto& t : mcpp::toolchain::apple_float_macro_words(*state.tc)) { - const auto q = " " + mcpp::xlings::shq(t); + const auto q = " " + mcpp::platform::shell::quote(t); if (flags.find(q) == std::string::npos) flags += q; } } // The same words the package's own units receive from this list // (mcpp.manifest.flag_words), one quoted word each. for (auto& w : mcpp::manifest::flag_words(state.targetSideUsage.cxxflags)) - flags += " " + mcpp::xlings::shq(w); + flags += " " + mcpp::platform::shell::quote(w); state.tc->stdModuleFlags = flags; break; } diff --git a/src/build/prepare/sources.cpp b/src/build/prepare/sources.cpp index da91c7fc0..ad70a2157 100644 --- a/src/build/prepare/sources.cpp +++ b/src/build/prepare/sources.cpp @@ -23,13 +23,14 @@ import mcpp.wire; import mcpp.manifest; import mcpp.platform; import mcpp.ui; -import mcpp.xlings; -import mcpp.xlings.address_set; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.address_set; namespace mcpp::build { namespace fs = std::filesystem; -namespace addrset = mcpp::xlings::addrset; +namespace addrset = mcpp::provider::addrset; namespace { @@ -383,7 +384,7 @@ void fill_xpkg_env(PrepareState& state, mcpp::build::BuildProgramEnv& e, std::set answered; // Namespaced first -- it is the exact spelling, and the bare form must not // shadow it (the receiver keeps the first value it is given for a name). - auto put = [&](const mcpp::xlings::paths::XpkgRef& ref, std::string_view suffix, + auto put = [&](const mcpp::provider::PackageRef& ref, std::string_view suffix, const std::string& value) { e.xpkgDirs.emplace_back(mcpp::build::xpkg_env_var(ref.ns, ref.name, suffix), value); e.xpkgDirs.emplace_back(mcpp::build::xpkg_env_var("", ref.name, suffix), value); @@ -398,7 +399,7 @@ void fill_xpkg_env(PrepareState& state, mcpp::build::BuildProgramEnv& e, if (!answered.insert(key).second) continue; auto wit = state.xlingsWinner.find(key); const std::string spec = wit == state.xlingsWinner.end() ? raw : wit->second; - auto ref = mcpp::xlings::paths::parse_xpkg_ref(spec); + auto ref = mcpp::provider::parse_ref(spec); // AN OVERRIDE ANSWERS INSTEAD OF THE REGISTRY (mcpp#755): its root as // the directory, so a plugin that reads `/bin/` needs no // change, its program where it named one, and the source. @@ -410,7 +411,7 @@ void fill_xpkg_env(PrepareState& state, mcpp::build::BuildProgramEnv& e, } continue; } - auto dir = mcpp::xlings::paths::xpkg_payload(xlEnv, ref); + auto dir = mcpp::provider::payload(*mcpp::provider::open(xlEnv), ref); if (dir) { put(ref, "DIR", dir->string()); put(ref, "SOURCE", "payload"); diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 7f5e91bdc..7bcaac840 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -51,9 +51,10 @@ import mcpp.build.provisions; // #359 build-time provisions: table + propagati import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.config; -import mcpp.xlings; -import mcpp.xlings.address_set; // mcpp#755: override checks reuse the unification's claims -import mcpp.xlings.runtime_selection; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.runtime_selection; +import mcpp.provider.address_set; import mcpp.runtime.binding; import mcpp.toolchain.post_install; import mcpp.platform; @@ -505,7 +506,7 @@ struct PrepareState { return out; } std::filesystem::path runtimeWorkspaceRoot; - mcpp::xlings::runtime::RuntimeSelection runtimeSelection; + mcpp::provider::runtime::RuntimeSelection runtimeSelection; std::filesystem::path workRoot; // The project whose `.mcpp/` holds the packages of the custom indices: // `overrides.project_index_root`, else the project being built. @@ -518,7 +519,7 @@ struct PrepareState { CacheMode cacheMode{}; // ── P1: toolchain spec resolution, target axis, L1 cfg merge ──────────── - // `get_cfg`, `provide_runtime_payload`, `report_fixup`, + // `get_cfg`, `provide_runtime_payload`, // `msvc_usable_either_origin`, `native_first_run_spec`, `tcSpecSource`, // `resolvedAccel`, `cfgCtx` and `add_once` are closures that outlive P1 // (later phases call them by name): each captures `state` itself, not the @@ -533,9 +534,7 @@ struct PrepareState { std::string runtimePayload; std::filesystem::path runtimeLibDir; bool runtimePayloadProvided = false; - std::function provide_runtime_payload; - std::function report_fixup; + std::function(const mcpp::toolchain::XimToolchainPackage&)> provide_runtime_payload; std::string effectiveProfile; std::vector storeRoots; std::optional tcSpec; @@ -973,8 +972,8 @@ payload_override(PrepareState& state, std::string_view key); // declared `provision = "on-request"`. std::expected, std::string> payloads_to_provision(PrepareState& state, - const mcpp::xlings::addrset::Resolution& unified, - std::span claims, + const mcpp::provider::addrset::Resolution& unified, + std::span claims, const std::vector& onRequest); // A dependency that states `[xlings.overrides]` is refused: where a payload // comes from is the root's statement (mcpp#755). diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index 7e92359f3..eb81c534a 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -42,8 +42,9 @@ import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.config; -import mcpp.xlings; -import mcpp.xlings.runtime_selection; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.runtime_selection; import mcpp.runtime.binding; import mcpp.toolchain.post_install; import mcpp.platform; @@ -173,13 +174,13 @@ static std::expected step1_define_early_toolchain_closures(Pr // was started with, byte for byte. // // NOT RE-DERIVED AT THE TWO DELIVERY SITES BELOW, AND NOT FROM - // `[xlings] deps`. `mcpp::xlings::runtime` is the sole runtime-selection + // `[xlings] deps`. `mcpp::provider::runtime` is the sole runtime-selection // policy and `RuntimeBinding::subosDir` is its resolved answer; a second // derivation is how a build ends up with two subos and no way to say which // one it used. The per-package payload paths a program may also need are // already answered, separately, by `MCPP_XPKG_*_DIR`. state.projectSubosBin = [&]() -> std::string { - using Mode = mcpp::xlings::runtime::RuntimeSelection::Mode; + using Mode = mcpp::provider::runtime::RuntimeSelection::Mode; if (state.runtimeBindingSnapshot.selection.mode != Mode::NamedSubos) return {}; auto bin = state.runtimeBindingSnapshot.subosDir / "bin"; @@ -192,50 +193,30 @@ static std::expected step1_define_early_toolchain_closures(Pr state.runtimeLibDir = state.runtimeBindingSnapshot.libraryDirs.empty() ? std::filesystem::path{} : state.runtimeBindingSnapshot.libraryDirs.front(); - // THE DECLARED RUNTIME PAYLOAD IS PROVIDED BEFORE THE FIRST FIXUP THAT - // CONSUMES IT (mcpp#660), and not earlier: a build whose toolchain needs - // no C runtime payload must not download one. At most once per build. An + // THE DECLARED RUNTIME PAYLOAD IS PROVIDED WITH THE TOOLCHAIN THAT RUNS ON + // IT (mcpp#660), and not earlier: a build whose toolchain needs no C + // runtime payload must not download one. At most once per build. An // inherited binding is not exempt, because the parent build may have used // a toolchain that needed no payload. The two values derived above are // refreshed with the binding, because detection and the fingerprint read - // them after the fixups. + // them afterwards. state.runtimePayloadProvided = false; - state.provide_runtime_payload = [&](const mcpp::toolchain::XimToolchainPackage& pkg) { - if (state.runtimePayloadProvided) return; - if (mcpp::toolchain::post_install_fixup_kind(pkg).empty()) return; + state.provide_runtime_payload = [&](const mcpp::toolchain::XimToolchainPackage& pkg) + -> std::expected { + if (state.runtimePayloadProvided) return {}; + if (mcpp::toolchain::runtime_payload_kind(pkg).empty()) return {}; state.runtimePayloadProvided = true; auto cfgP = state.get_cfg(true); - if (!cfgP) return; - if (!mcpp::toolchain::ensure_declared_runtime(**cfgP, state.runtimeBindingSnapshot)) - return; + if (!cfgP) return {}; + auto changed = mcpp::toolchain::ensure_declared_runtime(**cfgP, state.runtimeBindingSnapshot); + if (!changed) return std::unexpected(changed.error()); + if (!*changed) return {}; state.runtimePayload = state.runtimeBindingSnapshot.libc; state.runtimeLibDir = state.runtimeBindingSnapshot.libraryDirs.empty() ? std::filesystem::path{} : state.runtimeBindingSnapshot.libraryDirs.front(); + return {}; }; - // mcpp#427: a toolchain fixup that could not run is a DEGRADATION, not a - // failure — the build continues without it. But it has to be said, or the - // eventual `stdlib.h: No such file or directory` arrives with no way to - // connect it to its cause. - // - // Deduplicated by payload: `ensure_post_install_fixup` is called from up - // to four seams in one build (manifest toolchain, default toolchain, - // MinGW first-run, build.mcpp host toolchain) and they routinely resolve - // the SAME payload. Saying it once is the rule mcpp#417 already paid for. - auto fixupNoticed = std::make_shared>(); - state.report_fixup = [fixupNoticed]( - const mcpp::toolchain::FixupOutcome& outcome, - const std::filesystem::path& payloadRoot) { - if (outcome.skippedReason.empty()) return; - if (!fixupNoticed->insert(payloadRoot.generic_string()).second) return; - // Only the fact this line ADDS. The `Runtime` note above already gave - // the cause and the remedy for the same absence; repeating them here - // would be the second copy of one message, which is the habit mcpp#417 - // exists to break. - mcpp::ui::info("Toolchain", std::format( - "used as installed — not patched against a C runtime ({})", - outcome.skippedReason)); - }; // Toolchain resolution priority: see the table at the top of this @@ -295,12 +276,16 @@ static std::expected step1_define_early_toolchain_closures(Pr // the same fact twice is how two of them start disagreeing. state.storeRoots = [&]() -> std::vector { std::vector roots; - if (auto c = state.get_cfg(true)) roots.push_back((*c)->xlingsHome() / "data" / "xpkgs"); - for (auto& d : mcpp::config::project_xlings_data_roots(state.workRoot)) - roots.push_back(d / "xpkgs"); - if (!state.overrides.project_index_root.empty()) - for (auto& d : mcpp::config::project_xlings_data_roots(state.overrides.project_index_root)) - roots.push_back(d / "xpkgs"); + auto c = state.get_cfg(true); + if (c) for (auto& s : mcpp::provider::open(mcpp::config::make_xlings_env(**c))->stores()) roots.push_back(s); + auto project = [&](const std::filesystem::path& dir) { + for (auto& s : mcpp::provider::layout().stores_of(dir / ".mcpp")) + roots.push_back(s); + for (auto& d : mcpp::config::project_xlings_data_roots(dir)) + roots.push_back(mcpp::provider::layout().store_in(d)); + }; + project(state.workRoot); + if (!state.overrides.project_index_root.empty()) project(state.overrides.project_index_root); // The engine interface's package: written once per mcpp version and // never edited, like a payload (mcpp#797). roots.push_back(mcpp::build::engine::root()); @@ -417,7 +402,7 @@ static std::expected step1_define_early_toolchain_closures(Pr auto c = state.get_cfg(true); if (!c) return mcpp::toolchain::msvc::has_usable_msvc(); return mcpp::toolchain::msvc::msvc_available_here( - (*c)->xlingsHome() / "data" / "xpkgs"); + mcpp::provider::open(mcpp::config::make_xlings_env(**c))->store()); }; // THE PLATFORM'S CANONICAL NATIVE DEFAULT — a spec string only; no @@ -1439,13 +1424,8 @@ step2_resolve_explicit_spec(PrepareState& state, ToolchainResolveCtx& ctx) { // manifest [toolchain] path previously ran none, so a freshly // auto-installed payload kept its stale install-time cfg / // unpatched runtime libs. - state.provide_runtime_payload(pkg); - if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( - **cfg, payload->root, pkg, - state.runtimeBindingSnapshot.runtimeId, state.runtimeLibDir); !fixed) - return std::unexpected(std::format( - "toolchain post-install fixup: {}", fixed.error())); - else state.report_fixup(*fixed, payload->root); + if (auto provided = state.provide_runtime_payload(pkg); !provided) + return std::unexpected(provided.error()); // Canonical rendering, whatever spelling the manifest/config used: // "Resolved gcc@16.1.0 → x86_64-linux-musl → ". // @@ -1668,13 +1648,8 @@ step2_first_run_auto_install(PrepareState& state) { // `mcpp toolchain install` performs — without it a fresh sandbox // gcc cannot find the C library (stdlib.h: No such file or // directory) and a fresh llvm keeps its stale install-time cfg. - state.provide_runtime_payload(defaultPkg); - if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( - **cfg, payload->root, defaultPkg, - state.runtimeBindingSnapshot.runtimeId, state.runtimeLibDir); !fixed) - return std::unexpected(std::format( - "default toolchain post-install fixup: {}", fixed.error())); - else state.report_fixup(*fixed, payload->root); + if (auto provided = state.provide_runtime_payload(defaultPkg); !provided) + return std::unexpected(provided.error()); // Persist the default so we don't ask again next time. if (auto wr = mcpp::config::write_default_toolchain(**cfg, defaultSpec); wr) { @@ -2010,7 +1985,7 @@ step2_retarget_for_retargetable_driver(PrepareState& state) { // a path with a space in it. state.tc->stdModuleTargetFlags = " " + state.tc->crossTargetFlag - + " -isysroot " + mcpp::xlings::shq(sdk->string()); + + " -isysroot " + mcpp::platform::shell::quote(sdk->string()); } } } @@ -2054,7 +2029,7 @@ step2_retarget_for_retargetable_driver(PrepareState& state) { *want, sysroot_override(*state.m, *want)); !want_sysroot.empty()) { if (auto cfg3 = state.get_cfg(true); cfg3) { - auto ref = mcpp::xlings::paths::parse_xpkg_ref(want_sysroot); + auto ref = mcpp::provider::parse_ref(want_sysroot); auto xl = mcpp::config::make_xlings_env(**cfg3); // INSTALLED, NOT MERELY LOOKED UP — THE SAME CHANNEL // THE ROW'S TOOLCHAIN PIN GOES THROUGH. @@ -2089,7 +2064,7 @@ step2_retarget_for_retargetable_driver(PrepareState& state) { want_sysroot, /*autoInstall=*/true, &srProgress)) dir = p->root; else - dir = mcpp::xlings::paths::xpkg_payload(xl, ref); + dir = mcpp::provider::payload(*mcpp::provider::open(xl), ref); if (dir) { if (auto spec = mcpp::freestanding::resolve(*want)) { const auto inc = @@ -2243,13 +2218,8 @@ step2_msvc_abi_without_msvc_repair(PrepareState& state) { gnuPkg.target(), mcpp::toolchain::payload_frontend_dir(payloadR->root, gnuPkg).string())); } - state.provide_runtime_payload(gnuPkg); - if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( - **cfgR, payloadR->root, gnuPkg, - state.runtimeBindingSnapshot.runtimeId, state.runtimeLibDir); !fixed) - return std::unexpected(std::format( - "MinGW toolchain post-install fixup: {}", fixed.error())); - else state.report_fixup(*fixed, payloadR->root); + if (auto provided = state.provide_runtime_payload(gnuPkg); !provided) + return std::unexpected(provided.error()); // Persist both axes so the repair happens once, not on every build. if (mcpp::config::write_default_toolchain(**cfgR, pins::kFirstRunWinGnu)) diff --git a/src/build/prepare/toolchain_decision.cpp b/src/build/prepare/toolchain_decision.cpp index 2ea4d379b..ba710238e 100644 --- a/src/build/prepare/toolchain_decision.cpp +++ b/src/build/prepare/toolchain_decision.cpp @@ -134,7 +134,7 @@ std::expected phase5_toolchain_after_graph(PrepareState& stat auto pkg = mcpp::toolchain::to_xim_package(*spec); if (auto picked = mcpp::toolchain::resolve_version_match( "", mcpp::toolchain::list_installed_versions( - (*cfg)->xlingsHome() / "data" / "xpkgs", + mcpp::provider::open(mcpp::config::make_xlings_env(**cfg))->store(), pkg.ximName))) return std::format("{}@{}", family, *picked); } diff --git a/src/build/prepare/toolchain_env.cpp b/src/build/prepare/toolchain_env.cpp index 67d7ca4f8..8e95f1409 100644 --- a/src/build/prepare/toolchain_env.cpp +++ b/src/build/prepare/toolchain_env.cpp @@ -39,7 +39,8 @@ import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.config; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.build.resources; // the resource compiler the row uses (#734 E2) import mcpp.toolchain.post_install; import mcpp.platform; @@ -426,7 +427,7 @@ void fill_target_build_env(mcpp::build::BuildProgramEnv& e, // toolchain or link mode resolved, so this is set before the toolchain // gate below. if (cfg) { - const auto view = mcpp::xlings::paths::sysroot(mcpp::config::make_xlings_env(*cfg)); + const auto view = mcpp::provider::open(mcpp::config::make_xlings_env(*cfg))->sysroot(); e.pkgConfigLibdir = (view / "usr" / "lib" / "pkgconfig").generic_string() + mcpp::platform::env::path_list_separator() + (view / "usr" / "share" / "pkgconfig").generic_string(); diff --git a/src/build/prepare/windows_resources.cpp b/src/build/prepare/windows_resources.cpp index 96624b1dc..8fada107b 100644 --- a/src/build/prepare/windows_resources.cpp +++ b/src/build/prepare/windows_resources.cpp @@ -50,7 +50,8 @@ import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.lockfile; import mcpp.config; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.runtime.binding; import mcpp.platform.runtime_search; import mcpp.toolchain.post_install; @@ -206,19 +207,31 @@ std::expected step13_windows_resources(PrepareState& state, B if (!t.windowsCodePage.empty()) return t.windowsCodePage; return (hostToolBuild && t.is_program() && !ownManifest) ? "utf-8" : "legacy"; }; + // `windows_long_paths` (design 2026-10-16 §11), with the same + // defaults: a host tool is long-path aware unless it says otherwise. + auto longPathsOf = [&](const mcpp::manifest::Target& t) { + const auto& stated = t.windowsLongPaths; + if (stated) return *stated; + return hostToolBuild && t.is_program() && !ownManifest; + }; // A test program carries the code page `[test] windows_code_page` // gave it, and nothing by default: the host-tool default is a // program's. auto takesCodePage = [](const mcpp::manifest::Target& t) { return t.is_program() || t.kind == mcpp::manifest::Target::TestBinary; }; + // Whether an image carries an application manifest of mcpp's: a + // UTF-8 code page or long paths asked for. + auto manifested = [&](const mcpp::manifest::Target& t) { + return codePageOf(t) == "utf-8" || longPathsOf(t); + }; const bool anyUtf8Image = std::ranges::any_of(M.targets, [&](const auto& t) { - return takesCodePage(t) && codePageOf(t) == "utf-8"; + return takesCodePage(t) && manifested(t); }); // Author scripts reach programs and shared libraries, never a test // program, so only a program's manifest can collide with one. const bool anyUtf8Program = std::ranges::any_of(M.targets, [&](const auto& t) { - return t.is_program() && codePageOf(t) == "utf-8"; + return t.is_program() && manifested(t); }); if (M.resources.declared() || anyUtf8Image) { namespace rsrc = mcpp::build::resources; @@ -319,7 +332,7 @@ std::expected step13_windows_resources(PrepareState& state, B return lu.kind == mcpp::build::LinkUnit::TestBinary && owns(lu) && std::ranges::any_of(M.targets, [&](const auto& t) { return t.name == lu.targetName && takesCodePage(t) - && codePageOf(t) == "utf-8"; + && manifested(t); }); }); if (peUnits.empty()) { @@ -415,14 +428,17 @@ std::expected step13_windows_resources(PrepareState& state, B // the version block belong to a specific artifact, and the // manifest to a specific executable. const bool synthVersion = R.declared() && R.synthesize_version_info(); - auto wantsUtf8 = [&](const mcpp::build::LinkUnit& lu) { + auto targetOf = [&](const mcpp::build::LinkUnit& lu) -> const mcpp::manifest::Target* { if (lu.kind != mcpp::build::LinkUnit::Binary - && lu.kind != mcpp::build::LinkUnit::TestBinary) return false; - if (!lu.artifactOf.empty()) return false; + && lu.kind != mcpp::build::LinkUnit::TestBinary) return nullptr; + if (!lu.artifactOf.empty()) return nullptr; for (auto const& t : M.targets) - if (t.name == lu.targetName) - return takesCodePage(t) && codePageOf(t) == "utf-8"; - return false; + if (t.name == lu.targetName && takesCodePage(t)) return &t; + return nullptr; + }; + auto wantsUtf8 = [&](const mcpp::build::LinkUnit& lu) { + const auto* t = targetOf(lu); + return t && manifested(*t); }; if (!iconAbs.empty() || synthVersion || anyUtf8Image) { // A version key mcpp cannot order (an upstream build number) @@ -464,7 +480,9 @@ std::expected step13_windows_resources(PrepareState& state, B if (utf8) { manifestAbs = resDir / (imageName + ".mcpp.manifest"); std::filesystem::create_directories(manifestAbs.parent_path(), mkEc); - const auto manifestText = rsrc::utf8_code_page_manifest(); + const auto* t = targetOf(lu); + const auto manifestText = rsrc::application_manifest( + codePageOf(*t) == "utf-8", longPathsOf(*t)); std::string had; if (std::ifstream in(manifestAbs, std::ios::binary); in) had.assign(std::istreambuf_iterator(in), {}); diff --git a/src/build/prepare/xlings.cpp b/src/build/prepare/xlings.cpp index 7ace3eee7..a0c5d61e6 100644 --- a/src/build/prepare/xlings.cpp +++ b/src/build/prepare/xlings.cpp @@ -9,7 +9,9 @@ import mcpp.build.prepare_inputs; import std; import mcpp.diag; import mcpp.build.refusal; -import mcpp.xlings.address_set; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.address_set; import mcpp.build.version_floor; import mcpp.log; import mcpp.manifest; @@ -35,7 +37,6 @@ import mcpp.build.build_program; import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.config; -import mcpp.xlings; import mcpp.toolchain.post_install; import mcpp.platform; import mcpp.fetcher; @@ -256,13 +257,8 @@ step3_define_host_tc_closures_and_refresh_index(PrepareState& state) { pkg.target(), mcpp::toolchain::payload_frontend_dir(payload->root, pkg).string())); } - state.provide_runtime_payload(pkg); - if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( - **cfgH, payload->root, pkg, - state.runtimeBindingSnapshot.runtimeId, state.runtimeLibDir); !fixed) - return std::unexpected(std::format( - "host toolchain post-install fixup: {}", fixed.error())); - else state.report_fixup(*fixed, payload->root); + if (auto provided = state.provide_runtime_payload(pkg); !provided) + return std::unexpected(provided.error()); // SAME THREE ARGUMENTS THE NATIVE CALL USES (line ~3550), not the // one-argument form. `detect()` probes `payloadPaths` — the // fine-grained glibc/linux-headers xpkg directories `resolve_link_model` @@ -448,7 +444,7 @@ std::expected phase3_xlings_before_graph(PrepareState& state) if (!state.m->indices.empty() || materializeRootRuntime) { auto cfg2 = state.get_cfg(true); if (cfg2) { - mcpp::xlings::ProjectEnv penv; + mcpp::provider::ProjectEnv penv; if (materializeRootRuntime) { penv.deps = state.runtimeOwnerManifest->xlings.deps; // Appended, never substituted: a project may legitimately @@ -488,7 +484,7 @@ std::expected phase3_xlings_before_graph(PrepareState& state) // what it cannot see is `[xlings.workspace]` and // `[feature-xlings.]` naming one package at two versions, which // reaches here as two addresses and used to install both. - std::vector rootClaims; + std::vector rootClaims; std::vector rootOnRequest; for (auto const& spec : applicable_xlings_addresses( *state.runtimeOwnerManifest, @@ -498,7 +494,7 @@ std::expected phase3_xlings_before_graph(PrepareState& state) rootClaims.push_back({spec, "this project", 0}); rootOnRequest.push_back(state.runtimeOwnerManifest->xlings.on_request(spec)); } - auto rootUnified = mcpp::xlings::addrset::unify(rootClaims); + auto rootUnified = mcpp::provider::addrset::unify(rootClaims); if (!rootUnified) { refusal::record(refusal::Code::ToolVersionConflict); return std::unexpected(rootUnified.error()); @@ -514,7 +510,7 @@ std::expected phase3_xlings_before_graph(PrepareState& state) if (!rootInstall) return std::unexpected(rootInstall.error()); std::vector declaredDeps = std::move(*rootInstall); if (materializeRootRuntime && !state.xlingsOverridden.empty()) { - namespace addrset = mcpp::xlings::addrset; + namespace addrset = mcpp::provider::addrset; std::erase_if(penv.deps, [&](const std::string& a) { return state.xlingsOverridden.contains(addrset::package_key(a)); }); @@ -642,7 +638,7 @@ std::expected phase3_xlings_before_graph(PrepareState& state) if (needsRemoteUpdate && !refreshPolicy.offline) { mcpp::ui::status("Fetching", "custom index repos (first use)"); auto projEnv = mcpp::config::make_project_xlings_env(**cfg2, state.projectIndexRoot()); - int rc = mcpp::xlings::update_index(projEnv, /*quiet=*/true); + int rc = mcpp::provider::open(projEnv)->update_index(/*quiet=*/true); if (rc != 0) { return std::unexpected( "project custom index update failed; run `mcpp index update` for details"); diff --git a/src/build/resources.cppm b/src/build/resources.cppm index d744f7592..1aced3407 100644 --- a/src/build/resources.cppm +++ b/src/build/resources.cppm @@ -160,9 +160,12 @@ synthesize_rc(const mcpp::manifest::Package& pkg, const std::filesystem::path& iconAbs, const std::filesystem::path& manifestAbs = {}); -// The application manifest that makes a process's ANSI code page UTF-8 on -// Windows 10 1903 and later (#693), as Microsoft documents it. ASCII text; the -// same bytes for every image that carries it. +// An application manifest, as Microsoft documents its settings: the process +// ANSI code page UTF-8 on Windows 10 1903 and later (#693), and paths past +// MAX_PATH where the system enables them (`longPathAware`, design 2026-10-16 +// §11). ASCII text; the same bytes for every image that carries the same. +std::string application_manifest(bool utf8CodePage, bool longPathAware); +// The manifest of a host tool: both. std::string utf8_code_page_manifest(); // A compiled resource holding `utf8_code_page_manifest()`, for an executable @@ -315,19 +318,27 @@ std::optional find_rc_tool(const mcpp::toolchain::Toolchain& tc, return std::nullopt; } -std::string utf8_code_page_manifest() { - return +std::string application_manifest(bool utf8CodePage, bool longPathAware) { + std::string text = "\n" "\n" " \n" - " \n" - " " - "UTF-8\n" + " \n"; + if (utf8CodePage) + text += " " + "UTF-8\n"; + if (longPathAware) + text += " " + "true\n"; + text += " \n" " \n" "\n"; + return text; } +std::string utf8_code_page_manifest() { return application_manifest(true, true); } + std::expected compile_utf8_manifest(const mcpp::toolchain::Toolchain& tc, std::string_view dialectId, diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm index ae4c29711..3df3c0d6f 100644 --- a/src/build/runtime_validation.cppm +++ b/src/build/runtime_validation.cppm @@ -16,6 +16,8 @@ export module mcpp.build.runtime_validation; import std; +import mcpp.provider; +import mcpp.provider.select; import mcpp.build.loader_contract; import mcpp.build.plan; import mcpp.build.symbol_provision; @@ -341,7 +343,7 @@ std::string post_link_key(const mcpp::build::BuildPlan& plan) { material += '\x1f'; if (!plan.runtimeBinding.subosDir.empty()) { std::error_code ec; - auto stampPath = plan.runtimeBinding.subosDir / ".xlings.json"; + auto stampPath = mcpp::provider::layout().description(plan.runtimeBinding.subosDir); auto size = std::filesystem::file_size(stampPath, ec); if (!ec) material += std::to_string(size); ec.clear(); diff --git a/src/build/test_targets.cppm b/src/build/test_targets.cppm index 7b2f8db99..f872780b4 100644 --- a/src/build/test_targets.cppm +++ b/src/build/test_targets.cppm @@ -121,7 +121,10 @@ discover_test_targets(const std::filesystem::path& manifestRoot, target.name = name; target.kind = mcpp::manifest::Target::TestBinary; target.main = mainRelative->string(); - if (packageManifest) target.windowsCodePage = packageManifest->testWindowsCodePage; + if (packageManifest) { + target.windowsCodePage = packageManifest->testWindowsCodePage; + target.windowsLongPaths = packageManifest->testWindowsLongPaths; + } for (std::size_t i = 0; i < globFlags.size(); ++i) { if (!globHits[i].contains(file)) continue; for (auto const& define : globFlags[i].defines) diff --git a/src/cli.cppm b/src/cli.cppm index d076e30d8..bac2dc747 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -39,6 +39,7 @@ import mcpp.build.progress; // __action reports its start import mcpp.platform.runtime_search; // linker-wrapper path-injection opt-out import mcpp.ui; import mcpp.log; +import mcpp.home; // --system: the system home import mcpp.diag; // the single sink for the report below import mcpp.modgraph.glob; // take_unnarrowable_paths() @@ -97,6 +98,7 @@ void print_usage() { std::println(" --no-cache Deprecated alias for --cache=off (clears the build dir)"); std::println(" --no-color Disable colored output"); std::println(" --offline Never touch the network (also: MCPP_OFFLINE=1)"); + std::println(" --system Maintain the machine's system home (administrators)"); std::println(" --locked Fail if resolution differs from mcpp.lock (also: --frozen, MCPP_LOCKED=1)"); std::println(" --play-game[=NAME] Play snake, stack or runner in the status row while it builds"); std::println(" --jobs N|auto, -j Concurrent compiles ('auto' = cores + free RAM)"); @@ -147,6 +149,57 @@ struct ReportUnnarrowablePaths { } }; +// The words of the command line that name the command, up to two. +std::vector command_words(int argc, char** argv) { + std::vector words; + for (int i = 1; i < argc && words.size() < 2; ++i) { + std::string_view a = argv[i]; + if (a == "--") break; + if (a.starts_with("-")) { + if (a == "--jobs" || a == "-j" || a == "--toolchain") ++i; + continue; + } + words.push_back(a); + } + return words; +} + +// `--system`: the command is one that maintains a home, and this process may +// write the system home; MCPP_HOME then names it, files are written readable +// by everyone (umask 022), and the home is held exclusively while this runs. +int enter_system_home(int argc, char** argv) { + const auto words = command_words(argc, argv); + const auto is = [&](std::string_view a, std::initializer_list bs) { + return words.size() == 2 && words[0] == a && std::ranges::find(bs, words[1]) != bs.end(); + }; + if (!(is("self", {"init", "doctor", "store"}) || is("toolchain", {"install", "remove", "default"}) + || is("index", {"update"}))) { + mcpp::ui::error("`--system` maintains the machine's system home: self init|doctor|store, toolchain " + "install|remove|default, index update. A build as root would write root's files into a " + "user's project; build as yourself"); + return 2; + } + const auto home = mcpp::home::system_root(); + std::error_code ec; + std::filesystem::create_directories(home, ec); + const auto probe = home / std::format(".mcpp-write-{:x}", std::random_device{}()); + { std::ofstream(probe) << ""; } + if (ec || !std::filesystem::exists(probe, ec)) { + mcpp::ui::error(std::format("`--system` writes {}, which this user cannot; run it with sudo " + "(or as a member of the group that maintains it)", home.string())); + return 4; + } + std::filesystem::remove(probe, ec); + mcpp::platform::process::set_umask_022(); + mcpp::platform::env::set("MCPP_HOME", home.string()); + static std::optional held; + while (!(held = mcpp::platform::fs::FileLock::try_acquire(home))) + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + if (!mcpp::home::is_system_home(home)) + std::ofstream(home / std::string(mcpp::home::kHomeMarker)) << "# The machine's mcpp home: written by `mcpp --system`, read by every user.\nmode = \"system\"\n"; + return 0; +} + int run(int argc, char** argv) { namespace cl = mcpplibs::cmdline; ReportUnnarrowablePaths reportUnnarrowable_; @@ -218,6 +271,13 @@ int run(int argc, char** argv) { else if (a.starts_with("--play-game=")) mcpp::platform::env::set("MCPP_PLAY_GAME", std::string(a.substr(12))); } + // `--system` (design 2026-10-16 §9.2): the maintenance commands act on + // the machine's system home, whatever the caller's HOME and MCPP_HOME + // say. Building as root would write root's files into a user's project, + // so no other command takes it. + if (std::ranges::any_of(std::span(argv + 1, argv + argc), [](const char* a) { return std::string_view(a) == "--system"; })) { + if (int rc = enter_system_home(argc, argv); rc != 0) return rc; + } // Decline xlings' linker-wrapper path injection, for this process and // everything it spawns (openxlings/xlings#540). // @@ -342,6 +402,9 @@ int run(int argc, char** argv) { .option(cl::Option("offline") .help("Never touch the network (index refresh, downloads, toolchain install)") .global()) + .option(cl::Option("system") + .help("Maintain the machine's system home (self init|doctor|store, toolchain install|remove|default, index update)") + .global()) // Declared here as well as read in the pre-pass: the pre-pass sets the // env var, and this makes the parser accept the token instead of // rejecting it as unknown. Both halves are needed, which is exactly the @@ -916,8 +979,14 @@ int run(int argc, char** argv) { .subcommand(cl::App("explain") .description("Show extended description for an error code") .arg(cl::Arg("code").help("Error code such as E0001").required())) + .subcommand(cl::App("store") + .description("Compare this home's payloads with its upstreams") + .subcommand(cl::App("dedupe") + .description("List payloads an upstream has the same of; --apply links them") + .option(cl::Option("apply").help("Replace each by a link to the upstream's")))) .action(wrap_rc([&dispatch_sub](const cl::ParsedArgs& p) { return dispatch_sub("self", p, { + {"store", cmd_self_store}, {"init", cmd_self_init}, {"doctor", cmd_doctor}, {"env", cmd_env}, diff --git a/src/cli/cmd_self.cppm b/src/cli/cmd_self.cppm index f0a659023..1c574d240 100644 --- a/src/cli/cmd_self.cppm +++ b/src/cli/cmd_self.cppm @@ -8,6 +8,8 @@ module; export module mcpp.cli.cmd_self; import std; +import mcpp.provider; +import mcpp.provider.select; import mcpp.build.engine_package; import mcpplibs.cmdline; import mcpp.doctor; @@ -68,7 +70,7 @@ nlohmann::json env_data_readonly() { // the one function the first run uses. Probing for a usable MSVC is // read-only, as the rest of this path is. {"defaultToolchain", std::string(mcpp::toolchain::triple::pins::host_default_toolchain( - mcpp::toolchain::msvc::msvc_available_here(registry / "data" / "xpkgs")))}, + mcpp::toolchain::msvc::msvc_available_here(mcpp::provider::layout().store_of_home(registry))))}, // An Android host (Termux): the same default toolchain as GNU/Linux on // this architecture; reported because the shell and the loader rules // differ (D21, D28). @@ -154,6 +156,16 @@ export int cmd_self_config(const mcpplibs::cmdline::ParsedArgs& parsed) { return mcpp::doctor::self_config(parsed.option_or_empty("mirror").value()); } +// `mcpp self store dedupe [--apply]`. +export int cmd_self_store(const mcpplibs::cmdline::ParsedArgs& parsed) { + auto sub = parsed.has_subcommand() && parsed.subcommand_name() == "dedupe" ? parsed.subcommand() : std::nullopt; + if (!sub) { + std::println(stderr, "error: `mcpp self store` requires a subcommand: dedupe"); + return 2; + } + return mcpp::doctor::store_dedupe(sub->get().is_flag_set("apply")); +} + // Used both by `mcpp explain ` (top-level) and `mcpp self explain // ` (legacy alias). export int cmd_explain_action(const mcpplibs::cmdline::ParsedArgs& parsed) { diff --git a/src/config.cppm b/src/config.cppm index f3f6dc395..2dde831a7 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -26,16 +26,17 @@ import mcpp.modgraph.glob; // try_narrow: the one UTF-8 spelling of a path import mcpp.libs.toml; import mcpp.libs.json; import mcpp.pm.index_spec; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.platform; import mcpp.log; +import mcpp.diag; import mcpp.fallback.xlings_binary; import mcpp.fallback.config_migration; -import mcpp.fallback.install_integrity; export namespace mcpp::config { -inline constexpr std::string_view kXlingsPinnedVersion = mcpp::xlings::pinned::kXlingsVersion; +inline constexpr std::string_view kXlingsPinnedVersion = mcpp::provider::kPinnedVersion; struct IndexRepo { std::string name; @@ -50,6 +51,11 @@ struct IndexRepo { // default. The provisioning line names an index that such a table // redirected (#634, C4). bool fromConfig = false; + // An index mcpp owns keeps its data in /indices/ and its + // payloads in /xpkgs (design 2026-10-16 §5.1); empty for the + // provider's own indexes. + std::filesystem::path dir; + std::filesystem::path store; }; // Canonical mcpplibs index coordinates. The index repository moved from the @@ -124,6 +130,13 @@ struct GlobalConfig { // Empty means no global default; mcpp will hard-error unless the project // mcpp.toml declares its own [toolchain]. std::string defaultToolchain; + + // [xlings] upstream (design 2026-10-16 §8): read-only homes the registry + // installs from by link -- the user's configuration, else the system's, + // else none (self-contained). Each one passed the trust check. + std::vector upstreams; + // This home is the machine's system home (`mcpp --system`, §9). + bool systemHome = false; // default_target = "" e.g. "x86_64-linux-musl" // The target axis of the default pair (toolchain = family@version and // target = triple are orthogonal axes). Empty means host. @@ -149,7 +162,7 @@ struct GlobalConfig { }; // Create an xlings::Env from the resolved GlobalConfig. -mcpp::xlings::Env make_xlings_env(const GlobalConfig& cfg) { +mcpp::provider::Location make_xlings_env(const GlobalConfig& cfg) { #if defined(_WIN32) // On Windows, the copied xlings binary in the sandbox may not function // correctly for large package installs (missing runtime environment). @@ -167,7 +180,7 @@ mcpp::xlings::Env make_xlings_env(const GlobalConfig& cfg) { // Create an xlings::Env that targets the project-level .mcpp/ directory. // Used when custom (non-builtin) indices are configured in mcpp.toml. -mcpp::xlings::Env make_project_xlings_env(const GlobalConfig& cfg, +mcpp::provider::Location make_project_xlings_env(const GlobalConfig& cfg, const std::filesystem::path& projectDir) { return { cfg.xlingsBinary, cfg.xlingsHome(), projectDir / ".mcpp" }; } @@ -193,27 +206,8 @@ resolve_project_index_path(const std::filesystem::path& projectDir, bool project_index_data_initialized(const std::filesystem::path& projectDir) { - std::error_code ec; - for (auto const& data : project_xlings_data_roots(projectDir)) { - auto reposDir = data / "xim-index-repos"; - if (std::filesystem::exists(reposDir / "xim-indexrepos.json", ec)) { - return true; - } - if (std::filesystem::is_directory(reposDir, ec)) { - for (auto const& entry : std::filesystem::directory_iterator(reposDir, ec)) { - if (entry.is_directory(ec) && std::filesystem::exists(entry.path() / "pkgs", ec)) { - return true; - } - } - } - if (std::filesystem::is_directory(data, ec)) { - for (auto const& entry : std::filesystem::directory_iterator(data, ec)) { - if (entry.is_directory(ec) && std::filesystem::exists(entry.path() / "pkgs", ec)) { - return true; - } - } - } - } + for (auto const& data : project_xlings_data_roots(projectDir)) + if (mcpp::provider::layout().index_data_present(data)) return true; return false; } @@ -223,19 +217,17 @@ std::string check_base_init(const GlobalConfig& cfg) { auto xlEnv = make_xlings_env(cfg); struct Check { std::string_view name; std::filesystem::path path; }; - auto patchelfBin = mcpp::xlings::paths::xim_tool(xlEnv, "patchelf", - mcpp::xlings::pinned::kPatchelfVersion) / "bin" / "patchelf"; - auto ninjaRoot = mcpp::xlings::paths::xim_tool_root(xlEnv, "ninja"); + auto patchelfBin = mcpp::provider::open(xlEnv)->pinned("patchelf", mcpp::provider::kPatchelfVersion) / "bin" / "patchelf"; + auto ninjaRoot = mcpp::provider::open(xlEnv)->versions("ninja"); auto ninja_name = std::string("ninja") + std::string(mcpp::platform::exe_suffix); // Check xlings binary if (!std::filesystem::exists(cfg.xlingsBinary)) return std::format("xlings binary missing: {}", cfg.xlingsBinary.string()); - // Check sandbox init marker - auto marker = xlEnv.home / "subos" / "default" / ".xlings.json"; - if (!std::filesystem::exists(marker)) - return "sandbox not initialized (missing subos/default/.xlings.json)"; + // The provider's own environment, initialised once. + if (!mcpp::provider::open(xlEnv)->initialized()) + return "the provider's environment is not initialized"; // Check patchelf (Linux only) #if !defined(__APPLE__) && !defined(_WIN32) @@ -279,7 +271,7 @@ bool ensure_project_index_dir( const GlobalConfig& cfg, const std::filesystem::path& projectDir, const std::map& indices, - const mcpp::xlings::ProjectEnv& penv = {}); + const mcpp::provider::ProjectEnv& penv = {}); struct ConfigError { std::string message; }; @@ -296,9 +288,9 @@ struct ConfigError { std::string message; }; // // We can't use mcpp.fetcher's EventHandler here because fetcher imports // config — the dependency would be cyclic. -using BootstrapFile = mcpp::xlings::BootstrapFile; -using BootstrapProgress = mcpp::xlings::BootstrapProgress; -using BootstrapProgressCallback = mcpp::xlings::BootstrapProgressCallback; +using BootstrapFile = mcpp::provider::BootstrapFile; +using BootstrapProgress = mcpp::provider::BootstrapProgress; +using BootstrapProgressCallback = mcpp::provider::BootstrapProgressCallback; // `initial_mirror`: when seeding a *fresh* .xlings.json (file did not // already exist), use this as the mirror value instead of the default. @@ -361,7 +353,10 @@ void print_status(std::string_view verb, std::string_view msg) { } std::expected run_capture(const std::string& cmd) { - return mcpp::xlings::run_capture(cmd); + auto r = mcpp::platform::process::capture(cmd); + if (r.exit_code != 0 && r.output.empty()) + return std::unexpected("command failed: " + cmd); + return r.output; } void write_file(const std::filesystem::path& p, std::string_view content) { @@ -414,13 +409,13 @@ bool write_default_xlings_json(const std::filesystem::path& path, std::string_view mirror_override = {}) { // Delegate to xlings module. Convert IndexRepo vec to SeedRepo span. - std::vector pairs; + std::vector pairs; pairs.reserve(repos.size()); for (auto& r : repos) - pairs.push_back({ r.name, r.url, r.artifact, r.source, r.artifactCn }); + pairs.push_back({ r.name, r.url, r.artifact, r.source, r.artifactCn, r.dir, r.store }); // seed_xlings_json writes to env.home / ".xlings.json", so we // construct a temporary Env with home = path.parent_path(). - mcpp::xlings::Env env; + mcpp::provider::Location env; env.home = path.parent_path(); // No explicit --mirror: seed "auto" so xlings' own adaptive mirror module // (latency-probed, with per-download failover) picks the best reachable @@ -428,9 +423,9 @@ bool write_default_xlings_json(const std::filesystem::path& path, // mechanism, stranding overseas users and GitHub-hosted CI on a // slow/unreachable gitcode. An explicit --mirror always wins (else branch). if (mirror_override.empty()) - mcpp::xlings::seed_xlings_json(env, pairs); // default = "auto" + mcpp::provider::open(env)->seed(pairs); // default = "auto" else - mcpp::xlings::seed_xlings_json(env, pairs, mirror_override); + mcpp::provider::open(env)->seed(pairs, mirror_override); return std::filesystem::exists(path); } @@ -486,6 +481,8 @@ nlohmann::json index_repo_entry(const IndexRepo& r) { else if (!r.artifact.empty()) e["artifact"] = r.artifact; if (!r.source.empty()) e["source"] = r.source; + if (!r.dir.empty()) e["dir"] = r.dir.generic_string(); + if (!r.store.empty()) e["store"] = r.store.generic_string(); return e; } @@ -597,7 +594,7 @@ void reconcile_index_repos(const std::filesystem::path& xjson, // + patchelf in one shot, this function and ensure_sandbox_xlings_binary + // ensure_sandbox_patchelf below can collapse into a single call. void ensure_sandbox_init(const GlobalConfig& cfg, bool quiet) noexcept { - mcpp::xlings::ensure_init(make_xlings_env(cfg), quiet); + mcpp::provider::open(make_xlings_env(cfg))->ensure_initialized(quiet); } // With the 0.0.4 layout change (xlings binary at /registry/bin/ @@ -613,13 +610,13 @@ void ensure_sandbox_xlings_binary(const GlobalConfig& /*cfg*/, bool /*quiet*/) n void ensure_sandbox_ninja(const GlobalConfig& cfg, bool quiet, const BootstrapProgressCallback& cb) noexcept { - mcpp::xlings::ensure_ninja(make_xlings_env(cfg), quiet, cb); + mcpp::provider::open(make_xlings_env(cfg))->ensure_ninja(quiet, cb); } void ensure_sandbox_patchelf(const GlobalConfig& cfg, bool quiet, const BootstrapProgressCallback& cb) noexcept { - mcpp::xlings::ensure_patchelf(make_xlings_env(cfg), quiet, cb); + mcpp::provider::open(make_xlings_env(cfg))->ensure_patchelf(quiet, cb); } } // namespace @@ -763,18 +760,75 @@ std::expected load_or_init( } cfg.xlingsBinaryMode = doc->get_string("xlings.binary").value_or("bundled"); - if (auto h = doc->get_string("xlings.home"); h && !h->empty()) + if (auto h = doc->get_string("xlings.home"); h && !h->empty()) { cfg.xlingsHomeOverride = *h; + // The registry is mcpp's (design 2026-10-16 §12, D10): a provider + // home mcpp did not make -- one whose configuration names none of + // mcpp's indices -- is somebody else's, and reusing it is what an + // upstream is for. `mcpp-index` is the name an earlier mcpp gave it + // (COMPAT(internal:legacy-index-name)). + if (const auto served = mcpp::provider::layout().indices_of(cfg.xlingsHomeOverride); !served.empty()) { + const bool mcpps = std::ranges::any_of(served, [](const std::string& name) { + return name == "mcpplibs" || name == "mcpp-index"; + }); + if (!mcpps) + return std::unexpected(ConfigError{std::format( + "[xlings] home = \"{}\" is a home mcpp did not make; to reuse its packages, name it as " + "an upstream instead: [xlings] upstream = [\"{}\"]", + cfg.xlingsHomeOverride.string(), cfg.xlingsHomeOverride.string())}); + } + } cfg.defaultIndex = doc->get_string("index.default").value_or("mcpplibs"); cfg.indexAutoRefresh = doc->get_bool("index.auto_refresh").value_or(true); if (auto t = doc->get_int("index.refresh_timeout"); t && *t > 0) cfg.indexRefreshTimeoutSeconds = *t; - mcpp::xlings::set_index_refresh_timeout( - std::chrono::seconds{cfg.indexRefreshTimeoutSeconds}); + mcpp::provider::set_index_refresh_timeout(std::chrono::seconds{cfg.indexRefreshTimeoutSeconds}); cfg.searchTtlSeconds = doc->get_int("cache.search_ttl_seconds").value_or(3600); cfg.defaultJobs = doc->get_int("build.default_jobs").value_or(0); cfg.defaultToolchain = doc->get_string("toolchain.default").value_or(""); cfg.defaultTarget = doc->get_string("toolchain.default_target").value_or(""); + // The system-level configuration (design 2026-10-16 §8.1, §9): what an + // administrator states for every user, below what the user states. + cfg.systemHome = mcpp::home::is_system_home(cfg.mcppHome); + std::optional systemDoc; + if (auto sys = mcpp::libs::toml::parse_file(mcpp::home::system_config_file()); sys) systemDoc = std::move(*sys); + if (cfg.defaultToolchain.empty() && systemDoc) { + cfg.defaultToolchain = systemDoc->get_string("toolchain.default").value_or(""); + if (cfg.defaultTarget.empty()) + cfg.defaultTarget = systemDoc->get_string("toolchain.default_target").value_or(""); + } + { + auto declared = doc->get_string_array("xlings.upstream"); + if (!declared && systemDoc) declared = systemDoc->get_string_array("xlings.upstream"); + for (auto entry : declared.value_or(std::vector{})) { + std::filesystem::path dir = entry; + if (entry == "~" || entry.starts_with("~/")) + if (auto* h = std::getenv(mcpp::platform::is_windows ? "USERPROFILE" : "HOME"); h && *h) + dir = std::filesystem::path(h) / (entry.size() > 2 ? entry.substr(2) : std::string{}); + dir = dir.lexically_normal(); + std::error_code ec; + if (!dir.is_absolute() || !std::filesystem::is_directory(dir, ec)) { + // Fewer packages reused than asked for: said, not hidden. + mcpp::diag::degraded("xlings/upstream", std::format("[xlings] upstream {} is not a directory", entry), + "this run does not reuse it"); + continue; + } + // The trust boundary (§8.2): another user's home is code that + // user can replace at any time. + if (auto why = mcpp::platform::untrusted_because(dir); !why.empty()) + return std::unexpected(ConfigError{std::format( + "[xlings] upstream {}: not used, {}; an upstream belongs to this user or to the system " + "and only its owner can change it", dir.string(), why)}); + if (mcpp::provider::layout().upstream_of(dir).empty()) { + mcpp::diag::degraded("xlings/upstream", + std::format("[xlings] upstream {} is neither an mcpp home nor an xlings home", + dir.string()), + "this run does not reuse it"); + continue; + } + cfg.upstreams.push_back(dir); + } + } // [xlings.overrides] (mcpp#755), read by the same function the tests read. if (auto overrides = parse_payload_overrides(*doc); overrides) cfg.payloadOverrides = std::move(*overrides); @@ -862,11 +916,19 @@ std::expected load_or_init( add_default("mcpplibs", kMcpplibsIndexUrl, kMcpplibsIndexArtifact, kMcpplibsIndexArtifactCn); canonicalize_legacy_index_names(cfg); + // Ownership is by index (design 2026-10-16 §5): an index mcpp serves + // keeps its data and payloads in mcpp's home; the provider's own indexes + // stay where the provider keeps them. + for (auto& r : cfg.indexRepos) { + if (mcpp::provider::layout().is_provider_index(r.name)) continue; + r.dir = cfg.mcppHome / "indices" / r.name; + r.store = mcpp::provider::layout().own_store(cfg.mcppHome); + } // 5. Seed registry/.xlings.json if missing; migrate legacy cached // sandboxes in place so CI/user caches move from mcpp-index to mcpplibs // without losing xlings' active subos or version bindings. - auto xjson = cfg.xlingsHome() / ".xlings.json"; + auto xjson = mcpp::provider::open(make_xlings_env(cfg))->config_file(); if (!std::filesystem::exists(xjson)) { write_default_xlings_json(xjson, cfg.indexRepos, initial_mirror); record_seeded_index_repos(xjson, cfg.indexRepos, quiet); @@ -874,8 +936,36 @@ std::expected load_or_init( mcpp::fallback::migrate_xlings_json_index_names(xjson); reconcile_index_repos(xjson, cfg.indexRepos, quiet); } + { + // A home from before: its indices' places recorded, their data moved + // there once; payloads already installed stay where they are and are + // still found (design §5.3). + std::vector owned; + for (auto& r : cfg.indexRepos) + if (!r.dir.empty()) owned.push_back({ r.name, r.url, {}, {}, {}, r.dir, r.store }); + mcpp::provider::open(make_xlings_env(cfg))->own_indices(owned); + } + // The path budget (design 2026-10-16 §11): with Windows' long paths off, + // a payload whose deepest file (gcc's libstdc++ headers: 140 characters + // below , measured) would pass MAX_PATH cannot be installed. + if constexpr (mcpp::platform::is_windows) { + constexpr std::size_t kDeepestPayloadSuffix = 140; + const auto length = cfg.mcppHome.native().size(); + if (!mcpp::platform::windows::long_paths_enabled() && length + 1 + kDeepestPayloadSuffix > 259) + mcpp::diag::degraded("home/path-budget", + std::format("MCPP_HOME is {} characters long and Windows long paths are off", length), + "installing a toolchain may fail because a path is too long", + "enable long paths (LongPathsEnabled = 1, `mcpp self doctor` shows it) or use a shorter MCPP_HOME"); + } + { + std::vector upstreams; + for (auto& dir : cfg.upstreams) + for (auto& u : mcpp::provider::layout().upstream_of(dir)) upstreams.push_back(std::move(u)); + mcpp::provider::open(make_xlings_env(cfg))->set_upstreams(upstreams); + } // 6. Acquire xlings binary if needed + const auto bundledBinary = cfg.xlingsBinary; if (cfg.xlingsBinaryMode == "bundled") { auto xbin = mcpp::fallback::acquire_xlings_binary( cfg.xlingsBinary, quiet, kXlingsPinnedVersion, @@ -893,6 +983,21 @@ std::expected load_or_init( return std::unexpected(ConfigError{std::format( "configured xlings binary not found: {}", cfg.xlingsBinary.string())}); } + // The handshake (design 2026-10-16 §12): an xlings chosen by + // `[xlings] binary` that cannot serve this mcpp gives way to the one mcpp + // ships, with a note. + if (cfg.xlingsBinaryMode != "bundled") { + if (auto why = mcpp::provider::unmet(mcpp::provider::describe(cfg.xlingsBinary)); !why.empty()) { + if (!quiet) + std::println(stderr, "note: the configured xlings {} {}; using the one mcpp ships", + cfg.xlingsBinary.string(), why); + cfg.xlingsBinary = bundledBinary; + auto xbin = mcpp::fallback::acquire_xlings_binary( + cfg.xlingsBinary, quiet, kXlingsPinnedVersion, + cfg.metaCacheDir / "vendored-xlings.versions"); + if (!xbin) return std::unexpected(ConfigError{xbin.error()}); + } + } // 7. Sandbox bootstrap (mcpp self-contained xlings environment). // Order matters: @@ -924,40 +1029,20 @@ std::expected load_or_init( // Once per process: the doctor loads the configuration more than once, // and the same sentence twice reads as two events. static bool announced = false; - auto marker = mcpp::xlings::paths::sandbox_init_marker(make_xlings_env(cfg)); - if (!announced && !std::filesystem::exists(marker)) { + if (!announced && !mcpp::provider::open(make_xlings_env(cfg))->initialized()) { announced = true; print_status("Skipping", "sandbox bootstrap (offline mode; run without --offline to bootstrap)"); } } { - auto bsEnv = make_xlings_env(cfg); #if !defined(__APPLE__) && !defined(_WIN32) // patchelf is ELF-only; macOS uses Mach-O and Windows uses PE. if (bootstrapAllowed) ensure_sandbox_patchelf(cfg, quiet, onBootstrapProgress); - // Only mark complete if the actual binary exists (not just the dir). - { - auto pBin = mcpp::xlings::paths::xim_tool(bsEnv, "patchelf", - mcpp::xlings::pinned::kPatchelfVersion) / "bin" / "patchelf"; - if (std::filesystem::exists(pBin)) - mcpp::fallback::mark_install_complete(pBin.parent_path().parent_path()); - } #endif if (bootstrapAllowed) ensure_sandbox_ninja(cfg, quiet, onBootstrapProgress); - { - auto nRoot = mcpp::xlings::paths::xim_tool_root(bsEnv, "ninja"); - auto ninja_name = std::string("ninja") + std::string(mcpp::platform::exe_suffix); - std::error_code ec; - if (std::filesystem::exists(nRoot)) { - for (auto& v : std::filesystem::directory_iterator(nRoot, ec)) { - if (std::filesystem::exists(v.path() / ninja_name)) - mcpp::fallback::mark_install_complete(v.path()); - } - } - } } // 8. Bootstrap check is NOT done here — it's deferred to commands that @@ -995,12 +1080,12 @@ bool ensure_project_index_dir( const GlobalConfig& cfg, const std::filesystem::path& projectDir, const std::map& indices, - const mcpp::xlings::ProjectEnv& penv) + const mcpp::provider::ProjectEnv& penv) { // Collect custom non-builtin indices that need xlings project-scope data. // Local path indices are also seeded so xlings can create its own // project-local repo link and install packages from that index. - std::vector customRepos; + std::vector customRepos; for (auto& [name, spec] : indices) { if (spec.is_builtin()) continue; if (spec.is_local()) { @@ -1013,7 +1098,11 @@ bool ensure_project_index_dir( // #269: pass a declared artifact source through to xlings unless a // rev/tag/branch pin forces git (the artifact channel only tracks // the latest published pointer). - mcpp::xlings::SeedRepo repo{ name, spec.url, "", "" }; + // A project's index keeps its data and payloads in the project + // (design 2026-10-16 §5.1, D4). + mcpp::provider::SeedRepo repo{ name, spec.url, "", "" }; + repo.dir = projectDir / ".mcpp" / "indices" / name; + repo.store = mcpp::provider::layout().own_store(projectDir / ".mcpp"); if (spec.artifact_applicable()) { repo.artifact = spec.artifact; repo.source = spec.source; @@ -1043,9 +1132,9 @@ bool ensure_project_index_dir( std::filesystem::create_directories(dotMcpp, ec); // Seed .xlings.json with the custom index entries. - mcpp::xlings::Env env; + mcpp::provider::Location env; env.home = dotMcpp; - mcpp::xlings::seed_xlings_json(env, customRepos, "auto", penv); + mcpp::provider::open(env)->seed(customRepos, "auto", penv); auto exposeLocalIndex = [&](const std::string& name, const std::filesystem::path& source, diff --git a/src/doctor.cppm b/src/doctor.cppm index 946d03276..eee997038 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -23,7 +23,6 @@ import mcpp.build.runtime_validation; import mcpp.config; import mcpp.fallback.probe_sysroot; import mcpp.fallback.xlings_binary; -import mcpp.fallback.install_integrity; import mcpp.fetcher.progress; import mcpp.home; import mcpp.libs.json; @@ -42,7 +41,8 @@ import mcpp.toolchain.linkmodel; import mcpp.toolchain.stdmod; import mcpp.toolchain.abi; import mcpp.ui; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; namespace mcpp::doctor { @@ -239,8 +239,8 @@ export int doctor_report() { } { std::error_code sdkEc; - auto msvcRoot = mcpp::home::root() - / "registry" / "data" / "xpkgs" / "xim-x-msvc"; + auto msvcRoot = mcpp::provider::layout().tool_in( + mcpp::provider::home_store().value_or(std::filesystem::path{}), "msvc"); for (auto& v : std::filesystem::directory_iterator(msvcRoot, sdkEc)) { if (!v.is_directory(sdkEc)) continue; auto ver = v.path().filename().string(); @@ -266,8 +266,8 @@ export int doctor_report() { mcpp::ui::status("Checking", "mingw (xim:mingw-gcc)"); { - auto pkgs = mcpp::home::root() - / "registry" / "data" / "xpkgs" / "xim-x-mingw-gcc"; + auto pkgs = mcpp::provider::layout().tool_in( + mcpp::provider::home_store().value_or(std::filesystem::path{}), "mingw-gcc"); std::error_code ec; bool any = false; if (std::filesystem::exists(pkgs, ec)) { @@ -288,8 +288,8 @@ export int doctor_report() { auto triple = std::string(mcpp::platform::host_arch) + "-linux-musl"; auto label = std::format("linux cross (xim:{}-gcc)", triple); mcpp::ui::status("Checking", label); - auto pkgs = mcpp::home::root() / "registry" / "data" / "xpkgs" - / std::format("xim-x-{}-gcc", triple); + auto pkgs = mcpp::provider::layout().tool_in( + mcpp::provider::home_store().value_or(std::filesystem::path{}), triple + "-gcc"); std::error_code ec; bool any = false; if (std::filesystem::exists(pkgs, ec)) { @@ -375,19 +375,16 @@ export int doctor_report() { // that resolves what it needs has nothing to report), so doctor is // where the state is visible without a failing run. mcpp::ui::status("Checking", "package indexes"); - const auto dataRoot = mcpp::xlings::paths::index_data( - mcpp::config::make_xlings_env(*cfg)); std::error_code idxEc; bool anyIndex = false; - for (auto const& entry : std::filesystem::directory_iterator(dataRoot, idxEc)) { - if (!entry.is_directory(idxEc)) continue; - if (!std::filesystem::exists(entry.path() / "pkgs", idxEc)) continue; + for (auto const& root : mcpp::provider::open(mcpp::config::make_xlings_env(*cfg))->index_roots()) { + if (!std::filesystem::exists(root / "pkgs", idxEc)) continue; anyIndex = true; - const auto name = entry.path().filename().string(); - auto contract = mcpp::pm::read_index_contract(entry.path()); + const auto name = root.filename().string(); + auto contract = mcpp::pm::read_index_contract(root); if (!contract || contract->minMcpp.empty()) { ok(std::format("index '{}' declares no mcpp floor", name)); - } else if (mcpp::pm::index_usable(entry.path())) { + } else if (mcpp::pm::index_usable(root)) { ok(std::format("index '{}' requires mcpp >= {}", name, contract->minMcpp)); } else { warn(std::format( @@ -398,6 +395,26 @@ export int doctor_report() { } } if (!anyIndex) ok("no package index synced yet"); + + // Where the home is, and what it reuses (design 2026-10-16 §8, §10, §11). + mcpp::ui::status("Checking", "home"); + if ((*cfg).systemHome) ok(std::format("{} is the machine's system home", (*cfg).mcppHome.string())); + if ((*cfg).upstreams.empty()) + ok("self-contained: no upstream ([xlings] upstream)"); + for (auto const& u : (*cfg).upstreams) ok(std::format("upstream {}", u.string())); + const auto caps = mcpp::platform::fs_caps((*cfg).mcppHome); + if (caps.native()) + ok(std::format("{} on {}: every write is the filesystem's own", + (*cfg).mcppHome.string(), caps.type.empty() ? "this filesystem" : caps.type)); + else + ok(std::format("{} on {}: {}{}{}", (*cfg).mcppHome.string(), caps.type.empty() ? "this filesystem" : caps.type, + caps.rename_no_replace ? "" : "publishes by hard link or a rename to a free name; ", + caps.rename_exchange ? "" : "exchanges by moving aside; ", + caps.symlinks ? "" : "no symbolic links (no reuse from an upstream)")); + if constexpr (mcpp::platform::is_windows) + ok(std::format("Windows long paths: {}", mcpp::platform::windows::long_paths_enabled() + ? "on (LongPathsEnabled = 1)" + : "off (LongPathsEnabled = 0; mcpp does not change it)")); } mcpp::ui::status("Checking", "cache health"); @@ -526,7 +543,7 @@ export int doctor_report() { // (std::filesystem::exists follows symlinks → false for dangling). mcpp::ui::status("Checking", "toolchain runtime deps"); if (cfg) { - auto pkgsDir = (*cfg).xlingsHome() / "data" / "xpkgs"; + auto pkgsDir = mcpp::provider::open(mcpp::config::make_xlings_env(*cfg))->store(); std::error_code ec; bool sawAny = false; bool anyMissing = false; @@ -534,17 +551,16 @@ export int doctor_report() { if (std::filesystem::exists(pkgsDir, ec)) { // Mirror `mcpp toolchain list`: each xim-x-//bin // holds one installed toolchain frontend (clang++/g++/musl-gcc-…). - for (auto& entry : std::filesystem::directory_iterator(pkgsDir, ec)) { - auto name = entry.path().filename().string(); - if (name.rfind("xim-x-", 0) != 0) continue; // toolchains only - auto id = mcpp::toolchain::identify_xim_payload( - name.substr(std::string("xim-x-").size())); + for (const auto& entry : mcpp::provider::layout().entries(pkgsDir)) { + if (entry.ns != "xim") continue; // toolchains only + auto id = mcpp::toolchain::identify_xim_payload(entry.name); if (!id) continue; // not a compiler pkg - for (auto& vEntry : std::filesystem::directory_iterator(entry.path(), ec)) { + { + const auto& vDir = entry.dir; mcpp::toolchain::ToolchainSpec s; s.family = id->family; - s.version = vEntry.path().filename().string(); + s.version = vDir.filename().string(); s.target = id->target; // payload_frontend, not toolchain_frontend(root/"bin"): // cl.exe is four levels down at @@ -554,7 +570,7 @@ export int doctor_report() { // in `toolchain list`; doctor kept its own copy, so the // two commands disagreed about the same machine. auto binR = mcpp::toolchain::payload_frontend( - vEntry.path(), mcpp::toolchain::to_xim_package(s)); + vDir, mcpp::toolchain::to_xim_package(s)); // DOCTOR IS THE COMMAND FOR EXACTLY THIS. A payload whose // own description does not parse is a finding, and // skipping it would make the one command that reports on @@ -623,9 +639,9 @@ export int doctor_report() { } } - // Dangling symlinks under registry/subos/default/lib — these point - // into xim payload lib dirs; a removed package leaves them broken. - auto subosLib = (*cfg).xlingsHome() / "subos" / "default" / "lib"; + // Dangling symlinks under the provider environment's lib — these + // point into payload lib dirs; a removed package leaves them broken. + auto subosLib = mcpp::provider::open(mcpp::config::make_xlings_env(*cfg))->sysroot() / "lib"; if (std::filesystem::exists(subosLib, ec)) { bool anyDangling = false; for (auto& e : std::filesystem::directory_iterator(subosLib, ec)) { @@ -654,7 +670,7 @@ export int doctor_report() { // musl-gcc payload on a Linux, macOS or Windows mcpp. mcpp::ui::status("Checking", "gcc include-fixed headers"); if (cfg) { - auto pkgsDir = (*cfg).xlingsHome() / "data" / "xpkgs"; + auto pkgsDir = mcpp::provider::open(mcpp::config::make_xlings_env(*cfg))->store(); std::error_code ec; bool sawAnyGcc = false; bool anyFrozen = false; @@ -664,23 +680,21 @@ export int doctor_report() { // directory `identify_xim_payload` recognises, filtered to the // GCC family (native, musl-gcc, mingw-(cross-)gcc, and any // `-gcc` cross payload all answer Family::Gcc). - for (auto& entry : std::filesystem::directory_iterator(pkgsDir, ec)) { - auto name = entry.path().filename().string(); - if (name.rfind("xim-x-", 0) != 0) continue; - auto id = mcpp::toolchain::identify_xim_payload( - name.substr(std::string("xim-x-").size())); + for (const auto& entry : mcpp::provider::layout().entries(pkgsDir)) { + if (entry.ns != "xim") continue; + auto id = mcpp::toolchain::identify_xim_payload(entry.name); if (!id || id->family != mcpp::toolchain::Family::Gcc) continue; - for (auto& vEntry : std::filesystem::directory_iterator(entry.path(), ec)) { - if (!vEntry.is_directory(ec)) continue; + { + const auto& vDir = entry.dir; sawAnyGcc = true; - auto findings = find_fixincludes_banners(vEntry.path()); + auto findings = find_fixincludes_banners(vDir); if (findings.empty()) continue; anyFrozen = true; mcpp::toolchain::ToolchainSpec s; s.family = id->family; - s.version = vEntry.path().filename().string(); + s.version = vDir.filename().string(); s.target = id->target; auto specStr = s.spec_str(); auto targetFlag = id->target.empty() @@ -1124,11 +1138,11 @@ export int why_toolchain_json(std::string_view target, std::string_view tcSpec) // same question of the same authority rather than re-deriving an answer // that can drift from the one the build uses. const auto lm = mcpp::toolchain::resolve_link_model(tc); - auto path_origin = [](const std::filesystem::path& p) -> std::string_view { + auto path_origin = [](const std::filesystem::path& p) -> std::string { if (p.empty()) return "none"; - const auto sp = p.generic_string(); - if (sp.find("registry/data/xpkgs") != std::string::npos) return "payload"; - if (sp.find("registry/subos") != std::string::npos) return "subos"; + const auto kind = mcpp::provider::layout().kind_of(p); + if (kind == "payload") return "payload"; + if (kind == "environment") return "subos"; // provider-boundary-ok: doctor --json's published origin value return "host"; }; const auto& srPath = @@ -1644,14 +1658,6 @@ export int self_init(bool force) { return 1; } - // Clean any incomplete xpkg installations (interrupted downloads, etc.). - auto xpkgsBase = cfg->xlingsHome() / "data" / "xpkgs"; - int cleaned = mcpp::fallback::clean_all_incomplete(xpkgsBase); - if (cleaned > 0) { - mcpp::ui::info("Cleaned", std::format( - "{} incomplete installation(s)", cleaned)); - } - // Verify result. auto problem = mcpp::config::check_base_init(*cfg); if (!problem.empty()) { @@ -1671,6 +1677,42 @@ std::string upper_ascii(std::string s) { } +// `mcpp self store dedupe [--apply]` (design 2026-10-16 §8.5): the payloads +// this home installed itself that an upstream has the same of -- namespace, +// name, version and revision, complete there -- listed; with --apply each is +// replaced by a link to the upstream's, atomically, one at a time. +export int store_dedupe(bool apply) { + auto cfg = mcpp::config::load_or_init(/*quiet=*/true); + if (!cfg) { + mcpp::ui::error(cfg.error().message); + return 4; + } + if (cfg->upstreams.empty()) { + std::println("no upstream is configured ([xlings] upstream in config.toml); nothing to compare with"); + return 0; + } + auto provider = mcpp::provider::open(mcpp::config::make_xlings_env(*cfg)); + const auto found = provider->duplicates(); + int failed = 0; + for (const auto& d : found) { + if (!apply) { + std::println("{} = {}", d.local.string(), d.upstream.string()); + continue; + } + if (auto why = provider->link_to_upstream(d); why.empty()) + std::println("linked {} -> {}", d.local.string(), d.upstream.string()); + else { + mcpp::ui::error(why); + ++failed; + } + } + if (found.empty()) std::println("no payload here is also in an upstream"); + else if (!apply) + std::println("{} payload(s) could be links to an upstream's; `mcpp self store dedupe --apply` makes them so", + found.size()); + return failed == 0 ? 0 : 1; +} + // `mcpp self config [--mirror CN|GLOBAL]` (mirror = raw option value). export int self_config(std::string mirror) { if (!mirror.empty()) { @@ -1691,7 +1733,7 @@ export int self_config(std::string mirror) { // file via xlings. // // TODO(mirror-default): the default "CN" lives in - // mcpp::xlings::seed_xlings_json — see the matching note there for the + // mcpp::provider::xlings::seed_xlings_json — see the matching note there for the // long-term plan (flip default to GLOBAL, or auto-detect on first init). auto cfg = mcpp::config::load_or_init( /*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback(), mirror); @@ -1702,11 +1744,11 @@ export int self_config(std::string mirror) { auto env = mcpp::config::make_xlings_env(*cfg); if (mirror.empty()) { - auto rc = mcpp::xlings::config_show(env); + auto rc = mcpp::provider::open(env)->show_config(); return rc == 0 ? 0 : 1; } - auto rc = mcpp::xlings::config_set_mirror(env, mirror, /*quiet=*/true); + auto rc = mcpp::provider::open(env)->set_mirror(mirror, /*quiet=*/true); if (rc != 0) { mcpp::ui::error(std::format("failed to set xlings mirror to {}", mirror)); return 1; diff --git a/src/fallback/install_integrity.cppm b/src/fallback/install_integrity.cppm deleted file mode 100644 index 2a93b01bd..000000000 --- a/src/fallback/install_integrity.cppm +++ /dev/null @@ -1,336 +0,0 @@ -// mcpp.fallback.install_integrity — unified incomplete-install detection & cleanup. -// -// Every xpkg install (toolchain, bootstrap tool, modular lib) goes through -// the same lifecycle: -// -// 1. xlings creates the xpkg directory -// 2. downloads / extracts / elfpatches -// 3. mcpp writes `.mcpp_ok` marker on success -// -// If step 2 is interrupted (Ctrl+C, network failure, kill -9), the directory -// exists but is incomplete. This module provides a single mechanism to detect -// and clean up such residue, used by: -// -// - resolve_xpkg_path() (package_fetcher.cppm) -// - ensure_patchelf/ninja() (xlings.cppm bootstrap) -// - mcpp self init (cli.cppm) -// -// Marker file: `.mcpp_ok` — written ONLY by mcpp after verified install. -// Absence of marker + directory exists = incomplete → safe to delete. -// Backward compat: packages installed before this feature have no marker; -// fall back to heuristic check (bin/ or lib/ or include/ exists). - -module; -#include - -export module mcpp.fallback.install_integrity; - -import std; -import mcpp.log; - -export namespace mcpp::fallback { - -// Marker file name written into xpkg directories after successful install. -inline constexpr std::string_view kInstallMarker = ".mcpp_ok"; - -// Did the package put anything in its own directory? -// -// THE MARKER IS ONLY AS GOOD AS THE EVIDENCE IT IS WRITTEN FROM, and for -// three releases that evidence was `exitCode == 0 && exists(verdir)` — the -// installer's own report plus a directory the installer creates before it does -// any work. mcpp#533: a package-identity collision made xlings skip the -// descriptor's `install()`, the version directory was created anyway, mcpp -// stamped it complete, and the build failed four layers later in a linker -// command line. Because the stamp is what `is_install_complete` reads, the -// wrong state then survived every subsequent build. -// -// So: at least one entry that neither mcpp nor xlings wrote. That is the -// observed signature exactly — the directory held `.mcpp_ok`, -// `.xpkg-install.json` and `mcpp_generated/` and nothing else — and it needs -// no knowledge of what the descriptor was supposed to produce. -// -// DELIBERATELY WEAK, in both directions. It cannot tell a RIGHT tree from a -// wrong one, and it must not: a package that legitimately installs no payload -// exists (xlings has type-only packages, and #531 provisions `[xlings] deps` -// for packages mcpp consumes nothing from). So the caller WITHHOLDS THE MARKER -// rather than failing — the marker means "verified", we could not verify, and -// saying so costs one cheap re-check per build because xlings short-circuits -// its own install. If the package really was needed, the build now fails at -// the empty link unit, which names the target. -bool payload_is_substantive(const std::filesystem::path& xpkgDir); - -// Check whether an xpkg directory has the .mcpp_ok marker. -// STRICT marker-only — does not fall back to legacy heuristics. -// -// …with one exception, which is what makes the mcpp#533 fix reach a machine -// that already hit the bug: a marker over a directory holding nothing but -// mcpp's and xlings' own bookkeeping is not evidence, whoever wrote it. A -// store poisoned before this check existed heals on the next build instead of -// requiring the user to know which directory to delete. -bool is_install_complete(const std::filesystem::path& xpkgDir); - -// Heuristic check for pre-.mcpp_ok packages (upgrade compat). -// Returns true if the directory looks like a complete legacy install -// based on layout (top-level bin/lib/lib64/include/share, or a single -// subdirectory containing src/ or mcpp.toml). -// Use ONLY for one-time legacy adoption or to avoid deleting old packages; -// do NOT use this to decide "skip install" on the active install path — -// that's what is_install_complete()'s strict semantics protect against. -bool looks_complete_legacy(const std::filesystem::path& xpkgDir); - -// Write the .mcpp_ok marker into a directory, marking it as complete. -void mark_install_complete(const std::filesystem::path& xpkgDir); - -// If xpkgDir exists but is incomplete, remove it entirely. -// Returns true if residue was cleaned (directory was removed). -// Returns false if directory doesn't exist or is already complete. -bool clean_incomplete_install(const std::filesystem::path& xpkgDir); - -// Scan an xpkgs base directory and clean ALL incomplete installations. -// Only cleans directories without .mcpp_ok marker AND without legacy -// content (won't delete pre-upgrade packages). -// Used by `mcpp self init`. -// Returns number of directories cleaned. -int clean_all_incomplete(const std::filesystem::path& xpkgsBase); - -// RAII rollback guard for the resolve/install path. -// -// Replaces a bare clean_incomplete_install() before a reinstall attempt. -// Instead of DELETING a no-marker directory outright (which permanently -// destroys a content-complete *legacy* package if the reinstall then fails — -// the xim-x-zlib regression), it RENAMES the directory aside and only commits -// to deleting it once the reinstall is known to have produced a good install. -// -// Usage: -// InstallStash stash(verdir); // moves verdir -> verdir.mcpp-stash (no marker) -// ... attempt reinstall into verdir ... -// stash.commit(); // reinstall produced a good install: drop backup -// // On any return/throw WITHOUT commit(), the destructor: -// // - keeps the new install if verdir now has a marker (reinstall won); -// // - else, if the stashed dir looks_complete_legacy(), restores it and -// // marks it complete (a working package was wrongly cleaned); -// // - else discards it (genuine half-extracted residue). -class InstallStash { -public: - explicit InstallStash(std::filesystem::path xpkgDir); - ~InstallStash(); - InstallStash(const InstallStash&) = delete; - InstallStash& operator=(const InstallStash&) = delete; - - // Reinstall succeeded: discard the backup, leave the live dir in place. - void commit() noexcept; - // Whether anything was actually stashed (dir existed and had no marker). - bool stashed() const noexcept { return active_; } - -private: - std::filesystem::path orig_; - std::filesystem::path stash_; - bool active_ = false; - bool committed_ = false; -}; - -} // namespace mcpp::fallback - -// ─── Implementation ───────────────────────────────────────────────── - -namespace mcpp::fallback { - -bool looks_complete_legacy(const std::filesystem::path& xpkgDir) { - if (!std::filesystem::exists(xpkgDir)) return false; - // xim toolchain/tool packages: top-level bin/lib/lib64/include/share - for (auto dir : {"bin", "lib", "lib64", "include", "share"}) { - if (std::filesystem::exists(xpkgDir / dir)) - return true; - } - // mcpplibs layout: single subdirectory containing src/ or mcpp.toml - std::error_code ec; - std::vector subs; - for (auto& e : std::filesystem::directory_iterator(xpkgDir, ec)) { - if (e.is_directory()) subs.push_back(e.path()); - } - if (subs.size() == 1) { - auto& sub = subs[0]; - if (std::filesystem::exists(sub / "src") - || std::filesystem::exists(sub / "mcpp.toml") - || std::filesystem::exists(sub / "include") - || std::filesystem::exists(sub / "bin")) - return true; - } - return false; -} - -// Strict: has .mcpp_ok marker (written only on verified success). -bool has_marker(const std::filesystem::path& xpkgDir) { - return std::filesystem::exists(xpkgDir / std::string(kInstallMarker)); -} - -bool payload_is_substantive(const std::filesystem::path& xpkgDir) { - // Everything mcpp or xlings writes into a version directory itself. Kept - // here, beside `kInstallMarker`, because that is the list's subject — and - // an entry added by either tool has to be added here or this predicate - // starts calling an empty install substantive again. - // - // .mcpp_ok mcpp's completeness marker (kInstallMarker) - // .xpkg-install.json xlings' install record - // mcpp_generated/ mcpp's `generated_files` synthesis - static constexpr std::string_view kSelfWritten[] = { - kInstallMarker, ".xpkg-install.json", "mcpp_generated", - }; - std::error_code ec; - if (!std::filesystem::is_directory(xpkgDir, ec)) return false; - for (auto const& e : std::filesystem::directory_iterator(xpkgDir, ec)) { - auto name = e.path().filename().string(); - bool self = false; - for (auto s : kSelfWritten) if (name == s) { self = true; break; } - if (!self) return true; - } - return false; -} - -bool is_install_complete(const std::filesystem::path& xpkgDir) { - if (!std::filesystem::exists(xpkgDir)) return false; - // The marker is necessary but not sufficient — see payload_is_substantive. - // This is the arm that heals a store poisoned by mcpp#533 before the fix - // existed: the stale `.mcpp_ok` stops being believed, the install is - // retried, and nobody has to be told which directory to remove. - if (std::filesystem::exists(xpkgDir / std::string(kInstallMarker)) - && !payload_is_substantive(xpkgDir)) { - mcpp::log::verbose("integrity", std::format( - "{}: marked complete but holds nothing the package installed — " - "treating as incomplete", xpkgDir.string())); - return false; - } - - // STRICT marker-only. - // Used on the install/resolve path — half-extracted dirs with bin/ - // would otherwise be mistaken for complete packages. - // - // Legacy packages (installed before .mcpp_ok existed) will trigger - // a one-time reinstall after upgrade. This is the cost of strict - // semantics; the alternative (legacy heuristic) shields half-extracted - // packages from cleanup and re-introduces the very bug we're fixing. - // The reinstall is cheap because copy_xpkg_from_global() is the - // typical fallback path — it reuses the existing ~/.xlings/ copy. - return has_marker(xpkgDir); -} - -void mark_install_complete(const std::filesystem::path& xpkgDir) { - auto marker = xpkgDir / std::string(kInstallMarker); - if (std::filesystem::exists(marker)) return; - std::ofstream ofs(marker); - if (ofs) ofs << "1\n"; -} - -bool clean_incomplete_install(const std::filesystem::path& xpkgDir) { - if (!std::filesystem::exists(xpkgDir)) return false; - - // STRICT marker-only semantics. - // Used on the resolve/install path for the CURRENT target: we know - // mcpp just attempted to install this package, so absence of .mcpp_ok - // unambiguously means the attempt was incomplete (interrupted, failed - // mid-extract, etc.). Legacy heuristic compat does NOT apply here — - // a half-extracted dir that happens to have a `bin/` would otherwise - // escape cleanup and corrupt subsequent installs. - if (has_marker(xpkgDir)) return false; - - mcpp::log::verbose("integrity", - std::format("cleaning incomplete install: {}", xpkgDir.string())); - std::error_code ec; - std::filesystem::remove_all(xpkgDir, ec); - return !ec; -} - -int clean_all_incomplete(const std::filesystem::path& xpkgsBase) { - if (!std::filesystem::exists(xpkgsBase)) return 0; - - // Global scan (used by `mcpp self init`). Keeps legacy packages - // (no marker but has content) for backward compatibility — those - // were installed before the marker system existed. - int cleaned = 0; - std::error_code ec; - for (auto& pkgDir : std::filesystem::directory_iterator(xpkgsBase, ec)) { - if (!pkgDir.is_directory()) continue; - for (auto& verDir : std::filesystem::directory_iterator(pkgDir.path(), ec)) { - if (!verDir.is_directory()) continue; - if (has_marker(verDir.path())) continue; - if (looks_complete_legacy(verDir.path())) { - mcpp::log::debug("integrity", std::format( - "legacy package without marker, kept: {}", - verDir.path().string())); - continue; - } - std::filesystem::remove_all(verDir.path(), ec); - if (!ec) ++cleaned; - } - } - return cleaned; -} - -// ─── InstallStash ─────────────────────────────────────────────────── - -InstallStash::InstallStash(std::filesystem::path xpkgDir) - : orig_(std::move(xpkgDir)) -{ - std::error_code ec; - if (!std::filesystem::exists(orig_, ec)) return; // nothing to stash - if (has_marker(orig_)) return; // complete: leave untouched - - stash_ = orig_; - stash_ += ".mcpp-stash"; - std::filesystem::remove_all(stash_, ec); // clear any prior stash residue - std::filesystem::rename(orig_, stash_, ec); - if (ec) { - // Rename failed (e.g. cross-device); fall back to the historical - // delete-then-reinstall semantics so the caller still starts clean. - std::error_code rmec; - std::filesystem::remove_all(orig_, rmec); - stash_.clear(); - return; - } - active_ = true; -} - -void InstallStash::commit() noexcept { - committed_ = true; - if (!active_) return; - std::error_code ec; - std::filesystem::remove_all(stash_, ec); - active_ = false; -} - -InstallStash::~InstallStash() { - if (!active_ || committed_) return; - std::error_code ec; - - // Reinstall produced a complete install at orig_ — keep it, drop backup. - if (std::filesystem::exists(orig_, ec) && has_marker(orig_)) { - std::filesystem::remove_all(stash_, ec); - return; - } - - if (looks_complete_legacy(stash_)) { - // A content-complete legacy package (no marker) was cleaned for a - // reinstall that then failed. Roll it back so we never leave a - // working package deleted, and adopt it (write the marker) so the - // next resolve trusts it instead of repeating the delete-then-fail. - std::filesystem::remove_all(orig_, ec); // clear partial residue - std::error_code rnec; - std::filesystem::rename(stash_, orig_, rnec); - if (!rnec) { - mark_install_complete(orig_); - mcpp::log::verbose("integrity", std::format( - "rolled back stashed legacy install after failed reinstall: {}", - orig_.string())); - } else { - mcpp::log::verbose("integrity", std::format( - "failed to roll back stashed install {} -> {}: {}", - stash_.string(), orig_.string(), rnec.message())); - } - } else { - // Genuine interrupted residue — discard (historical semantics). - std::filesystem::remove_all(stash_, ec); - } -} - - -} // namespace mcpp::fallback diff --git a/src/fallback/probe_sysroot.cppm b/src/fallback/probe_sysroot.cppm index dc91adb90..3436ac9b6 100644 --- a/src/fallback/probe_sysroot.cppm +++ b/src/fallback/probe_sysroot.cppm @@ -11,7 +11,8 @@ module; export module mcpp.fallback.probe_sysroot; import std; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.platform; import mcpp.log; @@ -51,12 +52,12 @@ std::optional remap_xlings_baked_sysroot(std::string_view reportedPath, const std::filesystem::path& compilerBin) { if (reportedPath.empty()) return std::nullopt; - if (!reportedPath.ends_with("subos/default")) return std::nullopt; + if (!mcpp::provider::layout().is_default_environment(reportedPath)) return std::nullopt; - auto xpkgsOpt = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin); + auto xpkgsOpt = mcpp::provider::store_of(compilerBin); if (xpkgsOpt) { // Owned by this toolchain's registry? Then it is the right answer. - auto registry = xpkgsOpt->parent_path().parent_path(); + auto registry = mcpp::provider::layout().home_of_store(*xpkgsOpt); std::error_code ec; const bool inside = path_is_under( std::filesystem::path(std::string(reportedPath)), registry); @@ -64,10 +65,8 @@ remap_xlings_baked_sysroot(std::string_view reportedPath, return std::nullopt; } - if (auto xpkgs = std::move(xpkgsOpt)) { - // xpkgs is /data/xpkgs -> registry = xpkgs/../.. - auto registrySysroot = xpkgs->parent_path().parent_path() - / "subos" / "default"; + if (auto environment = mcpp::provider::environment_of(compilerBin)) { + auto registrySysroot = *environment; if (std::filesystem::exists(registrySysroot / "usr" / "include")) return registrySysroot; } @@ -81,11 +80,11 @@ remap_xlings_baked_sysroot(std::string_view reportedPath, bool sysroot_is_owned(std::string_view reportedPath, const std::filesystem::path& compilerBin) { if (reportedPath.empty()) return false; - auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin); + auto xpkgs = mcpp::provider::store_of(compilerBin); // No registry to compare against -- nothing to contradict, so accept. if (!xpkgs) return true; return path_is_under(std::filesystem::path(std::string(reportedPath)), - xpkgs->parent_path().parent_path()); + mcpp::provider::layout().home_of_store(*xpkgs)); } // Is this sysroot foreign -- neither this mcpp home's registry nor a tree diff --git a/src/fallback/xpkg_copy.cppm b/src/fallback/xpkg_copy.cppm deleted file mode 100644 index 8afa5d273..000000000 --- a/src/fallback/xpkg_copy.cppm +++ /dev/null @@ -1,62 +0,0 @@ -// mcpp.fallback.xpkg_copy — copy xpkg payload from global ~/.xlings/ to sandbox. -// -// Workaround: xlings may extract large packages (e.g. LLVM) into its -// global data dir instead of the mcpp sandbox, because the extraction -// subprocess doesn't always inherit XLINGS_HOME. This module detects -// the global copy and mirrors it into the sandbox so mcpp remains -// self-contained. - -module; -#include - -export module mcpp.fallback.xpkg_copy; - -import std; -import mcpp.log; - -export namespace mcpp::fallback { - -// Try to copy an xpkg verdir from the global xlings home into the -// sandbox. Returns true if the copy succeeded (verdir now exists). -bool copy_xpkg_from_global(const std::filesystem::path& sandboxVerdir) { - if (std::filesystem::exists(sandboxVerdir)) return true; - - mcpp::log::verbose("fallback", "verdir not in sandbox, checking global xlings"); - - const char* xhome = nullptr; -#if defined(_WIN32) - xhome = std::getenv("USERPROFILE"); -#endif - if (!xhome) xhome = std::getenv("HOME"); - if (!xhome) return false; - - mcpp::log::debug("fallback", std::format("HOME={}", xhome)); - - // xlings stores xpkgs at /.xlings/data/xpkgs/ or - // /.xlings/subos/default/data/xpkgs/ - auto pkgDir = sandboxVerdir.parent_path().filename().string(); - auto verName = sandboxVerdir.filename().string(); - std::filesystem::path candidates[] = { - std::filesystem::path(xhome) / ".xlings" / "data" / "xpkgs" / pkgDir / verName, - std::filesystem::path(xhome) / ".xlings" / "subos" / "default" / "data" / "xpkgs" / pkgDir / verName, - }; - - for (auto& src : candidates) { - std::error_code ec; - bool srcExists = std::filesystem::exists(src, ec) && std::filesystem::is_directory(src, ec); - mcpp::log::debug("fallback", std::format( - "candidate '{}' exists={}", src.string(), srcExists)); - if (srcExists) { - std::filesystem::create_directories(sandboxVerdir.parent_path(), ec); - std::filesystem::copy(src, sandboxVerdir, - std::filesystem::copy_options::recursive - | std::filesystem::copy_options::overwrite_existing, ec); - mcpp::log::verbose("fallback", std::format( - "copied from global xlings: ec={}", ec.message())); - if (!ec) return true; - } - } - return false; -} - -} // namespace mcpp::fallback diff --git a/src/home.cppm b/src/home.cppm index 0f1b5cb07..b5ab05e1d 100644 --- a/src/home.cppm +++ b/src/home.cppm @@ -22,6 +22,23 @@ import mcpp.platform; export namespace mcpp::home { +// The system-level configuration (design 2026-10-16 §8.1, §9): what an +// administrator states for every user of the machine -- upstreams, a default +// toolchain, the system home. $MCPP_SYSTEM_CONFIG when set; else +// /etc/mcpp/config.toml (Windows: %ProgramData%\mcpp\config.toml). +std::filesystem::path system_config_file(); + +// The system home (§9.1): `[system] home` of the system configuration; else +// /var/lib/mcpp (macOS: /Library/Application Support/mcpp; Windows: +// %ProgramData%\mcpp). Written only by `mcpp --system`; read by every user. +std::filesystem::path system_root(); + +// The marker a system home carries (`.mcpp-home`, mode = "system"), and +// whether `home` has it. +inline constexpr std::string_view kHomeMarker = ".mcpp-home"; +bool is_system_home(const std::filesystem::path& home); + + // $MCPP_HOME. See root() below for the resolution order. std::filesystem::path root(); @@ -93,7 +110,7 @@ std::filesystem::path root() { p.has_parent_path() && p != p.root_path(); p = p.parent_path()) { if (p.filename() == "target") { disqualified = true; break; } - if (p.filename() == "xpkgs") { + if (p.filename() == "xpkgs") { // provider-boundary-ok: mcpp.home sits below the port (the adapter imports it) auto parent = p.parent_path().filename().string(); if (parent == "data") { disqualified = true; break; } } @@ -105,6 +122,56 @@ std::filesystem::path root() { return default_home(); } +std::filesystem::path system_config_file() { + if (auto* e = std::getenv("MCPP_SYSTEM_CONFIG"); e && *e) return std::filesystem::path(e); + if constexpr (mcpp::platform::is_windows) { + auto* data = std::getenv("ProgramData"); + return std::filesystem::path(data && *data ? data : "C:\\ProgramData") / "mcpp" / "config.toml"; + } + return "/etc/mcpp/config.toml"; +} + +namespace { +// `key = "value"` under `[table]` of a small TOML file, read without a TOML +// reader (this module stays a leaf): the system configuration's `[system] +// home`, the marker's `mode`. Empty when absent. +std::string toml_string(const std::filesystem::path& file, std::string_view table, std::string_view key) { + std::ifstream in(file); + std::string line, current; + while (std::getline(in, line)) { + auto t = std::string_view(line); + while (!t.empty() && (t.front() == ' ' || t.front() == '\t')) t.remove_prefix(1); + if (t.empty() || t.front() == '#') continue; + if (t.front() == '[') { + current = std::string(t.substr(1, t.find(']') == std::string_view::npos ? 0 : t.find(']') - 1)); + continue; + } + if (current != table || !t.starts_with(key)) continue; + t.remove_prefix(key.size()); + while (!t.empty() && (t.front() == ' ' || t.front() == '\t')) t.remove_prefix(1); + if (t.empty() || t.front() != '=') continue; + const auto open = t.find('"'), close = t.find('"', open + 1); + if (open == std::string_view::npos || close == std::string_view::npos) continue; + return std::string(t.substr(open + 1, close - open - 1)); + } + return {}; +} +} // namespace + +std::filesystem::path system_root() { + if (auto h = toml_string(system_config_file(), "system", "home"); !h.empty()) return std::filesystem::path(h); + if constexpr (mcpp::platform::is_windows) { + auto* data = std::getenv("ProgramData"); + return std::filesystem::path(data && *data ? data : "C:\\ProgramData") / "mcpp"; + } + if constexpr (mcpp::platform::is_macos) return "/Library/Application Support/mcpp"; + return "/var/lib/mcpp"; +} + +bool is_system_home(const std::filesystem::path& home) { + return toml_string(home / std::string(kHomeMarker), "", "mode") == "system"; +} + std::filesystem::path cache_root() { return root() / "build-cache" / "v1"; } diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index c8a9185ff..71114cc95 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -48,7 +48,8 @@ import mcpp.pack.stage_tree; import mcpp.pack.strip; import mcpp.pack.zip; import mcpp.platform; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.manifest; import mcpp.toolchain.triple; @@ -848,11 +849,10 @@ ldd_parse(const std::filesystem::path& binary) std::filesystem::path sandbox_patchelf(const mcpp::config::GlobalConfig& cfg) { auto env = mcpp::config::make_xlings_env(cfg); - auto bin = mcpp::xlings::paths::xim_tool(env, "patchelf", - mcpp::xlings::pinned::kPatchelfVersion) / "bin" / "patchelf"; + auto bin = mcpp::provider::open(env)->pinned("patchelf", mcpp::provider::kPatchelfVersion) / "bin" / "patchelf"; if (std::filesystem::exists(bin)) return bin; // Fallback: scan all versions (in case a different version is installed). - auto root = mcpp::xlings::paths::xim_tool_root(env, "patchelf"); + auto root = mcpp::provider::open(env)->versions("patchelf"); std::error_code ec; if (!std::filesystem::exists(root, ec)) return {}; for (auto& v : std::filesystem::directory_iterator(root, ec)) { diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 7e940e141..a384ef9ac 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -33,7 +33,8 @@ import mcpp.toolchain.registry; import mcpp.toolchain.triple; import mcpp.log; import mcpp.ui; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; namespace mcpp::pack { @@ -98,7 +99,7 @@ struct DriverLibraryDirs { DriverLibraryDirs driver_library_dirs(const mcpp::toolchain::Toolchain& tc) { DriverLibraryDirs out; - const auto base = std::format("{} {}", mcpp::xlings::shq(tc.binaryPath.string()), + const auto base = std::format("{} {}", mcpp::platform::shell::quote(tc.binaryPath.string()), tc.crossTargetFlag); if (auto r = mcpp::toolchain::run_capture(std::format( "{} -print-search-dirs {}", base, mcpp::platform::null_redirect))) { diff --git a/src/pm/commands.cppm b/src/pm/commands.cppm index 5997228a5..413eb4794 100644 --- a/src/pm/commands.cppm +++ b/src/pm/commands.cppm @@ -25,7 +25,8 @@ import mcpp.pm.index_refresh; // shared refresh policy (with mcpp.build.prep import mcpp.pm.resolver; // is_version_constraint import mcpp.project; // shared find_manifest_root import mcpp.ui; -import mcpp.xlings; // index freshness +import mcpp.provider; +import mcpp.provider.select; // index freshness import mcpplibs.cmdline; namespace mcpp::pm::commands::detail { diff --git a/src/pm/index_management.cppm b/src/pm/index_management.cppm index d06d4eb96..47d14b764 100644 --- a/src/pm/index_management.cppm +++ b/src/pm/index_management.cppm @@ -18,7 +18,8 @@ import mcpp.platform; import mcpp.project; import mcpp.pm.index_route; import mcpp.ui; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; namespace mcpp::pm { @@ -43,9 +44,9 @@ export int search_packages(const std::string& keyword, bool allVersions) { // Offline is still honoured: update_index() returns without touching the // network, and search reports whatever the local copy holds. auto xlEnv = mcpp::config::make_xlings_env(*cfg); - if (!mcpp::xlings::is_index_fresh(xlEnv, cfg->searchTtlSeconds)) { + if (!mcpp::provider::open(xlEnv)->index_fresh(cfg->searchTtlSeconds)) { mcpp::ui::status("Updating", "package index (auto-refresh)"); - mcpp::xlings::ensure_index_fresh(xlEnv, cfg->searchTtlSeconds, /*quiet=*/true); + mcpp::provider::open(xlEnv)->ensure_index_fresh(cfg->searchTtlSeconds, /*quiet=*/true); } mcpp::fetcher::Fetcher f(*cfg); @@ -170,7 +171,7 @@ export int index_update(const std::string& filterName) { // Update global index repos. mcpp::ui::status("Updating", "all index repos"); auto xlEnv = mcpp::config::make_xlings_env(*cfg); - int rc = mcpp::xlings::update_index(xlEnv); + int rc = mcpp::provider::open(xlEnv)->update_index(); if (rc != 0) { mcpp::ui::error("index update failed"); return 1; } // Also update project-level custom indices if present. @@ -193,7 +194,7 @@ export int index_update(const std::string& filterName) { // Re-sync the project-level clone via xlings. mcpp::config::ensure_project_index_dir(*cfg, *root, m->indices); auto projEnv = mcpp::config::make_project_xlings_env(*cfg, *root); - int prc = mcpp::xlings::update_index(projEnv); + int prc = mcpp::provider::open(projEnv)->update_index(); if (prc != 0) { mcpp::ui::error(std::format("project index `{}` update failed", idxName)); } else { @@ -223,7 +224,7 @@ export int index_status() { if (s < 172800) return std::format("{}h ago", s / 3600); return std::format("{}d ago", s / 86400); }; - auto show = [&](const char* label, const mcpp::xlings::IndexStatus& st) { + auto show = [&](const char* label, const mcpp::provider::IndexStatus& st) { std::string state = !st.present ? "missing" : st.fresh ? "fresh" : "stale"; @@ -235,8 +236,8 @@ export int index_status() { st.rev.value_or("-"), st.dir.string()); }; - auto official = mcpp::xlings::official_index_status(xlEnv, cfg->searchTtlSeconds); - auto deflt = mcpp::xlings::default_index_status(xlEnv, cfg->searchTtlSeconds); + auto official = mcpp::provider::open(xlEnv)->official_index_status(cfg->searchTtlSeconds); + auto deflt = mcpp::provider::open(xlEnv)->default_index_status(cfg->searchTtlSeconds); std::println(""); std::println(" {:<10} {:<8} {:<12} {:<12} {}", diff --git a/src/pm/index_refresh.cppm b/src/pm/index_refresh.cppm index e3c2efc81..d417f684a 100644 --- a/src/pm/index_refresh.cppm +++ b/src/pm/index_refresh.cppm @@ -47,7 +47,8 @@ import mcpp.pm.index_route; export import mcpp.pm.refresh_policy; // the policy half, shared with the fetcher import mcpp.pm.resolver; import mcpp.ui; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; export namespace mcpp::pm { @@ -56,7 +57,7 @@ export namespace mcpp::pm { RefreshDecision decide_for_dependency(const IndexRoute& route, std::string_view depKey, const DependencySpec& spec, - const mcpp::xlings::Env& env, + const mcpp::provider::Location& env, const mcpp::platform::PlatformKey& platform, const RefreshPolicy& policy); @@ -113,7 +114,7 @@ std::vector coords_of(std::string_view depKey, RefreshDecision decide_for_dependency(const IndexRoute& route, std::string_view depKey, const DependencySpec& spec, - const mcpp::xlings::Env& env, + const mcpp::provider::Location& env, const mcpp::platform::PlatformKey& platform, const RefreshPolicy& policy) { @@ -155,7 +156,7 @@ RefreshDecision decide_for_dependency(const IndexRoute& route, return d; } - auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); + auto status = mcpp::provider::open(env)->default_index_status(policy.debounceSeconds); // 4-6. The three ways local state fails to answer. // diff --git a/src/pm/index_snapshot.cppm b/src/pm/index_snapshot.cppm deleted file mode 100644 index f6fcab1d8..000000000 --- a/src/pm/index_snapshot.cppm +++ /dev/null @@ -1,359 +0,0 @@ -// mcpp.pm.index_snapshot — local, known-good copies of index trees, and the -// guard that makes a refresh MONOTONE. -// -// THE INVARIANT -// -// An index-side change must never take mcpp from "works" to "does not work". -// -// It did not hold. An index tree can declare a client-version floor -// (`index.toml` `min_mcpp`, mcpp.pm.index_contract). When the published index -// raises that floor, `xlings update` replaces the local tree in place with one -// this binary cannot read, every descriptor read starts returning nothing, and -// the build dies — on a machine that was working sixty seconds earlier. There -// was no backup, so there was no way back either: the refresh is what broke it. -// -// The 2026-07-08 index design specified the cure ("staged refresh keeps the -// last compatible snapshot") and it was never implemented — the floor was -// checked in exactly one place, the descriptor reader, and the refresh path -// knew nothing about it. -// -// WHY BACKUP-AND-ROLLBACK RATHER THAN STAGE-AND-SWAP -// -// mcpp does not fetch indexes. `mcpp::xlings::update_index` shells out to -// `xlings update`, which rewrites the tree in place; mcpp cannot ask it to -// unpack somewhere else (the `index_repos` entry it seeds carries name / url / -// artifact / source and no version or destination). So the sequence is -// archive → let the refresh happen → judge the result → restore if it got -// worse. Same guarantee, no cross-repo dependency. -// -// The judgement is deliberately "did it get WORSE", not "is it good": -// -// usable before && !usable after → restore -// -// A tree that was already unusable has nothing to roll back to, and restoring -// an equally-unusable snapshot over a fresh one would only make the next -// refresh redo the same work. -// -// WHY KEEP MORE THAN ONE -// -// One backup answers "undo the refresh that just happened". A short history -// answers "give me the newest snapshot this binary can actually read", which -// is the same question the publishing side would answer with a version -// negotiation protocol (openxlings/xlings#476) — locally, for free, and today. -// It only covers snapshots this machine has seen, which is the overwhelmingly -// common case for a development box or a CI runner with a warm cache; a fresh -// install that meets a too-new index has no local history and correctly falls -// through to "upgrade mcpp". -// -// COST -// -// Index trees are small (measured: 912 KB for mcpplibs, 2.2 MB for -// xim-pkgindex) and archiving uses hard links where the filesystem allows it, -// so a snapshot costs approximately one directory walk and no extra bytes. - -export module mcpp.pm.index_snapshot; - -import std; -import mcpp.pm.index_contract; - -export namespace mcpp::pm::index_snapshot { - -// How many known-good snapshots to keep per index tree. Small on purpose: the -// value of the history is "the last few floors", and floors move rarely (the -// mcpp index has changed its floor 6 times in its entire life). -inline constexpr std::size_t kKeepPerIndex = 5; - -// The snapshot store is a SIBLING of the index data root, never inside it: -// -// /.. index-snapshots/// -// -// Everything that enumerates index repos does it by listing directories under -// `data/` (mcpp's own Fetcher::sorted_index_dirs takes every subdirectory, and -// xlings walks the same tree). A snapshot store living there would be handed to -// those scanners as if it were an index. Today that happens to be harmless — -// the store has no `pkgs/` so lookups miss and move on — but "harmless because -// of a detail of someone else's loop" is not a property worth depending on, -// and it costs nothing to put the store where no index scanner can reach it. -std::filesystem::path snapshots_root(const std::filesystem::path& dataRoot); -std::filesystem::path snapshot_dir(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir); - -// Identity of the tree currently in `indexDir`, taken from the marker xlings -// writes (`.xlings-index-version`). OPAQUE BY CONTRACT — a short sha for the -// artifact transport, a version string for others; never parsed, only compared -// and used as a directory name. Empty when the marker is absent, in which case -// the caller falls back to a content-independent name. -std::string snapshot_id(const std::filesystem::path& indexDir); - -// Archive `indexDir` under the snapshots root. No-op (returns false) when the -// tree is unusable — a snapshot exists to be restored, and restoring an -// unusable tree helps nobody. Overwrites an existing snapshot with the same id. -bool archive(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir); - -// Restore a specific snapshot over `indexDir`. The destination is replaced. -bool restore(const std::filesystem::path& snapshotPath, - const std::filesystem::path& indexDir); - -// Newest-first list of archived snapshots for one index tree. -std::vector -list_snapshots(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir); - -// The newest archived snapshot this binary can actually read, if any. -std::optional -newest_usable(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir); - -// Drop all but the newest `keep` snapshots of one index tree. -void prune(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir, - std::size_t keep = kKeepPerIndex); - -// Index trees directly under `dataRoot`: a directory containing `pkgs/`. -std::vector -index_dirs(const std::filesystem::path& dataRoot); - -// What a guarded refresh did, for the caller to report. -struct GuardOutcome { - // Index trees that were usable before the refresh and unusable after, and - // were therefore rolled back. - std::vector rolledBack; - // Index trees restored from an older snapshot because the refreshed tree - // was unusable and no pre-refresh tree existed to keep. - std::vector recovered; - // Index trees left unusable — nothing local could serve them. - std::vector stillUnusable; - // The `min_mcpp` of each refreshed tree found unusable, read before a - // rollback replaces it, so a notice can name the version it asks for. - std::map requiredMcpp; - - bool degraded() const { - return !rolledBack.empty() || !recovered.empty() || !stillUnusable.empty(); - } -}; - -// Run `refresh` with the monotonicity guarantee over every index tree under -// `dataRoot`. Returns the refresh's own exit code; `out` describes what the -// guard had to do. -// -// `refresh` is a callable so this module stays free of any dependency on the -// xlings process layer — and so it is testable without a network. -int guarded_refresh(const std::filesystem::path& dataRoot, - const std::function& refresh, - GuardOutcome& out); - -} // namespace mcpp::pm::index_snapshot - -namespace mcpp::pm::index_snapshot { - -namespace { - -// A REAL copy. Not hard links. -// -// Hard links were the obvious optimisation — an index tree is thousands of -// small files and a snapshot that costs real bytes is a reason not to take one -// — and they are wrong here, which the round-trip test caught immediately: a -// hard-linked snapshot ALIASES the live tree. Anything that rewrites an index -// file in place (a truncating write, a tar extraction over an existing path) -// writes straight through the link and destroys the backup. The one event this -// module exists to survive is precisely "something replaced the tree", so a -// snapshot that shares inodes with the tree is not a snapshot at all. -// -// The bytes are cheap in absolute terms (measured: 912 KB for mcpplibs, 2.2 MB -// for xim-pkgindex; five kept snapshots ≈ 15 MB) and `prune` bounds the total. -bool copy_tree(const std::filesystem::path& from, - const std::filesystem::path& to) -{ - std::error_code ec; - std::filesystem::remove_all(to, ec); - ec.clear(); - std::filesystem::create_directories(to.parent_path(), ec); - - ec.clear(); - std::filesystem::copy(from, to, - std::filesystem::copy_options::recursive - | std::filesystem::copy_options::overwrite_existing - | std::filesystem::copy_options::skip_symlinks, ec); - return !ec; -} - -// Directory mtime, as a sortable integer. Only ever compared against other -// snapshots of the same index, so file_clock's non-Unix epoch is irrelevant. -std::int64_t mtime_of(const std::filesystem::path& p) { - std::error_code ec; - auto t = std::filesystem::last_write_time(p, ec); - if (ec) return 0; - return t.time_since_epoch().count(); -} - -std::string sanitize_component(std::string s) { - for (auto& c : s) { - const bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') - || (c >= '0' && c <= '9') || c == '.' || c == '-' || c == '_'; - if (!ok) c = '_'; - } - if (s.empty()) s = "unknown"; - return s; -} - -} // namespace - -std::filesystem::path snapshots_root(const std::filesystem::path& dataRoot) { - auto parent = dataRoot.parent_path(); - // Degenerate path (relative "data", or a root) — fall back to staying put - // rather than climbing out of the home directory. - if (parent.empty()) return dataRoot / ".index-snapshots"; - return parent / "index-snapshots"; -} - -std::filesystem::path snapshot_dir(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir) { - return snapshots_root(dataRoot) / indexDir.filename(); -} - -std::string snapshot_id(const std::filesystem::path& indexDir) { - std::ifstream is(indexDir / ".xlings-index-version", std::ios::binary); - if (!is) return {}; - std::string v{std::istreambuf_iterator(is), {}}; - while (!v.empty() && (v.back() == '\n' || v.back() == '\r' || v.back() == ' ')) - v.pop_back(); - return sanitize_component(std::move(v)); -} - -std::vector -index_dirs(const std::filesystem::path& dataRoot) { - std::vector out; - std::error_code ec; - if (!std::filesystem::is_directory(dataRoot, ec)) return out; - for (auto& e : std::filesystem::directory_iterator(dataRoot, ec)) { - if (ec) break; - if (!e.is_directory()) continue; - // The store is a sibling of dataRoot (see snapshots_root), so it - // should never appear here. Skipped anyway: this loop decides what - // gets archived, and an archive of the archive is the one mistake - // that would grow without bound. - if (e.path().filename() == ".index-snapshots") continue; - std::error_code pec; - if (!std::filesystem::is_directory(e.path() / "pkgs", pec)) continue; - out.push_back(e.path()); - } - std::ranges::sort(out); - return out; -} - -bool archive(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir) -{ - if (!mcpp::pm::index_usable(indexDir)) return false; - - auto id = snapshot_id(indexDir); - if (id.empty()) id = "unversioned"; - auto dst = snapshot_dir(dataRoot, indexDir) / id; - - // Same id already archived: the tree did not move, so the existing - // snapshot is byte-equivalent and re-copying it buys nothing. - std::error_code ec; - if (std::filesystem::is_directory(dst, ec) && id != "unversioned") - return true; - - return copy_tree(indexDir, dst); -} - -bool restore(const std::filesystem::path& snapshotPath, - const std::filesystem::path& indexDir) -{ - std::error_code ec; - if (!std::filesystem::is_directory(snapshotPath, ec)) return false; - return copy_tree(snapshotPath, indexDir); -} - -std::vector -list_snapshots(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir) -{ - std::vector out; - auto base = snapshot_dir(dataRoot, indexDir); - std::error_code ec; - if (!std::filesystem::is_directory(base, ec)) return out; - for (auto& e : std::filesystem::directory_iterator(base, ec)) { - if (ec) break; - if (e.is_directory()) out.push_back(e.path()); - } - std::ranges::sort(out, [](auto& a, auto& b) { - return mtime_of(a) > mtime_of(b); // newest first - }); - return out; -} - -std::optional -newest_usable(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir) -{ - for (auto& s : list_snapshots(dataRoot, indexDir)) - if (mcpp::pm::index_usable(s)) return s; - return std::nullopt; -} - -void prune(const std::filesystem::path& dataRoot, - const std::filesystem::path& indexDir, - std::size_t keep) -{ - auto snaps = list_snapshots(dataRoot, indexDir); - for (std::size_t i = keep; i < snaps.size(); ++i) { - std::error_code ec; - std::filesystem::remove_all(snaps[i], ec); - } -} - -int guarded_refresh(const std::filesystem::path& dataRoot, - const std::function& refresh, - GuardOutcome& out) -{ - // Snapshot every tree that is currently readable. Doing this BEFORE the - // refresh is the whole point: afterwards the old bytes are gone. - auto before = index_dirs(dataRoot); - std::map usableBefore; - for (auto& dir : before) { - usableBefore[dir] = mcpp::pm::index_usable(dir); - if (usableBefore[dir]) { - archive(dataRoot, dir); - prune(dataRoot, dir); - } - } - - const int rc = refresh(); - - // Judge each tree. `index_dirs` is re-read: a refresh may add a repo. - for (auto& dir : index_dirs(dataRoot)) { - if (mcpp::pm::index_usable(dir)) { - // Improved or unchanged — record the new good state for next time. - archive(dataRoot, dir); - prune(dataRoot, dir); - continue; - } - if (auto c = mcpp::pm::read_index_contract(dir)) - out.requiredMcpp[dir] = c->minMcpp; - auto wasUsable = usableBefore.find(dir); - if (wasUsable != usableBefore.end() && wasUsable->second) { - // Got worse. This is the case the invariant exists for. - if (auto snap = newest_usable(dataRoot, dir); - snap && restore(*snap, dir)) { - out.rolledBack.push_back(dir); - continue; - } - } - // Was already unusable (or the rollback failed): try the local history - // anyway — it may hold a readable tree from before this binary ever - // met the raised floor. - if (auto snap = newest_usable(dataRoot, dir); - snap && restore(*snap, dir)) { - out.recovered.push_back(dir); - continue; - } - out.stillUnusable.push_back(dir); - } - return rc; -} - -} // namespace mcpp::pm::index_snapshot diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index 923e3b1e9..1f727f8dd 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -1,10 +1,10 @@ -// mcpp.pm.package_fetcher — subprocess-based xlings interface client (NDJSON over stdio). +// mcpp.pm.package_fetcher — packages through the provider port: search, +// plan, install, the index list, a package's descriptor and its payload. // -// Each Fetcher::call() invokes: -// XLINGS_HOME= interface --args '{...}' -// then parses the NDJSON event stream emitted on stdout. -// -// Wire format reference: xlings/src/interface.cppm. +// Each operation goes to the provider (mcpp.provider.select): which program +// runs, its protocol and its layout are the adapter's. A payload's +// completeness is the provider's answer (its `payload_status`, design §7.1): +// mcpp writes nothing into a payload -- no marker, no copy from another home. module; #include @@ -24,11 +24,9 @@ import mcpp.pm.refresh_policy; // decide_for_miss / apply (#648 A5) import mcpp.build.refusal; // offline-download-required (#648 A1) import mcpp.ui; import mcpp.pm.index_spec; -import mcpp.xlings; -import mcpp.libs.toml; // re-used for tiny JSON-ish parsing? no — stick with manual -import mcpp.fallback.xpkg_copy; -import mcpp.fallback.install_integrity; -import mcpp.fallback.legacy_dirs; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.libs.json; export namespace mcpp::pm { @@ -36,16 +34,16 @@ export namespace mcpp::pm { enum class EventKind { Progress, Log, Data, Error, Heartbeat, Result }; -using ProgressEvent = mcpp::xlings::ProgressEvent; -using LogEvent = mcpp::xlings::LogEvent; -using DataEvent = mcpp::xlings::DataEvent; -using ErrorEvent = mcpp::xlings::ErrorEvent; -using ResultEvent = mcpp::xlings::ResultEvent; -using Event = mcpp::xlings::Event; +using ProgressEvent = mcpp::provider::ProgressEvent; +using LogEvent = mcpp::provider::LogEvent; +using DataEvent = mcpp::provider::DataEvent; +using ErrorEvent = mcpp::provider::ErrorEvent; +using ResultEvent = mcpp::provider::ResultEvent; +using Event = mcpp::provider::Event; -// --- Listener interface (alias to mcpp::xlings::EventHandler) --- +// --- Listener interface (alias to mcpp::provider::EventHandler) --- -using EventHandler = mcpp::xlings::EventHandler; +using EventHandler = mcpp::provider::EventHandler; // --- Capability invocation --- @@ -53,7 +51,7 @@ struct CallError { std::string message; }; -using CallResult = mcpp::xlings::CallResult; +using CallResult = mcpp::provider::CallResult; class Fetcher { public: @@ -265,23 +263,31 @@ namespace mcpp::pm { namespace { -// JSON extraction and NDJSON parsing are now in mcpp::xlings. -// Bring them into this TU's anonymous namespace for use by search/plan_install -// helper code that still needs extract_string/extract_object. -using mcpp::xlings::extract_string; -using mcpp::xlings::extract_int; -using mcpp::xlings::extract_bool; -using mcpp::xlings::extract_object; -using mcpp::xlings::parse_event_line; -using mcpp::xlings::shq; +// A field of a data event's JSON: a string, or an object/array as compact +// JSON text (what the readers below walk). Empty when absent. +std::string extract_string(std::string_view text, std::string_view key) { + auto j = nlohmann::json::parse(text, nullptr, false); + if (!j.is_object()) return {}; + auto it = j.find(std::string(key)); + return it != j.end() && it->is_string() ? it->get() : std::string{}; +} +std::string extract_object(std::string_view text, std::string_view key) { + auto j = nlohmann::json::parse(text, nullptr, false); + if (!j.is_object()) return {}; + auto it = j.find(std::string(key)); + return it != j.end() && (it->is_object() || it->is_array()) ? it->dump() : std::string{}; +} + +mcpp::provider::Location location_of(const config::GlobalConfig& cfg) { + return mcpp::config::make_xlings_env(cfg); +} } // namespace std::string Fetcher::build_command(std::string_view capability, std::string_view argsJson) const { - mcpp::xlings::Env env{ cfg_.xlingsBinary, cfg_.xlingsHome() }; - return mcpp::xlings::build_interface_command(env, capability, argsJson); + return mcpp::provider::open(location_of(cfg_))->operation_command(capability, argsJson); } // ─── install_packages failure diagnostics (#238) ──────────────────── @@ -418,13 +424,13 @@ Fetcher::call(std::string_view capability, std::string_view argsJson, EventHandler* handler) { - mcpp::xlings::Env env{ cfg_.xlingsBinary, cfg_.xlingsHome() }; + auto provider = mcpp::provider::open(location_of(cfg_)); mcpp::log::debug("fetcher", std::format("call: cap='{}'", capability)); if (mcpp::log::is_enabled(mcpp::log::Level::debug)) { - auto cmd = mcpp::xlings::build_interface_command(env, capability, argsJson); + auto cmd = provider->operation_command(capability, argsJson); mcpp::log::debug("fetcher", std::format("cmd = {}", cmd)); } - auto r = mcpp::xlings::call(env, capability, argsJson, handler); + auto r = provider->call(capability, argsJson, handler); if (!r) { mcpp::log::error("fetcher", std::format("call '{}' failed: {}", capability, r.error())); return std::unexpected(CallError{r.error()}); @@ -805,15 +811,11 @@ Fetcher::read_xpkg_lua(std::string_view ns, std::string_view shortName) const { if (shortName.empty()) return std::nullopt; - auto data = cfg_.xlingsHome() / "data"; - std::error_code ec; - if (!std::filesystem::exists(data, ec)) return std::nullopt; - // Candidate filenames generated by compat module — canonical first, // then fallback candidates (see compat.cppm for deprecation schedule). auto filenames = mcpp::pm::compat::xpkg_lua_candidates(ns, shortName); - for (auto& dir : sorted_index_dirs(data)) { + for (auto& dir : mcpp::provider::open(location_of(cfg_))->index_roots()) { if (auto lua = read_identity_verified_xpkg_lua( dir / "pkgs", ns, shortName, filenames)) { return lua; @@ -856,14 +858,17 @@ Fetcher::read_xpkg_lua_from_project_data(const std::filesystem::path& projectDir auto filenames = mcpp::pm::compat::xpkg_lua_candidates(ns, shortName); - for (auto& data : mcpp::config::project_xlings_data_roots(projectDir)) { - std::error_code ec; - if (!std::filesystem::exists(data, ec)) continue; - for (auto& dir : sorted_index_dirs(data)) { - if (auto lua = read_identity_verified_xpkg_lua( - dir / "pkgs", ns, shortName, filenames)) { - return lua; - } + // The project's own indices (design §5.1), then the data roots a project + // index was cloned into before it had a place of its own. + std::vector dirs = + mcpp::provider::layout().index_roots_of(projectDir / ".mcpp"); + for (auto& data : mcpp::config::project_xlings_data_roots(projectDir)) + for (auto& dir : sorted_index_dirs(data)) + if (std::ranges::find(dirs, dir) == dirs.end()) dirs.push_back(dir); + for (auto& dir : dirs) { + if (auto lua = read_identity_verified_xpkg_lua( + dir / "pkgs", ns, shortName, filenames)) { + return lua; } } return std::nullopt; @@ -879,8 +884,11 @@ Fetcher::install_path_from_project_data(const std::filesystem::path& projectDir, std::string_view shortName, std::string_view version) { - for (auto& data : mcpp::config::project_xlings_data_roots(projectDir)) { - auto base = data / "xpkgs"; + std::vector bases = + mcpp::provider::layout().stores_of(projectDir / ".mcpp"); + for (auto& data : mcpp::config::project_xlings_data_roots(projectDir)) + bases.push_back(mcpp::provider::layout().store_in(data)); + for (auto& base : bases) { if (!std::filesystem::exists(base)) continue; // Try canonical directory name: ns.shortName @@ -959,12 +967,7 @@ Fetcher::scan_short_name_matches( std::vector Fetcher::builtin_index_roots() const { - std::vector roots; - auto data = cfg_.xlingsHome() / "data"; - std::error_code ec; - if (!std::filesystem::exists(data, ec)) return roots; - for (auto& dir : sorted_index_dirs(data)) roots.push_back(dir); - return roots; + return mcpp::provider::open(location_of(cfg_))->index_roots(); } // ─── Legacy read_xpkg_lua (COMPAT(internal:legacy-fetcher-api)) ───────────────── @@ -1020,7 +1023,7 @@ ParsedTarget parse_target(std::string_view target, std::string_view defaultIndex // says why (`code`, interface 1.5+; extraction since 2026.10.10.2). Every // failure used to end in "check network and retry": a phone whose app sandbox // refused hard links was sent to check its network (design 2026-10-10 §13). -std::string install_failure_hint(const std::optional& error, +std::string install_failure_hint(const std::optional& error, const std::filesystem::path& home) { const std::string code = error ? error->code : std::string{}; if (code == "E_NETWORK") @@ -1054,16 +1057,10 @@ Fetcher::resolve_xpkg_path(std::string_view target, target)}); } - auto base = cfg_.xlingsHome() / "data" / "xpkgs"; - std::filesystem::path verdir = base - / std::format("{}-x-{}", parsed.indexName, parsed.packageName) - / parsed.version; + auto provider = mcpp::provider::open(location_of(cfg_)); + const mcpp::provider::PackageRef ref{parsed.indexName, parsed.packageName, parsed.version}; - mcpp::log::verbose("fetcher", std::format( - "resolve: target='{}' verdir='{}'", target, verdir.string())); - mcpp::log::debug("fetcher", std::format( - " xlingsHome='{}' autoInstall={} verdir_exists={}", - cfg_.xlingsHome().string(), autoInstall, std::filesystem::exists(verdir))); + mcpp::log::verbose("fetcher", std::format("resolve: target='{}'", target)); auto fill_payload = [&](XpkgPayload& p) { p.binDir = p.root / "bin"; @@ -1083,20 +1080,16 @@ Fetcher::resolve_xpkg_path(std::string_view target, p.sourceDir = (subs.size() == 1) ? subs.front() : p.root; }; - // Build the payload from a verdir that is known to exist. - auto make_payload = [&]() -> XpkgPayload { + // The payload as the provider placed it: a version dir that holds bin/, + // include/ or lib/ is the root (xim packages); otherwise its one + // subdirectory (an extracted source tarball, mcpplibs packages). + auto make_payload = [&](const std::filesystem::path& verdir) -> XpkgPayload { XpkgPayload payload; - // For xim packages (gcc, cmake, ...) the version dir IS the root. - // For mcpplibs packages the version dir contains an extracted - // tarball subdirectory; we treat the wrapper subdir as the root - // when its content includes bin/ or include/. std::error_code ec; std::vector subs; for (auto& e : std::filesystem::directory_iterator(verdir, ec)) { if (e.is_directory()) subs.push_back(e.path()); } - // If verdir directly contains bin/ or include/ -> it's the root. - // Otherwise prefer the unique subdirectory. bool verdir_is_root = std::filesystem::exists(verdir / "bin") || std::filesystem::exists(verdir / "include") || std::filesystem::exists(verdir / "lib"); @@ -1107,38 +1100,38 @@ Fetcher::resolve_xpkg_path(std::string_view target, return payload; }; - // ─── Resolution chain: marker check → clean residue → install → fallback ─ + // What the provider says of this payload, nothing installed: there and + // complete is the only "installed" (a directory that merely exists may + // be an interrupted install, or a success that installed nothing, #533). + const auto complete_here = [&]() -> std::optional { + const std::array refs{ref}; + auto st = provider->status(refs); + if (st.size() == 1) { + if (st[0].complete && !st[0].dir.empty()) return st[0].dir; + // Present but not complete (an interrupted install, a success + // that installed nothing): not installed. + if (st[0].origin != "absent") return std::nullopt; + } + // The provider could not say, or has nothing where the index's store + // is: what is installed elsewhere in the home, by its version grammar, + // holding something -- a payload installed before its index had a + // store of its own (COMPAT(internal:legacy-registry-store), design + // 2026-10-16 §5.3). + auto dir = provider->payload(ref); + std::error_code ec; + if (!dir.empty() && std::filesystem::is_directory(dir, ec) && !std::filesystem::is_empty(dir, ec)) return dir; + return std::nullopt; + }; - // 1. Already installed and complete (has .mcpp_ok marker)? - // - // Strict marker-only. We do NOT do legacy heuristic adoption here - // because we cannot distinguish a legacy-complete install (has bin/) - // from a half-extracted residue (also has bin/). Adopting the latter - // would silently corrupt the user's toolchain. - // - // Cost for users upgrading mcpp: a one-time reinstall per toolchain. - // The install path normally hits copy_xpkg_from_global() as a fast - // fallback (reuses ~/.xlings/ copy), so this is rarely a real download. - if (mcpp::fallback::is_install_complete(verdir)) { - mcpp::log::debug("fetcher", "install complete in sandbox"); - return make_payload(); + if (auto dir = complete_here()) { + mcpp::log::debug("fetcher", "payload complete in the store"); + return make_payload(*dir); } - // 2. Directory exists without marker → either interrupted install - // or legacy package. Stash it aside (don't delete outright) so a - // failed reinstall can roll back to a content-complete legacy package - // instead of leaving it destroyed (the xim-x-zlib regression). The - // guard commits (drops the backup) on each success path below; on any - // failure path its destructor restores a looks_complete_legacy() stash. - mcpp::fallback::InstallStash stash(verdir); - - // 3. Install via xlings (primary path). if (autoInstall) { - // Offline is checked HERE, after the "already installed" fast paths - // above: a build whose packages are all present must stay fully - // functional offline — that is the whole point. Only an actual - // download attempt is refused, and it names the package so the user - // knows what to fetch rather than seeing a generic network error. + // Offline is checked HERE, after the "already installed" fast path + // above: a build whose packages are all present stays functional + // offline. Only an actual download is refused, naming the package. if (mcpp::platform::env::offline_mode()) { mcpp::build::refusal::record( mcpp::build::refusal::Code::OfflineDownloadRequired); @@ -1148,149 +1141,62 @@ Fetcher::resolve_xpkg_path(std::string_view target, parsed.packageName, parsed.version) }); } if (parsed.indexName == "xim") { - mcpp::xlings::Env xlEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; // A descriptor missing from the local xim index may be one the // remote index has. Whether that miss refreshes is the refresh - // policy's decision, so `[index] auto_refresh = false` and the - // debounce hold here as on every other path (#648 A5). The status - // line `apply` prints makes the one-time network pause visible. - if (!mcpp::xlings::official_package_present(xlEnv, parsed.packageName)) { + // policy's decision (#648 A5). + if (!provider->official_package_present(parsed.packageName)) { auto decision = mcpp::pm::decide_for_miss( - mcpp::pm::policy_for(cfg_), xlEnv, + mcpp::pm::policy_for(cfg_), location_of(cfg_), std::format("xim:{}", parsed.packageName)); - if (auto r = mcpp::pm::apply(decision, xlEnv); !r) + if (auto r = mcpp::pm::apply(decision, location_of(cfg_)); !r) mcpp::ui::warning(r.error()); } } - std::vector targets { - std::format("{}:{}@{}", parsed.indexName, parsed.packageName, parsed.version) - }; - mcpp::log::verbose("fetcher", std::format("xlings install: {}", targets[0])); + std::vector targets { mcpp::provider::to_string(ref) }; + mcpp::log::verbose("fetcher", std::format("install: {}", targets[0])); auto inst = install(targets, handler); - if (!inst) { - // xlings launch/protocol failure — propagate the real error. - return std::unexpected(inst.error()); - } - if (inst->exitCode == 0 && std::filesystem::exists(verdir)) { - // Normal success path — but "the callee exited 0 and the directory - // it creates before doing any work exists" is not evidence that - // anything was installed. mcpp#533: a package-identity collision - // made xlings skip the descriptor's `install()` while still - // reporting success, and stamping that state cemented it, because - // `.mcpp_ok` is what the fast path above reads. - // - // Withheld rather than failed: a package can legitimately install - // no payload, and refusing here would break those. The cost of - // being wrong is one cheap re-check next build (xlings - // short-circuits its own install); the cost of the marker being - // wrong is permanent. - if (mcpp::fallback::payload_is_substantive(verdir)) { - mcpp::fallback::mark_install_complete(verdir); - } else { - mcpp::log::warn("fetcher", std::format( - "'{}@{}' reported a successful install but {} holds " - "nothing the package installed — not marking it complete", - parsed.packageName, parsed.version, - verdir.string())); - } - stash.commit(); - return make_payload(); - } - if (inst->exitCode == 0 && !std::filesystem::exists(verdir)) { - // xlings reported success but the package didn't land in the - // sandbox. This is the documented XLINGS_HOME-propagation bug: - // a successful xlings install can extract to ~/.xlings/ instead - // of the sandbox. ONLY in this narrow case do we trust the - // global location enough to fall through to copy_xpkg_from_global. - mcpp::log::verbose("fetcher", - "xlings reported success but verdir is missing — " - "checking global xlings (XLINGS_HOME propagation fallback)"); - bool copyOk = mcpp::fallback::copy_xpkg_from_global(verdir); - if (copyOk && mcpp::fallback::looks_complete_legacy(verdir)) { - mcpp::fallback::mark_install_complete(verdir); - stash.commit(); - mcpp::log::verbose("fetcher", "resolved via copy fallback"); - return make_payload(); - } - // Copy didn't yield a usable package — clean any partial state. - mcpp::fallback::clean_incomplete_install(verdir); + if (!inst) return std::unexpected(inst.error()); + if (inst->exitCode == 0) { + if (auto dir = complete_here()) return make_payload(*dir); + // Reported a success and left nothing complete (mcpp#533's shape): + // said, not stamped over. + return std::unexpected(CallError{std::format( + "'{}' reported a successful install, but its payload is not complete\n" + " hint: {}", targets[0], install_failure_hint(inst->error, cfg_.xlingsHome()))}); } - if (inst->exitCode != 0) { - // xlings interface install actually failed (network, missing - // package, extraction subprocess issue, etc.). Retry once through - // direct `xlings install`: xlings' own CLI path is more reliable - // for large toolchain packages and keeps the real failure output - // visible to CI/users when it still fails. - mcpp::fallback::clean_incomplete_install(verdir); - - // Offline-first refresh-on-miss: the failure may be a stale local - // index that doesn't list this package@version (e.g. installing a - // NEW toolchain version when the cached index predates it). The - // pre-install gate only refreshes when the package *file* is - // missing, so a version-miss slips through. Refresh the index ONCE - // here — on an actual install miss — then retry. Steady-state - // installs (package+version already local) succeed on the first - // try and never reach this path, so they stay offline. - if (parsed.indexName == "xim") { - mcpp::xlings::Env refreshEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; - mcpp::log::verbose("fetcher", - std::format("install failed for {}; asking the refresh policy before retry", - targets[0])); - auto decision = mcpp::pm::decide_for_miss( - mcpp::pm::policy_for(cfg_), refreshEnv, targets[0]); - if (auto r = mcpp::pm::apply(decision, refreshEnv); !r) - mcpp::log::verbose("fetcher", r.error()); - } + // The install failed. The index may predate this version: the refresh + // policy decides whether to refresh once, then the provider's own + // command line tries again -- more robust for a large toolchain, and + // its output stays visible when it still fails. + if (parsed.indexName == "xim") { mcpp::log::verbose("fetcher", - std::format("interface install failed for {}; retrying direct xlings install", - targets[0])); - mcpp::xlings::Env directEnv{ cfg_.xlingsBinary, cfg_.xlingsHome() }; - int directRc = mcpp::xlings::install_direct(directEnv, targets[0]); - if (directRc == 0 && std::filesystem::exists(verdir)) { - mcpp::fallback::mark_install_complete(verdir); - stash.commit(); - return make_payload(); - } - if (directRc == 0 && !std::filesystem::exists(verdir)) { - bool copyOk = mcpp::fallback::copy_xpkg_from_global(verdir); - if (copyOk && mcpp::fallback::looks_complete_legacy(verdir)) { - mcpp::fallback::mark_install_complete(verdir); - stash.commit(); - mcpp::log::verbose("fetcher", "resolved via copy fallback after direct install"); - return make_payload(); - } - mcpp::fallback::clean_incomplete_install(verdir); - } - - // If both interface and direct install failed, do NOT try copy - // fallback: the global ~/.xlings/ state may itself be residue from - // the same failure, and looks_complete_legacy() can't tell residue - // from complete. - std::string installError = std::format( - "xlings install of '{}:{}@{}' failed (interface exit {}, direct exit {})", - parsed.indexName, parsed.packageName, parsed.version, - inst->exitCode, directRc); - if (inst->error) - installError += ": " + inst->error->message; - return std::unexpected(CallError{std::format( - "{}\n hint: {}", installError, - install_failure_hint(inst->error, cfg_.xlingsHome()))}); + std::format("install failed for {}; asking the refresh policy before retry", targets[0])); + auto decision = mcpp::pm::decide_for_miss( + mcpp::pm::policy_for(cfg_), location_of(cfg_), targets[0]); + if (auto r = mcpp::pm::apply(decision, location_of(cfg_)); !r) + mcpp::log::verbose("fetcher", r.error()); } + mcpp::log::verbose("fetcher", std::format("retrying {} through the provider's command line", targets[0])); + int directRc = provider->install_direct(targets[0]); + if (directRc == 0) + if (auto dir = complete_here()) return make_payload(*dir); + + std::string installError = std::format( + "install of '{}' failed (interface exit {}, direct exit {})", + targets[0], inst->exitCode, directRc); + if (inst->error) + installError += ": " + inst->error->message; + return std::unexpected(CallError{std::format( + "{}\n hint: {}", installError, + install_failure_hint(inst->error, cfg_.xlingsHome()))}); } - // No autoInstall fallback: when the caller explicitly disables - // auto-install, do NOT perform any implicit recovery from the global - // ~/.xlings/ location. Without "this session's xlings install - // reported success" as a witness, we can't tell a complete legacy - // package apart from interrupted residue, and silently marking the - // latter as complete would mask the underlying problem. - - // 4. All paths exhausted. + return std::unexpected(CallError{ - std::format("xpkg payload missing: {}\n" + std::format("payload of '{}' is not installed\n" " hint: check network and retry, or `mcpp self init --force`", - verdir.string())}); + mcpp::provider::to_string(ref))}); } // ─── Namespace-aware install_path (canonical, 0.0.10+) ────────────── @@ -1303,41 +1209,41 @@ Fetcher::install_path(std::string_view ns, std::string_view shortName, // Layout discrimination is done by the caller via the xpkg.lua's // `mcpp = ""` (Form A pointer) or `mcpp = { ... }` (Form B // inline) field. - auto base = cfg_.xlingsHome() / "data" / "xpkgs"; - if (!std::filesystem::exists(base)) return std::nullopt; - - auto try_dir = [&](std::string_view dirName) -> std::optional { - auto verdir = base / std::string(dirName) / std::string(version); - return std::filesystem::exists(verdir) - ? std::optional{verdir} - : std::nullopt; - }; - - // Candidate directory names generated by compat module — canonical first, - // then fallback candidates (see compat.cppm for deprecation schedule). - auto candidates = mcpp::pm::compat::install_dir_candidates( - ns, shortName, cfg_.defaultIndex); - for (auto& dirName : candidates) { - if (auto p = try_dir(dirName)) return *p; - } - - // Last-resort fallback scan (COMPAT(internal:legacy-fetcher-api)): walk xpkgs/ for a - // directory holding this package under an older layout. - // - // The bare `-x-` arm is bound to the namespaces the caller - // actually asked for. Unbound, it returned ANY namespace's directory with - // a matching short name — `ocornut:imgui` resolving to `compat-x-imgui` — - // and install_path's contract is "the verdir for THIS package", so the - // install was skipped and another package's tree read in its place. See - // the header of mcpp.fallback.legacy_dirs for why version alignment is - // what made it reachable. - auto qname = mcpp::pm::compat::qualified_name(ns, shortName); + // Every store a payload of this home may be in: the store of the index + // that serves it, then the home's, which holds what was installed before + // that index had a store of its own (design 2026-10-16 §5.3). + const auto qname = mcpp::pm::compat::qualified_name(ns, shortName); + const auto candidates = mcpp::pm::compat::install_dir_candidates(ns, shortName, cfg_.defaultIndex); std::vector acceptedPrefixes; if (!ns.empty()) acceptedPrefixes.emplace_back(ns); if (!cfg_.defaultIndex.empty()) acceptedPrefixes.emplace_back(cfg_.defaultIndex); - if (auto legacy = mcpp::fallback::scan_legacy_install_dirs( - base, qname, shortName, acceptedPrefixes)) { - if (auto p = try_dir(*legacy)) return *p; + for (const auto& base : mcpp::provider::open(location_of(cfg_))->stores()) { + std::error_code ec; + if (!std::filesystem::exists(base, ec)) continue; + auto try_dir = [&](std::string_view dirName) -> std::optional { + auto verdir = base / std::string(dirName) / std::string(version); + return std::filesystem::exists(verdir, ec) ? std::optional{verdir} : std::nullopt; + }; + // Candidate directory names generated by compat module -- canonical + // first, then fallback candidates (see compat.cppm for deprecation + // schedule). + for (auto& dirName : candidates) + if (auto p = try_dir(dirName)) return *p; + + // Last-resort fallback scan (COMPAT(internal:legacy-fetcher-api)): + // walk the store for a directory holding this package under an older + // layout. + // + // The bare `-x-` arm is bound to the namespaces the caller + // actually asked for. Unbound, it returned ANY namespace's directory + // with a matching short name -- `ocornut:imgui` resolving to + // `compat-x-imgui` -- and install_path's contract is "the verdir for + // THIS package", so the install was skipped and another package's + // tree read in its place. See the header of mcpp.fallback.legacy_dirs + // for why version alignment is what made it reachable. + if (auto legacy = mcpp::provider::layout().legacy_install_dir(base, qname, shortName, acceptedPrefixes); + !legacy.empty()) + if (auto p = try_dir(legacy)) return *p; } return std::nullopt; } diff --git a/src/pm/refresh_policy.cppm b/src/pm/refresh_policy.cppm index 00b06c9fe..785fa58e5 100644 --- a/src/pm/refresh_policy.cppm +++ b/src/pm/refresh_policy.cppm @@ -21,7 +21,8 @@ import mcpp.log; import mcpp.platform; import mcpp.pm.index_contract; import mcpp.ui; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; export namespace mcpp::pm { @@ -46,7 +47,7 @@ struct RefreshPolicy { bool autoRefresh = true; // Shared with the xim install gate — one constant, one rationale, in the // leaf module both layers can see (mcpp::xlings). - std::int64_t debounceSeconds = mcpp::xlings::kIndexRefreshDebounceSeconds; + std::int64_t debounceSeconds = mcpp::provider::kIndexRefreshDebounceSeconds; }; struct RefreshDecision { @@ -66,7 +67,7 @@ RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg); // judgement is left to make. Keeps the opt-outs in one place rather than // re-tested at each call site. RefreshDecision decide_for_miss(const RefreshPolicy& policy, - const mcpp::xlings::Env& env, + const mcpp::provider::Location& env, std::string_view subject); // Run the sync a decision asked for. At most ONE per process: a build whose @@ -74,16 +75,16 @@ RefreshDecision decide_for_miss(const RefreshPolicy& policy, // Returns an error only when the sync itself failed; callers decide whether // that is fatal (it is not, if the build can still resolve locally). std::expected apply(const RefreshDecision& d, - const mcpp::xlings::Env& env); + const mcpp::provider::Location& env); // Explicit user intent (`mcpp update`): ignores debounce and the once-per- // process guard. Still refuses when offline, and says so. -std::expected force_refresh(const mcpp::xlings::Env& env); +std::expected force_refresh(const mcpp::provider::Location& env); // One line of advisory context for a resolution failure: what the index is and // how old it is. Never a gate — only ever appended to an error the user is // already seeing. -std::string staleness_note(const mcpp::xlings::Env& env); +std::string staleness_note(const mcpp::provider::Location& env); } // namespace mcpp::pm @@ -130,7 +131,7 @@ RefreshPolicy policy_for(const mcpp::config::GlobalConfig& cfg) { } RefreshDecision decide_for_miss(const RefreshPolicy& policy, - const mcpp::xlings::Env& env, + const mcpp::provider::Location& env, std::string_view subject) { RefreshDecision d; d.subject = std::string(subject); @@ -144,7 +145,7 @@ RefreshDecision decide_for_miss(const RefreshPolicy& policy, // Same debounce as the build path: two `mcpp add` typos in a row should not // buy two multi-repo syncs, for the same reason a build with two missing // packages does not. - auto status = mcpp::xlings::default_index_status(env, policy.debounceSeconds); + auto status = mcpp::provider::open(env)->default_index_status(policy.debounceSeconds); if (status.present && status.ageSeconds >= 0 && status.ageSeconds < policy.debounceSeconds) { d.reason = RefreshReason::SuppressedDebounce; @@ -157,14 +158,14 @@ RefreshDecision decide_for_miss(const RefreshPolicy& policy, namespace { -std::expected run_sync(const mcpp::xlings::Env& env, +std::expected run_sync(const mcpp::provider::Location& env, std::string_view banner) { - auto before = mcpp::xlings::default_index_status(env, 0).rev; + auto before = mcpp::provider::open(env)->default_index_status(0).rev; mcpp::ui::status("Refreshing", banner); - int rc = mcpp::xlings::update_index(env, /*quiet=*/true); + int rc = mcpp::provider::open(env)->update_index(/*quiet=*/true); if (rc != 0) return std::unexpected(std::format("package index refresh failed (rc {})", rc)); - auto after = mcpp::xlings::default_index_status(env, 0).rev; + auto after = mcpp::provider::open(env)->default_index_status(0).rev; if (before && after && *before != *after) mcpp::ui::status("Updated", std::format("package index {} → {}", *before, *after)); else if (after) @@ -175,7 +176,7 @@ std::expected run_sync(const mcpp::xlings::Env& env, } // namespace std::expected apply(const RefreshDecision& d, - const mcpp::xlings::Env& env) + const mcpp::provider::Location& env) { if (!d.shouldRefresh) { mcpp::log::verbose("index", std::format( @@ -192,7 +193,7 @@ std::expected apply(const RefreshDecision& d, d.subject, reason_text(d.reason))); } -std::expected force_refresh(const mcpp::xlings::Env& env) { +std::expected force_refresh(const mcpp::provider::Location& env) { if (mcpp::platform::env::offline_mode()) return std::unexpected( "offline mode is on — cannot refresh the package index " @@ -201,8 +202,8 @@ std::expected force_refresh(const mcpp::xlings::Env& env) { return run_sync(env, "package index (requested)"); } -std::string staleness_note(const mcpp::xlings::Env& env) { - auto st = mcpp::xlings::default_index_status(env, 0); +std::string staleness_note(const mcpp::provider::Location& env) { + auto st = mcpp::provider::open(env)->default_index_status(0); if (!st.present) return "no local package index"; return st.rev ? std::format("local index {} ({})", *st.rev, age_phrase(st.ageSeconds)) diff --git a/src/project.cppm b/src/project.cppm index 6fac1eb07..1f71f3877 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -247,7 +247,7 @@ bool merge_xlings(mcpp::manifest::Manifest& member, const mcpp::manifest::XlingsConfig* top, std::span rows, XlingsPart part) { - // `(namespace, name)`, the identity `mcpp.xlings.address_set` defines, + // `(namespace, name)`, the identity `mcpp.provider.address_set` defines, // spelled here from the same parser rather than imported. Importing that // module here makes GCC 16.1 fail with an internal compiler error // (segmentation fault) at `import mcpp.cli;` in src/main.cpp. Measured. diff --git a/src/xlings/address_set.cppm b/src/provider/address_set.cppm similarity index 97% rename from src/xlings/address_set.cppm rename to src/provider/address_set.cppm index 2c2570624..32231a6b5 100644 --- a/src/xlings/address_set.cppm +++ b/src/provider/address_set.cppm @@ -1,4 +1,4 @@ -// mcpp.xlings.address_set — one package, one version. +// mcpp.provider.address_set — one package, one version. // // THE PROBLEM THIS EXISTS FOR. An xlings address is written // `[:][@]`, and mcpp had two definitions of when two such @@ -35,13 +35,13 @@ // winner that fails it is refused. The asymmetry is the point: an upgrade must // not turn every dependency that pinned a tool into a hard failure, while a // floor a rule package depends on must not be silently lowered. -export module mcpp.xlings.address_set; +export module mcpp.provider.address_set; import std; import mcpp.manifest; import mcpp.version_req; -export namespace mcpp::xlings::addrset { +export namespace mcpp::provider::addrset { // One declaration of one address, and where it came from. struct Claim { @@ -97,11 +97,11 @@ std::optional override_violation(std::span claims, std::vector requirements_for(std::span claims, std::string_view key); -} // namespace mcpp::xlings::addrset +} // namespace mcpp::provider::addrset // ── implementation ────────────────────────────────────────────────────────── -namespace mcpp::xlings::addrset { +namespace mcpp::provider::addrset { std::string package_key(std::string_view address) { const auto e = mcpp::manifest::parse_address(address); @@ -224,4 +224,4 @@ std::expected unify(std::span claims) { return out; } -} // namespace mcpp::xlings::addrset +} // namespace mcpp::provider::addrset diff --git a/src/provider/provider.cppm b/src/provider/provider.cppm new file mode 100644 index 000000000..f42840ffb --- /dev/null +++ b/src/provider/provider.cppm @@ -0,0 +1,466 @@ +// mcpp.provider — the package provider port: mcpp's own words for what it +// asks of the program that fetches, installs and describes packages +// (.agents/docs/2026-10-16-xlings-provider-port-and-shared-store-design.md §4). +// +// mcpp (build / toolchain / pm / runtime / doctor / pack / cli) +// │ import mcpp.provider.select this port, and the one provider +// ▼ +// mcpp.provider types and interfaces (this module) +// mcpp.provider.xlings the adapter: xlings' commands, protocol, layout +// +// Nothing outside src/provider/xlings/ names xlings' commands, its NDJSON, +// its home layout or its store naming; .github/tools/check_provider_boundary.py +// makes that a fact. A payload's path is what the provider answers, and it is +// used as answered: never canonicalised (a borrowed payload is a link entry +// whose path is its identity), never removed by mcpp. + +export module mcpp.provider; + +import std; + +export namespace mcpp::provider { + +// ─── Where the provider is ────────────────────────────────────────── + +// One provider instance: its program and its home, and the project whose +// scope it acts in (empty: the global scope). +struct Location { + std::filesystem::path binary; + std::filesystem::path home; + std::filesystem::path projectDir; +}; + +// ─── A package address ────────────────────────────────────────────── + +// `[:][@]`, as a manifest writes it. The namespace +// defaults to `xim`, the provider's own index: `ninja` and `xim:ninja` are +// one package (mcpp.provider.address_set holds the same rule). +struct PackageRef { + std::string ns; + std::string name; + std::string version; // empty = unpinned; otherwise a version or a requirement +}; + +PackageRef parse_ref(std::string_view spec) { + PackageRef r; + std::string_view rest = spec; + if (auto colon = rest.find(':'); colon != std::string_view::npos) { + r.ns = std::string(rest.substr(0, colon)); + rest = rest.substr(colon + 1); + } else { + r.ns = "xim"; + } + if (auto at = rest.find('@'); at != std::string_view::npos) { + r.name = std::string(rest.substr(0, at)); + r.version = std::string(rest.substr(at + 1)); + } else { + r.name = std::string(rest); + } + return r; +} + +std::string to_string(const PackageRef& r) { + return r.version.empty() ? std::format("{}:{}", r.ns, r.name) : std::format("{}:{}@{}", r.ns, r.name, r.version); +} + +// ─── What an operation says while it runs ────────────────────────── + +struct ProgressEvent { + std::string phase; // "download", "extract", "configure", ... + int percent = 0; + std::string message; +}; + +struct LogEvent { + std::string level; // "debug" | "info" | "warn" | "error" + std::string message; +}; + +// A structured fact the operation reports (`install_targets`, `install_plan`, +// `payload_status`, ...): its kind and its JSON, for the caller to read. +struct DataEvent { + std::string dataKind; + std::string payloadJson; +}; + +struct ErrorEvent { + std::string code; + std::string message; + std::string hint; + bool recoverable = false; +}; + +struct ResultEvent { + int exitCode = 0; + std::string dataJson; +}; + +using Event = std::variant; + +struct EventHandler { + virtual ~EventHandler() = default; + virtual void on_progress(const ProgressEvent&) {} + virtual void on_log (const LogEvent&) {} + virtual void on_data (const DataEvent&) {} + virtual void on_error (const ErrorEvent&) {} + virtual void on_result (const ResultEvent&) {} +}; + +struct CallResult { + int exitCode = 0; + std::vector dataEvents; + std::optional error; + std::string resultJson; + // The provider's own error-level lines, the last few, kept only when the + // operation failed: the line that names a rejection. + std::vector stderrTail; +}; + +// A download in flight, as the provider reports it. +struct BootstrapFile { + std::string name; + double downloadedBytes = 0; + double totalBytes = 0; + bool started = false; + bool finished = false; +}; + +struct BootstrapProgress { + std::vector files; + double elapsedSec = 0; +}; + +using BootstrapProgressCallback = std::function; + +// ─── Indexes ──────────────────────────────────────────────────────── + +// A local index, read without touching the network (`mcpp index status`). +struct IndexStatus { + std::filesystem::path dir; + bool present = false; + bool fresh = false; + std::int64_t ageSeconds = -1; + std::optional rev; // the index's content identity; opaque +}; + +// One index a home is seeded with: its name and where it comes from (a git +// URL, an artifact base and its CN half, the source policy). +struct SeedRepo { + std::string name; + std::string url; + std::string artifact; + std::string source; + std::string artifactCn; + // An index mcpp owns (design §5): where its data is and where the + // payloads it serves go. Empty: the provider's own places. + std::filesystem::path dir; + std::filesystem::path store; +}; + +// How long a completed index sync suppresses the next (a second sync because +// another package is missing cannot help: the first fetched everything). +inline constexpr std::int64_t kIndexRefreshDebounceSeconds = 120; +inline constexpr std::chrono::seconds kDefaultIndexRefreshTimeout{120}; + +// ─── A project's environment (mcpp.toml [xlings]) ─────────────────── + +struct ProjectEnv { + std::vector deps; + std::vector> workspace; + std::string subos; + bool empty() const { return deps.empty() && workspace.empty() && subos.empty(); } +}; + +// ─── A runtime environment's description ──────────────────────────── + +namespace runtime_info { + +// The schema this build understands; a higher one is still read. +inline constexpr int kSupportedSchema = 1; + +struct EnvDecl { + std::string var; + std::string op; // "set" | "prepend" + std::string value; // may name the environment's directory +}; + +struct Declarer { + std::string binding; // "@" + std::vector decls; +}; + +struct PackageIdentity { + std::string namespace_; + std::string name; + std::string version; + std::string source; +}; + +struct CapabilityProvider { + std::string capability; + PackageIdentity provider; +}; + +struct RuntimeArtifact { + std::string role; + PackageIdentity provider; + std::filesystem::path path; + std::string provenance; + std::string abi; + std::string digest; + std::string hostFingerprint; +}; + +struct Info { + int schema = 0; + std::string runtime; // "glibc@2.39" + std::string hostGlibc; // optional creation-host floor + std::vector providers; // sorted by binding + std::vector runtimeProviders; + std::vector runtimeArtifacts; + bool present = false; + // Non-empty: the caller surfaces it. Never an error. + std::string note; +}; + +// "glibc@2.39" -> "linux--glibc": the runtime string says its family. +std::string family_of(std::string_view runtime, std::string_view arch = "x86_64") { + const auto at = runtime.find('@'); + const auto name = runtime.substr(0, at == std::string_view::npos ? runtime.size() : at); + if (name == "glibc") return std::format("linux-{}-glibc", arch); + if (name == "musl") return std::format("linux-{}-musl", arch); + if (name == "wasi-libc") return "wasm32-wasi"; + if (name == "macos_sdk") return std::format("darwin-{}", arch); + if (name == "ucrt") return std::format("windows-{}-ucrt", arch); + return "unknown"; +} + +} // namespace runtime_info + +// ─── The provider, as mcpp asks of it ─────────────────────────────── + +// What a provider program is, read by its handshake (design §12). +struct ProviderInfo { + std::string version; // "2026.10.16.1" + std::string protocol; // "1.10" + std::vector capabilities; + std::vector features; // index_dir, upstreams, payload_status, ... + std::vector homeLayouts; + bool answered = false; // false: an older provider without the handshake +}; + +// A read-only home this provider installs from by link (design 2026-10-16 +// §8): its home, its store (empty: the home's), the namespaces taken from it +// (empty: every one). +struct Upstream { + std::filesystem::path home; + std::filesystem::path store; + std::vector namespaces; +}; + +// A payload installed here that an upstream has with the same identity +// (namespace, name, version, revision) and complete: the copy here could be +// a link to it (`mcpp self store dedupe`, §8.5). +struct Duplicate { + std::filesystem::path local; + std::filesystem::path upstream; +}; + +// One payload in a store: its package and version, and its directory. +struct StoreEntry { + std::string ns; + std::string name; + std::string version; + std::filesystem::path dir; +}; + +// What a payload path says without asking the provider program: the store +// it is in and what is beside it. Each answer is a path as the store spells +// it, never canonicalised; an empty path is "none" (std::optional and +// std::expected are kept out of the virtual signatures of this +// module: GCC 16 writes an unreadable module for its importers when an +// abstract class returns one). +class Layout { +public: + virtual ~Layout() = default; + // The store `path` (a payload, or a file inside one) is in. + virtual std::filesystem::path store_of(const std::filesystem::path& path) const = 0; + // A version of `name` (any namespace) beside `path`'s payload, holding + // something (and `requiredRel`, when given). + virtual std::filesystem::path beside(const std::filesystem::path& path, std::string_view name, + std::string_view requiredRel) const = 0; + // A version directory of the provider-index tool `name` beside `path`. + virtual std::filesystem::path beside_tool(const std::filesystem::path& path, std::string_view name) const = 0; + // `binaryRel` inside a version of the provider-index tool `name` beside `path`. + virtual std::filesystem::path beside_binary(const std::filesystem::path& path, std::string_view name, + std::string_view binaryRel) const = 0; + // The runtime environment a toolchain at `compiler` belongs to. + virtual std::filesystem::path environment_of(const std::filesystem::path& compiler) const = 0; + // The packaging revision recorded in a payload; -1: no record. + virtual int revision_of(const std::filesystem::path& payloadDir) const = 0; + // The store of mcpp's own home, and a package found in it. + virtual std::filesystem::path home_store() const = 0; + virtual std::filesystem::path in_home_store(std::string_view name, std::string_view requiredRel) const = 0; + // The store under a provider's data root (a project's). + virtual std::filesystem::path store_in(const std::filesystem::path& dataRoot) const = 0; + // A directory of `store` that holds the package under an older naming + // (the namespaces `prefixes` allow); empty: none. + virtual std::string legacy_install_dir(const std::filesystem::path& store, std::string_view qualifiedName, + std::string_view shortName, + const std::vector& prefixes) const = 0; + // The store of a home. + virtual std::filesystem::path store_of_home(const std::filesystem::path& home) const = 0; + // The home a store belongs to. + virtual std::filesystem::path home_of_store(const std::filesystem::path& store) const = 0; + // Whether `dir` is a home's default runtime environment. + virtual bool is_default_environment(std::string_view dir) const = 0; + // Whether `dir` is a store (a payload's path is /...). + virtual bool is_store(const std::filesystem::path& dir) const = 0; + // The directory of every version of the provider-index package `name` + // in `store`. + virtual std::filesystem::path tool_in(const std::filesystem::path& store, std::string_view name) const = 0; + // Every payload in `store`, and the payload a path is in (empty name: none). + virtual std::vector entries(const std::filesystem::path& store) const = 0; + virtual StoreEntry entry_of(const std::filesystem::path& path) const = 0; + // What a path of a provider home is: "payload", "environment", "index", + // or "" (nothing the provider keeps). + virtual std::string kind_of(const std::filesystem::path& path) const = 0; + // A runtime environment of a home ("default"), a project's provider + // home, and the file an environment describes itself in (its changes + // mark the environment changed). + virtual std::filesystem::path environment(const std::filesystem::path& home, std::string_view name) const = 0; + virtual std::filesystem::path project_home(const std::filesystem::path& projectRoot) const = 0; + virtual std::filesystem::path description(const std::filesystem::path& environment) const = 0; + // The index directories under a data root, whether any index has data + // there, and where a provider-index package's descriptor may be (in the + // order an index is consulted). + virtual std::vector index_dirs(const std::filesystem::path& dataRoot) const = 0; + virtual bool index_data_present(const std::filesystem::path& dataRoot) const = 0; + virtual std::vector descriptor_candidates(const std::filesystem::path& dataRoot, + std::string_view name) const = 0; + // A runtime environment's description of itself. + virtual runtime_info::Info runtime_info(const std::filesystem::path& environment) const = 0; + // An index's content identity (opaque: compared and printed, never + // parsed); empty: none. + virtual std::string index_revision(const std::filesystem::path& indexDir) const = 0; + // A provider home's index directories and stores (the global home, or a + // project's: project_home), read from its configuration alone. + virtual std::vector index_roots_of(const std::filesystem::path& home) const = 0; + virtual std::vector stores_of(const std::filesystem::path& home) const = 0; + // The indexes a provider home is configured to serve, its default + // included; empty: the directory is not a provider home (no + // configuration there). + virtual std::vector indices_of(const std::filesystem::path& home) const = 0; + // The store an mcpp home (or a project's .mcpp) keeps the payloads of + // mcpp's own indices in: the provider names a store by its last + // component (design §5.1). + virtual std::filesystem::path own_store(const std::filesystem::path& mcppHome) const = 0; + // What a directory offers as an upstream: an mcpp home (its registry for + // the provider's own packages, its store for mcpp's), a provider home (its + // own packages only), or nothing (empty). + virtual std::vector upstream_of(const std::filesystem::path& dir) const = 0; + // Whether `name` is one of the provider's own indexes (its default and + // the ones it adds itself), which stay where the provider keeps them. + virtual bool is_provider_index(std::string_view name) const = 0; +}; + +// What a provider says of a package's payload without installing anything +// (its `payload_status`, design §7.1 / X4): where it is, whether it is +// complete (there, holding something, no recorded failure, the recipe's +// revision), and whether it is this home's own or an upstream's link entry. +struct PayloadStatus { + std::string request; // the address as asked + std::string ns, name, version; // what it resolved to + int revision = 0; // the recipe's + std::filesystem::path dir; // the payload directory (empty: does not resolve) + std::string origin; // "local" | "upstream" | "absent" + int installedRevision = -1; // what the payload's stamp records; -1: none + bool complete = false; + std::string error; // non-empty: the request did not resolve +}; + +// One provider at one Location. +class Provider { +public: + virtual ~Provider() = default; + virtual const Location& location() const = 0; + + // ── payloads ── + // Where an installed payload of `ref` is (the provider's own answer + // first, then its version grammar over what is installed). + virtual std::filesystem::path payload(const PackageRef& ref) const = 0; // empty: not installed + // A pinned tool's payload directory, and the directory of all its versions. + virtual std::filesystem::path pinned(std::string_view tool, std::string_view version) const = 0; + virtual std::filesystem::path versions(std::string_view tool) const = 0; + // Remember what an install resolved each request to (its `install_targets`). + virtual void remember(const DataEvent& installTargets) const = 0; + // The store this provider installs into, and every store a payload here + // may be in (the indices' own, then the home's). + virtual std::filesystem::path store() const = 0; + virtual std::vector stores() const = 0; + // What each request resolves to and what its payload is here, nothing + // installed; one entry per request, in order (empty: the provider cannot + // say -- a provider before the handshake). + virtual std::vector status(std::span refs) const = 0; + + // ── the environment it keeps ── + virtual std::filesystem::path sysroot() const = 0; // headers and libraries a build targets + virtual std::filesystem::path bin_dir() const = 0; // the programs it placed + virtual bool initialized() const = 0; + virtual std::filesystem::path data_root() const = 0; // where its indexes are + virtual std::filesystem::path config_file() const = 0; // the home's own configuration + + // ── operations ── + virtual std::expected + call(std::string_view operation, std::string_view argsJson, EventHandler* handler = nullptr) const = 0; + virtual int install_with_progress(std::string_view target, const BootstrapProgressCallback& cb, + bool quiet = false) const = 0; + virtual int install_direct(std::string_view target, bool quiet = false) const = 0; + // The command line that runs the provider program here (diagnostics and + // the few commands a person is told to run). + virtual std::string command_prefix() const = 0; + virtual std::string operation_command(std::string_view operation, std::string_view argsJson) const = 0; + + // ── setup ── + virtual void seed(std::span repos, std::string_view mirror = "auto", + const ProjectEnv& penv = {}) const = 0; + // The upstreams this home installs from (replacing what it had; none: + // self-contained). Never written to. + virtual void set_upstreams(std::span upstreams) const = 0; + // What this home installed itself that an upstream has the same of, and + // one of them replaced by a link to the upstream's (atomically; empty: + // done, else why not). + virtual std::vector duplicates() const = 0; + virtual std::string link_to_upstream(const Duplicate& d) const = 0; + virtual int show_config() const = 0; + virtual int set_mirror(std::string_view mirror, bool quiet = false) const = 0; + virtual void ensure_initialized(bool quiet) const = 0; + // The runtime the toolchains this provider installs run on (design + // 2026-10-16 §7.2): mcpp's selected one ("glibc@2.44.3"), pinned once. + // Pinning makes the provider's links to it current; `relink` makes them + // again when the pin already says so (its payload was installed since). + virtual void pin_runtime(std::string_view runtimeId, bool relink) const = 0; + virtual void ensure_patchelf(bool quiet, const BootstrapProgressCallback& cb) const = 0; + virtual void ensure_ninja(bool quiet, const BootstrapProgressCallback& cb) const = 0; + virtual std::filesystem::path usable_nasm() const = 0; // empty: none + virtual std::filesystem::path placed_nasm() const = 0; + virtual bool nasm_is_from_host(const std::filesystem::path& bin) const = 0; + + // ── indexes ── + // Every directory an index of this scope keeps its data in, sorted. + virtual std::vector index_roots() const = 0; + // Take over the indices mcpp owns (`dir`, `store` set): their places + // recorded in the provider's configuration, data the provider kept at its + // own place moved there once. + virtual void own_indices(std::span owned) const = 0; + virtual IndexStatus default_index_status(std::int64_t ttlSeconds) const = 0; + virtual IndexStatus official_index_status(std::int64_t ttlSeconds) const = 0; + virtual bool index_fresh(std::int64_t ttlSeconds) const = 0; + virtual bool official_index_fresh(std::int64_t ttlSeconds) const = 0; + virtual bool official_package_index_fresh(std::string_view package, std::int64_t ttlSeconds) const = 0; + virtual bool official_package_present(std::string_view package) const = 0; + virtual int update_index(bool quiet = false) const = 0; + virtual void ensure_index_fresh(std::int64_t ttlSeconds, bool quiet = false) const = 0; +}; + +} // namespace mcpp::provider diff --git a/src/xlings/runtime_selection.cppm b/src/provider/runtime_selection.cppm similarity index 91% rename from src/xlings/runtime_selection.cppm rename to src/provider/runtime_selection.cppm index 5d1eb432e..ba795113d 100644 --- a/src/xlings/runtime_selection.cppm +++ b/src/provider/runtime_selection.cppm @@ -1,4 +1,4 @@ -// mcpp.xlings.runtime_selection — the sole project runtime-selection policy. +// mcpp.provider.runtime_selection — the sole project runtime-selection policy. // // This module chooses a name and an owner. It deliberately does not inspect // the process environment, xlings' active/current state, the compiler path, or @@ -6,12 +6,12 @@ // any of them to choose the build OS would make one mcpp.toml mean different // ABIs in different shells. -export module mcpp.xlings.runtime_selection; +export module mcpp.provider.runtime_selection; import std; import mcpp.manifest; -export namespace mcpp::xlings::runtime { +export namespace mcpp::provider::runtime { struct RuntimeSelection { enum class Mode { McppDefault, NamedSubos } mode = Mode::McppDefault; @@ -67,4 +67,4 @@ select_runtime( return out; } -} // namespace mcpp::xlings::runtime +} // namespace mcpp::provider::runtime diff --git a/src/provider/select.cppm b/src/provider/select.cppm new file mode 100644 index 000000000..65aa0a503 --- /dev/null +++ b/src/provider/select.cppm @@ -0,0 +1,103 @@ +// mcpp.provider.select — the one provider mcpp uses, and the only module +// outside src/provider/xlings/ that names the adapter +// (.github/tools/check_provider_boundary.py). Everything else imports this +// and speaks the port's words (mcpp.provider). + +export module mcpp.provider.select; + +import std; +import mcpp.provider; + + +import mcpp.provider.xlings; +import mcpp.provider.xlings.provider; +import mcpp.provider.xlings.subos_info; + +export namespace mcpp::provider { + +// What a payload path says (its store, what is beside it), asked of no program. +const Layout& layout() { return mcpp::provider::xlings::layout_instance(); } + +// The provider at `at`: its program, its home, the project it acts for. +std::unique_ptr open(const Location& at) { + return std::make_unique(at); +} + +// The port's questions, each "none" as std::nullopt. +namespace detail { +inline std::optional some(std::filesystem::path p) { + if (p.empty()) return std::nullopt; + return p; +} +} // namespace detail +std::optional store_of(const std::filesystem::path& path) { + return detail::some(layout().store_of(path)); +} +std::optional beside(const std::filesystem::path& path, std::string_view name, + std::string_view requiredRel = {}) { + return detail::some(layout().beside(path, name, requiredRel)); +} +std::optional beside_tool(const std::filesystem::path& path, std::string_view name) { + return detail::some(layout().beside_tool(path, name)); +} +std::optional beside_binary(const std::filesystem::path& path, std::string_view name, + std::string_view binaryRel) { + return detail::some(layout().beside_binary(path, name, binaryRel)); +} +std::optional environment_of(const std::filesystem::path& compiler) { + return detail::some(layout().environment_of(compiler)); +} +std::optional revision_of(const std::filesystem::path& payloadDir) { + const int r = layout().revision_of(payloadDir); + if (r < 0) return std::nullopt; + return r; +} +std::optional home_store() { return detail::some(layout().home_store()); } +std::optional in_home_store(std::string_view name, std::string_view requiredRel = {}) { + return detail::some(layout().in_home_store(name, requiredRel)); +} +std::optional index_revision(const std::filesystem::path& indexDir) { + auto r = layout().index_revision(indexDir); + if (r.empty()) return std::nullopt; + return r; +} +std::optional payload(const Provider& p, const PackageRef& ref) { + return detail::some(p.payload(ref)); +} +// The nasm a build may use (the host's first, then the provider's), and the +// one the provider placed. +std::optional usable_nasm(const Provider& p) { return detail::some(p.usable_nasm()); } +std::optional placed_nasm(const Provider& p) { return detail::some(p.placed_nasm()); } + +// A runtime environment's declared variables, merged into `ambient` (`set` +// fills an absent one, `prepend` goes in front of what is there). +std::vector> +resolve_env(const runtime_info::Info& info, const std::filesystem::path& environment, + const std::function(std::string_view)>& ambient = {}) { + return mcpp::provider::xlings::subos::resolve_env(info, environment, ambient); +} + +// What the provider program at `binary` says it is, and why it cannot serve +// this mcpp (empty: it can) -- the handshake every provider source passes +// (design §12). +ProviderInfo describe(const std::filesystem::path& binary) { return mcpp::provider::xlings::describe(binary); } +std::string unmet(const ProviderInfo& info) { return mcpp::provider::xlings::unmet(info); } + +// The provider version this mcpp ships with and requires at least. +inline constexpr std::string_view kPinnedVersion = mcpp::provider::xlings::pinned::kXlingsVersion; +// The pinned tools mcpp provisions through the provider. +inline constexpr std::string_view kPatchelfVersion = mcpp::provider::xlings::pinned::kPatchelfVersion; +inline constexpr std::string_view kNinjaVersion = mcpp::provider::xlings::pinned::kNinjaVersion; +inline constexpr std::string_view kNasmVersion = mcpp::provider::xlings::pinned::kNasmVersion; + +// The provider's error lines of a failed run, the last `limit`. +std::vector error_tail(std::string_view text, std::size_t limit = 20) { + return mcpp::provider::xlings::stderr_error_tail(text, limit); +} + +// A bound on an index refresh, set from `[index] refresh_timeout`. +void set_index_refresh_timeout(std::chrono::seconds timeout) { + mcpp::provider::xlings::set_index_refresh_timeout(timeout); +} + +} // namespace mcpp::provider diff --git a/src/fallback/legacy_dirs.cppm b/src/provider/xlings/legacy_dirs.cppm similarity index 93% rename from src/fallback/legacy_dirs.cppm rename to src/provider/xlings/legacy_dirs.cppm index 9b6c2ff22..a5b81b405 100644 --- a/src/fallback/legacy_dirs.cppm +++ b/src/provider/xlings/legacy_dirs.cppm @@ -1,4 +1,4 @@ -// mcpp.fallback.legacy_dirs — legacy xpkg directory scan. +// mcpp.provider.xlings.legacy_dirs — legacy store directory scan (the adapter's: store naming is xlings'). // // Last-resort fallback scan (COMPAT(internal:legacy-xpkgs-scan)): walk xpkgs/ for a // directory that holds the requested package under an older naming layout. @@ -30,11 +30,11 @@ // for (`-x-`, `-x-`) and refuses the one it // was never meant to serve: some other namespace's package. -export module mcpp.fallback.legacy_dirs; +export module mcpp.provider.xlings.legacy_dirs; import std; -export namespace mcpp::fallback { +export namespace mcpp::provider::xlings { // Scan the xpkgs base directory for a legacy install directory holding // (namespace, shortName). Returns the matching directory name (not the full @@ -73,4 +73,4 @@ scan_legacy_install_dirs(const std::filesystem::path& xpkgsBase, return std::nullopt; } -} // namespace mcpp::fallback +} // namespace mcpp::provider::xlings diff --git a/src/provider/xlings/provider.cppm b/src/provider/xlings/provider.cppm new file mode 100644 index 000000000..bd882975d --- /dev/null +++ b/src/provider/xlings/provider.cppm @@ -0,0 +1,421 @@ +// mcpp.provider.xlings.provider — xlings as mcpp's package provider: the +// port's Layout and Provider over the adapter's commands and layout +// (mcpp.provider.xlings, mcpp.provider.xlings.subos_info). + +export module mcpp.provider.xlings.provider; + +import std; +import mcpp.provider; +import mcpp.provider.xlings; +import mcpp.provider.xlings.subos_info; +import mcpp.provider.xlings.legacy_dirs; +import mcpp.libs.json; +import mcpp.platform; + +export namespace mcpp::provider::xlings { + +class XlingsLayout final : public mcpp::provider::Layout { +public: + std::filesystem::path store_of(const std::filesystem::path& path) const override { + return paths::xpkgs_from_compiler(path).value_or(std::filesystem::path{}); + } + std::filesystem::path beside(const std::filesystem::path& path, std::string_view name, + std::string_view requiredRel) const override { + return paths::find_sibling_package(path, name, requiredRel).value_or(std::filesystem::path{}); + } + std::filesystem::path beside_tool(const std::filesystem::path& path, std::string_view name) const override { + return paths::find_sibling_tool(path, name).value_or(std::filesystem::path{}); + } + std::filesystem::path beside_binary(const std::filesystem::path& path, std::string_view name, + std::string_view binaryRel) const override { + return paths::find_sibling_binary(path, name, binaryRel).value_or(std::filesystem::path{}); + } + std::filesystem::path environment_of(const std::filesystem::path& compiler) const override { + return paths::subos_dir_of(compiler).value_or(std::filesystem::path{}); + } + int revision_of(const std::filesystem::path& payloadDir) const override { + return paths::installed_revision(payloadDir).value_or(-1); + } + std::filesystem::path home_store() const override { + return paths::active_home_xpkgs().value_or(std::filesystem::path{}); + } + std::filesystem::path in_home_store(std::string_view name, std::string_view requiredRel) const override { + return paths::find_home_tool(name, requiredRel).value_or(std::filesystem::path{}); + } + std::filesystem::path store_in(const std::filesystem::path& dataRoot) const override { return dataRoot / "xpkgs"; } + std::string legacy_install_dir(const std::filesystem::path& store, std::string_view qualifiedName, + std::string_view shortName, + const std::vector& prefixes) const override { + return scan_legacy_install_dirs(store, qualifiedName, shortName, prefixes).value_or(std::string{}); + } + std::filesystem::path store_of_home(const std::filesystem::path& home) const override { + return home / "data" / "xpkgs"; + } + std::filesystem::path home_of_store(const std::filesystem::path& store) const override { + return store.parent_path().parent_path(); + } + bool is_default_environment(std::string_view dir) const override { return dir.ends_with("subos/default"); } + bool is_store(const std::filesystem::path& dir) const override { + return dir.filename() == "xpkgs" && dir.parent_path().filename() == "data"; + } + std::filesystem::path tool_in(const std::filesystem::path& store, std::string_view name) const override { + return store / std::format("xim-x-{}", name); + } + std::vector entries(const std::filesystem::path& store) const override { + std::vector out; + std::error_code ec; + for (auto it = std::filesystem::directory_iterator(store, ec); !ec && it != std::filesystem::directory_iterator{}; + it.increment(ec)) { + if (!it->is_directory(ec)) continue; + const auto pkg = it->path().filename().string(); + const auto sep = pkg.find("-x-"); + if (sep == std::string::npos) continue; + std::error_code vec; + for (auto v = std::filesystem::directory_iterator(it->path(), vec); !vec && v != std::filesystem::directory_iterator{}; + v.increment(vec)) { + if (!v->is_directory(vec)) continue; + const auto ver = v->path().filename().string(); + if (ver.empty() || ver.front() == '.') continue; + out.push_back({pkg.substr(0, sep), pkg.substr(sep + 3), ver, v->path()}); + } + } + return out; + } + mcpp::provider::StoreEntry entry_of(const std::filesystem::path& path) const override { + // /-x-//...: the chain of ancestors, leaf first. + std::vector chain; + for (auto p = path;; p = p.parent_path()) { + chain.push_back(p); + if (!p.has_parent_path() || p.parent_path() == p) break; + } + for (std::size_t i = 1; i < chain.size(); ++i) { + if (chain[i].filename() != "xpkgs") continue; + const auto pkg = chain[i - 1].filename().string(); + const auto sep = pkg.find("-x-"); + if (sep == std::string::npos) return {}; + mcpp::provider::StoreEntry e{pkg.substr(0, sep), pkg.substr(sep + 3), {}, {}}; + if (i >= 2) { + e.version = chain[i - 2].filename().string(); + e.dir = chain[i - 2]; + } + return e; + } + return {}; + } + std::string kind_of(const std::filesystem::path& path) const override { + const auto s = path.generic_string(); + if (s.find("registry/data/xpkgs") != std::string::npos) return "payload"; + if (s.find("registry/subos") != std::string::npos) return "environment"; + if (s.find("registry/data/") != std::string::npos) return "index"; + return {}; + } + std::filesystem::path environment(const std::filesystem::path& home, std::string_view name) const override { + return home / "subos" / std::string(name); + } + std::filesystem::path project_home(const std::filesystem::path& projectRoot) const override { + return projectRoot / ".mcpp" / ".xlings"; + } + std::filesystem::path description(const std::filesystem::path& environment) const override { + return environment / ".xlings.json"; + } + std::vector index_dirs(const std::filesystem::path& dataRoot) const override { + std::vector out; + std::error_code ec; + for (auto it = std::filesystem::directory_iterator(dataRoot, ec); !ec && it != std::filesystem::directory_iterator{}; + it.increment(ec)) { + if (!it->is_directory(ec)) continue; + const auto n = it->path().filename().string(); + if (n == "xpkgs" || n == "runtimedir" || n == "stale" || n.starts_with(".")) continue; + if (n == "xim-index-repos") { + std::error_code rec; + for (auto r = std::filesystem::directory_iterator(it->path(), rec); + !rec && r != std::filesystem::directory_iterator{}; r.increment(rec)) + if (r->is_directory(rec)) out.push_back(r->path()); + continue; + } + out.push_back(it->path()); + } + std::ranges::sort(out); + return out; + } + bool index_data_present(const std::filesystem::path& data) const override { + std::error_code ec; + auto reposDir = data / "xim-index-repos"; + if (std::filesystem::exists(reposDir / "xim-indexrepos.json", ec)) return true; + for (const auto& base : {reposDir, data}) { + if (!std::filesystem::is_directory(base, ec)) continue; + for (auto const& entry : std::filesystem::directory_iterator(base, ec)) + if (entry.is_directory(ec) && std::filesystem::exists(entry.path() / "pkgs", ec)) return true; + } + return false; + } + std::vector descriptor_candidates(const std::filesystem::path& data, + std::string_view name) const override { + std::vector out; + if (name.empty()) return out; + const std::string subdir(1, name[0]), fname = std::string(name) + ".lua"; + // xim-pkgindex first, then each sub-index. + out.push_back(data / "xim-pkgindex" / "pkgs" / subdir / fname); + std::error_code ec; + std::vector repos; + for (auto it = std::filesystem::directory_iterator(data / "xim-index-repos", ec); + !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) + if (it->is_directory(ec)) repos.push_back(it->path()); + std::ranges::sort(repos); + for (const auto& r : repos) out.push_back(r / "pkgs" / subdir / fname); + return out; + } + mcpp::provider::runtime_info::Info runtime_info(const std::filesystem::path& environment) const override { + return subos::read(environment); + } + std::string index_revision(const std::filesystem::path& indexDir) const override { + return xlings::index_revision(indexDir).value_or(std::string{}); + } + std::vector index_roots_of(const std::filesystem::path& home) const override { + return paths::index_roots_of(home); + } + std::vector stores_of(const std::filesystem::path& home) const override { + return paths::stores_of(home); + } + std::vector indices_of(const std::filesystem::path& home) const override { + return paths::indices_of(home); + } + std::filesystem::path own_store(const std::filesystem::path& mcppHome) const override { + // xlings takes a store only by the name `xpkgs` (a payload's path is + // its identity, /-x-/). + return mcppHome / "xpkgs"; + } + std::vector upstream_of(const std::filesystem::path& dir) const override { + std::error_code ec; + // An mcpp home: the provider home under it serves xim, its store the + // namespaces of mcpp's indices -- never xim from a plain xlings home's + // view of mcpp's packages (design §8.3). + if (std::filesystem::is_regular_file(dir / "registry" / ".xlings.json", ec)) + return {{dir / "registry", {}, {"xim"}}, {dir / "registry", own_store(dir), {}}}; + if (std::filesystem::is_regular_file(dir / ".xlings.json", ec)) + return {{dir, {}, {"xim"}}}; + return {}; + } + bool is_provider_index(std::string_view name) const override { + // xlings' default index and the ones it adds beside it. + return name == "xim" || name == "awesome" || name == "scode" || name == "d2x"; + } +}; + +class XlingsProvider final : public mcpp::provider::Provider { +public: + explicit XlingsProvider(mcpp::provider::Location at) : env_(std::move(at)) {} + const mcpp::provider::Location& location() const override { return env_; } + + std::filesystem::path payload(const mcpp::provider::PackageRef& ref) const override { + return paths::xpkg_payload(env_, ref).value_or(std::filesystem::path{}); + } + std::filesystem::path pinned(std::string_view tool, std::string_view version) const override { + return paths::xim_tool(env_, tool, version); + } + std::filesystem::path versions(std::string_view tool) const override { return paths::xim_tool_root(env_, tool); } + void remember(const mcpp::provider::DataEvent& installTargets) const override { + paths::record_resolutions(env_, paths::parse_install_targets(installTargets.payloadJson)); + } + + std::filesystem::path store() const override { return paths::xpkgs_base(env_); } + std::vector stores() const override { return paths::stores(env_); } + std::vector status(std::span refs) const override { + std::vector out; + if (refs.empty()) return out; + nlohmann::json args{{"targets", nlohmann::json::array()}}; + for (const auto& r : refs) args["targets"].push_back(mcpp::provider::to_string(r)); + auto r = xlings::call(env_, "payload_status", args.dump(), nullptr); + if (!r || r->exitCode != 0) return out; + for (const auto& d : r->dataEvents) { + if (d.dataKind != "payload_status") continue; + auto j = nlohmann::json::parse(d.payloadJson, nullptr, false); + if (!j.is_object() || !j.contains("targets") || !j["targets"].is_array()) continue; + const auto str = [](const nlohmann::json& o, const char* k) { + return o.contains(k) && o[k].is_string() ? o[k].get() : std::string{}; + }; + for (const auto& t : j["targets"]) { + mcpp::provider::PayloadStatus ps; + ps.request = str(t, "request"); + ps.ns = str(t, "namespace"); + ps.name = str(t, "name"); + ps.version = str(t, "version"); + if (t.contains("revision") && t["revision"].is_number_integer()) ps.revision = t["revision"].get(); + ps.dir = str(t, "payload_dir"); + ps.origin = str(t, "origin"); + if (t.contains("installed_revision") && t["installed_revision"].is_number_integer()) + ps.installedRevision = t["installed_revision"].get(); + ps.complete = t.value("complete", false); + ps.error = str(t, "error"); + out.push_back(std::move(ps)); + } + } + if (out.size() != refs.size()) out.clear(); + return out; + } + std::filesystem::path sysroot() const override { return paths::sysroot(env_); } + std::filesystem::path bin_dir() const override { return paths::sandbox_bin(env_); } + bool initialized() const override { + std::error_code ec; + return std::filesystem::exists(paths::sandbox_init_marker(env_), ec); + } + std::filesystem::path data_root() const override { return paths::index_data(env_); } + std::filesystem::path config_file() const override { + return (env_.projectDir.empty() ? env_.home : env_.projectDir) / ".xlings.json"; + } + + std::expected + call(std::string_view operation, std::string_view argsJson, mcpp::provider::EventHandler* handler) const override { + return xlings::call(env_, operation, argsJson, handler); + } + int install_with_progress(std::string_view target, const mcpp::provider::BootstrapProgressCallback& cb, + bool quiet) const override { + return xlings::install_with_progress(env_, target, cb, quiet); + } + int install_direct(std::string_view target, bool quiet) const override { + return xlings::install_direct(env_, target, quiet); + } + std::string command_prefix() const override { return build_command_prefix(env_); } + std::string operation_command(std::string_view operation, std::string_view argsJson) const override { + return build_interface_command(env_, operation, argsJson); + } + + void seed(std::span repos, std::string_view mirror, + const mcpp::provider::ProjectEnv& penv) const override { + seed_xlings_json(env_, repos, mirror, penv); + } + void set_upstreams(std::span upstreams) const override { + const auto file = env_.home / ".xlings.json"; + std::ifstream in(file); + if (!in) return; + auto j = nlohmann::json::parse(in, nullptr, false); + in.close(); + if (!j.is_object()) return; + nlohmann::json list = nlohmann::json::array(); + for (const auto& u : upstreams) { + nlohmann::json e{{"home", u.home.generic_string()}}; + if (!u.store.empty()) e["store"] = u.store.generic_string(); + if (!u.namespaces.empty()) e["namespaces"] = u.namespaces; + list.push_back(std::move(e)); + } + if (list.empty() ? !j.contains("upstreams") : (j.contains("upstreams") && j["upstreams"] == list)) return; + if (list.empty()) j.erase("upstreams"); + else j["upstreams"] = std::move(list); + auto tmp = file; + tmp += std::format(".mcpp-{:x}", std::random_device{}()); + { std::ofstream(tmp) << j.dump(2) << "\n"; } + std::error_code ec; + std::filesystem::rename(tmp, file, ec); + if (ec) std::filesystem::remove(tmp, ec); + } + std::vector duplicates() const override { + std::vector out; + std::ifstream in(env_.home / ".xlings.json"); + auto j = nlohmann::json::parse(in, nullptr, false); + if (!j.is_object() || !j.contains("upstreams") || !j["upstreams"].is_array()) return out; + std::error_code ec; + for (const auto& store : paths::stores(env_)) { + for (const auto& entry : XlingsLayout{}.entries(store)) { + if (std::filesystem::is_symlink(entry.dir, ec) || std::filesystem::is_empty(entry.dir, ec)) continue; + const auto mine = paths::installed_revision(entry.dir); + if (!mine) continue; + for (const auto& u : j["upstreams"]) { + if (!u.is_object() || !u.contains("home") || !u["home"].is_string()) continue; + if (u.contains("namespaces") && u["namespaces"].is_array() && !u["namespaces"].empty() + && std::ranges::none_of(u["namespaces"], [&](const nlohmann::json& n) { return n == entry.ns; })) + continue; + const std::filesystem::path home = u["home"].get(); + const std::filesystem::path ustore = u.contains("store") && u["store"].is_string() + ? std::filesystem::path(u["store"].get()) : home / "data" / "xpkgs"; + const auto candidate = ustore / entry.dir.parent_path().filename() / entry.dir.filename(); + if (!std::filesystem::is_directory(candidate, ec) || std::filesystem::is_empty(candidate, ec)) continue; + if (paths::installed_revision(candidate) != mine) continue; + out.push_back({entry.dir, candidate}); + break; + } + } + } + return out; + } + std::string link_to_upstream(const mcpp::provider::Duplicate& d) const override { + std::error_code ec; + auto link = d.local; + link += std::format(".mcpp-link-{:x}", std::random_device{}()); + if (!mcpp::platform::create_directory_link(d.upstream, link)) + return std::format("cannot link {} to {}", link.string(), d.upstream.string()); + if (auto swapped = mcpp::platform::exchange_paths(link, d.local); !swapped) { + std::filesystem::remove(link, ec); + return swapped.error(); + } + // The copy this home installed, now at the scratch name. + std::filesystem::remove_all(link, ec); + return {}; + } + int show_config() const override { return config_show(env_); } + int set_mirror(std::string_view mirror, bool quiet) const override { return config_set_mirror(env_, mirror, quiet); } + void ensure_initialized(bool quiet) const override { ensure_init(env_, quiet); } + void pin_runtime(std::string_view runtimeId, bool relink) const override { + // `.xlings.json` `runtime: {"glibc": "2.44.3"}` is the pin; set + // through xlings itself, and only when it says otherwise or the + // links must be made again: `config --runtime` makes /abi + // (the loader a toolchain names) point at the pinned payload, and + // installing that payload alone does not. + const auto at = runtimeId.find('@'); + if (at == std::string_view::npos) return; + const std::string family(runtimeId.substr(0, at)), version(runtimeId.substr(at + 1)); + std::ifstream in(env_.home / ".xlings.json"); + auto j = nlohmann::json::parse(in, nullptr, false); + std::error_code ec; + if (!relink && std::filesystem::exists(std::filesystem::symlink_status(env_.home / "abi", ec)) + && j.is_object() && j.contains("runtime") && j["runtime"].is_object() + && j["runtime"].value(family, std::string{}) == version) + return; + mcpp::provider::Location global = env_; + global.projectDir.clear(); + (void)run_capture(build_command_prefix(global) + " config --runtime " + + mcpp::platform::shell::quote(runtimeId) + " " + + std::string(mcpp::platform::shell::silent_redirect)); + } + void ensure_patchelf(bool quiet, const mcpp::provider::BootstrapProgressCallback& cb) const override { + xlings::ensure_patchelf(env_, quiet, cb); + } + void ensure_ninja(bool quiet, const mcpp::provider::BootstrapProgressCallback& cb) const override { + xlings::ensure_ninja(env_, quiet, cb); + } + std::filesystem::path usable_nasm() const override { return find_usable_nasm(env_).value_or(std::filesystem::path{}); } + std::filesystem::path placed_nasm() const override { return find_sandbox_nasm(env_).value_or(std::filesystem::path{}); } + bool nasm_is_from_host(const std::filesystem::path& bin) const override { return xlings::nasm_is_from_host(env_, bin); } + + std::vector index_roots() const override { return paths::index_roots(env_); } + void own_indices(std::span owned) const override { + paths::own_indices(env_, owned); + } + mcpp::provider::IndexStatus default_index_status(std::int64_t ttlSeconds) const override { + return xlings::default_index_status(env_, ttlSeconds); + } + mcpp::provider::IndexStatus official_index_status(std::int64_t ttlSeconds) const override { + return xlings::official_index_status(env_, ttlSeconds); + } + bool index_fresh(std::int64_t ttlSeconds) const override { return is_index_fresh(env_, ttlSeconds); } + bool official_index_fresh(std::int64_t ttlSeconds) const override { return is_official_index_fresh(env_, ttlSeconds); } + bool official_package_index_fresh(std::string_view package, std::int64_t ttlSeconds) const override { + return is_official_package_index_fresh(env_, package, ttlSeconds); + } + bool official_package_present(std::string_view package) const override { + return xlings::official_package_present(env_, package); + } + int update_index(bool quiet) const override { return xlings::update_index(env_, quiet); } + void ensure_index_fresh(std::int64_t ttlSeconds, bool quiet) const override { + xlings::ensure_index_fresh(env_, ttlSeconds, quiet); + } + +private: + mcpp::provider::Location env_; +}; + +const mcpp::provider::Layout& layout_instance() { + static const XlingsLayout instance; + return instance; +} + +} // namespace mcpp::provider::xlings diff --git a/src/xlings/subos_info.cppm b/src/provider/xlings/subos_info.cppm similarity index 81% rename from src/xlings/subos_info.cppm rename to src/provider/xlings/subos_info.cppm index 70d478829..0207d32cb 100644 --- a/src/xlings/subos_info.cppm +++ b/src/provider/xlings/subos_info.cppm @@ -1,4 +1,4 @@ -// mcpp.xlings.subos_info — read the `subos_info` block xlings writes into a +// mcpp.provider.xlings.subos_info — read the `subos_info` block xlings writes into a // subos's own `.xlings.json`. // // WHAT THIS IS FOR @@ -33,96 +33,23 @@ // // Design: .agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md §3-S3 -export module mcpp.xlings.subos_info; +export module mcpp.provider.xlings.subos_info; import std; import mcpp.libs.json; import mcpp.platform; +import mcpp.provider; -export namespace mcpp::xlings::subos { - -// The schema this build understands. A HIGHER one on disk is still read — we -// take the fields we know and say so. Refusing outright would let a newer -// xlings break an older mcpp, which is the failure shape the index-floor -// incident already paid for once: publishing data must not invalidate the -// program that reads it. -inline constexpr int kSupportedSchema = 1; +export namespace mcpp::provider::xlings::subos { +// The description's types and the runtime's family are the port's +// (mcpp.provider runtime_info); this module reads xlings' block of them. +inline constexpr int kSupportedSchema = mcpp::provider::runtime_info::kSupportedSchema; inline constexpr std::string_view kBlock = "subos_info"; +std::string family_of(std::string_view runtime, std::string_view arch = "x86_64") { return mcpp::provider::runtime_info::family_of(runtime, arch); } -struct EnvDecl { - std::string var; - std::string op; // "set" | "prepend" - std::string value; // may contain ${subosdir} -}; - -struct Provider { - std::string binding; // "@" - std::vector decls; -}; - -struct PackageIdentity { - std::string namespace_; - std::string name; - std::string version; - std::string source; -}; - -struct CapabilityProvider { - std::string capability; - PackageIdentity provider; -}; - -struct RuntimeArtifact { - std::string role; - PackageIdentity provider; - std::filesystem::path path; - std::string provenance; - std::string abi; - std::string digest; - std::string hostFingerprint; -}; - -struct Info { - int schema = 0; - std::string runtime; // "glibc@2.39" - std::string hostGlibc; // optional creation-host floor, e.g. "2.43" - std::vector providers; // sorted by binding - // Optional generic contract exported by xlings after it resolves provider - // recipes. mcpp interprets these facts and never performs provider/hardware - // discovery itself. - std::vector runtimeProviders; - std::vector runtimeArtifacts; - bool present = false; - // Non-empty ⇒ the caller MUST surface it. Never an error: a missing or - // newer block degrades the experience; it does not invalidate a build. - std::string note; -}; - -// The runtime string is self-describing: "glibc@2.39" says Linux/glibc -// without a second field that could disagree with it. -// -// Mirrors xlings's `subos::manifest::family_of`. Two implementations of one -// mapping is a cost, and the alternative — asking the xlings binary — costs -// a subprocess in the build's hot path and fails on exactly the machines -// where it matters (a sandbox xlings that has not been updated). The mapping -// is five rows and stable, and test_subos_info.cpp pins every one of them, so -// a drift is a test failure rather than a silent ABI disagreement. -std::string family_of(std::string_view runtime, - std::string_view arch = "x86_64") { - const auto at = runtime.find('@'); - const auto name = runtime.substr(0, at == std::string_view::npos - ? runtime.size() : at); - if (name == "glibc") return std::format("linux-{}-glibc", arch); - if (name == "musl") return std::format("linux-{}-musl", arch); - if (name == "wasi-libc") return "wasm32-wasi"; - if (name == "macos_sdk") return std::format("darwin-{}", arch); - if (name == "ucrt") return std::format("windows-{}-ucrt", arch); - return "unknown"; -} - -Info read(const std::filesystem::path& subosDir) { - Info info; +mcpp::provider::runtime_info::Info read(const std::filesystem::path& subosDir) { + mcpp::provider::runtime_info::Info info; auto path = subosDir / ".xlings.json"; std::error_code ec; if (!std::filesystem::exists(path, ec)) { @@ -193,11 +120,11 @@ Info read(const std::filesystem::path& subosDir) { if (auto envs = it->find("envs"); envs != it->end() && envs->is_object()) { for (auto e = envs->begin(); e != envs->end(); ++e) { if (!e.value().is_array()) continue; - Provider prov; + mcpp::provider::runtime_info::Declarer prov; prov.binding = e.key(); for (auto const& d : e.value()) { if (!d.is_object()) continue; - EnvDecl decl; + mcpp::provider::runtime_info::EnvDecl decl; if (auto x = d.find("var"); x != d.end() && x->is_string()) decl.var = x->get(); if (auto x = d.find("op"); x != d.end() && x->is_string()) @@ -223,7 +150,7 @@ Info read(const std::filesystem::path& subosDir) { if (auto contract = it->find("runtime_contract"); contract != it->end() && contract->is_object()) { auto read_identity = [](const nlohmann::json& value) { - PackageIdentity id; + mcpp::provider::runtime_info::PackageIdentity id; if (!value.is_object()) return id; id.namespace_ = value.value("namespace", ""); id.name = value.value("name", ""); @@ -239,7 +166,7 @@ Info read(const std::filesystem::path& subosDir) { providers != contract->end() && providers->is_array()) { for (auto const& value : *providers) { if (!value.is_object()) continue; - CapabilityProvider provider; + mcpp::provider::runtime_info::CapabilityProvider provider; provider.capability = value.value("capability", ""); if (auto identity = value.find("provider"); identity != value.end()) provider.provider = read_identity(*identity); @@ -254,7 +181,7 @@ Info read(const std::filesystem::path& subosDir) { artifacts != contract->end() && artifacts->is_array()) { for (auto const& value : *artifacts) { if (!value.is_object()) continue; - RuntimeArtifact artifact; + mcpp::provider::runtime_info::RuntimeArtifact artifact; artifact.role = value.value("role", ""); if (auto identity = value.find("provider"); identity != value.end()) artifact.provider = read_identity(*identity); @@ -297,7 +224,7 @@ Info read(const std::filesystem::path& subosDir) { // cosmetic here: it decides which provider wins a list variable, and // libglvnd resolves GL vendors by exactly that order. std::sort(info.providers.begin(), info.providers.end(), - [](Provider const& a, Provider const& b) { + [](mcpp::provider::runtime_info::Declarer const& a, mcpp::provider::runtime_info::Declarer const& b) { return a.binding < b.binding; }); @@ -326,7 +253,7 @@ Info read(const std::filesystem::path& subosDir) { // never matches and the joined value is a corrupt list. Caught by CI on // Windows, not by any amount of reading. std::vector> -resolve_env(const Info& info, const std::filesystem::path& subosDir, +resolve_env(const mcpp::provider::runtime_info::Info& info, const std::filesystem::path& subosDir, const std::function(std::string_view)>& ambient = {}) { std::vector> out; @@ -412,4 +339,4 @@ resolve_env(const Info& info, const std::filesystem::path& subosDir, return out; } -} // namespace mcpp::xlings::subos +} // namespace mcpp::provider::xlings::subos diff --git a/src/xlings/xlings.cppm b/src/provider/xlings/xlings.cppm similarity index 85% rename from src/xlings/xlings.cppm rename to src/provider/xlings/xlings.cppm index c64d7e5e2..af191cd4a 100644 --- a/src/xlings/xlings.cppm +++ b/src/provider/xlings/xlings.cppm @@ -11,12 +11,12 @@ module; #include // stderr #include -export module mcpp.xlings; +export module mcpp.provider.xlings; import std; +import mcpp.provider; import mcpp.pm.compat; import mcpp.pm.index_contract; -import mcpp.pm.index_snapshot; import mcpp.platform; import mcpp.log; import mcpp.ui; // closing notices of the refresh guard @@ -24,15 +24,11 @@ import mcpp.home; import mcpp.xpkg_version; import mcpp.libs.json; -export namespace mcpp::xlings { +export namespace mcpp::provider::xlings { // ─── Env: resolved xlings binary + home directory ─────────────────── -struct Env { - std::filesystem::path binary; // xlings binary path - std::filesystem::path home; // XLINGS_HOME directory - std::filesystem::path projectDir; // XLINGS_PROJECT_DIR (empty = global mode) -}; +using Env = mcpp::provider::Location; // ─── Pinned version constants ─────────────────────────────────────── @@ -122,7 +118,15 @@ namespace pinned { // depended on a failed one ran their hooks anyway, and an extraction said // nothing until it ended. mcpp reads the failure's code for its hint and // the `extract` and `hook` progress events for its `Installing` lines. - inline constexpr std::string_view kXlingsVersion = "2026.10.10.2"; + // + // Eighth, at 2026.10.16.1 (openxlings/xlings#676). Below it, an index's + // data and payloads could only live in xlings' own home, `update` fetched + // every index including the ones mcpp refreshes itself, a home could not + // install from another by link, and nothing answered which payloads were + // complete. mcpp now places its indices and store in its home (`dir`, + // `store`), stages each refresh (`sync_index`), names upstreams, reads + // `payload_status`, and checks all of it through `interface --version`. + inline constexpr std::string_view kXlingsVersion = "2026.10.16.1"; inline constexpr std::string_view kNasmVersion = "3.02"; } @@ -158,11 +162,7 @@ namespace paths { // that layout (`xpkgs_base`, `xim_tool`). A second place deriving it is // the shape this codebase has paid for repeatedly: it does not fail when // you add it, it fails later, somewhere else. - struct XpkgRef { - std::string ns; // "xim" when the spec omitted one - std::string name; - std::string version; // empty = unpinned - }; + using XpkgRef = mcpp::provider::PackageRef; // Parse `[:][@]` exactly as a manifest writes it. XpkgRef parse_xpkg_ref(std::string_view spec); @@ -294,6 +294,39 @@ namespace paths { // index data root: env.home / "data" std::filesystem::path index_data(const Env& env); + // ── Where each index is, and where its payloads go (design 2026-10-16 + // §5): an `index_repos` entry may name its own data directory and store + // (xlings 2026.10.16.1, X1/X2); mcpp names them for the indices it owns. + struct DeclaredIndex { + std::string name; + std::filesystem::path dir; // where its data is (absolute) + std::filesystem::path store; // where its payloads go (absolute; empty: the home's) + bool owned = false; // a `dir` was declared: its owner refreshes it + }; + // The entries of this env's configuration (a project's, then the home's + // when the env has a project). + std::vector declared_indices(const Env& env); + // Where the index `name` keeps its data here. + std::filesystem::path index_dir_of(const Env& env, std::string_view name); + // Every store a payload of this env may be in: the declared ones, then + // the home's. A payload mcpp installed into the home's store before its + // index had a store of its own is found there (COMPAT(internal:legacy-registry-store)). + std::vector stores(const Env& env); + // Every directory an index of this env's scope (its project's when it + // has one, else the home's) keeps its data in, sorted. + std::vector index_roots(const Env& env); + // The same of a provider home (the global one, or a project's), read + // from its configuration alone: its index directories, its stores. + std::vector index_roots_of(const std::filesystem::path& home); + std::vector stores_of(const std::filesystem::path& home); + // The indexes a provider home serves: xim, then its configuration's + // `index_repos`; empty when it has no configuration. + std::vector indices_of(const std::filesystem::path& home); + // Record where the indices mcpp owns keep their data and payloads in + // this env's configuration, and move data an index kept at the home's + // own place there, once (design §5.3). + void own_indices(const Env& env, std::span owned); + // sandbox init marker: env.home / "subos" / "default" / ".xlings.json" std::filesystem::path sandbox_init_marker(const Env& env); @@ -388,36 +421,17 @@ std::string build_interface_command(const Env& env, // ─── NDJSON event types ───────────────────────────────────────────── -struct ProgressEvent { - std::string phase; // "download", "extract", "configure", ... - int percent; // 0..100 - std::string message; -}; +using ProgressEvent = mcpp::provider::ProgressEvent; -struct LogEvent { - std::string level; // "debug" | "info" | "warn" | "error" - std::string message; -}; +using LogEvent = mcpp::provider::LogEvent; -struct DataEvent { - std::string dataKind; // "install_plan", "styled_list", ... - std::string payloadJson;// raw JSON (let caller parse) -}; +using DataEvent = mcpp::provider::DataEvent; -struct ErrorEvent { - std::string code; - std::string message; - std::string hint; - bool recoverable = false; -}; +using ErrorEvent = mcpp::provider::ErrorEvent; -struct ResultEvent { - int exitCode = 0; - std::string dataJson; // additional payload, may be empty -}; +using ResultEvent = mcpp::provider::ResultEvent; -using Event = std::variant; +using Event = mcpp::provider::Event; // Parse one NDJSON line into an Event. std::optional parse_event_line(std::string_view line); @@ -431,17 +445,7 @@ std::string extract_object(std::string_view text, std::string_view key); // ─── Subprocess call ──────────────────────────────────────────────── -struct CallResult { - int exitCode = 0; - std::vector dataEvents; - std::optional error; - std::string resultJson; - // xlings' own error-level lines from its stderr, the last few, kept only - // when the call failed (#614). The NDJSON error event carries a summary - // ("config hook failed"); the line that says WHICH binding was rejected and - // why is a log line on stderr, which this call used to discard. - std::vector stderrTail; -}; +using CallResult = mcpp::provider::CallResult; // The error-level lines of an xlings invocation's stderr, the last `limit` of // them (#614). A line is kept when it contains "error" in any case, contains @@ -450,14 +454,7 @@ struct CallResult { // install. std::vector stderr_error_tail(std::string_view text, std::size_t limit = 20); -struct EventHandler { - virtual ~EventHandler() = default; - virtual void on_progress(const ProgressEvent&) {} - virtual void on_log (const LogEvent&) {} - virtual void on_data (const DataEvent&) {} - virtual void on_error (const ErrorEvent&) {} - virtual void on_result (const ResultEvent&) {} -}; +using EventHandler = mcpp::provider::EventHandler; std::expected call(const Env& env, std::string_view capability, @@ -465,20 +462,11 @@ call(const Env& env, std::string_view capability, // ─── Bootstrap progress types ─────────────────────────────────────── -struct BootstrapFile { - std::string name; // xim package id, e.g. "xim:patchelf@0.18.0" - double downloadedBytes = 0; - double totalBytes = 0; - bool started = false; - bool finished = false; -}; +using BootstrapFile = mcpp::provider::BootstrapFile; -struct BootstrapProgress { - std::vector files; - double elapsedSec = 0; -}; +using BootstrapProgress = mcpp::provider::BootstrapProgress; -using BootstrapProgressCallback = std::function; +using BootstrapProgressCallback = mcpp::provider::BootstrapProgressCallback; // Run xlings install with progress callback (used by bootstrap functions). // When not `quiet` and stderr is a TTY, an elapsed-time spinner is shown @@ -507,30 +495,13 @@ int install_direct(const Env& env, std::string_view target, bool quiet = false); // The project's build-environment payload (L-1), materialized 1:1 into the // `.xlings.json` keys xlings already understands. Plain types (no manifest // dependency); the caller fills it from a manifest's [xlings] section. -struct ProjectEnv { - std::vector deps; // → "deps" - std::vector> workspace; // → "workspace" - std::string subos; // → "subos" - // No `envs`: the key was materialised here and read by nothing. See - // .agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md §4. - bool empty() const { - return deps.empty() && workspace.empty() && subos.empty(); - } -}; +using ProjectEnv = mcpp::provider::ProjectEnv; // One index_repos entry for seed_xlings_json. `artifact`/`source` are the // xlings >= 0.4.68 per-repo artifact-sync fields (#269); empty means "do not // emit the key", which keeps pre-artifact output byte-identical and matches // xlings' "undeclared = plain git" semantics. Older xlings ignores both keys. -struct SeedRepo { - std::string name; - std::string url; - std::string artifact; // artifact source base, e.g. https://github.com/xlings-res/mcpp-index - std::string source; // "auto" | "artifact" | "git" - // Non-empty: `artifact` is written as the region object - // {"GLOBAL": artifact, "CN": artifactCn} (xlings #377). - std::string artifactCn; -}; +using SeedRepo = mcpp::provider::SeedRepo; void seed_xlings_json(const Env& env, std::span repos, @@ -583,7 +554,7 @@ std::optional find_sandbox_nasm(const Env& env); // still absent afterwards is absent upstream. Lives here, in the leaf module, // because both users need it and the policy layer (mcpp.pm.index_refresh) // imports this module — the reverse would be a cycle. -inline constexpr std::int64_t kIndexRefreshDebounceSeconds = 120; +inline constexpr std::int64_t kIndexRefreshDebounceSeconds = mcpp::provider::kIndexRefreshDebounceSeconds; // ─── Bounds on the xlings children mcpp starts (#648 A3) ─────────────── // @@ -605,7 +576,7 @@ inline constexpr std::int64_t kIndexRefreshDebounceSeconds = 120; // only a total bound can apply, and it is generous: a toolchain archive on a // slow link is not a hang; // - a local command (the one-time sandbox init) gets a total bound. -inline constexpr std::chrono::seconds kDefaultIndexRefreshTimeout{120}; +inline constexpr std::chrono::seconds kDefaultIndexRefreshTimeout = mcpp::provider::kDefaultIndexRefreshTimeout; inline constexpr std::chrono::seconds kInterfaceIdleTimeout{300}; inline constexpr std::chrono::seconds kDirectInstallTimeout{3 * 3600}; inline constexpr std::chrono::seconds kLocalCommandTimeout{600}; @@ -651,13 +622,7 @@ bool official_package_present(const Env& env, std::string_view packageName); // ─── Index status (read-only, offline) ────────────────────────────── // Snapshot of a local index directory — computed without touching the // network, for `mcpp index status`. -struct IndexStatus { - std::filesystem::path dir; // on-disk index directory - bool present; // pkgs/ tree exists locally - bool fresh; // refreshed within ttlSeconds - std::int64_t ageSeconds; // since last refresh marker, -1 if unknown - std::optional rev; // index CONTENT identity, nullopt if absent -}; +using IndexStatus = mcpp::provider::IndexStatus; IndexStatus default_index_status(const Env& env, std::int64_t ttlSeconds); IndexStatus official_index_status(const Env& env, std::int64_t ttlSeconds); @@ -676,13 +641,22 @@ std::optional index_revision(const std::filesystem::path& indexDir) std::expected run_capture(const std::string& cmd); -} // namespace mcpp::xlings +// What the xlings at `binary` is (`xlings interface --version`, protocol +// 1.10+): its version, protocol, capabilities, features and home layouts; +// `answered` false for an xlings before the handshake. +mcpp::provider::ProviderInfo describe(const std::filesystem::path& binary); + +// Why that xlings cannot serve this mcpp (design 2026-10-16 §12): empty when +// it can. The pinned version, and every feature mcpp relies on. +std::string unmet(const mcpp::provider::ProviderInfo& info); + +} // namespace mcpp::provider::xlings // ═══════════════════════════════════════════════════════════════════════ // Implementation // ═══════════════════════════════════════════════════════════════════════ -namespace mcpp::xlings { +namespace mcpp::provider::xlings { namespace { @@ -698,7 +672,7 @@ void print_status(std::string_view verb, std::string_view msg) { } std::filesystem::path default_index_dir(const Env& env) { - return paths::index_data(env) / "mcpplibs"; + return paths::index_dir_of(env, "mcpplibs"); } std::filesystem::path official_index_dir(const Env& env) { @@ -919,6 +893,43 @@ std::optional index_revision(const std::filesystem::path& indexDir) return read_index_revision(indexDir); } +// ─── The handshake ────────────────────────────────────────────────── + +mcpp::provider::ProviderInfo describe(const std::filesystem::path& binary) { + mcpp::provider::ProviderInfo info; + auto out = run_capture(std::format("{} interface --version {}", shq(binary.string()), + mcpp::platform::null_redirect)); + if (!out) return info; + auto j = nlohmann::json::parse(*out, nullptr, false); + if (!j.is_object()) return info; + const auto strings = [&](const char* key) { + std::vector v; + if (j.contains(key) && j[key].is_array()) + for (auto& e : j[key]) if (e.is_string()) v.push_back(e.get()); + return v; + }; + info.version = j.value("version", std::string{}); + info.protocol = j.value("protocol_version", std::string{}); + info.capabilities = strings("capabilities"); + info.features = strings("features"); + if (j.contains("home_layouts") && j["home_layouts"].is_array()) + for (auto& e : j["home_layouts"]) if (e.is_number_integer()) info.homeLayouts.push_back(e.get()); + info.answered = !info.version.empty(); + return info; +} + +std::string unmet(const mcpp::provider::ProviderInfo& info) { + if (!info.answered) return "answers no handshake (an xlings before 2026.10.16.1)"; + if (mcpp::xpkg_version::compare_keys(info.version, pinned::kXlingsVersion) < 0) + return std::format("is {}, older than {}", info.version, pinned::kXlingsVersion); + for (auto feature : {"index_dir", "index_store", "upstreams", "payload_status", "sync_index"}) + if (std::ranges::find(info.features, feature) == info.features.end()) + return std::format("does not offer `{}`", feature); + if (std::ranges::find(info.capabilities, "install_packages") == info.capabilities.end()) + return "does not offer `install_packages`"; + return {}; +} + // ─── run_capture ──────────────────────────────────────────────────── std::expected run_capture(const std::string& cmd) { @@ -967,25 +978,7 @@ std::filesystem::path xim_tool(const Env& env, std::string_view tool, } // Parse `[:][@]` exactly as a manifest writes it. -XpkgRef parse_xpkg_ref(std::string_view spec) { - XpkgRef r; - if (auto colon = spec.find(':'); colon != std::string_view::npos) { - r.ns = std::string(spec.substr(0, colon)); - spec = spec.substr(colon + 1); - } else { - // The manifest's own default, spelled once here rather than at each - // call site: `deps = ["ninja"]` and `deps = ["xim:ninja"]` name the - // same package. - r.ns = "xim"; - } - if (auto at = spec.find('@'); at != std::string_view::npos) { - r.name = std::string(spec.substr(0, at)); - r.version = std::string(spec.substr(at + 1)); - } else { - r.name = std::string(spec); - } - return r; -} +XpkgRef parse_xpkg_ref(std::string_view spec) { return mcpp::provider::parse_ref(spec); } // Where that package's payload is, or nullopt if it is not installed. // @@ -1004,7 +997,9 @@ XpkgRef parse_xpkg_ref(std::string_view spec) { std::optional xpkg_payload(const Env& env, const XpkgRef& ref) { if (auto recorded = recorded_payload(env, ref)) return recorded; - return xpkg_payload_at(xpkgs_base(env), ref); + for (const auto& store : stores(env)) + if (auto found = xpkg_payload_at(store, ref)) return found; + return std::nullopt; } std::optional @@ -1116,9 +1111,11 @@ recorded_payload(const Env& env, const XpkgRef& ref) { // provisioning check treats "no answer" as not installed (#716). std::error_code ec; if (!std::filesystem::is_directory(dir, ec)) return std::nullopt; - const auto root = xpkgs_base(env) / std::format("{}-x-{}", ref.ns, ref.name); - if (dir.parent_path().lexically_normal() != root.lexically_normal()) return std::nullopt; - return dir; + for (const auto& store : stores(env)) { + const auto root = store / std::format("{}-x-{}", ref.ns, ref.name); + if (dir.parent_path().lexically_normal() == root.lexically_normal()) return dir; + } + return std::nullopt; } std::optional @@ -1305,6 +1302,150 @@ std::filesystem::path index_data(const Env& env) { return env.home / "data"; } +namespace { +std::vector declared_in(const std::filesystem::path& file, const std::filesystem::path& base, + const std::filesystem::path& dataRoot) { + std::vector out; + std::ifstream in(file); + if (!in) return out; + auto j = nlohmann::json::parse(in, nullptr, false); + if (!j.is_object() || !j.contains("index_repos") || !j["index_repos"].is_array()) return out; + const auto absolute = [&](const std::string& p) { + const std::filesystem::path path = p; + return path.is_absolute() ? path.lexically_normal() : (base / path).lexically_normal(); + }; + for (const auto& e : j["index_repos"]) { + if (!e.is_object() || !e.contains("name") || !e["name"].is_string()) continue; + DeclaredIndex d; + d.name = e["name"].get(); + if (e.contains("dir") && e["dir"].is_string() && !e["dir"].get().empty()) { + d.dir = absolute(e["dir"].get()); + d.owned = true; + } else { + d.dir = dataRoot / d.name; + } + if (e.contains("store") && e["store"].is_string() && !e["store"].get().empty()) + d.store = absolute(e["store"].get()); + out.push_back(std::move(d)); + } + return out; +} +} // namespace + +std::vector declared_indices(const Env& env) { + std::vector out; + if (!env.projectDir.empty()) + out = declared_in(env.projectDir / ".xlings.json", env.projectDir, env.projectDir / "data"); + for (auto& d : declared_in(env.home / ".xlings.json", env.home, index_data(env))) + if (std::ranges::none_of(out, [&](const DeclaredIndex& o) { return o.name == d.name; })) + out.push_back(std::move(d)); + return out; +} + +std::filesystem::path index_dir_of(const Env& env, std::string_view name) { + for (auto& d : declared_indices(env)) + if (d.name == name) return d.dir; + return index_data(env) / std::string(name); +} + +std::vector indices_of(const std::filesystem::path& home) { + std::error_code ec; + if (!std::filesystem::is_regular_file(home / ".xlings.json", ec)) return {}; + std::vector out{"xim"}; + for (auto& d : declared_in(home / ".xlings.json", home, home / "data")) + if (std::ranges::find(out, d.name) == out.end()) out.push_back(d.name); + return out; +} + +std::vector index_roots_of(const std::filesystem::path& home) { + const auto data = home / "data"; + std::vector out; + std::error_code ec; + const auto declared = declared_in(home / ".xlings.json", home, data); + // An owned index is read where its owner keeps it; a copy left at the + // provider's own place (written by a provider that did not know the + // owner's place) is not a second index. + const auto owned_name = [&](const std::filesystem::path& p) { + return std::ranges::any_of(declared, [&](const DeclaredIndex& d) { return d.owned && d.name == p.filename(); }); + }; + for (auto& e : std::filesystem::directory_iterator(data, ec)) + if (e.is_directory(ec) && !owned_name(e.path())) out.push_back(e.path()); + for (auto& d : declared) + if (d.owned && std::filesystem::is_directory(d.dir, ec) && std::ranges::find(out, d.dir) == out.end()) + out.push_back(d.dir); + std::ranges::sort(out); + return out; +} + +std::vector stores_of(const std::filesystem::path& home) { + std::vector out; + for (auto& d : declared_in(home / ".xlings.json", home, home / "data")) + if (!d.store.empty() && std::ranges::find(out, d.store) == out.end()) out.push_back(d.store); + if (auto own = home / "data" / "xpkgs"; std::ranges::find(out, own) == out.end()) out.push_back(own); + return out; +} + +std::vector index_roots(const Env& env) { + return index_roots_of(env.projectDir.empty() ? env.home : env.projectDir); +} + +void own_indices(const Env& env, std::span owned) { + const bool project = !env.projectDir.empty(); + const auto base = project ? env.projectDir : env.home; + const auto data = project ? env.projectDir / "data" : index_data(env); + const auto file = base / ".xlings.json"; + std::ifstream in(file); + if (!in) return; + auto j = nlohmann::json::parse(in, nullptr, false); + in.close(); + if (!j.is_object() || !j.contains("index_repos") || !j["index_repos"].is_array()) return; + bool changed = false; + for (const auto& o : owned) { + if (o.dir.empty()) continue; + for (auto& e : j["index_repos"]) { + if (!e.is_object() || e.value("name", std::string{}) != o.name) continue; + // Its data, from where the provider kept it, once: a rename on one + // filesystem, a copy and a removal across two. + std::error_code ec; + const auto old = data / o.name; + if (std::filesystem::is_directory(old, ec) && !std::filesystem::exists(o.dir, ec)) { + std::filesystem::create_directories(o.dir.parent_path(), ec); + std::filesystem::rename(old, o.dir, ec); + if (ec) { + ec.clear(); + std::filesystem::copy(old, o.dir, std::filesystem::copy_options::recursive + | std::filesystem::copy_options::copy_symlinks, ec); + if (!ec) std::filesystem::remove_all(old, ec); + else std::filesystem::remove_all(o.dir, ec); + } + } + if (e.value("dir", std::string{}) != o.dir.generic_string()) { + e["dir"] = o.dir.generic_string(); + changed = true; + } + if (!o.store.empty() && e.value("store", std::string{}) != o.store.generic_string()) { + e["store"] = o.store.generic_string(); + changed = true; + } + } + } + if (!changed) return; + auto tmp = file; + tmp += std::format(".mcpp-{:x}", std::random_device{}()); + write_file(tmp, j.dump(2) + "\n"); + std::error_code ec; + std::filesystem::rename(tmp, file, ec); + if (ec) std::filesystem::remove(tmp, ec); +} + +std::vector stores(const Env& env) { + std::vector out; + for (auto& d : declared_indices(env)) + if (!d.store.empty() && std::ranges::find(out, d.store) == out.end()) out.push_back(d.store); + if (auto home = xpkgs_base(env); std::ranges::find(out, home) == out.end()) out.push_back(home); + return out; +} + std::filesystem::path sandbox_init_marker(const Env& env) { return env.home / "subos" / "default" / ".xlings.json"; } @@ -1907,6 +2048,10 @@ void seed_xlings_json(const Env& env, if (!repos[i].source.empty()) json += std::format(", \"source\": \"{}\"", json_escape(repos[i].source)); + if (!repos[i].dir.empty()) + json += std::format(", \"dir\": \"{}\"", json_escape(repos[i].dir.generic_string())); + if (!repos[i].store.empty()) + json += std::format(", \"store\": \"{}\"", json_escape(repos[i].store.generic_string())); json += std::format(" }}{}\n", i + 1 == repos.size() ? "" : ","); } json += " ],\n"; @@ -2146,28 +2291,69 @@ bool is_official_package_index_fresh(const Env& env, && official_index_cache_matches_package_file(env, packageName); } -namespace { -// The raw sync. Everything that makes a refresh SAFE lives in the wrapper -// below; this function's only job is to run `xlings update` and report. -int update_index_unguarded(const Env& env, bool quiet); -} // namespace - -// Guarded entry point — the ONE place every refresh in this process passes -// through, which is why the monotonicity guarantee is installed here rather -// than at the seven call sites. +// The one entry point every refresh in this process passes through. // // an index-side change must never take mcpp from "works" to "does not work" // -// A published index can raise its client-version floor (index.toml min_mcpp). -// `xlings update` rewrites the tree in place, so before this guard existed a -// floor bump replaced a readable index with an unreadable one and left no way -// back — the refresh itself was the thing that broke the machine. See -// mcpp.pm.index_snapshot for why the shape is archive/judge/restore rather -// than the stage-and-swap the original design assumed. +// A published index can raise its client-version floor (index.toml +// min_mcpp). An index mcpp owns (one whose data directory mcpp declared, +// design 2026-10-16 §6) is therefore fetched into a staging directory beside +// it (xlings' `sync_index` with a destination, X1), judged there, and swapped +// in only when this mcpp can read it -- or when the copy in use is no better. +// The rest (the provider's own indexes, which carry no floor of mcpp's) are +// refreshed by the provider in place; it leaves an owned index that is there +// to its owner. namespace { std::atomic g_index_refresh_timeout_s{kDefaultIndexRefreshTimeout.count()}; + +struct StagedOutcome { + std::vector kept; // the refreshed copy needs a newer mcpp; the old one stays + std::vector unusable; // no copy this mcpp can read + std::map requiredMcpp; +}; + +// One owned index: .staging- fetched, judged, then exchanged with +// (or renamed onto it when there is none yet); the staging directory +// holds the old copy afterwards and is removed. +int refresh_owned(const Env& env, const paths::DeclaredIndex& d, StagedOutcome& out) { + std::error_code ec; + auto staging = d.dir; + staging += std::format(".staging-{:x}", std::random_device{}()); + std::filesystem::create_directories(d.dir.parent_path(), ec); + const auto args = nlohmann::json{{"name", d.name}, {"dest", staging.generic_string()}}.dump(); + auto r = call(env, "sync_index", args, nullptr); + if (!r || r->exitCode != 0 || !std::filesystem::is_directory(staging, ec)) { + std::filesystem::remove_all(staging, ec); + return r ? (r->exitCode != 0 ? r->exitCode : 1) : 1; + } + const bool present = std::filesystem::exists(d.dir, ec); + const bool before = present && mcpp::pm::index_usable(d.dir); + if (!mcpp::pm::index_usable(staging)) { + if (auto c = mcpp::pm::read_index_contract(staging)) out.requiredMcpp[d.dir] = c->minMcpp; + if (before) { + std::filesystem::remove_all(staging, ec); + out.kept.push_back(d.dir); + return 0; + } + out.unusable.push_back(d.dir); + } + auto swapped = present ? mcpp::platform::exchange_paths(staging, d.dir) + : mcpp::platform::rename_no_replace(staging, d.dir); + std::filesystem::remove_all(staging, ec); + if (!swapped) { + mcpp::log::warn("index", std::format("the refreshed index `{}` could not replace the one in use: {}", + d.name, swapped.error())); + return 1; + } + mark_index_refreshed(d.dir); + return 0; } +// The raw sync. Everything that makes a refresh SAFE lives in the wrapper +// below; this function's only job is to run the provider's refresh and report. +int update_index_unguarded(const Env& env, bool quiet); +} // namespace + void set_index_refresh_timeout(std::chrono::seconds timeout) { g_index_refresh_timeout_s.store( timeout.count() > 0 ? timeout.count() : kDefaultIndexRefreshTimeout.count(), @@ -2179,46 +2365,7 @@ std::chrono::seconds index_refresh_timeout() { } int update_index(const Env& env, bool quiet) { - namespace snap = mcpp::pm::index_snapshot; - const auto dataRoot = paths::index_data(env); - - snap::GuardOutcome out; - int rc = snap::guarded_refresh(dataRoot, - [&] { return update_index_unguarded(env, quiet); }, out); - - // Report ONLY when the guard had to act. The common path — refresh keeps - // the index readable — must stay silent, or the notice becomes noise that - // users learn to skip past, which is the same as not printing it. - // - // A floor is not an error of the run (an index is data; mcpp is the - // program). Each case is one closing notice, printed after the command's - // own output (mcpp::ui::add_closing_notice), and only because this run - // refreshed the index. `quiet` governs the refresh's own narration, not - // these: they are the one thing the refresh has to say. - (void)quiet; - auto advice = [&](const std::filesystem::path& dir) { - auto it = out.requiredMcpp.find(dir); - return mcpp::pm::index_floor_upgrade_advice( - it == out.requiredMcpp.end() ? std::string_view{} : std::string_view{it->second}); - }; - for (auto& dir : out.rolledBack) { - mcpp::ui::add_closing_notice("MCPP_INDEX_REQUIRES_NEWER_MCPP", std::format( - "the refreshed package index `{}` requires a newer mcpp; this run " - "used the previous index. {}", dir.filename().string(), advice(dir))); - } - for (auto& dir : out.recovered) { - mcpp::ui::add_closing_notice("MCPP_INDEX_REQUIRES_NEWER_MCPP", std::format( - "the package index `{}` was restored from a local snapshot this mcpp " - "can read; the published index requires a newer mcpp. {}", - dir.filename().string(), advice(dir))); - } - for (auto& dir : out.stillUnusable) { - mcpp::ui::add_closing_notice("MCPP_INDEX_REQUIRES_NEWER_MCPP", std::format( - "the package index `{}` requires a newer mcpp and no earlier copy is " - "usable; packages it serves cannot be resolved. {}", - dir.filename().string(), advice(dir))); - } - return rc; + return update_index_unguarded(env, quiet); } namespace { @@ -2251,6 +2398,28 @@ int update_index_unguarded(const Env& env, bool quiet) { return 0; } + // The indices mcpp owns, each staged and judged before it replaces the + // copy in use. A floor is not an error of the run (an index is data; + // mcpp is the program): each case is one closing notice, printed after + // the command's own output, and only because this run refreshed. + StagedOutcome staged; + int ownedRc = 0; + for (const auto& d : paths::declared_indices(env)) + if (d.owned) ownedRc = std::max(ownedRc, refresh_owned(env, d, staged)); + auto advice = [&](const std::filesystem::path& dir) { + auto it = staged.requiredMcpp.find(dir); + return mcpp::pm::index_floor_upgrade_advice( + it == staged.requiredMcpp.end() ? std::string_view{} : std::string_view{it->second}); + }; + for (auto& dir : staged.kept) + mcpp::ui::add_closing_notice("MCPP_INDEX_REQUIRES_NEWER_MCPP", std::format( + "the refreshed package index `{}` requires a newer mcpp; this run " + "used the previous index. {}", dir.filename().string(), advice(dir))); + for (auto& dir : staged.unusable) + mcpp::ui::add_closing_notice("MCPP_INDEX_REQUIRES_NEWER_MCPP", std::format( + "the package index `{}` requires a newer mcpp and no earlier copy is " + "usable; packages it serves cannot be resolved. {}", dir.filename().string(), advice(dir))); + // Through the NDJSON interface, as installs are (W11): the refresh reports // its steps as events and is drawn by the same renderer, instead of the // bare CLI's terminal text, which was either reprinted verbatim or, for @@ -2280,7 +2449,7 @@ int update_index_unguarded(const Env& env, bool quiet) { if (rc == 0 && renderer.result_exit() > 0) rc = renderer.result_exit(); if (rc != 0 || timedOut) renderer.fail(); } - if (rc == 0 && !timedOut) { mark_known_indexes_refreshed(env); return 0; } + if (rc == 0 && !timedOut) { mark_known_indexes_refreshed(env); return ownedRc; } // A refresh that exceeded its bound is not retried: the retries exist for // a transient failure that ends, and a connection that never answers // would only be waited on three times. The caller treats the refresh as @@ -2325,4 +2494,4 @@ bool official_package_present(const Env& env, std::string_view packageName) { return !pkg.empty() && std::filesystem::exists(pkg, ec); } -} // namespace mcpp::xlings +} // namespace mcpp::provider::xlings diff --git a/src/runtime/binding.cppm b/src/runtime/binding.cppm index 01e965b8b..70ef5c797 100644 --- a/src/runtime/binding.cppm +++ b/src/runtime/binding.cppm @@ -14,9 +14,9 @@ import std; // its own. The caller passes the path; see subos_path. import mcpp.libs.json; import mcpp.platform; -import mcpp.xlings.runtime_selection; -import mcpp.xlings.subos_info; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.runtime_selection; export namespace mcpp::platform::runtime { @@ -66,14 +66,14 @@ struct RuntimeBinding { // rule. A payload directory and a farm directory are not interchangeable // even when they currently resolve to the same file. std::vector searchDirs; - std::vector environment; + std::vector environment; std::vector providerBindings; std::vector capabilities; - std::vector runtimeProviders; - std::vector runtimeArtifacts; + std::vector runtimeProviders; + std::vector runtimeArtifacts; std::string provenance; std::filesystem::path subosDir; - mcpp::xlings::runtime::RuntimeSelection selection; + mcpp::provider::runtime::RuntimeSelection selection; // Did the SubOS describe itself (does it carry a `subos_info` block)? // @@ -170,7 +170,7 @@ std::string canonical_contract(const RuntimeBinding& binding) { append_field(out, binding.arch); append_field(out, binding.runtimeId); append_field(out, binding.selection.mode - == mcpp::xlings::runtime::RuntimeSelection::Mode::McppDefault + == mcpp::provider::runtime::RuntimeSelection::Mode::McppDefault ? "mcpp-default" : "named"); append_field(out, binding.selection.subosName); append_field(out, binding.provenance); @@ -185,7 +185,7 @@ std::string canonical_contract(const RuntimeBinding& binding) { // before revisions existed keeps the contract, and the output directory, // it had. for (auto const& p : binding.libraryDirs) - if (const int r = mcpp::xlings::paths::installed_revision(p.parent_path()) + if (const int r = mcpp::provider::revision_of(p.parent_path()) .value_or(0); r > 0) append_field(out, std::format("revision={}", r)); // The farm participates in the hash because it participates in the @@ -228,13 +228,13 @@ std::string canonical_contract(const RuntimeBinding& binding) { } std::filesystem::path subos_path( - const mcpp::xlings::runtime::RuntimeSelection& selection, + const mcpp::provider::runtime::RuntimeSelection& selection, const std::filesystem::path& xlingsHome) { - using Mode = mcpp::xlings::runtime::RuntimeSelection::Mode; + using Mode = mcpp::provider::runtime::RuntimeSelection::Mode; + const auto& layout = mcpp::provider::layout(); if (selection.mode == Mode::McppDefault || selection.subosName == "default") - return xlingsHome / "subos" / "default"; - return selection.ownerRoot / ".mcpp" / ".xlings" / "subos" - / selection.subosName; + return layout.environment(xlingsHome, "default"); + return layout.environment(layout.project_home(selection.ownerRoot), selection.subosName); } // `subos_info.runtime` is the provider's declared identity, while the SubOS @@ -249,8 +249,9 @@ std::optional managed_glibc_identity( const std::filesystem::path& libraryDir, const std::filesystem::path& xlingsRoot) { std::error_code bec, lec; + const auto& layout = mcpp::provider::layout(); auto base = std::filesystem::weakly_canonical( - xlingsRoot / "data" / "xpkgs" / "xim-x-glibc", bec); + layout.tool_in(layout.store_of_home(xlingsRoot), "glibc"), bec); auto real = std::filesystem::weakly_canonical(libraryDir, lec); if (bec || lec || base.empty() || real.empty()) return std::nullopt; @@ -286,7 +287,8 @@ std::optional exact_declared_glibc_payload( || version.find('\\') != std::string_view::npos) return std::nullopt; - auto payload = xlingsRoot / "data" / "xpkgs" / "xim-x-glibc" + const auto& layout = mcpp::provider::layout(); + auto payload = layout.tool_in(layout.store_of_home(xlingsRoot), "glibc") / std::string(version); for (auto const& leaf : {"lib64", "lib"}) { auto lib = payload / leaf; @@ -314,7 +316,7 @@ std::optional exact_declared_glibc_payload( std::expected resolve_runtime_binding( - const mcpp::xlings::runtime::RuntimeSelection& selection, + const mcpp::provider::runtime::RuntimeSelection& selection, const std::filesystem::path& /*compiler*/, const std::filesystem::path& xlingsHome) { RuntimeBinding out; @@ -344,7 +346,7 @@ resolve_runtime_binding( out.note += std::move(message); }; - auto info = mcpp::xlings::subos::read(out.subosDir); + auto info = mcpp::provider::layout().runtime_info(out.subosDir); out.declared = info.present; if (!info.present) { note(std::format( @@ -360,7 +362,7 @@ resolve_runtime_binding( "it described (openxlings/xlings#547).", selection.subosName, info.note.empty() ? "no `subos_info` block" : info.note)); - } else if (info.schema > mcpp::xlings::subos::kSupportedSchema) { + } else if (info.schema > mcpp::provider::runtime_info::kSupportedSchema) { // Mirrors `subos_info::read`, which already reads a HIGHER schema and // says so. A consumer stricter than its own reader is a time bomb: // the day xlings writes schema 2, an equality check stops every build @@ -369,7 +371,7 @@ resolve_runtime_binding( "SubOS '{}' declares runtime contract schema {}, newer than the {} " "this mcpp understands; using the fields it knows", selection.subosName, info.schema, - mcpp::xlings::subos::kSupportedSchema)); + mcpp::provider::runtime_info::kSupportedSchema)); } if (info.present && info.runtime.empty()) { note(std::format( @@ -384,7 +386,7 @@ resolve_runtime_binding( out.arch = std::string(mcpp::platform::host_arch); out.runtimeId = info.runtime; out.provenance = selection.mode - == mcpp::xlings::runtime::RuntimeSelection::Mode::McppDefault + == mcpp::provider::runtime::RuntimeSelection::Mode::McppDefault ? "mcpp_default" : "named_subos"; // libc/ELF are Linux concepts. macOS and Windows still carry the same @@ -542,10 +544,10 @@ std::string serialize_runtime_binding(const RuntimeBinding& binding) { j["subos_dir"] = binding.subosDir.generic_string(); j["selection"] = { {"mode", binding.selection.mode - == mcpp::xlings::runtime::RuntimeSelection::Mode::McppDefault + == mcpp::provider::runtime::RuntimeSelection::Mode::McppDefault ? "mcpp_default" : "named_subos"}, {"source", binding.selection.source - == mcpp::xlings::runtime::RuntimeSelection::Source::DefaultPolicy + == mcpp::provider::runtime::RuntimeSelection::Source::DefaultPolicy ? "default_policy" : "manifest"}, {"name", binding.selection.subosName}, {"owner_root", binding.selection.ownerRoot.generic_string()}, @@ -580,7 +582,7 @@ deserialize_runtime_binding(std::string_view encoded) { if (auto it = j.find("environment"); it != j.end() && it->is_array()) { for (auto const& v : *it) { if (!v.is_object()) continue; - mcpp::xlings::subos::EnvDecl decl; + mcpp::provider::runtime_info::EnvDecl decl; decl.var = v.value("var", ""); decl.op = v.value("op", ""); decl.value = v.value("value", ""); @@ -593,7 +595,7 @@ deserialize_runtime_binding(std::string_view encoded) { out.capabilities = j.value( "capabilities", std::vector{}); auto read_identity = [](const nlohmann::json& value) { - mcpp::xlings::subos::PackageIdentity id; + mcpp::provider::runtime_info::PackageIdentity id; if (!value.is_object()) return id; id.namespace_ = value.value("namespace", ""); id.name = value.value("name", ""); @@ -605,7 +607,7 @@ deserialize_runtime_binding(std::string_view encoded) { it != j.end() && it->is_array()) { for (auto const& value : *it) { if (!value.is_object()) continue; - mcpp::xlings::subos::CapabilityProvider provider; + mcpp::provider::runtime_info::CapabilityProvider provider; provider.capability = value.value("capability", ""); if (auto identity = value.find("provider"); identity != value.end()) provider.provider = read_identity(*identity); @@ -617,7 +619,7 @@ deserialize_runtime_binding(std::string_view encoded) { it != j.end() && it->is_array()) { for (auto const& value : *it) { if (!value.is_object()) continue; - mcpp::xlings::subos::RuntimeArtifact artifact; + mcpp::provider::runtime_info::RuntimeArtifact artifact; artifact.role = value.value("role", ""); if (auto identity = value.find("provider"); identity != value.end()) artifact.provider = read_identity(*identity); @@ -635,11 +637,11 @@ deserialize_runtime_binding(std::string_view encoded) { out.subosDir = j.value("subos_dir", ""); auto s = j.at("selection"); out.selection.mode = s.value("mode", "") == "named_subos" - ? mcpp::xlings::runtime::RuntimeSelection::Mode::NamedSubos - : mcpp::xlings::runtime::RuntimeSelection::Mode::McppDefault; + ? mcpp::provider::runtime::RuntimeSelection::Mode::NamedSubos + : mcpp::provider::runtime::RuntimeSelection::Mode::McppDefault; out.selection.source = s.value("source", "") == "manifest" - ? mcpp::xlings::runtime::RuntimeSelection::Source::Manifest - : mcpp::xlings::runtime::RuntimeSelection::Source::DefaultPolicy; + ? mcpp::provider::runtime::RuntimeSelection::Source::Manifest + : mcpp::provider::runtime::RuntimeSelection::Source::DefaultPolicy; out.selection.subosName = s.value("name", "default"); out.selection.ownerRoot = s.value("owner_root", ""); // Completeness is conditional on `declared`. An UNDECLARED binding @@ -666,13 +668,13 @@ std::vector> resolve_runtime_environment( const RuntimeBinding& binding, const std::function(std::string_view)>& ambient = {}) { - mcpp::xlings::subos::Info info; + mcpp::provider::runtime_info::Info info; info.present = true; - mcpp::xlings::subos::Provider snapshot; + mcpp::provider::runtime_info::Declarer snapshot; snapshot.binding = "runtime-binding-snapshot"; snapshot.decls = binding.environment; info.providers.push_back(std::move(snapshot)); - return mcpp::xlings::subos::resolve_env(info, binding.subosDir, ambient); + return mcpp::provider::resolve_env(info, binding.subosDir, ambient); } } // namespace mcpp::platform::runtime diff --git a/src/toolchain/clang.cppm b/src/toolchain/clang.cppm index cbd849d0f..75dd42a0e 100644 --- a/src/toolchain/clang.cppm +++ b/src/toolchain/clang.cppm @@ -6,7 +6,8 @@ import std; import mcpp.toolchain.model; import mcpp.toolchain.msvc; import mcpp.toolchain.probe; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.platform; export namespace mcpp::toolchain::clang { @@ -109,7 +110,7 @@ std::optional find_libcxx_std_module_source( auto manifest_r = mcpp::toolchain::run_capture(std::format( "{}{} -print-library-module-manifest-path {}", envPrefix, - mcpp::xlings::shq(cxx_binary.string()), + mcpp::platform::shell::quote(cxx_binary.string()), mcpp::platform::null_redirect)); if (manifest_r) { auto manifestPath = std::filesystem::path( @@ -165,7 +166,7 @@ std::optional find_libcxx_std_module_source( // replaces made with `bin/../share`. if (auto lib_r = mcpp::toolchain::run_capture(std::format( "{}{} --print-file-name=libc++.a {}", - envPrefix, mcpp::xlings::shq(cxx_binary.string()), + envPrefix, mcpp::platform::shell::quote(cxx_binary.string()), mcpp::platform::null_redirect))) { std::filesystem::path lib(mcpp::toolchain::trim_line(*lib_r)); // A driver that cannot place the library echoes the bare name back. @@ -369,8 +370,8 @@ std::vector std_module_build_commands(const Toolchain& tc, ixxFlags, sysrootFlag, precompileFlags, - mcpp::xlings::shq(tc.stdModuleSource.string()), - mcpp::xlings::shq(absBmi)), + mcpp::platform::shell::quote(tc.stdModuleSource.string()), + mcpp::platform::shell::quote(absBmi)), std::format( "{} {}{}{}{} " "{} -c -o {}", @@ -379,35 +380,35 @@ std::vector std_module_build_commands(const Toolchain& tc, crtToken, sysrootFlag, codegenFlags, - mcpp::xlings::shq(absBmi), - mcpp::xlings::shq((cacheDir / "std.o").string())) + mcpp::platform::shell::quote(absBmi), + mcpp::platform::shell::quote((cacheDir / "std.o").string())) }; #else return { std::format( "cd {} && {}{} {}{} -Wno-reserved-module-identifier{}{}{} " "--precompile {} -o {} 2>&1", - mcpp::xlings::shq(cacheDir.string()), + mcpp::platform::shell::quote(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), - mcpp::xlings::shq(tc.binaryPath.string()), + mcpp::platform::shell::quote(tc.binaryPath.string()), cppStandardFlag, crtToken, sysrootFlag, precompileFlags, module_source_language_flags(tc.stdModuleSource), - mcpp::xlings::shq(tc.stdModuleSource.string()), - mcpp::xlings::shq(relBmi)), + mcpp::platform::shell::quote(tc.stdModuleSource.string()), + mcpp::platform::shell::quote(relBmi)), std::format( "cd {} && {}{} {}{} -Wno-reserved-module-identifier{}{} " "{} -c -o std.o 2>&1", - mcpp::xlings::shq(cacheDir.string()), + mcpp::platform::shell::quote(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), - mcpp::xlings::shq(tc.binaryPath.string()), + mcpp::platform::shell::quote(tc.binaryPath.string()), cppStandardFlag, crtToken, sysrootFlag, codegenFlags, - mcpp::xlings::shq(relBmi)) + mcpp::platform::shell::quote(relBmi)) }; #endif } @@ -461,7 +462,7 @@ std::filesystem::path runtime_archive_path(const Toolchain& tc, if (auto r = mcpp::toolchain::run_capture(std::format( "{}{}{} --print-file-name={} {}", mcpp::toolchain::compiler_env_prefix(tc), - mcpp::xlings::shq(tc.binaryPath.string()), + mcpp::platform::shell::quote(tc.binaryPath.string()), driverFlags, archive, mcpp::platform::null_redirect))) { std::filesystem::path lib(mcpp::toolchain::trim_line(*r)); std::error_code ec; @@ -579,27 +580,27 @@ std::vector std_compat_build_commands(const Toolchain& tc, "-fmodule-file=std={} " "--precompile {} -o {} 2>&1", mcpp::toolchain::compiler_env_prefix(tc), - mcpp::xlings::shq(tc.binaryPath.string()), + mcpp::platform::shell::quote(tc.binaryPath.string()), cppStandardFlag, crtToken, sysrootFlag, precompileFlags, ixxFlags, absStdBmi, - mcpp::xlings::shq(tc.stdCompatSource.string()), - mcpp::xlings::shq(absBmi)), + mcpp::platform::shell::quote(tc.stdCompatSource.string()), + mcpp::platform::shell::quote(absBmi)), std::format("{}{} {}{} -Wno-reserved-module-identifier{}{} " "-fmodule-file=std={} " "{} -c -o {} 2>&1", mcpp::toolchain::compiler_env_prefix(tc), - mcpp::xlings::shq(tc.binaryPath.string()), + mcpp::platform::shell::quote(tc.binaryPath.string()), cppStandardFlag, crtToken, sysrootFlag, codegenFlags, absStdBmi, - mcpp::xlings::shq(absBmi), - mcpp::xlings::shq(absObj)) + mcpp::platform::shell::quote(absBmi), + mcpp::platform::shell::quote(absObj)) }; } diff --git a/src/toolchain/detect.cppm b/src/toolchain/detect.cppm index 91766727b..9c5a15bd1 100644 --- a/src/toolchain/detect.cppm +++ b/src/toolchain/detect.cppm @@ -9,7 +9,9 @@ import std; import mcpp.toolchain.clang; import mcpp.toolchain.gcc; import mcpp.toolchain.msvc; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.platform; export namespace mcpp::toolchain { @@ -63,7 +65,7 @@ detect(const std::filesystem::path& explicit_compiler, auto ver_r = run_capture(std::format("{}{} --version 2>&1", envPrefix, - mcpp::xlings::shq(tc.binaryPath.string()))); + mcpp::platform::shell::quote(tc.binaryPath.string()))); if (!ver_r) return std::unexpected(ver_r.error()); const auto& vstr = *ver_r; @@ -100,7 +102,7 @@ detect(const std::filesystem::path& explicit_compiler, auto vr = run_capture(std::format( "{}{} -print-effective-triple 2>NUL", envPrefix, - mcpp::xlings::shq(tc.binaryPath.string()))); + mcpp::platform::shell::quote(tc.binaryPath.string()))); if (vr) { auto effective = trim_line(*vr); if (!effective.empty() && effective != tc.targetTriple) diff --git a/src/toolchain/gcc.cppm b/src/toolchain/gcc.cppm index 547f29c76..f44f4ddf8 100644 --- a/src/toolchain/gcc.cppm +++ b/src/toolchain/gcc.cppm @@ -6,7 +6,8 @@ import std; import mcpp.platform; import mcpp.toolchain.model; import mcpp.toolchain.probe; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; export namespace mcpp::toolchain::gcc { @@ -167,8 +168,7 @@ void enrich_toolchain(Toolchain& tc) { std::optional find_binutils_bin(const std::filesystem::path& compilerBin) { - if (auto as = mcpp::xlings::paths::find_sibling_binary( - compilerBin, "binutils", "bin/as")) { + if (auto as = mcpp::provider::beside_binary(compilerBin, "binutils", "bin/as")) { return as->parent_path(); } return std::nullopt; @@ -211,18 +211,18 @@ static std::string module_source_command(const Toolchain& tc, std::string_view object) { std::string bFlag; if (auto binutilsBin = binutils_prefix_dir(tc); !binutilsBin.empty()) - bFlag = std::format(" -B{}", mcpp::xlings::shq(binutilsBin.string())); + bFlag = std::format(" -B{}", mcpp::platform::shell::quote(binutilsBin.string())); const char* cd = mcpp::platform::is_windows ? "cd /d" : "cd"; return std::format( "{} {} && {}{} {} -fmodules -O2{}{} -c {} -o {} 2>&1", cd, - mcpp::xlings::shq(cacheDir.string()), + mcpp::platform::shell::quote(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), - mcpp::xlings::shq(tc.binaryPath.string()), + mcpp::platform::shell::quote(tc.binaryPath.string()), cppStandardFlag, sysrootFlag, bFlag, - mcpp::xlings::shq(source.string()), + mcpp::platform::shell::quote(source.string()), object); } @@ -245,7 +245,7 @@ std::string std_module_build_command(const Toolchain& tc, std::string_view cppStandardFlag) { std::string bFlag; if (auto binutilsBin = binutils_prefix_dir(tc); !binutilsBin.empty()) - bFlag = std::format(" -B{}", mcpp::xlings::shq(binutilsBin.string())); + bFlag = std::format(" -B{}", mcpp::platform::shell::quote(binutilsBin.string())); // Windows (MinGW): cmd.exe needs `/d` to change DRIVE (project on D:, // BMI cache on C: is the real CI layout — same-drive runs masked this). @@ -253,13 +253,13 @@ std::string std_module_build_command(const Toolchain& tc, return std::format( "{} {} && {}{} {} -fmodules -O2{}{} -c {} -o std.o 2>&1", cd, - mcpp::xlings::shq(cacheDir.string()), + mcpp::platform::shell::quote(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), - mcpp::xlings::shq(tc.binaryPath.string()), + mcpp::platform::shell::quote(tc.binaryPath.string()), cppStandardFlag, sysrootFlag, bFlag, - mcpp::xlings::shq(tc.stdModuleSource.string())); + mcpp::platform::shell::quote(tc.stdModuleSource.string())); } std::vector std_module_build_commands( diff --git a/src/toolchain/lifecycle.cppm b/src/toolchain/lifecycle.cppm index b0f2a95d5..67ea87e4c 100644 --- a/src/toolchain/lifecycle.cppm +++ b/src/toolchain/lifecycle.cppm @@ -25,8 +25,9 @@ import mcpp.ui; import mcpp.wire; import mcpp.libs.json; import mcpp.log; -import mcpp.xlings; -import mcpp.xlings.runtime_selection; +import mcpp.provider; +import mcpp.provider.select; +import mcpp.provider.runtime_selection; import mcpp.runtime.binding; namespace mcpp::toolchain { @@ -105,7 +106,7 @@ list_installed_versions(const std::filesystem::path& pkgsDir, std::string_view ximName) { std::vector out; - auto root = pkgsDir / std::format("xim-x-{}", ximName); + auto root = mcpp::provider::layout().tool_in(pkgsDir, ximName); std::error_code ec; if (!std::filesystem::exists(root, ec)) return out; for (auto& v : std::filesystem::directory_iterator(root, ec)) { @@ -118,17 +119,13 @@ list_installed_versions(const std::filesystem::path& pkgsDir, // Falls back to an empty list silently — the caller will then either error // out with a clear message or just keep the partial as-is. // -// Index layout in mcpp's sandbox is two-tier: -// /data/xim-pkgindex/pkgs//.lua — primary -// /data/xim-index-repos//pkgs//.lua -// We scan both so a package living in either tier resolves. +// Where a descriptor may be is the provider's answer (its primary index, then +// its sub-indexes), so a package living in either resolves. std::vector list_available_xpkg_versions(const mcpp::config::GlobalConfig& cfg, std::string_view ximName) { if (ximName.empty()) return {}; - std::string subdir(1, ximName[0]); - std::string fname = std::string(ximName) + ".lua"; auto try_load = [&](const std::filesystem::path& p) -> std::optional> @@ -147,18 +144,9 @@ list_available_xpkg_versions(const mcpp::config::GlobalConfig& cfg, body, mcpp::platform::HostPlatform::current()); }; - auto data = cfg.xlingsHome() / "data"; - if (auto v = try_load(data / "xim-pkgindex" / "pkgs" / subdir / fname); v) - return std::move(*v); - - std::error_code ec; - auto repos = data / "xim-index-repos"; - if (std::filesystem::exists(repos, ec)) { - for (auto& repo : std::filesystem::directory_iterator(repos, ec)) { - auto cand = repo.path() / "pkgs" / subdir / fname; - if (auto v = try_load(cand); v) return std::move(*v); - } - } + auto data = mcpp::provider::open(mcpp::config::make_xlings_env(cfg))->data_root(); + for (const auto& cand : mcpp::provider::layout().descriptor_candidates(data, ximName)) + if (auto v = try_load(cand); v) return std::move(*v); return {}; } @@ -452,7 +440,7 @@ bool version_greater(const std::string& a, const std::string& b) { // the two come to disagree about what is installed. export int toolchain_list(const mcpp::config::GlobalConfig& cfg, bool json = false) { - auto pkgsDir = cfg.xlingsHome() / "data" / "xpkgs"; + auto pkgsDir = mcpp::provider::open(mcpp::config::make_xlings_env(cfg))->store(); auto effective = effective_default_toolchain(cfg); // The default is a PAIR: toolchain axis (family@version) + target axis. @@ -478,24 +466,20 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, }; std::vector payloads; std::error_code ec; - if (std::filesystem::exists(pkgsDir, ec)) { - for (auto& entry : std::filesystem::directory_iterator(pkgsDir, ec)) { - auto name = entry.path().filename().string(); - if (auto sep = name.find("-x-"); sep != std::string::npos) - name = name.substr(sep + 3); - auto id = mcpp::toolchain::identify_xim_payload(name); + { + for (const auto& entry : mcpp::provider::layout().entries(pkgsDir)) { + auto id = mcpp::toolchain::identify_xim_payload(entry.name); if (!id) continue; // ninja/glibc/… — not a toolchain - for (auto& vEntry : std::filesystem::directory_iterator(entry.path(), ec)) { - if (!vEntry.is_directory(ec)) continue; + { mcpp::toolchain::ToolchainSpec s; s.family = id->family; - s.version = vEntry.path().filename().string(); + s.version = entry.version; s.target = id->target; auto pkg = mcpp::toolchain::to_xim_package(s); // From the payload ROOT, not `root/bin`: msvc keeps cl.exe // four levels deeper, and asking for `root/bin` skipped every // installed toolset silently. - auto bin = mcpp::toolchain::payload_frontend(vEntry.path(), pkg); + auto bin = mcpp::toolchain::payload_frontend(entry.dir, pkg); // A MALFORMED DESCRIPTOR IS REPORTED, NOT SKIPPED. This // enumeration's `continue` means "nothing usable here", and a // payload whose own description does not parse would then be @@ -876,16 +860,25 @@ int attach_target_arg(mcpp::toolchain::ToolchainSpec& spec, // already locates a payload, off Linux, and for a provider other than glibc, // so a home in its normal state starts no process. // -// A failed install is reported at verbose level and is not an error here. The -// consumer that needs the payload, the fixup, reports its absence with the -// exact coordinate, and a caller whose toolchain needs no C runtime payload -// is unaffected. Returns whether `binding` changed. -export bool ensure_declared_runtime(const mcpp::config::GlobalConfig& cfg, - mcpp::platform::runtime::RuntimeBinding& binding) { +// A toolchain that runs on this runtime cannot start without it (design +// 2026-10-16 §7.2: its loader is the runtime's), so a failed install is the +// caller's error, naming the exact coordinate that provides the payload. The +// callers ask only for a toolchain that runs on it. Returns whether `binding` +// changed. +export std::expected +ensure_declared_runtime(const mcpp::config::GlobalConfig& cfg, + mcpp::platform::runtime::RuntimeBinding& binding) { namespace rt = mcpp::platform::runtime; if constexpr (!mcpp::platform::is_linux) return false; - if (!binding.libraryDirs.empty()) return false; if (rt::runtime_provider(binding.runtimeId) != "glibc") return false; + // The toolchains the provider installs run on this runtime (design + // 2026-10-16 §7.2), not on whatever it installs newest; pinned once its + // payload is there, which is what the provider's links point at. + const auto provider = mcpp::provider::open(mcpp::config::make_xlings_env(cfg)); + if (!binding.libraryDirs.empty()) { + provider->pin_runtime(binding.runtimeId, /*relink=*/false); + return false; + } const auto target = "xim:" + binding.runtimeId; mcpp::log::verbose("toolchain", std::format( @@ -893,13 +886,10 @@ export bool ensure_declared_runtime(const mcpp::config::GlobalConfig& cfg, binding.runtimeId, target)); mcpp::fetcher::Fetcher fetcher(cfg); mcpp::fetcher::InstallProgressHandler progress; - if (auto installed = fetcher.resolve_xpkg_path(target, /*autoInstall=*/true, &progress); - !installed) { - mcpp::log::verbose("toolchain", std::format( - "installing the declared runtime {} failed: {}", - target, installed.error().message)); - return false; - } + if (auto installed = fetcher.resolve_xpkg_path(target, /*autoInstall=*/true, &progress); !installed) + return std::unexpected(std::format( + "the C runtime the toolchain runs on, {}, is not installed: {}", target, installed.error().message)); + provider->pin_runtime(binding.runtimeId, /*relink=*/true); auto again = rt::resolve_runtime_binding(binding.selection, {}, cfg.xlingsHome()); if (!again) return false; binding = std::move(*again); @@ -999,8 +989,7 @@ export int toolchain_install(const mcpp::config::GlobalConfig& cfg, // by now whatever held it has exited, so free the bytes before adding // another few hundred MB. { - auto vdir = mcpp::xlings::paths::xim_tool( - mcpp::config::make_xlings_env(cfg), pkg.ximName, pkg.ximVersion); + auto vdir = mcpp::provider::open(mcpp::config::make_xlings_env(cfg))->pinned(pkg.ximName, pkg.ximVersion); sweep_parked_payloads(vdir.parent_path(), vdir); } auto payload = fetcher.resolve_xpkg_path(pkg.target(), /*autoInstall=*/true, &progress); @@ -1074,45 +1063,20 @@ export int toolchain_install(const mcpp::config::GlobalConfig& cfg, return 1; } - // Post-install fixup (patchelf / specs / cfg regeneration) — ONE - // pipeline shared by every toolchain install path, dispatched and - // made idempotent inside ensure_post_install_fixup. - // - // RESOLVE THE RUNTIME BINDING HERE, do not let the fixup guess. - // The fixup is a CONSUMER of RuntimeBinding (`prepare.cppm` says so - // where it does the same thing) and it no longer derives an identity - // of its own — so a caller that passes nothing gets no fixup at all. - // `mcpp toolchain install` is the path that needs it MOST: without - // the patchelf/specs wiring a fresh sandbox gcc cannot find the C - // library. There is no project here, so the selection is mcpp's - // default SubOS, which is exactly what `RuntimeSelection{}` means. - std::string runtimeId; - std::filesystem::path runtimeLibDir; - if (auto rb = mcpp::platform::runtime::resolve_runtime_binding( - mcpp::xlings::runtime::RuntimeSelection{}, {}, cfg.xlingsHome())) { - if (!mcpp::toolchain::post_install_fixup_kind(pkg).empty()) - ensure_declared_runtime(cfg, *rb); - runtimeId = rb->runtimeId; - if (!rb->libraryDirs.empty()) runtimeLibDir = rb->libraryDirs.front(); - } // a binding that cannot be resolved degrades below, it does not stop - // the install — same rule the build path follows. - - if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( - cfg, payload->root, pkg, runtimeId, runtimeLibDir); !fixed) { - mcpp::ui::error(std::format( - "post-install fixup failed: {}", fixed.error())); - return 1; - } else if (!fixed->skippedReason.empty()) { - // The user asked for this install explicitly, so the degradation - // is louder here than on a build — but it is still a warning, and - // the install itself succeeded. - mcpp::ui::warning(std::format( - "installed, but not bound to a C runtime: {}.\n" - " The install itself succeeded. To complete the wiring, " - "run `xlings self doctor --fix` from inside this SubOS, then " - "re-run this command (openxlings/xlings#547).", - fixed->skippedReason)); - } + // The toolchain runs on the runtime its provider bound it to (design + // 2026-10-16 §7.2); mcpp writes nothing into it. What it needs is + // that runtime installed: `mcpp toolchain install` is the path that + // needs it MOST, before any build has provided it. There is no project + // here, so the selection is mcpp's default environment, which is what + // `RuntimeSelection{}` means. + if (!mcpp::toolchain::runtime_payload_kind(pkg).empty()) { + if (auto rb = mcpp::platform::runtime::resolve_runtime_binding( + mcpp::provider::runtime::RuntimeSelection{}, {}, cfg.xlingsHome())) + if (auto provided = ensure_declared_runtime(cfg, *rb); !provided) { + mcpp::ui::error(provided.error()); + return 1; + } + } // a binding that cannot be resolved degrades, as on the build path mcpp::ui::status("Installed", std::format("{} → {}", pkg.display_spec(), bin.string())); @@ -1133,7 +1097,7 @@ export int toolchain_install(const mcpp::config::GlobalConfig& cfg, export int toolchain_set_default(const mcpp::config::GlobalConfig& cfg, const std::string& pos0, const std::string& pos1, const std::string& targetArg = {}) { - auto pkgsDir = cfg.xlingsHome() / "data" / "xpkgs"; + auto pkgsDir = mcpp::provider::open(mcpp::config::make_xlings_env(cfg))->store(); auto xlEnv = mcpp::config::make_xlings_env(cfg); // Accept three input shapes (mirrors `install`): // mcpp toolchain default gcc@16.1.0 @@ -1215,7 +1179,7 @@ export int toolchain_set_default(const mcpp::config::GlobalConfig& cfg, pkg = mcpp::toolchain::to_xim_package(*spec); } - auto installDir = mcpp::xlings::paths::xim_tool(xlEnv, pkg.ximName, pkg.ximVersion); + auto installDir = mcpp::provider::open(xlEnv)->pinned(pkg.ximName, pkg.ximVersion); // A RESOLVABLE COMPILER, not a directory that exists. A removal that // could not delete the last empty directories leaves a skeleton // behind (see remove_payload_tree), and `exists()` would call that @@ -1300,7 +1264,7 @@ export int toolchain_remove(const mcpp::config::GlobalConfig& cfg, if (int rc = attach_target_arg(*parsedSpec, targetArg); rc != 0) return rc; auto pkg = mcpp::toolchain::to_xim_package(*parsedSpec); auto spec = parsedSpec->display(); - auto installDir = mcpp::xlings::paths::xim_tool(xlEnv, pkg.ximName, pkg.ximVersion); + auto installDir = mcpp::provider::open(xlEnv)->pinned(pkg.ximName, pkg.ximVersion); std::error_code ec; if (!std::filesystem::exists(installDir, ec)) { mcpp::ui::error(std::format("{} is not installed", spec)); diff --git a/src/toolchain/msvc.cppm b/src/toolchain/msvc.cppm index f1ff12d63..036532e8f 100644 --- a/src/toolchain/msvc.cppm +++ b/src/toolchain/msvc.cppm @@ -33,7 +33,8 @@ import mcpp.platform; import mcpp.toolchain.model; import mcpp.toolchain.probe; import mcpp.toolchain.triple; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; export namespace mcpp::toolchain::msvc { @@ -1168,10 +1169,10 @@ std::optional detect_installation() { std::vector sibling_sdk_roots(const std::filesystem::path& clPath) { std::vector out; - auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(clPath); + auto xpkgs = mcpp::provider::store_of(clPath); if (!xpkgs) return out; // a system cl.exe: not in any store std::error_code ec; - auto pkgRoot = *xpkgs / "xim-x-windows-sdk"; + auto pkgRoot = mcpp::provider::layout().tool_in(*xpkgs, "windows-sdk"); for (auto& e : std::filesystem::directory_iterator(pkgRoot, ec)) { if (e.is_directory(ec)) out.push_back(e.path()); } @@ -1362,7 +1363,7 @@ std::string describe_msvc_probe(const MsvcProbe& p) { } Origin origin_of(const std::filesystem::path& clPath) { - return mcpp::xlings::paths::xpkgs_from_compiler(clPath) + return mcpp::provider::store_of(clPath) ? Origin::Managed : Origin::SystemMsvc; } @@ -1445,7 +1446,7 @@ bool msvc_available_here([[maybe_unused]] const std::filesystem::path& pkgsDir) // gets the wrong answer. Any installed version whose cl.exe resolves // counts; `installation_at` is the same resolution install and build use. std::error_code ec; - auto root = pkgsDir / "xim-x-msvc"; + auto root = mcpp::provider::layout().tool_in(pkgsDir, "msvc"); if (!std::filesystem::is_directory(root, ec)) return false; for (auto& v : std::filesystem::directory_iterator(root, ec)) { if (!v.is_directory(ec)) continue; @@ -1574,13 +1575,13 @@ std::string cl_stage_command(const Toolchain& tc, // cache on C: is the real CI layout). return std::format( "cd /d {} && {} /nologo {}{} /EHsc /O2 /W0{} /c {} /ifcOutput {} /Fo:{} 2>&1", - mcpp::xlings::shq(cacheDir.string()), - mcpp::xlings::shq(tc.binaryPath.string()), + mcpp::platform::shell::quote(cacheDir.string()), + mcpp::platform::shell::quote(tc.binaryPath.string()), cppStandardFlag, crtFlag.empty() ? std::string{} : std::format(" {}", crtFlag), extraRef, - mcpp::xlings::shq(source.string()), - mcpp::xlings::shq(ifcOut.string()), + mcpp::platform::shell::quote(source.string()), + mcpp::platform::shell::quote(ifcOut.string()), objName); } @@ -1648,7 +1649,7 @@ std::vector std_compat_build_commands( std::string_view cppStandardFlag, std::string_view crtFlag) { // std.compat imports std — reference the freshly staged std.ifc. auto ref = std::format(" /reference std={}", - mcpp::xlings::shq(std_bmi_path(cacheDir).string())); + mcpp::platform::shell::quote(std_bmi_path(cacheDir).string())); return { cl_stage_command(tc, cacheDir, cppStandardFlag, tc.stdCompatSource, std_compat_bmi_path(cacheDir), "std.compat.obj", diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index 282258e47..44d945242 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -1,9 +1,13 @@ -// mcpp.toolchain.post_install — toolchain payload post-install fixups (patchelf / specs / cfg) +// mcpp.toolchain.post_install — what an installed toolchain payload says +// about the runtime it was built against, read and never written: its baked +// runtime binding, the pristine link spec a build generates into its own +// directory, the glibc payload a runtime identity names. // -// Extracted verbatim from cli.cppm (cli modularization, see -// .agents/docs/2026-06-10-cli-modularization.md). Zero behavior change: -// bodies are byte-identical moves; only the surrounding module/namespace -// changed (mcpp::cli::detail -> mcpp::cli). +// A payload is immutable (design 2026-10-16 §7): the post-install rewrites +// that used to live here (PT_INTERP/RUNPATH by patchelf, gcc specs, clang cfg, +// the `.mcpp-fixup.json` marker) are gone. A toolchain runs on the runtime +// its provider bound it to; what mcpp's artifacts load is decided per build +// (mcpp.toolchain.linkmodel). module; #include @@ -20,153 +24,11 @@ import mcpp.runtime.binding; import mcpp.toolchain.linkmodel; import mcpp.toolchain.registry; import mcpp.ui; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; namespace mcpp::toolchain { -// ── #273 sandbox containment ───────────────────────────────────────────── -// patchelf_walk once escaped its sandbox through a symlinked payload -// directory (an e2e sandbox seeded `registry/data/xpkgs/xim-x-gcc` from the -// user's real ~/.xlings via `ln -s`) and rewrote PT_INTERP/RUNPATH of the -// REAL installation against loader paths inside a soon-deleted mktemp dir. -// -// Containment rule: every rewrite in this module is fenced by ONE explicit -// trust root — the owning sandbox's registry (`cfg.registryDir`), resolved -// once at the entry point via `containment_root` and threaded down as a -// parameter. The root is a first-class fact of the system; it is never -// re-derived from the payload path (canonicalizing the payload first would -// resolve the malicious symlink and collapse the fence into a tautology). -export std::filesystem::path -containment_root(const std::filesystem::path& registryDir) { - std::error_code ec; - auto canon = std::filesystem::weakly_canonical(registryDir, ec); - if (ec) return {}; // empty root → escapes_containment fails closed - return canon; -} - -// True when `file`'s physical (symlink-resolved) location falls outside -// `rootCanon`. Fails CLOSED: an empty root or an unresolvable path counts -// as escaping — never patch what cannot be proven contained. Comparison is -// per path component (a `registry-evil` sibling sharing the string prefix -// is outside). -export bool escapes_containment(const std::filesystem::path& file, - const std::filesystem::path& rootCanon) { - if (rootCanon.empty()) return true; - std::error_code ec; - auto real = std::filesystem::weakly_canonical(file, ec); - if (ec) return true; - auto r = rootCanon.generic_string(); - auto f = real.generic_string(); - if (f == r) return false; - return !(f.size() > r.size() && f.starts_with(r) && f[r.size()] == '/'); -} - -// Run patchelf on every dynamic ELF in `dir` (recursively): -// - Set PT_INTERP to `loader` (the sandbox-local glibc loader). -// - Set RUNPATH to `rpath` (colon-separated list of sandbox lib dirs). -// Idempotent; skips static binaries and shared libs without PT_INTERP. -// -// TODO(xlings/libxpkg-upstream): xim 0.4.10's `elfpatch.auto({interpreter=...})` -// is supposed to do this in install hooks but currently scans 0 files for -// some packages (verified empirically: `binutils: elfpatch auto: 0 0 0`). -// Once the upstream legacy elfpatch path is fixed, this mcpp-side walker -// can be deleted. -export void patchelf_walk(const std::filesystem::path& dir, - const std::filesystem::path& loader, - const std::string& rpath, - const std::filesystem::path& patchelfBin, - const std::filesystem::path& fenceRoot) -{ - if (!std::filesystem::exists(dir) || !std::filesystem::exists(patchelfBin)) - return; - std::error_code ec; - for (auto it = std::filesystem::recursive_directory_iterator(dir, ec); - it != std::filesystem::recursive_directory_iterator{}; it.increment(ec)) - { - if (ec) { ec.clear(); continue; } - if (!it->is_regular_file(ec)) continue; - auto path = it->path(); - // Skip non-ELF (cheap magic check) - std::ifstream is(path, std::ios::binary); - char m[4]{}; - is.read(m, 4); - if (!is || m[0] != 0x7f || m[1] != 'E' || m[2] != 'L' || m[3] != 'F') - continue; - is.close(); - // #273 fence: never rewrite files that physically live outside the - // sandbox (payload reached through a symlink → foreign installation). - if (escapes_containment(path, fenceRoot)) { - mcpp::log::verbose("toolchain", std::format( - "patchelf_walk: skip (outside sandbox, #273 fence): {}", - path.string())); - continue; - } - // Probe PT_INTERP — skip static binaries (no interp). - auto probe = std::format("{} --print-interpreter {} 2>/dev/null", - mcpp::platform::shell::quote(patchelfBin.string()), - mcpp::platform::shell::quote(path.string())); - auto probeResult = mcpp::platform::process::capture(probe); - bool hasInterp = (probeResult.exit_code == 0 && !probeResult.output.empty()); - - // Patch a COPY and atomically rename it into place. The payload can - // contain libraries the CURRENT process has mmapped (a self-hosted - // mcpp links the sandbox glibc/libgcc_s, and since the fixup - // pipeline runs on every install path, the patching process may BE - // such a consumer). In-place patchelf rewrites the backing file of - // those live mappings and corrupts the running process — observed - // on CI as an exit-time SIGSEGV in _dl_fini jumping to an - // unrelocated address. rename() gives the patched content a fresh - // inode while live processes keep the old one. - auto tmp = path; - tmp += ".mcpp-patch.tmp"; - { - std::error_code cec; - std::filesystem::copy_file( - path, tmp, std::filesystem::copy_options::overwrite_existing, cec); - if (cec) continue; - std::filesystem::permissions( - tmp, std::filesystem::status(path, cec).permissions(), - std::filesystem::perm_options::replace, cec); - } - bool patched = true; - if (hasInterp) { - patched = (mcpp::platform::process::run_silent(std::format( - "{} --set-interpreter {} {} 2>/dev/null", - mcpp::platform::shell::quote(patchelfBin.string()), - mcpp::platform::shell::quote(loader.string()), - mcpp::platform::shell::quote(tmp.string()))) == 0) && patched; - } - // Always set RUNPATH (works on .so too — they need to find deps). - if (!rpath.empty()) { - patched = (mcpp::platform::process::run_silent(std::format( - "{} --set-rpath {} {} 2>/dev/null", - mcpp::platform::shell::quote(patchelfBin.string()), - mcpp::platform::shell::quote(rpath), - mcpp::platform::shell::quote(tmp.string()))) == 0) && patched; - } - std::error_code rec; - if (patched) std::filesystem::rename(tmp, path, rec); - if (!patched || rec) std::filesystem::remove(tmp, rec); - } -} - -// xim bakes the installing user's XLINGS_HOME into gcc specs at install -// time (as `--dynamic-linker` and `-rpath`). When mcpp uses its own -// isolated sandbox (MCPP_HOME/registry/), the baked-in paths point to -// xlings' home, not mcpp's sandbox glibc — binaries would fail to exec. -// -// Mcpp does a post-install spec rewrite: -// - Dynamically detects the baked-in loader path from the specs file -// - Replaces it with the sandbox glibc payload's loader -// - Replaces the rpath with : -// NOT idempotent across homes -- and that is a defect, not a caveat. The -// needle is one path (the baked glibc dir) while the replacement is two -// (glibc + gccLib), so a second home patching the same shared payload leaves -// the first home's gccLib entry behind: it never appears in any later needle. -// One stale rpath per run, forever. Measured at 68 on a developer machine, -// all of them gccLib, none glibc -- a distribution only this mechanism -// produces. See 2026-08-08-payload-version-and-contract-drift-design.md §2.1; -// this function is slated for deletion, with the flags moving to the build. // Extract the baked-in glibc loader path (".../ld-linux-.so.N") from a // gcc specs file. xim bakes the installing user's XLINGS_HOME into specs at // install time, so the DIR varies per machine, and the loader NAME varies @@ -257,126 +119,6 @@ export std::string detect_baked_loader(const std::string& specsContent) { // which prints what is compiled into the binary and is therefore always // recoverable. // -// patchelf_walk stays: making the compiler itself runnable is a different -// question from what the compiler produces. - - -// Regenerate the clang driver cfg files after the LLVM payload landed in the -// sandbox. The cfg xlings authored at install time is a per-machine, -// per-install-path artifact (its content depended on what existed when the -// package was installed); mcpp's builds bypass it entirely -// (--no-default-config), so its only remaining job is to make a HUMAN -// running `clang++` directly get a working, hermetic compiler. We therefore -// regenerate it deterministically from the same link model the builds use, -// instead of line-patching whatever a given install produced: -// C + C++: -B/-L glibc payload, payload dynamic linker + rpath, -// lld / compiler-rt / libunwind -// C++ only: -nostdinc++ -stdlib=libc++ + payload libc++ headers/libs -// On macOS the C library comes from the SDK: --sysroot= + libc++ headers. -export void fixup_clang_cfg(const std::filesystem::path& payloadRoot, - const std::filesystem::path& glibcLibDir, - const std::filesystem::path& fenceRoot) { - // #273 fence — same rule the removed gcc specs fixup used. - if (escapes_containment(payloadRoot, fenceRoot)) { - mcpp::log::verbose("toolchain", - "fixup_clang_cfg: skip (payload outside sandbox, #273 fence)"); - return; - } - auto binDir = payloadRoot / "bin"; - if (!std::filesystem::exists(binDir)) return; - - // Target triple from the payload layout (lib/), used for the - // loader lookup and the per-target libc++ include/lib dirs. - std::string triple; - std::error_code ec; - for (auto it = std::filesystem::directory_iterator(payloadRoot / "lib", ec); - !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { - auto name = it->path().filename().string(); - if (it->is_directory(ec) && name.find("-linux-") != std::string::npos) { - triple = name; - break; - } - } - - std::string common, cxxOnly, cHdr; - auto cxxInclude = payloadRoot / "include" / "c++" / "v1"; - if constexpr (mcpp::platform::is_macos) { - // macOS keeps its historical cfg semantics: the C library and the - // C++ runtime LINK both come from the SDK; only the libc++ HEADERS - // come from the payload. Do NOT add -nostdinc++/-stdlib=libc++ - // here — a bare cfg-driven link has no libc++abi handling (that - // lives in the main build's needs_explicit_libcxx path) and dies - // with undefined __cxa_* / __gxx_personality_v0. - if (auto sdk = mcpp::platform::macos::sdk_path()) - common += "--sysroot=" + sdk->string() + "\n"; - if (std::filesystem::exists(cxxInclude)) - cxxOnly += "-isystem " + cxxInclude.string() + "\n"; - } else { - if constexpr (mcpp::platform::is_linux) - common += "-nostdlibinc\n"; - if (!glibcLibDir.empty()) { - auto loader = resolve_loader(glibcLibDir, triple); - common += "-B" + glibcLibDir.string() + "\n"; - common += "-L" + glibcLibDir.string() + "\n"; - if (!loader.empty()) - common += "-Wl,--dynamic-linker=" + loader.string() + "\n"; - common += "-Wl,--enable-new-dtags,-rpath," + glibcLibDir.string() + "\n"; - } - common += "-fuse-ld=lld\n--rtlib=compiler-rt\n--unwindlib=libunwind\n"; - - // HEADER axis (C and C++ drivers alike): the C library and kernel - // headers come from the same payloads the link axis uses. Without - // these, a direct `clang hello.c` only works when the HOST happens - // to ship /usr/include — silently non-hermetic, broken on - // header-less machines. For C++ they must come AFTER the libc++ - // block (its C-header wrappers reach libc via #include_next), so - // they are collected separately and appended in order below — - // byte-consistent with what llvm.lua's install hook generates. - if (!glibcLibDir.empty()) { - auto glibcInclude = glibcLibDir.parent_path() / "include"; - if (std::filesystem::exists(glibcInclude / "features.h")) - cHdr += "-isystem " + glibcInclude.string() + "\n"; - constexpr std::string_view kLinuxLimits = "include/linux/limits.h"; - auto linuxHeaders = mcpp::xlings::paths::find_sibling_package( - payloadRoot / "bin" / "clang++", "linux-headers", kLinuxLimits); - if (!linuxHeaders) - linuxHeaders = mcpp::xlings::paths::find_home_tool( - "linux-headers", kLinuxLimits); - if (linuxHeaders) - cHdr += "-isystem " + (*linuxHeaders / "include").string() + "\n"; - } - - if (std::filesystem::exists(cxxInclude)) { - cxxOnly += "-nostdinc++\n-stdlib=libc++\n"; - cxxOnly += "-isystem " + cxxInclude.string() + "\n"; - } - if (!triple.empty()) { - auto tripleInclude = payloadRoot / "include" / triple / "c++" / "v1"; - if (std::filesystem::exists(tripleInclude)) - cxxOnly += "-isystem " + tripleInclude.string() + "\n"; - } - cxxOnly += cHdr; - if (!triple.empty()) { - auto tripleLib = payloadRoot / "lib" / triple; - if (std::filesystem::exists(tripleLib)) { - cxxOnly += "-L" + tripleLib.string() + "\n"; - cxxOnly += "-Wl,-rpath," + tripleLib.string() + "\n"; - } - } - } - - // Regenerate every existing cfg in bin/ (clang.cfg, clang++.cfg, and any - // versioned clang-.cfg xlings created), classified C vs C++ by - // whether the driver name contains "++". - for (auto it = std::filesystem::directory_iterator(binDir, ec); - !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { - auto name = it->path().filename().string(); - if (!name.ends_with(".cfg")) continue; - const bool isCxx = name.find("++") != std::string::npos; - std::ofstream os(it->path()); - os << common << (isCxx ? cxxOnly : cHdr); - } -} // Resolve one glibc payload from the RuntimeBinding identity. The lookup is // exact: `glibc@2.44.3` means the `2.44.3` directory and no other, which is @@ -439,239 +181,20 @@ select_glibc_payload_lib(const std::filesystem::path& glibcRoot, return lib; } -std::expected -find_sandbox_glibc_lib(const mcpp::xlings::Env& xlEnv, - std::string_view runtimeId) { - return select_glibc_payload_lib( - mcpp::xlings::paths::xim_tool_root(xlEnv, "glibc"), runtimeId); -} - -// Post-install fixup for a freshly-installed GNU gcc payload: patchelf -// PT_INTERP/RUNPATH for gcc/binutils binaries + linker-specs wiring against -// the sandbox glibc — without it a fresh-sandbox glibc gcc cannot find the -// C library (stdlib.h not found). -void gcc_post_install_fixup(const mcpp::config::GlobalConfig& cfg, - const std::filesystem::path& payloadRoot, - const std::filesystem::path& glibcLibDir) { - auto xlEnv = mcpp::config::make_xlings_env(cfg); - auto gccLibDir = payloadRoot / "lib64"; - auto patchelfBin = mcpp::xlings::paths::xim_tool(xlEnv, "patchelf", - mcpp::xlings::pinned::kPatchelfVersion) / "bin" / "patchelf"; - - if (!glibcLibDir.empty() && std::filesystem::exists(gccLibDir) - && std::filesystem::exists(patchelfBin)) - { - auto loader = resolve_loader(glibcLibDir, /*targetTriple=*/{}); - auto rpath = std::format("{}:{}", - glibcLibDir.string(), gccLibDir.string()); - - mcpp::log::verbose("toolchain", std::format( - "gcc fixup: patchelf_walk rpath='{}'", rpath)); - // Single trust root for every walk below — including the binutils - // SIBLING payloads, which the entry-point ownership guard does not - // cover (it only vets the gcc payload itself). - auto fence = containment_root(cfg.registryDir); - auto binutilsRoot = mcpp::xlings::paths::xim_tool_root(xlEnv, "binutils"); - if (std::filesystem::exists(binutilsRoot)) { - for (auto& v : std::filesystem::directory_iterator(binutilsRoot)) - patchelf_walk(v.path(), loader, rpath, patchelfBin, fence); - } - patchelf_walk(payloadRoot, loader, rpath, patchelfBin, fence); - - } else { - mcpp::ui::warning( - "could not locate sandbox glibc/gcc/patchelf paths; " - "gcc-built binaries may have unresolved PT_INTERP/RUNPATH"); - } -} - -// LLVM payload fixup: RUNPATH for the bundled runtime shared libraries -// (libc++.so / libunwind.so need to find siblings like libatomic.so.1 after -// the payload moved) + deterministic cfg regeneration. Only lib/ dirs are -// walked — NOT bin/: the clang++ binary's own RUNPATH (zlib, libxml2, …) was -// set by xlings and must be preserved. -void llvm_post_install_fixup(const mcpp::config::GlobalConfig& cfg, - const std::filesystem::path& payloadRoot, - const std::filesystem::path& glibcLibDir) { - auto xlEnv = mcpp::config::make_xlings_env(cfg); - auto patchelfBin = mcpp::xlings::paths::xim_tool(xlEnv, "patchelf", - mcpp::xlings::pinned::kPatchelfVersion) / "bin" / "patchelf"; - if (!glibcLibDir.empty() && std::filesystem::exists(patchelfBin)) { - auto loader = resolve_loader(glibcLibDir, /*targetTriple=*/{}); - auto llvmLib = payloadRoot / "lib"; - std::string rpath; - std::error_code ec; - for (auto it = std::filesystem::directory_iterator(llvmLib, ec); - !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { - if (it->is_directory(ec) - && it->path().filename().string().find("-linux-") != std::string::npos) - rpath += it->path().string() + ":"; - } - rpath += llvmLib.string() + ":" + glibcLibDir.string(); - mcpp::log::verbose("toolchain", std::format( - "llvm fixup: patchelf_walk lib/ rpath='{}'", rpath)); - patchelf_walk(llvmLib, loader, rpath, patchelfBin, - containment_root(cfg.registryDir)); - } - mcpp::log::verbose("toolchain", "llvm fixup: fixup_clang_cfg"); - fixup_clang_cfg(payloadRoot, glibcLibDir, containment_root(cfg.registryDir)); -} - -// ── the single fixup pipeline entry ────────────────────────────────────── -// -// Called from the payload-resolution seam shared by ALL toolchain install -// paths (explicit `mcpp toolchain install`, default-toolchain auto-install, -// and manifest `[toolchain]` auto-install). Previously each path remembered -// (or forgot) its own subset of fixups: the manifest path ran none, which is -// how a fresh llvm install kept a stale install-time cfg and unpatched -// runtime libs. Idempotent via a content-fingerprinted marker. -// -// Bump when the fixup logic changes so existing installs re-run it. -constexpr std::string_view kFixupRev = "hermetic-5-managed-headers"; - -// What the fixup DID, so the caller can decide how loud to be about it. -// -// mcpp#427: this used to be `expected`, and "the runtime identity is -// unknown" was an ERROR inside this function. That made one severity for every -// caller, and it was the wrong one: `mcpp build` died on a machine whose -// default SubOS predates xlings' `subos_info` block — an absence, not a -// contradiction — while the very function this gate protects already degrades -// correctly on an empty `glibcLibDir` (it warns and skips patching). The -// degradation existed and was unreachable. -// -// Severity belongs to the caller. `build` continues; `toolchain install`, -// which the user asked for explicitly, says so at warning level. -export struct FixupOutcome { - bool applied = false; // patching actually ran - std::string skippedReason; // non-empty ⇒ degraded, and why -}; - -// WHICH TOOLCHAIN PAYLOADS THE FIXUP PATCHES AGAINST A C RUNTIME, stated once. -// -// Empty for every payload the fixup leaves alone. On Windows it is always -// empty: a PE toolchain has nothing to patch. Callers that must provide the -// runtime payload before a fixup ask this rather than repeating the dispatch, -// so a toolchain that needs no C runtime payload never causes one to be -// installed. -export std::string_view post_install_fixup_kind(const XimToolchainPackage& pkg) { +// WHICH TOOLCHAIN PAYLOADS RUN ON A C RUNTIME PAYLOAD, stated once: a GNU +// gcc and an llvm on Linux. The runtime they run on is the one the provider +// bound them to when it installed them (its home's runtime pin, design +// 2026-10-16 §7.2); mcpp makes sure that runtime is installed and writes +// nothing into the toolchain (§7.1: the patchelf, specs and cfg rewrites +// that once lived here are gone). Empty on every other payload and on +// Windows, where a PE toolchain has no C runtime payload. +export std::string_view runtime_payload_kind(const XimToolchainPackage& pkg) { if constexpr (mcpp::platform::is_windows) return {}; if (pkg.needsGccPostInstallFixup) return "gcc"; if (pkg.ximName == "llvm") return "llvm"; return {}; } -export std::expected -ensure_post_install_fixup(const mcpp::config::GlobalConfig& cfg, - const std::filesystem::path& payloadRoot, - const XimToolchainPackage& pkg, - std::string_view runtimeId = {}, - const std::filesystem::path& selectedRuntimeLibDir = {}) { - const std::string kind{post_install_fixup_kind(pkg)}; - if (kind.empty()) return FixupOutcome{}; - - // Ownership guard: payloads inherited via symlink from another MCPP_HOME - // are not ours to patch — their owner already ran the fixup, and patching - // through the symlink would rewrite the canonical files against OUR - // (possibly ephemeral) paths, bricking the owner's toolchain. - // Rewritten on the shared containment predicate (#273): the old inline - // check reused one error_code across both canonicalizations (the second - // success cleared the first failure) and compared raw strings without a - // component boundary (a `registry-evil` sibling passed). The predicate - // fails closed on any resolution error. - if (escapes_containment(payloadRoot, containment_root(cfg.registryDir))) { - mcpp::log::verbose("toolchain", std::format( - "skip {} fixup: payload '{}' resolves outside this home — " - "inherited payload, owner is responsible for its fixup", - kind, payloadRoot.string())); - return FixupOutcome{}; - } - - auto xlEnv = mcpp::config::make_xlings_env(cfg); - std::filesystem::path glibcLibDir; - std::string skipped; - if constexpr (mcpp::platform::is_linux) { - // ONE DERIVATION, AND IT IS NOT THIS FUNCTION'S. - // - // The caller resolves a RuntimeBinding first and hands the identity in; - // `prepare.cppm` says so where it does it ("The fixup is itself a - // consumer of RuntimeBinding: doing it first would recreate #392 by - // letting directory order choose a libc"). This function used to ALSO - // read `/subos/default` when the identity was empty — - // a second derivation that (a) contradicted that architecture, (b) - // consulted a HARDCODED SubOS even when the project selected another - // one, and (c) turned an absent description into a fatal error. - // - // Unknown degrades. A CONTRADICTION — an identity that is declared but - // cannot be honoured — still fails, below. - std::string selected(runtimeId); - if (selected.empty()) { - skipped = "the selected SubOS declares no runtime identity, so " - "there is no C runtime to bind this toolchain to"; - } else if (!selectedRuntimeLibDir.empty()) { - constexpr std::string_view prefix = "glibc@"; - auto version = selected.starts_with(prefix) - ? std::string_view(selected).substr(prefix.size()) - : std::string_view{}; - auto payload = selectedRuntimeLibDir.parent_path(); - auto discovered = payload_lib_dir_with_loader(payload); - std::error_code dec, sec; - auto discoveredReal = std::filesystem::weakly_canonical(discovered, dec); - auto selectedReal = std::filesystem::weakly_canonical( - selectedRuntimeLibDir, sec); - if (version.empty() || payload.filename() != version - || discovered.empty() || dec || sec - || discoveredReal != selectedReal) { - return std::unexpected(std::format( - "selected RuntimeBinding {} points at incompatible/stale " - "runtime directory '{}'; expected the exact payload version", - selected, selectedRuntimeLibDir.string())); - } - glibcLibDir = selectedRuntimeLibDir; - } else { - auto exact = find_sandbox_glibc_lib(xlEnv, selected); - if (!exact) return std::unexpected(exact.error()); - glibcLibDir = std::move(*exact); - } - } - - // Nothing to bind to. Report it and change nothing — patching a payload - // against a runtime we cannot name would be a guess, and a guess here is - // an ABI. NO MARKER IS WRITTEN: "we did nothing" must never be able to - // read back as "already applied", so the day the SubOS learns to describe - // itself the fixup runs on the next invocation. - if (!skipped.empty()) - return FixupOutcome{.applied = false, .skippedReason = std::move(skipped)}; - - // Content-fingerprinted marker: a marker whose INPUTS drifted (different - // glibc payload, newer fixup logic) re-runs the fixup — "a process once - // exited 0" is not evidence the current inputs were ever applied. - auto markerPath = payloadRoot / ".mcpp-fixup.json"; - nlohmann::json expected; - expected["schema"] = 1; - expected["kind"] = kind; - expected["rev"] = std::string(kFixupRev); - expected["glibcLib"] = glibcLibDir.generic_string(); - { - std::ifstream is(markerPath); - if (is) { - try { - nlohmann::json actual; - is >> actual; - if (actual == expected) // fixup already applied - return FixupOutcome{.applied = true, .skippedReason = {}}; - } catch (...) { /* corrupt marker → re-run */ } - } - } - - if (kind == "gcc") gcc_post_install_fixup(cfg, payloadRoot, glibcLibDir); - else llvm_post_install_fixup(cfg, payloadRoot, glibcLibDir); - - std::ofstream os(markerPath); - os << expected.dump(2) << "\n"; - return FixupOutcome{.applied = true, .skippedReason = {}}; -} - - // PT_INTERP of an ELF, read directly. // // Not via `patchelf --print-interpreter`: this runs during prepare, where @@ -768,8 +291,8 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { // host loader. A fossil that names something no longer present is not an // authority; it is just old. std::vector installed; - if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { - for (auto it = std::filesystem::directory_iterator(*xpkgs / "xim-x-glibc", ec); + if (auto xpkgs = mcpp::provider::store_of(compilerBin)) { + for (auto it = std::filesystem::directory_iterator(mcpp::provider::layout().tool_in(*xpkgs, "glibc"), ec); !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { if (!it->is_directory(ec)) continue; auto v = it->path().filename().string(); diff --git a/src/toolchain/probe.cppm b/src/toolchain/probe.cppm index b9379d728..642ec4468 100644 --- a/src/toolchain/probe.cppm +++ b/src/toolchain/probe.cppm @@ -15,7 +15,8 @@ export module mcpp.toolchain.probe; import std; import mcpp.toolchain.model; import mcpp.toolchain.triple; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.platform; import mcpp.log; import mcpp.fallback.sysroot_complete; @@ -192,7 +193,7 @@ discover_compiler_runtime_dirs(const std::filesystem::path& compilerBin) { auto rootStr = root.string(); auto exe = compilerBin.filename().string(); - bool looksLikeLlvm = rootStr.find("xim-x-llvm") != std::string::npos + bool looksLikeLlvm = mcpp::provider::layout().entry_of(root).name == "llvm" || exe.find("clang") != std::string::npos; if (looksLikeLlvm) { append_existing_unique(dirs, root / "lib"); @@ -202,7 +203,7 @@ discover_compiler_runtime_dirs(const std::filesystem::path& compilerBin) { append_existing_unique(dirs, d); } - if (auto rt = mcpp::xlings::paths::find_sibling_tool(compilerBin, "gcc-runtime")) { + if (auto rt = mcpp::provider::beside_tool(compilerBin, "gcc-runtime")) { append_existing_unique(dirs, *rt / "lib64"); append_existing_unique(dirs, *rt / "lib"); } @@ -248,7 +249,7 @@ discover_link_runtime_dirs(const std::filesystem::path& compilerBin, append_existing_unique(dirs, root / "lib"); if constexpr (mcpp::platform::is_linux) { - if (auto rt = mcpp::xlings::paths::find_sibling_tool(compilerBin, "gcc-runtime")) { + if (auto rt = mcpp::provider::beside_tool(compilerBin, "gcc-runtime")) { append_existing_unique(dirs, *rt / "lib64"); append_existing_unique(dirs, *rt / "lib"); } @@ -292,7 +293,7 @@ probe_target_triple(const std::filesystem::path& compilerBin, const std::string& envPrefix) { auto triple_r = run_capture(std::format("{}{} -dumpmachine {}", envPrefix, - mcpp::xlings::shq(compilerBin.string()), + mcpp::platform::shell::quote(compilerBin.string()), mcpp::platform::null_redirect)); if (!triple_r) return std::unexpected(triple_r.error()); return trim_line(*triple_r); @@ -313,7 +314,7 @@ probe_sysroot(const std::filesystem::path& compilerBin, // 1. Ask the compiler directly (works for GCC; Clang often doesn't support it). auto r = run_capture(std::format("{}{} -print-sysroot {}", envPrefix, - mcpp::xlings::shq(compilerBin.string()), + mcpp::platform::shell::quote(compilerBin.string()), mcpp::platform::null_redirect)); if (r) { auto s = trim_line(*r); @@ -411,17 +412,17 @@ payload_root_for_binding(const std::filesystem::path& compilerBin, // bits/os_defines.h:39: fatal error: features.h: No such file const auto exact = [&](const std::filesystem::path& xpkgs) -> std::optional { - auto root = xpkgs / std::format("xim-x-{}", name) / version; + auto root = mcpp::provider::layout().tool_in(xpkgs, name) / version; std::error_code ec; if (std::filesystem::is_directory(root, ec)) return root; return std::nullopt; }; // Compiler siblings: <...>/xpkgs/xim-x-/ - if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { + if (auto xpkgs = mcpp::provider::store_of(compilerBin)) { if (auto root = exact(*xpkgs)) return *root; } // Active home. - if (auto xpkgs = mcpp::xlings::paths::active_home_xpkgs()) { + if (auto xpkgs = mcpp::provider::home_store()) { if (auto root = exact(*xpkgs)) return *root; } return std::nullopt; @@ -458,8 +459,6 @@ probe_payload_paths(const std::filesystem::path& compilerBin, return std::nullopt; } - namespace paths = mcpp::xlings::paths; - // Layout WITHIN the chosen payload. This convention stays: it answers // "where inside", not "which one". auto glibcInclude = *glibc / "include"; @@ -484,9 +483,9 @@ probe_payload_paths(const std::filesystem::path& compilerBin, // needs , so a silent miss breaks every glibc build). constexpr std::string_view kLinuxLimits = "include/linux/limits.h"; auto linuxHeaders = - paths::find_sibling_package(compilerBin, "linux-headers", kLinuxLimits); + mcpp::provider::beside(compilerBin, "linux-headers", kLinuxLimits); if (!linuxHeaders) - linuxHeaders = paths::find_home_tool("linux-headers", kLinuxLimits); + linuxHeaders = mcpp::provider::in_home_store("linux-headers", kLinuxLimits); if (linuxHeaders) { pp.linuxInclude = *linuxHeaders / "include"; } else { diff --git a/src/toolchain/registry.cppm b/src/toolchain/registry.cppm index f77bfd1c8..c96c4879f 100644 --- a/src/toolchain/registry.cppm +++ b/src/toolchain/registry.cppm @@ -19,7 +19,8 @@ export module mcpp.toolchain.registry; import std; import mcpp.libs.json; import mcpp.platform; -import mcpp.xlings; +import mcpp.provider; +import mcpp.provider.select; import mcpp.toolchain.clang; import mcpp.toolchain.compat; import mcpp.toolchain.gcc; @@ -396,7 +397,7 @@ Origin origin_of(const ToolchainSpec& spec); // `identifyVersion = false` skips running cl.exe for its banner; callers that // only need to know whether a toolset is there should pass false. std::expected -resolve_managed_msvc(const mcpp::xlings::Env& env, +resolve_managed_msvc(const mcpp::provider::Location& env, const XimToolchainPackage& pkg, bool identifyVersion = true); @@ -664,7 +665,7 @@ Origin origin_of(const ToolchainSpec& spec) { } std::expected -resolve_managed_msvc(const mcpp::xlings::Env& env, +resolve_managed_msvc(const mcpp::provider::Location& env, const XimToolchainPackage& pkg, bool identifyVersion) { // NOT the fetcher's `root`: that field is its guess at where the useful @@ -672,7 +673,7 @@ resolve_managed_msvc(const mcpp::xlings::Env& env, // dir has no bin/ include/ lib/. An msvc payload's only entry is `VC/`, // so the guess lands exactly one level too deep. (store, name, version) // is known — compose it. - auto verDir = mcpp::xlings::paths::xim_tool(env, pkg.ximName, pkg.ximVersion); + auto verDir = mcpp::provider::open(env)->pinned(pkg.ximName, pkg.ximVersion); // The package version IS the toolset directory name, so cl.exe is derived // rather than searched for: nothing here can silently pick a different // toolset, which is the whole reason the version axis exists. diff --git a/tests/e2e/151_index_url_artifact_migration.sh b/tests/e2e/151_index_url_artifact_migration.sh index 21c9728ae..cee05ff3a 100755 --- a/tests/e2e/151_index_url_artifact_migration.sh +++ b/tests/e2e/151_index_url_artifact_migration.sh @@ -21,6 +21,18 @@ REGION="{ \"GLOBAL\": \"$ART\", \"CN\": \"$ART_CN\" }" CFG="$MCPP_HOME/config.toml" XJ="$MCPP_HOME/registry/.xlings.json" +# The mcpplibs entry of .xlings.json has these fields (a JSON object), however +# the file is laid out. +entry_has() { + python3 - "$XJ" "$1" <<'PYEOF2' +import json, sys +doc = json.load(open(sys.argv[1])) +want = json.loads(sys.argv[2]) +e = next(r for r in doc["index_repos"] if r["name"] == "mcpplibs") +sys.exit(0 if all(e.get(k) == v for k, v in want.items()) else 1) +PYEOF2 +} + # 1. Fresh init: new org URL + artifact declaration in both files. # (xlings' own bootstrap may re-serialize .xlings.json with its writer, so # assert on the key/value pairs, not on mcpp's seed line layout.) @@ -64,7 +76,7 @@ cat > "$XJ" < /dev/null grep -q "url = \"$NEW_URL\"" "$CFG" || { echo "config.toml url not healed"; cat "$CFG"; exit 1; } -grep -qF "\"url\": \"$NEW_URL\", \"artifact\": $REGION" "$XJ" \ +entry_has "{\"url\": \"$NEW_URL\", \"artifact\": $REGION}" \ || { echo ".xlings.json not healed"; cat "$XJ"; exit 1; } grep -q '"subos": "default"' "$XJ" || { echo "unrelated .xlings.json key lost"; exit 1; } if grep -q 'mcpp-community/mcpp-index' "$CFG" "$XJ"; then @@ -80,7 +92,7 @@ cmp -s "$XJ" "$TMP/xj1" || { echo ".xlings.json heal not idempotent"; exit 1; # 3. Legacy heal (compact spacing, old index name): xlings-writer format. printf '{"index_repos":[{"name":"mcpp-index","url":"%s"}],"mirror":"auto"}' "$OLD_URL" > "$XJ" "$MCPP" self env > /dev/null -grep -q '"name":"mcpplibs"' "$XJ" || { echo "compact name not healed"; cat "$XJ"; exit 1; } +entry_has '{}' || { echo "compact name not healed"; cat "$XJ"; exit 1; } grep -q 'xlings-res/mcpp-index' "$XJ" || { echo "compact artifact not injected"; cat "$XJ"; exit 1; } grep -q 'gitcode.com/xlings-res/mcpp-index' "$XJ" || { echo "compact artifact has no CN half"; cat "$XJ"; exit 1; } @@ -88,7 +100,7 @@ grep -q 'gitcode.com/xlings-res/mcpp-index' "$XJ" || { echo "compact artifact ha # gains the CN half, once; a base the user wrote is left alone. printf '{\n "index_repos": [\n { "name": "mcpplibs", "url": "%s", "artifact": "%s" }\n ],\n "mirror": "CN"\n}\n' "$NEW_URL" "$ART" > "$XJ" "$MCPP" self env > /dev/null -grep -qF "\"artifact\": $REGION" "$XJ" || { echo "flat default artifact not upgraded"; cat "$XJ"; exit 1; } +entry_has "{\"artifact\": $REGION}" || { echo "flat default artifact not upgraded"; cat "$XJ"; exit 1; } cp "$XJ" "$TMP/xj4"; "$MCPP" self env > /dev/null cmp -s "$XJ" "$TMP/xj4" || { echo "region upgrade not idempotent"; diff "$TMP/xj4" "$XJ"; exit 1; } # A base the user states in config.toml owns the entry, and has no CN half. diff --git a/tests/e2e/163_identity_first_resolution.sh b/tests/e2e/163_identity_first_resolution.sh index 2c470fd30..c34dd848f 100755 --- a/tests/e2e/163_identity_first_resolution.sh +++ b/tests/e2e/163_identity_first_resolution.sh @@ -30,7 +30,7 @@ cd "$TMP" # A real, downloadable payload so this exercises the whole install path rather # than stopping at resolution. Reuse whatever asio the registry already has. SRC="" -for c in "$MCPP_HOME/registry/data/mcpplibs/pkgs/c/chriskohlhoff.asio.lua"; do +for c in "$MCPP_HOME/indices/mcpplibs/pkgs/c/chriskohlhoff.asio.lua"; do [ -f "$c" ] && SRC="$c" && break done if [ -z "$SRC" ]; then diff --git a/tests/e2e/165_bare_name_cross_namespace_wire_address.sh b/tests/e2e/165_bare_name_cross_namespace_wire_address.sh index dc5e91cff..6bb12e0ce 100755 --- a/tests/e2e/165_bare_name_cross_namespace_wire_address.sh +++ b/tests/e2e/165_bare_name_cross_namespace_wire_address.sh @@ -29,7 +29,7 @@ cd "$TMP" # Reuse a real, downloadable descriptor so this exercises the whole install # path rather than stopping at resolution. -SRC="$MCPP_HOME/registry/data/mcpplibs/pkgs/c/chriskohlhoff.asio.lua" +SRC="$MCPP_HOME/indices/mcpplibs/pkgs/c/chriskohlhoff.asio.lua" if [ ! -f "$SRC" ]; then echo "SKIP 165_bare_name_cross_namespace_wire_address (no asio descriptor in registry)" exit 0 diff --git a/tests/e2e/173_index_refresh_policy.sh b/tests/e2e/173_index_refresh_policy.sh index 344a50718..54f88ef0e 100755 --- a/tests/e2e/173_index_refresh_policy.sh +++ b/tests/e2e/173_index_refresh_policy.sh @@ -20,7 +20,7 @@ trap "rm -rf $TMP" EXIT export MCPP_HOME="$TMP/mcpp-home" source "$(dirname "$0")/_inherit_toolchain.sh" -INDEX="$MCPP_HOME/registry/data/mcpplibs" +INDEX="$MCPP_HOME/indices/mcpplibs" mkdir -p "$INDEX/pkgs/w" cat > "$INDEX/pkgs/w/widget.lua" <<'EOF' diff --git a/tests/e2e/204_new_transactional_scaffold.sh b/tests/e2e/204_new_transactional_scaffold.sh index 85c3d90be..fba4054d3 100755 --- a/tests/e2e/204_new_transactional_scaffold.sh +++ b/tests/e2e/204_new_transactional_scaffold.sh @@ -48,7 +48,7 @@ mkdir -p "$MCPP_HOME" >/dev/null 2>&1 || true # Exact foreign-namespace provider with a same-short default-namespace sibling. -INDEX="$MCPP_HOME/registry/data/mcpplibs/pkgs/t" +INDEX="$MCPP_HOME/indices/mcpplibs/pkgs/t" mkdir -p "$INDEX" for row in 'acme tx-template acme-x-tx-template 2.0.0' \ 'mcpplibs tx-template mcpplibs-x-tx-template 9.0.0'; do diff --git a/tests/e2e/32_semver_merge.sh b/tests/e2e/32_semver_merge.sh index f0c7e4303..bd4f1364a 100755 --- a/tests/e2e/32_semver_merge.sh +++ b/tests/e2e/32_semver_merge.sh @@ -16,14 +16,14 @@ export MCPP_HOME="$TMP/mcpp-home" source "$(dirname "$0")/_inherit_toolchain.sh" # Index is needed since we exercise version-source deps. -mkdir -p "$MCPP_HOME/registry/data" -# Link pre-cached index data (may be under old "mcpp-index" or new "mcpplibs" name) -for idx_name in mcpplibs mcpp-index; do - if [[ -d "$HOME/.mcpp/registry/data/$idx_name" ]]; then - ln -sf "$HOME/.mcpp/registry/data/$idx_name" \ - "$MCPP_HOME/registry/data/$idx_name" 2>/dev/null \ - || cp -r "$HOME/.mcpp/registry/data/$idx_name" \ - "$MCPP_HOME/registry/data/$idx_name" +mkdir -p "$MCPP_HOME/registry/data" "$MCPP_HOME/indices" +# Link the pre-cached mcpplibs index: where mcpp keeps it (indices/), or where +# an earlier mcpp kept it, under the old or the new name. +for src in "$HOME/.mcpp/indices/mcpplibs" "$HOME/.mcpp/registry/data/mcpplibs" "$HOME/.mcpp/registry/data/mcpp-index"; do + if [[ -d "$src" ]]; then + ln -sf "$src" "$MCPP_HOME/indices/mcpplibs" 2>/dev/null \ + || cp -r "$src" "$MCPP_HOME/indices/mcpplibs" + break fi done # Pre-cached xpkg downloads so the test doesn't re-fetch the world. diff --git a/tests/e2e/33_multi_version_mangling.sh b/tests/e2e/33_multi_version_mangling.sh index a489a706e..bfe80e8f6 100755 --- a/tests/e2e/33_multi_version_mangling.sh +++ b/tests/e2e/33_multi_version_mangling.sh @@ -20,12 +20,14 @@ export MCPP_HOME="$TMP/mcpp-home" source "$(dirname "$0")/_inherit_toolchain.sh" # Index + xpkgs need to be visible since we exercise version-source deps. -mkdir -p "$MCPP_HOME/registry/data" -# Link pre-cached index data (may be under old "mcpp-index" or new "mcpplibs" name) -for idx_name in mcpplibs mcpp-index; do - if [[ -d "$HOME/.mcpp/registry/data/$idx_name" ]]; then - ln -sf "$HOME/.mcpp/registry/data/$idx_name" \ - "$MCPP_HOME/registry/data/$idx_name" +mkdir -p "$MCPP_HOME/registry/data" "$MCPP_HOME/indices" +# Link the pre-cached mcpplibs index: where mcpp keeps it (indices/), or where +# an earlier mcpp kept it, under the old or the new name. +for src in "$HOME/.mcpp/indices/mcpplibs" "$HOME/.mcpp/registry/data/mcpplibs" "$HOME/.mcpp/registry/data/mcpp-index"; do + if [[ -d "$src" ]]; then + ln -sf "$src" "$MCPP_HOME/indices/mcpplibs" 2>/dev/null \ + || cp -r "$src" "$MCPP_HOME/indices/mcpplibs" + break fi done if [[ -d "$HOME/.mcpp/registry/data/xpkgs" ]]; then diff --git a/tests/e2e/40_llvm_bmi_cache.sh b/tests/e2e/40_llvm_bmi_cache.sh index 94361a06d..3f7b2b566 100755 --- a/tests/e2e/40_llvm_bmi_cache.sh +++ b/tests/e2e/40_llvm_bmi_cache.sh @@ -31,11 +31,11 @@ source "$(dirname "$0")/_inherit_toolchain.sh" # mcpplibs packages live in a separate registry namespace; inherit it so the # index lookup for mcpplibs.cmdline succeeds in the isolated MCPP_HOME. USER_MCPP="${HOME}/.mcpp" -if [[ -d "$USER_MCPP/registry/data/mcpplibs" ]]; then - mkdir -p "$MCPP_HOME/registry/data" - [[ -e "$MCPP_HOME/registry/data/mcpplibs" ]] \ - || ln -sf "$USER_MCPP/registry/data/mcpplibs" "$MCPP_HOME/registry/data/mcpplibs" 2>/dev/null \ - || cp -r "$USER_MCPP/registry/data/mcpplibs" "$MCPP_HOME/registry/data/mcpplibs" +if [[ -d "$USER_MCPP/indices/mcpplibs" ]]; then + mkdir -p "$MCPP_HOME/indices" + [[ -e "$MCPP_HOME/indices/mcpplibs" ]] \ + || ln -sf "$USER_MCPP/indices/mcpplibs" "$MCPP_HOME/indices/mcpplibs" 2>/dev/null \ + || cp -r "$USER_MCPP/indices/mcpplibs" "$MCPP_HOME/indices/mcpplibs" fi mkdir -p "$TMP/proj/src" diff --git a/tests/e2e/52_local_path_namespaced_index.sh b/tests/e2e/52_local_path_namespaced_index.sh index 7b231895e..100f55856 100755 --- a/tests/e2e/52_local_path_namespaced_index.sh +++ b/tests/e2e/52_local_path_namespaced_index.sh @@ -116,6 +116,8 @@ if [[ -d "$USER_MCPP/registry/data/xpkgs" ]]; then fi cat > "$TMP/fake-bin/xlings" <<'EOF' #!/usr/bin/env bash +# The handshake mcpp asks of every xlings (docs/25 §5). +[[ "${1:-}" == "interface" && "${2:-}" == "--version" ]] && { echo '{"protocol_version":"1.10","version":"9999.1.1.1","capabilities":["install_packages"],"home_layouts":[2],"features":["index_dir","index_store","upstreams","payload_status","sync_index"]}'; exit 0; } set -e if [[ "${1:-}" == "self" && "${2:-}" == "init" ]]; then diff --git a/tests/e2e/53_namespaced_cache_label.sh b/tests/e2e/53_namespaced_cache_label.sh index 1ffbc43b7..23ccee6f7 100755 --- a/tests/e2e/53_namespaced_cache_label.sh +++ b/tests/e2e/53_namespaced_cache_label.sh @@ -90,7 +90,7 @@ if grep -Eq '^\[[0-9]+/[0-9]+\] ' build.log; then exit 1 fi -[[ ! -e "$MCPP_HOME/registry/data/mcpplibs/.mcpp-index-updated" ]] || { +[[ ! -e "$MCPP_HOME/indices/mcpplibs/.mcpp-index-updated" ]] || { echo "FAIL: local path index dependency should not refresh builtin mcpplibs index" find "$MCPP_HOME/registry/data" -maxdepth 3 -type f | sort cat build.log diff --git a/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh b/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh index 0328c85a5..36467432b 100644 --- a/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh +++ b/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh @@ -87,6 +87,8 @@ fi cat > "$TMP/fake-bin/xlings" <<'EOF' #!/usr/bin/env bash +# The handshake mcpp asks of every xlings (docs/25 §5). +[[ "${1:-}" == "interface" && "${2:-}" == "--version" ]] && { echo '{"protocol_version":"1.10","version":"9999.1.1.1","capabilities":["install_packages"],"home_layouts":[2],"features":["index_dir","index_store","upstreams","payload_status","sync_index"]}'; exit 0; } set -e if [[ "${1:-}" == "self" && "${2:-}" == "init" ]]; then diff --git a/tests/e2e/60_stale_xpkg_cache_reinstall.sh b/tests/e2e/60_stale_xpkg_cache_reinstall.sh index f16d649d4..29093523a 100644 --- a/tests/e2e/60_stale_xpkg_cache_reinstall.sh +++ b/tests/e2e/60_stale_xpkg_cache_reinstall.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # requires: gcc fresh-sandbox -# Version dependencies must not reuse stale xpkg directories that predate -# mcpp's .mcpp_ok install marker. Such directories can have an old extracted +# Version dependencies must not reuse stale xpkg directories the provider +# does not call complete. Such directories can have an old extracted # layout that no longer matches the package index's mcpp metadata. set -e source "$(dirname "$0")/_host_path.sh" @@ -45,6 +45,8 @@ mkdir -p "$TMP/fake-bin" FAKE_INSTALL_LOG="$TMP/fake-install.log" cat > "$TMP/fake-bin/xlings" <<'EOF' #!/usr/bin/env bash +# The handshake mcpp asks of every xlings (docs/25 §5). +[[ "${1:-}" == "interface" && "${2:-}" == "--version" ]] && { echo '{"protocol_version":"1.10","version":"9999.1.1.1","capabilities":["install_packages"],"home_layouts":[2],"features":["index_dir","index_store","upstreams","payload_status","sync_index"]}'; exit 0; } set -e if [[ "${1:-}" == "self" && "${2:-}" == "init" ]]; then @@ -155,8 +157,9 @@ grep -Fq 'compat:compat.stale@1.0.0' "$FAKE_INSTALL_LOG" || { exit 1 } -test -f .mcpp/.xlings/data/xpkgs/compat-x-compat.stale/1.0.0/.mcpp_ok || { - echo "FAIL: successful reinstall should be marked complete" +# The payload is the provider's: mcpp writes no marker into it (docs/25). +test ! -e .mcpp/.xlings/data/xpkgs/compat-x-compat.stale/1.0.0/.mcpp_ok || { + echo "FAIL: mcpp wrote a marker into the payload" find .mcpp/.xlings/data/xpkgs/compat-x-compat.stale/1.0.0 -maxdepth 2 -print exit 1 } diff --git a/tests/e2e/63_bare_dependency_peer_root_priority.sh b/tests/e2e/63_bare_dependency_peer_root_priority.sh index f132ceba6..6117200bd 100755 --- a/tests/e2e/63_bare_dependency_peer_root_priority.sh +++ b/tests/e2e/63_bare_dependency_peer_root_priority.sh @@ -10,7 +10,7 @@ trap "rm -rf $TMP" EXIT export MCPP_HOME="$TMP/mcpp-home" source "$(dirname "$0")/_inherit_toolchain.sh" -INDEX_DIR="$MCPP_HOME/registry/data/mcpplibs" +INDEX_DIR="$MCPP_HOME/indices/mcpplibs" PKG_ROOT="$MCPP_HOME/registry/data/xpkgs/mcpplibs-x-imgui/1.0.0" mkdir -p "$INDEX_DIR/pkgs/i" "$PKG_ROOT/src" printf 'ok\n' > "$INDEX_DIR/.mcpp-index-updated" diff --git a/tests/e2e/69_package_templates.sh b/tests/e2e/69_package_templates.sh index 6731a7450..9620f3ccb 100755 --- a/tests/e2e/69_package_templates.sh +++ b/tests/e2e/69_package_templates.sh @@ -16,8 +16,8 @@ mkdir -p "$MCPP_HOME" "$MCPP" self config --mirror "${MCPP_E2E_TOOLCHAIN_MIRROR:-GLOBAL}" >/dev/null 2>&1 || true # ── index entry (local official-index clone) ──────────────────────────── -mkdir -p "$MCPP_HOME/registry/data/mcpplibs/pkgs/t" -cat > "$MCPP_HOME/registry/data/mcpplibs/pkgs/t/tpl-demo.lua" <<'EOF' +mkdir -p "$MCPP_HOME/indices/mcpplibs/pkgs/t" +cat > "$MCPP_HOME/indices/mcpplibs/pkgs/t/tpl-demo.lua" <<'EOF' package = { spec = "1", namespace = "mcpplibs", @@ -143,25 +143,25 @@ version = "0.1.0" EOF } -write_descriptor "$MCPP_HOME/registry/data/mcpplibs/pkgs/a/acme.tpl-demo.lua" \ +write_descriptor "$MCPP_HOME/indices/mcpplibs/pkgs/a/acme.tpl-demo.lua" \ acme tpl-demo 3.0.0 seed_template acme-x-tpl-demo tpl-demo 3.0.0 solo false acme-solo -write_descriptor "$MCPP_HOME/registry/data/mcpplibs/pkgs/m/mcpplibs.capi.lua.lua" \ +write_descriptor "$MCPP_HOME/indices/mcpplibs/pkgs/m/mcpplibs.capi.lua.lua" \ mcpplibs.capi lua 5.4.7 seed_template mcpplibs.capi-x-lua lua 5.4.7 module false nested-lua -write_descriptor "$MCPP_HOME/registry/data/mcpplibs/pkgs/a/ambiguous.lua" \ +write_descriptor "$MCPP_HOME/indices/mcpplibs/pkgs/a/ambiguous.lua" \ mcpplibs ambiguous 1.0.0 seed_template mcpplibs-x-ambiguous ambiguous 1.0.0 one false ambiguous-one seed_template mcpplibs-x-ambiguous ambiguous 1.0.0 two false ambiguous-two -write_descriptor "$MCPP_HOME/registry/data/mcpplibs/pkgs/d/dupe-default.lua" \ +write_descriptor "$MCPP_HOME/indices/mcpplibs/pkgs/d/dupe-default.lua" \ mcpplibs dupe-default 1.0.0 seed_template mcpplibs-x-dupe-default dupe-default 1.0.0 one true dupe-one seed_template mcpplibs-x-dupe-default dupe-default 1.0.0 two true dupe-two -write_descriptor "$MCPP_HOME/registry/data/mcpplibs/pkgs/n/no-templates.lua" \ +write_descriptor "$MCPP_HOME/indices/mcpplibs/pkgs/n/no-templates.lua" \ mcpplibs no-templates 1.0.0 mkdir -p "$MCPP_HOME/registry/data/xpkgs/mcpplibs-x-no-templates/1.0.0/no-templates-1.0.0" cat > "$MCPP_HOME/registry/data/xpkgs/mcpplibs-x-no-templates/1.0.0/no-templates-1.0.0/mcpp.toml" <<'EOF' @@ -225,8 +225,8 @@ grep -qi "deprecated" out6.log || { cat out6.log; echo "missing gui deprecation" # 7. Form B (mcpp = { ... }) compat package is selected only when compat is # explicit, then rejected as a non-template payload. Bare `tpl-repro` must not # cross from mcpplibs into compat. -mkdir -p "$MCPP_HOME/registry/data/mcpplibs/pkgs/c" -cat > "$MCPP_HOME/registry/data/mcpplibs/pkgs/c/compat.tpl-repro.lua" <<'EOF' +mkdir -p "$MCPP_HOME/indices/mcpplibs/pkgs/c" +cat > "$MCPP_HOME/indices/mcpplibs/pkgs/c/compat.tpl-repro.lua" <<'EOF' package = { spec = "1", namespace = "compat", diff --git a/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh b/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh index 720b53e33..8d70040a6 100755 --- a/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh +++ b/tests/e2e/730_bare_compat_dependency_is_not_a_refresh_miss.sh @@ -18,7 +18,7 @@ TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } -INDEX_ROOT="${MCPP_HOME:-$HOME/.mcpp}/registry/data/mcpplibs/pkgs/c" +INDEX_ROOT="${MCPP_HOME:-$HOME/.mcpp}/indices/mcpplibs/pkgs/c" DESC="" for cand in cjson argparse gtest ftxui; do if [ -f "$INDEX_ROOT/compat.$cand.lua" ]; then DESC="$cand"; break; fi diff --git a/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh index d4c669b0f..f3e20100d 100755 --- a/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh +++ b/tests/e2e/732_xlings_refresh_is_bounded_and_owned.sh @@ -33,6 +33,8 @@ bounded() { # mkdir -p "$TMP/bin" "$TMP/home" cat > "$TMP/bin/xlings" <<'EOF' #!/usr/bin/env bash +# The handshake mcpp asks of every xlings (docs/25 §5). +[[ "${1:-}" == "interface" && "${2:-}" == "--version" ]] && { echo '{"protocol_version":"1.10","version":"9999.1.1.1","capabilities":["install_packages"],"home_layouts":[2],"features":["index_dir","index_store","upstreams","payload_status","sync_index"]}'; exit 0; } case "${1:-}:${2:-}" in update:*|interface:update_packages) echo "$$" > "${STUB_PID:?}"; exec sleep 1000 ;; esac diff --git a/tests/e2e/733_envelope_reports_observed_network_access.sh b/tests/e2e/733_envelope_reports_observed_network_access.sh index 33a7b2bbb..bc08b3cf3 100755 --- a/tests/e2e/733_envelope_reports_observed_network_access.sh +++ b/tests/e2e/733_envelope_reports_observed_network_access.sh @@ -18,11 +18,13 @@ fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } export MCPP_HOME="$TMP/home" source "$(dirname "$0")/_inherit_toolchain.sh" -rm -rf "$MCPP_HOME/registry/data/mcpplibs" +rm -rf "$MCPP_HOME/indices/mcpplibs" mkdir -p "$TMP/bin" cat > "$TMP/bin/xlings" <<'EOF' #!/usr/bin/env bash +# The handshake mcpp asks of every xlings (docs/25 §5). +[[ "${1:-}" == "interface" && "${2:-}" == "--version" ]] && { echo '{"protocol_version":"1.10","version":"9999.1.1.1","capabilities":["install_packages"],"home_layouts":[2],"features":["index_dir","index_store","upstreams","payload_status","sync_index"]}'; exit 0; } echo "$*" >> "${STUB_LOG:?}" exit 0 EOF diff --git a/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh b/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh index 4953ef96d..e536fc72e 100755 --- a/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh +++ b/tests/e2e/734_auto_refresh_false_governs_first_index_sync.sh @@ -21,6 +21,8 @@ source "$(dirname "$0")/_host_path.sh" mkdir -p "$TMP/bin" cat > "$TMP/bin/xlings" <<'EOF' #!/usr/bin/env bash +# The handshake mcpp asks of every xlings (docs/25 §5). +[[ "${1:-}" == "interface" && "${2:-}" == "--version" ]] && { echo '{"protocol_version":"1.10","version":"9999.1.1.1","capabilities":["install_packages"],"home_layouts":[2],"features":["index_dir","index_store","upstreams","payload_status","sync_index"]}'; exit 0; } echo "$*" >> "${STUB_LOG:?}" exit 0 EOF diff --git a/tests/e2e/76_qualified_custom_ns_noncanonical_filename.sh b/tests/e2e/76_qualified_custom_ns_noncanonical_filename.sh index 80961840c..e9f666b74 100755 --- a/tests/e2e/76_qualified_custom_ns_noncanonical_filename.sh +++ b/tests/e2e/76_qualified_custom_ns_noncanonical_filename.sh @@ -23,7 +23,7 @@ trap "rm -rf $TMP" EXIT export MCPP_HOME="$TMP/mcpp-home" source "$(dirname "$0")/_inherit_toolchain.sh" -INDEX_DIR="$MCPP_HOME/registry/data/mcpplibs" +INDEX_DIR="$MCPP_HOME/indices/mcpplibs" # Observed real install layout is -x-: aimol-x-tensorvia-cpu. PKG_ROOT="$MCPP_HOME/registry/data/xpkgs/aimol-x-tensorvia-cpu/0.1.1" mkdir -p "$INDEX_DIR/pkgs/t" "$PKG_ROOT/src" diff --git a/tests/e2e/812_an_index_floor_is_a_closing_tip.sh b/tests/e2e/812_an_index_floor_is_a_closing_tip.sh index 0a6651093..d2d287cf2 100755 --- a/tests/e2e/812_an_index_floor_is_a_closing_tip.sh +++ b/tests/e2e/812_an_index_floor_is_a_closing_tip.sh @@ -20,23 +20,31 @@ fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } export MCPP_HOME="$TMP/home" source "$(dirname "$0")/_inherit_toolchain.sh" -DATA="$MCPP_HOME/registry/data/mcpplibs" +DATA="$MCPP_HOME/indices/mcpplibs" rm -rf "$DATA" mkdir -p "$DATA/pkgs/f" printf '[index]\nspec = "1"\nmin_mcpp = "0.0.1"\n' > "$DATA/index.toml" printf 'package = { spec = "1", name = "fixture", type = "package" }\n' > "$DATA/pkgs/f/fixture.lua" -# The stub stands in for xlings: an index update rewrites the tree so that it -# requires a newer mcpp than any release, which is what a published floor bump -# looks like to an older client. Every other call succeeds and does nothing. +# The stub stands in for xlings: an owned index is refreshed into a staging +# directory (sync_index with a destination, docs/25 §2), and the copy it +# fetches requires a newer mcpp than any release, which is what a published +# floor bump looks like to an older client. Every other call succeeds and does +# nothing. mkdir -p "$TMP/bin" cat > "$TMP/bin/xlings" < "$DATA/index.toml" - case " \$* " in *" interface "*) echo '{"kind":"result","exitCode":0}' ;; esac + *" sync_index "*) + dest=\$(printf '%s' "\$4" | sed -n 's/.*"dest":"\([^"]*\)".*/\1/p') + mkdir -p "\$dest/pkgs/f" + cp "$DATA/pkgs/f/fixture.lua" "\$dest/pkgs/f/fixture.lua" + printf '[index]\nspec = "1"\nmin_mcpp = "9999.9.9.9"\n' > "\$dest/index.toml" + echo '{"kind":"result","exitCode":0}' ;; + *" interface "*) echo '{"kind":"result","exitCode":0}' ;; esac exit 0 EOF diff --git a/tests/e2e/817_emit_writes_no_xlings_json_under_private_work_dir.sh b/tests/e2e/817_emit_writes_no_xlings_json_under_private_work_dir.sh index 32c37be3f..f6c801b79 100755 --- a/tests/e2e/817_emit_writes_no_xlings_json_under_private_work_dir.sh +++ b/tests/e2e/817_emit_writes_no_xlings_json_under_private_work_dir.sh @@ -39,6 +39,8 @@ source "$(dirname "$0")/_inherit_toolchain.sh" mkdir -p "$OUT/bin" cat > "$OUT/bin/xlings" <<'EOF' #!/usr/bin/env bash +# The handshake mcpp asks of every xlings (docs/25 §5). +[[ "${1:-}" == "interface" && "${2:-}" == "--version" ]] && { echo '{"protocol_version":"1.10","version":"9999.1.1.1","capabilities":["install_packages"],"home_layouts":[2],"features":["index_dir","index_store","upstreams","payload_status","sync_index"]}'; exit 0; } echo "$*" >> "${STUB_LOG:?}" exit 0 EOF diff --git a/tests/e2e/914_an_upstream_home_is_linked_never_written.sh b/tests/e2e/914_an_upstream_home_is_linked_never_written.sh new file mode 100755 index 000000000..80e7652d1 --- /dev/null +++ b/tests/e2e/914_an_upstream_home_is_linked_never_written.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# 914 -- a home with an upstream installs from it by link, and never writes it +# (docs/25 §3). +# +# The upstream is an mcpp home of this user's (prepared from the machine's +# toolchain, as every isolated test home is). Criteria: +# A. The isolated home builds and runs a program; its C runtime payload is a +# link into the upstream's store, and the program's loader is the path in +# the isolated home's own store. +# B. The upstream's store is the same before and after (names, link targets). +# C. `mcpp self doctor` names the upstream; `mcpp self store dedupe` finds +# nothing to link that is not linked already. +# D. An upstream another user could change is refused when the configuration +# is read (a directory this user can write but does not own: /tmp). +# E. A provider home mcpp did not make is refused as `[xlings] home`, and +# the refusal names `upstream`. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +# The upstream: an mcpp home holding the machine's toolchain. +UP="$TMP/upstream" +export MCPP_HOME="$UP" +source "$(dirname "$0")/_inherit_toolchain.sh" +"$MCPP" self env > "$TMP/up.env" 2>&1 || fail "the upstream home does not initialise" "$TMP/up.env" +snapshot() { (cd "$UP/registry/data/xpkgs" && find . -maxdepth 2 -printf '%p %l\n' | sort); } +snapshot > "$TMP/up.before" + +# The home under test: nothing installed, the upstream named. +export MCPP_HOME="$TMP/home" +mkdir -p "$MCPP_HOME" +"$MCPP" self env > /dev/null 2>&1 || true +sed -i.bak "s|^\[xlings\]\$|[xlings]\nupstream = [\"$(host_path "$UP")\"]|" "$MCPP_HOME/config.toml" +grep -q '^upstream = ' "$MCPP_HOME/config.toml" || fail "the test could not name the upstream" "$MCPP_HOME/config.toml" + +# ── A ───────────────────────────────────────────────────────────────────────── +cd "$TMP" +"$MCPP" new hello > new.log 2>&1 || fail "A: mcpp new" new.log +cd hello +"$MCPP" build > build.log 2>&1 || fail "A: the build failed" build.log +BIN=$(find target -path '*/bin/hello' -type f | head -1) +"$BIN" > run.log 2>&1 || fail "A: the program does not run" run.log +GLIBC=$(find "$MCPP_HOME/registry/data/xpkgs/xim-x-glibc" -mindepth 1 -maxdepth 1 | head -1) +[ -L "$GLIBC" ] || fail "A: the C runtime was installed here, not linked from the upstream ($GLIBC)" +case "$(readlink "$GLIBC")" in "$UP"/*|"$(host_path "$UP")"/*) ;; *) fail "A: $GLIBC links elsewhere: $(readlink "$GLIBC")" ;; esac +if command -v readelf > /dev/null; then + readelf -lW "$BIN" | grep -q "interpreter: $MCPP_HOME/registry/data/xpkgs/xim-x-glibc/" \ + || fail "A: the program's loader is not the path in this home's store" <(readelf -lW "$BIN") +fi +echo "ok: A. built against the upstream's runtime, by link" + +# ── B ───────────────────────────────────────────────────────────────────────── +snapshot > "$TMP/up.after" +cmp -s "$TMP/up.before" "$TMP/up.after" || { diff "$TMP/up.before" "$TMP/up.after"; fail "B: the upstream changed"; } +echo "ok: B. the upstream is unchanged" + +# ── C ───────────────────────────────────────────────────────────────────────── +"$MCPP" self doctor > doctor.log 2>&1 || true +grep -q "upstream $(host_path "$UP")" doctor.log || fail "C: doctor does not name the upstream" doctor.log +"$MCPP" self store dedupe > dedupe.log 2>&1 || fail "C: dedupe failed" dedupe.log +grep -q "$MCPP_HOME/registry/data/xpkgs/xim-x-glibc" dedupe.log && fail "C: a linked payload is listed again" dedupe.log +echo "ok: C. doctor names it; dedupe finds the linked payloads linked" + +# ── D ───────────────────────────────────────────────────────────────────────── +if [ "$(id -u)" != 0 ] && [ "$(stat -c %u /tmp 2>/dev/null || stat -f %u /tmp)" = 0 ]; then + cp "$MCPP_HOME/config.toml" "$TMP/config.keep" + sed -i.bak '/^upstream = /d' "$MCPP_HOME/config.toml" + sed -i.bak 's|^\[xlings\]$|[xlings]\nupstream = ["/tmp"]|' "$MCPP_HOME/config.toml" + if "$MCPP" self env > untrusted.log 2>&1; then fail "D: an upstream this user can write but does not own was used" untrusted.log; fi + grep -q "upstream /tmp: not used" untrusted.log || fail "D: the refusal does not name the upstream" untrusted.log + cp "$TMP/config.keep" "$MCPP_HOME/config.toml" + echo "ok: D. an upstream another user could change is refused" +fi + +# ── E ───────────────────────────────────────────────────────────────────────── +FOREIGN="$TMP/foreign-xlings" +mkdir -p "$FOREIGN" +printf '{"index_repos":[{"name":"xim","url":"https://example.invalid/xim.git"}]}\n' > "$FOREIGN/.xlings.json" +cp "$MCPP_HOME/config.toml" "$TMP/config.keep" +sed -i.bak '/^home *=/d' "$MCPP_HOME/config.toml" +sed -i.bak "s|^\[xlings\]\$|[xlings]\nhome = \"$(host_path "$FOREIGN")\"|" "$MCPP_HOME/config.toml" +if "$MCPP" self env > foreign.log 2>&1; then fail "E: a home mcpp did not make was taken as its registry" foreign.log; fi +grep -q 'name it as an upstream' foreign.log || fail "E: the refusal does not name upstream" foreign.log +# A home an earlier mcpp made names its index `mcpp-index`: it is mcpp's. +printf '{"index_repos":[{"name":"mcpp-index","url":"https://example.invalid/mcpp-index.git"}]}\n' > "$FOREIGN/.xlings.json" +"$MCPP" self env > earlier.log 2>&1 || true +if grep -q 'name it as an upstream' earlier.log; then fail "E: a home an earlier mcpp made was refused" earlier.log; fi +cp "$TMP/config.keep" "$MCPP_HOME/config.toml" +echo "ok: E. a home mcpp did not make is not its registry" + +echo "PASS: 914_an_upstream_home_is_linked_never_written" diff --git a/tests/e2e/915_the_system_home_is_maintained_and_read.sh b/tests/e2e/915_the_system_home_is_maintained_and_read.sh new file mode 100755 index 000000000..2242d82eb --- /dev/null +++ b/tests/e2e/915_the_system_home_is_maintained_and_read.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# requires: unix-shell fresh-sandbox +# 915 -- `--system` maintains the machine's system home; users read it as an +# upstream (docs/25 §4). +# +# The system configuration is the test's own ($MCPP_SYSTEM_CONFIG), naming a +# system home under the test's directory. Criteria: +# A. `--system` with a command that is not a maintenance command is refused, +# naming why, and writes nothing. +# B. `--system self init` makes the system home: marked mode = "system", +# every file readable by every user, whatever MCPP_HOME the caller has. +# C. A system home this user cannot write is refused, naming sudo. +# D. A user whose home names nothing reads the system configuration: its +# upstream is the system home, and doctor says so. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap 'chmod -R u+w "$TMP" 2>/dev/null; rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +SYS="$TMP/system-home" +export MCPP_SYSTEM_CONFIG="$TMP/system.toml" +printf '[system]\nhome = "%s"\n\n[xlings]\nupstream = ["%s"]\n' "$(host_path "$SYS")" "$(host_path "$SYS")" > "$MCPP_SYSTEM_CONFIG" +# The machine's toolchain, as every isolated home has it. +export MCPP_HOME="$SYS" +source "$(dirname "$0")/_inherit_toolchain.sh" +export MCPP_HOME="$TMP/not-this-one" +cd "$TMP" + +# ── A ───────────────────────────────────────────────────────────────────────── +if "$MCPP" --system build > a.log 2>&1; then fail "A: --system build was not refused" a.log; fi +grep -q "maintains the machine's system home" a.log || fail "A: the refusal does not say why" a.log +[ ! -e "$SYS/.mcpp-home" ] || fail "A: a refused --system wrote the system home" +echo "ok: A. --system takes maintenance commands only" + +# ── B ───────────────────────────────────────────────────────────────────────── +"$MCPP" --system self init > b.log 2>&1 || fail "B: --system self init failed" b.log +grep -q '^mode = "system"' "$SYS/.mcpp-home" || fail "B: the system home is not marked" "$SYS/.mcpp-home" +[ ! -e "$TMP/not-this-one/registry" ] || fail "B: --system wrote the caller's MCPP_HOME" +unreadable=$(find "$SYS" -newer "$MCPP_SYSTEM_CONFIG" -type f ! -perm -o=r -print | head -5) +[ -z "$unreadable" ] || fail "B: files of the system home other users cannot read: $unreadable" +echo "ok: B. the system home is made, marked and readable by everyone" + +# ── C ───────────────────────────────────────────────────────────────────────── +if [ "$(id -u)" != 0 ]; then + chmod a-w "$SYS" + if "$MCPP" --system index update > c.log 2>&1; then fail "C: an unwritable system home was written" c.log; fi + grep -q "sudo" c.log || fail "C: the refusal does not name sudo" c.log + chmod u+w "$SYS" + echo "ok: C. a system home this user cannot write is refused" +fi + +# ── D ───────────────────────────────────────────────────────────────────────── +export MCPP_HOME="$TMP/user" +mkdir -p "$MCPP_HOME" +"$MCPP" self doctor > d.log 2>&1 || true +grep -q "upstream $(host_path "$SYS")" d.log || fail "D: the user's home does not read the system configuration" d.log +echo "ok: D. a user reads the system home as an upstream" + +echo "PASS: 915_the_system_home_is_maintained_and_read" diff --git a/tests/e2e/916_a_home_from_before_keeps_working.sh b/tests/e2e/916_a_home_from_before_keeps_working.sh new file mode 100755 index 000000000..56688cf29 --- /dev/null +++ b/tests/e2e/916_a_home_from_before_keeps_working.sh @@ -0,0 +1,64 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 916 -- a home an earlier mcpp made keeps working (docs/25 §1). +# +# An earlier mcpp kept the data of its own index under the provider's data +# directory and wrote markers into payloads. Criteria: +# A. The first run moves the index data to /indices/, once, and +# records its place and its store in the provider's configuration. +# B. The index is read from there (doctor lists it), and the copy is not +# read twice when a stale one reappears at the old place. +# C. A payload installed under the old store, with the old marker, is left +# exactly as it was. +set -e +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +export MCPP_HOME="$TMP/home" +source "$(dirname "$0")/_inherit_toolchain.sh" +REG="$MCPP_HOME/registry" +mkdir -p "$REG/data/mcpplibs/pkgs/f" +printf '[index]\nspec = "1"\n' > "$REG/data/mcpplibs/index.toml" +printf 'package = { spec = "1", name = "fixture", type = "package" }\n' > "$REG/data/mcpplibs/pkgs/f/fixture.lua" +cat > "$REG/.xlings.json" < "$OLD/src/legacy.h" +printf 'ok\n' > "$OLD/.mcpp_ok" +before=$(cd "$OLD" && find . -type f -exec sha256sum {} + | sort) + +cd "$TMP" +"$MCPP" self env > a.log 2>&1 || fail "A: the first run failed" a.log +[ -f "$MCPP_HOME/indices/mcpplibs/pkgs/f/fixture.lua" ] || fail "A: the index data did not move" +[ ! -e "$REG/data/mcpplibs" ] || fail "A: the index data is still at the old place" +python3 - "$REG/.xlings.json" "$MCPP_HOME" <<'PY' || fail "A: the configuration does not record the index's place" "$REG/.xlings.json" +import json, os, sys +doc = json.load(open(sys.argv[1])) +e = next(r for r in doc["index_repos"] if r["name"] == "mcpplibs") +home = sys.argv[2] +assert os.path.normpath(e["dir"]) == os.path.normpath(os.path.join(home, "indices", "mcpplibs")), e +assert os.path.normpath(e["store"]) == os.path.normpath(os.path.join(home, "xpkgs")), e +PY +echo "ok: A. the index moved once and its place is recorded" + +"$MCPP" self doctor > b.log 2>&1 || true +[ "$(grep -c "index 'mcpplibs'" b.log)" = 1 ] || fail "B: doctor does not list the index once" b.log +mkdir -p "$REG/data/mcpplibs/pkgs" +"$MCPP" self doctor > b2.log 2>&1 || true +[ "$(grep -c "index 'mcpplibs'" b2.log)" = 1 ] || fail "B: a stale copy at the old place is read as a second index" b2.log +echo "ok: B. the index is read where it is, once" + +after=$(cd "$OLD" && find . -type f -exec sha256sum {} + | sort) +[ "$before" = "$after" ] || fail "C: a payload of the old store changed" +echo "ok: C. the old store's payload is as it was" + +echo "PASS: 916_a_home_from_before_keeps_working" diff --git a/tests/e2e/917_a_long_home_on_windows.sh b/tests/e2e/917_a_long_home_on_windows.sh new file mode 100755 index 000000000..671c9fd97 --- /dev/null +++ b/tests/e2e/917_a_long_home_on_windows.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +# requires: windows +# 917 -- Windows long paths (docs/25 §6). +# +# Criteria: +# A. mcpp.exe declares itself long-path aware in its application manifest. +# B. With long paths on (LongPathsEnabled = 1), an MCPP_HOME 157 characters +# long builds and runs a program; with them off, mcpp reports the path +# budget as a degradation naming both remedies. +set -e +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +# ── A ───────────────────────────────────────────────────────────────────────── +grep -a -q 'true' "$MCPP" \ + || fail "A: mcpp.exe does not declare longPathAware" +echo "ok: A. mcpp.exe is long-path aware" + +# ── B ───────────────────────────────────────────────────────────────────────── +LONG="$TMP" +while [ ${#LONG} -lt 150 ]; do LONG="$LONG/deep-directory"; done +LONG="${LONG:0:157}" +export MCPP_HOME="$LONG" +mkdir -p "$MCPP_HOME" +enabled=$(reg query 'HKLM\SYSTEM\CurrentControlSet\Control\FileSystem' //v LongPathsEnabled 2>/dev/null | grep -o '0x[0-9a-f]*' || echo 0x0) +if [ "$enabled" = "0x1" ]; then + source "$(dirname "$0")/_inherit_toolchain.sh" + cd "$TMP" + "$MCPP" new hello > new.log 2>&1 || fail "B: mcpp new" new.log + cd hello + "$MCPP" build > build.log 2>&1 || fail "B: a build under a 157-character MCPP_HOME failed" build.log + "$MCPP" run > run.log 2>&1 || fail "B: the program does not run" run.log + echo "ok: B. a 157-character MCPP_HOME builds with long paths on" +else + "$MCPP" self env > env.log 2>&1 || true + grep -q "Windows long paths are off" env.log || fail "B: the path budget is not reported" env.log + grep -q "shorter MCPP_HOME" env.log || fail "B: the report does not name the remedies" env.log + echo "ok: B. with long paths off, the path budget is reported" +fi + +echo "PASS: 917_a_long_home_on_windows" diff --git a/tests/e2e/_inherit_toolchain.sh b/tests/e2e/_inherit_toolchain.sh index ba3eb9bd2..ef37cd37f 100644 --- a/tests/e2e/_inherit_toolchain.sh +++ b/tests/e2e/_inherit_toolchain.sh @@ -33,7 +33,9 @@ fi # existing payload remains the first shape of #293. link_xpkg_payloads() { local source_dir="$1" - local target_dir="$MCPP_HOME/registry/data/xpkgs" + # The provider's store by default; the store of mcpp's own indices when + # named (docs/25 §1). + local target_dir="${2:-$MCPP_HOME/registry/data/xpkgs}" [[ -d "$source_dir" ]] || return 0 mkdir -p "$target_dir" @@ -61,6 +63,7 @@ link_xpkg_payloads() { if [[ -d "$USER_MCPP/registry/data/xpkgs" || -d "$HOME/.xlings/data/xpkgs" || ( -n "${USERPROFILE:-}" && -d "$USERPROFILE/.xlings/data/xpkgs" ) ]]; then mkdir -p "$MCPP_HOME/registry/data" link_xpkg_payloads "$USER_MCPP/registry/data/xpkgs" + link_xpkg_payloads "$USER_MCPP/xpkgs" "$MCPP_HOME/xpkgs" link_xpkg_payloads "$HOME/.xlings/data/xpkgs" if [[ -n "${USERPROFILE:-}" ]]; then link_xpkg_payloads "$USERPROFILE/.xlings/data/xpkgs" diff --git a/tests/scripts/test_check_provider_boundary.py b/tests/scripts/test_check_provider_boundary.py new file mode 100644 index 000000000..a56a293d3 --- /dev/null +++ b/tests/scripts/test_check_provider_boundary.py @@ -0,0 +1,72 @@ +#!/usr/bin/env python3 +"""Fixture tests for .github/tools/check_provider_boundary.py. + +The check passes on the repository's src/, and on a tree that names the +provider's layout only inside the adapter, in comments, or with a stated +exception; it fails on an import of the adapter outside the composition root, +on a layout spelling in code, and on a payload canonicalised or removed. +""" + +from __future__ import annotations + +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[2] +SCRIPT = REPO_ROOT / ".github" / "tools" / "check_provider_boundary.py" + + +def run(root: Path) -> subprocess.CompletedProcess: + return subprocess.run([sys.executable, str(SCRIPT), "--root", str(root)], + capture_output=True, text=True, check=False) + + +def tree(files: dict[str, str]) -> tempfile.TemporaryDirectory: + d = tempfile.TemporaryDirectory() + for rel, text in files.items(): + p = Path(d.name) / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(text, encoding="utf-8") + return d + + +CLEAN = { + "src/provider/select.cppm": "export module mcpp.provider.select;\nimport mcpp.provider.xlings;\n", + "src/provider/xlings/xlings.cppm": 'auto p = home / "xpkgs" / "xim-x-gcc";\n', + "src/build/a.cppm": ('// the payload under xpkgs/xim-x-gcc is the provider\'s\n' + 'import mcpp.provider.select;\n' + 'auto q = std::filesystem::canonical(sourceDir);\n' + 'auto r = dir / "subos"; // provider-boundary-ok: a fixture\'s reason\n'), +} + + +class ProviderBoundary(unittest.TestCase): + def test_the_repository_is_clean(self) -> None: + r = run(REPO_ROOT) + self.assertEqual(r.returncode, 0, r.stdout) + + def test_the_adapter_comments_and_stated_exceptions_pass(self) -> None: + with tree(CLEAN) as d: + r = run(Path(d)) + self.assertEqual(r.returncode, 0, r.stdout) + + def test_each_crossing_is_found(self) -> None: + cases = { + "import": "import mcpp.provider.xlings.subos_info;\n", + "layout": 'auto p = home / "xpkgs";\n', + "marker": 'auto ok = dir / ".mcpp_ok";\n', + "payload": "auto p = std::filesystem::canonical(payloadDir);\n", + "removal": "std::filesystem::remove_all(payload);\n", + } + for name, line in cases.items(): + with self.subTest(case=name), tree({**CLEAN, "src/build/b.cppm": line}) as d: + r = run(Path(d)) + self.assertEqual(r.returncode, 1, r.stdout) + self.assertIn("src/build/b.cppm:1:", r.stdout) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/test_build_resources.cpp b/tests/unit/test_build_resources.cpp index 2fe1ff570..d3f6e7682 100644 --- a/tests/unit/test_build_resources.cpp +++ b/tests/unit/test_build_resources.cpp @@ -332,6 +332,21 @@ TEST(BuildResources, AManifestOnlyScriptCarriesTheManifestAndNothingElse) { std::string::npos); } +// An application manifest states what was asked of it and nothing else; a +// host tool's states both (design 2026-10-16 §11). +TEST(BuildResources, AnApplicationManifestStatesWhatWasAskedOfIt) { + const std::string codePage = "UTF-8"; + const std::string longPaths = "true"; + const auto both = res::application_manifest(true, true); + EXPECT_NE(both.find(codePage), std::string::npos); + EXPECT_NE(both.find(longPaths), std::string::npos); + const auto onlyLong = res::application_manifest(false, true); + EXPECT_EQ(onlyLong.find("activeCodePage"), std::string::npos); + EXPECT_NE(onlyLong.find(longPaths), std::string::npos); + EXPECT_EQ(res::application_manifest(true, false).find("longPathAware"), std::string::npos); + EXPECT_EQ(res::utf8_code_page_manifest(), both); +} + TEST(BuildResources, ScanNamesWhatItCouldNotResolve) { TempDir d; auto rc = d.write("app.rc", "1 ICON APP_ICON\n"); diff --git a/tests/unit/test_fallback_legacy_dirs.cpp b/tests/unit/test_fallback_legacy_dirs.cpp index f8c7871f0..b38868241 100644 --- a/tests/unit/test_fallback_legacy_dirs.cpp +++ b/tests/unit/test_fallback_legacy_dirs.cpp @@ -1,7 +1,7 @@ #include import std; -import mcpp.fallback.legacy_dirs; +import mcpp.provider.xlings.legacy_dirs; // The legacy install-dir scan must not cross namespaces. // @@ -35,7 +35,7 @@ TEST(FallbackLegacyDirs, DoesNotReturnAnotherNamespacesDirectory) { auto base = make_xpkgs({"compat-x-imgui"}); // Asking for ocornut:imgui. Only compat's directory exists, and it is NOT // an answer to this question. - auto hit = mcpp::fallback::scan_legacy_install_dirs( + auto hit = mcpp::provider::xlings::scan_legacy_install_dirs( base, "ocornut.imgui", "imgui", {"ocornut", "mcpplibs"}); EXPECT_FALSE(hit.has_value()) << "returned '" << hit.value_or("") << "' for ocornut:imgui"; @@ -44,7 +44,7 @@ TEST(FallbackLegacyDirs, DoesNotReturnAnotherNamespacesDirectory) { TEST(FallbackLegacyDirs, StillFindsOwnNamespaceShortNameLayout) { auto base = make_xpkgs({"compat-x-imgui", "ocornut-x-imgui"}); - auto hit = mcpp::fallback::scan_legacy_install_dirs( + auto hit = mcpp::provider::xlings::scan_legacy_install_dirs( base, "ocornut.imgui", "imgui", {"ocornut", "mcpplibs"}); ASSERT_TRUE(hit.has_value()); EXPECT_EQ(*hit, "ocornut-x-imgui"); @@ -54,7 +54,7 @@ TEST(FallbackLegacyDirs, StillFindsOwnNamespaceShortNameLayout) { TEST(FallbackLegacyDirs, StillFindsIndexPrefixedLegacyLayout) { // The old layout this arm exists for: -x-. auto base = make_xpkgs({"mcpplibs-x-tinyhttps"}); - auto hit = mcpp::fallback::scan_legacy_install_dirs( + auto hit = mcpp::provider::xlings::scan_legacy_install_dirs( base, "mcpplibs.tinyhttps", "tinyhttps", {"mcpplibs", "mcpplibs"}); ASSERT_TRUE(hit.has_value()); EXPECT_EQ(*hit, "mcpplibs-x-tinyhttps"); @@ -65,7 +65,7 @@ TEST(FallbackLegacyDirs, QualifiedSuffixNeedsNoPrefixAllowance) { // `-x-.` carries the namespace in the suffix itself, so it is // accepted whatever the prefix is — that is the legacy FQN layout. auto base = make_xpkgs({"someindex-x-ocornut.imgui"}); - auto hit = mcpp::fallback::scan_legacy_install_dirs( + auto hit = mcpp::provider::xlings::scan_legacy_install_dirs( base, "ocornut.imgui", "imgui", {"ocornut"}); ASSERT_TRUE(hit.has_value()); EXPECT_EQ(*hit, "someindex-x-ocornut.imgui"); diff --git a/tests/unit/test_fetcher_install_path.cpp b/tests/unit/test_fetcher_install_path.cpp new file mode 100644 index 000000000..435d2bc9a --- /dev/null +++ b/tests/unit/test_fetcher_install_path.cpp @@ -0,0 +1,60 @@ +// Fetcher::install_path -- where an installed package of mcpp's indices is +// (design 2026-10-16 §5): the store of the index that serves it, then the +// provider home's, which holds what was installed before that index had a +// store of its own. + +#include + +import std; +import mcpp.config; +import mcpp.pm.package_fetcher; + +namespace { + +struct TempDir { + std::filesystem::path path = std::filesystem::temp_directory_path() + / std::format("mcpp-install-path-{}", std::chrono::steady_clock::now().time_since_epoch().count()); + TempDir() { std::filesystem::create_directories(path); } + ~TempDir() { + std::error_code ec; + std::filesystem::remove_all(path, ec); + } +}; + +// A home whose mcpplibs index keeps its payloads in /xpkgs. +mcpp::config::GlobalConfig home_at(const std::filesystem::path& home) { + mcpp::config::GlobalConfig cfg{}; + cfg.mcppHome = home; + cfg.registryDir = home / "registry"; + cfg.defaultIndex = "mcpplibs"; + std::filesystem::create_directories(cfg.registryDir); + std::ofstream(cfg.registryDir / ".xlings.json") << std::format( + R"({{"index_repos":[{{"name":"mcpplibs","url":"u","dir":"{}","store":"{}"}}]}})", + (home / "indices" / "mcpplibs").generic_string(), (home / "xpkgs").generic_string()); + return cfg; +} + +TEST(FetcherInstallPath, APackageIsFoundInItsIndexStore) { + TempDir temp; + auto cfg = home_at(temp.path); + const auto verdir = temp.path / "xpkgs" / "compat-x-gadget" / "1.0.0"; + std::filesystem::create_directories(verdir); + mcpp::pm::Fetcher f(cfg); + const auto found = f.install_path("compat", "gadget", "1.0.0"); + ASSERT_TRUE(found); + EXPECT_EQ(found->lexically_normal(), verdir.lexically_normal()); +} + +TEST(FetcherInstallPath, APackageFromBeforeIsFoundInTheHomesStore) { + TempDir temp; + auto cfg = home_at(temp.path); + const auto verdir = cfg.registryDir / "data" / "xpkgs" / "compat-x-gadget" / "1.0.0"; + std::filesystem::create_directories(verdir); + mcpp::pm::Fetcher f(cfg); + const auto found = f.install_path("compat", "gadget", "1.0.0"); + ASSERT_TRUE(found); + EXPECT_EQ(found->lexically_normal(), verdir.lexically_normal()); + EXPECT_FALSE(f.install_path("compat", "gadget", "2.0.0")); +} + +} // namespace diff --git a/tests/unit/test_freestanding.cpp b/tests/unit/test_freestanding.cpp index df2f898c7..f87bfe792 100644 --- a/tests/unit/test_freestanding.cpp +++ b/tests/unit/test_freestanding.cpp @@ -5,7 +5,7 @@ import mcpp.freestanding.target; import mcpp.freestanding.linkline; import mcpp.freestanding.runner; import mcpp.toolchain.triple; -import mcpp.xlings; +import mcpp.provider.xlings; import mcpp.build.build_program; using namespace mcpp::freestanding; @@ -160,7 +160,7 @@ TEST(FreestandingRunner, TheMissingRunnerMessageNamesTheKeyAndTheTriple) { // to store internals mcpp is free to change. TEST(XpkgRef, ParsesEverySpellingAManifestMayWrite) { - using mcpp::xlings::paths::parse_xpkg_ref; + using mcpp::provider::xlings::paths::parse_xpkg_ref; auto full = parse_xpkg_ref("xim:picolibc-riscv@1.8.12"); EXPECT_EQ(full.ns, "xim"); @@ -183,7 +183,7 @@ TEST(XpkgRef, ParsesEverySpellingAManifestMayWrite) { } TEST(XpkgPayload, APinnedRefResolvesToThatVersionOrToNothing) { - namespace xp = mcpp::xlings::paths; + namespace xp = mcpp::provider::xlings::paths; auto base = std::filesystem::temp_directory_path() / std::format("mcpp-xpkg-test-{}", ::getpid()); std::filesystem::remove_all(base); @@ -203,7 +203,7 @@ TEST(XpkgPayload, APinnedRefResolvesToThatVersionOrToNothing) { } TEST(XpkgPayload, AnUnpinnedRefTakesTheHighestVersionNumerically) { - namespace xp = mcpp::xlings::paths; + namespace xp = mcpp::provider::xlings::paths; auto base = std::filesystem::temp_directory_path() / std::format("mcpp-xpkg-test2-{}", ::getpid()); std::filesystem::remove_all(base); @@ -226,7 +226,7 @@ TEST(XpkgPayload, AnUnpinnedRefTakesTheHighestVersionNumerically) { // nothing was installed. That is what made a rule package unable to state a // floor: it declared `>=8.5.0`, got it, and `xpkg_dir` answered "". TEST(XpkgPayload, AConstrainedRefTakesTheHighestInstalledVersionSatisfyingIt) { - namespace xp = mcpp::xlings::paths; + namespace xp = mcpp::provider::xlings::paths; auto base = std::filesystem::temp_directory_path() / std::format("mcpp-xpkg-test3-{}", ::getpid()); std::filesystem::remove_all(base); @@ -261,7 +261,7 @@ TEST(XpkgPayload, AConstrainedRefTakesTheHighestInstalledVersionSatisfyingIt) { // An installed version whose directory name is not a SemVer is addressable by // its exact spelling. `8.0.RC1` is a real CANN version. TEST(XpkgPayload, AnUnparseableVersionIsStillAddressableExactly) { - namespace xp = mcpp::xlings::paths; + namespace xp = mcpp::provider::xlings::paths; auto base = std::filesystem::temp_directory_path() / std::format("mcpp-xpkg-test4-{}", ::getpid()); std::filesystem::remove_all(base); @@ -289,7 +289,7 @@ TEST(XpkgPayload, AnUnparseableVersionIsStillAddressableExactly) { // three or more are written-prefix equality, and a fourth segment is an // ordinary version. The lookup answered "" for every one of these before. TEST(XpkgPayload, ABareVersionSelectsWhatXlingsSelected) { - namespace xp = mcpp::xlings::paths; + namespace xp = mcpp::provider::xlings::paths; auto base = std::filesystem::temp_directory_path() / std::format("mcpp-xpkg-test5-{}", ::getpid()); std::filesystem::remove_all(base); @@ -317,7 +317,7 @@ TEST(XpkgPayload, ABareVersionSelectsWhatXlingsSelected) { // field when present, 0 for a record that predates it, and nothing at all when // there is no record -- which is the case mcpp judges by its marker alone. TEST(XpkgPayload, InstalledRevisionReadsTheXlingsRecord) { - namespace xp = mcpp::xlings::paths; + namespace xp = mcpp::provider::xlings::paths; auto dir = std::filesystem::temp_directory_path() / std::format("mcpp-xpkg-rev-{}", ::getpid()); std::filesystem::remove_all(dir); diff --git a/tests/unit/test_index_snapshot.cpp b/tests/unit/test_index_snapshot.cpp deleted file mode 100644 index 11b3210ef..000000000 --- a/tests/unit/test_index_snapshot.cpp +++ /dev/null @@ -1,227 +0,0 @@ -// The invariant this file exists for: -// -// An index-side change must never take mcpp from "works" to "does not work". -// -// A published index can raise its client-version floor (index.toml min_mcpp). -// `xlings update` rewrites the tree in place, so before the guard existed a -// floor bump replaced a readable index with an unreadable one and left no way -// back — the refresh was the thing that broke the machine. -// -// The 2026-07-08 index design specified this behaviour ("staged refresh keeps -// the last compatible snapshot") and it was never implemented; the floor was -// checked in exactly one place, the descriptor reader. There was no test that -// could have noticed, because the only coverage was of the pure predicate. -// These are that missing test. - -#include - -import std; -import mcpp.pm.index_snapshot; -import mcpp.pm.index_contract; -import mcpp.version; - -using namespace mcpp::pm::index_snapshot; - -namespace { - -struct Tmp { - std::filesystem::path path; - Tmp() { - path = std::filesystem::temp_directory_path() - / std::format("mcpp_idx_snap_{}", std::random_device{}()); - std::filesystem::create_directories(path); - } - ~Tmp() { std::error_code ec; std::filesystem::remove_all(path, ec); } -}; - -void writeFile(const std::filesystem::path& p, std::string_view body) { - std::filesystem::create_directories(p.parent_path()); - std::ofstream(p) << body; -} - -std::string readFile(const std::filesystem::path& p) { - std::ifstream is(p); - return std::string((std::istreambuf_iterator(is)), {}); -} - -// A minimal index tree: pkgs/ makes it an index, index.toml carries the floor, -// .xlings-index-version is the snapshot identity. -void makeIndex(const std::filesystem::path& dir, - std::string_view minMcpp, - std::string_view version, - std::string_view marker) { - writeFile(dir / "pkgs" / "z" / "zlib.lua", std::format("-- {}\n", marker)); - writeFile(dir / ".xlings-index-version", version); - if (minMcpp.empty()) { - std::error_code ec; - std::filesystem::remove(dir / "index.toml", ec); - } else { - writeFile(dir / "index.toml", - std::format("[index]\nspec = \"1\"\nmin_mcpp = \"{}\"\n", minMcpp)); - } -} - -constexpr std::string_view kTooNew = "9999.9.9.9"; // no mcpp satisfies this - -} // namespace - -TEST(IndexSnapshot, DetectsIndexTreesAndIgnoresTheSnapshotStore) { - Tmp t; - auto data = t.path / "data"; - makeIndex(data / "mcpplibs", "", "v1", "a"); - makeIndex(data / "xim-pkgindex", "", "v1", "b"); - std::filesystem::create_directories(data / "not-an-index"); // no pkgs/ - std::filesystem::create_directories(data / ".index-snapshots" / "x" / "pkgs"); - - auto dirs = index_dirs(data); - ASSERT_EQ(dirs.size(), 2u); - EXPECT_EQ(dirs[0].filename(), "mcpplibs"); - EXPECT_EQ(dirs[1].filename(), "xim-pkgindex"); -} - -TEST(IndexSnapshot, ArchiveRefusesAnUnusableTree) { - Tmp t; - auto data = t.path / "data"; - auto idx = data / "mcpplibs"; - makeIndex(idx, kTooNew, "v1", "unusable"); - - // A snapshot exists to be restored; archiving one nobody can read would - // only give the recovery path a useless candidate. - EXPECT_FALSE(archive(data, idx)); - EXPECT_TRUE(list_snapshots(data, idx).empty()); -} - -TEST(IndexSnapshot, ArchiveAndRestoreRoundTrip) { - Tmp t; - auto data = t.path / "data"; - auto idx = data / "mcpplibs"; - makeIndex(idx, "0.0.85", "v1", "original"); - - ASSERT_TRUE(archive(data, idx)); - auto snaps = list_snapshots(data, idx); - ASSERT_EQ(snaps.size(), 1u); - - makeIndex(idx, "0.0.85", "v2", "replaced"); - EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("replaced"), - std::string::npos); - - ASSERT_TRUE(restore(snaps[0], idx)); - EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("original"), - std::string::npos); -} - -// THE load-bearing test. A refresh that raises the floor beyond this binary -// must leave the machine working. -TEST(IndexSnapshot, RefreshThatBreaksTheIndexIsRolledBack) { - Tmp t; - auto data = t.path / "data"; - auto idx = data / "mcpplibs"; - makeIndex(idx, "0.0.85", "good", "usable-tree"); - - GuardOutcome out; - int rc = guarded_refresh(data, [&] { - // This is what `xlings update` does: rewrite the tree in place, with a - // floor this binary cannot satisfy. - makeIndex(idx, kTooNew, "too-new", "unusable-tree"); - return 0; - }, out); - - EXPECT_EQ(rc, 0); - ASSERT_EQ(out.rolledBack.size(), 1u) << "the refresh made the index unusable " - "and the guard did not roll it back"; - EXPECT_TRUE(out.stillUnusable.empty()); - - // The machine still works: the tree on disk is readable and is the old one. - EXPECT_TRUE(mcpp::pm::index_usable(idx)); - EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("usable-tree"), - std::string::npos); -} - -// The mirror case: a refresh that keeps the index readable must be left alone, -// and must silently become the new known-good snapshot. -TEST(IndexSnapshot, NormalRefreshIsUntouchedAndBecomesTheNewSnapshot) { - Tmp t; - auto data = t.path / "data"; - auto idx = data / "mcpplibs"; - makeIndex(idx, "0.0.85", "v1", "first"); - - GuardOutcome out; - guarded_refresh(data, [&] { - makeIndex(idx, "0.0.85", "v2", "second"); - return 0; - }, out); - - EXPECT_FALSE(out.degraded()) << "a healthy refresh must not report anything"; - EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("second"), - std::string::npos); - // Both the pre- and post-refresh trees are now archived, so the next - // refresh has somewhere to fall back to. - EXPECT_EQ(list_snapshots(data, idx).size(), 2u); -} - -// Already-unusable on entry: there is nothing to "roll back" to, but a local -// snapshot from before the floor moved is still the right answer. -TEST(IndexSnapshot, RecoversFromLocalHistoryWhenAlreadyUnusable) { - Tmp t; - auto data = t.path / "data"; - auto idx = data / "mcpplibs"; - - // A good tree was seen at some point in the past... - makeIndex(idx, "0.0.85", "old-good", "known-good"); - ASSERT_TRUE(archive(data, idx)); - // ...and the current tree is already too new (e.g. mcpp was downgraded, or - // the guard was introduced after the damage was done). - makeIndex(idx, kTooNew, "too-new", "unusable"); - - GuardOutcome out; - guarded_refresh(data, [&] { return 0; }, out); // refresh changes nothing - - ASSERT_EQ(out.recovered.size(), 1u); - EXPECT_TRUE(out.rolledBack.empty()); - EXPECT_TRUE(mcpp::pm::index_usable(idx)); - EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("known-good"), - std::string::npos); -} - -// No local history and a too-new tree is the one case nothing can rescue. It -// must be reported honestly rather than silently left looking fine. -TEST(IndexSnapshot, ReportsStillUnusableWhenNoSnapshotCanHelp) { - Tmp t; - auto data = t.path / "data"; - auto idx = data / "mcpplibs"; - makeIndex(idx, kTooNew, "too-new", "unusable"); - - GuardOutcome out; - guarded_refresh(data, [&] { return 0; }, out); - - ASSERT_EQ(out.stillUnusable.size(), 1u); - EXPECT_TRUE(out.rolledBack.empty()); - EXPECT_TRUE(out.recovered.empty()); -} - -TEST(IndexSnapshot, PruneKeepsTheNewest) { - Tmp t; - auto data = t.path / "data"; - auto idx = data / "mcpplibs"; - for (int i = 0; i < 4; ++i) { - makeIndex(idx, "0.0.85", std::format("v{}", i), std::format("gen{}", i)); - ASSERT_TRUE(archive(data, idx)); - } - ASSERT_EQ(list_snapshots(data, idx).size(), 4u); - prune(data, idx, /*keep=*/2); - EXPECT_EQ(list_snapshots(data, idx).size(), 2u); -} - -// A tree with no index.toml declares no contract, so it is usable by anyone. -// Third-party indexes rely on this and must never be rolled back. -TEST(IndexSnapshot, TreeWithoutAContractIsAlwaysUsable) { - Tmp t; - auto data = t.path / "data"; - auto idx = data / "third-party"; - makeIndex(idx, "", "v1", "no-contract"); - EXPECT_TRUE(mcpp::pm::index_usable(idx)); - - GuardOutcome out; - guarded_refresh(data, [&] { return 0; }, out); - EXPECT_FALSE(out.degraded()); -} diff --git a/tests/unit/test_install_integrity.cpp b/tests/unit/test_install_integrity.cpp deleted file mode 100644 index 0c99b21a9..000000000 --- a/tests/unit/test_install_integrity.cpp +++ /dev/null @@ -1,241 +0,0 @@ -#include - -import std; -import mcpp.fallback.install_integrity; - -// Regression coverage for InstallStash — the rollback-safe replacement for a -// bare clean_incomplete_install() on the resolve/install path. -// -// Background: a content-complete *legacy* package (no .mcpp_ok marker) used to -// be deleted outright before a reinstall. If the reinstall then failed, the -// working package was gone for good (the xim-x-zlib / clang `libz.so.1` 127 -// regression). InstallStash renames it aside and restores it on failure. - -namespace { - -namespace fs = std::filesystem; - -fs::path make_tempdir(std::string_view name) { - auto base = fs::temp_directory_path() - / std::format("{}-{}", name, - std::chrono::steady_clock::now() - .time_since_epoch().count()); - fs::create_directories(base); - return base; -} - -void write_file(const fs::path& p, std::string_view content) { - fs::create_directories(p.parent_path()); - std::ofstream os(p); - os << content; -} - -// A directory that looks_complete_legacy(): top-level lib/ with a payload. -void make_legacy_pkg(const fs::path& verdir, std::string_view marker) { - write_file(verdir / "lib" / "libz.so.1", marker); -} - -bool has_ok_marker(const fs::path& verdir) { - return fs::exists(verdir / ".mcpp_ok"); -} - -// Read the first line and CLOSE the handle before returning. On Windows an -// open file handle blocks fs::remove_all() ("being used by another process"), -// so readers must not outlive the read. -std::string read_first_line(const fs::path& p) { - std::ifstream in(p); - std::string line; - std::getline(in, line); - return line; -} - -} // namespace - -// Reinstall fails (no commit) → a content-complete legacy package is rolled -// back AND adopted (marker written) so the next resolve trusts it. -TEST(InstallIntegrityStash, RestoresLegacyPackageOnFailedReinstall) { - auto root = make_tempdir("mcpp-stash-restore"); - auto verdir = root / "compat-x-compat.zlib" / "1.3.1"; - make_legacy_pkg(verdir, "ORIGINAL"); - - { - mcpp::fallback::InstallStash stash(verdir); - EXPECT_TRUE(stash.stashed()); - // verdir was moved aside, leaving the live path clean for a reinstall. - EXPECT_FALSE(fs::exists(verdir)); - // ... reinstall attempt fails here: no commit() ... - } // destructor runs - - // Original content is back, and it is now marked complete. - ASSERT_TRUE(fs::exists(verdir / "lib" / "libz.so.1")); - EXPECT_EQ(read_first_line(verdir / "lib" / "libz.so.1"), "ORIGINAL"); - EXPECT_TRUE(has_ok_marker(verdir)); - EXPECT_FALSE(fs::exists(fs::path(verdir.string() + ".mcpp-stash"))); - - fs::remove_all(root); -} - -// commit() (reinstall produced a good install) → backup is dropped, the live -// directory is left exactly as the reinstall produced it. -TEST(InstallIntegrityStash, CommitDropsBackupAndKeepsNewInstall) { - auto root = make_tempdir("mcpp-stash-commit"); - auto verdir = root / "compat-x-compat.zlib" / "1.3.1"; - make_legacy_pkg(verdir, "ORIGINAL"); - - { - mcpp::fallback::InstallStash stash(verdir); - ASSERT_TRUE(stash.stashed()); - // Simulate a successful reinstall landing fresh content. - write_file(verdir / "lib" / "libz.so.1", "REINSTALLED"); - stash.commit(); - } - - ASSERT_TRUE(fs::exists(verdir / "lib" / "libz.so.1")); - EXPECT_EQ(read_first_line(verdir / "lib" / "libz.so.1"), "REINSTALLED"); - EXPECT_FALSE(fs::exists(fs::path(verdir.string() + ".mcpp-stash"))); - - fs::remove_all(root); -} - -// No commit, but the reinstall DID produce a complete install (marker present) -// → keep the new install, discard the stash (don't clobber a good result). -TEST(InstallIntegrityStash, KeepsNewCompleteInstallWhenUncommitted) { - auto root = make_tempdir("mcpp-stash-newcomplete"); - auto verdir = root / "compat-x-compat.zlib" / "1.3.1"; - make_legacy_pkg(verdir, "ORIGINAL"); - - { - mcpp::fallback::InstallStash stash(verdir); - ASSERT_TRUE(stash.stashed()); - // Reinstall produced a marked-complete install but commit() was missed. - write_file(verdir / "lib" / "libz.so.1", "REINSTALLED"); - mcpp::fallback::mark_install_complete(verdir); - } // destructor: live dir is complete → keep it - - EXPECT_EQ(read_first_line(verdir / "lib" / "libz.so.1"), "REINSTALLED"); - EXPECT_FALSE(fs::exists(fs::path(verdir.string() + ".mcpp-stash"))); - - fs::remove_all(root); -} - -// Genuine half-extracted residue (NOT looks_complete_legacy) and a failed -// reinstall → discard it, matching the historical delete semantics. -TEST(InstallIntegrityStash, DiscardsNonLegacyResidueOnFailure) { - auto root = make_tempdir("mcpp-stash-residue"); - auto verdir = root / "compat-x-compat.zlib" / "1.3.1"; - write_file(verdir / "partial.tmp", "half-extracted"); // no bin/lib/... → not legacy-complete - - { - mcpp::fallback::InstallStash stash(verdir); - ASSERT_TRUE(stash.stashed()); - // reinstall fails, no commit - } - - EXPECT_FALSE(fs::exists(verdir)); // residue discarded, not restored - EXPECT_FALSE(fs::exists(fs::path(verdir.string() + ".mcpp-stash"))); - - fs::remove_all(root); -} - -// A directory that already has the .mcpp_ok marker must not be stashed. -TEST(InstallIntegrityStash, NoopWhenAlreadyComplete) { - auto root = make_tempdir("mcpp-stash-complete"); - auto verdir = root / "compat-x-compat.zlib" / "1.3.1"; - make_legacy_pkg(verdir, "ORIGINAL"); - mcpp::fallback::mark_install_complete(verdir); - - { - mcpp::fallback::InstallStash stash(verdir); - EXPECT_FALSE(stash.stashed()); - EXPECT_TRUE(fs::exists(verdir / "lib" / "libz.so.1")); // untouched - } - - EXPECT_TRUE(fs::exists(verdir / "lib" / "libz.so.1")); - EXPECT_TRUE(has_ok_marker(verdir)); - - fs::remove_all(root); -} - -// ─── mcpp#533: the marker is only as good as the evidence behind it ───────── -// -// `.mcpp_ok` used to be written from `exitCode == 0 && exists(verdir)` — the -// installer's own report, plus a directory the installer creates before doing -// any work. A package-identity collision made xlings skip the descriptor's -// `install()` while still reporting success; mcpp stamped the result complete, -// and because the stamp is what `is_install_complete` reads, the wrong state -// then survived every later build. -// -// The observed directory held exactly three entries — `.mcpp_ok`, -// `.xpkg-install.json`, `mcpp_generated/` — and nothing else. That signature -// needs no knowledge of what the descriptor was supposed to build. - -namespace { - -// Everything mcpp and xlings write into a version directory themselves. -// Written out rather than taken from the code under test: this list IS the -// claim, and sharing it would let both sides drift together. -void make_self_written_only(const fs::path& verdir) { - write_file(verdir / ".mcpp_ok", "1\n"); - write_file(verdir / ".xpkg-install.json", "{}\n"); - write_file(verdir / "mcpp_generated" / "libdrm" / "lib" / "libdrm.so", ""); -} - -} // namespace - -TEST(InstallEvidence, ADirectoryHoldingOnlyOurOwnFilesIsNotAPayload) { - auto dir = make_tempdir("mcpp-evidence-self"); - auto verdir = dir / "compat-x-libdrm" / "2.4.123"; - make_self_written_only(verdir); - - EXPECT_FALSE(mcpp::fallback::payload_is_substantive(verdir)); - fs::remove_all(dir); -} - -TEST(InstallEvidence, OneEntryTheInstallWroteIsEnough) { - auto dir = make_tempdir("mcpp-evidence-real"); - auto verdir = dir / "compat-x-libdrm" / "2.4.123"; - make_self_written_only(verdir); - // …plus the tree the descriptor's install() was supposed to build. - write_file(verdir / "include" / "drm.h", "#pragma once\n"); - - EXPECT_TRUE(mcpp::fallback::payload_is_substantive(verdir)); - fs::remove_all(dir); -} - -// THE HALF THAT MAKES THE UPGRADE SEAMLESS. `is_install_complete` is -// marker-only by design, so a store poisoned before this check existed carries -// a `.mcpp_ok` that short-circuits forever. Guarding only the WRITE site would -// help users who have not hit the bug and do nothing for the ones who filed -// it. Nobody should have to be told which directory to delete. -TEST(InstallEvidence, AStaleMarkerOverAnEmptyPayloadDoesNotCountAsComplete) { - auto dir = make_tempdir("mcpp-evidence-heal"); - auto verdir = dir / "compat-x-libdrm" / "2.4.123"; - make_self_written_only(verdir); - ASSERT_TRUE(has_ok_marker(verdir)); // the poisoned state, exactly - - EXPECT_FALSE(mcpp::fallback::is_install_complete(verdir)); - fs::remove_all(dir); -} - -// The other direction, which is the one that would make every build slower if -// it were wrong: a real install must still be believed on the strength of its -// marker, without re-running anything. -TEST(InstallEvidence, ARealInstallIsStillCompleteOnItsMarker) { - auto dir = make_tempdir("mcpp-evidence-good"); - auto verdir = dir / "xim-x-gcc" / "16.1.0"; - make_legacy_pkg(verdir, "payload"); - write_file(verdir / ".mcpp_ok", "1\n"); - - EXPECT_TRUE(mcpp::fallback::payload_is_substantive(verdir)); - EXPECT_TRUE(mcpp::fallback::is_install_complete(verdir)); - fs::remove_all(dir); -} - -// An absent directory is absent, not incomplete — the new arm must not change -// what the function says about a package that was never installed. -TEST(InstallEvidence, AnAbsentDirectoryIsStillJustAbsent) { - auto dir = make_tempdir("mcpp-evidence-absent"); - EXPECT_FALSE(mcpp::fallback::is_install_complete(dir / "nothing" / "here")); - EXPECT_FALSE(mcpp::fallback::payload_is_substantive(dir / "nothing" / "here")); - fs::remove_all(dir); -} diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 3a6328f98..2a82c38e2 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -2661,6 +2661,47 @@ windows_code_page = "{}" } } +// `windows_long_paths` (design 2026-10-16 §11): an executable's statement +// that it handles paths past MAX_PATH; unset when the author says nothing. +TEST(Manifest, ParsesWindowsLongPathsOnABinaryTargetAndForTests) { + for (bool value : {true, false}) { + const auto src = std::format(R"( +[package] +name = "app" +version = "0.1.0" +[targets.app] +kind = "bin" +main = "src/main.cpp" +windows_long_paths = {0} +[test] +windows_long_paths = {0} +)", value); + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + ASSERT_EQ(m->targets.size(), 1u); + ASSERT_TRUE(m->targets[0].windowsLongPaths.has_value()); + EXPECT_EQ(*m->targets[0].windowsLongPaths, value); + ASSERT_TRUE(m->testWindowsLongPaths.has_value()); + EXPECT_EQ(*m->testWindowsLongPaths, value); + EXPECT_TRUE(m->schemaWarnings.empty()); + } + auto unset = mcpp::manifest::parse_string( + "[package]\nname = \"app\"\nversion = \"0.1.0\"\n[targets.app]\nkind = \"bin\"\nmain = \"m.cpp\"\n"); + ASSERT_TRUE(unset.has_value()); + EXPECT_FALSE(unset->targets[0].windowsLongPaths.has_value()); + auto wrong = mcpp::manifest::parse_string( + "[package]\nname = \"app\"\nversion = \"0.1.0\"\n[targets.app]\nkind = \"bin\"\nmain = \"m.cpp\"\n" + "windows_long_paths = \"yes\"\n"); + ASSERT_FALSE(wrong.has_value()); + EXPECT_NE(wrong.error().message.find("windows_long_paths must be true or false"), std::string::npos); + auto library = mcpp::manifest::parse_string( + "[package]\nname = \"app\"\nversion = \"0.1.0\"\n[targets.core]\nkind = \"lib\"\n" + "windows_long_paths = true\n"); + ASSERT_FALSE(library.has_value()); + EXPECT_NE(library.error().message.find("targets.core.windows_long_paths"), std::string::npos) + << library.error().message; +} + TEST(Manifest, RefusesAnUnknownWindowsCodePageNamingTheAcceptedOnes) { constexpr auto src = R"( [package] diff --git a/tests/unit/test_pm_index_refresh.cpp b/tests/unit/test_pm_index_refresh.cpp index c7604be14..f8a7a2d37 100644 --- a/tests/unit/test_pm_index_refresh.cpp +++ b/tests/unit/test_pm_index_refresh.cpp @@ -21,7 +21,7 @@ import mcpp.pm.index_refresh; import mcpp.pm.index_route; import mcpp.pm.index_spec; import mcpp.platform.axis; -import mcpp.xlings; +import mcpp.provider.xlings; namespace { @@ -94,7 +94,7 @@ package = {{ void remove_index() { std::filesystem::remove_all(index_dir()); } - mcpp::xlings::Env env() const { return mcpp::xlings::Env{ .home = home }; } + mcpp::provider::xlings::Env env() const { return mcpp::provider::xlings::Env{ .home = home }; } mcpp::config::GlobalConfig config() const { mcpp::config::GlobalConfig cfg; @@ -117,7 +117,7 @@ mcpp::pm::DependencySpec version_dep(std::string_view ns, RefreshReason decide(const mcpp::pm::IndexRoute& route, const mcpp::pm::DependencySpec& spec, - const mcpp::xlings::Env& env, + const mcpp::provider::xlings::Env& env, mcpp::pm::RefreshPolicy policy = {}) { return mcpp::pm::decide_for_dependency( route, spec.shortName, spec, env, @@ -126,7 +126,7 @@ RefreshReason decide(const mcpp::pm::IndexRoute& route, bool refreshes(const mcpp::pm::IndexRoute& route, const mcpp::pm::DependencySpec& spec, - const mcpp::xlings::Env& env, + const mcpp::provider::xlings::Env& env, mcpp::pm::RefreshPolicy policy = {}) { return mcpp::pm::decide_for_dependency( route, spec.shortName, spec, env, diff --git a/tests/unit/test_post_install.cpp b/tests/unit/test_post_install.cpp index bd5fd96e5..2d90e4767 100644 --- a/tests/unit/test_post_install.cpp +++ b/tests/unit/test_post_install.cpp @@ -3,7 +3,7 @@ import std; import mcpp.toolchain.post_install; -import mcpp.xlings; +import mcpp.provider.xlings; import mcpp.config; import mcpp.toolchain.registry; import mcpp.platform; @@ -64,134 +64,9 @@ TEST(DetectBakedLoader, EmptyWhenNoGnuLoaderPresent) { } // namespace -// ── the post-install fixup gate (mcpp#427) ─────────────────────────────── -// -// A unit test rather than an e2e ON PURPOSE, and the reason is the defect -// itself. `ensure_post_install_fixup` returns early — before any of the logic -// under test — for a payload that resolves OUTSIDE the caller's registry -// ("inherited payload, owner is responsible for its fixup"). Every affordable -// e2e inherits its toolchain by symlink to avoid a multi-gigabyte download, so -// no e2e can reach this code. That is precisely the shape of mcpp#221: a test -// that creates the missing thing somewhere the code under test never looks, -// and passes. -// -// Here the payload is a real directory inside the test's own registry, so the -// containment guard passes and the gate actually runs. - -namespace fixup_gate { - -struct Sandbox { - std::filesystem::path root; - mcpp::config::GlobalConfig cfg; - - Sandbox() { - root = std::filesystem::temp_directory_path() - / std::format("mcpp-fixup-{}", std::random_device{}()); - std::filesystem::create_directories(root / "registry" / "data" / "xpkgs"); - cfg.registryDir = root / "registry"; - } - ~Sandbox() { - std::error_code ec; - std::filesystem::remove_all(root, ec); - } - - // A payload physically inside the registry, so the #273 containment guard - // treats it as ours to patch and the gate is actually reached. - std::filesystem::path payload(std::string_view name) { - auto p = root / "registry" / "data" / "xpkgs" / name / "1.0.0"; - std::filesystem::create_directories(p / "bin"); - return p; - } -}; - -mcpp::toolchain::XimToolchainPackage gcc_pkg() { - mcpp::toolchain::XimToolchainPackage pkg; - pkg.ximName = "gcc"; - pkg.ximVersion = "16.1.0"; - pkg.needsGccPostInstallFixup = true; - return pkg; -} - -} // namespace fixup_gate - -// THE REGRESSION. An empty runtime identity used to be -// std::unexpected("… default SubOS has no RuntimeBinding identity …") -// and `prepare.cppm` turned that into `error: toolchain post-install fixup: …`, -// so `mcpp build` AND `mcpp toolchain install` both died on any Linux machine -// whose default SubOS predated xlings' `subos_info` block. Absence is not a -// contradiction: it degrades. -TEST(PostInstallFixup, AnAbsentRuntimeIdentityDegradesInsteadOfFailing) { - if constexpr (!mcpp::platform::is_linux) { - GTEST_SKIP() << "the runtime binding gate is Linux-only"; - } else { - fixup_gate::Sandbox sb; - auto result = mcpp::toolchain::ensure_post_install_fixup( - sb.cfg, sb.payload("xim-x-gcc"), fixup_gate::gcc_pkg(), - /*runtimeId=*/"", /*selectedRuntimeLibDir=*/{}); - - ASSERT_TRUE(result.has_value()) - << "an undescribed SubOS became a hard error again: " - << result.error(); - EXPECT_FALSE(result->applied); - // ...and it must SAY why. A degradation nobody can report is - // indistinguishable from a fixup that silently stopped working. - EXPECT_FALSE(result->skippedReason.empty()) - << "degraded silently — the caller has nothing to print"; - } -} - -// The other direction, so relaxing absence cannot quietly relax everything. A -// runtime that is DECLARED but cannot be produced is wrong information, not -// missing information: guessing a different glibc would make one mcpp.toml mean -// different ABIs on different machines. -TEST(PostInstallFixup, ADeclaredRuntimeThatCannotBeHonouredStillFails) { - if constexpr (!mcpp::platform::is_linux) { - GTEST_SKIP() << "the runtime binding gate is Linux-only"; - } else { - fixup_gate::Sandbox sb; - auto result = mcpp::toolchain::ensure_post_install_fixup( - sb.cfg, sb.payload("xim-x-gcc"), fixup_gate::gcc_pkg(), - /*runtimeId=*/"glibc@0.0.0-does-not-exist", - /*selectedRuntimeLibDir=*/{}); - - EXPECT_FALSE(result.has_value()) - << "a runtime identity that names nothing installed was accepted"; - } -} - -// No marker may be written for a fixup that did not happen. The marker is a -// content fingerprint whose whole job is to answer "were these inputs ever -// applied"; one written for "we did nothing" reads back as "already applied", -// and the fixup would then never run again — including on the day the user -// moves to a SubOS that does describe itself and the identity becomes knowable. -TEST(PostInstallFixup, ASkippedFixupLeavesNoMarkerBehind) { - if constexpr (!mcpp::platform::is_linux) { - GTEST_SKIP() << "the runtime binding gate is Linux-only"; - } else { - fixup_gate::Sandbox sb; - auto payload = sb.payload("xim-x-gcc"); - auto result = mcpp::toolchain::ensure_post_install_fixup( - sb.cfg, payload, fixup_gate::gcc_pkg(), /*runtimeId=*/"", {}); - ASSERT_TRUE(result.has_value()) << result.error(); - EXPECT_FALSE(std::filesystem::exists(payload / ".mcpp-fixup.json")) - << "'we did nothing' was recorded as 'already applied'"; - } -} - -// A package that needs no fixup at all is not a degradation and must not report -// one, or the caller warns about every msvc/system install. -TEST(PostInstallFixup, APackageWithNoFixupReportsNothingToReport) { - fixup_gate::Sandbox sb; - mcpp::toolchain::XimToolchainPackage none; - none.ximName = "msvc"; - none.ximVersion = "system"; - auto result = mcpp::toolchain::ensure_post_install_fixup( - sb.cfg, sb.payload("xim-x-msvc"), none, "", {}); - ASSERT_TRUE(result.has_value()) << result.error(); - EXPECT_TRUE(result->skippedReason.empty()) - << "a toolchain with no fixup reported a degradation: " - << result->skippedReason; -} +// The fixup gate tests (mcpp#427) went with the fixup: a payload is not +// written by mcpp (design 2026-10-16 §7.1), so there is no gate to test. +// What stays is read-only: the baked loader above, the glibc payload below. // THE DECLARED VERSION NAMES ITS PAYLOAD DIRECTORY EXACTLY (mcpp#660). // diff --git a/tests/unit/test_post_install_containment.cpp b/tests/unit/test_post_install_containment.cpp deleted file mode 100644 index 6af7ced4b..000000000 --- a/tests/unit/test_post_install_containment.cpp +++ /dev/null @@ -1,116 +0,0 @@ -// #273 regression: patchelf_walk followed a symlinked payload directory out -// of the sandbox and corrupted the user's real ~/.xlings gcc installation. -// These tests pin the containment predicate that fences every rewrite in -// post_install: the trust root is the owning sandbox's registry -// (cfg.registryDir), canonicalized ONCE via containment_root() and threaded -// down explicitly; escapes_containment() compares a file's PHYSICAL -// (symlink-resolved) location against it and fails closed. -#include - -import std; -import mcpp.toolchain.post_install; - -namespace fs = std::filesystem; -using mcpp::toolchain::containment_root; -using mcpp::toolchain::escapes_containment; - -namespace { - -// Temp tree reproducing the incident topology: -// /sandbox/registry/data/xpkgs/pkg-real/1.0/bin/tool (real file) -// /foreign/pkg/1.0/bin/tool (real file) -// /sandbox/registry/data/xpkgs/pkg-link -> /foreign/pkg -struct Fixture { - fs::path tmp; - fs::path registry, realPayload, foreign, linkedPayload; - bool hasSymlink = false; // Windows may lack symlink privilege → SKIP - - Fixture() { - auto tick = std::chrono::steady_clock::now().time_since_epoch().count(); - tmp = fs::temp_directory_path() / - fs::path(std::format("mcpp-containment-test-{}", tick)); - fs::remove_all(tmp); - registry = tmp / "sandbox" / "registry"; - realPayload = registry / "data" / "xpkgs" / "pkg-real" / "1.0"; - foreign = tmp / "foreign" / "pkg"; - fs::create_directories(realPayload / "bin"); - fs::create_directories(foreign / "1.0" / "bin"); - std::ofstream(realPayload / "bin" / "tool") << "x"; - std::ofstream(foreign / "1.0" / "bin" / "tool") << "x"; - linkedPayload = registry / "data" / "xpkgs" / "pkg-link"; - std::error_code ec; - fs::create_directory_symlink(foreign, linkedPayload, ec); - hasSymlink = !ec; - } - ~Fixture() { - std::error_code ec; - fs::remove_all(tmp, ec); - } -}; - -} // namespace - -TEST(Containment, RootIsCanonicalRegistryDir) { - Fixture f; - EXPECT_EQ(containment_root(f.registry), fs::weakly_canonical(f.registry)); -} - -TEST(Containment, RealFileInsideRegistryDoesNotEscape) { - Fixture f; - auto root = containment_root(f.registry); - EXPECT_FALSE(escapes_containment(f.realPayload / "bin" / "tool", root)); -} - -TEST(Containment, FileBehindSymlinkedPayloadEscapes) { - Fixture f; - if (!f.hasSymlink) GTEST_SKIP() << "no symlink privilege"; - // The incident's core: spelled under the registry, physically foreign. - // Canonicalizing the payload path FIRST would make this a tautology — - // the fence must be the independently-resolved registry root. - auto root = containment_root(f.registry); - EXPECT_TRUE(escapes_containment(f.linkedPayload / "1.0" / "bin" / "tool", root)); -} - -TEST(Containment, SymlinkedPayloadRootItselfEscapes) { - Fixture f; - if (!f.hasSymlink) GTEST_SKIP() << "no symlink privilege"; - // What the entry-point ownership guard checks before running any fixup. - auto root = containment_root(f.registry); - EXPECT_TRUE(escapes_containment(f.linkedPayload / "1.0", root)); -} - -TEST(Containment, DotDotTraversalEscapes) { - Fixture f; - auto root = containment_root(f.registry); - // bin → 1.0 → pkg-real → xpkgs → data → registry → sandbox, then into - // a path outside the registry: physically out even though the spelled - // prefix starts inside it. - auto sneaky = f.realPayload / "bin" / ".." / ".." / ".." / ".." / ".." / ".." - / "elsewhere" / "tool"; - EXPECT_TRUE(escapes_containment(sneaky, root)); -} - -TEST(Containment, SiblingStringPrefixIsNotContained) { - Fixture f; - // "-evil" shares the raw string prefix but is a different - // directory — the component-boundary check must reject it. (The old - // inline guard compared raw strings and passed this.) - auto evil = fs::path(f.registry.string() + "-evil"); - fs::create_directories(evil); - std::ofstream(evil / "tool") << "x"; - auto root = containment_root(f.registry); - EXPECT_TRUE(escapes_containment(evil / "tool", root)); -} - -TEST(Containment, EmptyRootFailsClosed) { - Fixture f; - // containment_root() returns empty when the registry cannot be resolved; - // the predicate must then refuse everything rather than allow everything. - EXPECT_TRUE(escapes_containment(f.realPayload / "bin" / "tool", {})); -} - -TEST(Containment, RootItselfIsContained) { - Fixture f; - auto root = containment_root(f.registry); - EXPECT_FALSE(escapes_containment(f.registry, root)); -} diff --git a/tests/unit/test_runtime_contract.cpp b/tests/unit/test_runtime_contract.cpp index 768634f59..405ea8b23 100644 --- a/tests/unit/test_runtime_contract.cpp +++ b/tests/unit/test_runtime_contract.cpp @@ -10,7 +10,7 @@ import mcpp.platform; import mcpp.platform.axis; import mcpp.runtime.binding; import mcpp.platform.runtime_search; -import mcpp.xlings.subos_info; +import mcpp.provider.xlings.subos_info; namespace build = mcpp::build; namespace mf = mcpp::manifest; diff --git a/tests/unit/test_runtime_selection.cpp b/tests/unit/test_runtime_selection.cpp index 1e459d058..dd1bb435d 100644 --- a/tests/unit/test_runtime_selection.cpp +++ b/tests/unit/test_runtime_selection.cpp @@ -5,9 +5,9 @@ import mcpp.config; import mcpp.manifest; import mcpp.platform; import mcpp.runtime.binding; -import mcpp.xlings.runtime_selection; +import mcpp.provider.runtime_selection; -namespace runtime = mcpp::xlings::runtime; +namespace runtime = mcpp::provider::runtime; namespace { diff --git a/tests/unit/test_sources.cpp b/tests/unit/test_sources.cpp index 1fdfccb8c..b13fb7943 100644 --- a/tests/unit/test_sources.cpp +++ b/tests/unit/test_sources.cpp @@ -12,10 +12,10 @@ import mcpp.libs.toml; import mcpp.build.directives; import mcpp.toolchain.dialect; import mcpp.toolchain.registry; -import mcpp.xlings.address_set; +import mcpp.provider.address_set; namespace dirs = mcpp::build::directives; -namespace addrset = mcpp::xlings::addrset; +namespace addrset = mcpp::provider::addrset; namespace { diff --git a/tests/unit/test_subos_info.cpp b/tests/unit/test_subos_info.cpp index f76229064..9aee8f159 100644 --- a/tests/unit/test_subos_info.cpp +++ b/tests/unit/test_subos_info.cpp @@ -11,9 +11,9 @@ import std; import mcpp.platform; -import mcpp.xlings.subos_info; +import mcpp.provider.xlings.subos_info; -namespace su = mcpp::xlings::subos; +namespace su = mcpp::provider::xlings::subos; namespace { diff --git a/tests/unit/test_toolchain_lifecycle.cpp b/tests/unit/test_toolchain_lifecycle.cpp index dcb150be9..27c6b5a83 100644 --- a/tests/unit/test_toolchain_lifecycle.cpp +++ b/tests/unit/test_toolchain_lifecycle.cpp @@ -168,7 +168,7 @@ TEST(DeclaredRuntime, ABindingThatLocatesItsPayloadIsLeftAlone) { binding.runtimeId = "glibc@2.44.3"; binding.libraryDirs = {"/store/xim-x-glibc/2.44.3/lib64"}; const auto before = binding.libraryDirs; - EXPECT_FALSE(ensure_declared_runtime(cfg, binding)); + EXPECT_EQ(ensure_declared_runtime(cfg, binding).value_or(true), false); EXPECT_EQ(binding.libraryDirs, before); EXPECT_EQ(binding.runtimeId, "glibc@2.44.3"); } @@ -180,7 +180,7 @@ TEST(DeclaredRuntime, OnlyAGlibcPayloadIsInstalled) { for (auto id : {"", "ucrt@10.0.26100.0", "macos_sdk@15.0", "glibc"}) { mcpp::platform::runtime::RuntimeBinding binding; binding.runtimeId = id; - EXPECT_FALSE(ensure_declared_runtime(cfg, binding)) << id; + EXPECT_EQ(ensure_declared_runtime(cfg, binding).value_or(true), false) << id; EXPECT_TRUE(binding.libraryDirs.empty()) << id; } } @@ -192,24 +192,23 @@ TEST(DeclaredRuntime, NothingIsInstalledOffLinux) { auto cfg = unreachable_cfg(); mcpp::platform::runtime::RuntimeBinding binding; binding.runtimeId = "glibc@2.44.3"; - EXPECT_FALSE(ensure_declared_runtime(cfg, binding)); + EXPECT_EQ(ensure_declared_runtime(cfg, binding).value_or(true), false); } -// The dispatch that decides whether a toolchain needs the payload at all. A -// toolchain whose fixup does not patch against a C runtime must not cause a -// download. -TEST(DeclaredRuntime, OnlyToolchainsTheFixupPatchesNeedThePayload) { +// Which toolchains run on a C runtime payload: a GNU gcc and an llvm on +// Linux. Any other toolchain must not cause a runtime download. +TEST(DeclaredRuntime, OnlyToolchainsOnARuntimePayloadNeedIt) { XimToolchainPackage gcc; gcc.ximName = "gcc"; gcc.needsGccPostInstallFixup = true; XimToolchainPackage llvm; llvm.ximName = "llvm"; XimToolchainPackage musl; musl.ximName = "musl-gcc"; XimToolchainPackage msvc; msvc.ximName = "msvc"; if constexpr (mcpp::platform::is_windows) { for (auto const* p : {&gcc, &llvm, &musl, &msvc}) - EXPECT_TRUE(post_install_fixup_kind(*p).empty()) << p->ximName; + EXPECT_TRUE(runtime_payload_kind(*p).empty()) << p->ximName; } else { - EXPECT_EQ(post_install_fixup_kind(gcc), "gcc"); - EXPECT_EQ(post_install_fixup_kind(llvm), "llvm"); - EXPECT_TRUE(post_install_fixup_kind(musl).empty()); - EXPECT_TRUE(post_install_fixup_kind(msvc).empty()); + EXPECT_EQ(runtime_payload_kind(gcc), "gcc"); + EXPECT_EQ(runtime_payload_kind(llvm), "llvm"); + EXPECT_TRUE(runtime_payload_kind(musl).empty()); + EXPECT_TRUE(runtime_payload_kind(msvc).empty()); } } diff --git a/tests/unit/test_xlings.cpp b/tests/unit/test_xlings.cpp index f7abc8069..2bd37dd0c 100644 --- a/tests/unit/test_xlings.cpp +++ b/tests/unit/test_xlings.cpp @@ -1,7 +1,7 @@ #include import std; -import mcpp.xlings; +import mcpp.provider.xlings; import mcpp.platform.env; namespace { @@ -96,9 +96,9 @@ TEST(FindUsableNasm, FindsOnPathWhenVersionIsAdequate) { mcpp::platform::env::ScopedEnv path("PATH", pathDir.string() + ":" + base); auto home = make_tempdir("mcpp-nasm-home"); // no sandbox nasm present - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - auto found = mcpp::xlings::find_usable_nasm(env); + auto found = mcpp::provider::xlings::find_usable_nasm(env); ASSERT_TRUE(found.has_value()); EXPECT_EQ(*found, expected); @@ -114,9 +114,9 @@ TEST(FindUsableNasm, RejectsPathNasmBelowMinVersionAndFindsNoSandboxFallback) { mcpp::platform::env::ScopedEnv path("PATH", pathDir.string() + ":" + base); auto home = make_tempdir("mcpp-nasm-home-empty"); // no sandbox nasm - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::find_usable_nasm(env).has_value()); + EXPECT_FALSE(mcpp::provider::xlings::find_usable_nasm(env).has_value()); std::filesystem::remove_all(pathDir); std::filesystem::remove_all(home); @@ -131,9 +131,9 @@ TEST(FindUsableNasm, FallsBackToSandboxWhenPathHasNoNasm) { auto sandboxDir = home / "data" / "xpkgs" / "xim-x-nasm" / "3.02"; auto expected = make_fake_nasm(sandboxDir, "NASM version 3.02 compiled on Jan 1 2026"); - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - auto found = mcpp::xlings::find_usable_nasm(env); + auto found = mcpp::provider::xlings::find_usable_nasm(env); ASSERT_TRUE(found.has_value()); EXPECT_EQ(*found, expected); @@ -145,9 +145,9 @@ TEST(FindSandboxNasm, FindsUnderVersionedBinSubdir) { auto sandboxDir = home / "data" / "xpkgs" / "xim-x-nasm" / "3.02" / "bin"; auto expected = make_fake_nasm(sandboxDir, "NASM version 3.02 compiled on Jan 1 2026"); - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - auto found = mcpp::xlings::find_sandbox_nasm(env); + auto found = mcpp::provider::xlings::find_sandbox_nasm(env); ASSERT_TRUE(found.has_value()); EXPECT_EQ(*found, expected); @@ -159,18 +159,18 @@ TEST(FindSandboxNasm, RejectsInstalledNasmBelowMinVersion) { auto sandboxDir = home / "data" / "xpkgs" / "xim-x-nasm" / "2.10"; make_fake_nasm(sandboxDir, "NASM version 2.10.00 compiled on Jan 1 2020"); - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::find_sandbox_nasm(env).has_value()); + EXPECT_FALSE(mcpp::provider::xlings::find_sandbox_nasm(env).has_value()); std::filesystem::remove_all(home); } TEST(FindSandboxNasm, NoSandboxDirectoryYieldsNullopt) { auto home = make_tempdir("mcpp-nasm-home-absent"); - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::find_sandbox_nasm(env).has_value()); + EXPECT_FALSE(mcpp::provider::xlings::find_sandbox_nasm(env).has_value()); std::filesystem::remove_all(home); } @@ -181,9 +181,9 @@ TEST(XlingsIndexFreshness, RequiresDefaultMcpplibsIndex) { auto home = make_tempdir("mcpp-xlings-index-freshness"); std::filesystem::create_directories(home / "data" / "xim-pkgindex" / "pkgs"); - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::is_index_fresh(env, 3600)); + EXPECT_FALSE(mcpp::provider::xlings::is_index_fresh(env, 3600)); std::filesystem::remove_all(home); } @@ -193,9 +193,9 @@ TEST(XlingsIndexFreshness, AcceptsFreshDefaultMcpplibsIndex) { std::filesystem::create_directories(home / "data" / "mcpplibs" / "pkgs"); std::ofstream(home / "data" / "mcpplibs" / ".mcpp-index-updated") << "ok\n"; - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_TRUE(mcpp::xlings::is_index_fresh(env, 3600)); + EXPECT_TRUE(mcpp::provider::xlings::is_index_fresh(env, 3600)); std::filesystem::remove_all(home); } @@ -204,9 +204,9 @@ TEST(XlingsIndexFreshness, RequiresRefreshMarkerForDefaultMcpplibsIndex) { auto home = make_tempdir("mcpp-xlings-index-freshness"); std::filesystem::create_directories(home / "data" / "mcpplibs" / "pkgs"); - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::is_index_fresh(env, 3600)); + EXPECT_FALSE(mcpp::provider::xlings::is_index_fresh(env, 3600)); std::filesystem::remove_all(home); } @@ -219,9 +219,9 @@ TEST(XlingsIndexFreshness, RejectsStaleRefreshMarker) { std::filesystem::last_write_time( marker, std::filesystem::file_time_type::clock::now() - std::chrono::seconds(7200)); - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::is_index_fresh(env, 3600)); + EXPECT_FALSE(mcpp::provider::xlings::is_index_fresh(env, 3600)); std::filesystem::remove_all(home); } @@ -231,9 +231,9 @@ TEST(XlingsIndexFreshness, RequiresOfficialXimIndexEvenWhenDefaultIndexIsFresh) std::filesystem::create_directories(home / "data" / "mcpplibs" / "pkgs"); std::ofstream(home / "data" / "mcpplibs" / ".mcpp-index-updated") << "ok\n"; - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::is_official_index_fresh(env, 3600)); + EXPECT_FALSE(mcpp::provider::xlings::is_official_index_fresh(env, 3600)); std::filesystem::remove_all(home); } @@ -243,9 +243,9 @@ TEST(XlingsIndexFreshness, AcceptsFreshOfficialXimIndex) { std::filesystem::create_directories(home / "data" / "xim-pkgindex" / "pkgs"); std::ofstream(home / "data" / "xim-pkgindex" / ".mcpp-index-updated") << "ok\n"; - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_TRUE(mcpp::xlings::is_official_index_fresh(env, 3600)); + EXPECT_TRUE(mcpp::provider::xlings::is_official_index_fresh(env, 3600)); std::filesystem::remove_all(home); } @@ -255,9 +255,9 @@ TEST(XlingsIndexFreshness, RequiresOfficialPackageFileEvenWhenOfficialIndexIsFre std::filesystem::create_directories(home / "data" / "xim-pkgindex" / "pkgs"); std::ofstream(home / "data" / "xim-pkgindex" / ".mcpp-index-updated") << "ok\n"; - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::is_official_package_index_fresh(env, "musl-gcc", 3600)); + EXPECT_FALSE(mcpp::provider::xlings::is_official_package_index_fresh(env, "musl-gcc", 3600)); std::filesystem::remove_all(home); } @@ -268,9 +268,9 @@ TEST(XlingsIndexFreshness, AcceptsFreshOfficialPackageFile) { std::ofstream(home / "data" / "xim-pkgindex" / ".mcpp-index-updated") << "ok\n"; std::ofstream(home / "data" / "xim-pkgindex" / "pkgs" / "m" / "musl-gcc.lua") << "package = {}\n"; - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_TRUE(mcpp::xlings::is_official_package_index_fresh(env, "musl-gcc", 3600)); + EXPECT_TRUE(mcpp::provider::xlings::is_official_package_index_fresh(env, "musl-gcc", 3600)); std::filesystem::remove_all(home); } @@ -284,9 +284,9 @@ TEST(XlingsIndexFreshness, RejectsOfficialPackageCacheWithForeignPath) { std::ofstream(home / "data" / "xim-pkgindex" / ".xlings-index-cache.json") << R"({"entries":{"musl-gcc":{"path":"/tmp/foreign/xim-pkgindex/pkgs/m/musl-gcc.lua"}}})"; - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_FALSE(mcpp::xlings::is_official_package_index_fresh(env, "musl-gcc", 3600)); + EXPECT_FALSE(mcpp::provider::xlings::is_official_package_index_fresh(env, "musl-gcc", 3600)); std::filesystem::remove_all(home); } @@ -300,9 +300,9 @@ TEST(XlingsIndexFreshness, AcceptsOfficialPackageCacheWithCurrentPath) { std::ofstream(home / "data" / "xim-pkgindex" / ".xlings-index-cache.json") << std::format(R"({{"entries":{{"musl-gcc":{{"path":"{}"}}}}}})", pkg.string()); - mcpp::xlings::Env env{.home = home}; + mcpp::provider::xlings::Env env{.home = home}; - EXPECT_TRUE(mcpp::xlings::is_official_package_index_fresh(env, "musl-gcc", 3600)); + EXPECT_TRUE(mcpp::provider::xlings::is_official_package_index_fresh(env, "musl-gcc", 3600)); std::filesystem::remove_all(home); } @@ -337,7 +337,7 @@ TEST(XlingsSiblingPackage, MetadataOnlyHuskIsNotContent) { // Isolate from the host's ~/.xlings fallback. const char* oldHome = std::getenv("HOME"); mcpp::platform::env::set("HOME", tmp.string()); - auto found = mcpp::xlings::paths::find_sibling_package(gccBin, "linux-headers"); + auto found = mcpp::provider::xlings::paths::find_sibling_package(gccBin, "linux-headers"); mcpp::platform::env::set("HOME", oldHome ? oldHome : ""); EXPECT_FALSE(found.has_value()); @@ -358,7 +358,7 @@ TEST(XlingsSiblingPackage, SkipsHuskAndFindsPayloadUnderOtherPrefix) { auto real = xpkgs / "scode-x-linux-headers" / "5.11.1"; touch(real / "include" / "linux" / "limits.h"); - auto found = mcpp::xlings::paths::find_sibling_package(gccBin, "linux-headers"); + auto found = mcpp::provider::xlings::paths::find_sibling_package(gccBin, "linux-headers"); ASSERT_TRUE(found.has_value()); EXPECT_EQ(*found, real); @@ -378,7 +378,7 @@ TEST(XlingsSiblingPackage, RequiredRelPathRejectsContentfulButWrongCandidate) { auto real = xpkgs / "scode-x-linux-headers" / "5.11.1"; touch(real / "include" / "linux" / "limits.h"); - auto found = mcpp::xlings::paths::find_sibling_package( + auto found = mcpp::provider::xlings::paths::find_sibling_package( gccBin, "linux-headers", "include/linux/limits.h"); ASSERT_TRUE(found.has_value()); EXPECT_EQ(*found, real); @@ -399,7 +399,7 @@ TEST(XlingsHomeTool, FindsPayloadUnderNonXimPrefix) { const char* oldMcppHome = std::getenv("MCPP_HOME"); mcpp::platform::env::set("MCPP_HOME", tmp.string()); - auto found = mcpp::xlings::paths::find_home_tool( + auto found = mcpp::provider::xlings::paths::find_home_tool( "linux-headers", "include/linux/limits.h"); mcpp::platform::env::set("MCPP_HOME", oldMcppHome ? oldMcppHome : ""); @@ -425,11 +425,11 @@ TEST(XlingsSeed, PlainRepoEmissionIsByteStable) { // Zero-diff gate: repos without artifact/source must serialize exactly // as before the SeedRepo extension. auto dir = make_tempdir("mcpp-seed-plain"); - mcpp::xlings::Env env; + mcpp::provider::xlings::Env env; env.home = dir; - std::vector repos{ + std::vector repos{ { "mcpplibs", "https://x.git", "", "" } }; - mcpp::xlings::seed_xlings_json(env, repos); + mcpp::provider::xlings::seed_xlings_json(env, repos); auto text = read_seeded(dir); EXPECT_NE(text.find( " { \"name\": \"mcpplibs\", \"url\": \"https://x.git\" }\n"), @@ -441,12 +441,12 @@ TEST(XlingsSeed, PlainRepoEmissionIsByteStable) { TEST(XlingsSeed, ArtifactAndSourceEmittedWhenSet) { auto dir = make_tempdir("mcpp-seed-art"); - mcpp::xlings::Env env; + mcpp::provider::xlings::Env env; env.home = dir; - std::vector repos{ + std::vector repos{ { "mcpplibs", "https://x.git", "https://github.com/xlings-res/mcpp-index", "auto" } }; - mcpp::xlings::seed_xlings_json(env, repos); + mcpp::provider::xlings::seed_xlings_json(env, repos); auto text = read_seeded(dir); EXPECT_NE(text.find( "\"url\": \"https://x.git\", \"artifact\": " @@ -455,6 +455,99 @@ TEST(XlingsSeed, ArtifactAndSourceEmittedWhenSet) { std::filesystem::remove_all(dir); } +// ─── Indices mcpp owns: their places (design 2026-10-16 §5) ────────────── + +TEST(XlingsOwnedIndex, SeedNamesItsDataAndStore) { + auto dir = make_tempdir("mcpp-seed-owned"); + mcpp::provider::xlings::Env env; + env.home = dir / "registry"; + mcpp::provider::xlings::SeedRepo repo{ "mcpplibs", "https://x.git", "", "" }; + repo.dir = dir / "indices" / "mcpplibs"; + repo.store = dir / "xpkgs"; + std::vector repos{ repo }; + mcpp::provider::xlings::seed_xlings_json(env, repos); + const auto declared = mcpp::provider::xlings::paths::declared_indices(env); + ASSERT_EQ(declared.size(), 1u); + EXPECT_TRUE(declared[0].owned); + EXPECT_EQ(declared[0].dir, (dir / "indices" / "mcpplibs").lexically_normal()); + EXPECT_EQ(declared[0].store, (dir / "xpkgs").lexically_normal()); + // Its store first, the home's after it (payloads from before, §5.3). + const auto stores = mcpp::provider::xlings::paths::stores(env); + ASSERT_EQ(stores.size(), 2u); + EXPECT_EQ(stores[0], (dir / "xpkgs").lexically_normal()); + EXPECT_EQ(stores[1], env.home / "data" / "xpkgs"); + std::filesystem::remove_all(dir); +} + +TEST(XlingsOwnedIndex, AHomeServesItsDefaultAndItsConfiguredIndexes) { + auto dir = make_tempdir("mcpp-indices-of"); + EXPECT_TRUE(mcpp::provider::xlings::paths::indices_of(dir).empty()); // not a provider home + std::ofstream(dir / ".xlings.json") << "{}"; + EXPECT_EQ(mcpp::provider::xlings::paths::indices_of(dir), std::vector{"xim"}); + std::ofstream(dir / ".xlings.json") << R"({"index_repos":[{"name":"mcpp-index","url":"u"},{"name":"xim","url":"v"}]})"; + EXPECT_EQ(mcpp::provider::xlings::paths::indices_of(dir), (std::vector{"xim", "mcpp-index"})); + std::filesystem::remove_all(dir); +} + +TEST(XlingsOwnedIndex, AHomeFromBeforeHasItsIndexMovedOnce) { + auto dir = make_tempdir("mcpp-own-indices"); + mcpp::provider::xlings::Env env; + env.home = dir / "registry"; + std::vector plain{ { "mcpplibs", "https://x.git", "", "" } }; + mcpp::provider::xlings::seed_xlings_json(env, plain); + const auto old = env.home / "data" / "mcpplibs"; + std::filesystem::create_directories(old / "pkgs" / "f"); + { std::ofstream(old / "pkgs" / "f" / "foo.lua") << "package = {}\n"; } + std::filesystem::create_directories(env.home / "data" / "xim-pkgindex" / "pkgs"); + + mcpp::provider::xlings::SeedRepo owned{ "mcpplibs", "https://x.git", "", "" }; + owned.dir = dir / "indices" / "mcpplibs"; + owned.store = dir / "xpkgs"; + std::vector repos{ owned }; + mcpp::provider::xlings::paths::own_indices(env, repos); + EXPECT_FALSE(std::filesystem::exists(old)); + EXPECT_TRUE(std::filesystem::exists(owned.dir / "pkgs" / "f" / "foo.lua")); + const auto declared = mcpp::provider::xlings::paths::declared_indices(env); + ASSERT_EQ(declared.size(), 1u); + EXPECT_TRUE(declared[0].owned); + // Read from both places: the provider's data dir and the owned one. + const auto roots = mcpp::provider::xlings::paths::index_roots(env); + EXPECT_NE(std::ranges::find(roots, owned.dir.lexically_normal()), roots.end()); + EXPECT_NE(std::ranges::find(roots, env.home / "data" / "xim-pkgindex"), roots.end()); + + // Once: data that reappears at the old place is not moved over the new. + std::filesystem::create_directories(old / "pkgs"); + mcpp::provider::xlings::paths::own_indices(env, repos); + EXPECT_TRUE(std::filesystem::exists(old)); + EXPECT_TRUE(std::filesystem::exists(owned.dir / "pkgs" / "f" / "foo.lua")); + // And it is not read as a second index. + const auto again = mcpp::provider::xlings::paths::index_roots(env); + EXPECT_EQ(std::ranges::find(again, old), again.end()); + std::filesystem::remove_all(dir); +} + +TEST(XlingsOwnedIndex, APayloadIsFoundInItsIndexStoreThenInTheHomes) { + auto dir = make_tempdir("mcpp-owned-payload"); + mcpp::provider::xlings::Env env; + env.home = dir / "registry"; + mcpp::provider::xlings::SeedRepo owned{ "mcpplibs", "https://x.git", "", "" }; + owned.dir = dir / "indices" / "mcpplibs"; + owned.store = dir / "xpkgs"; + std::vector repos{ owned }; + mcpp::provider::xlings::seed_xlings_json(env, repos); + // One installed before the index had a store of its own, one after. + std::filesystem::create_directories(env.home / "data" / "xpkgs" / "compat-x-zlib" / "1.3.1" / "lib"); + std::filesystem::create_directories(dir / "xpkgs" / "mcpplibs-x-cmdline" / "0.0.1" / "src"); + using mcpp::provider::xlings::paths::xpkg_payload; + auto zlib = xpkg_payload(env, { "compat", "zlib", "1.3.1" }); + ASSERT_TRUE(zlib.has_value()); + EXPECT_EQ(*zlib, env.home / "data" / "xpkgs" / "compat-x-zlib" / "1.3.1"); + auto cmdline = xpkg_payload(env, { "mcpplibs", "cmdline", "" }); + ASSERT_TRUE(cmdline.has_value()); + EXPECT_EQ(*cmdline, (dir / "xpkgs").lexically_normal() / "mcpplibs-x-cmdline" / "0.0.1"); + std::filesystem::remove_all(dir); +} + // ─── Index identity + clock-skew hardening (#315) ──────────────────────── TEST(XlingsIndexFreshness, FutureMarkerIsStaleNotEternallyFresh) { @@ -470,8 +563,8 @@ TEST(XlingsIndexFreshness, FutureMarkerIsStaleNotEternallyFresh) { std::filesystem::last_write_time( marker, std::filesystem::file_time_type::clock::now() + std::chrono::hours{72}, ec); - mcpp::xlings::Env env{.home = home}; - auto st = mcpp::xlings::default_index_status(env, 3600); + mcpp::provider::xlings::Env env{.home = home}; + auto st = mcpp::provider::xlings::default_index_status(env, 3600); EXPECT_FALSE(st.fresh); EXPECT_EQ(st.ageSeconds, -1); // unusable timestamp reports as unknown @@ -485,23 +578,23 @@ TEST(XlingsIndexRevision, ReadsTrimmedValueAndDegradesToNullopt) { std::filesystem::create_directories(dir / "pkgs"); // Absent: a local `path` index has no such file — must not be an error. - EXPECT_FALSE(mcpp::xlings::index_revision(dir).has_value()); + EXPECT_FALSE(mcpp::provider::xlings::index_revision(dir).has_value()); // Present. The observed files carry no trailing newline, but a writer that // added one (or a \r from Windows) must not change the value, or every run // would look like the index had changed. std::ofstream(dir / ".xlings-index-version") << " 8d67478\r\n"; - auto rev = mcpp::xlings::index_revision(dir); + auto rev = mcpp::provider::xlings::index_revision(dir); ASSERT_TRUE(rev.has_value()); EXPECT_EQ(*rev, "8d67478"); // Blank is not an identity. std::ofstream(dir / ".xlings-index-version", std::ios::trunc) << "\n"; - EXPECT_FALSE(mcpp::xlings::index_revision(dir).has_value()); + EXPECT_FALSE(mcpp::provider::xlings::index_revision(dir).has_value()); // The value is OPAQUE: sub-indexes carry a date version, not a sha. std::ofstream(dir / ".xlings-index-version", std::ios::trunc) << "2026.7.30.1"; - EXPECT_EQ(mcpp::xlings::index_revision(dir).value(), "2026.7.30.1"); + EXPECT_EQ(mcpp::provider::xlings::index_revision(dir).value(), "2026.7.30.1"); std::filesystem::remove_all(home); } @@ -513,8 +606,8 @@ TEST(XlingsIndexRevision, StatusCarriesTheRevision) { std::ofstream(dir / ".mcpp-index-updated") << "ok\n"; std::ofstream(dir / ".xlings-index-version") << "abc1234"; - mcpp::xlings::Env env{.home = home}; - auto st = mcpp::xlings::default_index_status(env, 3600); + mcpp::provider::xlings::Env env{.home = home}; + auto st = mcpp::provider::xlings::default_index_status(env, 3600); ASSERT_TRUE(st.rev.has_value()); EXPECT_EQ(*st.rev, "abc1234"); @@ -529,7 +622,7 @@ TEST(XlingsIndexRevision, StatusCarriesTheRevision) { // child cannot bury the diagnostic they are attached to. TEST(XlingsStderrTail, KeepsOnlyErrorLevelLines) { - auto tail = mcpp::xlings::stderr_error_tail( + auto tail = mcpp::provider::xlings::stderr_error_tail( "[xim] resolving acme.widget\r\n" "downloading 42%\n" "Error: checksum mismatch\n" @@ -544,14 +637,14 @@ TEST(XlingsStderrTail, KeepsOnlyErrorLevelLines) { TEST(XlingsStderrTail, KeepsTheLastLinesUpToTheLimit) { std::string text; for (int i = 0; i < 30; ++i) text += std::format("error {}\n", i); - auto tail = mcpp::xlings::stderr_error_tail(text); + auto tail = mcpp::provider::xlings::stderr_error_tail(text); ASSERT_EQ(tail.size(), 20u); EXPECT_EQ(tail.front(), "error 10"); EXPECT_EQ(tail.back(), "error 29"); - EXPECT_EQ(mcpp::xlings::stderr_error_tail(text, 2), + EXPECT_EQ(mcpp::provider::xlings::stderr_error_tail(text, 2), (std::vector{"error 28", "error 29"})); - EXPECT_TRUE(mcpp::xlings::stderr_error_tail("").empty()); - EXPECT_EQ(mcpp::xlings::stderr_error_tail("error without a newline"), + EXPECT_TRUE(mcpp::provider::xlings::stderr_error_tail("").empty()); + EXPECT_EQ(mcpp::provider::xlings::stderr_error_tail("error without a newline"), (std::vector{"error without a newline"})); } @@ -563,8 +656,8 @@ TEST(XlingsStderrTail, KeepsTheLastLinesUpToTheLimit) { namespace { -mcpp::xlings::Env xlings_env(std::string_view projectDir) { - mcpp::xlings::Env env; +mcpp::provider::xlings::Env xlings_env(std::string_view projectDir) { + mcpp::provider::xlings::Env env; env.home = std::filesystem::temp_directory_path() / "mcpp-xlings-home"; env.binary = env.home / "bin" / "xlings"; env.projectDir = std::filesystem::path(projectDir); @@ -573,8 +666,8 @@ mcpp::xlings::Env xlings_env(std::string_view projectDir) { // The decided entry for `key`; a default entry named "" when the decision omits // the key. -mcpp::xlings::InvocationVar decided(const mcpp::xlings::Env& env, std::string_view key) { - for (auto const& var : mcpp::xlings::invocation_env(env)) +mcpp::provider::xlings::InvocationVar decided(const mcpp::provider::xlings::Env& env, std::string_view key) { + for (auto const& var : mcpp::provider::xlings::invocation_env(env)) if (var.name == key) return var; return {}; } @@ -605,8 +698,8 @@ TEST(XlingsInvocationEnv, TheProcessEnvironmentIsUnchangedAfterwards) { { auto global = xlings_env(""); - mcpp::xlings::ScopedInvocationEnv scope(global); - (void)mcpp::xlings::build_command_prefix(global); + mcpp::provider::xlings::ScopedInvocationEnv scope(global); + (void)mcpp::provider::xlings::build_command_prefix(global); #if defined(_WIN32) EXPECT_FALSE(env::get("XLINGS_PROJECT_DIR").has_value()); #endif @@ -615,8 +708,8 @@ TEST(XlingsInvocationEnv, TheProcessEnvironmentIsUnchangedAfterwards) { { auto project = xlings_env("proj-dir"); - mcpp::xlings::ScopedInvocationEnv scope(project); - (void)mcpp::xlings::build_command_prefix(project); + mcpp::provider::xlings::ScopedInvocationEnv scope(project); + (void)mcpp::provider::xlings::build_command_prefix(project); #if defined(_WIN32) EXPECT_EQ(env::get("XLINGS_PROJECT_DIR"), std::optional(project.projectDir.string())); @@ -637,8 +730,8 @@ TEST(XlingsInvocationEnv, NeitherPathNorTheHomeOutlivesTheInvocation) { auto e = xlings_env(""); for (int i = 0; i < 3; ++i) { - mcpp::xlings::ScopedInvocationEnv scope(e); - (void)mcpp::xlings::build_command_prefix(e); + mcpp::provider::xlings::ScopedInvocationEnv scope(e); + (void)mcpp::provider::xlings::build_command_prefix(e); #if defined(_WIN32) EXPECT_EQ(env::get("XLINGS_HOME"), std::optional(e.home.string())); const auto path = env::get("PATH"); @@ -651,7 +744,7 @@ TEST(XlingsInvocationEnv, NeitherPathNorTheHomeOutlivesTheInvocation) { EXPECT_EQ(env::get("XLINGS_HOME"), std::optional("shell-home")); // Building a command alone changes nothing on any platform. - (void)mcpp::xlings::build_command_prefix(e); + (void)mcpp::provider::xlings::build_command_prefix(e); EXPECT_EQ(env::get("PATH"), pathBefore); EXPECT_EQ(env::get("XLINGS_HOME"), std::optional("shell-home")); } @@ -660,22 +753,22 @@ TEST(XlingsInvocationEnv, NeitherPathNorTheHomeOutlivesTheInvocation) { // directory mcpp was started from, so a project's `.xlings.json` cannot put it // in that project's mode. TEST(XlingsInvocationEnv, TheWindowsPrefixStartsInTheHome) { - mcpp::xlings::Env e; + mcpp::provider::xlings::Env e; e.home = "C:\\Users\\a b\\.mcpp\\registry"; e.binary = "C:\\Users\\a b\\.mcpp\\registry\\bin\\xlings.exe"; - EXPECT_EQ(mcpp::xlings::windows_command_prefix(e), + EXPECT_EQ(mcpp::provider::xlings::windows_command_prefix(e), "cd /d \"C:\\Users\\a b\\.mcpp\\registry\" && " "\"C:\\Users\\a b\\.mcpp\\registry\\bin\\xlings.exe\""); } #if !defined(_WIN32) TEST(XlingsInvocationEnv, ThePosixPrefixRendersTheDecision) { - auto global = mcpp::xlings::build_command_prefix(xlings_env("")); + auto global = mcpp::provider::xlings::build_command_prefix(xlings_env("")); EXPECT_NE(global.find("env -u XLINGS_PROJECT_DIR -u XLINGS_ACTIVE_SUBOS PATH="), std::string::npos) << global; EXPECT_EQ(global.find("XLINGS_PROJECT_DIR="), std::string::npos) << global; - auto project = mcpp::xlings::build_command_prefix(xlings_env("/work/proj")); + auto project = mcpp::provider::xlings::build_command_prefix(xlings_env("/work/proj")); EXPECT_EQ(project.find("-u XLINGS_PROJECT_DIR"), std::string::npos) << project; EXPECT_NE(project.find("env -u XLINGS_ACTIVE_SUBOS PATH="), std::string::npos) << project; EXPECT_NE(project.find("XLINGS_PROJECT_DIR="), std::string::npos) << project; @@ -700,8 +793,8 @@ TEST(XlingsInvocationEnv, TheShellsActiveSubosIsNeverInherited) { env::ScopedEnv prior("XLINGS_ACTIVE_SUBOS", std::string("shell-subos")); { auto e = xlings_env(""); - mcpp::xlings::ScopedInvocationEnv scope(e); - (void)mcpp::xlings::build_command_prefix(e); + mcpp::provider::xlings::ScopedInvocationEnv scope(e); + (void)mcpp::provider::xlings::build_command_prefix(e); #if defined(_WIN32) EXPECT_FALSE(env::get("XLINGS_ACTIVE_SUBOS").has_value()); #endif diff --git a/tests/unit/test_xlings_address_set.cpp b/tests/unit/test_xlings_address_set.cpp index b756c2423..182af0422 100644 --- a/tests/unit/test_xlings_address_set.cpp +++ b/tests/unit/test_xlings_address_set.cpp @@ -1,9 +1,9 @@ #include import std; -import mcpp.xlings.address_set; +import mcpp.provider.address_set; -namespace addrset = mcpp::xlings::addrset; +namespace addrset = mcpp::provider::addrset; namespace {