initial commit
This commit is contained in:
@@ -0,0 +1,97 @@
|
||||
# Ubuntu 24.04 の作業専用 VM
|
||||
|
||||
この手順は Arch 開発ホスト上の既存 KVM/QEMU を使い、Ubuntu のカーネル・AppArmor・ユーザー空間で DocView をビルドして試験するためのもの。コンテナーの Ubuntu ユーザー空間だけの試験とは区別する。ホストへパッケージをインストールせず、ホストの既存文書・ホーム・Docker socket を guest に共有しない。
|
||||
|
||||
作業先の既定値は `build-ubuntu-vm`。変更する場合は全コマンドで `DOCVIEW_VM_WORK` を同じ絶対パスに設定する。`private/` の SSH 鍵・seed・known_hosts・serial console は報告物へ含めない。作業先は `.gitignore` の `/build*/` により追跡対象外。4 vCPU、8 GiB RAM、24 GiB の差分ディスクを使い、SSH は `127.0.0.1:22224` のみ。QMP socket も作業先だけに置く。
|
||||
|
||||
## 固定入力
|
||||
|
||||
- Ubuntu cloud image: `https://cloud-images.ubuntu.com/noble/20260911/noble-server-cloudimg-amd64.img`、625,256,960 bytes、SHA-256 `612b2c0cc1bc413a6cb8c38fd611794caf0f2b436c50013d8b3794db12ad7354`。同じ公式ディレクトリーの `SHA256SUMS.gpg` を、[Canonical 公開 fingerprint](https://ubuntu.com/docs/public-images/public-images-how-to/verify-image-checksum/) `D2EB44626FDDC30B513D5BB71A5D6C4C7DB87C81` に照合する。
|
||||
- ホスト補助の qemu-img 11.1.1-2: Arch の公開 package を workspace へだけ展開。固定 SHA-256 とホストに既存の読み取り専用パッケージ鍵リングによる署名検証を行う。既存 QEMU 11.1.1、OVMF `/usr/share/edk2-ovmf/x64/`、GnuPG、bsdtar を前提とする。
|
||||
- ISO 作成の pycdlib 1.14.0: PyPI wheel 213,201 bytes を取得し、公開 SHA-256 を照合して workspace にだけ展開。
|
||||
- Qt 6.11.2: [公式 basic SDK metadata](https://download.qt.io/online/qtsdkrepository/linux_x64/desktop/qt6_6112/qt6_6112/Updates.xml) と [公式 WebEngine extension metadata](https://download.qt.io/online/qtsdkrepository/linux_x64/extensions/qtwebengine/6112/x86_64/Updates.xml)。basic、WebChannel、Positioning、WebEngine の実 archive 合計 325,996,106 bytes。Qt 公開 SHA-1 sidecar と照合し、各ファイルの SHA-256 も記録する。Qt Creator と debug symbols は取得しない。これらは公式ファイル名で RHEL 9.6 build とされており、Arch パッケージと同一バイナリーではない。
|
||||
- qpdf 12.4.1: [公式 release](https://github.com/qpdf/qpdf/releases/tag/v12.4.1) source 19,713,921 bytes、release API の SHA-256 と照合。
|
||||
- libzip 1.11.4: [公式 source](https://libzip.org/download/)、793,340 bytes。固定 SHA-256 は KDE Ark の [公開ビルド定義](https://github.com/flathub/org.kde.ark/blob/master/org.kde.ark.json) とも対応する。上流が detached signature を公開しているとは主張しない。
|
||||
- PDFium 155.0.8057.0: 既存 `.deps/pdfium` を source snapshot に含め、リポジトリーの lock と supplemental notice 検証を通常 CMake で維持する。
|
||||
|
||||
## 作成と起動
|
||||
|
||||
リポジトリー root から実行する。取得・VM 起動は通常の OS 権限が必要。
|
||||
|
||||
```sh
|
||||
python3 tests/ubuntu_vm/prepare_downloads.py
|
||||
python3 tests/ubuntu_vm/verify_inputs.py
|
||||
python3 tests/ubuntu_vm/create_guest.py
|
||||
python3 tests/ubuntu_vm/run_guest.py
|
||||
```
|
||||
|
||||
最後のコマンドは VM の前景プロセスを維持する。別ターミナルから `python3 tests/ubuntu_vm/probe_guest.py` で起動・SSH・OS・Landlock・AppArmor を記録する。初回の cloud-init が完了するまで待つ。seed はパスワード認証と root SSH を無効にし、作業専用 `docview` user と新規公開鍵を設定する。guest の root 作業には、この guest user の sudo だけを使う。
|
||||
|
||||
`qmp.py query-status`、`qmp.py stop`、`qmp.py cont` で状態確認・一時停止・再開、`qmp.py system_powerdown` で正常終了できる。性能測定の並行実行を避ける場合は guest を停止する。正常終了して QEMU が終了してからだけディスクをコピー・削除する。
|
||||
|
||||
## 依存環境と本体
|
||||
|
||||
1. guest 内で APT update、必要 package の `--assume-no` simulation、`--no-install-recommends` install を行う。固定 package 一覧は [apt-packages.json](apt-packages.json)、初回の実行記録は作業先 `metadata/apt-packages.json`、候補・取得量・実配置ログは `logs/apt-*.txt`。Ubuntu 自身の署名付きリポジトリーを使う。ホスト APT/Pacman は変更しない。
|
||||
2. `python3 tests/ubuntu_vm/fetch_sdk.py` で固定 SDK/source を取得する。`sdk-downloads/` と `metadata/sdk-downloads.json`、`install_guest_sdk.py` を専用 SSH で guest の `~/incoming/` へコピーする。`python3 incoming/install_guest_sdk.py` を guest 内で実行する。
|
||||
3. Qt は `/opt/docview-qt/6.11.2/gcc_64`、qpdf/libzip は `/opt/docview-deps` へ配置される。QtWebEngineProcess のこの固定パスだけを対象に、Ubuntu 標準と同じ `flags=(unconfined) { userns, }` 形式の AppArmor profile を追加する。これは Chromium 内部の sandbox を無効にする設定ではない。`kernel.apparmor_restrict_unprivileged_userns=1` を維持し、`--no-sandbox` 等は使わない。
|
||||
4. 本体ソースが安定した時点で `python3 tests/ubuntu_vm/snapshot.py --name source-snapshot` を実行する。既存出力があれば別名にする。snapshot archive と manifest を guest の `~/incoming/source-snapshot.tar.gz`、`source-snapshot.json` へ転送し、`build_guest.py` を同ディレクトリーへコピーして実行する。全ファイルの SHA-256 を再照合してから `~/docview-source`、`~/docview-build` でビルドする。
|
||||
|
||||
guest での通常 build/test 環境は次のとおり。
|
||||
|
||||
```sh
|
||||
export PATH=/opt/docview-deps/bin:/opt/docview-qt/6.11.2/gcc_64/bin:$PATH
|
||||
export PKG_CONFIG_PATH=/opt/docview-deps/lib/pkgconfig
|
||||
export LD_LIBRARY_PATH=/opt/docview-deps/lib:/opt/docview-qt/6.11.2/gcc_64/lib
|
||||
export QT_QPA_PLATFORM=xcb
|
||||
export QT_QUICK_BACKEND=software
|
||||
mkdir -p ~/validation/ctest
|
||||
dbus-run-session -- xvfb-run -a -s '-screen 0 1920x1080x24' \
|
||||
ctest --test-dir ~/docview-build --output-on-failure \
|
||||
--output-junit ~/validation/ctest/tests.xml
|
||||
```
|
||||
|
||||
guest 内の実版・kernel・LSM、入出力の hash、CTest ログは個別に保存する。後続ソース変更を同期する際は前の manifest/結果を保持し、`src`/`qml` 差分を明示する。この VM の Xvfb/Sway headless の結果は、物理 GPU・実ディスプレイ・人の IME 操作・配布ライセンス確認まで合格した証拠にはしない。Arch 用 tar/告知 manifest の一致を、異なる Ubuntu/公式 Qt SDK の配布証拠へ流用しない。
|
||||
|
||||
|
||||
## 差分と追加検証
|
||||
|
||||
`sync_patch.py --name <新しい名前> <repository相対file...>` は指定したsourceだけを転送し、guestの変更前本文と前後SHA-256を `~/validation/source-deltas/` へ保存する。既存名は再使用しない。秘密鍵・設定・ユーザー文書はsource集合へ含めない。
|
||||
|
||||
追加Wayland試験にはguestの `sway` packageを使う。runnerは独立D-Busと一時XDG保存先、1920×1080のheadless/pixman compositorを作る。実際のサイズ変更がQt側とcompositor側で確定してから次の操作へ進む。小さい出力では縦960pxの試験前提を満たせない。
|
||||
|
||||
```sh
|
||||
python3 ~/docview-source/tests/run_wayland_validation.py --build-dir ~/docview-build --output ~/validation/wayland/gui --mode gui
|
||||
python3 ~/docview-source/tests/run_wayland_validation.py --build-dir ~/docview-build --output ~/validation/wayland/smoke --mode smoke
|
||||
cmake --install ~/docview-build --prefix ~/docview-install
|
||||
```
|
||||
|
||||
UbuntuのPython 3.12では、Arch package cacheのzstd専用fixture 5件を明示skipする。Python 3.14で実行したArchの35件を、Ubuntuの35件成功として扱わない。Ubuntuでは残り30件と、別のinstalled runtime inventoryを検査する。
|
||||
|
||||
## 配置先の検査
|
||||
|
||||
guest内で、install済みの本体に対して以下を実行できる。出力先は新しいdirectoryを選ぶ。
|
||||
|
||||
```sh
|
||||
python3 ~/docview-source/tools/collect_ubuntu_validation_runtime.py \
|
||||
--prefix ~/docview-install \
|
||||
--qtpaths /opt/docview-qt/6.11.2/gcc_64/bin/qtpaths \
|
||||
--dependency-prefix /opt/docview-deps \
|
||||
--input-manifest ~/incoming/sdk-downloads.json \
|
||||
--source-root ~/dependency-sources/qpdf-12.4.1 \
|
||||
--source-root ~/dependency-sources/libzip-1.11.4 \
|
||||
--output ~/validation/new-installed-runtime
|
||||
```
|
||||
|
||||
collectorは選択した信頼済みbuild/SDKにだけ`ldd`を実行し、ファイル・symbolic link・実体hash・依存解決・dpkg所有packageを確認する。任意の未信頼実行ファイルを調べるためには使わない。runtimeをコピーした配布物は作らず、告知の欠落と完全なChromium告知未確認も記録する。SDKのSBOMには10MBを越えるものがあるため、metadataは1件16MiB・合計64MiB、告知本文は1件8MiB・合計32MiBの別上限を使う。
|
||||
|
||||
本体起動時は上記の`LD_LIBRARY_PATH`を維持し、`tests/smoke.py --binary ~/docview-install/bin/docview --output ~/validation/new-installed-smoke`を専用Xvfb内で実行する。これは別prefixへのbuild/install検査であり、自己完結packageの配布確認ではない。
|
||||
|
||||
[保存済みの実行結果](../results/ui-final/ubuntu/README.md)には失敗と修正後の成功、配置後の6条件起動、依存台帳を区別して残した。試験の途中でフォントRPCの待機を修正したため、現行の`record_guest_validation.py`はこの検証で使った`~/validation/ctest-font-final/`を照合対象にする。
|
||||
|
||||
追加の限定試験では`--ctest-name canvas-ctest --output-name canvas-build-record.json --expected-groups 2 --scope-note 'Canvas focused tests'`のように記録先と範囲を指定する。既存の全体試験の台帳を上書きせず、変更後のsource・実行ファイル・JUnit・ログのhashを別に保存する。WaylandのCanvas単独実行は`run_wayland_validation.py --mode gui --suite pdf_canvas`で選択できる。
|
||||
|
||||
## Ubuntu用deb
|
||||
|
||||
[パッケージ生成手順](../../docs/UBUNTU-PACKAGE.md)と[実行結果](../results/ubuntu-package/README.md)を追加した。`package_smoke.py --output <新規先>`は専用VMでインストール済みの`/usr/bin/docview`を使い、元SDK・依存・build/installを私有mount namespaceで隠して通常ユーザーでX11/Wayland各6条件を検査する。`package_lifecycle.py --smoke <成功したsmoke先> --output <新規先>`は、そのバイナリー一致を確認してDocViewだけをremove/purgeし、新しく作ったユーザーデータprobeの保持を検査する。後者の検証後、VM内のDocViewパッケージは未インストールになる。開発環境のSDKとソースは保持する。
|
||||
|
||||
クリーンOS検証は`DOCVIEW_VM_WORK=build-ubuntu-package-clean`で同じ読取専用base imageから別overlayを作り、開発VMを停止してから同じSSHポートを使う。SDKやbuildを転送せず、debと試験入力だけを転送した。`clean_package.py --phase install|smoke|remove --archive <検証済みdeb> --output <新規記録先>`を順に実行する。install phaseは宣言依存で全ELFが解決することを確認してからGUI試験ツールを追加する。[新規OSの結果](../results/ubuntu-package/clean-vm/README.md)に元image・転送内容・apt変更・3 phaseを保存した。
|
||||
@@ -0,0 +1,56 @@
|
||||
[
|
||||
"build-essential",
|
||||
"cmake",
|
||||
"ninja-build",
|
||||
"pkg-config",
|
||||
"libseccomp-dev",
|
||||
"libfontconfig1-dev",
|
||||
"libtomlplusplus-dev",
|
||||
"zlib1g-dev",
|
||||
"libjpeg-dev",
|
||||
"libssl-dev",
|
||||
"libgnutls28-dev",
|
||||
"libbz2-dev",
|
||||
"liblzma-dev",
|
||||
"libzstd-dev",
|
||||
"libgl1-mesa-dev",
|
||||
"libegl1-mesa-dev",
|
||||
"libxkbcommon-x11-0",
|
||||
"libxcb-cursor0",
|
||||
"libxcb-icccm4",
|
||||
"libxcb-image0",
|
||||
"libxcb-keysyms1",
|
||||
"libxcb-render-util0",
|
||||
"libxcb-randr0",
|
||||
"libxcb-shape0",
|
||||
"libxcb-sync1",
|
||||
"libxcb-xfixes0",
|
||||
"libxcb-xinerama0",
|
||||
"libxcb-xkb1",
|
||||
"libxcomposite1",
|
||||
"libxdamage1",
|
||||
"libxrandr2",
|
||||
"libxtst6",
|
||||
"libnss3",
|
||||
"libasound2t64",
|
||||
"libpulse0",
|
||||
"libxss1",
|
||||
"libgbm1",
|
||||
"libopengl0",
|
||||
"libvulkan1",
|
||||
"xvfb",
|
||||
"xauth",
|
||||
"dbus-x11",
|
||||
"fonts-dejavu-core",
|
||||
"fonts-liberation",
|
||||
"fonts-noto-cjk",
|
||||
"locales",
|
||||
"mesa-utils",
|
||||
"weston",
|
||||
"p7zip-full",
|
||||
"python3-pil",
|
||||
"python3-fonttools",
|
||||
"python3-venv",
|
||||
"poppler-utils",
|
||||
"sway"
|
||||
]
|
||||
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Verify and build an explicitly transferred source snapshot inside the guest."""
|
||||
import hashlib,json,os,subprocess,tarfile
|
||||
from pathlib import Path
|
||||
incoming=Path.home()/'incoming';source=Path.home()/'docview-source';build=Path.home()/'docview-build'
|
||||
record=json.loads((incoming/'source-snapshot.json').read_text())
|
||||
with (incoming/'source-snapshot.tar.gz').open('rb') as f:assert hashlib.file_digest(f,'sha256').hexdigest()==record['archiveSha256']
|
||||
source.mkdir(exist_ok=True)
|
||||
with tarfile.open(incoming/'source-snapshot.tar.gz') as t:t.extractall(source,filter='data')
|
||||
for row in record['files']:
|
||||
with (source/row['path']).open('rb') as f:assert hashlib.file_digest(f,'sha256').hexdigest()==row['sha256']
|
||||
env=os.environ.copy();env['PATH']='/opt/docview-deps/bin:/opt/docview-qt/6.11.2/gcc_64/bin:'+env['PATH'];env['PKG_CONFIG_PATH']='/opt/docview-deps/lib/pkgconfig';env['LD_LIBRARY_PATH']='/opt/docview-deps/lib:/opt/docview-qt/6.11.2/gcc_64/lib'
|
||||
print('Verified',len(record['files']),'input files; snapshot',record['archiveSha256'],flush=True)
|
||||
subprocess.run(['cmake','-S',str(source),'-B',str(build),'-G','Ninja','-DCMAKE_BUILD_TYPE=Release','-DCMAKE_PREFIX_PATH=/opt/docview-qt/6.11.2/gcc_64;/opt/docview-deps'],env=env,check=True)
|
||||
subprocess.run(['cmake','--build',str(build),'--parallel','4'],env=env,check=True)
|
||||
@@ -0,0 +1,150 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Test installation or upgrade of a deb on the dedicated VM without an SDK."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
APP = Path('/opt/docview')
|
||||
ARCHIVE_SHA = '978904fd5986694f7b053381dcb6ca1ac07b92884ef9768c320923d179d873e5'
|
||||
|
||||
|
||||
def sha(path):
|
||||
with path.open('rb') as stream:
|
||||
return hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--phase', choices=['install', 'smoke', 'remove'], required=True)
|
||||
parser.add_argument('--archive', type=Path, required=True)
|
||||
parser.add_argument('--archive-sha256', default=ARCHIVE_SHA)
|
||||
parser.add_argument('--upgrade', action='store_true', help='Require an existing DocView package during install')
|
||||
parser.add_argument('--output', type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
assert os.getuid() != 0 and Path.home() == Path('/home/docview')
|
||||
assert re.fullmatch('[0-9a-f]{64}', args.archive_sha256)
|
||||
assert sha(args.archive) == args.archive_sha256
|
||||
for name in ['/opt/docview-qt', '/opt/docview-deps', '/home/docview/docview-build', '/home/docview/docview-install']:
|
||||
assert not Path(name).exists(), name
|
||||
output = args.output.resolve() / args.phase
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
record = {'success': False, 'phase': args.phase, 'archiveSha256': args.archive_sha256,
|
||||
'installationMode': 'upgrade' if args.upgrade else 'fresh-install',
|
||||
'runnerSha256': sha(Path(__file__)), 'commands': [], 'originalSdkOrBuildPresent': False,
|
||||
'osRelease': Path('/etc/os-release').read_text(), 'uid': os.getuid()}
|
||||
|
||||
def run(name, command, environment=None, allowed=(0,)):
|
||||
with (output / (name + '.log')).open('w') as stream:
|
||||
result = subprocess.run(command, env=environment, stdout=stream, stderr=subprocess.STDOUT, timeout=900)
|
||||
record['commands'].append({'name': name, 'command': command, 'exitCode': result.returncode,
|
||||
'logSha256': sha(output / (name + '.log'))})
|
||||
assert result.returncode in allowed, name
|
||||
return (output / (name + '.log')).read_text()
|
||||
|
||||
def packages(name):
|
||||
return run(name, ['dpkg-query', '-W', '-f=${Package}\t${Version}\t${db:Status-Status}\n'])
|
||||
|
||||
minimal = {'PATH': '/usr/bin:/bin', 'HOME': '/home/docview', 'USER': 'docview',
|
||||
'LOGNAME': 'docview', 'LANG': 'C.UTF-8'}
|
||||
try:
|
||||
if args.phase == 'install':
|
||||
before = packages('packages-before')
|
||||
installed = [line for line in before.splitlines() if line.startswith('docview\t')]
|
||||
if args.upgrade:
|
||||
assert APP.is_dir() and Path('/usr/bin/docview').is_file()
|
||||
assert len(installed) == 1 and installed[0].endswith('\tinstalled')
|
||||
record['previousPackage'] = installed[0]
|
||||
record['previousManifestSha256'] = sha(APP / 'share/doc/docview/package-manifest.json')
|
||||
else:
|
||||
assert not APP.exists() and not Path('/usr/bin/docview').exists() and not installed
|
||||
run('apt-update', ['sudo', 'apt-get', 'update'])
|
||||
run('apt-plan', ['sudo', 'apt-get', '-s', '--no-install-recommends', 'install', str(args.archive)])
|
||||
run('apt-install', ['sudo', 'env', 'DEBIAN_FRONTEND=noninteractive', 'apt-get', '-y',
|
||||
'--no-install-recommends', 'install', str(args.archive)])
|
||||
after = packages('packages-after-install')
|
||||
rows = json.loads((APP / 'share/doc/docview/package-manifest.json').read_text())['files']
|
||||
for row in rows:
|
||||
if row['path'].startswith('DEBIAN/'): continue
|
||||
path = Path('/') / row['path']; info = path.stat()
|
||||
assert info.st_uid == 0 and info.st_gid == 0
|
||||
assert oct(info.st_mode & 0o7777) == row['mode']
|
||||
assert info.st_size == row['size'] and sha(path) == row['sha256']
|
||||
environment = {**minimal, 'LD_LIBRARY_PATH': '/opt/docview/lib:/opt/docview/qt/lib'}
|
||||
dependencies = []
|
||||
for path in sorted(APP.rglob('*')):
|
||||
if not path.is_file(): continue
|
||||
with path.open('rb') as stream:
|
||||
if stream.read(4) != b'\x7fELF': continue
|
||||
result = subprocess.run(['ldd', str(path)], env=environment, capture_output=True, text=True, timeout=30)
|
||||
assert result.returncode == 0 and 'not found' not in result.stdout, str(path) + result.stdout
|
||||
resolved = re.findall(r'(?:=>\s+)?(/[^\s]+)\s+\(', result.stdout)
|
||||
assert resolved or result.stdout.strip() == 'statically linked'
|
||||
assert all(name.startswith(('/opt/docview/', '/lib/', '/lib64/', '/usr/lib/')) for name in resolved)
|
||||
dependencies.append({'path': str(path), 'sha256': sha(path), 'resolved': sorted(set(resolved))})
|
||||
record['elfDependenciesBeforeTestHelperInstallStep'] = dependencies
|
||||
record['installedFilesVerified'] = sum(not row['path'].startswith('DEBIAN/') for row in rows)
|
||||
record['executables'] = {name: sha(APP / 'bin' / name) for name in ['docview', 'docview-pdf-worker', 'docview-archive-worker']}
|
||||
profiles = run('apparmor-installed', ['sudo', 'cat', '/sys/kernel/security/apparmor/profiles'])
|
||||
assert 'docview-bundled-qtwebengine ' in profiles and 'docview-qtwebengine ' not in profiles
|
||||
assert run('userns-before', ['sysctl', '-n', 'kernel.apparmor_restrict_unprivileged_userns']).strip() == '1'
|
||||
record['newInstalledPackages'] = sorted(set(after.splitlines()) - set(before.splitlines()))
|
||||
# Dependency resolution is recorded before adding test infrastructure.
|
||||
run('apt-test-helpers', ['sudo', 'env', 'DEBIAN_FRONTEND=noninteractive', 'apt-get', '-y',
|
||||
'--no-install-recommends', 'install', 'xvfb', 'xauth', 'sway', 'dbus-x11'])
|
||||
packages('packages-after-test-helpers')
|
||||
elif args.phase == 'smoke':
|
||||
installed = json.loads((args.output / 'install/report.json').read_text())
|
||||
assert installed['success']
|
||||
record['executables'] = installed['executables']
|
||||
record['launcherSha256'] = sha(Path('/usr/bin/docview'))
|
||||
record['smokeSourceSha256'] = sha(ROOT / 'tests/smoke.py')
|
||||
record['waylandRunnerSha256'] = sha(ROOT / 'tests/run_wayland_validation.py')
|
||||
for name, digest in installed['executables'].items(): assert sha(APP / 'bin' / name) == digest
|
||||
run('x11', ['dbus-run-session', '--', 'xvfb-run', '-a', '-s', '-screen 0 1100x760x24', sys.executable,
|
||||
str(ROOT / 'tests/smoke.py'), '--binary', '/usr/bin/docview', '--output', str(output / 'x11')],
|
||||
{**minimal, 'QT_QPA_PLATFORM': 'xcb', 'QT_QUICK_BACKEND': 'software', 'QTWEBENGINE_CHROMIUM_FLAGS': '--disable-gpu'})
|
||||
run('wayland', [sys.executable, str(ROOT / 'tests/run_wayland_validation.py'), '--build-dir', '/opt/docview/bin',
|
||||
'--smoke-binary', '/usr/bin/docview', '--output', str(output / 'wayland'), '--mode', 'smoke'], minimal)
|
||||
for relative in ['x11/results.json', 'wayland/smoke/results.json']:
|
||||
cases = json.loads((output / relative).read_text())
|
||||
assert len(cases) == 6 and all(row['state'] == 'Ready' and row['binarySha256'] == record['launcherSha256'] for row in cases)
|
||||
for name, digest in installed['executables'].items(): assert sha(APP / 'bin' / name) == digest
|
||||
record.update(smokeConditions=12, executableBytesUnchanged=True, callerRuntimeVariablesAbsent=True)
|
||||
else:
|
||||
assert json.loads((args.output / 'smoke/report.json').read_text())['success']
|
||||
probes = []
|
||||
try:
|
||||
for directory in ['.config/docview', '.local/state/docview', '.local/share/docview', 'validation/clean-user-document']:
|
||||
path = Path.home() / directory / 'clean-package-probe.txt'; path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open('x') as stream: stream.write('保持する文書と設定\n')
|
||||
probes.append({'path': str(path), 'sha256': sha(path)})
|
||||
run('apt-remove', ['sudo', 'apt-get', '-y', 'remove', 'docview'])
|
||||
assert not APP.exists() and not Path('/usr/bin/docview').exists()
|
||||
assert not Path('/usr/share/applications/docview.desktop').exists()
|
||||
assert Path('/etc/apparmor.d/docview').is_file()
|
||||
assert 'docview-bundled-qtwebengine ' not in run('apparmor-removed', ['sudo', 'cat', '/sys/kernel/security/apparmor/profiles'])
|
||||
assert all(sha(Path(row['path'])) == row['sha256'] for row in probes)
|
||||
run('apt-purge', ['sudo', 'apt-get', '-y', 'purge', 'docview'])
|
||||
assert not Path('/etc/apparmor.d/docview').exists()
|
||||
assert all(sha(Path(row['path'])) == row['sha256'] for row in probes)
|
||||
assert run('userns-after', ['sysctl', '-n', 'kernel.apparmor_restrict_unprivileged_userns']).strip() == '1'
|
||||
record.update(payloadRemoved=True, profileUnloaded=True, conffilePurged=True,
|
||||
userProbesPreserved=probes, kernelRestrictionUnchanged=True)
|
||||
finally:
|
||||
for row in probes:
|
||||
path = Path(row['path'])
|
||||
if path.is_file() and sha(path) == row['sha256']: path.unlink()
|
||||
record['success'] = True
|
||||
finally:
|
||||
(output / 'report.json').write_text(json.dumps(record, indent=2, sort_keys=True) + '\n')
|
||||
print(json.dumps({'phase': args.phase, 'success': record['success']}))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build/package the pinned v2 candidate only in the dedicated Ubuntu guest.
|
||||
|
||||
Input transfer is recorded separately. Existing source/build/install/provider
|
||||
prefixes and earlier evidence are preserved. This does not stop the VM.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
HOME = Path('/home/docview')
|
||||
PREFIX = HOME / 'validation/pdfium-context-prefix'
|
||||
BUILD = HOME / 'docview-context-build'
|
||||
INSTALL = HOME / 'docview-context-install'
|
||||
RESULTS = HOME / 'validation/pdfium-context-package'
|
||||
VERSION = '0.1.0~validation4'
|
||||
|
||||
|
||||
def sha(path):
|
||||
with path.open('rb') as stream: return hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--phase', choices=('build', 'package', 'smoke'), required=True)
|
||||
parser.add_argument('--run-name', required=True)
|
||||
args = parser.parse_args()
|
||||
assert os.getuid() != 0 and Path.home() == HOME and ROOT == HOME / 'docview-context-source'
|
||||
assert re.fullmatch('[a-z0-9-]{1,64}', args.run_name)
|
||||
output = RESULTS / args.run_name; output.mkdir(parents=True, exist_ok=False)
|
||||
env = {**os.environ, 'PATH': '/opt/docview-deps/bin:/opt/docview-qt/6.11.2/gcc_64/bin:' + os.environ['PATH'],
|
||||
'PKG_CONFIG_PATH': '/opt/docview-deps/lib/pkgconfig',
|
||||
'LD_LIBRARY_PATH': '/opt/docview-deps/lib:/opt/docview-qt/6.11.2/gcc_64/lib',
|
||||
'QT_QPA_PLATFORM': 'xcb', 'QT_QUICK_BACKEND': 'software',
|
||||
'QTWEBENGINE_CHROMIUM_FLAGS': '--disable-gpu', 'QT_SCALE_FACTOR': '1',
|
||||
'QT_AUTO_SCREEN_SCALE_FACTOR': '0', 'XDG_SESSION_TYPE': 'x11'}
|
||||
for key in ('LD_PRELOAD', 'LD_AUDIT', 'QTWEBENGINE_DISABLE_SANDBOX', 'WAYLAND_DISPLAY'):
|
||||
assert not env.get(key), key
|
||||
preserved = [HOME / 'docview-source/.deps/pdfium/lib/libpdfium.so', HOME / 'docview-build/docview',
|
||||
HOME / 'docview-build/docview-pdf-worker', HOME / 'docview-install/bin/docview']
|
||||
before = {str(path): sha(path) for path in preserved}
|
||||
report = {'success': False, 'phase': args.phase, 'commands': [], 'runnerSha256': sha(Path(__file__)),
|
||||
'providerAndOriginalBuildBefore': before, 'scope': 'Local Ubuntu candidate integration only; no public release or complete Chromium notice claim'}
|
||||
|
||||
def run(label, command, timeout=1800, environment=env):
|
||||
start = time.monotonic(); log = output / (label + '.log')
|
||||
print('Starting', label, flush=True)
|
||||
with log.open('w') as stream:
|
||||
result = subprocess.run(command, cwd=ROOT, env=environment, stdout=stream, stderr=subprocess.STDOUT, timeout=timeout)
|
||||
report['commands'].append({'name': label, 'command': list(map(str, command)), 'exitCode': result.returncode,
|
||||
'seconds': time.monotonic() - start, 'logSha256': sha(log)})
|
||||
print('Finished', label, result.returncode, flush=True)
|
||||
assert result.returncode == 0, label + ' failed; inspect ' + str(log)
|
||||
|
||||
try:
|
||||
sys.path.insert(0, str(ROOT / 'tools'))
|
||||
from verify_pdfium_candidate import verify_prefix, verify_installed
|
||||
report['candidatePrefix'], _ = verify_prefix(PREFIX, PREFIX / 'lib/libpdfium.so', PREFIX / 'include')
|
||||
if args.phase == 'build':
|
||||
run('configure', ['cmake', '-S', str(ROOT), '-B', str(BUILD), '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=Release',
|
||||
'-DCMAKE_PREFIX_PATH=/opt/docview-qt/6.11.2/gcc_64;/opt/docview-deps',
|
||||
'-DDOCVIEW_PDFIUM_ROOT=' + str(PREFIX)])
|
||||
run('build', ['cmake', '--build', str(BUILD), '--parallel', '4'])
|
||||
run('candidate-integration-tests', [sys.executable, str(ROOT / 'tests/test_pdfium_candidate_integration.py'), '-v'])
|
||||
run('ctest', ['dbus-run-session', '--', 'xvfb-run', '-a', '-s', '-screen 0 1920x1080x24',
|
||||
'ctest', '--test-dir', str(BUILD), '--output-on-failure', '--output-junit', str(output / 'tests.xml')])
|
||||
shutil.copyfile(BUILD / 'Testing/Temporary/LastTest.log', output / 'LastTest.log')
|
||||
run('install', ['cmake', '--install', str(BUILD), '--prefix', str(INSTALL)])
|
||||
report['installedCandidate'] = verify_installed(INSTALL)
|
||||
report['binaries'] = {name: sha(BUILD / name) for name in ('docview', 'docview-pdf-worker', 'docview-archive-worker')}
|
||||
elif args.phase == 'package':
|
||||
package_command = [sys.executable, str(ROOT / 'tools/package_ubuntu.py'), '--prefix', str(INSTALL),
|
||||
'--qtpaths', '/opt/docview-qt/6.11.2/gcc_64/bin/qtpaths', '--dependency-prefix', '/opt/docview-deps',
|
||||
'--source-root', str(HOME / 'dependency-sources/qpdf-12.4.1'),
|
||||
'--source-root', str(HOME / 'dependency-sources/libzip-1.11.4'),
|
||||
'--input-manifest', str(HOME / 'incoming/sdk-downloads.json'),
|
||||
'--sdk-notices', str(ROOT / 'tests/results/source-archives/qt-sdk-notices-final'),
|
||||
'--sdk-supplement', str(ROOT / 'tests/results/qt-notice-supplement/collection'),
|
||||
'--qt-licenses', str(ROOT / 'tests/results/ubuntu-package/inputs/qt-licenses'), '--version', VERSION]
|
||||
for name in ('package', 'repeat'):
|
||||
destination = output / name
|
||||
run(name, [*package_command, '--output', str(destination)])
|
||||
run(name + '-verify', [sys.executable, str(ROOT / 'tools/verify_ubuntu_package.py'), '--archive',
|
||||
str(destination / ('docview_' + VERSION + '_amd64.deb')), '--output', str(destination / 'verification.json')])
|
||||
archive = 'docview_' + VERSION + '_amd64.deb'
|
||||
report['archiveSha256'] = sha(output / 'package' / archive)
|
||||
report['repeatSha256'] = sha(output / 'repeat' / archive)
|
||||
assert report['archiveSha256'] == report['repeatSha256']
|
||||
report['archive'] = str(output / 'package' / archive)
|
||||
else:
|
||||
package = json.loads((RESULTS / 'package/report.json').read_text())
|
||||
assert package['success'] and sha(Path(package['archive'])) == package['archiveSha256']
|
||||
run('apt-install', ['sudo', 'env', 'DEBIAN_FRONTEND=noninteractive', 'apt-get', '-y',
|
||||
'--no-install-recommends', 'install', package['archive']])
|
||||
report['installedCandidate'] = verify_installed(Path('/opt/docview'))
|
||||
run('installed-smoke', [sys.executable, str(ROOT / 'tests/ubuntu_vm/package_smoke.py'),
|
||||
'--output', str(output / 'installed-smoke'), '--hide-prefix', str(BUILD), '--hide-prefix', str(INSTALL),
|
||||
'--hide-prefix', str(PREFIX), '--hide-prefix', str(HOME / 'validation/pdfium-intent-context')],
|
||||
environment={key: value for key, value in os.environ.items() if key not in (
|
||||
'LD_LIBRARY_PATH', 'QT_PLUGIN_PATH', 'QT_QPA_PLATFORM_PLUGIN_PATH', 'QML_IMPORT_PATH',
|
||||
'QML2_IMPORT_PATH', 'QTWEBENGINEPROCESS_PATH', 'QTWEBENGINE_DISABLE_SANDBOX')})
|
||||
run('lifecycle', [sys.executable, str(ROOT / 'tests/ubuntu_vm/package_lifecycle.py'), '--smoke',
|
||||
str(output / 'installed-smoke'), '--output', str(output / 'lifecycle')])
|
||||
report['packagePurged'] = not Path('/opt/docview').exists() and not Path('/etc/apparmor.d/docview').exists()
|
||||
assert report['packagePurged']
|
||||
after = {str(path): sha(path) for path in preserved}
|
||||
report['providerAndOriginalBuildAfter'] = after
|
||||
assert before == after
|
||||
report['originalsUnchanged'] = report['success'] = True
|
||||
finally:
|
||||
report['evidenceSha256'] = {str(path.relative_to(output)): sha(path)
|
||||
for path in sorted(output.rglob('*')) if path.is_file() and path.suffix != '.deb'}
|
||||
(output / 'report.json').write_text(json.dumps(report, indent=2) + '\n')
|
||||
print(json.dumps({'phase': args.phase, 'success': report['success']}))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Disposable local Ubuntu validation VM helper; no host package installation."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import os,sys,subprocess,shutil,io,json
|
||||
r=Path(os.environ.get('DOCVIEW_VM_WORK', 'build-ubuntu-vm')).resolve()
|
||||
sys.path.insert(0,str(r/'tools/python'))
|
||||
import pycdlib
|
||||
key=r/'private/id_ed25519'
|
||||
if not key.exists():subprocess.run(['ssh-keygen','-q','-t','ed25519','-N','','-C','docview-ubuntu-vm-local','-f',str(key)],check=True)
|
||||
os.chmod(key,0o600)
|
||||
public=key.with_suffix('.pub').read_text().strip()
|
||||
userdata='#cloud-config\nhostname: docview-ubuntu2404\nmanage_etc_hosts: true\nssh_pwauth: false\ndisable_root: true\nssh_quiet_keygen: true\nno_ssh_fingerprints: true\nssh_publish_hostkeys:\n enabled: false\nusers:\n - name: docview\n groups: [adm, sudo]\n shell: /bin/bash\n lock_passwd: true\n sudo: ALL=(ALL) NOPASSWD:ALL\n ssh_authorized_keys:\n - '+public+'\n'
|
||||
userdata += '\nssh:\n emit_keys_to_console: false\nruncmd:\n - [sh, -c, \"systemctl restart ssh.service; echo DOCVIEW_SSH_DIAG; ip -4 addr show; ss -ltn; systemctl --no-pager status ssh.service; journalctl -u ssh.service -n 20 --no-pager\"]\n'
|
||||
metadata='instance-id: docview-ubuntu2404-20260919b\nlocal-hostname: docview-ubuntu2404\n'
|
||||
iso=pycdlib.PyCdlib();iso.new(interchange_level=3,joliet=3,rock_ridge='1.09',vol_ident='CIDATA')
|
||||
for data,short,long in [(userdata,'USER_DAT.;1','user-data'),(metadata,'META_DAT.;1','meta-data')]:
|
||||
b=data.encode();iso.add_fp(io.BytesIO(b),len(b),iso_path='/'+short,rr_name=long,joliet_path='/'+long)
|
||||
iso.write(str(r/'private/seed.iso'));iso.close();os.chmod(r/'private/seed.iso',0o600)
|
||||
qimg=str(r/'tools/usr/bin/qemu-img')
|
||||
subprocess.run([qimg,'--version'],check=True)
|
||||
base=r/'downloads/noble-server-cloudimg-amd64.img';base.chmod(0o444)
|
||||
overlay=r/'guest.qcow2'
|
||||
if not overlay.exists():subprocess.run([qimg,'create','-f','qcow2','-F','qcow2','-b',str(base),str(overlay),'24G'],check=True)
|
||||
subprocess.run([qimg,'check',str(overlay)],check=True)
|
||||
vars=r/'OVMF_VARS.4m.fd'
|
||||
if not vars.exists():shutil.copyfile('/usr/share/edk2-ovmf/x64/OVMF_VARS.4m.fd',vars)
|
||||
print('Created dedicated SSH key, seed ISO and 24 GiB qcow2 overlay; no key contents logged.')
|
||||
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Download the fixed SDK/source inputs into a disposable VM workspace.
|
||||
|
||||
Qt's public repository provides SHA-1 sidecars; record independent SHA-256
|
||||
digests as well. No archives are executed/extracted by this script.
|
||||
"""
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import urllib.request
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--work', type=Path, default=Path('build-ubuntu-vm'))
|
||||
args = parser.parse_args()
|
||||
root = args.work.resolve()
|
||||
downloads = root / 'sdk-downloads'
|
||||
downloads.mkdir(parents=True, exist_ok=True)
|
||||
base = 'https://download.qt.io/online/qtsdkrepository/'
|
||||
repositories = [
|
||||
('qt-base-Updates.xml', 'linux_x64/desktop/qt6_6112/qt6_6112/'),
|
||||
('qt-webengine-Updates.xml', 'linux_x64/extensions/qtwebengine/6112/x86_64/'),
|
||||
]
|
||||
selected = {
|
||||
'qt.qt6.6112.linux_gcc_64',
|
||||
'qt.qt6.6112.addons.qtwebchannel.linux_gcc_64',
|
||||
'qt.qt6.6112.addons.qtpositioning.linux_gcc_64',
|
||||
'extensions.qtwebengine.6112.linux_gcc_64',
|
||||
}
|
||||
inputs = []
|
||||
for metadata, repository in repositories:
|
||||
for package in ET.fromstring((root / 'downloads' / metadata).read_bytes()).findall('PackageUpdate'):
|
||||
name = package.findtext('Name')
|
||||
if name not in selected:
|
||||
continue
|
||||
for archive in package.findtext('DownloadableArchives').split(','):
|
||||
filename = package.findtext('Version') + archive.strip()
|
||||
url = base + repository + name + '/' + filename
|
||||
inputs.append({'name': filename, 'url': url, 'kind': 'qt',
|
||||
'checksumUrl': url + '.sha1', 'checksumAlgorithm': 'sha1'})
|
||||
inputs += [
|
||||
{'name': 'qpdf-12.4.1.tar.gz', 'kind': 'source',
|
||||
'url': 'https://github.com/qpdf/qpdf/releases/download/v12.4.1/qpdf-12.4.1.tar.gz',
|
||||
'checksumAlgorithm': 'sha256',
|
||||
'expected': 'f045aa277be2356ff53a89a8622945958291177d2483afc20ede7c8a8cd3873c',
|
||||
'checksumSource': 'https://api.github.com/repos/qpdf/qpdf/releases/tags/v12.4.1'},
|
||||
{'name': 'libzip-1.11.4.tar.xz', 'kind': 'source',
|
||||
'url': 'https://libzip.org/download/libzip-1.11.4.tar.xz',
|
||||
'checksumAlgorithm': 'sha256',
|
||||
'expected': '8a247f57d1e3e6f6d11413b12a6f28a9d388de110adc0ec608d893180ed7097b',
|
||||
'checksumSource': 'https://github.com/flathub/org.kde.ark/blob/master/org.kde.ark.json'},
|
||||
]
|
||||
|
||||
|
||||
def fetch(item):
|
||||
item = dict(item)
|
||||
target = downloads / item['name']
|
||||
if 'checksumUrl' in item:
|
||||
with urllib.request.urlopen(item['checksumUrl'], timeout=45) as response:
|
||||
checksum = response.read(512).decode().strip().split()[0]
|
||||
if len(checksum) != 40 or any(c not in '0123456789abcdef' for c in checksum):
|
||||
raise ValueError('invalid SHA-1 sidecar')
|
||||
item['expected'] = checksum
|
||||
(downloads / (item['name'] + '.sha1')).write_text(checksum + '\n')
|
||||
if not target.exists():
|
||||
partial = target.with_suffix(target.suffix + '.part')
|
||||
count = 0
|
||||
with urllib.request.urlopen(item['url'], timeout=90) as response, partial.open('wb') as stream:
|
||||
item['finalUrl'] = response.url
|
||||
while block := response.read(1024 * 1024):
|
||||
count += len(block)
|
||||
if count > 250 * 1024 * 1024:
|
||||
raise ValueError('archive exceeds per-file download bound')
|
||||
stream.write(block)
|
||||
partial.replace(target)
|
||||
with target.open('rb') as stream:
|
||||
actual = hashlib.file_digest(stream, item['checksumAlgorithm']).hexdigest()
|
||||
if actual != item['expected']:
|
||||
raise ValueError('checksum mismatch: ' + item['name'])
|
||||
with target.open('rb') as stream:
|
||||
item['sha256'] = hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
item['size'] = target.stat().st_size
|
||||
item['checksumMatched'] = True
|
||||
print(item['name'], item['size'], 'verified', flush=True)
|
||||
return item
|
||||
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as pool:
|
||||
records = list(pool.map(fetch, inputs))
|
||||
(root / 'metadata' / 'sdk-downloads.json').write_text(json.dumps(records, indent=2) + '\n')
|
||||
print('Total verified bytes:', sum(item['size'] for item in records), flush=True)
|
||||
@@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Disposable local Ubuntu validation VM helper; no host package installation."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess,sys,shlex
|
||||
r=Path(os.environ.get('DOCVIEW_VM_WORK', 'build-ubuntu-vm')).resolve()
|
||||
ssh=['ssh','-i',str(r/'private/id_ed25519'),'-p','22224','-o','IdentitiesOnly=yes','-o','BatchMode=yes','-o','StrictHostKeyChecking=yes','-o','UserKnownHostsFile='+str(r/'private/known_hosts'),'-o','ConnectTimeout=5','[email protected]']
|
||||
logname=sys.argv[1]; command=sys.argv[2:]
|
||||
p=subprocess.run(ssh+[shlex.join(command)],capture_output=True,text=True)
|
||||
(r/'logs'/logname).write_text(p.stdout+p.stderr)
|
||||
print('guest command exit',p.returncode,'log',logname)
|
||||
print((p.stdout+p.stderr)[-3000:])
|
||||
raise SystemExit(p.returncode)
|
||||
@@ -0,0 +1,61 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run inside the disposable Ubuntu VM, as its docview user."""
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tarfile
|
||||
|
||||
incoming = Path.home() / 'incoming'
|
||||
qt = Path('/opt/docview-qt/6.11.2/gcc_64')
|
||||
prefix = Path('/opt/docview-deps')
|
||||
records = json.loads((incoming / 'sdk-downloads.json').read_text())
|
||||
for item in records:
|
||||
with (incoming / item['name']).open('rb') as stream:
|
||||
assert hashlib.file_digest(stream, 'sha256').hexdigest() == item['sha256']
|
||||
subprocess.run(['sudo', 'mkdir', '-p', str(qt), str(prefix)], check=True)
|
||||
for item in records:
|
||||
if item['kind'] == 'qt':
|
||||
target = qt / 'lib' if 'icu-linux' in item['name'] else qt
|
||||
subprocess.run(['sudo', '7z', 'x', '-y', '-bso0', '-bsp0', '-o' + str(target),
|
||||
str(incoming / item['name'])], check=True)
|
||||
sources = Path.home() / 'dependency-sources'
|
||||
sources.mkdir(exist_ok=True)
|
||||
for name in ['qpdf-12.4.1.tar.gz', 'libzip-1.11.4.tar.xz']:
|
||||
with tarfile.open(incoming / name) as archive:
|
||||
archive.extractall(sources, filter='data')
|
||||
print('All SDK/source hashes verified again inside guest; extracted into dedicated prefixes.', flush=True)
|
||||
subprocess.run([str(qt / 'bin/qtpaths'), '--query', 'QT_VERSION'], check=True)
|
||||
subprocess.run([str(qt / 'bin/qtpaths'), '--query', 'QT_INSTALL_PREFIX'], check=True)
|
||||
|
||||
for project, extra in [
|
||||
('qpdf-12.4.1', ['-DBUILD_STATIC_LIBS=OFF', '-DBUILD_DOC=OFF', '-DINSTALL_EXAMPLES=OFF']),
|
||||
('libzip-1.11.4', ['-DBUILD_TOOLS=OFF', '-DBUILD_REGRESS=OFF', '-DBUILD_EXAMPLES=OFF', '-DBUILD_DOC=OFF']),
|
||||
]:
|
||||
source = sources / project
|
||||
build = Path.home() / 'dependency-builds' / project
|
||||
subprocess.run(['cmake', '-S', str(source), '-B', str(build), '-G', 'Ninja',
|
||||
'-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_INSTALL_PREFIX=' + str(prefix),
|
||||
'-DCMAKE_INSTALL_LIBDIR=lib', *extra], check=True)
|
||||
targets = ['--target', 'libqpdf', 'qpdf', 'fix-qdf', 'zlib-flate'] if project.startswith('qpdf') else []
|
||||
subprocess.run(['cmake', '--build', str(build), '--parallel', '4', *targets], check=True)
|
||||
if project.startswith('qpdf'):
|
||||
for component in ['lib', 'dev', 'cli']:
|
||||
subprocess.run(['sudo', 'cmake', '--install', str(build), '--component', component], check=True)
|
||||
else:
|
||||
subprocess.run(['sudo', 'cmake', '--install', str(build)], check=True)
|
||||
|
||||
profile = '''# Dedicated fixed Qt SDK in this disposable test VM only.
|
||||
# Same userns opt-in pattern as Ubuntu's QtWebEngineProcess profile.
|
||||
abi <abi/4.0>,
|
||||
include <tunables/global>
|
||||
profile docview-qtwebengine /opt/docview-qt/6.11.2/gcc_64/libexec/QtWebEngineProcess flags=(unconfined) {
|
||||
userns,
|
||||
}
|
||||
'''
|
||||
local = Path.home() / 'docview-qtwebengine.apparmor'
|
||||
local.write_text(profile)
|
||||
subprocess.run(['sudo', 'install', '-m', '0644', str(local),
|
||||
'/etc/apparmor.d/docview-qtwebengine'], check=True)
|
||||
subprocess.run(['sudo', 'apparmor_parser', '-r', '/etc/apparmor.d/docview-qtwebengine'], check=True)
|
||||
print('Guest dependency prefixes and exact QtWebEngineProcess AppArmor userns profile installed.', flush=True)
|
||||
@@ -0,0 +1,89 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Remove/purge the tested local package in the dedicated validation VM."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
|
||||
|
||||
def sha(path):
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--smoke', type=Path, required=True)
|
||||
parser.add_argument('--output', type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
assert os.getuid() != 0 and Path.home() == Path('/home/docview')
|
||||
smoke = json.loads((args.smoke / 'report.json').read_text())
|
||||
assert smoke['success'] and smoke['smokeConditions'] == 12
|
||||
for name, digest in smoke['executables'].items():
|
||||
assert sha(Path('/opt/docview/bin') / name) == digest
|
||||
args.output.mkdir(parents=True, exist_ok=False)
|
||||
record = {'success': False, 'runnerSha256': sha(Path(__file__)),
|
||||
'smokeReportSha256': sha(args.smoke / 'report.json'), 'commands': []}
|
||||
|
||||
def run(name, command, allowed=(0,)):
|
||||
result = subprocess.run(command, capture_output=True, text=True, timeout=180)
|
||||
log = args.output / (name + '.log')
|
||||
log.write_text(result.stdout + result.stderr)
|
||||
record['commands'].append({'name': name, 'command': command, 'exitCode': result.returncode,
|
||||
'logSha256': sha(log)})
|
||||
assert result.returncode in allowed, name
|
||||
return result.stdout
|
||||
|
||||
def profiles(label):
|
||||
return run(label, ['sudo', 'cat', '/sys/kernel/security/apparmor/profiles'])
|
||||
|
||||
bundled = 'docview-bundled-qtwebengine '
|
||||
sentinels = []
|
||||
try:
|
||||
before = profiles('profiles-installed')
|
||||
assert bundled in before and 'docview-qtwebengine ' in before
|
||||
record['installedPackage'] = run('package-installed', ['dpkg-query', '-W',
|
||||
'-f=${Package} ${Version} ${db:Status-Status} ${Installed-Size}\n', 'docview']).strip()
|
||||
assert ' installed ' in record['installedPackage']
|
||||
assert run('userns-before', ['sysctl', '-n', 'kernel.apparmor_restrict_unprivileged_userns']).strip() == '1'
|
||||
for relative in ['.config/docview', '.local/state/docview', '.local/share/docview',
|
||||
'validation/package-user-document']:
|
||||
directory = Path.home() / relative
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
path = directory / 'deb-preservation-probe.txt'
|
||||
with path.open('x') as stream:
|
||||
stream.write('DocView package removal preservation probe — 日本語\n')
|
||||
sentinels.append({'path': str(path), 'sha256': sha(path)})
|
||||
run('apt-remove', ['sudo', 'env', 'DEBIAN_FRONTEND=noninteractive', 'apt-get', '-y', 'remove', 'docview'])
|
||||
after = profiles('profiles-removed')
|
||||
assert bundled not in after and 'docview-qtwebengine ' in after
|
||||
for name in ['/opt/docview', '/usr/bin/docview', '/usr/share/applications/docview.desktop']:
|
||||
assert not Path(name).exists(), name
|
||||
assert Path('/etc/apparmor.d/docview').is_file()
|
||||
assert all(sha(Path(row['path'])) == row['sha256'] for row in sentinels)
|
||||
record['remove'] = {'payloadRemoved': True, 'profileUnloaded': True,
|
||||
'conffilePreserved': True, 'userProbesPreserved': True}
|
||||
run('apt-purge', ['sudo', 'env', 'DEBIAN_FRONTEND=noninteractive', 'apt-get', '-y', 'purge', 'docview'])
|
||||
assert not Path('/etc/apparmor.d/docview').exists()
|
||||
assert all(sha(Path(row['path'])) == row['sha256'] for row in sentinels)
|
||||
assert run('userns-after', ['sysctl', '-n', 'kernel.apparmor_restrict_unprivileged_userns']).strip() == '1'
|
||||
assert Path('/opt/docview-qt/6.11.2/gcc_64/bin/qtpaths').is_file()
|
||||
assert Path('/opt/docview-deps/lib/libqpdf.so.30').is_file()
|
||||
assert Path('/home/docview/docview-install/bin/docview').is_file()
|
||||
assert Path('/home/docview/docview-build/docview').is_file()
|
||||
record['purge'] = {'conffileRemoved': True, 'userProbesPreserved': True}
|
||||
record.update(success=True, userProbes=sentinels, originalDevelopmentPrefixesRestored=True,
|
||||
kernelUserNamespaceRestrictionUnchanged=True)
|
||||
finally:
|
||||
# Delete only these newly-created probes after preserving their hashes.
|
||||
for row in sentinels:
|
||||
path = Path(row['path'])
|
||||
if path.is_file() and sha(path) == row['sha256']:
|
||||
path.unlink()
|
||||
(args.output / 'report.json').write_text(json.dumps(record, indent=2, sort_keys=True) + '\n')
|
||||
print(json.dumps({key: record[key] for key in ('success', 'remove', 'purge')}))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,139 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate the installed local deb with original SDK/build prefixes hidden.
|
||||
|
||||
Runs only in the dedicated Ubuntu guest. Root is used for a private mount
|
||||
namespace; the viewer, compositor, and document workers run as docview.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
APP = Path('/opt/docview')
|
||||
HIDDEN = ['/opt/docview-qt', '/opt/docview-deps', '/home/docview/docview-install',
|
||||
'/home/docview/docview-build']
|
||||
|
||||
|
||||
def sha(path):
|
||||
with path.open('rb') as stream:
|
||||
return hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
|
||||
|
||||
def user_run(output):
|
||||
assert os.getuid() != 0
|
||||
assert not any(name in os.environ for name in ('LD_LIBRARY_PATH', 'QT_PLUGIN_PATH',
|
||||
'QT_QPA_PLATFORM_PLUGIN_PATH', 'QML_IMPORT_PATH', 'QML2_IMPORT_PATH',
|
||||
'QTWEBENGINEPROCESS_PATH', 'QTWEBENGINE_DISABLE_SANDBOX'))
|
||||
hidden = {name: not Path(name).exists() or not any(Path(name).iterdir()) for name in HIDDEN}
|
||||
assert all(hidden.values())
|
||||
manifest = json.loads((APP / 'share/doc/docview/package-manifest.json').read_text())
|
||||
installed = []
|
||||
for row in manifest['files']:
|
||||
if row['path'].startswith('DEBIAN/'):
|
||||
continue
|
||||
path = Path('/') / row['path']
|
||||
value = path.stat()
|
||||
assert value.st_uid == 0 and value.st_gid == 0
|
||||
assert oct(value.st_mode & 0o7777) == row['mode']
|
||||
assert value.st_size == row['size'] and sha(path) == row['sha256']
|
||||
installed.append(row['path'])
|
||||
environment = {**os.environ, 'LD_LIBRARY_PATH': str(APP / 'lib') + ':' + str(APP / 'qt/lib')}
|
||||
dependencies = []
|
||||
for path in sorted(APP.rglob('*')):
|
||||
if not path.is_file(): continue
|
||||
with path.open('rb') as stream:
|
||||
if stream.read(4) != b'\x7fELF': continue
|
||||
result = subprocess.run(['ldd', str(path)], env=environment, capture_output=True, text=True, timeout=30)
|
||||
assert result.returncode == 0 and 'not found' not in result.stdout, str(path)
|
||||
resolved = re.findall(r'(?:=>\s+)?(/[^\s]+)\s+\(', result.stdout)
|
||||
no_needed = False
|
||||
if not resolved:
|
||||
dynamic = subprocess.check_output(['readelf', '-d', str(path)], text=True, timeout=30)
|
||||
no_needed = '(NEEDED)' not in dynamic and result.stdout.strip() == 'statically linked'
|
||||
assert (resolved or no_needed) and all(name.startswith(('/opt/docview/', '/lib/', '/lib64/', '/usr/lib/')) for name in resolved), str(path) + ': ' + result.stdout
|
||||
dependencies.append({'path': str(path), 'sha256': sha(path), 'resolved': sorted(set(resolved)),
|
||||
'noDynamicDependencies': no_needed})
|
||||
identities = {name: sha(APP / 'bin' / name) for name in
|
||||
['docview', 'docview-pdf-worker', 'docview-archive-worker']}
|
||||
launcher = sha(Path('/usr/bin/docview'))
|
||||
record = {'success': False, 'hiddenOriginalPrefixes': hidden, 'uid': os.getuid(),
|
||||
'callerRuntimeVariablesAbsent': True, 'installedFilesVerified': len(installed),
|
||||
'executables': identities, 'launcherSha256': launcher, 'elfDependencies': dependencies,
|
||||
'runnerSha256': sha(Path(__file__)), 'smokeSourceSha256': sha(ROOT / 'tests/smoke.py'),
|
||||
'waylandRunnerSha256': sha(ROOT / 'tests/run_wayland_validation.py'),
|
||||
'scope': 'Dedicated existing Ubuntu VM, private mount namespace, Xvfb and headless Sway/software; not a clean OS or physical desktop'}
|
||||
commands = [
|
||||
('x11', ['dbus-run-session', '--', 'xvfb-run', '-a', '-s', '-screen 0 1100x760x24',
|
||||
sys.executable, str(ROOT / 'tests/smoke.py'), '--binary', '/usr/bin/docview',
|
||||
'--output', str(output / 'x11')]),
|
||||
('wayland', [sys.executable, str(ROOT / 'tests/run_wayland_validation.py'),
|
||||
'--build-dir', '/opt/docview/bin', '--smoke-binary', '/usr/bin/docview',
|
||||
'--output', str(output / 'wayland'), '--mode', 'smoke'])]
|
||||
runs = []
|
||||
try:
|
||||
for name, command in commands:
|
||||
env = dict(os.environ)
|
||||
if name == 'x11':
|
||||
env.update(QT_QPA_PLATFORM='xcb', QT_QUICK_BACKEND='software',
|
||||
QTWEBENGINE_CHROMIUM_FLAGS='--disable-gpu')
|
||||
with (output / (name + '.log')).open('w') as log:
|
||||
result = subprocess.run(command, env=env, stdout=log, stderr=subprocess.STDOUT, timeout=300)
|
||||
runs.append({'name': name, 'exitCode': result.returncode})
|
||||
assert result.returncode == 0, name + ' smoke failed'
|
||||
values = json.loads((output / name / ('smoke/results.json' if name == 'wayland' else 'results.json')).read_text())
|
||||
assert len(values) == 6 and all(row['state'] == 'Ready' and row['binarySha256'] == launcher for row in values)
|
||||
assert all(sha(APP / 'bin' / name) == digest for name, digest in identities.items())
|
||||
assert sha(Path('/usr/bin/docview')) == launcher
|
||||
record.update(success=True, executableBytesUnchanged=True, smokeConditions=12)
|
||||
finally:
|
||||
record['runs'] = runs
|
||||
(output / 'report.json').write_text(json.dumps(record, indent=2, sort_keys=True) + '\n')
|
||||
print(json.dumps({key: record[key] for key in ('success', 'installedFilesVerified', 'smokeConditions')}))
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--output', type=Path, required=True)
|
||||
parser.add_argument('--isolated', action='store_true')
|
||||
parser.add_argument('--user-run', action='store_true')
|
||||
parser.add_argument('--hide-prefix', action='append', default=[], type=Path,
|
||||
help='Additional dedicated guest build/install prefix to hide in the private namespace')
|
||||
args = parser.parse_args()
|
||||
output = args.output.resolve()
|
||||
assert len(args.hide_prefix) <= 8
|
||||
additional = []
|
||||
for prefix in args.hide_prefix:
|
||||
assert prefix.is_absolute() and not prefix.is_symlink()
|
||||
prefix = prefix.absolute()
|
||||
assert prefix.is_relative_to('/home/docview') and prefix != Path('/home/docview')
|
||||
assert '..' not in prefix.parts and not ROOT.is_relative_to(prefix) and not output.is_relative_to(prefix)
|
||||
additional.extend(['--hide-prefix', str(prefix)])
|
||||
if str(prefix) not in HIDDEN: HIDDEN.append(str(prefix))
|
||||
if args.user_run:
|
||||
user_run(output)
|
||||
elif args.isolated:
|
||||
assert os.getuid() == 0
|
||||
with tempfile.TemporaryDirectory(prefix='docview-hidden-runtime-') as temporary:
|
||||
Path(temporary).chmod(0o755)
|
||||
for name in HIDDEN:
|
||||
if Path(name).is_dir():
|
||||
subprocess.run(['mount', '--bind', temporary, name], check=True)
|
||||
subprocess.run(['runuser', '-u', 'docview', '--', 'env', '-i',
|
||||
'HOME=/home/docview', 'USER=docview', 'LOGNAME=docview',
|
||||
'PATH=/usr/bin:/bin', 'LANG=C.UTF-8', sys.executable,
|
||||
str(Path(__file__)), '--user-run', '--output', str(output), *additional], check=True)
|
||||
else:
|
||||
assert os.getuid() != 0 and Path.home() == Path('/home/docview')
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
subprocess.run(['sudo', 'unshare', '--mount', '--propagation', 'private',
|
||||
sys.executable, str(Path(__file__)), '--isolated', '--output', str(output), *additional], check=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,42 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Disposable local Ubuntu validation VM helper; no host package installation."""
|
||||
import os
|
||||
import urllib.request, hashlib, json, pathlib, time, subprocess, concurrent.futures
|
||||
ROOT=pathlib.Path(os.environ.get('DOCVIEW_VM_WORK', 'build-ubuntu-vm')).resolve()
|
||||
D=ROOT/'downloads'
|
||||
for sub in ['downloads','metadata','logs','tools','private']: (ROOT/sub).mkdir(parents=True,exist_ok=True)
|
||||
os.chmod(ROOT/'private',0o700)
|
||||
entries=[
|
||||
('SHA256SUMS','https://cloud-images.ubuntu.com/noble/20260911/SHA256SUMS',100000),
|
||||
('SHA256SUMS.gpg','https://cloud-images.ubuntu.com/noble/20260911/SHA256SUMS.gpg',100000),
|
||||
('ubuntu-cloud-key.asc','https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xD2EB44626FDDC30B513D5BB71A5D6C4C7DB87C81',100000),
|
||||
('noble-server-cloudimg-amd64.img','https://cloud-images.ubuntu.com/noble/20260911/noble-server-cloudimg-amd64.img',700*1024*1024),
|
||||
('noble-server-cloudimg-amd64.manifest','https://cloud-images.ubuntu.com/noble/20260911/noble-server-cloudimg-amd64.manifest',100000),
|
||||
('qemu-img-11.1.1-2-x86_64.pkg.tar.zst','https://geo.mirror.pkgbuild.com/extra/os/x86_64/qemu-img-11.1.1-2-x86_64.pkg.tar.zst',5*1024*1024),
|
||||
('qemu-img-11.1.1-2-x86_64.pkg.tar.zst.sig','https://geo.mirror.pkgbuild.com/extra/os/x86_64/qemu-img-11.1.1-2-x86_64.pkg.tar.zst.sig',100000),
|
||||
('qt-base-Updates.xml','https://download.qt.io/online/qtsdkrepository/linux_x64/desktop/qt6_6112/qt6_6112/Updates.xml',1000000),
|
||||
('qt-webengine-Updates.xml','https://download.qt.io/online/qtsdkrepository/linux_x64/extensions/qtwebengine/6112/x86_64/Updates.xml',1000000),
|
||||
('pycdlib.json','https://pypi.org/pypi/pycdlib/1.14.0/json',1000000)]
|
||||
def fetch(e):
|
||||
name,url,limit=e;p=D/name;start=time.time();h=hashlib.sha256();n=0
|
||||
if p.exists():
|
||||
with p.open('rb') as f:
|
||||
for b in iter(lambda:f.read(1024*1024),b''):h.update(b);n+=len(b)
|
||||
return dict(name=name,url=url,size=n,sha256=h.hexdigest(),reused=True)
|
||||
try:
|
||||
with urllib.request.urlopen(url,timeout=60) as r,p.with_suffix(p.suffix+'.part').open('wb') as f:
|
||||
final=r.url
|
||||
while True:
|
||||
b=r.read(1024*1024)
|
||||
if not b:break
|
||||
n+=len(b)
|
||||
if n>limit:raise RuntimeError('download size bound exceeded')
|
||||
f.write(b);h.update(b)
|
||||
p.with_suffix(p.suffix+'.part').rename(p)
|
||||
print(name,n,'downloaded',flush=True)
|
||||
return dict(name=name,url=url,finalUrl=final,size=n,sha256=h.hexdigest(),seconds=round(time.time()-start,3))
|
||||
except Exception as ex:
|
||||
print(name,'FAILED',str(ex),flush=True);return dict(name=name,url=url,error=str(ex))
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as pool: records=list(pool.map(fetch,entries))
|
||||
(ROOT/'metadata/downloads.json').write_text(json.dumps(records,indent=2)+'\n')
|
||||
if any('error' in e for e in records):raise SystemExit(1)
|
||||
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Disposable local Ubuntu validation VM helper; no host package installation."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess,json,time
|
||||
r=Path(os.environ.get('DOCVIEW_VM_WORK', 'build-ubuntu-vm')).resolve()
|
||||
ssh=['ssh','-i',str(r/'private/id_ed25519'),'-p','22224','-o','IdentitiesOnly=yes','-o','BatchMode=yes','-o','PasswordAuthentication=no','-o','StrictHostKeyChecking=accept-new','-o','UserKnownHostsFile='+str(r/'private/known_hosts'),'-o','ConnectTimeout=3','[email protected]']
|
||||
probe='import subprocess,json,pathlib,ctypes,platform,os\nlibc=ctypes.CDLL(None,use_errno=True);abi=libc.syscall(444,0,0,1)\nresult={\'osRelease\':pathlib.Path(\'/etc/os-release\').read_text(),\'uname\':platform.uname()._asdict(),\'uid\':os.getuid(),\'landlockABI\':abi,\'landlockErrno\':ctypes.get_errno()}\ncommands={\'lsm\':[\'sudo\',\'cat\',\'/sys/kernel/security/lsm\'],\'kernelConfig\':[\'bash\',\'-c\',\'grep -E "^CONFIG_(SECURITY_LANDLOCK|SECCOMP|SECCOMP_FILTER|USER_NS|SECURITY_APPARMOR)=" /boot/config-$(uname -r)\'],\'apparmor\':[\'sudo\',\'aa-status\',\'--json\'],\'sysctls\':[\'sysctl\',\'kernel.unprivileged_userns_clone\',\'kernel.apparmor_restrict_unprivileged_userns\'],\'unprivilegedUserNamespace\':[\'unshare\',\'--user\',\'--map-root-user\',\'true\'],\'disk\':[\'df\',\'-h\',\'/\'],\'cloudInit\':[\'cloud-init\',\'status\']}\nfor k,v in commands.items():\n p=subprocess.run(v,capture_output=True,text=True);result[k]={\'command\':v,\'exitCode\':p.returncode,\'stdout\':p.stdout,\'stderr\':p.stderr}\nprint(json.dumps(result,indent=2))\n'
|
||||
for attempt in range(10):
|
||||
p=subprocess.run(ssh+['python3','-'],input=probe,capture_output=True,text=True)
|
||||
if p.returncode==0:
|
||||
(r/'metadata/guest-probe.json').write_text(p.stdout);(r/'logs/ssh-probe.txt').write_text(p.stderr);j=json.loads(p.stdout);print('Guest ready:',j['uname']['release'],'Landlock ABI',j['landlockABI'],'unprivileged namespace exit',j['unprivilegedUserNamespace']['exitCode']);break
|
||||
time.sleep(2)
|
||||
else:print(p.stderr);raise SystemExit(p.returncode)
|
||||
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Read status or pause/resume/power down this dedicated local VM."""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('command', choices=['query-status', 'stop', 'cont', 'system_powerdown'])
|
||||
args = parser.parse_args()
|
||||
root = Path(os.environ.get('DOCVIEW_VM_WORK', 'build-ubuntu-vm')).resolve()
|
||||
with socket.socket(socket.AF_UNIX) as stream:
|
||||
stream.settimeout(5)
|
||||
stream.connect(str(root / 'qmp.sock'))
|
||||
reader = stream.makefile('rb')
|
||||
json.loads(reader.readline())
|
||||
for operation in ['qmp_capabilities', args.command]:
|
||||
stream.sendall(json.dumps({'execute': operation}).encode() + b'\n')
|
||||
while True:
|
||||
response = json.loads(reader.readline())
|
||||
if 'return' in response or 'error' in response:
|
||||
break
|
||||
if operation == args.command:
|
||||
print(json.dumps(response))
|
||||
if 'error' in response:
|
||||
raise SystemExit(1)
|
||||
@@ -0,0 +1,112 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Repackage the fixed Ubuntu v2 install with pinned qpdf notices, without rebuilding."""
|
||||
from pathlib import Path
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tarfile
|
||||
import time
|
||||
|
||||
HOME = Path('/home/docview')
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
OUTPUT = HOME / 'validation/qpdf-notice-package'
|
||||
PRIOR_SOURCE = HOME / 'docview-context-source'
|
||||
INSTALL = HOME / 'docview-context-install'
|
||||
VERSION = '0.1.0~validation5'
|
||||
|
||||
|
||||
def sha(path):
|
||||
with path.open('rb') as stream:
|
||||
return hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
|
||||
|
||||
def main():
|
||||
assert Path.home() == HOME and os.getuid() == 1000
|
||||
assert ROOT == HOME / 'validation/qpdf-notice-tools'
|
||||
OUTPUT.mkdir(parents=True, exist_ok=False)
|
||||
sys.path.insert(0, str(ROOT / 'tools'))
|
||||
from verify_ubuntu_package import inspect, verify, MANIFEST
|
||||
from collect_ubuntu_validation_runtime import QPDF_NOTICE_PIN
|
||||
env = {**os.environ, 'PATH': '/opt/docview-deps/bin:/opt/docview-qt/6.11.2/gcc_64/bin:' + os.environ['PATH'],
|
||||
'LD_LIBRARY_PATH': '/opt/docview-deps/lib:/opt/docview-qt/6.11.2/gcc_64/lib'}
|
||||
for key in ('LD_PRELOAD', 'LD_AUDIT', 'QTWEBENGINE_DISABLE_SANDBOX'):
|
||||
env.pop(key, None)
|
||||
preserved = [INSTALL/'bin'/name for name in ('docview','docview-pdf-worker','docview-archive-worker')]
|
||||
preserved += [INSTALL/'lib/libpdfium.so', HOME/'validation/pdfium-context-prefix/BUILDINFO.json',
|
||||
Path('/opt/docview-deps/lib/libqpdf.so.30')]
|
||||
before = {str(p):sha(p) for p in preserved}
|
||||
report = {'success': False, 'version': VERSION, 'scope': 'Notice collection and packaging only; no C++ rebuild or repeated full CTest',
|
||||
'runtimeBefore': before, 'runnerSha256': sha(Path(__file__)), 'commands': []}
|
||||
def run(name, command):
|
||||
start = time.monotonic()
|
||||
with (OUTPUT/(name+'.log')).open('w') as log:
|
||||
result = subprocess.run(command, env=env, stdout=log, stderr=subprocess.STDOUT, timeout=900)
|
||||
report['commands'].append({'name':name, 'command':command, 'exitCode':result.returncode,
|
||||
'seconds':time.monotonic()-start, 'logSha256':sha(OUTPUT/(name+'.log'))})
|
||||
assert result.returncode == 0, name
|
||||
try:
|
||||
for name in ('test_ubuntu_validation_runtime','test_qpdf_notice_package','test_pdfium_candidate_integration'):
|
||||
run(name,[sys.executable,str(ROOT/'tests'/(name+'.py')),'-v'])
|
||||
command = [sys.executable,str(ROOT/'tools/package_ubuntu.py'),'--prefix',str(INSTALL),
|
||||
'--qtpaths','/opt/docview-qt/6.11.2/gcc_64/bin/qtpaths','--dependency-prefix','/opt/docview-deps',
|
||||
'--source-root',str(HOME/'dependency-sources/qpdf-12.4.1'),
|
||||
'--source-root',str(HOME/'dependency-sources/libzip-1.11.4'),
|
||||
'--qpdf-source-archive',str(HOME/'incoming/qpdf-12.4.1.tar.gz'),
|
||||
'--input-manifest',str(HOME/'incoming/sdk-downloads.json'),
|
||||
'--sdk-notices',str(PRIOR_SOURCE/'tests/results/source-archives/qt-sdk-notices-final'),
|
||||
'--sdk-supplement',str(PRIOR_SOURCE/'tests/results/qt-notice-supplement/collection'),
|
||||
'--qt-licenses',str(PRIOR_SOURCE/'tests/results/ubuntu-package/inputs/qt-licenses'), '--version',VERSION]
|
||||
archives = []
|
||||
for name in ('package','repeat'):
|
||||
run(name,[*command,'--output',str(OUTPUT/name)])
|
||||
archive = OUTPUT/name/('docview_'+VERSION+'_amd64.deb'); archives.append(archive)
|
||||
run(name+'-verify',[sys.executable,str(ROOT/'tools/verify_ubuntu_package.py'),'--archive',str(archive),
|
||||
'--output',str(OUTPUT/name/'verification.json')])
|
||||
assert sha(archives[0]) == sha(archives[1])
|
||||
prior = HOME/'validation/pdfium-context-package/package/package/docview_0.1.0~validation4_amd64.deb'
|
||||
assert sha(prior) == 'bd4f00912078d406cb466cd6910ed01c0c9fdd32b76cb2f47b74cfb9719f1bd9'
|
||||
def inventory(path):
|
||||
data, _, _ = inspect(path, '--fsys-tarfile')
|
||||
control, _, _ = inspect(path, '--ctrl-tarfile')
|
||||
return {**data, **{'DEBIAN/'+name:dict(row,path='DEBIAN/'+name) for name,row in control.items()}}
|
||||
old, new = inventory(prior), inventory(archives[0])
|
||||
differences = {'added':[new[k] for k in sorted(new.keys()-old.keys())],
|
||||
'removed':[old[k] for k in sorted(old.keys()-new.keys())],
|
||||
'changed':[{'path':k,'before':old[k],'after':new[k]} for k in sorted(old.keys()&new.keys()) if old[k]!=new[k]]}
|
||||
protected = lambda k: not k.startswith(('DEBIAN/','opt/docview/share/doc/docview/'))
|
||||
protected_paths = {k for k in old.keys()|new.keys() if protected(k)}
|
||||
unchanged = all(old.get(k)==new.get(k) for k in protected_paths)
|
||||
assert unchanged, 'Runtime or application payload changed'
|
||||
diff = {'success': True, 'method':'Bounded raw data/control inventory; old validation4 is not accepted by the current notice verifier',
|
||||
'oldArchiveSha256':sha(prior),'newArchiveSha256':sha(archives[0]),'oldFiles':len(old),'newFiles':len(new),
|
||||
'allRuntimeAndApplicationFilesUnchanged':unchanged,'protectedFileCount':len(protected_paths),**differences}
|
||||
(OUTPUT/'payload-difference.json').write_text(json.dumps(diff,indent=2)+'\n')
|
||||
result = verify(archives[0])
|
||||
notice = next(row for row in result['qpdfNoticeCorrespondence']['notices'] if row['source']=='NOTICE.md')
|
||||
wanted = './opt/docview/share/doc/docview/third-party/runtime/'+notice['copiedPath']
|
||||
process = subprocess.Popen(['dpkg-deb','--fsys-tarfile',str(archives[0])],stdout=subprocess.PIPE)
|
||||
found = None
|
||||
with tarfile.open(fileobj=process.stdout,mode='r|') as tar:
|
||||
for member in tar:
|
||||
if member.name == wanted:
|
||||
assert member.isfile() and member.size==QPDF_NOTICE_PIN['notices']['NOTICE.md']['size']
|
||||
found = tar.extractfile(member).read()
|
||||
process.stdout.close(); assert process.wait(timeout=15)==0
|
||||
assert found is not None and hashlib.sha256(found).hexdigest()==QPDF_NOTICE_PIN['notices']['NOTICE.md']['sha256']
|
||||
assert found==(HOME/'dependency-sources/qpdf-12.4.1/NOTICE.md').read_bytes()
|
||||
(OUTPUT/'NOTICE.from-deb.md').write_bytes(found)
|
||||
report.update(success=True, archive=str(archives[0]),archiveSha256=sha(archives[0]),
|
||||
repeatSha256=sha(archives[1]),archiveBytes=archives[0].stat().st_size,
|
||||
qpdfNoticeCorrespondence=result['qpdfNoticeCorrespondence'],pdfiumCorrespondence=result['pdfiumCorrespondence'],
|
||||
actualNoticeBytesMatchSelectedSource=True, actualPayloadCompared=True)
|
||||
finally:
|
||||
report['runtimeAfter']={str(p):sha(p) for p in preserved}
|
||||
report['runtimeUnchanged']=report['runtimeBefore']==report['runtimeAfter']
|
||||
report['success']=report['success'] and report['runtimeUnchanged']
|
||||
(OUTPUT/'report.json').write_text(json.dumps(report,indent=2)+'\n')
|
||||
print(json.dumps({k:report[k] for k in ('success','archiveSha256','archiveBytes','runtimeUnchanged')}))
|
||||
|
||||
|
||||
if __name__=='__main__': main()
|
||||
@@ -0,0 +1,72 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Record this dedicated guest build without copying VM credentials or user data."""
|
||||
import argparse
|
||||
import ctypes
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import re
|
||||
import subprocess
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
|
||||
def sha(path):
|
||||
with path.open('rb') as handle:
|
||||
return hashlib.file_digest(handle, 'sha256').hexdigest()
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--ctest-name', default='ctest-font-final')
|
||||
parser.add_argument('--output-name', default='build-record.json')
|
||||
parser.add_argument('--expected-groups', type=int, default=22)
|
||||
parser.add_argument('--scope-note', default='Full 22-group baseline; later focused results are recorded separately.')
|
||||
args = parser.parse_args()
|
||||
if (not re.fullmatch(r'[a-z0-9][a-z0-9-]{0,63}', args.ctest_name)
|
||||
or not re.fullmatch(r'[a-z0-9][a-z0-9-]{0,63}\.json', args.output_name)
|
||||
or not 1 <= args.expected_groups <= 100):
|
||||
parser.error('Use bounded local evidence names and 1–100 expected groups')
|
||||
|
||||
home = Path.home()
|
||||
source, build = home / 'docview-source', home / 'docview-build'
|
||||
validation = home / 'validation'
|
||||
output = validation / args.output_name
|
||||
if output.exists():
|
||||
raise SystemExit('Keep previous evidence; output already exists')
|
||||
ctest = validation / args.ctest_name
|
||||
cases = ET.parse(ctest / 'tests.xml').findall('.//testcase')
|
||||
if len(cases) != args.expected_groups or any(x.find('failure') is not None or x.find('skipped') is not None for x in cases):
|
||||
raise SystemExit('Expected all selected CTest groups to pass')
|
||||
names = ['docview', 'docview-pdf-worker', 'docview-archive-worker', 'test_app',
|
||||
'test_core', 'test_pdf', 'test_pdf_canvas', 'test_web_qml', 'test_translations']
|
||||
files = [source / n for n in ('CMakeLists.txt', 'resources.qrc')]
|
||||
for folder in ('src', 'qml', 'cmake', 'resources', 'tools'):
|
||||
files += [p for p in (source / folder).rglob('*')
|
||||
if p.is_file() and not p.is_symlink() and '__pycache__' not in p.parts]
|
||||
files += [p for p in (source / 'tests').rglob('*')
|
||||
if p.is_file() and not p.is_symlink() and 'results' not in p.parts
|
||||
and '__pycache__' not in p.parts and p.suffix in ('.cpp', '.h', '.py', '.qml')]
|
||||
libc = ctypes.CDLL(None, use_errno=True)
|
||||
landlock_abi = libc.syscall(444, 0, 0, 1)
|
||||
record = {
|
||||
'scope': 'Ubuntu 24.04 dedicated KVM guest; Xvfb/Sway software rendering, no physical GPU or IME acceptance',
|
||||
'os': platform.freedesktop_os_release(), 'kernel': platform.release(),
|
||||
'compiler': subprocess.check_output(['c++', '--version'], text=True).splitlines()[0],
|
||||
'cmake': subprocess.check_output(['cmake', '--version'], text=True).splitlines()[0],
|
||||
'qt': subprocess.check_output(['/opt/docview-qt/6.11.2/gcc_64/bin/qtpaths', '--query', 'QT_VERSION'], text=True).strip(),
|
||||
'landlockABI': landlock_abi,
|
||||
'apparmorRestrictUnprivilegedUserns': Path('/proc/sys/kernel/apparmor_restrict_unprivileged_userns').read_text().strip(),
|
||||
'qtWebEngineApparmorProfileSha256': sha(Path('/etc/apparmor.d/docview-qtwebengine')),
|
||||
'binaries': {n: sha(build / n) for n in names},
|
||||
'installedBinaries': {n: sha(home / 'docview-install/bin' / n) for n in names[:3]},
|
||||
'sourceSha256': {str(p.relative_to(source)): sha(p) for p in sorted(set(files))},
|
||||
'ctest': {'directory': args.ctest_name, 'groups': len(cases), 'failures': 0, 'junitSha256': sha(ctest / 'tests.xml'),
|
||||
'logSha256': sha(ctest / 'LastTest.log')},
|
||||
'focusedTestScope': args.scope_note,
|
||||
'pythonProvenanceScope': '30 passed and 5 explicit zstd skips with Python 3.12; Arch Python 3.14 executes all 35 cases.',
|
||||
'productionBinariesChangedByTestFixes': False,
|
||||
}
|
||||
with output.open('x') as handle:
|
||||
handle.write(json.dumps(record, ensure_ascii=False, indent=2) + '\n')
|
||||
print(json.dumps({'ctestGroups': len(cases), 'sourceFiles': len(record['sourceSha256']),
|
||||
'productionBinarySha256': record['binaries']['docview']}))
|
||||
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Disposable local Ubuntu validation VM helper; no host package installation."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess,json,socket
|
||||
r=Path(os.environ.get('DOCVIEW_VM_WORK', 'build-ubuntu-vm')).resolve()
|
||||
with socket.socket() as port_probe:
|
||||
port_probe.bind(('127.0.0.1',22224))
|
||||
cmd=['qemu-system-x86_64','-name','docview-ubuntu2404-validation',
|
||||
'-machine','q35,accel=kvm','-cpu','host','-smp','4','-m','8192',
|
||||
'-display','none','-monitor','none','-serial','file:'+str(r/'private/serial.log'),
|
||||
'-pidfile',str(r/'qemu.pid'),'-qmp','unix:'+str(r/'qmp.sock')+',server=on,wait=off',
|
||||
'-drive','if=pflash,format=raw,readonly=on,file=/usr/share/edk2-ovmf/x64/OVMF_CODE.4m.fd',
|
||||
'-drive','if=pflash,format=raw,file='+str(r/'OVMF_VARS.4m.fd'),
|
||||
'-drive','if=virtio,format=qcow2,file='+str(r/'guest.qcow2'),
|
||||
'-drive','if=virtio,format=raw,readonly=on,file='+str(r/'private/seed.iso'),
|
||||
'-netdev','user,id=net0,hostfwd=tcp:127.0.0.1:22224-:22',
|
||||
'-device','virtio-net-pci,netdev=net0','-device','virtio-rng-pci',
|
||||
'-sandbox','on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny']
|
||||
(r/'metadata/qemu-active-command.json').write_text(json.dumps(cmd,indent=2)+'\n')
|
||||
print('Running dedicated Ubuntu guest; 4 vCPU / 8 GiB / 24 GiB / SSH 127.0.0.1:22224',flush=True)
|
||||
with (r/'logs/qemu-runtime.txt').open('ab') as log:
|
||||
p=subprocess.run(cmd,stdout=log,stderr=subprocess.STDOUT)
|
||||
print('QEMU exit',p.returncode,flush=True)
|
||||
raise SystemExit(p.returncode)
|
||||
@@ -0,0 +1,48 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Create a bounded source/PDFium snapshot, excluding results, builds and VM keys."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import tarfile
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--name', required=True)
|
||||
args = parser.parse_args()
|
||||
if not re.fullmatch(r'[a-z0-9][a-z0-9-]{0,63}', args.name):
|
||||
parser.error('Use a short lowercase snapshot name')
|
||||
work = Path(os.environ.get('DOCVIEW_VM_WORK', ROOT / 'build-ubuntu-vm')).resolve()
|
||||
archive = work / (args.name + '.tar.gz')
|
||||
manifest = work / 'metadata' / (args.name + '.json')
|
||||
if archive.exists() or manifest.exists():
|
||||
parser.error('Choose a new name to preserve previous source evidence')
|
||||
files = [ROOT / name for name in ('CMakeLists.txt', 'resources.qrc', 'README.md')]
|
||||
for name in ('src', 'qml', 'cmake', 'resources', 'tools', 'tests', 'docs', '.deps/pdfium'):
|
||||
directory = ROOT / name
|
||||
for base, directories, leaves in os.walk(directory, followlinks=False):
|
||||
directories[:] = [d for d in directories if d not in ('results', '__pycache__', '.git')
|
||||
and not (Path(base) / d).is_symlink()]
|
||||
files.extend(Path(base) / leaf for leaf in leaves if leaf != 'validation-record.json')
|
||||
rows, total = [], 0
|
||||
for path in sorted(set(files)):
|
||||
if path.is_symlink() or not path.is_file() or not path.resolve().is_relative_to(ROOT):
|
||||
raise SystemExit('Unexpected non-regular source input')
|
||||
size = path.stat().st_size
|
||||
total += size
|
||||
if len(rows) >= 10000 or size > 256 * 1024 * 1024 or total > 512 * 1024 * 1024:
|
||||
raise SystemExit('Source snapshot exceeds finite limits')
|
||||
with path.open('rb') as stream:
|
||||
digest = hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
rows.append({'path': str(path.relative_to(ROOT)), 'size': size, 'sha256': digest})
|
||||
work.mkdir(parents=True, exist_ok=True)
|
||||
manifest.parent.mkdir(parents=True, exist_ok=True)
|
||||
with tarfile.open(archive, 'x:gz') as stream:
|
||||
for row in rows:
|
||||
stream.add(ROOT / row['path'], arcname=row['path'], recursive=False)
|
||||
with archive.open('rb') as stream:
|
||||
digest = hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
manifest.write_text(json.dumps({'archiveSha256': digest, 'files': rows}, indent=2) + '\n')
|
||||
print(json.dumps({'files': len(rows), 'sourceBytes': total, 'archiveSha256': digest}))
|
||||
@@ -0,0 +1,85 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Transfer an explicit source delta to this task's dedicated Ubuntu guest."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import tarfile
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--name', required=True)
|
||||
parser.add_argument('files', nargs='+')
|
||||
args = parser.parse_args()
|
||||
if not re.fullmatch(r'[a-z0-9][a-z0-9-]{0,63}', args.name):
|
||||
parser.error('Use a short lowercase name for a new delta')
|
||||
work = Path(os.environ.get('DOCVIEW_VM_WORK', ROOT / 'build-ubuntu-vm')).resolve()
|
||||
output = work / 'source-deltas' / args.name
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
rows, payload = [], {}
|
||||
for name in sorted(set(args.files)):
|
||||
relative = Path(name)
|
||||
if relative.is_absolute() or '..' in relative.parts or str(relative) != name:
|
||||
raise SystemExit('Only canonical repository-relative source paths are allowed')
|
||||
if relative.parts[0] not in ('src', 'qml', 'resources', 'cmake', 'tools', 'tests', 'CMakeLists.txt', 'README.md'):
|
||||
raise SystemExit('Path is not in the allowed source set')
|
||||
path = ROOT / relative
|
||||
if path.is_symlink() or not path.resolve().is_relative_to(ROOT) or not path.is_file():
|
||||
raise SystemExit('Only regular local source files are allowed')
|
||||
data = path.read_bytes()
|
||||
if len(data) > 8 * 1024 * 1024 or len(payload) >= 256:
|
||||
raise SystemExit('Source delta exceeds finite limits')
|
||||
payload[name] = data
|
||||
rows.append({'path': name, 'size': len(data), 'sha256': hashlib.sha256(data).hexdigest()})
|
||||
archive = output / 'patch.tar.gz'
|
||||
with tarfile.open(archive, 'w:gz') as stream:
|
||||
for name, data in payload.items():
|
||||
info = tarfile.TarInfo(name)
|
||||
info.size, info.mode = len(data), 0o644
|
||||
stream.addfile(info, io.BytesIO(data))
|
||||
record = {'name': args.name, 'files': rows,
|
||||
'archiveSha256': hashlib.sha256(archive.read_bytes()).hexdigest()}
|
||||
manifest = output / 'patch.json'
|
||||
manifest.write_text(json.dumps(record, indent=2) + '\n')
|
||||
apply = output / 'apply.py'
|
||||
apply.write_text('''import hashlib,json,shutil,tarfile
|
||||
from pathlib import Path
|
||||
here=Path(__file__).resolve().parent
|
||||
record=json.loads((here/'patch.json').read_text())
|
||||
assert hashlib.sha256((here/'patch.tar.gz').read_bytes()).hexdigest()==record['archiveSha256']
|
||||
source=Path.home()/'docview-source'
|
||||
history=Path.home()/'validation/source-deltas'/record['name']
|
||||
history.mkdir(parents=True,exist_ok=False)
|
||||
changes=[]
|
||||
with tarfile.open(here/'patch.tar.gz') as archive:
|
||||
for row in record['files']:
|
||||
rel=Path(row['path'])
|
||||
assert not rel.is_absolute() and '..' not in rel.parts
|
||||
data=archive.extractfile(row['path']).read()
|
||||
assert len(data)==row['size'] and hashlib.sha256(data).hexdigest()==row['sha256']
|
||||
target=source/rel
|
||||
assert target.resolve().is_relative_to(source) and not target.is_symlink()
|
||||
before=None
|
||||
if target.exists():
|
||||
before=hashlib.sha256(target.read_bytes()).hexdigest()
|
||||
original=history/'originals'/rel
|
||||
original.parent.mkdir(parents=True,exist_ok=True)
|
||||
shutil.copyfile(target,original)
|
||||
target.parent.mkdir(parents=True,exist_ok=True)
|
||||
target.write_bytes(data)
|
||||
changes.append({**row,'previousSha256':before,'changed':before!=row['sha256']})
|
||||
(history/'record.json').write_text(json.dumps({**record,'files':changes},indent=2)+'\\n')
|
||||
print(json.dumps({'sourceDelta':record['name'],'files':len(changes),'changed':sum(x['changed'] for x in changes)}))
|
||||
''')
|
||||
base = ['-i', str(work / 'private/id_ed25519'), '-o', 'IdentitiesOnly=yes', '-o', 'BatchMode=yes',
|
||||
'-o', 'StrictHostKeyChecking=yes', '-o', 'UserKnownHostsFile=' + str(work / 'private/known_hosts')]
|
||||
remote = 'incoming/delta-' + args.name
|
||||
ssh = ['ssh', *base, '-p', '22224', '[email protected]']
|
||||
subprocess.run([*ssh, 'mkdir -p ' + remote], check=True)
|
||||
subprocess.run(['scp', *base, '-P', '22224', str(archive), str(manifest), str(apply),
|
||||
'[email protected]:' + remote + '/'], check=True)
|
||||
subprocess.run([*ssh, 'python3 ' + remote + '/apply.py'], check=True)
|
||||
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Verify VM inputs and install small helper tools into the VM workspace only."""
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import urllib.request
|
||||
import zipfile
|
||||
|
||||
root = Path(os.environ.get('DOCVIEW_VM_WORK', 'build-ubuntu-vm')).resolve()
|
||||
downloads = root / 'downloads'
|
||||
keyring = root / 'private/gnupg'
|
||||
keyring.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
fingerprint = 'D2EB44626FDDC30B513D5BB71A5D6C4C7DB87C81'
|
||||
|
||||
|
||||
def run(command, log):
|
||||
result = subprocess.run(command, capture_output=True, text=True)
|
||||
(root / 'logs' / log).write_text(result.stdout + result.stderr)
|
||||
result.check_returncode()
|
||||
return result.stdout + result.stderr
|
||||
|
||||
|
||||
run(['gpg', '--homedir', str(keyring), '--batch', '--import',
|
||||
str(downloads / 'ubuntu-cloud-key.asc')], 'ubuntu-key-import.txt')
|
||||
identity = run(['gpg', '--homedir', str(keyring), '--batch', '--with-colons',
|
||||
'--fingerprint'], 'ubuntu-key-fingerprint.txt')
|
||||
assert 'fpr:::::::::' + fingerprint + ':' in identity
|
||||
verification = run(['gpg', '--homedir', str(keyring), '--batch', '--status-fd', '1',
|
||||
'--verify', str(downloads / 'SHA256SUMS.gpg'),
|
||||
str(downloads / 'SHA256SUMS')], 'ubuntu-sha-signature.txt')
|
||||
assert 'VALIDSIG ' + fingerprint + ' ' in verification
|
||||
filename = 'noble-server-cloudimg-amd64.img'
|
||||
expected = next(line.split()[0] for line in (downloads / 'SHA256SUMS').read_text().splitlines()
|
||||
if line.split()[1].lstrip('*') == filename)
|
||||
with (downloads / filename).open('rb') as stream:
|
||||
actual = hashlib.file_digest(stream, 'sha256').hexdigest()
|
||||
assert actual == expected
|
||||
qemu = downloads / 'qemu-img-11.1.1-2-x86_64.pkg.tar.zst'
|
||||
with qemu.open('rb') as stream:
|
||||
assert hashlib.file_digest(stream, 'sha256').hexdigest() == 'a095493f3fffa82cc5db3a8409950124e67e45cb6ca30456a5851362be5c396a'
|
||||
# This host-only bootstrap recipe was tested on Arch. Its trusted repository
|
||||
# keyring is read-only; an Ubuntu guest does not use this host package.
|
||||
run(['gpgv', '--keyring', '/etc/pacman.d/gnupg/pubring.gpg', str(qemu) + '.sig',
|
||||
str(qemu)], 'qemu-package-signature.txt')
|
||||
subprocess.run(['bsdtar', '-xf', str(qemu), '-C', str(root / 'tools'),
|
||||
'usr/bin/qemu-img'], check=True)
|
||||
package = json.loads((downloads / 'pycdlib.json').read_text())
|
||||
wheel = next(item for item in package['urls'] if item['filename'].endswith('.whl'))
|
||||
target = downloads / wheel['filename']
|
||||
if not target.exists():
|
||||
with urllib.request.urlopen(wheel['url'], timeout=45) as response:
|
||||
target.write_bytes(response.read(300000))
|
||||
with target.open('rb') as stream:
|
||||
assert hashlib.file_digest(stream, 'sha256').hexdigest() == wheel['digests']['sha256']
|
||||
assert target.stat().st_size == wheel['size']
|
||||
with zipfile.ZipFile(target) as archive:
|
||||
archive.extractall(root / 'tools/python')
|
||||
report = {'ubuntuImageSignatureValid': True, 'ubuntuSigningFingerprint': fingerprint,
|
||||
'ubuntuImageSha256': actual, 'ubuntuImageSize': (downloads / filename).stat().st_size,
|
||||
'qemuPackageSha256MatchesLocalRepositoryDatabase': True, 'qemuPackageSignatureValid': True}
|
||||
(root / 'metadata/verification.json').write_text(json.dumps(report, indent=2) + '\n')
|
||||
print('Verified Ubuntu signed checksum/image, QEMU package and local ISO helper.')
|
||||
Reference in New Issue
Block a user