kamat/docview
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
kamat
2026-09-21 13:41:40 +09:00
commit 855c7328df
411 changed files with 85352 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/PDF-VALIDATION.md
+232
View File
@@ -0,0 +1,232 @@
# PDF implementation validation
Validated on Linux x64 with Qt 6.11.2 and PDFium 155.0.8057.0, pinned to
`a5a7089234f121990b336b3841008009dca143bf`. The non-V8 build disables PDF
JavaScript and XFA. Exact binary provenance and SHA-256 are in
`cmake/pdfium.lock.json`; `python3 cmake/fetch_pdfium.py` installs that archive.
The current [v2 source-build candidate](results/pdfium-intent-context/README.md)
preserves rendering intents, pattern initial state and uncolored tiling stroke color.
Dedicated Arch and Ubuntu builds each pass 972 color probes and all 24 CTest groups;
the upstream run passes 1,979 tests. It is integrated into `build-context` and the
Ubuntu validation4 deb. Original dependencies and earlier packages remain available
for comparison. The v1 corpus had eight incorrect shading-pattern expectations;
v2 corrects those rows without changing the PDF/profile bytes and retains the old
failures and measurements. See the [current integration record](results/pdfium-intent-context/integration-record.json).
The historical [CMYK CLUT comparison](results/cmyk-lut/README.md) exposes a provider
rendering-intent discrepancy: on both Arch and Ubuntu, 30 of 48 ICC vector/image probes
exceed the unchanged analytic tolerance. The fixed PDFium uses perceptual ICC conversion
despite this PDF's relative-colorimetric intent. Diagnosis does not grant acceptance;
the [additional review page](results/cmyk-lut/index.html) retains the difference.
The [current human review gallery](results/pdfium-intent-context/render-review/index.html) assembles 21 image
pairs with per-page expectations and known differences. Nineteen compare against
Poppler; two compare TTF/TTC within PDFium. Sources, images and the worker used
for that recorded run are bound by hashes. All 83 decoded images match the prior
gallery exactly. All remain pending human review; creating or testing the
gallery does not grant G-RENDER acceptance.
## Validation revision
The older integrated, comparison and performance records remain historical evidence
for the binary hashes recorded with each run. Current checks include the latest
identity, navigation, metadata-omission and translation changes.
The historical completion-final run passed all 20 CTest groups; its [log](results/completion-final/ctest/LastTest.log)
and [build hashes](results/completion-final/build-record.json) are retained. The [six renderer comparisons](results/completion-final/comparison-run.json),
[relocated package checks](results/completion-final/linux-development-package/README.md)
and [watchdog](results/completion-final/worker-deadlines/README.md) have also completed.
The [v2 PDF performance run](results/pdfium-intent-context/performance/README.md)
passed four conditions, 120 cold process starts, 750 operations and 960 scroll inputs,
plus the approximately 1 GB / 5,000-page stress case. No measured provisional budget
was exceeded. These are headless software-rendering reference measurements; physical
display and reference-hardware acceptance remain incomplete. See [validation status](../docs/VALIDATION.md).
## Automated checks
The current `test_pdf` run records 75 passes (the earlier run had 69), including annotation geometry and lifetime checks,
26 tagged-PDF boundary rows, bounded shared-metadata checks, production worker
cases, Japanese broker integration, fatal-font-error rows, and QtTest setup and cleanup: metadata/page labels/CropBox/Rotate; explicit and named outline
destinations; RoleMap and multiple MCIDs in H1/H2; external versus blocked Launch
links and comment contents; text search; saved widget appearance; full-page
versus tile pixels; inverse coordinates; password required/invalid/correct;
corruption and render limits; and real sandboxed PDF worker round trips. A
malformed page tree with a missing first page is rejected before commit, and
its document handle is closed; it cannot trigger repeated page-metadata loads.
The suite also checks 1,200-item outline pagination, encoded-byte limits with
long Unicode labels, explicit 20,000-item truncation, string-limit warnings,
eight capability states/reasons, and the tagged versus untagged declaration.
The latest metadata regression cases are `annotationContentsLengthBoundary`,
`annotationAndLinkCountLimitsAreExplicit` (comments/links at 1,000 and 1,001), and
`annotationResponsePreservesBoundedPrefixThroughWorker`. A Contents value over
4,095 UTF-16 units (8,192-byte buffer including its terminator) keeps a fixed
explanation in the annotation list and reports an omission warning. Page-local
link/annotation count limits and the 512 KiB metadata response budget return an
explicit warning with the retained prefix. The production-worker case uses 100
long Japanese comments and verifies bounded replies and unchanged source bytes.
These cases pass in the current 75-pass suite, without increasing the safety limits.
Controller regressions now distinguish an outline containing only external or
blocked actions from one with usable internal destinations. Only internal targets
are considered for heading fallback and its capability state. The same GUI run
also replaces an open PDF's source file, verifies that its old reading position is
not rebound to the replacement, and checks the unresolved-location notice on
explicit reopen. [Focused and full-GUI evidence](results/completion-regressions/controller-state.md).
The worker test uses the production `WorkerProcess` launcher, completes the
bootstrap ping, parses the fixture only after Linux Landlock and seccomp are
active, renders a tile, and stops cleanly. It must run in an environment
permitting the test harness to create its own local socket. The worker never
disables its own sandbox for testing. The launcher also has a Windows handle
transport, but these results were obtained on Linux and establish no Windows
sandbox result.
The [tagged-PDF boundary corpus](fixtures/pdf/headings/README.md) adds a transformed
Form XObject, a rotated CropBox with structure order different from draw order,
a heading spanning two pages with nested and direct MCR children, and explicit
page-precision fallbacks for vector-only headings and reused stream-local MCIDs.
The regression checks authored PDF-point regions, title/order/page/level, all
four user rotations, and inverse coordinates within 0.5 pt. The Form-only ParentTree fixture now yields an exact H2 using the shared
worker-side qpdf structure adapter; a GUI regression follows the target.
The expanded corpus also checks nested Forms, owner-qualified MCID collisions,
repeated calls, structure cycles and bounded stream decompression.
The corpus exposed and fixed mixed-child ordering and non-text MCID collision
bugs; the ordinary RoleMap/multiple-MCID cases continue to pass.
`test_pdf_canvas` includes 21 passes including setup and cleanup. It distinguishes
the page nearest the viewport centre (ties choose the previous page) from the
leading logical reading location, and preserves the central PDF point on zoom. A separate
`pdf_canvas_hidpi` CTest group repeats the annotation pixel/hit-target case at DPR 2
(three QtTest passes including setup and cleanup). It compares both directions of the canvas coordinate transform
against PDFium for a nonzero CropBox, document Rotate 90 and user rotations
0/90/180/270. It verifies PDF-point restoration after zoom and horizontal pan,
separate jumps to two headings on the same page, invalid/clipped coordinates,
a jump whose page metadata arrives later, high zoom limits and search-result navigation.
The actual sandboxed worker also verifies that prefetch disabled produces no
adjacent-page requests, enabled prefetch warms a bounded adjacent viewport,
and a page jump uses those pixels immediately. An in-flight prefetch test
changes the zoom before completion, rejects the obsolete reply, and confirms
the previous visible resolution remains cached. Prefetch is limited to eight
tiles per adjacent page, runs only after visible work and queued operations,
and cannot insert a tile when doing so would evict existing cache entries.
Real-worker cases also inject a delayed fatal font reply: visible and prefetch
failures stop the poisoned worker even after a zoom revision, while a reply for
a replaced document is ignored. Missing-font warnings remain nonfatal and
reach document information. GUI regressions reopen the same unchanged source
with a new worker after failure in prefetch, search, cancelled search or page
metadata.
Late metadata cannot resurrect a destination cancelled by a newer jump, zoom,
or explicit cancellation. Theme background changes preserve white PDF paper.
The real-worker cache regression checks the 64–512 MiB budget clamp, nonzero
charged image bytes after rendering, cost below budget, cache clearing at
document replacement, actual visible/prefetch request counters, and exactly one
discarded old-revision reply. Repeated viewport readiness checks do not inflate
the request counts. These counters are lifetime samples; budget/cost report
QCache's KiB-rounded image charge and are separate from measured process RSS.
A two-pixel offscreen margin was insufficient to preserve antialiasing for a
glyph crossing a tile boundary. The worker now renders a sixteen-pixel margin,
then copies only the requested tile. The fixture's tile and full-page crop
compare byte for byte, including the red saved form appearance. IPC remains a
512-pixel tile, and the margin is not transmitted.
The base fixtures are generated locally by `tests/fixtures/pdf/generate.py`;
`tests/fixtures/pdf/generate_headings.py` regenerates 26 structure fixtures and
four shared-context boundary fixtures, with a SHA-256/expectation manifest,
without external dependencies.
They contain no third-party document content. The `qpdf` CLI regenerates
the encrypted fixture (reader password: `reader`); libqpdf is also a worker runtime
dependency for supplemental link-border and structure metadata. No password is saved by the
application.
## Independent renderer comparison
Run `python3 tests/compare_pdf_renderers.py` after building `pdf-worker-evidence`.
The exporter uses the production sandboxed worker, font broker and IPC; it does
not link PDFium into the main process. Set
`DOCVIEW_BUILD_DIR=build-pdf` to use another build directory. The comparison
uses system Poppler when available, because the bundled document-tools wrapper
has a different fontconfig environment.
Fixture SHA-256:
`e8b0b037679a3449de5d48f1b13a8562486c9ce19385a036d682cab906815a22`.
PDFium and Poppler 26.08.0 at 144 DPI both produced a 400 × 400 CropBox image.
154,110 of 160,000 pixels matched exactly. 4,024 pixels had a maximum channel
difference greater than 16; mean absolute channel error was 2.26274 / 255.
These descriptive metrics are not a performance or compatibility acceptance
threshold. The record contains the worker and font-broker binary hashes.
Evidence is written to `tests/results/pdf/{pdfium,poppler,difference,comparison}.png`
and `comparison.json`. Inspection confirms all three text strings, the black
rectangle and the saved red widget appearance are present in both images. Text
antialiasing differs. The no-appearance text-comment icon differs by renderer.
The former missing default link borders now match Poppler in the 8,448-pixel
comparison region. Other differences remain visible in the evidence and are
not accepted as golden-image success.
The [link-border corpus](results/link-borders/README.md) adds 20 annotation styles,
four intrinsic and four user rotations, tiled/full output equality, original
rectangle and file preservation, encrypted input, malformed data and bounded
metadata/AP handling. Supplemental metadata is read through libqpdf in the
worker, and numeric viewing-only appearances are generated through PDFium's
public API. Supplied appearances are preserved. NoZoom/NoRotate now use temporary
Rect and page-rotation compensation, with logical zoom separated from DPR.
The [annotation-flag corpus](results/pdf-annotation-flags/README.md) exercises
generated borders, saved link/comment appearances, and native Text icons.
Other missing-AP annotation types retain an explicit NoZoom limitation. Canvas tests compare
actual colored pixels with focus/click bounds at 100% and 200% DPR. These are
scoped rendering results, not a complete G-RENDER gate.
The [expanded synthetic corpus](fixtures/pdf/extended/README.md) adds five
documents and nine pages with embedded Japanese text, authored vertical glyph
positions and ruby, RGB/RGBA images, soft masks, alpha blending, clipping,
shading, CMYK swatches, Standard-14 substitution, varied CropBox/rotation, and
ICCBased RGB vector and raster content.
The [extended comparison evidence](results/pdf-extended/README.md) records
matching dimensions and geometry/alpha probes, Japanese extraction/search,
independent Poppler images and visual inspection. Run
`python3 tests/compare_pdf_extended.py` to regenerate that evidence.
The [ICC comparison](results/pdf-icc/README.md) renders a self-authored linear-RGB
matrix/TRC profile at 50%, 100% and 400%. All 54 probes agree with the analytic
sRGB transfer function within the stated rounding tolerance. A DeviceRGB control
distinguishes actual profile application from ignored ICC data. Dimensions and
the original file hash also match. Run `python3 tests/compare_pdf_icc.py` with
Pillow and pypdf installed. This is not calibrated CMYK/printing acceptance.
The [large-PDF stress record](STRESS-PDF-VALIDATION.md) covers a 5,000-page,
1,009,828,104-byte generated PDF with 1,280 genuinely referenced uncompressed
raster streams. Three opens, 100 keyboard operations and a direct page-5,000
screen capture completed with the sandbox enabled; the large input was removed
after hashes, timings, memory and screenshots were retained. This workload is
separate from rendering compatibility and reference-machine acceptance.
The [real TTC experiment](results/font-collection/README.md) constructs a private
two-face collection from locally installed OFL fonts. The production broker
selects regular face 0 and bold face 1; both pages' 892,800 pixels match the
separate-TTF condition exactly, and regular/bold remain distinct. Font programs,
configuration and cache are temporary and removed after verification. This
checks real Linux TrueType collection selection, not Windows/CJK/CFF collections.
## Outstanding acceptance coverage
G-RENDER is **not complete**: the synthetic comparisons have no human-approved
golden image and do not establish arbitrary CID/CMap compatibility,
calibrated ICC/CMYK, complex transparency
groups, malformed appearances or production-corpus compatibility. The original embedded Japanese
fixture uses explicitly positioned vertical glyphs; the new unembedded fixture
adds native UniJIS-UCS2-V writing mode for two requested Japanese families. Generated link borders, saved link/comment appearances, and native Text icons
now cover NoZoom/NoRotate; other subtype-specific synthesized appearances
still need wider validation.
G-HEADINGS remains scoped to the generated corpus: RoleMap, direct/nested/interleaved
MCIDs, transformed Form text, rotated pages and cross-page fragments are covered.
A shared qpdf context now reads Form-only StructParents and verifies stream owners,
ParentTree membership and actual Form invocation. Public PDFium APIs supply text
and geometry. Repeated or ambiguously matched Forms keep explicit page-precision
fallbacks; malformed/cyclic/bounded-out structures report limitations. The main
process has no added PDF parser or native PDF objects. Annotation-owned marked
content remains explicitly unsupported, and broader production structures require
corpus validation. These results do not establish comprehensive tagged-PDF support.
tests/STRESS-PDF-VALIDATION.md
+72
View File
@@ -0,0 +1,72 @@
# 大容量 PDF の負荷試験記録
最新のUI修正後の再測定は [ui-finalの記録](results/ui-final/stress/README.md) を参照してください。[completion-finalの記録](results/completion-final/stress/README.md)と以下は、以前の本体による測定履歴です。資料構造と再現手順は共通です。
2026-09-19、Arch Linux x86_64 / kernel 7.2.2-zen1-1-zen / Qt 6.11.2、Xvfb の専用 X11 表示と Qt Quick software rendering で当時のビルドを測定した。画面は 1400×900、アプリ窓は 1100×800、DPR 1。PDFium ワーカーの Landlock/seccomp と資源制限は有効のままである。以下は [stress-final/report.json](results/stress-final/report.json) の結果で、[従来の記録](results/stress/report.json) は旧ビルドの履歴として保持する。
## 入力と再現方法
`generate_stress_pdf.py` は、オリジナルの紙面テクスチャ・ブロック文字・バーコードを持つ **1,280 個の異なる 512 × 512 RGB 画像**を、フィルターなしの各 768 KiB PDF image stream として書く。5,000 ページの全てに画像を描画し、画像を `pageIndex % 1280` で再利用する。全画像が少なくとも一度参照される。サイズを稼ぐだけの未参照データや padding はない。
- PDF サイズ: **1,009,828,104 bytes**（約 963.05 MiB / 0.940 GiB）。
- 実ラスター payload: **1,006,632,960 bytes**（960 MiB）。
- SHA-256: `a8657fdaef99e4e6d3eb650bfc4afe9ded74dee4873c69698fcd73021617dc2e`。
- ページ: 600 × 800 pt。画像矩形: `[44, 180, 556, 692]`。ページ番号・画像番号の本文マーカーと二つの色マーカーを持つ。
- 生成前空き容量: 318,780,846,080 bytes。出力見積りに加え 512 MiB の余裕を要求する。
- ラスター用バッファ: テンプレートと現在の画像の計 1.5 MiB。全文書を RAM に保持せず、逐次書き込みと SHA-256 更新を行う。
通常のビルド後、次で生成・検査・GUI 試験・削除まで実行する。
```sh
python3 tests/test_stress_generator.py
python3 tests/test_stress_runner.py
python3 tests/run_stress_pdf.py --xvfb --binary build/docview --output tests/results/stress-final
```
最初のコマンドは 5 ページ・2 画像のみを使用する軽量試験で、再現性、画像の固有性と参照、xref、空き容量不足、既存ファイル保護を確認する。二番目の新しい runner 回帰は GUI／大容量 PDF を使わず、出力 schema、実 open 数、メモリ圧迫、no-op 除外、close 後解放、起動失敗時の旧集計除去、実行ファイル改変時の失敗を検査する。今回 3 試験が通過し、[軽量試験ログ](results/stress-final/runner-tests.log)を保存した。三番目は `build/stress/` 内の一時ディレクトリに大容量 PDF を作り、成功・失敗のどちらでも回収する。利用者の設定・履歴には一時 XDG ディレクトリを使用する。
## 測定結果
アプリ自身が focused window にキーイベントを送り、`frameSwapped` で初回応答を記録し、全可視タイルがそろった後のフレームを最終品質とする。これは提示キュー時刻であり、物理ディスプレイの scanout を確認する値ではない。`--benchmark-runs 3 --benchmark-close-cycles 2 --benchmark-operations 100 --benchmark-scroll-steps 0` を明示し、実際に 3 open、操作前の単純 open／close 2 回、操作後の close 1 回を行った。3 回の open は毎回新しいワーカー、100 操作はページ移動 80 回と拡大／fit-width 20 回から成る。キャッシュ分類は位置が変わったページ移動だけを含み、no-op は 0 件だった。継続スクロールはこの有限試験では実施していない。生成直後の OS ファイルキャッシュは残している。
| 項目 | 件数 | P50 | P95 | 最大 |
|---|---:|---:|---:|---:|
| open → 可視本文の提示 | 3 | 96.71 ms | 97.70 ms | 97.70 ms |
| キー → 初回応答フレーム | 100 | 6.61 ms | 8.14 ms | 11.69 ms |
| キー → 全可視タイルの最終品質 | 100 | 12.01 ms | 24.12 ms | 40.35 ms |
| キャッシュ済みページ移動 | 68 | 11.86 ms | 15.87 ms | 18.87 ms |
| 新規可視タイルを要するページ移動 | 12 | 18.67 ms | 40.35 ms | 40.35 ms |
| メモリ・要求 | 実測 |
|---|---:|
| アプリのピーク RSS | 410,173,440 bytes |
| アプリと子プロセス合計のピーク RSS | 752,041,984 bytes |
| 各 open 完了時のプロセス合計 RSS | 379,916,288 / 384,729,088 / 386,531,328 bytes |
| タイルキャッシュのピーク charged bytes | 268,423,168 bytes |
| タイルキャッシュ上限 | 268,435,456 bytes（256 MiB） |
| 可視／先読みタイル要求 | 205 / 281 |
| 描画待ち／queued render／active render の最大 | 4 / 3 / 1 |
| 破棄済み queued requests | 28 |
| 最大メモリ圧迫段階 | 0（通常解像度条件を維持） |
表はアプリ内の 100 ms 間隔の標本である。独立した 50 ms 間隔の `/proc` 監視はアプリ 413,401,088 bytes、子を含む合計 755,404,800 bytes を観測した。標本時刻が異なるため値は一致しない。合計 RSS は共有ページをプロセスごとに数えるため PSS ではなく、Xvfb は含めない。有限間隔のサンプルなので瞬間的な全ピークを保証しない。
全 3 close で Empty とタイル cache 0 を確認した。単純 open／close 2 回後の合計 RSS は 132,026,368 / 134,127,616 bytes、操作後の最終 close は 376,856,576 bytes だった。プロセスの allocator 等に残る RSS を含み、2 回の均一条件だけで長期リークを判定する試験ではない。
## 終端ページとクリーンアップ
独立した `pdfinfo` が 5,000 ページを認識し、`pdftotext` が 1・1,280・5,000 ページの予定マーカーを抽出した。1,280 個の画像 digest は全て異なる。先頭・中間・末尾の image stream を再読込してハッシュ一致と xref を検査した。アプリが試験後に算出した全文書 SHA-256 も生成時の値と一致した。
別プロセスで `--page 5000 --smoke-output ...` を実行し、Ready / pdf、可視 PDF 寸法、viewportReady、PNG／runtime 保存の成功を満たして終了した。[保存画面](results/stress-final/last-page/screen.png)を `view_image` で目視し、`DOCVIEW STRESS PAGE 05000 OF 05000 IMAGE 1159`、ラスター画像、ステータス `5000 / 5000` を確認した。自動 OCR 検証ではない。メモリ圧迫 0 の集計対象は benchmark プロセスであり、smoke の runtime 出力には圧迫段階の時系列は含まれない。
証跡は [report.json](results/stress-final/report.json)、[benchmark.json](results/stress-final/benchmark.json)、[source-manifest.json](results/stress-final/source-manifest.json) に保存した。原本 SHA は benchmark 終了時と smoke 後の独立読込の両方で生成時と一致した。大容量 PDF は削除済みで、`sourceRemoved=true` と `build/stress/` に PDF が残っていないことを確認した。全工程 14.291 s、benchmark プロセス 7.468 s、smoke プロセス 4.893 s。
次の主 3 実行ファイルは試験前後で SHA-256 が一致した。report の `executableSha256`／`executableSha256After`／`executablesUnchanged` に保存する。
| 実行ファイル | SHA-256 |
|---|---|
| docview | `9612bccb5ce864174c1898843a90fa75c2598ac47e8abb3b9822d35c78d766c5` |
| docview-pdf-worker | `b199d491f411dc32c60c5f19bd07c24d0cc85fb5e43ca1f08f085615574dc461` |
| docview-archive-worker | `312dc77c52f9b40441f339381ae5209cac4d11d8131c9f1f239aa025713f0966` |
この試験は大容量・大量ページの取扱い、遠方移動、キャッシュ上限の観測である。3 回の open は 30 回の性能受入条件を置き換えず、仮想表示器は設計書の基準機ではない。1,280 画像を再利用する合成文書は、5,000 ページ全てが異なる高解像度スキャン、圧縮画像の複雑な復号、実書籍の互換性、長時間のリーク検証を網羅しない。
tests/benchmark.py
+193
View File
@@ -0,0 +1,193 @@
#!/usr/bin/env python3
"""Fixed input sequences and explicit close/reopen memory measurements.
Use a dedicated display or Xvfb. Chromium and worker sandboxes stay enabled.
Runtime completion is distinct from budget observations and target acceptance.
"""
import argparse
import hashlib
import json
import math
import os
import pathlib
import statistics
import subprocess
import tempfile
root = pathlib.Path(__file__).resolve().parents[1]
p = argparse.ArgumentParser()
p.add_argument('--binary', type=pathlib.Path, default=root/'build/docview')
p.add_argument('--runs', type=int, default=30)
p.add_argument('--operations', type=int, default=100, help='operations in each applicable navigation series')
p.add_argument('--close-cycles', type=int, default=10)
p.add_argument('--scroll-steps', type=int, default=240)
p.add_argument('--cold-process-runs', type=int, default=0, help='fresh process and fresh XDG app-cache opens; OS file cache retained')
p.add_argument('--formats', nargs='+', default=['pdf', 'html', 'zip', 'epub'])
p.add_argument('--corpus', choices=['smoke', 'standard', 'fonts'], default='smoke')
p.add_argument('--output', type=pathlib.Path, default=root/'tests/results/performance')
a = p.parse_args()
if not 0 <= a.cold_process_runs <= 100:
p.error('--cold-process-runs must be between 0 and 100')
a.output.mkdir(parents=True, exist_ok=True)
def file_hash(path):
with path.open('rb') as source:
return hashlib.file_digest(source, 'sha256').hexdigest()
def stats(samples):
if not samples:
return {'n': 0}
ordered = sorted(samples)
return {'n': len(samples), 'p50_ms': statistics.median(samples),
'p95_ms': ordered[max(0, math.ceil(len(ordered)*.95)-1)],
'min_ms': ordered[0], 'max_ms': ordered[-1]}
def budget(stats_, limit):
return {'limit_ms': limit, 'observed_met': stats_['p95_ms'] <= limit} if stats_.get('n') else {'status': 'not-measured', 'limit_ms': limit}
executables = {name: file_hash(a.binary.resolve().parent / name) for name in
[a.binary.name, 'docview-pdf-worker', 'docview-archive-worker']}
runner_hash = file_hash(pathlib.Path(__file__))
fixtures = {'pdf': 'pdf/navigation.pdf', 'html': 'web/html/index.html', 'zip': 'web/html-book.zip', 'epub': 'web/reflow-rtl.epub'}
if a.corpus == 'standard':
fixtures = {'pdf': 'performance/standard-500.pdf', 'html': 'performance/html/index.html',
'zip': 'performance/standard-html.zip', 'epub': 'performance/standard-500.epub',
'pdf-headings': 'performance/standard-headings-500.pdf'}
if 'pdf' in a.formats and 'pdf-headings' not in a.formats:
a.formats.append('pdf-headings')
elif a.corpus == 'fonts':
fixtures = {'pdf': 'pdf/fonts/unembedded-japanese.pdf'}
a.formats = ['pdf']
unknown = set(a.formats) - fixtures.keys()
if unknown:
p.error('formats unavailable in this corpus: '+', '.join(sorted(unknown)))
# Clear this invocation's selected evidence before starting any process. A
# failure or interruption must not leave later selected cases from an old run.
for kind in a.formats:
(a.output/(kind+'.json')).unlink(missing_ok=True)
cold_directory = a.output/'cold'/kind
for old in cold_directory.glob('[0-9][0-9][0-9].json'):
old.unlink()
failures = []
with tempfile.TemporaryDirectory(prefix='docview-benchmark-') as temporary:
directory = pathlib.Path(temporary)
config = directory/'config.toml'
config.write_text('schema_version=1\n[privacy]\nremember_position=false\nrecent_documents=0\n', encoding='utf-8')
env = os.environ.copy()
for name, leaf in [('XDG_CONFIG_HOME', 'config'), ('XDG_STATE_HOME', 'state'), ('XDG_CACHE_HOME', 'cache')]:
env[name] = str(directory/leaf)
for kind in a.formats:
result = a.output/(kind+'.json')
# Never reuse a previous success when this invocation fails before it
# produces a record (including a timeout or an executable startup error).
result.unlink(missing_ok=True)
source = root/'tests/fixtures'/fixtures[kind]
source_hash = file_hash(source)
command = [str(a.binary.resolve()), '--config', str(config), '--benchmark-output', str(result.resolve()),
'--benchmark-runs', str(a.runs), '--benchmark-operations', str(a.operations),
'--benchmark-close-cycles', str(a.close_cycles), '--benchmark-scroll-steps', str(a.scroll_steps), str(source)]
try:
completed = subprocess.run(command, env=env, check=False, timeout=max(180, a.runs*3+a.operations*6+a.scroll_steps*.05))
except subprocess.TimeoutExpired:
failures.append(kind+': process timeout')
continue
except OSError:
failures.append(kind+': executable startup failed')
continue
if not result.exists():
failures.append(kind+': no measurement record')
continue
values = json.loads(result.read_text())
values['harnessReportedSuccess'] = values.get('success', False)
values['success'] = values.get('success', False) and completed.returncode == 0
values['processExitCode'] = completed.returncode
actions = values['operations']
summary = {'open': stats(values['openFrameMs']),
'response': stats([v['firstResponseFrameMs'] for v in actions if 'firstResponseFrameMs' in v]),
'final_quality': stats([v['finalQualityFrameMs'] for v in actions if 'finalQualityFrameMs' in v])}
summary['cached_pdf_navigation'] = stats([v['finalQualityFrameMs'] for v in actions if v['command'] == 'nav.page' and v.get('cacheHit') is True and v.get('positionChanged') is True])
summary['uncached_pdf_navigation'] = stats([v['finalQualityFrameMs'] for v in actions if v['command'] == 'nav.page' and v.get('cacheHit') is False and v.get('positionChanged') is True])
summary['excluded_noop_page_navigation'] = sum(v['command'] == 'nav.page' and v.get('positionChanged') is not True for v in actions)
summary['first_open_in_process'] = stats(values['openFrameMs'][:1])
summary['same_process_document_reopen'] = stats(values['openFrameMs'][1:])
values['openTimingMode'] = 'First sample opens in a fresh application process; subsequent samples explicitly close/reopen the document while retaining process-global/browser/font and OS caches.'
cold_samples, cold_failures, cold_maximum_pressure = [], 0, 0
for trial in range(a.cold_process_runs):
cold_output = a.output/'cold'/kind/f'{trial + 1:03}.json'
cold_output.parent.mkdir(parents=True, exist_ok=True)
cold_output.unlink(missing_ok=True)
with tempfile.TemporaryDirectory(prefix='docview-cold-') as cold:
cold_env = env.copy()
for variable, leaf in [('XDG_CONFIG_HOME', 'config'), ('XDG_STATE_HOME', 'state'), ('XDG_CACHE_HOME', 'cache')]:
cold_env[variable] = str(pathlib.Path(cold)/leaf)
cold_command = [str(a.binary.resolve()), '--config', str(config), '--benchmark-output', str(cold_output.resolve()),
'--benchmark-runs', '1', '--benchmark-operations', '0', '--benchmark-close-cycles', '0', '--benchmark-scroll-steps', '0', str(source)]
try:
cold_run = subprocess.run(cold_command, env=cold_env, check=False, timeout=45)
cold_value = json.loads(cold_output.read_text()) if cold_output.exists() else {}
except (subprocess.TimeoutExpired, json.JSONDecodeError, OSError):
cold_run, cold_value = None, {}
pressure = cold_value.get('maximumMemoryPressureLevel')
if isinstance(pressure, int):
cold_maximum_pressure = max(cold_maximum_pressure, pressure)
if cold_run is None or cold_run.returncode or not cold_value.get('success') or pressure != 0 or len(cold_value.get('openFrameMs', [])) != 1:
cold_failures += 1
else:
cold_samples.append(cold_value['openFrameMs'][0])
summary['fresh_process_fresh_app_cache_open'] = stats(cold_samples)
values['coldProcessOpen'] = {'requested': a.cold_process_runs, 'failed': cold_failures,
'maximumMemoryPressureLevel': cold_maximum_pressure,
'samplesMs': cold_samples, 'status': 'measured' if a.cold_process_runs else 'not-measured',
'method': 'New application process and empty XDG config/state/cache roots per sample. System fonts and OS file cache retained; not fully OS-cold. Qt startup precedes the timed file-open event.'}
if cold_failures: failures.append(kind+': cold process open failure')
summary['navigation_series'] = {name: stats([v['finalQualityFrameMs'] for v in actions if v.get('series') == name])
for name in ['page-scroll', 'heading', 'chapter']}
summary['continuous_scroll_frames'] = stats(values.get('continuousScroll', {}).get('frameIntervalMs', []))
summary['continuous_scroll_input_dispatch'] = stats(values.get('continuousScroll', {}).get('dispatchIntervalMs', []))
summary['ui_timer_intervals'] = stats(values.get('uiTimerIntervalMs', []))
expected = values.get('uiTimerExpectedIntervalMs', 16)
summary['ui_timer_excess_delay_ms'] = max([0]+[max(0, v-expected) for v in values.get('uiTimerIntervalMs', [])])
cycles = values.get('openCloseCycles', [])
summary['explicit_close_cycles'] = len(cycles)
uniform_cycles = [v for v in cycles if v.get('workload') == 'open-first-visible-only']
summary['uniform_open_close_cycles'] = len(uniform_cycles)
summary['all_closes_empty'] = bool(cycles) and all(v['stateEmpty'] and v['cacheBytesAfterClose'] == 0 for v in cycles)
for name in ['applicationBytes', 'processGroupBytes']:
samples = [v['afterClose'][name] for v in uniform_cycles if v['afterClose'].get('available')]
if samples:
summary['post_close_'+name] = {'n': len(samples), 'first': samples[0], 'last': samples[-1],
'min': min(samples), 'max': max(samples), 'last_minus_first': samples[-1]-samples[0],
'interpretation': 'Finite repeated-close observation; retained RSS alone is not a leak verdict.'}
values['summary'] = summary
values['navigationCoverage'] = {v['series']: {'status': v['status'], 'measured': v['count'],
'minimum100Operations': v['count'] >= 100 if v['status'] == 'measured' else None}
for v in values.get('operationSeries', [])}
values['proposedBudgetObservations'] = {
'visual_response': budget(summary['response'], 50),
'app_cache_cold_open': budget(summary['fresh_process_fresh_app_cache_open'], 2000 if kind in ('pdf', 'pdf-headings', 'html') else 5000),
'cached_pdf_navigation': budget(summary['cached_pdf_navigation'], 100),
'uncached_pdf_navigation': budget(summary['uncached_pdf_navigation'], 500),
'continuous_scroll': budget(summary['continuous_scroll_frames'], 33),
'process_group_memory': {'limit_bytes': 1024**3, 'observed_met': values['processGroupRssPeakBytes'] <= 1024**3} if values['processGroupRssPeakBytes'] >= 0 else {'status': 'not-measured'},
'tile_cache_within_configured_budget': values['tileCachePeakChargedBytes'] <= values['tileCacheBudgetBytes'],
'acceptance': 'Informational comparison with original proposed budgets; not reference-hardware/target-OS acceptance.'}
values['originalUnchanged'] = source_hash == file_hash(source) == values['documentSha256']
values['corpus'] = a.corpus
values['executableSha256'] = executables
values['runnerSha256'] = runner_hash
checks = {'requested_open_timings': len(values['openFrameMs']) == a.runs,
'full_resolution_conditions': values.get('maximumMemoryPressureLevel') == 0 and cold_maximum_pressure == 0,
'uniform_open_close_cycles': len(uniform_cycles) >= a.close_cycles,
'close_releases_document': summary['all_closes_empty'],
'source_unchanged': values['originalUnchanged'],
'navigation_moved': all(v.get('positionChanged') is True for v in actions if v.get('series') in ('heading', 'chapter')),
'page_navigation_moved': summary['excluded_noop_page_navigation'] == 0,
'scroll_dispatched': values.get('continuousScroll', {}).get('dispatchedInputs') == a.scroll_steps}
values['measurementChecks'] = checks
values['success'] = values['success'] and not cold_failures and all(checks.values())
if not values['success']:
failures.append(kind+': measurement failed')
result.write_text(json.dumps(values, ensure_ascii=False, indent=2)+'\n', encoding='utf-8')
print(kind, json.dumps(summary), flush=True)
if failures:
raise SystemExit('; '.join(failures))
tests/cmyk_lut/correct_intent_oracle.py
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Preserve the original PDF bytes while correcting pattern entry-state expectations."""
import argparse
import hashlib
import json
from pathlib import Path
import shutil
ROOT = Path(__file__).resolve().parents[2]
ORIGINAL = ROOT / 'tests/fixtures/pdf/rendering-intents'
MANIFEST_SHA = 'b0db4ee3e57658b09d0ae10fcf8175986c3ab80ccd64442e3718802257ecf543'
def sha(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
assert sha(ORIGINAL / 'manifest.json') == MANIFEST_SHA
spec = json.loads((ORIGINAL / 'manifest.json').read_text())
files = {spec['file']: spec['sha256'], **{
row['file']: row['sha256'] for row in spec['profiles'].values()}}
for name, expected in files.items():
assert sha(ORIGINAL / name) == expected
spec['oracleRevision'] = 2
spec['oracleCorrection'] = {
'sourceManifestSha256': MANIFEST_SHA,
'correctorSha256': sha(Path(__file__)),
'case': 'shading-pattern-inherit',
'affectedProbesPerScale': 8,
'reason': 'A shading pattern without an ExtGState RI uses the graphics state at entry to its defining content stream, not the paint-time RI. These patterns are defined in page streams whose entry RI is RelativeColorimetric.',
'reference': 'https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/pdfreference1.4.pdf',
'section': '4.6.3 Table 4.23, printed page 231',
'pdfAndProfilesByteIdentical': True,
}
changed = []
for index, probe in enumerate(spec['probes']):
if probe['case'] == 'shading-pattern-inherit':
changed.append({'probeIndex': index, 'previousIntentIndex': probe['expectedIntentIndex']})
probe['expectedIntentIndex'] = 1
probe['expectedIntent'] = 'RelativeColorimetric'
assert len(changed) == 8
spec['oracleCorrection']['rows'] = changed
args.output.mkdir(parents=True, exist_ok=False)
for name in files:
shutil.copyfile(ORIGINAL / name, args.output / name)
(args.output / 'manifest.json').write_text(json.dumps(spec, indent=2) + '\n')
print(json.dumps({'probes': len(spec['probes']), 'correctedRows': len(changed),
'manifestSha256': sha(args.output / 'manifest.json')}))
if __name__ == '__main__':
main()
tests/cmyk_lut/diagnose.py
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env python3
"""Explain the retained failing CMYK comparison without relaxing its criterion."""
from io import BytesIO
import hashlib
import json
from pathlib import Path
from PIL import Image, ImageCms
from pypdf import PdfReader
ROOT = Path(__file__).resolve().parents[2]
RESULTS = ROOT / 'tests/results/cmyk-lut'
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def main():
corpus=ROOT/'tests/fixtures/pdf/cmyk-lut'
spec=json.loads((corpus/'manifest.json').read_text())
pdf=PdfReader(corpus/spec['file'])
assert b'/RelativeColorimetric ri' in pdf.pages[0].get_contents().get_data()
assert pdf.pages[0]['/Resources']['/XObject']['/Sample']['/Intent']=='/RelativeColorimetric'
source=json.loads((RESULTS/'pdfium-source/record.json').read_text())
for row in source['files']:
assert sha(RESULTS/'pdfium-source'/row['path'])==row['sha256']
transform_source=(RESULTS/'pdfium-source/core/fxcodec/icc/icc_transform.cpp').read_text()
assert 'TYPE_BGR_8, INTENT_PERCEPTUAL, /*dwFlags=*/0' in transform_source
profile=ImageCms.ImageCmsProfile(BytesIO((corpus/spec['profileFile']).read_bytes()))
transformations={intent:ImageCms.buildTransformFromOpenProfiles(profile,ImageCms.createProfile('sRGB'),
'CMYK','RGB',renderingIntent=value) for intent,value in [('perceptual',0),('relative',1)]}
colors={}
for probe in spec['probes']:
if probe['kind']!='icc-image': continue
pixel=Image.new('CMYK',(1,1),tuple(round(c*255) for c in probe['components']))
colors[probe['sample']]={name:list(ImageCms.applyTransform(pixel,t).getpixel((0,0))) for name,t in transformations.items()}
rows=[]
for os_name in ('arch','ubuntu'):
report=json.loads((RESULTS/os_name/'report.json').read_text())
assert not report['success'] and report['fixture']['sha256']==spec['sha256']
assert report['originalUnchanged'] and report['binariesUnchanged']
assert report['runnerSha256']==sha(ROOT/'tests/cmyk_lut/run.py')
for name,digest in report['evidenceSha256'].items():
assert sha(RESULTS/os_name/name)==digest
failed=0; matches=[]
for scale in report['scales']:
failed+=sum(not p['success'] for p in scale['probes'])
for probe in scale['probes']:
if probe['kind']!='icc-image': continue
pair=colors[probe['sample']]
errors={name:max(abs(a-b) for a,b in zip(probe['actualRgb'][name],pair[intent]))
for name,intent in [('pdfium','perceptual'),('poppler','relative')]}
matches.append({'scale':scale['scale'],'sample':probe['sample'],'actualRgb':probe['actualRgb'],
'cmmIntentColors':pair,'errorsAgainstRespectiveIntent':errors})
failed_by_renderer={name:sum(p['maxChannelError'].get(name,0)>spec['tolerance8Bit']
for scale in report['scales'] for p in scale['probes'])
for name in ('pdfium','poppler')}
maximum=max(p['maxChannelError'].get('pdfium',0) for scale in report['scales'] for p in scale['probes'])
assert failed==failed_by_renderer['pdfium']==30 and failed_by_renderer['poppler']==0 and maximum==74
assert all(max(m['errorsAgainstRespectiveIntent'].values())<=1 for m in matches)
rows.append({'os':os_name,'comparisonReportSha256':sha(RESULTS/os_name/'report.json'),
'analyticProbes':48,'failedPdfiumAnalyticProbes':failed,'failedPopplerAnalyticProbes':failed_by_renderer['poppler'],
'maximumPdfiumAnalyticChannelDifference':maximum,'intentMatchedImageProbes':matches})
result={'diagnosisVerified':True,'renderingAcceptance':False,'relativeIntentPreserved':False,
'toleranceUnchanged':spec['tolerance8Bit'],'profileSha256':spec['profileSha256'],
'sourceRevision':source['revision'],'sourceEvidenceSha256':sha(RESULTS/'pdfium-source/record.json'),
'diagnoserSha256':sha(Path(__file__)),'cmmVersion':ImageCms.core.littlecms_version,'results':rows,
'finding':'At this fixed PDFium revision, ICC conversion selects INTENT_PERCEPTUAL regardless of the '
'relative-colorimetric intent in the test PDF. On both OSs, all eight image swatches at all '
'three scales match LittleCMS perceptual output within one channel level. Poppler matches '
'relative-colorimetric output. This explains, but does not accept or repair, the difference.',
'remaining':'G-RENDER requires review and resolution/explicit compatibility judgment. No renderer '
'patch, engine substitution or user-visible detection was implemented in this comparison.'}
(RESULTS/'diagnosis.json').write_text(json.dumps(result,indent=2)+'\n')
print(json.dumps({'diagnosisVerified':True,'renderingAcceptance':False,'failedAnalyticProbes':60,
'intentMatchedImageProbes':48}))
if __name__=='__main__': main()
tests/cmyk_lut/export_pdfium_context_patch.py
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env python3
"""Export the second PDFium candidate, preserving the first candidate evidence."""
import difflib
import hashlib
import json
from pathlib import Path
import subprocess
import tarfile
import tempfile
ROOT = Path(__file__).resolve().parents[2]
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def main():
results = ROOT / 'tests/results/pdfium-intent-context'
materialization = json.loads((ROOT / 'tests/results/pdfium-intent-build/source-materialization.json').read_text())
repository = next(row for row in materialization['repositories'] if row['path'] == '.')
archive = ROOT / repository['archive']
inventory = ROOT / repository['inventory']
assert sha(archive) == repository['archiveSha256']
assert sha(inventory) == repository['inventorySha256']
source = ROOT / 'build-pdfium-intent-context/source'
originals = {row['path']: row for line in inventory.open()
if (row := json.loads(line))['archived'] and row['path'].startswith('core/')}
changed = {}
for name, row in originals.items():
assert row['mode'] == '100644'
if sha(source / name) != row['sha256']:
changed[name] = row
added = {str(path.relative_to(source)) for path in (source / 'core').rglob('*')
if path.is_file() and str(path.relative_to(source)) not in originals}
assert all(Path(name).suffix in ('.h', '.cpp') for name in added)
chunks = []
records = []
with tarfile.open(archive) as tar:
for name in sorted(changed.keys() | added):
before = b'' if name in added else tar.extractfile('pdfium/' + name).read()
if name in changed:
assert hashlib.sha256(before).hexdigest() == changed[name]['sha256']
after = (source / name).read_bytes()
diff = ''.join(difflib.unified_diff(
before.decode().splitlines(keepends=True),
after.decode().splitlines(keepends=True),
fromfile='/dev/null' if name in added else 'a/' + name,
tofile='b/' + name))
assert diff
chunks.append('diff --git a/' + name + ' b/' + name + '\n' + diff)
records.append({'path': name, 'added': name in added,
'beforeSha256': None if name in added else hashlib.sha256(before).hexdigest(),
'afterSha256': hashlib.sha256(after).hexdigest()})
destination = ROOT / 'cmake/patches/pdfium-rendering-intent-context.patch'
destination.parent.mkdir(parents=True, exist_ok=True)
destination.write_text(''.join(chunks))
snapshot = results / 'patches' / (sha(destination) + '.patch')
snapshot.parent.mkdir(exist_ok=True)
if snapshot.exists():
assert snapshot.read_bytes() == destination.read_bytes()
else:
snapshot.write_bytes(destination.read_bytes())
# Check that this reviewable patch, rather than an unrecorded local change,
# reproduces every modified core source without fuzzy matching.
with tempfile.TemporaryDirectory(prefix='docview-pdfium-patch-') as folder:
check_root = Path(folder)
with tarfile.open(archive) as tar:
for name in changed:
target = check_root / name
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(tar.extractfile('pdfium/' + name).read())
result = subprocess.run(['patch', '-p1', '--batch', '--fuzz=0', '-i', str(destination)],
cwd=check_root, capture_output=True, text=True)
assert result.returncode == 0, result.stdout + result.stderr
for row in records:
assert sha(check_root / row['path']) == row['afterSha256']
record = {'success': True, 'exporterSha256': sha(Path(__file__)),
'source': str(source.relative_to(ROOT)),
'parentRecord': 'tests/results/pdfium-intent-build/record.json',
'parentRecordSha256': sha(ROOT / 'tests/results/pdfium-intent-build/record.json'),
'baseRevision': repository['revision'], 'archiveSha256': sha(archive),
'patch': str(destination.relative_to(ROOT)), 'patchSha256': sha(destination),
'patchSnapshot': str(snapshot.relative_to(ROOT)),
'appliesWithoutFuzz': True, 'appliedSourcesByteIdentical': True,
'files': records,
'scope': 'Candidate core patch only. Provider shared-library/public-header patches are recorded separately. Production dependency is not replaced by this exporter.'}
(results / 'candidate-patch.json').write_text(json.dumps(record, indent=2) + '\n')
print(json.dumps({'files': len(records), 'patchBytes': destination.stat().st_size,
'patchSha256': record['patchSha256']}))
if __name__ == '__main__':
main()
tests/cmyk_lut/export_pdfium_patch.py
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/env python3
"""Export the isolated PDFium core changes against the verified fixed archive."""
import difflib
import hashlib
import json
from pathlib import Path
import subprocess
import tarfile
import tempfile
ROOT = Path(__file__).resolve().parents[2]
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def main():
results = ROOT / 'tests/results/pdfium-intent-build'
materialization = json.loads((results / 'source-materialization.json').read_text())
repository = next(row for row in materialization['repositories'] if row['path'] == '.')
archive = ROOT / repository['archive']
inventory = ROOT / repository['inventory']
assert sha(archive) == repository['archiveSha256']
assert sha(inventory) == repository['inventorySha256']
source = ROOT / materialization['source']
originals = {row['path']: row for line in inventory.open()
if (row := json.loads(line))['archived'] and row['path'].startswith('core/')}
changed = {}
for name, row in originals.items():
assert row['mode'] == '100644'
if sha(source / name) != row['sha256']:
changed[name] = row
added = {str(path.relative_to(source)) for path in (source / 'core').rglob('*')
if path.is_file() and str(path.relative_to(source)) not in originals}
assert all(Path(name).suffix in ('.h', '.cpp') for name in added)
chunks = []
records = []
with tarfile.open(archive) as tar:
for name in sorted(changed.keys() | added):
before = b'' if name in added else tar.extractfile('pdfium/' + name).read()
if name in changed:
assert hashlib.sha256(before).hexdigest() == changed[name]['sha256']
after = (source / name).read_bytes()
diff = ''.join(difflib.unified_diff(
before.decode().splitlines(keepends=True),
after.decode().splitlines(keepends=True),
fromfile='/dev/null' if name in added else 'a/' + name,
tofile='b/' + name))
assert diff
chunks.append('diff --git a/' + name + ' b/' + name + '\n' + diff)
records.append({'path': name, 'added': name in added,
'beforeSha256': None if name in added else hashlib.sha256(before).hexdigest(),
'afterSha256': hashlib.sha256(after).hexdigest()})
destination = ROOT / 'cmake/patches/pdfium-rendering-intent.patch'
destination.parent.mkdir(parents=True, exist_ok=True)
destination.write_text(''.join(chunks))
snapshot = results / 'patches' / (sha(destination) + '.patch')
snapshot.parent.mkdir(exist_ok=True)
if snapshot.exists():
assert snapshot.read_bytes() == destination.read_bytes()
else:
snapshot.write_bytes(destination.read_bytes())
# Check that this reviewable patch, rather than an unrecorded local change,
# reproduces every modified core source without fuzzy matching.
with tempfile.TemporaryDirectory(prefix='docview-pdfium-patch-') as folder:
check_root = Path(folder)
with tarfile.open(archive) as tar:
for name in changed:
target = check_root / name
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(tar.extractfile('pdfium/' + name).read())
result = subprocess.run(['patch', '-p1', '--batch', '--fuzz=0', '-i', str(destination)],
cwd=check_root, capture_output=True, text=True)
assert result.returncode == 0, result.stdout + result.stderr
for row in records:
assert sha(check_root / row['path']) == row['afterSha256']
record = {'success': True, 'exporterSha256': sha(Path(__file__)),
'baseRevision': repository['revision'], 'archiveSha256': sha(archive),
'patch': str(destination.relative_to(ROOT)), 'patchSha256': sha(destination),
'patchSnapshot': str(snapshot.relative_to(ROOT)),
'appliesWithoutFuzz': True, 'appliedSourcesByteIdentical': True,
'files': records,
'scope': 'Candidate core patch only. Provider shared-library/public-header patches are recorded separately. Production dependency is not replaced by this exporter.'}
(results / 'candidate-patch.json').write_text(json.dumps(record, indent=2) + '\n')
print(json.dumps({'files': len(records), 'patchBytes': destination.stat().st_size,
'patchSha256': record['patchSha256']}))
if __name__ == '__main__':
main()
tests/cmyk_lut/fetch_pdfium_build_tools.py
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env python3
"""Fetch only the fixed Linux GN, Clang and sysroot inputs for the isolated build."""
from concurrent.futures import ThreadPoolExecutor
import hashlib
import json
from pathlib import Path
import time
import urllib.request
ROOT=Path(__file__).resolve().parents[2]
def sha(path):
with path.open('rb') as stream: return hashlib.file_digest(stream,'sha256').hexdigest()
def main():
pins=json.loads((ROOT/'tests/results/source-correspondence/pdfium/dependency-pins.json').read_text())
source=ROOT/'build-pdfium-intent/source'
clang=next(row for row in pins['root']['deps']['third_party/llvm-build/Release+Asserts']['objects']
if row['object_name'].startswith('Linux_x64/clang-llvmorg-'))
sysroot=json.loads((source/'build/linux/sysroot_scripts/sysroots.json').read_text())['bullseye_amd64']
gn_version=pins['root']['vars']['gn_version']
inputs=[{'name':'clang.tar.xz','url':'https://commondatastorage.googleapis.com/chromium-browser-clang/'+clang['object_name']+'?generation='+str(clang['generation']),
'expectedSha256':clang['sha256sum'],'expectedBytes':clang['size_bytes']},
{'name':'sysroot.tar.xz','url':sysroot['URL']+'/'+sysroot['Sha256Sum'],'expectedSha256':sysroot['Sha256Sum']},
{'name':'gn.zip','url':'https://chrome-infra-packages.appspot.com/dl/gn/gn/linux-amd64/+/'+gn_version,
'cipdPackage':'gn/gn/linux-amd64','cipdVersion':gn_version}]
cache=ROOT/'build-pdfium-intent/downloads'; cache.mkdir(parents=True,exist_ok=False)
def fetch(row):
path=cache/row['name']; total=0; started=time.monotonic()
with urllib.request.urlopen(row['url'],timeout=60) as response, path.open('xb') as stream:
content_length=response.headers.get('Content-Length')
if content_length: assert int(content_length)<=512*1024**2
for chunk in iter(lambda:response.read(1024*1024),b''):
total+=len(chunk)
assert total<=512*1024**2 and time.monotonic()-started<=600
stream.write(chunk)
digest=sha(path)
if 'expectedSha256' in row: assert row['expectedSha256']==digest
if 'expectedBytes' in row: assert row['expectedBytes']==total
print(row['name']+' '+str(total)+' bytes; SHA-256 '+digest,flush=True)
return {**row,'path':str(path.relative_to(ROOT)),'bytes':total,'sha256':digest}
with ThreadPoolExecutor(max_workers=3) as pool: rows=list(pool.map(fetch,inputs))
record={'success':True,'fetcherSha256':sha(Path(__file__)),'inputs':rows,
'scope':'Clang and sysroot checked against source-pinned SHA-256. GN fetched from the official CIPD fixed Git-revision selector; archive SHA-256 recorded. No downloaded executable or installer has been run.'}
(ROOT/'tests/results/pdfium-intent-build/tool-downloads.json').write_text(json.dumps(record,indent=2)+'\n')
if __name__=='__main__':main()
tests/cmyk_lut/intents.py
+307
View File
@@ -0,0 +1,307 @@
#!/usr/bin/env python3
"""Strict ICC rendering-intent probes against direct CMM, PDFium and Poppler.
The direct oracle never reads rendered pixels. It uses the embedded profile and
an independently selected expected intent. It may share LittleCMS algorithms
with the renderers, and is not claimed to be an independent CMM implementation.
"""
import argparse
import ctypes as C
import ctypes.util
from io import BytesIO
import hashlib
import itertools
import json
import math
import os
from pathlib import Path
import platform
import re
import shutil
import struct
import subprocess
from PIL import Image, ImageChops, ImageCms, ImageDraw
from pypdf import PdfReader
ROOT=Path(__file__).resolve().parents[2]
CORPUS=ROOT/'tests/fixtures/pdf/rendering-intents'
INTENTS=('Perceptual','RelativeColorimetric','Saturation','AbsoluteColorimetric')
def sha(path):
with path.open('rb') as source:return hashlib.file_digest(source,'sha256').hexdigest()
class Cmm:
"""Public LittleCMS API, normalized CMYK doubles -> unsigned 8-bit sRGB."""
def __init__(self,profiles):
library=ctypes.util.find_library('lcms2')
if not library:raise RuntimeError('LittleCMS2 shared library is required')
self.lib=C.CDLL(library);self.transforms={};self.profiles=[];self.buffers=[]
declarations={
'cmsOpenProfileFromMem':([C.c_void_p,C.c_uint32],C.c_void_p),
'cmsCreate_sRGBProfile':([],C.c_void_p),
'cmsCreateTransform':([C.c_void_p,C.c_uint32,C.c_void_p,C.c_uint32,C.c_uint32,C.c_uint32],C.c_void_p),
'cmsDoTransform':([C.c_void_p,C.c_void_p,C.c_void_p,C.c_uint32],None),
'cmsDeleteTransform':([C.c_void_p],None),'cmsCloseProfile':([C.c_void_p],C.c_int),
'cmsGetEncodedCMMversion':([],C.c_uint32)}
for name,(args,result) in declarations.items():
function=getattr(self.lib,name);function.argtypes=args;function.restype=result
output=self.lib.cmsCreate_sRGBProfile();assert output;self.profiles.append(output)
for identifier,data in profiles.items():
buffer=C.create_string_buffer(data)
self.buffers.append(buffer)
profile=self.lib.cmsOpenProfileFromMem(buffer,len(data));assert profile;self.profiles.append(profile)
for intent in range(4):
# lcms2.h public pixel-format macros: FLOAT_SH(1), PT_CMYK=6,
# PT_RGB=4, CHANNELS_SH(n), BYTES_SH(n). Double CMYK uses %.
transform=self.lib.cmsCreateTransform(profile,(1<<22)|(6<<16)|(4<<3),
output,(4<<16)|(3<<3)|1,intent,0)
assert transform;self.transforms[identifier,intent]=transform
self.version=self.lib.cmsGetEncodedCMMversion()
self.libraryPaths=[]
maps=Path('/proc/self/maps')
if maps.exists():
paths={line.split()[-1] for line in maps.read_text().splitlines()
if '/' in line and 'liblcms2.so' in line}
self.libraryPaths=[{'file':Path(p).name,'sha256':sha(Path(p))} for p in sorted(paths) if Path(p).is_file()]
def color(self,profile,intent,values):
assert len(values)==4 and all(math.isfinite(x) and 0<=x<=1 for x in values)
source=(C.c_double*4)(*(x*100 for x in values));target=(C.c_ubyte*3)()
self.lib.cmsDoTransform(self.transforms[profile,intent],source,target,1)
return list(target)
def close(self):
for transform in self.transforms.values():self.lib.cmsDeleteTransform(transform)
for profile in self.profiles:self.lib.cmsCloseProfile(profile)
self.transforms.clear();self.profiles.clear();self.buffers.clear()
def tags_from(profile):
assert len(profile)==struct.unpack_from('>I',profile)[0]
assert profile[8:24]==bytes.fromhex('02100000')+b'scnrCMYKLab ' and profile[36:40]==b'acsp'
tags={}
for index in range(struct.unpack_from('>I',profile,128)[0]):
signature,offset,size=struct.unpack_from('>4sII',profile,132+12*index)
assert offset%4==0 and offset>=132 and offset+size<=len(profile) and signature not in tags
tags[signature]=profile[offset:offset+size]
return tags
def verify_fixture(corpus,spec):
source=corpus/spec['file'];assert sha(source)==spec['sha256']
assert sha(ROOT/'tests/fixtures/pdf/generate_rendering_intents.py')==spec['generatorSha256']
corrected=spec.get('oracleRevision')==2
if corrected:
correction=spec['oracleCorrection']
assert correction['correctorSha256']==sha(ROOT/'tests/cmyk_lut/correct_intent_oracle.py')
assert correction['sourceManifestSha256']=='b0db4ee3e57658b09d0ae10fcf8175986c3ab80ccd64442e3718802257ecf543'
original=ROOT/'tests/fixtures/pdf/cmyk-lut/synthetic-cmyk-lab.icc'
assert sha(original)==spec['baseProfileSha256']
profiles={identifier:(corpus/item['file']).read_bytes() for identifier,item in spec['profiles'].items()}
assert profiles['original']==original.read_bytes()
vertex_count=0
# Independent coefficient representation of the written linear CLUTs.
coefficients={b'A2B0':[(100,(-12,-15,-10,-35)),(0,(-15,18,0,0)),(0,(-12,0,20,0))],
b'A2B1':[(100,(-20,-20,-20,-40)),(0,(-20,40,0,-20)),(0,(-20,0,40,-20))],
b'A2B2':[(100,(-25,-25,-20,-30)),(0,(20,-40,0,20)),(0,(0,-20,35,-15))]}
for identifier,data in profiles.items():
item=spec['profiles'][identifier]
assert hashlib.sha256(data).hexdigest()==item['sha256'] and len(data)==item['size']
tags=tags_from(data)
assert set(tags)=={b'desc',b'cprt',b'wtpt',*(t.encode() for t in item['tables'])}
white=struct.unpack_from('>3i',tags[b'wtpt'],8)
assert all(abs(actual/65536-expected)<=1/65536 for actual,expected in zip(white,item['mediaWhitePoint']))
for tag in item['tables']:
lut=tags[tag.encode()]
assert len(lut)==176 and lut[:12]==b'mft2'+bytes(4)+bytes((4,3,2,0))
assert struct.unpack_from('>HH',lut,48)==(2,2)
assert lut[52:68]==struct.pack('>HH',0,65535)*4 and lut[164:176]==struct.pack('>HH',0,65535)*3
for i,values in enumerate(itertools.product((0,1),repeat=4)):
encoded=struct.unpack_from('>3H',lut,68+i*6)
actual=(encoded[0]*100/65280,encoded[1]/256-128,encoded[2]/256-128)
expected=[offset+sum(a*b for a,b in zip(vector,values)) for offset,vector in coefficients[tag.encode()]]
assert max(abs(a-b) for a,b in zip(actual,expected))<.002
vertex_count+=1
pdf=PdfReader(source);assert len(pdf.pages)==spec['pageCount']
identities={}
for index,page in enumerate(pdf.pages,1):
assert list(page.mediabox)==[0,0,*spec['pageSizePt']]
probes=[p for p in spec['probes'] if p['page']==index];assert probes
identifier=probes[0]['profile'];resources=page['/Resources'];space=resources['/ColorSpace']['/CS']
# pypdf 4 leaves array members indirect; dictionary lookup alone dereferences.
embedded_profile=space[1].get_object()
assert space[0]=='/ICCBased' and embedded_profile['/N']==4 and embedded_profile.get_data()==profiles[identifier]
assert resources['/ColorSpace']['/IX'][0]=='/Indexed'
lookup=resources['/ColorSpace']['/IX'][3]
assert (lookup.original_bytes if isinstance(lookup,str) else bytes(lookup))==bytes(spec['sampleBytes'])
xobjects=resources['/XObject'];shared=xobjects.raw_get('/Shared')
identities.setdefault(identifier,shared.idnum);assert identities[identifier]==shared.idnum
assert '/Intent' not in xobjects['/Shared'] and xobjects['/Shared'].get_data()==bytes(spec['sampleBytes'])
assert xobjects['/Indexed'].get_data()==b'\0'
for font_name in ('T3Rect','T3Image',*(f'T3Override{i}' for i in range(4))):
glyph_font=resources['/Font']['/'+font_name]
assert glyph_font['/Subtype']=='/Type3' and glyph_font['/FirstChar']==glyph_font['/LastChar']==65
glyph=glyph_font['/CharProcs']['/A'].get_data();assert glyph.startswith(b'1000 0 d0\n')
if font_name=='T3Image':assert b'/Shared Do' in glyph
elif font_name.startswith('T3Override'):assert ('/'+INTENTS[int(font_name[-1])]+' ri').encode() in glyph
else:assert b' ri' not in glyph
assert '/RelativeColorimetric ri' not in xobjects['/Inherited'].get_data().decode()
for i,intent in enumerate(INTENTS):
assert resources['/ExtGState']['/I'+str(i)]['/RI']=='/'+intent
assert xobjects['/Image'+str(i)]['/Intent']=='/'+intent
assert ('/'+intent+' ri').encode() in xobjects['/Form'+str(i)].get_data()
assert ('/I'+str(i)+' gs').encode() in xobjects['/FormGs'+str(i)].get_data()
assert resources['/Pattern']['/P'+str(i)]['/ExtGState']['/RI']=='/'+intent
content=page.get_contents().get_data()
for probe in probes:
assert ('\n'.join(probe['commandSequence'])+'\n').encode() in content
expected=probe['expectedIntentIndex'];assert INTENTS[expected]==probe['expectedIntent']
current=1 if probe['graphicsStateIntent']=='default' else INTENTS.index(probe['graphicsStateIntent'])
if corrected and probe['case']=='shading-pattern-inherit':
assert expected==1
elif probe['case'] in ('nested-q-inner','form-ri-override','form-gs-override','image-intent-override','shading-pattern-RI-override','type3-ri-override'):
assert expected==(current+1)%4
else:assert expected==current
assert len(spec['probes'])==len(spec['cases'])*4*2
return profiles,{'pagesVerified':len(pdf.pages),'probesVerified':len(spec['probes']),
'clutVerticesVerified':vertex_count,'sharedImageObjectIds':identities,
'originalProfileByteIdentical':True,'oracleRevision':2 if corrected else 1,
'knownIncorrectPatternOracle':not corrected}
def cmm_checks(profiles,spec,cmm):
sample=tuple(spec['sampleBytes']);checks=[];colors={};bpc_checks=[]
for identifier,data in profiles.items():
profile=ImageCms.ImageCmsProfile(BytesIO(data));colors[identifier]=[]
for intent in range(4):
expected=cmm.color(identifier,intent,[x/255 for x in sample]);colors[identifier].append(expected)
transform=ImageCms.buildTransformFromOpenProfiles(profile,ImageCms.createProfile('sRGB'),'CMYK','RGB',renderingIntent=intent)
actual=list(ImageCms.applyTransform(Image.new('CMYK',(1,1),sample),transform).getpixel((0,0)))
error=max(abs(a-b) for a,b in zip(actual,expected))
checks.append({'profile':identifier,'intent':INTENTS[intent],'directCmmRgb':expected,
'pillow8BitRgb':actual,'maxChannelError':error,'success':error<=1})
if identifier=='distinct':
# Pillow 10.2 (Ubuntu 24.04) exposes the same public flag in
# FLAGS; later releases expose the Flags enum instead.
bpc_flag=(ImageCms.Flags.BLACKPOINTCOMPENSATION if hasattr(ImageCms,'Flags')
else ImageCms.FLAGS['BLACKPOINTCOMPENSATION'])
assert int(bpc_flag)==8192
with_bpc=ImageCms.buildTransformFromOpenProfiles(profile,ImageCms.createProfile('sRGB'),'CMYK','RGB',
renderingIntent=intent,flags=bpc_flag)
bpc=list(ImageCms.applyTransform(Image.new('CMYK',(1,1),sample),with_bpc).getpixel((0,0)))
assert max(abs(a-b) for a,b in zip(bpc,expected))<=1
bpc_checks.append({'intent':INTENTS[intent],'rgb':bpc,'flags':8192,'matchesWithoutBpc':True})
assert all(c['success'] for c in checks)
separation=[max(abs(a-b) for a,b in zip(colors['distinct'][i],colors['distinct'][j])) for i,j in itertools.combinations(range(4),2)]
assert min(separation)>2*spec['tolerance8Bit'],'Derived profile must distinguish every intent beyond tolerance'
return {'checks':checks,'distinctProfileBlackPointCompensationChecks':bpc_checks,
'distinctProfileMinimumPairwiseMaxChannelSeparation':min(separation),
'originalProfileIntentEquivalence':'Relative/Saturation/Absolute may coincide; distinct profile covers their dispatch.'}
def loaded_pdfium(worker,environment,requested):
text=subprocess.run(['ldd',str(worker)],capture_output=True,text=True,env=environment,check=True,timeout=30).stdout
match=re.search(r'^\s*libpdfium\.so\s+=>\s+(/.+?)\s+\(0x',text,re.M)
assert match,'ldd did not resolve worker libpdfium.so'
path=Path(match[1]).resolve(strict=True)
if requested:assert path==requested.resolve(strict=True),'Requested library lost to loader/RPATH precedence'
return {'path':str(path),'sha256':sha(path),'ldd':text,'selectionVerified':True}
def main():
parser=argparse.ArgumentParser(description=__doc__)
parser.add_argument('--build-dir',type=Path)
parser.add_argument('--pdfium-library',type=Path,help='Prepend this library directory to LD_LIBRARY_PATH and verify ldd resolves it')
parser.add_argument('--corpus',type=Path,default=CORPUS)
parser.add_argument('--output',type=Path,required=True)
parser.add_argument('--self-check',action='store_true',help='Validate fixture/CMM only; not rendering acceptance')
parser.add_argument('--scales',type=float,nargs='+',default=[.5,1.,2.])
args=parser.parse_args()
if not args.self_check and not args.build_dir:parser.error('--build-dir is required unless --self-check')
if not 1<=len(args.scales)<=4 or any(not math.isfinite(s) or s<.25 or s>4 for s in args.scales):parser.error('scales must be 1..4 finite values in .25..4')
if len(set(args.scales))!=len(args.scales):parser.error('scales must be unique')
output=args.output.resolve();output.mkdir(parents=True,exist_ok=False)
spec=json.loads((args.corpus/'manifest.json').read_text());source=(args.corpus/spec['file']).resolve()
report={'schemaVersion':1,'success':False,'fixtureValidationSuccess':False,'renderingAcceptance':False,
'mode':'fixture-self-check' if args.self_check else 'renderer-comparison','commands':[],'scales':[],
'runnerSha256':sha(Path(__file__)),'fixtureManifestSha256':sha(args.corpus/'manifest.json'),
'fixtureSha256':sha(source),'pythonVersion':platform.python_version(),'pillowLcmsVersion':ImageCms.core.littlecms_version,
'osRelease':platform.freedesktop_os_release(),'tolerance8Bit':spec['tolerance8Bit'],
'oracleRevision':spec.get('oracleRevision',1),'oracleKnownIssue':spec.get('oracleRevision')!=2,
'scope':'Direct explicit-intent CMM expected colors; production isolated PDF worker and independent Poppler. '
'CMM may share LittleCMS code with PDF renderers. Synthetic profiles only; no physical display/press, '
'human-approved golden or general ICC conformance assertion.'}
cmm=None;binaries={};environment=os.environ.copy()
try:
profiles,verified=verify_fixture(args.corpus,spec);report['fixtureVerification']=verified
cmm=Cmm(profiles);report['directCmm']={'encodedVersion':cmm.version,'libraries':cmm.libraryPaths,
'inputFormat':'TYPE_CMYK_DBL, 0..100 percent','outputFormat':'TYPE_RGB_8, cmsCreate_sRGBProfile','flags':0}
report['cmmValidation']=cmm_checks(profiles,spec,cmm);report['fixtureValidationSuccess']=True
if not args.self_check:
build=args.build_dir.resolve();binaries={name:sha(build/name) for name in ('pdf-worker-evidence','docview-pdf-worker')}
report['binaries']=binaries
if args.pdfium_library:
environment['LD_LIBRARY_PATH']=str(args.pdfium_library.resolve().parent)+(':'+environment['LD_LIBRARY_PATH'] if environment.get('LD_LIBRARY_PATH') else '')
assert not environment.get('LD_PRELOAD') and not environment.get('LD_AUDIT'),'Loader injection would invalidate evidence'
report['pdfiumLibrary']=loaded_pdfium(build/'docview-pdf-worker',environment,args.pdfium_library)
for program in ('qpdf','pdftoppm'):assert shutil.which(program),program+' required'
report['popplerVersion']=subprocess.run(['pdftoppm','-v'],capture_output=True,text=True,check=True).stderr.splitlines()[0]
def run(label,command,env=None):
log=output/(label+'.log')
with log.open('w') as f:result=subprocess.run(command,stdout=f,stderr=subprocess.STDOUT,env=env,timeout=300)
report['commands'].append({'name':label,'command':command,'exitCode':result.returncode,'logSha256':sha(log)})
assert result.returncode==0,label+' failed'
run('qpdf-check',['qpdf','--check',str(source)])
for scale in args.scales:
label='scale-'+str(scale).replace('.','_');folder=output/label;(folder/'pdfium').mkdir(parents=True)
run(label+'-pdfium',[str(build/'pdf-worker-evidence'),str(source),str(folder/'pdfium'),str(scale)],environment)
run(label+'-poppler',['pdftoppm','-cropbox','-r',str(72*scale),'-png',str(source),str(folder/'poppler')])
metadata=json.loads((folder/'pdfium/metadata.json').read_text())
assert metadata['pageCount']==spec['pageCount'] and metadata['renderScale']==scale and 'sandbox enabled' in metadata['transport']
row={'scale':scale,'probes':[],'pdfiumSuccess':True,'popplerSuccess':True,'pages':[]}
for page in range(1,spec['pageCount']+1):
paths={'pdfium':folder/'pdfium'/f'page-{page}.png','poppler':folder/f'poppler-{page}.png'}
images={key:Image.open(path).convert('RGB') for key,path in paths.items()}
dimensions=[math.ceil(v*scale) for v in spec['pageSizePt']]
assert all(list(image.size)==dimensions for image in images.values())
for probe in (p for p in spec['probes'] if p['page']==page):
x,y=probe['pointPt'];pixel=(round(x*scale),round((spec['pageSizePt'][1]-y)*scale))
components=probe['components']
if 'shading' in probe:
shading=probe['shading'];t=((pixel[0]+.5)/scale-shading['xStart'])/(shading['xEnd']-shading['xStart'])
components=[a+(b-a)*t for a,b in zip(shading['c0'],shading['c1'])]
expected=cmm.color(probe['profile'],probe['expectedIntentIndex'],components)
actual={key:list(image.getpixel(pixel)) for key,image in images.items()}
errors={key:max(abs(a-b) for a,b in zip(color,expected)) for key,color in actual.items()}
matches={key:error<=spec['tolerance8Bit'] for key,error in errors.items()}
row['probes'].append({**probe,'pointPx':pixel,'sampledComponents':components,'expectedRgb':expected,
'actualRgb':actual,'maxChannelError':errors,'matches':matches,'success':all(matches.values())})
for key,matched in matches.items():row[key+'Success'] &= matched
diff=ImageChops.difference(images['pdfium'],images['poppler']);diff.save(folder/f'difference-{page}.png')
sheet=Image.new('RGB',(1260,420),'#eeeeee');draw=ImageDraw.Draw(sheet)
for column,(label2,image) in enumerate([*images.items(),('difference',diff)]):
draw.text((column*420+8,7),f'{scale:g}x page {page}: {label2}',fill='black')
thumb=image.copy();thumb.thumbnail((410,380));sheet.paste(thumb,(column*420+5,30))
sheet.save(folder/f'comparison-{page}.png')
row['pages'].append({'page':page,'images':{key:sha(path) for key,path in paths.items()}})
row['success']=row['pdfiumSuccess'] and row['popplerSuccess'];report['scales'].append(row)
report['originalUnchanged']=sha(source)==spec['sha256'] and all(sha(args.corpus/item['file'])==item['sha256'] for item in spec['profiles'].values())
report['binariesUnchanged']=all(sha(build/name)==value for name,value in binaries.items()) and sha(Path(report['pdfiumLibrary']['path']))==report['pdfiumLibrary']['sha256']
report['pdfiumSuccess']=all(row['pdfiumSuccess'] for row in report['scales'])
report['popplerSuccess']=all(row['popplerSuccess'] for row in report['scales'])
report['renderingAcceptance']=report['success']=report['pdfiumSuccess'] and report['popplerSuccess'] and report['originalUnchanged'] and report['binariesUnchanged'] and not report['oracleKnownIssue']
except Exception as error:
report['error']=type(error).__name__+': '+str(error)
raise
finally:
if cmm:cmm.close()
report['evidenceSha256']={str(p.relative_to(output)):sha(p) for p in sorted(output.rglob('*')) if p.is_file()}
(output/'report.json').write_text(json.dumps(report,indent=2)+'\n')
print(json.dumps({key:report.get(key) for key in ('mode','fixtureValidationSuccess','success','pdfiumSuccess','popplerSuccess')}))
if not args.self_check and not report['success']:raise SystemExit('Intent validation failed; retained report preserves every failed probe')
if __name__=='__main__':main()
tests/cmyk_lut/prepare_pdfium_source.py
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""Materialize previously verified fixed Git archives for an isolated PDFium build."""
import hashlib
import json
from pathlib import Path, PurePosixPath
import tarfile
ROOT = Path(__file__).resolve().parents[2]
def sha(path):
with path.open('rb') as stream: return hashlib.file_digest(stream,'sha256').hexdigest()
def main():
reports=[ROOT/'tests/results/source-archives/pdfium-git/report.json',
ROOT/'tests/results/source-archives/pdfium-gitlink/report.json']
repositories=[]
for path in reports:
report=json.loads(path.read_text()); assert report['success']
repositories.extend(report['repositories'])
source=ROOT/'build-pdfium-intent/source'
source.mkdir(parents=True,exist_ok=False)
output=ROOT/'tests/results/pdfium-intent-build'; output.mkdir(parents=True,exist_ok=False)
materialized=[]; links=[]; total=0; count=0
for repository in repositories:
archive=ROOT/repository['archive']; inventory=ROOT/repository['inventory']
assert sha(archive)==repository['archiveSha256'] and sha(inventory)==repository['inventorySha256']
expected={r['path']:r for line in inventory.open() if (r:=json.loads(line))['archived']}
prefix='pdfium/'+('' if repository['path']=='.' else repository['path']+'/')
found=set()
with tarfile.open(archive) as stream:
for member in stream:
assert member.name.startswith(prefix)
relative=member.name.removeprefix(prefix)
if member.isdir(): continue
assert relative in expected and relative not in found
found.add(relative); row=expected[relative]
name=PurePosixPath(member.name.removeprefix('pdfium/'))
assert not name.is_absolute() and '..' not in name.parts and '\\' not in str(name)
path=source/name; assert path.resolve().is_relative_to(source) and not path.exists()
path.parent.mkdir(parents=True,exist_ok=True)
if member.issym():
assert row['mode']=='120000' and member.linkname==row['linkTarget']
target=PurePosixPath(member.linkname)
assert not target.is_absolute() and (path.parent/target).resolve().is_relative_to(source)
assert hashlib.sha256(member.linkname.encode()).hexdigest()==row['sha256']
links.append((path,member.linkname))
continue
assert member.isfile() and row['mode'] in ('100644','100755') and member.size==row['bytes']
total+=member.size; count+=1
assert member.size<=256*1024**2 and total<=2*1024**3 and count<=100000
digest=hashlib.sha256()
with stream.extractfile(member) as incoming, path.open('xb') as outgoing:
for chunk in iter(lambda:incoming.read(1024*1024),b''):
digest.update(chunk); outgoing.write(chunk)
assert digest.hexdigest()==row['sha256']
path.chmod(0o755 if row['mode']=='100755' else 0o644)
assert found==expected.keys()
materialized.append({k:repository[k] for k in ('path','revision','archive','archiveSha256','inventory','inventorySha256')})
for path,target in links:
assert (path.parent/target).resolve().is_file()
path.symlink_to(target)
record={'success':True,'preparerSha256':sha(Path(__file__)),'source':str(source.relative_to(ROOT)),
'repositories':materialized,'regularFiles':count,'regularBytes':total,'symlinks':len(links),
'sourceReports':{str(p.relative_to(ROOT)):sha(p) for p in reports},
'scope':'Fixed sources materialized for a separate local build. No source hooks, downloads or build executed by this preparer. Production dependency remains unchanged.'}
(output/'source-materialization.json').write_text(json.dumps(record,indent=2)+'\n')
print(json.dumps({'repositories':len(materialized),'regularFiles':count,'regularBytes':total,'symlinks':len(links)}))
if __name__=='__main__':main()
tests/cmyk_lut/prepare_pdfium_tools.py
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env python3
"""Materialize the three verified, pinned tools without running download hooks."""
import hashlib
import json
from pathlib import Path, PurePosixPath
import subprocess
import tarfile
import zipfile
ROOT = Path(__file__).resolve().parents[2]
SOURCE = ROOT / 'build-pdfium-intent/source'
RESULTS = ROOT / 'tests/results/pdfium-intent-build'
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def main():
downloads = json.loads((RESULTS / 'tool-downloads.json').read_text())
destinations = {
'clang.tar.xz': 'third_party/llvm-build/Release+Asserts',
'sysroot.tar.xz': 'build/linux/debian_bullseye_amd64-sysroot',
'gn.zip': 'buildtools/linux64',
}
records = []
for row in downloads['inputs']:
archive = ROOT / row['path']
assert sha(archive) == row['sha256']
destination = SOURCE / destinations[row['name']]
already_extracted = destination.exists()
destination.mkdir(parents=True, exist_ok=True)
if row['name'].endswith('.tar.xz'):
with tarfile.open(archive) as tar:
members = tar.getmembers()
assert len(members) < 30000
assert sum(member.size for member in members) < 1024**3
for member in members:
assert member.isfile() or member.isdir() or member.issym() or member.islnk()
assert not PurePosixPath(member.name).is_absolute()
assert '..' not in PurePosixPath(member.name).parts
# Python's data filter also rejects links that escape the destination.
tarfile.data_filter(member, str(destination))
if not already_extracted:
tar.extractall(destination, members=members, filter='data')
for member in members:
target = destination / member.name
if member.isfile() or member.islnk():
with tar.extractfile(member) as stream:
assert sha(target) == hashlib.file_digest(stream, 'sha256').hexdigest()
elif member.issym():
assert target.is_symlink() and str(target.readlink()) == member.linkname
else:
with zipfile.ZipFile(archive) as zip_file:
assert set(zip_file.namelist()) == {'gn', '.cipdpkg/manifest.json'}
manifest = json.loads(zip_file.read('.cipdpkg/manifest.json'))
assert manifest['package_name'] == row['cipdPackage']
for member in zip_file.infolist():
assert member.file_size < 16 * 1024**2
target = destination / member.filename
target.parent.mkdir(parents=True, exist_ok=True)
if not already_extracted:
target.write_bytes(zip_file.read(member))
target.chmod((member.external_attr >> 16) & 0o777)
assert target.read_bytes() == zip_file.read(member)
records.append({'archive': row['path'], 'sha256': row['sha256'],
'destination': str(destination.relative_to(ROOT))})
stamp = SOURCE / 'third_party/llvm-build/Release+Asserts/cr_build_revision'
stamp.write_text('llvmorg-24-init-3796-g20e97c4b-4\n')
gclient = SOURCE / 'build/config/gclient_args.gni'
gclient_text = ('android_ndk_version = "[email protected]"\n'
'build_with_chromium = false\ncheckout_android = false\n'
'checkout_libpng = true\ncheckout_skia = false\n')
if gclient.exists():
assert gclient.read_text() == gclient_text
else:
gclient.write_text(gclient_text)
versions = {}
for name, relative in [('gn', 'buildtools/linux64/gn'),
('clang', 'third_party/llvm-build/Release+Asserts/bin/clang'),
('lld', 'third_party/llvm-build/Release+Asserts/bin/ld.lld')]:
result = subprocess.run([str(SOURCE / relative), '--version'], check=True,
capture_output=True, text=True)
versions[name] = result.stdout.strip()
assert '150a9d6ba0aa' in versions['gn']
assert 'clang version 24.0.0git' in versions['clang']
record = {'success': True, 'preparerSha256': sha(Path(__file__)),
'tools': records, 'versions': versions,
'gclientArgsSha256': sha(gclient),
'scope': 'Pinned tools extracted; version commands only. No system installation or source download hooks.'}
(RESULTS / 'tool-materialization.json').write_text(json.dumps(record, indent=2) + '\n')
print(json.dumps(record, indent=2))
if __name__ == '__main__':
main()
tests/cmyk_lut/prepare_reference_cmm.py
+216
View File
@@ -0,0 +1,216 @@
#!/usr/bin/env python3
"""Prepare an isolated, pinned LittleCMS reference for the dedicated Ubuntu VM.
No package installation, system linker change, PDFium replacement, or downloads.
The PDFium allocator integration is reversed only in the dedicated build copy.
Other vendored changes remain and are explicitly identified in the build record.
"""
import argparse
import hashlib
import io
import json
import os
from pathlib import Path
import platform
import re
import shutil
import subprocess
import sys
import tarfile
ROOT=Path(__file__).resolve().parents[2]
REVISION='b76633e60c8387a77268fb3359277ca25b5fd75c'
def sha(path): return hashlib.sha256(path.read_bytes()).hexdigest()
def write_json(path,value): path.write_text(json.dumps(value,indent=2)+'\n')
def json_hash(value): return hashlib.sha256(json.dumps(value,sort_keys=True,separators=(',',':')).encode()).hexdigest()
def pack(args):
source=args.source.resolve(strict=True);output=args.output.resolve();output.mkdir(parents=True,exist_ok=False)
readme=(source/'README.pdfium').read_text()
assert 'Version: 2.19\n' in readme and 'Revision: '+REVISION+'\n' in readme
assert re.search(r'#define\s+LCMS_VERSION\s+2190\b',(source/'include/lcms2.h').read_text())
files={}
for p in sorted(source.rglob('*')):
if p.is_dir():continue
assert p.is_file() and not p.is_symlink() and p.resolve().is_relative_to(source)
relative=p.relative_to(source)
allowed=(len(relative.parts)==1 and (p.name in ('LICENSE','README.pdfium') or p.suffix=='.patch')) or (relative.parts[0] in ('src','include') and p.suffix in ('.c','.h'))
assert allowed and p.stat().st_size<=2*1024*1024
files['third_party/lcms/'+str(relative)]=p.read_bytes()
assert len(files)<100 and sum(map(len,files.values()))<16*1024*1024
reference=json.loads(args.reference_report.read_text());other=json.loads(args.comparison_report.read_text())
assert reference['directCmm']['encodedVersion']==2190 and other['directCmm']['encodedVersion']==2140
rows=[];comparisons=[]
for sr,so in zip(reference['scales'],other['scales'],strict=True):
assert sr['scale']==so['scale']
for a,b in zip(sr['probes'],so['probes'],strict=True):
fields=('page','profile','case','column','expectedIntentIndex','sampledComponents')
assert all(a[k]==b[k] for k in fields)
row={k:a[k] for k in fields};row.update(scale=sr['scale'],expectedRgb=a['expectedRgb'])
rows.append(row)
comparisons.append({'scale':sr['scale'],'page':a['page'],'case':a['case'],'column':a['column'],'profile':a['profile'],
'referenceExpectedRgb':a['expectedRgb'],'system214ExpectedRgb':b['expectedRgb'],
'pdfiumRgbEqual':a['actualRgb']['pdfium']==b['actualRgb']['pdfium'],
'pdfiumRgb':a['actualRgb']['pdfium'],'expectedRgbEqual':a['expectedRgb']==b['expectedRgb']})
assert len(rows)==600
profiles={}
corpus=ROOT/'tests/fixtures/pdf/rendering-intents'
manifest=json.loads((corpus/'manifest.json').read_text())
for key,item in manifest['profiles'].items():
path=corpus/item['file'];assert sha(path)==item['sha256']
files['profiles/'+path.name]=path.read_bytes();profiles[key]={'file':'profiles/'+path.name,'sha256':sha(path)}
expectations={'referenceReportSha256':sha(args.reference_report),'referenceDirectCmm':reference['directCmm'],
'comparisonReportSha256':sha(args.comparison_report),'profiles':profiles,'probes':rows,
'scope':'CMM numerical reproducibility only; the older shading-pattern-inherit state expectation is not adopted as PDF specification acceptance.'}
files['expected.json']=(json.dumps(expectations,indent=2)+'\n').encode()
files['intents.py']=Path(__file__).with_name('intents.py').read_bytes()
files['prepare_reference_cmm.py']=Path(__file__).read_bytes()
archive=output/'reference-lcms219.tar'
with tarfile.open(archive,'w',format=tarfile.PAX_FORMAT) as tar:
for name,data in sorted(files.items()):
info=tarfile.TarInfo(name);info.size=len(data);info.mode=0o644;info.mtime=0
tar.addfile(info,io.BytesIO(data))
inventory=[{'path':name,'size':len(data),'sha256':hashlib.sha256(data).hexdigest()} for name,data in sorted(files.items())]
record={'schemaVersion':1,'upstreamVersion':'2.19','upstreamRevision':REVISION,
'sourceScope':'Fixed PDFium-vendored source; allocator integration will be reversed in build copy only.',
'archive':archive.name,'archiveSha256':sha(archive),'files':inventory,
'sourceInventorySha256':json_hash([r for r in inventory if r['path'].startswith('third_party/')]),
'referenceReportSha256':sha(args.reference_report),'comparisonReportSha256':sha(args.comparison_report)}
write_json(output/'bundle.json',record)
write_json(output/'cross-os-before.json',{'scope':expectations['scope'],'rows':comparisons,
'total':len(comparisons),'allPdfiumRgbEqual':all(r['pdfiumRgbEqual'] for r in comparisons),
'differentExpectedRgb':sum(not r['expectedRgbEqual'] for r in comparisons)})
print(json.dumps({'archive':str(archive),'sha256':record['archiveSha256'],'files':len(inventory)}))
def build(args):
bundle=args.bundle.resolve(strict=True);record=json.loads(bundle.read_text());archive=bundle.parent/record['archive']
assert sha(archive)==record['archiveSha256']
output=args.output.resolve();output.mkdir(parents=True,exist_ok=False)
source=output/'source';source.mkdir()
rows={r['path']:r for r in record['files']};assert len(rows)==len(record['files'])<=100
with tarfile.open(archive,'r:') as tar:
entries=tar.getmembers();assert len(entries)==len(rows)
seen=set()
for entry in entries:
name=entry.name;relative=Path(name)
assert name in rows and name not in seen and entry.isfile() and not relative.is_absolute() and '..' not in relative.parts
assert entry.size==rows[name]['size']<=2*1024*1024
seen.add(name);data=tar.extractfile(entry).read()
assert hashlib.sha256(data).hexdigest()==rows[name]['sha256']
path=source/relative;path.parent.mkdir(parents=True,exist_ok=True);path.write_bytes(data)
lcms=source/'third_party/lcms';cmserr=lcms/'src/cmserr.c';before=sha(cmserr)
patch_source=lcms/'0000-cmserr-changes.patch'
# This file first includes a historical patch-file diff. Select only its
# final, actual cmserr.c delta, never execute arbitrary packaged hooks.
marker='diff --git a/third_party/lcms/src/cmserr.c b/third_party/lcms/src/cmserr.c\n'
patch_text=patch_source.read_text();assert patch_text.count('\n'+marker)==1
delta=patch_text[patch_text.index('\n'+marker)+1:]
assert delta.count('\ndiff --git ')==0
patch_file=output/'restore-standard-allocator.patch';patch_file.write_text(delta)
commands=[]
def run(label,command,**kwargs):
result=subprocess.run(command,capture_output=True,text=True,timeout=300,**kwargs)
log=output/(label+'.log');log.write_text(result.stdout+result.stderr)
commands.append({'command':command,'exitCode':result.returncode,'log':log.name,'logSha256':sha(log)})
assert result.returncode==0,label+' failed; see '+str(log)
return result.stdout
run('allocator-restore',['patch','--batch','--fuzz=0','--reverse','-p1','-i',str(patch_file)],cwd=source)
assert 'FXMEM_' not in cmserr.read_text() and '#include "core/' not in cmserr.read_text()
changes=[]
for row in record['files']:
now=sha(source/row['path'])
if now!=row['sha256']:changes.append({'path':row['path'],'before':row['sha256'],'after':now})
assert len(changes)==1 and changes[0]['path']=='third_party/lcms/src/cmserr.c' and changes[0]['before']==before
compiler=shutil.which('gcc');assert compiler
version=run('compiler-version',[compiler,'--version']);target=run('compiler-target',[compiler,'-dumpmachine']).strip()
library_dir=output/'lib';library_dir.mkdir();library=library_dir/'liblcms2.so.2.0.19'
inputs=[str(p) for p in sorted((lcms/'src').glob('*.c'))];assert len(inputs)==26
compile_command=[compiler,'-std=c99','-O2','-fPIC','-D_POSIX_C_SOURCE=200809L','-shared',
'-Wl,-soname,liblcms2.so.2','-Wl,-z,defs','-I'+str(source),'-I'+str(lcms/'include'),
*inputs,'-lm','-lpthread','-o',str(library)]
run('compile',compile_command)
(library_dir/'liblcms2.so.2').symlink_to(library.name);(library_dir/'liblcms2.so').symlink_to(library.name)
dependencies=run('ldd',['ldd',str(library)]);assert 'not found' not in dependencies
dynamic=run('readelf',['readelf','-d',str(library)])
environment=os.environ.copy();environment['LD_LIBRARY_PATH']=str(library_dir)
assert not environment.get('LD_PRELOAD') and not environment.get('LD_AUDIT')
run('runtime-version',[sys.executable,'-c',
'import ctypes; from PIL import ImageCms; c=ctypes.CDLL("liblcms2.so.2"); print(c.cmsGetEncodedCMMversion(), ImageCms.core.littlecms_version); assert c.cmsGetEncodedCMMversion()==2190; assert ImageCms.core.littlecms_version=="2.19"'],env=environment)
result={'schemaVersion':1,'success':True,'bundleSha256':sha(bundle),'archiveSha256':record['archiveSha256'],
'sourceInventorySha256':record['sourceInventorySha256'],'upstreamVersion':'2.19','upstreamRevision':REVISION,
'derivedSourceChanges':changes,'allocatorPatchSha256':sha(patch_file),
'remainingVendorPatches':'Retained; this is a pinned PDFium-vendored reference build with only its allocator integration reversed.',
'compiler':{'path':compiler,'sha256':sha(Path(compiler).resolve()),'version':version,'target':target},
'library':{'path':str(library),'size':library.stat().st_size,'sha256':sha(library),'soname':'liblcms2.so.2'},
'dependencies':dependencies,'dynamicSection':dynamic,'commands':commands,
'osRelease':platform.freedesktop_os_release(),'pythonVersion':platform.python_version(),
'scope':'Isolated reference library only; no system installation or loader configuration changes.'}
write_json(output/'build.json',result)
print(json.dumps({'success':True,'library':str(library),'sha256':sha(library)}))
def proof(args):
# Must start a fresh Python interpreter with the desired LD_LIBRARY_PATH;
# changing it after import cannot select an already loaded native library.
output=args.output.resolve();output.mkdir(parents=True,exist_ok=False)
data=json.loads(args.expected.read_text());source=args.expected.resolve().parent
sys.path.insert(0,str(source))
from intents import Cmm
from PIL import Image,ImageCms
profiles={key:(source/row['file']).read_bytes() for key,row in data['profiles'].items()}
assert all(sha(source/row['file'])==row['sha256'] for row in data['profiles'].values())
cmm=Cmm(profiles)
try:
assert cmm.version==args.version
paths={line.split()[-1] for line in Path('/proc/self/maps').read_text().splitlines() if 'liblcms2.so' in line and '/' in line}
assert len(paths)==1
actual=Path(next(iter(paths))).resolve()
if args.library:assert actual==args.library.resolve(strict=True)
assert ImageCms.core.littlecms_version==f'{args.version//1000}.{(args.version%1000)//10}'
module=Path(ImageCms.core.__file__)
ldd=subprocess.run(['ldd',str(module)],capture_output=True,text=True,check=True,timeout=20).stdout
match=re.search(r'liblcms2\.so\.2 => (\S+)',ldd);assert match and Path(match[1]).resolve()==actual
pillow={}
for key,profile in profiles.items():
for i in range(4):
pillow[key,i]=ImageCms.buildTransformFromOpenProfiles(ImageCms.ImageCmsProfile(io.BytesIO(profile)),ImageCms.createProfile('sRGB'),'CMYK','RGB',renderingIntent=i)
rows=[];quantized=[]
for p in data['probes']:
color=cmm.color(p['profile'],p['expectedIntentIndex'],p['sampledComponents'])
rows.append({**p,'actualRgb':color,'equal':color==p['expectedRgb'],
'maxChannelError':max(abs(a-b) for a,b in zip(color,p['expectedRgb']))})
for key in profiles:
for i in range(4):
pixel=(51,128,204,77)
direct=cmm.color(key,i,[v/255 for v in pixel])
pil=list(ImageCms.applyTransform(Image.new('CMYK',(1,1),pixel),pillow[key,i]).getpixel((0,0)))
quantized.append({'profile':key,'intent':i,'directRgb':direct,'pillowRgb':pil,
'maxChannelError':max(abs(a-b) for a,b in zip(direct,pil))})
report={'schemaVersion':1,'success':all(p['equal'] for p in rows) and all(p['maxChannelError']<=1 for p in quantized),
'referenceReportSha256':data['referenceReportSha256'],'expectedFileSha256':sha(args.expected),
'expectedScope':data['scope'],'pythonVersion':platform.python_version(),'encodedVersion':cmm.version,
'pillowVersion':__import__('PIL').__version__,'pillowLcmsVersion':ImageCms.core.littlecms_version,
'loadedLibrary':{'path':str(actual),'sha256':sha(actual)},'mappedLibraries':sorted(paths),
'pillowExtension':{'path':str(module),'sha256':sha(module),'ldd':ldd},
'total':len(rows),'identical':sum(p['equal'] for p in rows),'different':sum(not p['equal'] for p in rows),
'pillowChecks':quantized,'probes':rows,
'limitations':'Numerical matching at the declared probes, not independent-CMM or PDF semantic correctness.'}
write_json(output/'proof.json',report)
print(json.dumps({k:report[k] for k in ('success','encodedVersion','pillowLcmsVersion','total','identical','different')}))
if args.version==2190:assert report['success'],'Pinned CMM differs from host reference'
finally:cmm.close()
def main():
parser=argparse.ArgumentParser(description=__doc__);commands=parser.add_subparsers(dest='command',required=True)
p=commands.add_parser('pack');p.add_argument('--source',type=Path,required=True);p.add_argument('--reference-report',type=Path,required=True);p.add_argument('--comparison-report',type=Path,required=True);p.add_argument('--output',type=Path,required=True);p.set_defaults(function=pack)
p=commands.add_parser('build');p.add_argument('--bundle',type=Path,required=True);p.add_argument('--output',type=Path,required=True);p.set_defaults(function=build)
p=commands.add_parser('proof');p.add_argument('--expected',type=Path,required=True);p.add_argument('--output',type=Path,required=True);p.add_argument('--version',type=int,choices=(2140,2190),required=True);p.add_argument('--library',type=Path);p.set_defaults(function=proof)
args=parser.parse_args();args.function(args)
if __name__=='__main__':main()
tests/cmyk_lut/record.py
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env python3
"""Publish failing color evidence without converting diagnosis into acceptance."""
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path
ROOT=Path(__file__).resolve().parents[2]
RESULTS=ROOT/'tests/results/cmyk-lut'
def sha(path):
with path.open('rb') as stream: return hashlib.file_digest(stream,'sha256').hexdigest()
def main():
prior=json.loads((RESULTS/'prior-validation-record.json').read_text())
assert sha(RESULTS/'prior-validation-record.json')=='c80457d32c246e40743f59f5e602ff0f71d59d496b33a72618a97475659c9013'
for name,digest in prior['evidenceSha256'].items(): assert sha(ROOT/name)==digest,name
sources={str(p.relative_to(ROOT)):sha(p) for directory in ('src','qml','cmake','resources')
for p in (ROOT/directory).rglob('*') if p.is_file() and '__pycache__' not in p.parts}
sources.update({name:sha(ROOT/name) for name in ('CMakeLists.txt','resources.qrc')})
assert sources==prior['sourceSha256']
for name,digest in prior['builds']['arch']['binaries'].items(): assert sha(ROOT/'build'/name)==digest
package=prior['ubuntuPackage']
assert sha(ROOT/'build-ubuntu-vm/packages'/package['archive'])==package['archiveSha256']
diagnosis=json.loads((RESULTS/'diagnosis.json').read_text())
assert diagnosis['diagnosisVerified'] and not diagnosis['renderingAcceptance']
assert not diagnosis['relativeIntentPreserved'] and diagnosis['toleranceUnchanged']==4
assert diagnosis['diagnoserSha256']==sha(ROOT/'tests/cmyk_lut/diagnose.py')
reports={}
for os_name in ('arch','ubuntu'):
report=json.loads((RESULTS/os_name/'report.json').read_text())
assert not report['success'] and report['originalUnchanged'] and report['binariesUnchanged']
assert len(report['scales'])==3
for name,digest in report['evidenceSha256'].items(): assert sha(RESULTS/os_name/name)==digest
assert report['binaries']['docview-pdf-worker']==prior['builds'][os_name]['binaries']['docview-pdf-worker']
assert report['runnerSha256']==sha(ROOT/'tests/cmyk_lut/run.py')
entry=next(r for r in diagnosis['results'] if r['os']==os_name)
assert entry['comparisonReportSha256']==sha(RESULTS/os_name/'report.json')
assert entry['failedPdfiumAnalyticProbes']==30 and entry['failedPopplerAnalyticProbes']==0
reports[os_name]=report
assert sha(ROOT/'build/pdf-worker-evidence')==reports['arch']['binaries']['pdf-worker-evidence']
assert sha(ROOT/'.deps/pdfium/lib/libpdfium.so')=='08f6ea53a64f6fbb9f5ba8d9c02e0051987c6a9175f3fb5ba25f219174731aaa'
reproduced=json.loads((RESULTS/'reproduction.json').read_text())
for name,digest in reproduced['filesByteIdentical'].items():
assert sha(ROOT/'tests/fixtures/pdf/cmyk-lut'/name)==digest==sha(RESULTS/'reproduction'/name)
shutdown=json.loads((RESULTS/'shutdown.json').read_text())
assert shutdown['qemuExitCode']==0 and shutdown['guestPowerDownObserved']
assert sha(RESULTS/'shutdown-tail.txt')==shutdown['tailSha256']
gallery=['''<!doctype html><html lang="ja"><meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1"><title>ICC CMYK 描画比較</title>
<style>body{margin:0;background:#f5f6f8;color:#202734;font:16px/1.7 system-ui,sans-serif}main{max-width:1500px;margin:auto;padding:36px 24px}h1{font-size:28px;line-height:1.4}h2{margin-top:40px}h3{margin:20px 0 8px}.status{background:#fff0ee;border-left:4px solid #bb3b30;padding:16px 20px}a{color:#1459a0}.row{display:grid;grid-template-columns:repeat(3,minmax(260px,1fr));gap:16px;overflow:auto}figure{margin:0;background:white;padding:12px}figcaption{font-weight:600;margin-bottom:8px}img{display:block;width:100%;height:auto}.hint{color:#596476;font-size:14px}</style>
<main><h1>ICC CMYK の追加描画比較</h1>
<p class="status"><strong>色の一致は未達です。</strong> Arch・Ubuntu各48点中30点が許容差を超えました。原因の照合は完了していますが、描画の受入や人による承認は完了していません。</p>
<p>相対的な測色を指定した資料に対して、固定PDFiumは知覚的なICC変換を使用します。上2段の右端の暗色に明瞭な差があります。下段はプロファイルを適用しないDeviceCMYKの対照です。</p>
<p><a href="README.md">試験条件・原因・未完了事項</a> / <a href="../../fixtures/pdf/cmyk-lut/icc-cmyk-lut.pdf">生成PDF</a> / <a href="diagnosis.json">数値による原因照合</a></p>
<p class="hint">各画像を開くと元の解像度で確認できます。差分画像は絶対RGB差で、色差の許容性や印刷色を判定する画像ではありません。</p>''']
for os_name,label in [('arch','Arch Linux'),('ubuntu','Ubuntu 24.04')]:
gallery.append('<h2>'+label+'</h2>')
for scale in reports[os_name]['scales']:
folder=os_name+'/scale-'+str(scale['scale']).replace('.','_')
gallery.append(f'<h3>{scale["scale"]:g}倍 / {scale["dpi"]:g} dpi</h3><div class="row">')
for title,name in [('PDFium / 本番隔離ワーカー','pdfium/page-1.png'),('Poppler / 独立した比較','poppler.png'),('絶対RGB差','difference.png')]:
path=folder+'/'+name
assert (RESULTS/path).is_file()
gallery.append(f'<figure><figcaption>{title}</figcaption><a href="{path}"><img src="{path}" alt="{title}の描画比較" loading="lazy"></a></figure>')
gallery.append('</div>')
gallery.append('</main></html>')
(RESULTS/'index.html').write_text('\n'.join(gallery))
tools=['tests/cmyk_lut/run.py','tests/cmyk_lut/diagnose.py','tests/cmyk_lut/record.py','tests/fixtures/pdf/generate_cmyk_lut.py','tests/render_pdf_worker.cpp']
new_requirement='Resolve or explicitly review the fixed PDFium ICC rendering-intent discrepancy demonstrated by the CMYK CLUT fixture; runtime detection/notice is not yet implemented'
record={**prior,'schemaVersion':4,'recordedAtUtc':datetime.now(timezone.utc).isoformat(),
'scope':'Unchanged validation3 product plus a new, failing CMYK ICC rendering comparison on Arch and Ubuntu. '
'Prior passing functional/package tests remain scoped to their original runs; color acceptance is not achieved.',
'goalComplete':False,'colorRenderingFinding':diagnosis,
'colorRenderingReports':{name:{'path':'tests/results/cmyk-lut/'+name+'/report.json','sha256':sha(RESULTS/name/'report.json')} for name in reports},
'previousValidation':{'record':'tests/results/cmyk-lut/prior-validation-record.json','sha256':sha(RESULTS/'prior-validation-record.json'),
'scope':'Validation3 package, all prior test executions and their exact product identities. No product bytes changed in the ICC investigation.'},
'remainingAcceptance':prior['remainingAcceptance']+[new_requirement],
'validationToolsSha256':{**prior['validationToolsSha256'],**{name:sha(ROOT/name) for name in tools}},
'evidenceSha256':{**prior['evidenceSha256'],**{str(p.relative_to(ROOT)):sha(p) for folder in (RESULTS,ROOT/'tests/fixtures/pdf/cmyk-lut')
for p in sorted(folder.rglob('*')) if p.is_file() and p.suffix!='.pyc' and p!=RESULTS/'record.json'}}}
encoded=json.dumps(record,ensure_ascii=False,indent=2)+'\n'
(RESULTS/'record.json').write_text(encoded); (ROOT/'docs/validation-record.json').write_text(encoded)
print(json.dumps({'colorRenderingAcceptance':False,'failedAnalyticProbes':60,'productUnchanged':True,'goalComplete':False}))
if __name__=='__main__': main()
tests/cmyk_lut/record_candidate.py
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env python3
"""Add isolated-candidate evidence while preserving production failure history."""
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path
import xml.etree.ElementTree as ET
ROOT = Path(__file__).resolve().parents[2]
RESULTS = ROOT / 'tests/results/pdfium-intent-build'
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def read(relative):
return json.loads((RESULTS / relative).read_text())
def main():
previous = RESULTS / 'prior-validation-record.json'
assert sha(previous) == 'e171862993beb61a9786ffee1f6a4a869ba30e3c3bd03223918a20d8e41f297b'
prior = json.loads(previous.read_text())
for name, digest in prior['evidenceSha256'].items():
assert sha(ROOT / name) == digest, name
for name, digest in prior['sourceSha256'].items():
assert sha(ROOT / name) == digest, name
for name, digest in prior['validationToolsSha256'].items():
assert sha(ROOT / name) == digest, name
for name, digest in prior['builds']['arch']['binaries'].items():
assert sha(ROOT / 'build' / name) == digest, name
package = prior['ubuntuPackage']
assert sha(ROOT / 'build-ubuntu-vm/packages' / package['archive']) == package['archiveSha256']
assert sha(ROOT / '.deps/pdfium/lib/libpdfium.so') == '08f6ea53a64f6fbb9f5ba8d9c02e0051987c6a9175f3fb5ba25f219174731aaa'
candidate = read('candidate-final/report.json')
assert candidate['upstreamTestsPass'] and candidate['candidatePdfiumProbesPass']
assert candidate['allBinaryInputsUnchanged'] and not candidate['productionDependencyReplaced']
assert not candidate['success'] and not candidate['intentPopplerSuccess']
assert [(r['tests'], r['failures']) for r in candidate['upstreamTests']] == [(1104, 0), (872, 0)]
for name, digest in candidate['evidenceSha256'].items():
assert sha(RESULTS / 'candidate-final' / name) == digest, name
for name, digest in candidate['inputs'].items():
assert sha(Path(name)) == digest, name
intents = read('candidate-final/intents/report.json')
probes = [p for scale in intents['scales'] for p in scale['probes']]
assert len(probes) == 600 and all(p['matches']['pdfium'] for p in probes)
comparison_failures = [p for p in probes if not p['matches']['poppler']]
assert len(comparison_failures) == 18
assert {p['case'] for p in comparison_failures} == {'shading-pattern-RI-override'}
baseline = read('intents-baseline/report.json')
assert sum(not p['matches']['pdfium'] for s in baseline['scales'] for p in s['probes']) == 150
ctest = read('docview-ctest/report.json')
assert ctest['exitCode'] == 0 and ctest['libraryUnchanged']
assert ctest['librarySha256'] == intents['pdfiumLibrary']['sha256']
suite = ET.parse(RESULTS / 'docview-ctest/tests.xml').getroot()
assert suite.tag == 'testsuite' and suite.attrib['tests'] == '22' and suite.attrib['failures'] == '0'
patch = read('candidate-patch.json')
assert patch['success'] and patch['appliesWithoutFuzz'] and patch['appliedSourcesByteIdentical']
assert sha(ROOT / patch['patch']) == patch['patchSha256']
assert sha(ROOT / patch['patchSnapshot']) == patch['patchSha256']
for row in patch['files']:
assert sha(ROOT / 'build-pdfium-intent/source' / row['path']) == row['afterSha256']
reproduced = read('fixture-reproduction.json')
assert reproduced['success'] and len(reproduced['filesByteIdentical']) == 4
for name, digest in reproduced['filesByteIdentical'].items():
assert sha(RESULTS / 'reproduced-fixture' / name) == digest
assert sha(ROOT / 'tests/fixtures/pdf/rendering-intents' / name) == digest
tools = ['tests/cmyk_lut/' + name + '.py' for name in
('prepare_pdfium_source', 'fetch_pdfium_build_tools', 'prepare_pdfium_tools',
'export_pdfium_patch', 'validate_pdfium_candidate', 'record_candidate', 'intents')]
tools.append('tests/fixtures/pdf/generate_rendering_intents.py')
candidate_summary = {
'productionDependencyReplaced': False, 'candidatePdfiumProbesPass': True,
'humanRenderingAcceptance': False, 'crossRendererComparisonAccepted': False,
'sourceRevision': patch['baseRevision'], 'patch': patch['patch'],
'patchSha256': patch['patchSha256'], 'changedCoreFiles': len(patch['files']),
'patchSnapshot': patch['patchSnapshot'],
'library': intents['pdfiumLibrary']['path'], 'librarySha256': intents['pdfiumLibrary']['sha256'],
'upstreamTests': candidate['upstreamTests'], 'docviewCtestGroups': 22,
'cmykAnalyticProbesPassed': 48, 'intentProbesPassed': 600,
'popplerFailedProbes': 18, 'popplerFailureCase': 'shading-pattern-RI-override',
'report': 'tests/results/pdfium-intent-build/candidate-final/report.json',
'reportSha256': sha(RESULTS / 'candidate-final/report.json'),
'scope': 'Arch isolated candidate with existing application/worker binaries; no adoption into the Ubuntu package, Windows execution, calibrated-output or human golden acceptance.'}
evidence = {str(path.relative_to(ROOT)): sha(path)
for folder in (RESULTS, ROOT / 'tests/fixtures/pdf/rendering-intents')
for path in sorted(folder.rglob('*')) if path.is_file()
and '__pycache__' not in path.parts and path != RESULTS / 'record.json'}
record = {**prior, 'schemaVersion': 5, 'recordedAtUtc': datetime.now(timezone.utc).isoformat(),
'scope': 'Unchanged validation3 product plus an isolated source-built PDFium rendering-intent candidate. Candidate numeric/upstream/application tests pass; comparator differences and final adoption/acceptance remain.',
'goalComplete': False, 'pdfiumRenderingIntentCandidate': candidate_summary,
'previousValidation': {'record': str(previous.relative_to(ROOT)), 'sha256': sha(previous),
'scope': 'Unchanged product and original failing ICC evidence; retained without rewriting its verdict.'},
'sourceSha256': {**prior['sourceSha256'], patch['patch']: patch['patchSha256']},
'validationToolsSha256': {**prior['validationToolsSha256'], **{name: sha(ROOT / name) for name in tools}},
'evidenceSha256': {**prior['evidenceSha256'], **evidence},
'remainingAcceptance': [*prior['remainingAcceptance'],
'Review and adopt the source-built rendering-intent patch; validate Ubuntu packaging and remaining rendering boundaries (including dedicated soft-mask/tiling probes). Independent Poppler shading-pattern intent differences remain recorded.']}
encoded = json.dumps(record, ensure_ascii=False, indent=2) + '\n'
(RESULTS / 'record.json').write_text(encoded)
(ROOT / 'docs/validation-record.json').write_text(encoded)
print(json.dumps({'goalComplete': False, 'candidateProbesPassed': 648,
'upstreamTestsPassed': 1976, 'docviewGroupsPassed': 22,
'evidenceFiles': len(record['evidenceSha256'])}))
if __name__ == '__main__':
main()
tests/cmyk_lut/record_context_candidate.py
+105
View File
@@ -0,0 +1,105 @@
#!/usr/bin/env python3
"""Freeze a build/test anchor for the corrected PDFium candidate, before staging."""
import hashlib
import json
from pathlib import Path
import xml.etree.ElementTree as ET
ROOT = Path(__file__).resolve().parents[2]
RESULTS = ROOT / 'tests/results/pdfium-intent-context'
PARENT = ROOT / 'tests/results/pdfium-intent-build/record.json'
PARENT_SHA = '47b4dc2efbf8e671f904610b3047400dbd3ceb0fa10d6bd53fb04f3727943e7e'
LIBRARY_SHA = 'cb049fe434f4c911b5eb047c0aca572d4dd9f5e405dde329b76c05a6aa771403'
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def rel(path):
return str(path.relative_to(ROOT))
def check_nested(path):
data = json.loads(path.read_text())
for name, digest in data.get('evidenceSha256', {}).items():
assert sha(path.parent / name) == digest, name
return data
def main():
destination = RESULTS / 'record.json'
assert not destination.exists(), 'The build anchor is immutable; create another version for changed inputs.'
assert sha(PARENT) == PARENT_SHA
parent = json.loads(PARENT.read_text())
source = ROOT / 'build-pdfium-intent-context/source'
library = source / 'out/patched/libpdfium.so'
assert sha(library) == LIBRARY_SHA
patch = json.loads((RESULTS / 'candidate-patch.json').read_text())
assert patch['success'] and patch['appliesWithoutFuzz'] and patch['appliedSourcesByteIdentical']
assert patch['parentRecordSha256'] == PARENT_SHA
assert sha(ROOT / patch['patch']) == sha(ROOT / patch['patchSnapshot']) == patch['patchSha256']
for row in patch['files']:
assert sha(source / row['path']) == row['afterSha256'], row['path']
build = json.loads((RESULTS / 'build-2.json').read_text())
assert build['exitCode'] == 0
upstream = json.loads((RESULTS / 'upstream-tests-execution.json').read_text())
assert upstream['success'] and upstream['librarySha256'] == LIBRARY_SHA
tests = 0
for row in upstream['commands']:
assert row['exitCode'] == 0
data = row['results']
assert data['failures'] == data['errors'] == data['disabled'] == 0
assert sha(Path(row['command'][0])) == row['executableSha256']
tests += data['tests']
assert tests == 1979
arch = check_nested(RESULTS / 'arch/report.json')
assert arch['candidateTestsPass'] and arch['allBinaryInputsUnchanged']
assert arch['inputs'][str(library)] == LIBRARY_SHA
assert arch['intents']['probeCount'] == 600 and arch['transparency']['probeCount'] == 324
for name in ('cmyk', 'intents', 'transparency'):
check_nested(RESULTS / 'arch' / name / 'report.json')
junit = ET.parse(RESULTS / 'arch/tests.xml').getroot()
cases = list(junit.iter('testcase'))
assert len(cases) == 22 and not list(junit.iter('failure')) and not list(junit.iter('error'))
evidence = {rel(path): sha(path) for path in sorted(RESULTS.rglob('*'))
if path.is_file() and path != destination}
for name in ('source-materialization', 'tool-materialization', 'provider-patches'):
path = ROOT / ('tests/results/pdfium-intent-build/' + name + '.json')
assert sha(path) == parent['evidenceSha256'][rel(path)]
evidence[rel(path)] = sha(path)
evidence[rel(PARENT)] = PARENT_SHA
sources = {name: value for name, value in parent['sourceSha256'].items()
if name.startswith('resources/licenses/pdfium-supplemental/')}
for name, value in sources.items():
assert sha(ROOT / name) == value
tools = [Path(__file__), ROOT / 'tests/cmyk_lut/export_pdfium_context_patch.py',
ROOT / 'tests/cmyk_lut/intents.py', ROOT / 'tests/cmyk_lut/transparency.py',
ROOT / 'tests/cmyk_lut/correct_intent_oracle.py', ROOT / 'tests/cmyk_lut/validate_context_candidate.py',
ROOT / 'tests/fixtures/pdf/generate_intent_transparency.py']
sources.update({rel(path): sha(path) for path in tools})
for folder in ('rendering-intents-context', 'intent-transparency'):
sources.update({rel(path): sha(path) for path in (ROOT / 'tests/fixtures/pdf' / folder).iterdir() if path.is_file()})
record = {'schemaVersion': 6, 'goalComplete': False,
'parent': {'path': rel(PARENT), 'sha256': PARENT_SHA},
'scope': 'Immutable Linux candidate build and Arch test anchor before staging. Ubuntu, package adoption and release acceptance are recorded separately.',
'pdfiumRenderingIntentCandidate': {
'sourceRevision': patch['baseRevision'], 'source': rel(source),
'library': str(library), 'librarySha256': LIBRARY_SHA,
'patch': patch['patch'], 'patchSha256': patch['patchSha256'],
'patchSnapshot': patch['patchSnapshot'], 'report': rel(RESULTS / 'arch/report.json'),
'reportSha256': sha(RESULTS / 'arch/report.json'),
'productionDependencyReplaced': False, 'comparisonAccepted': False,
'candidateProbesPass': True, 'candidateProbeCount': 972,
'upstreamTestCount': tests, 'docviewCTestGroups': 22,
'oracleRevision': 2,
'supersedes': 'The old shading-pattern-inherit expectation was incorrect. The historical v1 pass remains a numerical result against that old expectation, not pattern conformance evidence.'},
'evidenceSha256': evidence, 'sourceSha256': sources}
destination.write_text(json.dumps(record, indent=2) + '\n')
print(json.dumps({'recordSha256': sha(destination), 'evidenceFiles': len(evidence),
'sourceFiles': len(sources), 'candidateProbeCount': 972}))
if __name__ == '__main__':
main()
tests/cmyk_lut/record_context_integration.py
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env python3
"""Record the tested v2 deliverables without rewriting their fixed build anchor."""
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
OUT = ROOT / 'tests/results/pdfium-intent-context'
PARENT = ROOT / 'tests/results/pdfium-intent-build/record.json'
PARENT_SHA = '47b4dc2efbf8e671f904610b3047400dbd3ceb0fa10d6bd53fb04f3727943e7e'
ANCHOR_SHA = '80af72b06d5c9a6a3ab8b9311373e6a23e8df28668ce575e1124350e69aa160b'
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def relative(path):
return str(path.relative_to(ROOT))
def reference(path):
return {'path': relative(path), 'sha256': sha(path)}
def main():
current = ROOT / 'docs/validation-record.json'
destination = OUT / 'integration-record.json'
assert not destination.exists(), 'Integration records are immutable.'
assert sha(current) == sha(PARENT) == PARENT_SHA
assert sha(OUT / 'record.json') == ANCHOR_SHA
parent = json.loads(PARENT.read_text())
anchor = json.loads((OUT / 'record.json').read_text())
required = {
'archCandidate': OUT / 'arch/report.json',
'ubuntuCandidate': OUT / 'ubuntu/report.json',
'archNewBuild': OUT / 'arch-integration/report.json',
'ubuntuNewBuildAndPackage': OUT / 'ubuntu-package/report.json',
'sdkFreeUbuntu': OUT / 'ubuntu-package/clean-vm/report.json',
'performance': OUT / 'performance/report.json',
'renderReview': OUT / 'render-review/report.json',
}
reports = {key: json.loads(path.read_text()) for key, path in required.items()}
for key, data in reports.items():
assert data.get('candidateTestsPass', data.get('success')) is True, key
assert reports['renderReview']['humanApproved'] is False
arch = reports['archNewBuild']
for name, digest in arch['binaries'].items():
assert sha(ROOT / 'build-context' / name) == digest, name
library = ROOT / '.deps/pdfium-context/lib/libpdfium.so'
assert sha(library) == anchor['pdfiumRenderingIntentCandidate']['librarySha256']
assert sha(ROOT / '.deps/pdfium/lib/libpdfium.so') == '08f6ea53a64f6fbb9f5ba8d9c02e0051987c6a9175f3fb5ba25f219174731aaa'
source_paths = {ROOT / name for name in parent['sourceSha256']}
source_paths.update(ROOT / name for name in parent['validationToolsSha256'])
for folder in ('tools', 'cmake', 'tests/cmyk_lut'):
source_paths.update(path for path in (ROOT / folder).rglob('*')
if path.is_file() and '__pycache__' not in path.parts)
required_sources = tuple(ROOT / name for name in (
'CMakeLists.txt', 'tests/test_stage_pdfium_candidate.py',
'tests/test_pdfium_candidate_integration.py', 'tests/ubuntu_vm/context_package.py',
'tests/ubuntu_vm/package_smoke.py', 'tests/ubuntu_vm/clean_package.py',
'tests/compare_pdf_extended.py', 'tests/generate_render_review.py'))
for path in required_sources:
assert path.is_file(), relative(path)
source_paths.update(required_sources)
sources = {relative(path): sha(path) for path in sorted(source_paths) if path.is_file()}
changed = [{'path': name, 'previousSha256': digest, 'currentSha256': sources[name]}
for name, digest in {**parent['sourceSha256'], **parent['validationToolsSha256']}.items()
if name in sources and sources[name] != digest]
evidence = {}
for folder in (OUT, ROOT / 'tests/results/pdfium-intent-transparency',
ROOT / 'tests/results/pdfium-intent-ubuntu'):
for path in sorted(folder.rglob('*')):
if path.is_file() and path != destination and '__pycache__' not in path.parts:
evidence[relative(path)] = sha(path)
record = {
'schemaVersion': 7, 'recordedAtUtc': datetime.now(timezone.utc).isoformat(),
'goalComplete': False,
'scope': 'Linux v2 candidate integrated into dedicated Arch and Ubuntu builds and a local validation deb. Earlier evidence is retained by immutable parent references. Target Windows/physical/human/release gates remain incomplete.',
'previousValidation': reference(PARENT),
'candidateBuildAnchor': reference(OUT / 'record.json'),
'pdfiumRenderingIntentCandidate': anchor['pdfiumRenderingIntentCandidate'],
'candidateIntegration': {
'selectedLibrary': reference(library),
'originalProviderPreserved': True,
'buildInfo': reference(ROOT / '.deps/pdfium-context/BUILDINFO.json'),
'reviewedLock': reference(ROOT / 'cmake/pdfium-candidate-v2.lock.json'),
'archExecutable': 'build-context/docview',
'archInstalledExecutable': 'build-context/install/bin/docview',
'ubuntuPackageRecord': reference(required['ubuntuNewBuildAndPackage']),
},
'executables': arch['binaries'],
'validationReports': {key: reference(path) for key, path in required.items()},
'oracleCorrection': reference(ROOT / 'tests/fixtures/pdf/rendering-intents-context/manifest.json'),
'historicalOracleLimitation': 'The v1 shading-pattern-inherit expectations selected paint-time RI incorrectly. Its numerical pass is preserved as historical evidence, not pattern conformance. Revision 2 keeps the original PDF/profile bytes and corrects eight rows.',
'referenceCmm': reference(ROOT / 'tests/results/pdfium-intent-ubuntu/reference-cmm/verification.json'),
'remainingAcceptance': [
'Windows 11 native build, sandbox/pipe/IME execution and distribution tests',
'Target-OS physical GPU/display/input, reference hardware and long-duration acceptance',
'Human approval of G-RENDER reference images and documented comparison differences',
'Final public distribution/license decision and complete corresponding-source/notice assessment',
],
'sourceSha256': sources, 'changedSincePreviousRecord': changed,
'documentationSha256': {name: sha(ROOT / name) for name in (
'README.md', 'docs/PDFIUM-CANDIDATE.md', 'docs/VALIDATION.md',
'docs/ACCEPTANCE-AUDIT.md', 'docs/UBUNTU-PACKAGE.md',
'tests/PDF-VALIDATION.md')},
'evidenceSha256': evidence,
}
encoded = json.dumps(record, indent=2) + '\n'
destination.write_text(encoded)
current.write_text(encoded)
print(json.dumps({'schemaVersion': 7, 'sha256': sha(current),
'evidenceFiles': len(evidence), 'sourceFiles': len(sources),
'goalComplete': False}))
if __name__ == '__main__':
main()
tests/cmyk_lut/run.py
+150
View File
@@ -0,0 +1,150 @@
#!/usr/bin/env python3
"""Validate a known CMYK CLUT through the production isolated worker and Poppler."""
import argparse
import hashlib
from io import BytesIO
import json
import math
import os
from pathlib import Path
import platform
import struct
import subprocess
from PIL import Image, ImageChops, ImageCms, ImageDraw, ImageStat
from pypdf import PdfReader
ROOT = Path(__file__).resolve().parents[2]
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--build-dir', required=True, type=Path)
parser.add_argument('--output', required=True, type=Path)
args = parser.parse_args()
build = args.build_dir.resolve(); output = args.output.resolve()
output.mkdir(parents=True, exist_ok=False)
corpus = ROOT / 'tests/fixtures/pdf/cmyk-lut'
spec = json.loads((corpus / 'manifest.json').read_text())
source = corpus / spec['file']; icc_file = corpus / spec['profileFile']
assert sha(source) == spec['sha256'] and sha(icc_file) == spec['profileSha256']
assert sha(ROOT / 'tests/fixtures/pdf/generate_cmyk_lut.py') == spec['generatorSha256']
binaries = {name: sha(build / name) for name in ('pdf-worker-evidence', 'docview-pdf-worker')}
profile = icc_file.read_bytes()
assert len(profile) == spec['profileBytes'] == struct.unpack_from('>I', profile)[0]
assert profile[8:24] == bytes.fromhex('02100000') + b'scnrCMYKLab ' and profile[36:40] == b'acsp'
tags = {}
for i in range(struct.unpack_from('>I', profile, 128)[0]):
sig, offset, size = struct.unpack_from('>4sII', profile, 132+12*i)
assert offset % 4 == 0 and offset+size <= len(profile) and sig not in tags
tags[sig] = profile[offset:offset+size]
assert set(tags) == {b'desc', b'cprt', b'wtpt', b'A2B0'}
lut = tags[b'A2B0']
assert len(lut) == 176 and lut[:12] == b'mft2' + bytes(4) + bytes((4,3,2,0))
assert struct.unpack_from('>HH', lut, 48) == (2,2)
# Decode and check all 16 four-dimensional CLUT vertices independently.
for index in range(16):
c, m, y, k = [(index >> shift) & 1 for shift in (3,2,1,0)]
encoded = struct.unpack_from('>3H', lut, 68+index*6)
lab = [encoded[0]*100/65280, encoded[1]/256-128, encoded[2]/256-128]
expected = [100-12*c-15*m-10*y-35*k, 18*m-15*c, 20*y-12*c]
assert max(abs(a-b) for a,b in zip(lab,expected)) < .002
pdf = PdfReader(source)
assert len(pdf.pages) == 1 and list(pdf.pages[0].mediabox)[2:] == spec['pageSizePt']
resources = pdf.pages[0]['/Resources']
vector_space = resources['/ColorSpace']['/TestCMYK']
image_object = resources['/XObject']['/Sample'].get_object()
for space in (vector_space, image_object['/ColorSpace']):
assert space[0] == '/ICCBased' and space[1].get_object()['/N'] == 4
assert space[1].get_object()['/Alternate'] == '/DeviceCMYK'
assert space[1].get_object().get_data() == profile
assert image_object['/Width'] == 8 and image_object['/Height'] == 1 and image_object['/BitsPerComponent'] == 8
pixel_values = bytes(round(c*255) for p in spec['probes'] if p['kind'] == 'icc-image' for c in p['components'])
assert image_object.get_data() == pixel_values
icc = ImageCms.ImageCmsProfile(BytesIO(profile))
transform = ImageCms.buildTransformFromOpenProfiles(icc, ImageCms.createProfile('sRGB'), 'CMYK', 'RGB', renderingIntent=1)
cmm_probes = []
for probe in spec['probes']:
if probe['kind'] != 'icc-image': continue
pixel = Image.new('CMYK',(1,1),tuple(round(c*255) for c in probe['components']))
actual = ImageCms.applyTransform(pixel, transform).getpixel((0,0))
error = max(abs(a-b) for a,b in zip(actual,probe['expectedRgb']))
cmm_probes.append({'sample':probe['sample'], 'expectedRgb':probe['expectedRgb'], 'actualRgb':actual,
'maxChannelError':error, 'success':error <= spec['tolerance8Bit']})
report = {'success':False, 'osRelease':platform.freedesktop_os_release(), 'fixture':spec,
'python':platform.python_version(), 'pillowLcmsVersion':ImageCms.core.littlecms_version,
'runnerSha256':sha(Path(__file__)), 'binaries':binaries,
'rendererVersion':subprocess.run(['/usr/bin/pdftoppm','-v'],capture_output=True,text=True,check=True).stderr.splitlines()[0],
'embeddedProfileAndImageBytesVerified':True, 'clutVerticesVerified':16, 'cmmProbes':cmm_probes,
'scope':'Synthetic ICC v2 CMYK input -> Lab four-dimensional CLUT, vectors and images. '
'Production isolated worker at three scales and independent Poppler. '
'The CMM probe uses Pillow/LittleCMS and is not a third independent PDF renderer. '
'No calibrated press/display, arbitrary ICC conformance or human-approved golden.',
'commands':[], 'scales':[]}
def run(name, command):
with (output / (name+'.log')).open('w') as log:
result = subprocess.run(command, stdout=log, stderr=subprocess.STDOUT, timeout=180)
report['commands'].append({'name':name, 'command':command, 'exitCode':result.returncode,
'logSha256':sha(output / (name+'.log'))})
assert result.returncode == 0, name
try:
assert all(p['success'] for p in cmm_probes), 'Profile differs from the analytic model'
run('qpdf-check',['/usr/bin/qpdf','--check',str(source)])
sheets=[]
for scale in (.5, 1., 4.):
name = 'scale-'+str(scale).replace('.','_'); folder=output/name
(folder/'pdfium').mkdir(parents=True)
run(name+'-pdfium',[str(build/'pdf-worker-evidence'),str(source),str(folder/'pdfium'),str(scale)])
run(name+'-poppler',['/usr/bin/pdftoppm','-singlefile','-cropbox','-r',str(72*scale),'-png',str(source),str(folder/'poppler')])
meta=json.loads((folder/'pdfium/metadata.json').read_text())
assert meta['renderScale']==scale and 'sandbox enabled' in meta['transport']
images={'pdfium':Image.open(folder/'pdfium/page-1.png').convert('RGB'),
'poppler':Image.open(folder/'poppler.png').convert('RGB')}
dimensions=[math.ceil(v*scale) for v in spec['pageSizePt']]
assert all(list(im.size)==dimensions for im in images.values())
row={'scale':scale, 'dpi':72*scale, 'dimensions':dimensions, 'probes':[], 'success':True}
for probe in spec['probes']:
x,y=probe['pointPt']; point=(round(x*scale),round((spec['pageSizePt'][1]-y)*scale))
colors={key:list(im.getpixel(point)) for key,im in images.items()}
errors={key:max(abs(a-b) for a,b in zip(color,probe['expectedRgb'])) for key,color in colors.items()} if 'expectedRgb' in probe else {}
matched=all(error<=spec['tolerance8Bit'] for error in errors.values())
row['probes'].append({**probe,'pointPx':point,'actualRgb':colors,'maxChannelError':errors,
'checkedAgainstAnalyticColor':bool(errors),'success':matched})
row['success'] &= matched
# A profile ignored in favor of its DeviceCMYK Alternate cannot pass.
black={p['kind']:p for p in row['probes'] if p['sample']==4}
differences={key:min(black['icc-vector']['actualRgb'][key][i]-black['device-control']['actualRgb'][key][i]
for i in range(3)) for key in images}
row['blackProfileMinusAlternateMinChannel']=differences
row['profileEffectDetected']=all(value>=80 for value in differences.values())
row['success'] &= row['profileEffectDetected']
diff=ImageChops.difference(images['pdfium'],images['poppler']); diff.save(folder/'difference.png')
row['meanAbsoluteChannelError']=sum(ImageStat.Stat(diff).mean)/3
sheet=Image.new('RGB',(1500,340),'#eeeeee'); draw=ImageDraw.Draw(sheet)
for col,(label,im) in enumerate([('PDFium - isolated worker',images['pdfium']),('Poppler - independent',images['poppler']),('Absolute RGB difference',diff)]):
draw.text((col*500+10,10),f'{scale:g}x / {label}',fill='black')
thumb=im.copy(); thumb.thumbnail((488,295)); sheet.paste(thumb,(col*500+(500-thumb.width)//2,35))
sheet.save(folder/'comparison.png'); sheets.append(sheet); report['scales'].append(row)
overview=Image.new('RGB',(1500,340*len(sheets)),'white')
for i,sheet in enumerate(sheets): overview.paste(sheet,(0,340*i))
overview.save(output/'overview.png')
report['originalUnchanged']=sha(source)==spec['sha256'] and sha(icc_file)==spec['profileSha256']
report['binariesUnchanged']=all(sha(build/name)==digest for name,digest in binaries.items())
report['success']=all(row['success'] for row in report['scales']) and report['originalUnchanged'] and report['binariesUnchanged']
finally:
report['evidenceSha256']={str(path.relative_to(output)):sha(path) for path in sorted(output.rglob('*')) if path.is_file()}
(output/'report.json').write_text(json.dumps(report,indent=2)+'\n')
print(json.dumps({'success':report['success'],'scales':len(report['scales']),
'analyticImageAndVectorProbes':sum(p['checkedAgainstAnalyticColor'] for row in report['scales'] for p in row['probes'])}))
if not report['success']: raise SystemExit('Color validation failed; inspect the retained report')
if __name__=='__main__':
main()
tests/cmyk_lut/run_ubuntu_candidate.py
+99
View File
@@ -0,0 +1,99 @@
#!/usr/bin/env python3
"""Validate the fixed rendering-intent candidate in the dedicated Ubuntu guest."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import platform
import shutil
import subprocess
import sys
import time
ROOT = Path(__file__).resolve().parents[2]
EXPECTED_LIBRARY = '0c968f503ce549bad5301af2cc3fd0b83c37437315ac0e62dff439d6a1c6cc4d'
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--library', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
parser.add_argument('--build-dir', type=Path, default=Path.home() / 'docview-build')
args = parser.parse_args()
os_release = platform.freedesktop_os_release()
assert os_release['ID'] == 'ubuntu' and os_release['VERSION_ID'] == '24.04'
library = args.library.resolve()
assert sha(library) == EXPECTED_LIBRARY
output = args.output.resolve()
build = args.build_dir.resolve()
inputs = [library, ROOT / '.deps/pdfium/lib/libpdfium.so']
inputs += [build / name for name in ('docview', 'docview-pdf-worker', 'docview-archive-worker', 'pdf-worker-evidence')]
identities = {str(path): sha(path) for path in inputs}
output.mkdir(parents=True, exist_ok=False)
env = os.environ.copy()
assert not env.get('LD_PRELOAD') and not env.get('LD_AUDIT')
assert not env.get('QTWEBENGINE_DISABLE_SANDBOX')
env.pop('WAYLAND_DISPLAY', None)
env.update(LD_LIBRARY_PATH=str(library.parent) + ':/opt/docview-deps/lib:/opt/docview-qt/6.11.2/gcc_64/lib',
QT_QPA_PLATFORM='xcb', QT_QUICK_BACKEND='software',
QTWEBENGINE_CHROMIUM_FLAGS='--disable-gpu', QT_SCALE_FACTOR='1',
QT_AUTO_SCREEN_SCALE_FACTOR='0', XDG_SESSION_TYPE='x11')
report = {'schemaVersion': 1, 'success': False, 'osRelease': os_release,
'platform': platform.platform(), 'pythonVersion': platform.python_version(),
'runnerSha256': sha(Path(__file__)), 'inputs': identities, 'commands': [],
'productionDependencyReplaced': False,
'scope': 'Same fixed Linux candidate library as Arch, used by the existing Ubuntu-built application and workers. Dedicated Ubuntu VM, Xvfb/software, ordinary worker sandbox. No package installation or physical-display acceptance.'}
def run(label, command):
start = time.monotonic()
log = output / (label + '.log')
with log.open('w') as stream:
result = subprocess.run(command, cwd=ROOT, env=env,
stdout=stream, stderr=subprocess.STDOUT, timeout=900)
report['commands'].append({'name': label, 'command': command, 'exitCode': result.returncode,
'seconds': time.monotonic() - start, 'logSha256': sha(log)})
print(label, result.returncode, flush=True)
return result.returncode
try:
linked = subprocess.run(['ldd', str(build / 'docview-pdf-worker')], env=env,
capture_output=True, text=True, check=True).stdout
assert str(library) in linked and 'not found' not in linked
(output / 'worker-ldd.txt').write_text(linked)
run('cmyk', [sys.executable, str(ROOT / 'tests/cmyk_lut/run.py'),
'--build-dir', str(build), '--output', str(output / 'cmyk')])
run('intents', [sys.executable, str(ROOT / 'tests/cmyk_lut/intents.py'),
'--build-dir', str(build), '--pdfium-library', str(library),
'--scales', '.5', '1', '4', '--output', str(output / 'intents')])
ctest_status = run('ctest', ['xvfb-run', '-a', '-s', '-screen 0 1100x760x24',
'dbus-run-session', '--', 'ctest', '--test-dir', str(build),
'--output-on-failure', '--output-junit', str(output / 'tests.xml')])
shutil.copyfile(build / 'Testing/Temporary/LastTest.log', output / 'LastTest.log')
cmyk = json.loads((output / 'cmyk/report.json').read_text())
intents = json.loads((output / 'intents/report.json').read_text())
report['cmykSuccess'] = cmyk['success']
report['intentPdfiumSuccess'] = intents.get('pdfiumSuccess', False)
report['intentPopplerSuccess'] = intents.get('popplerSuccess', False)
report['ctestSuccess'] = ctest_status == 0
report['allBinaryInputsUnchanged'] = all(sha(Path(path)) == digest for path, digest in identities.items())
report['candidateTestsPass'] = (report['cmykSuccess'] and report['intentPdfiumSuccess']
and report['ctestSuccess'] and report['allBinaryInputsUnchanged'])
report['success'] = report['candidateTestsPass'] and intents['renderingAcceptance']
finally:
report['evidenceSha256'] = {str(path.relative_to(output)): sha(path)
for path in sorted(output.rglob('*')) if path.is_file()}
(output / 'report.json').write_text(json.dumps(report, indent=2) + '\n')
print(json.dumps({key: report.get(key) for key in ('success', 'candidateTestsPass',
'intentPopplerSuccess', 'allBinaryInputsUnchanged')}))
if not report['success']:
raise SystemExit('Comparison not fully accepted; candidate and comparator verdicts are retained separately.')
if __name__ == '__main__':
main()
tests/cmyk_lut/transparency.py
+222
View File
@@ -0,0 +1,222 @@
#!/usr/bin/env python3
"""Check RI masks/patterns against declared PDF state rules and a direct CMM.
This runner never treats agreement between renderers as the expected value.
The numerical DeviceRGB mask convention is distinct from intent selection.
"""
import argparse
from io import BytesIO
import itertools
import json
import math
import os
from pathlib import Path
import platform
import shutil
import subprocess
from PIL import Image, ImageChops, ImageCms, ImageDraw
from pypdf import PdfReader
from intents import Cmm, INTENTS, loaded_pdfium, sha
ROOT=Path(__file__).resolve().parents[2]
CORPUS=ROOT/'tests/fixtures/pdf/intent-transparency'
PROFILE_SHA='b9a40bfc18e8280eefe0fedc4e607467e1cc27adeee3beb77f4a2b83fc7e9315'
def preserved_inputs():
paths=[ROOT/'tests/fixtures/pdf/generate_rendering_intents.py',ROOT/'tests/cmyk_lut/intents.py']
paths += sorted((ROOT/'tests/fixtures/pdf/rendering-intents').glob('*'))
return {str(p.relative_to(ROOT)):sha(p) for p in paths if p.is_file()}
def verify(corpus,spec):
assert spec['schemaVersion']==1 and spec['pageCount']==4
assert spec['pageSizePt']==[880,850] and spec['sample']==[0,.75,.25,0]
assert spec['tolerance8Bit']==4 and len(spec['cases'])==27
assert len(spec['probes'])==108
assert sha(corpus/spec['file'])==spec['sha256']
assert sha(ROOT/'tests/fixtures/pdf/generate_intent_transparency.py')==spec['generatorSha256']
profile=(corpus/spec['profile']['file']).read_bytes()
assert len(profile)==984 and sha(corpus/spec['profile']['file'])==PROFILE_SHA==spec['profile']['sha256']
assert profile==(ROOT/spec['profile']['origin']).read_bytes()
pdf=PdfReader(corpus/spec['file']);assert len(pdf.pages)==4
identities={};probe_ids=set()
for page_index,page in enumerate(pdf.pages,1):
assert list(page.mediabox)==[0,0,880,850]
resources=page['/Resources'];patterns=resources['/Pattern'];gs=resources['/ExtGState']
assert resources['/ColorSpace']['/CS'][1].get_object().get_data()==profile
assert resources['/ColorSpace']['/PCS'][0]=='/Pattern'
for key in ('U','ULarge'):
stencil=patterns['/'+key]
assert stencil['/PaintType']==2
assert stencil.get_data().split()==[b'0',b'0',str(stencil['/XStep']).encode(),str(stencil['/YStep']).encode(),b're',b'f']
for key in ('Tile','TileForm','ShadeForm'):
obj=patterns.raw_get('/'+key) if key=='Tile' else resources['/XObject'].raw_get('/'+key)
identities.setdefault(key,obj.idnum);assert identities[key]==obj.idnum
assert b' ri' not in patterns['/Tile'].get_data()
assert '/ExtGState' not in patterns['/Shade']
for label in ('Tile','Shade'):
form=resources['/XObject']['/'+label+'Form']
assert form.get_data().startswith(b'/AbsoluteColorimetric ri\n')
assert form['/Resources']['/Pattern'].raw_get('/Tile').idnum==identities['Tile']
for i,intent in enumerate(INTENTS):
assert ('/'+intent+' ri').encode() in patterns['/TileRi'+str(i)].get_data()
assert patterns['/ShadeRi'+str(i)]['/ExtGState']['/RI']=='/'+intent
for key in ('Mask','OpaqueBC','BC0','BC1','BC2','BC3',*[f'MaskRi{i}' for i in range(4)],
*[f'MaskGs{i}' for i in range(4)],*[f'MaskQ{i}' for i in range(4)]):
mask=gs['/'+key]['/SMask'];group=mask['/G']
assert mask['/S']=='/Luminosity' and mask['/TR']=='/Identity'
assert group['/Group']['/CS']=='/DeviceRGB' and bool(group['/Group']['/I'])
assert group['/Resources']['/ColorSpace']['/CS'][1].get_object().get_data()==profile
if key.startswith('BC'):assert group.get_data().strip()==b''
assert gs['/NoMask']['/SMask']=='/None'
assert gs['/Mask']['/SMask'].raw_get('/G').idnum==gs['/OpaqueBC']['/SMask'].raw_get('/G').idnum
content=page.get_contents().get_data()
assert content.startswith(b'1 g 0 0 880 850 re f\n')
for p in (p for p in spec['probes'] if p['page']==page_index):
assert p['id'] not in probe_ids;probe_ids.add(p['id'])
assert ('\n'.join(p['commandSequence'])+'\n').encode() in content
case=p['case'];column=p['column'];current=(2,0,2,1)[column] if case.endswith('-reuse') else column
assert p['callerIntentIndex']==current and p['callerIntent']==INTENTS[current]
expected=current;oracle='icc'
if case in ('tiling-page-initial','tiling-colored-qQ','shading-page-initial'):expected=1
elif case in ('tiling-colored-ri','tiling-colored-gs','tiling-form-ri','shading-pattern-ri','shading-form-ri',
'mask-internal-ri','mask-internal-gs'):expected=(current+1)%4
if case.startswith('mask-'):
oracle='luminosity-icc'
if case in ('mask-q-inner-none','mask-none'):oracle='black'
elif case=='mask-BC-only':oracle='luminosity-bc'
assert p['expectedIntentIndex']==expected and p['expectedIntent']==INTENTS[expected]
assert p['oracle']==oracle and p['components']==[0,.75,.25,0]
if oracle=='luminosity-bc':assert p['backgroundRgb']==[[1,0,0],[0,1,0],[0,0,1],[0,.5,1]][column]
return profile,{'pages':4,'probes':len(probe_ids),'reusedObjectIds':identities,
'uncoloredStencilsContainOnlyGeometry':True,'profileByteIdentical':True,
'expectedIntentIndependentlyChecked':True}
def oracle(probe,cmm):
kind=probe['oracle']
if kind=='black':return [0,0,0],{'kind':kind}
if kind=='luminosity-bc':rgb=[255*c for c in probe['backgroundRgb']]
else:rgb=cmm.color('distinct',probe['expectedIntentIndex'],probe['components'])
if kind=='icc':return rgb,{'kind':kind,'directCmmRgb8':rgb}
luminance=sum(w*c for w,c in zip((.30,.59,.11),rgb))
return [round(255-luminance)]*3,{'kind':kind,'groupRgb8':rgb,'luminosity8Bit':luminance,
'recommendedDeviceRgbWeights':[.30,.59,.11]}
def cmm_checks(profile,cmm,spec):
colors=[cmm.color('distinct',i,spec['sample']) for i in range(4)]
lumas=[sum(w*c for w,c in zip((.30,.59,.11),rgb)) for rgb in colors]
separation=min(abs(a-b) for a,b in itertools.combinations(lumas,2))
assert separation>2*spec['tolerance8Bit']
source=ImageCms.ImageCmsProfile(BytesIO(profile));sample=tuple(round(v*255) for v in spec['sample'])
checks=[]
for i in range(4):
transform=ImageCms.buildTransformFromOpenProfiles(source,ImageCms.createProfile('sRGB'),'CMYK','RGB',renderingIntent=i)
quantized=list(ImageCms.applyTransform(Image.new('CMYK',(1,1),sample),transform).getpixel((0,0)))
assert max(abs(a-b) for a,b in zip(quantized,colors[i]))<=1
checks.append({'intent':INTENTS[i],'directCmmRgb8':colors[i],'pillowQuantizedRgb8':quantized,
'luminosity8Bit':lumas[i],'blackOnWhiteGray8Bit':round(255-lumas[i])})
return {'values':checks,'minimumLuminositySeparation':separation,
'noRenderedPixelsUsedForExpectedValues':True}
def main():
parser=argparse.ArgumentParser(description=__doc__)
parser.add_argument('--build-dir',type=Path)
parser.add_argument('--pdfium-library',type=Path)
parser.add_argument('--corpus',type=Path,default=CORPUS)
parser.add_argument('--output',type=Path,required=True)
parser.add_argument('--self-check',action='store_true')
parser.add_argument('--scales',type=float,nargs='+',default=[.5,1.,2.])
args=parser.parse_args()
if not args.self_check and not args.build_dir:parser.error('--build-dir required for rendering')
if not 1<=len(args.scales)<=4 or len(set(args.scales))!=len(args.scales) or any(not math.isfinite(v) or not .25<=v<=4 for v in args.scales):
parser.error('scales must be 1..4 unique finite values in .25..4')
output=args.output.resolve();output.mkdir(parents=True,exist_ok=False)
old=preserved_inputs();spec=json.loads((args.corpus/'manifest.json').read_text());source=(args.corpus/spec['file']).resolve()
report={'schemaVersion':1,'success':False,'renderingAcceptance':False,'fixtureValidationSuccess':False,
'mode':'fixture-self-check' if args.self_check else 'renderer-comparison',
'runnerSha256':sha(Path(__file__)),'sharedCmmAdapterSha256':sha(Path(__file__).with_name('intents.py')),
'fixtureManifestSha256':sha(args.corpus/'manifest.json'),'fixtureSha256':sha(source),
'pythonVersion':platform.python_version(),'pillowLcmsVersion':ImageCms.core.littlecms_version,
'tolerance8Bit':4,'commands':[],'scales':[],'preservedLegacyInputs':old,
'scope':'Synthetic ICC RI dispatch and state probes. Direct public LittleCMS calls supply colors, not renderer pixels; '
'the renderers may use the same CMM. DeviceRGB luminosity uses the PDF recommended weights, which are device-dependent. '
'No physical-display, press, general ICC, or all-PDF acceptance claim.'}
cmm=None;environment=os.environ.copy();binaries={}
try:
profile,verification=verify(args.corpus,spec);report['fixtureVerification']=verification
cmm=Cmm({'distinct':profile})
report['directCmm']={'encodedVersion':cmm.version,'libraries':cmm.libraryPaths,
'input':'CMYK double percent','output':'8-bit sRGB','flags':0}
report['cmmChecks']=cmm_checks(profile,cmm,spec);report['fixtureValidationSuccess']=True
if not args.self_check:
build=args.build_dir.resolve()
binaries={name:sha(build/name) for name in ('pdf-worker-evidence','docview-pdf-worker')};report['binaries']=binaries
if args.pdfium_library:
environment['LD_LIBRARY_PATH']=str(args.pdfium_library.resolve().parent)+(':'+environment['LD_LIBRARY_PATH'] if environment.get('LD_LIBRARY_PATH') else '')
assert not environment.get('LD_PRELOAD') and not environment.get('LD_AUDIT')
report['pdfiumLibrary']=loaded_pdfium(build/'docview-pdf-worker',environment,args.pdfium_library)
for program in ('qpdf','pdftoppm'):assert shutil.which(program)
report['popplerVersion']=subprocess.run(['pdftoppm','-v'],capture_output=True,text=True,check=True).stderr.splitlines()[0]
def run(label,command,env=None):
log=output/(label+'.log')
with log.open('w') as f:result=subprocess.run(command,stdout=f,stderr=subprocess.STDOUT,env=env,timeout=300)
report['commands'].append({'name':label,'command':command,'exitCode':result.returncode,'logSha256':sha(log)})
assert result.returncode==0,label+' failed'
run('qpdf-check',['qpdf','--check',str(source)])
for scale in args.scales:
label='scale-'+str(scale).replace('.','_');folder=output/label;(folder/'pdfium').mkdir(parents=True)
run(label+'-pdfium',[str(build/'pdf-worker-evidence'),str(source),str(folder/'pdfium'),str(scale)],environment)
run(label+'-poppler',['pdftoppm','-cropbox','-r',str(72*scale),'-png',str(source),str(folder/'poppler')])
metadata=json.loads((folder/'pdfium/metadata.json').read_text())
assert metadata['pageCount']==spec['pageCount'] and metadata['renderScale']==scale and 'sandbox enabled' in metadata['transport']
row={'scale':scale,'probes':[],'pdfiumSuccess':True,'popplerSuccess':True,'pages':[]}
for page in range(1,5):
paths={'pdfium':folder/'pdfium'/f'page-{page}.png','poppler':folder/f'poppler-{page}.png'}
images={key:Image.open(path).convert('RGB') for key,path in paths.items()}
assert all(list(im.size)==[math.ceil(v*scale) for v in spec['pageSizePt']] for im in images.values())
for probe in (p for p in spec['probes'] if p['page']==page):
x,y=probe['pointPt'];pixel=(round(x*scale),round((850-y)*scale))
expected,details=oracle(probe,cmm)
# Interior 3x3 minimum/maximum detects seams/unstable edge
# samples instead of silently selecting a favorable pixel.
samples={key:[list(im.getpixel((pixel[0]+dx,pixel[1]+dy))) for dy in (-1,0,1) for dx in (-1,0,1)] for key,im in images.items()}
actual={key:values[4] for key,values in samples.items()}
errors={key:max(abs(a-b) for color in values for a,b in zip(color,expected)) for key,values in samples.items()}
matches={key:err<=4 for key,err in errors.items()}
row['probes'].append({**probe,'pointPx':pixel,'oracleDetails':details,'expectedRgb':expected,
'actualRgb':actual,'neighborhoodMinRgb':{k:[min(c[i] for c in v) for i in range(3)] for k,v in samples.items()},
'neighborhoodMaxRgb':{k:[max(c[i] for c in v) for i in range(3)] for k,v in samples.items()},
'maxChannelError':errors,'matches':matches})
for key,match in matches.items():row[key+'Success'] &= match
diff=ImageChops.difference(images['pdfium'],images['poppler']);diff.save(folder/f'difference-{page}.png')
sheet=Image.new('RGB',(1260,440),'#eeeeee');draw=ImageDraw.Draw(sheet)
for col,(key,im) in enumerate([*images.items(),('difference',diff)]):
draw.text((col*420+8,7),f'{scale:g}x page {page}: {key}',fill='black')
thumb=im.copy();thumb.thumbnail((410,400));sheet.paste(thumb,(col*420+5,30))
sheet.save(folder/f'comparison-{page}.png')
row['pages'].append({'page':page,'images':{key:sha(path) for key,path in paths.items()}})
row['failedProbes']={engine:sum(not p['matches'][engine] for p in row['probes']) for engine in ('pdfium','poppler')}
row['success']=row['pdfiumSuccess'] and row['popplerSuccess'];report['scales'].append(row)
report['pdfiumSuccess']=all(r['pdfiumSuccess'] for r in report['scales'])
report['popplerSuccess']=all(r['popplerSuccess'] for r in report['scales'])
report['binariesUnchanged']=all(sha(build/n)==value for n,value in binaries.items()) and sha(Path(report['pdfiumLibrary']['path']))==report['pdfiumLibrary']['sha256']
report['originalUnchanged']=sha(source)==spec['sha256'] and sha(args.corpus/spec['profile']['file'])==PROFILE_SHA
report['legacyInputsUnchanged']=preserved_inputs()==old
if not args.self_check:
report['success']=report['renderingAcceptance']=all(report[k] for k in ('pdfiumSuccess','popplerSuccess','binariesUnchanged','originalUnchanged','legacyInputsUnchanged'))
except Exception as error:
report['error']=type(error).__name__+': '+str(error)
raise
finally:
if cmm:cmm.close()
report['evidenceSha256']={str(p.relative_to(output)):sha(p) for p in sorted(output.rglob('*')) if p.is_file()}
(output/'report.json').write_text(json.dumps(report,indent=2)+'\n')
print(json.dumps({key:report.get(key) for key in ('mode','fixtureValidationSuccess','success','pdfiumSuccess','popplerSuccess','legacyInputsUnchanged')}))
if not args.self_check and not report['success']:raise SystemExit('Pixel criteria failed; every failing probe remains in report.json')
if __name__=='__main__':main()
tests/cmyk_lut/validate_context_candidate.py
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""Run corrected RI, transparency and original CMYK probes plus DocView CTest."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import platform
import shutil
import subprocess
import sys
import time
ROOT = Path(__file__).resolve().parents[2]
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--library', type=Path, required=True)
parser.add_argument('--build-dir', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
library, build = args.library.resolve(), args.build_dir.resolve()
binaries = [library, ROOT / '.deps/pdfium/lib/libpdfium.so'] + [build / name for name in
('docview', 'docview-pdf-worker', 'docview-archive-worker', 'pdf-worker-evidence')]
identities = {str(path): sha(path) for path in binaries}
environment = os.environ.copy()
assert not environment.get('LD_PRELOAD') and not environment.get('LD_AUDIT')
assert not environment.get('QTWEBENGINE_DISABLE_SANDBOX')
environment.pop('WAYLAND_DISPLAY', None)
environment.update(LD_LIBRARY_PATH=str(library.parent) + (
':' + environment['LD_LIBRARY_PATH'] if environment.get('LD_LIBRARY_PATH') else ''),
QT_QPA_PLATFORM='xcb', QT_QUICK_BACKEND='software',
QTWEBENGINE_CHROMIUM_FLAGS='--disable-gpu', QT_SCALE_FACTOR='1',
QT_AUTO_SCREEN_SCALE_FACTOR='0', XDG_SESSION_TYPE='x11')
output = args.output.resolve()
output.mkdir(parents=True, exist_ok=False)
report = {'schemaVersion': 1, 'candidateTestsPass': False, 'acceptanceComplete': False,
'platform': platform.platform(), 'osRelease': platform.freedesktop_os_release(),
'inputs': identities, 'runnerSha256': sha(Path(__file__)), 'commands': [],
'productionDependencyReplaced': False,
'scope': 'Fixed candidate selected only for this test process tree. Generated fixtures and corrected stream-entry RI expectations; no physical display, Windows, human golden or release acceptance.'}
def run(label, command):
start = time.monotonic()
log = output / (label + '.log')
with log.open('w') as stream:
result = subprocess.run(command, cwd=ROOT, env=environment,
stdout=stream, stderr=subprocess.STDOUT, timeout=900)
report['commands'].append({'name': label, 'command': command,
'exitCode': result.returncode, 'seconds': time.monotonic() - start,
'logSha256': sha(log)})
print(label, result.returncode, flush=True)
return result.returncode
try:
linked = subprocess.run(['ldd', str(build / 'docview-pdf-worker')], env=environment,
capture_output=True, text=True, check=True).stdout
assert str(library) in linked and 'not found' not in linked
(output / 'worker-ldd.txt').write_text(linked)
run('cmyk', [sys.executable, str(ROOT / 'tests/cmyk_lut/run.py'),
'--build-dir', str(build), '--output', str(output / 'cmyk')])
run('intents', [sys.executable, str(ROOT / 'tests/cmyk_lut/intents.py'),
'--build-dir', str(build), '--pdfium-library', str(library),
'--corpus', str(ROOT / 'tests/fixtures/pdf/rendering-intents-context'),
'--scales', '.5', '1', '4', '--output', str(output / 'intents')])
run('transparency', [sys.executable, str(ROOT / 'tests/cmyk_lut/transparency.py'),
'--build-dir', str(build), '--pdfium-library', str(library),
'--scales', '.5', '1', '2', '--output', str(output / 'transparency')])
ctest_status = run('ctest', ['xvfb-run', '-a', '-s', '-screen 0 1100x760x24',
'dbus-run-session', '--', 'ctest', '--test-dir', str(build),
'--output-on-failure', '--output-junit', str(output / 'tests.xml')])
shutil.copyfile(build / 'Testing/Temporary/LastTest.log', output / 'LastTest.log')
reports = {name: json.loads((output / name / 'report.json').read_text())
for name in ('cmyk', 'intents', 'transparency')}
report['cmykSuccess'] = reports['cmyk']['success']
report['correctedOracle'] = reports['intents']['oracleRevision'] == 2 and not reports['intents']['oracleKnownIssue']
for name in ('intents', 'transparency'):
data = reports[name]
report[name] = {key: data.get(key, False) for key in
('pdfiumSuccess', 'popplerSuccess', 'fixtureValidationSuccess', 'binariesUnchanged', 'originalUnchanged')}
report[name]['probeCount'] = sum(len(row['probes']) for row in data['scales'])
report['ctestSuccess'] = ctest_status == 0
report['allBinaryInputsUnchanged'] = all(sha(Path(path)) == value for path, value in identities.items())
report['candidateTestsPass'] = all((report['cmykSuccess'], report['correctedOracle'],
report['ctestSuccess'], report['allBinaryInputsUnchanged'], *[
report[name][key] for name in ('intents', 'transparency') for key in
('pdfiumSuccess', 'fixtureValidationSuccess', 'binariesUnchanged', 'originalUnchanged')]))
report['comparisonAgreement'] = all(report[name]['popplerSuccess'] for name in ('intents', 'transparency'))
finally:
report['evidenceSha256'] = {str(path.relative_to(output)): sha(path)
for path in sorted(output.rglob('*')) if path.is_file()}
(output / 'report.json').write_text(json.dumps(report, indent=2) + '\n')
print(json.dumps({key: report.get(key) for key in
('candidateTestsPass', 'comparisonAgreement', 'allBinaryInputsUnchanged', 'acceptanceComplete')}))
if not report['candidateTestsPass']:
raise SystemExit('Candidate regression failed; all individual outcomes are retained.')
if __name__ == '__main__':
main()
tests/cmyk_lut/validate_pdfium_candidate.py
+99
View File
@@ -0,0 +1,99 @@
#!/usr/bin/env python3
"""Run the isolated candidate's upstream tests and both ICC regression corpora."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import subprocess
import sys
import time
ROOT = Path(__file__).resolve().parents[2]
def sha(path):
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--output', type=Path, required=True)
parser.add_argument('--build-dir', type=Path, default=ROOT / 'build')
parser.add_argument('--source', type=Path, default=ROOT / 'build-pdfium-intent/source')
parser.add_argument('--variant', choices=('baseline', 'patched'), default='patched')
args = parser.parse_args()
output = args.output.resolve()
output.mkdir(parents=True, exist_ok=False)
source = args.source.resolve()
binaries = source / 'out' / args.variant
library = binaries / 'libpdfium.so'
environment = os.environ.copy()
assert not environment.get('LD_PRELOAD') and not environment.get('LD_AUDIT')
environment['LD_LIBRARY_PATH'] = str(binaries) + (
':' + environment['LD_LIBRARY_PATH'] if environment.get('LD_LIBRARY_PATH') else '')
environment['LOCPATH'] = str(ROOT / 'build-pdfium-intent/locales')
inputs = [library, binaries / 'pdfium_unittests', binaries / 'pdfium_embeddertests',
args.build_dir / 'pdf-worker-evidence', args.build_dir / 'docview-pdf-worker',
ROOT / '.deps/pdfium/lib/libpdfium.so']
identities = {str(path.resolve()): sha(path) for path in inputs}
report = {'schemaVersion': 1, 'variant': args.variant, 'success': False,
'runnerSha256': sha(Path(__file__)), 'inputs': identities, 'commands': [],
'productionDependencyReplaced': False,
'scope': 'Isolated Linux source-build candidate. No Windows, Ubuntu, calibrated display, human golden, or release acceptance claimed.'}
def run(label, command, cwd=ROOT):
start = time.monotonic()
log = output / (label + '.log')
with log.open('w') as stream:
result = subprocess.run(command, cwd=cwd, env=environment,
stdout=stream, stderr=subprocess.STDOUT, timeout=600)
report['commands'].append({'name': label, 'command': command,
'exitCode': result.returncode,
'seconds': time.monotonic() - start,
'logSha256': sha(log)})
return result.returncode
try:
loader = subprocess.run(['ldd', str(args.build_dir.resolve() / 'docview-pdf-worker')],
env=environment, capture_output=True, text=True, check=True)
assert str(library) in loader.stdout
(output / 'worker-ldd.txt').write_text(loader.stdout)
upstream = []
for target in ('pdfium_unittests', 'pdfium_embeddertests'):
result = output / (target + '.json')
status = run(target, [str(binaries / target), '--gtest_output=json:' + str(result)], source)
data = json.loads(result.read_text()) if result.exists() else {}
upstream.append({'target': target, 'exitCode': status,
**{key: data.get(key) for key in ('tests', 'failures', 'disabled', 'errors', 'time')}})
report['upstreamTests'] = upstream
run('cmyk', [sys.executable, str(ROOT / 'tests/cmyk_lut/run.py'),
'--build-dir', str(args.build_dir.resolve()), '--output', str(output / 'cmyk')])
run('intents', [sys.executable, str(ROOT / 'tests/cmyk_lut/intents.py'),
'--build-dir', str(args.build_dir.resolve()),
'--pdfium-library', str(library), '--scales', '.5', '1', '4',
'--output', str(output / 'intents')])
cmyk = json.loads((output / 'cmyk/report.json').read_text())
intents = json.loads((output / 'intents/report.json').read_text())
report['cmykSuccess'] = cmyk['success']
report['intentPdfiumSuccess'] = intents.get('pdfiumSuccess', False)
report['intentPopplerSuccess'] = intents.get('popplerSuccess', False)
report['intentRenderingAcceptance'] = intents['renderingAcceptance']
report['upstreamTestsPass'] = all(row['exitCode'] == 0 and row['failures'] == 0 and row['errors'] == 0 for row in upstream)
report['allBinaryInputsUnchanged'] = all(sha(Path(path)) == digest for path, digest in identities.items())
report['candidatePdfiumProbesPass'] = cmyk['success'] and report['intentPdfiumSuccess']
report['success'] = (report['upstreamTestsPass'] and report['candidatePdfiumProbesPass']
and report['intentRenderingAcceptance'] and report['allBinaryInputsUnchanged'])
finally:
report['evidenceSha256'] = {str(path.relative_to(output)): sha(path)
for path in sorted(output.rglob('*')) if path.is_file()}
(output / 'report.json').write_text(json.dumps(report, indent=2) + '\n')
print(json.dumps({key: report.get(key) for key in ('success', 'upstreamTestsPass',
'candidatePdfiumProbesPass', 'intentPopplerSuccess', 'allBinaryInputsUnchanged')}))
if not report['success']:
raise SystemExit('Validation not fully accepted; see separate candidate and comparison results.')
if __name__ == '__main__':
main()
tests/compare_font_collection.py
+165
View File
@@ -0,0 +1,165 @@
#!/usr/bin/env python3
"""Compare real production-worker TTC face 0/1 rendering with separate TTFs.
Fonts/config/cache exist only inside TemporaryDirectory. No font is installed
or retained in the evidence directory. Requires fontTools, Pillow, fontconfig,
and the pdf-worker-evidence build target (which does not link PDFium itself).
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
import shutil
import struct
import subprocess
import tempfile
import time
from xml.sax.saxutils import escape
import fontTools
from fontTools.ttLib import TTCollection, TTFont
from PIL import Image, ImageChops, ImageStat
ROOT = Path(__file__).resolve().parent.parent
FAMILY = "DocView Collection Test"
def digest(path):
with path.open("rb") as source:
return hashlib.file_digest(source, "sha256").hexdigest()
def make_pdf(path):
stream = (b"BT /Regular 24 Tf 30 150 Td (Collection face 0123456789 AVWA) Tj ET\n"
b"BT /Bold 24 Tf 30 70 Td (Collection face 0123456789 AVWA) Tj ET\n")
objects = [b"<< /Type /Catalog /Pages 2 0 R >>", b"<< /Type /Pages /Count 1 /Kids [3 0 R] >>",
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 430 240] /Resources << /Font << /Regular 4 0 R /Bold 5 0 R >> >> /Contents 6 0 R >>",
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>",
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding >>",
b"<< /Length " + str(len(stream)).encode() + b" >>\nstream\n" + stream + b"endstream"]
output = bytearray(b"%PDF-1.7\n"); offsets = [0]
for index, obj in enumerate(objects, 1):
offsets.append(len(output)); output += f"{index} 0 obj\n".encode() + obj + b"\nendobj\n"
xref = len(output); output += f"xref\n0 {len(offsets)}\n0000000000 65535 f \n".encode()
for offset in offsets[1:]:
output += f"{offset:010d} 00000 n \n".encode()
output += f"trailer\n<< /Size {len(offsets)} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n".encode()
path.write_bytes(output)
def rename_copy(source, target, style):
# Format conversion is a modified font. Rename primary names so no
# Reserved Font Name is used by our private derived test fonts.
font = TTFont(source, recalcTimestamp=False)
replacements = {1: FAMILY, 3: f"DocViewCollectionTest-{style}", 4: f"{FAMILY} {style}",
6: f"DocViewCollectionTest-{style}", 16: FAMILY, 18: f"{FAMILY} {style}",
21: FAMILY, 25: "DocViewCollectionTest"}
for record in font["name"].names:
if record.nameID in replacements:
record.string = replacements[record.nameID].encode(record.getEncoding())
font.save(target); font.close()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--build-dir", type=Path, default=ROOT / os.environ.get("DOCVIEW_BUILD_DIR", "build"))
parser.add_argument("--regular", type=Path, default=Path("/usr/share/fonts/liberation/LiberationSans-Regular.ttf"))
parser.add_argument("--bold", type=Path, default=Path("/usr/share/fonts/liberation/LiberationSans-Bold.ttf"))
parser.add_argument("--license", type=Path, default=Path("/usr/share/licenses/ttf-liberation/LICENSE"))
parser.add_argument("--output", type=Path, default=ROOT / "tests/results/font-collection")
args = parser.parse_args()
executable = args.build_dir.resolve() / "pdf-worker-evidence"
if not executable.is_file():
raise SystemExit("Build the pdf-worker-evidence target first.")
license_text = args.license.read_text()
assert "SIL OPEN FONT LICENSE Version 1.1" in license_text
assert all(p.is_file() for p in (args.regular, args.bold))
assert shutil.disk_usage(tempfile.gettempdir()).free > 64 * 1024 * 1024
output = args.output.resolve(); output.mkdir(parents=True, exist_ok=True)
pair_pdf = output / "regular-bold.pdf"; make_pdf(pair_pdf)
cases = {"paired-styles": pair_pdf, "existing-standard14": ROOT / "tests/fixtures/pdf/extended/unembedded-font.pdf"}
report = {"status": "incomplete", "fontTools_version": fontTools.__version__,
"fontconfig_version": subprocess.run(["fc-match", "--version"], capture_output=True, text=True, check=True).stderr.strip(),
"worker_sha256": digest(args.build_dir / "docview-pdf-worker"),
"evidence_exporter_sha256": digest(executable),
"font_sources": [{"path": str(p), "sha256": digest(p), "bytes": p.stat().st_size} for p in (args.regular, args.bold)],
"license": {"path": str(args.license), "sha256": digest(args.license), "identifier": "OFL-1.1"},
"generated_family": FAMILY, "fonts_redistributed": False,
"selection_transport": "production WorkerProcess + FontBroker + sandboxed PDF worker; no direct PDFium exporter", "runs": [], "comparisons": []}
temporary = None
with tempfile.TemporaryDirectory(prefix="docview-ttc-") as name:
temporary = Path(name)
standalone = temporary / "standalone"; collection_dir = temporary / "collection"
standalone.mkdir(); collection_dir.mkdir()
paths = [standalone / "regular.ttf", standalone / "bold.ttf"]
for source, path, style in zip((args.regular, args.bold), paths, ("Regular", "Bold")):
rename_copy(source, path, style)
collection = TTCollection()
collection.fonts = [TTFont(path, recalcTimestamp=False) for path in paths]
ttc_path = collection_dir / "two-faces.ttc"
collection.save(ttc_path, shareTables=False); collection.close()
ttc_data = ttc_path.read_bytes()
assert ttc_data[:4] == b"ttcf" and struct.unpack_from(">I", ttc_data, 8)[0] == 2
report["generated_fonts"] = {
"standalone": [{"style": style, "bytes": p.stat().st_size, "sha256": digest(p)} for p, style in zip(paths, ("Regular", "Bold"))],
"collection": {"bytes": len(ttc_data), "sha256": digest(ttc_path), "face_offsets": list(struct.unpack_from(">2I", ttc_data, 12)), "shared_tables": False}}
for mode, fonts in (("standalone", standalone), ("collection", collection_dir)):
cache = temporary / (mode + "-cache"); cache.mkdir()
config = temporary / (mode + ".conf")
config.write_text('<?xml version="1.0"?>\n<!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd">\n'
'<fontconfig>\n<dir>' + escape(str(fonts)) + '</dir>\n<cachedir>' + escape(str(cache))
+ '</cachedir>\n<alias><family>sans-serif</family><prefer><family>' + FAMILY
+ '</family></prefer></alias>\n</fontconfig>\n')
env = os.environ.copy(); env["FONTCONFIG_FILE"] = str(config); env["FONTCONFIG_PATH"] = str(temporary)
listing = subprocess.check_output(["fc-list", "--format=%{file}\t%{index}\t%{style}\n"], env=env, text=True)
index = []
for line in listing.splitlines():
path, face, style = line.split("\t"); assert Path(path).parent == fonts
index.append({"file": Path(path).name, "face_index": int(face), "style": style})
assert len(index) == 2
if mode == "collection": assert {item["face_index"] for item in index} == {0, 1}
for case, source in cases.items():
destination = output / mode / case
started = time.monotonic()
process = subprocess.run([str(executable), str(source), str(destination)], env=env,
text=True, capture_output=True, timeout=60)
if process.returncode:
raise RuntimeError(f"{mode}/{case}: exit {process.returncode}\n{process.stdout}\n{process.stderr}")
metadata = json.loads((destination / "metadata.json").read_text())
selected = [s for s in metadata["fontSelections"] if s.get("found")]
assert selected and all(s["actualFace"] == FAMILY for s in selected)
assert metadata["fontBytesTransferred"] > 0
if mode == "collection" and case == "paired-styles":
assert {s["faceIndex"] for s in selected} == {0, 1}
assert all(s["faceIndex"] == (1 if s["requestedWeight"] >= 600 else 0) for s in selected)
report["runs"].append({"mode": mode, "case": case, "source_sha256": digest(source),
"elapsed_seconds": time.monotonic() - started, "font_index": index,
"selections": selected, "font_bytes_transferred": metadata["fontBytesTransferred"]})
for case in cases:
baseline = output / "standalone" / case / "page-1.png"
collected = output / "collection" / case / "page-1.png"
left = Image.open(baseline).convert("RGB"); right = Image.open(collected).convert("RGB")
assert left.size == right.size
difference = ImageChops.difference(left, right)
histogram = ImageChops.lighter(ImageChops.lighter(difference.getchannel("R"), difference.getchannel("G")), difference.getchannel("B")).histogram()
report["comparisons"].append({"case": case, "dimensions": list(left.size), "equal_pixels": histogram[0],
"different_pixels": sum(histogram[1:]), "mean_absolute_channel_error": sum(ImageStat.Stat(difference).mean) / 3,
"standalone_png_sha256": digest(baseline), "collection_png_sha256": digest(collected)})
assert difference.getbbox() is None, f"Raster mismatch in {case}"
if case == "paired-styles":
regular = right.crop((50, 120, 830, 200)).convert("L")
bold = right.crop((50, 280, 830, 360)).convert("L")
regular_ink = sum(regular.histogram()[:128]); bold_ink = sum(bold.histogram()[:128])
assert bold_ink > regular_ink > 0 and ImageChops.difference(regular, bold).getbbox() is not None
report["style_probe"] = {"regular_dark_pixels": regular_ink, "bold_dark_pixels": bold_ink,
"equal_text_at_equal_font_size": True, "regular_bold_distinct": True}
report["temporary_fonts_removed"] = not temporary.exists()
assert report["temporary_fonts_removed"]
report["status"] = "pass"
(output / "comparison.json").write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n")
print(json.dumps(report, ensure_ascii=False, indent=2))
if __name__ == "__main__":
main()
tests/compare_link_borders.py
+96
View File
@@ -0,0 +1,96 @@
#!/usr/bin/env python3
"""Read-only link annotation evidence through the production sandboxed worker.
Poppler is an independent comparison, not an all-features correctness oracle.
Run after building pdf-worker-evidence. Requires Pillow and Poppler utils.
"""
import argparse
import hashlib
import json
from pathlib import Path
import subprocess
import time
from PIL import Image, ImageChops, ImageStat
ROOT = Path(__file__).resolve().parent.parent
def sha(path):
with path.open('rb') as source:
return hashlib.file_digest(source, 'sha256').hexdigest()
def stats(a, b):
assert a.size == b.size
diff = ImageChops.difference(a, b)
pixels = list(diff.getdata())
return {'size': list(a.size), 'pixels': len(pixels), 'equal_pixels': sum(max(p) == 0 for p in pixels),
'pixels_max_delta_gt_16': sum(max(p) > 16 for p in pixels),
'mean_absolute_channel_error': sum(ImageStat.Stat(diff).mean) / 3}
def region(rect, rotation, margin=6):
points = []
for x in [rect[0], rect[2]]:
for y in [rect[1], rect[3]]:
x0, y0 = (x - 20) * 2, (450 - y) * 2
points.append([(x0, y0), (840 - y0, x0), (1020 - x0, 840 - y0), (y0, 1020 - x0)][rotation // 90])
return (min(p[0] for p in points) - margin, min(p[1] for p in points) - margin,
max(p[0] for p in points) + margin, max(p[1] for p in points) + margin)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--build-dir', type=Path, default=ROOT / 'build')
parser.add_argument('--output', type=Path, default=ROOT / 'tests/results/link-borders')
args = parser.parse_args()
out = args.output.resolve(); out.mkdir(parents=True, exist_ok=True)
build = args.build_dir.resolve(); helper = build / 'pdf-worker-evidence'
fixture = ROOT / 'tests/fixtures/pdf/link-borders/styles.pdf'
before = sha(fixture)
start = time.monotonic()
subprocess.run([str(helper), str(fixture), str(out / 'worker')], check=True, timeout=60)
elapsed = time.monotonic() - start
reference = subprocess.run(['/usr/bin/pdftoppm', '-cropbox', '-r', '144', '-png', str(fixture), str(out / 'poppler')], capture_output=True, check=True, timeout=60)
(out / 'poppler-stderr.txt').write_bytes(reference.stderr)
manifest = json.loads((fixture.parent / 'manifest.json').read_text())['fixtures'][0]
common = {'default', 'solid-rgb', 'dashed-border', 'dashed-bs-odd', 'dashed-default', 'underline',
'bs-overrides-border', 'zero-width', 'transparent', 'hidden', 'no-view', 'print-flag-visible'}
records = []
for index, rotation in enumerate([0, 90, 180, 270], 1):
a = Image.open(out / f'worker/page-{index}.png').convert('RGB')
b = Image.open(out / f'poppler-{index}.png').convert('RGB')
diff = ImageChops.difference(a, b); diff.save(out / f'difference-{index}.png')
comparisons = []
for case in manifest['annotations']:
bounds = region(case['rect'], rotation)
row = {'name': case['name'], 'reference_supported': case['name'] in common, **stats(a.crop(bounds), b.crop(bounds))}
if case['name'] in common:
# At 144dpi the grid-aligned borders should match except small
# independent rasterizer antialias/color-rounding differences.
assert row['mean_absolute_channel_error'] < 3, row
comparisons.append(row)
records.append({'page': index, 'rotation': rotation, 'whole_page': stats(a, b), 'annotations': comparisons})
navigation = ROOT / 'tests/fixtures/pdf/navigation.pdf'
navigation_before = sha(navigation)
subprocess.run([str(helper), str(navigation), str(out / 'navigation-worker')], check=True, timeout=60)
subprocess.run(['/usr/bin/pdftoppm', '-cropbox', '-f', '1', '-singlefile', '-r', '144', '-png', str(navigation), str(out / 'navigation-poppler')], capture_output=True, check=True, timeout=60)
a = Image.open(out / 'navigation-worker/page-1.png').convert('RGB')
b = Image.open(out / 'navigation-poppler.png').convert('RGB')
# The exact region where the original default link border was absent.
navigation_comparison = stats(a.crop((18, 368, 282, 400)), b.crop((18, 368, 282, 400)))
assert navigation_comparison['mean_absolute_channel_error'] < 3, navigation_comparison
assert before == sha(fixture) and navigation_before == sha(navigation)
result = {'transport': 'production WorkerProcess, sandboxed PDFWorker and FontBroker; no direct PDFium exporter',
'worker_sha256': sha(build / 'docview-pdf-worker'), 'exporter_sha256': sha(helper),
'qpdf_version': subprocess.check_output(['qpdf', '--version'], text=True).splitlines()[0],
'poppler_version': subprocess.run(['/usr/bin/pdftoppm', '-v'], capture_output=True, text=True).stderr.splitlines()[0],
'fixture_sha256': before, 'source_unchanged': True, 'four_page_export_seconds': elapsed,
'reference_status': 'independent Poppler comparison; manually inspect images; not a golden-image acceptance gate',
'reference_limitations': manifest['reference_limitations'],
'known_limitations': ['NoZoom/NoRotate geometry for generated links, saved normal APs, and native Text icons is covered separately by tests/results/pdf-annotation-flags.',
'Other annotation types without a supplied AP retain PDFium generation behavior; this corpus does not certify their NoZoom handling.',
'Text annotation icon appearance is viewer-specific and is not a link border defect.',
'This closes tested standard link border cases, not the entire G-RENDER gate.'],
'pages': records, 'navigation_fixture_sha256': navigation_before, 'navigation_default_link_region': navigation_comparison}
(out / 'comparison.json').write_text(json.dumps(result, indent=2) + '\n')
print(json.dumps({'source_unchanged': True, 'four_page_export_seconds': elapsed,
'standard_case_comparisons': len(common) * 4, 'navigation_default_link_region': navigation_comparison}, indent=2))
if __name__ == '__main__': main()
tests/compare_pdf_extended.py
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""Compare the original extended corpus with independent Poppler rendering.
No pixel metric is interpreted as a calibrated compatibility acceptance score.
"""
from pathlib import Path
import argparse
import hashlib
import json
import os
import subprocess
import sys
from PIL import Image, ImageChops, ImageDraw, ImageStat
ROOT = Path(__file__).resolve().parent.parent
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--build-dir", type=Path, default=ROOT / os.environ.get("DOCVIEW_BUILD_DIR", "build"))
parser.add_argument("--output", type=Path, default=ROOT / "tests/results/pdf-extended")
parser.add_argument("--pdfium-library", type=Path, default=ROOT / ".deps/pdfium/lib/libpdfium.so",
help="Selected worker library to identify in the evidence record.")
args = parser.parse_args()
BUILD = args.build_dir.resolve()
CORPUS = ROOT / "tests/fixtures/pdf/extended"
OUT = args.output.resolve()
OUT.mkdir(parents=True, exist_ok=True)
# The runtime wrapper can carry a private Fontconfig setup. Prefer an explicit
# system installation so Standard-14 fallback uses that installation's fonts.
def poppler_binary(name):
return os.environ.get("DOCVIEW_" + name.upper(), str(Path("/usr/bin") / name) if (Path("/usr/bin") / name).is_file() else name)
PDFTOPPM, PDFTOTEXT, PDFFONTS = (poppler_binary(name) for name in ("pdftoppm", "pdftotext", "pdffonts"))
manifest = json.loads((CORPUS / "manifest.json").read_text())
helper = BUILD / "pdf-worker-evidence"
if not helper.is_file():
raise SystemExit("Build the pdf-worker-evidence target before comparison.")
report = {"reference_status": "Independent Poppler comparison of synthetic source; not a human-approved golden image or full compatibility acceptance",
"platform": sys.platform, "transport": "production sandboxed PDF worker and FontBroker", "worker_sha256": hashlib.sha256((BUILD / "docview-pdf-worker").read_bytes()).hexdigest(), "broker_sha256": hashlib.sha256((BUILD / "libdocview_font_broker.a").read_bytes()).hexdigest(), "resolution_dpi": 144, "pdfium_adapter_sha256": hashlib.sha256((BUILD / "libdocview_pdf.a").read_bytes()).hexdigest(),
"pdfium_library_sha256": hashlib.sha256(args.pdfium_library.resolve().read_bytes()).hexdigest(),
"poppler_version": subprocess.run([PDFTOPPM, "-v"], text=True, capture_output=True).stderr.splitlines()[0],
"poppler_binary": PDFTOPPM, "documents": []}
report["fontconfig_matches"] = {family: subprocess.check_output(["fc-match", family], text=True).strip()
for family in ["Helvetica", "Times-Roman", "Courier"]}
overview = []
for specification in manifest["documents"]:
source = CORPUS / specification["file"]
if hashlib.sha256(source.read_bytes()).hexdigest() != specification["sha256"]:
raise SystemExit("Fixture hash differs from source manifest: " + source.name)
target = OUT / source.stem
(target / "pdfium").mkdir(parents=True, exist_ok=True)
(target / "poppler").mkdir(exist_ok=True)
subprocess.run([str(helper), str(source), str(target / "pdfium")], check=True)
subprocess.run([PDFTOPPM, "-cropbox", "-r", "144", "-png", str(source), str(target / "poppler/page")], check=True)
subprocess.run([PDFTOTEXT, "-layout", str(source), str(target / "poppler/text.txt")], check=True)
fonts = subprocess.check_output([PDFFONTS, str(source)], text=True)
(target / "fonts.txt").write_text(fonts)
doc = {"file": specification["file"], "fixture_sha256": hashlib.sha256(source.read_bytes()).hexdigest(), "pages": []}
for index in range(1, specification["page_count"] + 1):
a = Image.open(target / "pdfium" / f"page-{index}.png").convert("RGB")
b = Image.open(target / "poppler" / f"page-{index}.png").convert("RGB")
same_size = a.size == b.size
page = {"page": index, "pdfium_size": list(a.size), "poppler_size": list(b.size), "dimensions_match": same_size}
expected_size = specification["expected"].get("page_sizes_pt")
if expected_size:
page["expected_size_px"] = [v * 2 for v in expected_size[index - 1]]
elif specification["file"] == "page-geometry.pdf":
geometry = specification["expected"]["pages"][index - 1]
page["expected_size_px"] = [v * 2 for v in geometry["display_size_pt"]]
left, bottom, right, top = geometry["crop_box_pt"]
w, h = right - left, top - bottom
checks = []
for marker in geometry["markers_pdf_coordinates"]:
x, y = marker["x"] - left, marker["y"] - bottom
coordinates = {0: (x, h - y), 90: (y, x), 180: (w - x, y), 270: (h - y, w - x)}[geometry["rotation"]]
point = tuple(int(v * 2) for v in coordinates)
actual_a, actual_b = a.getpixel(point), b.getpixel(point)
expected = marker["rgb"]
checks.append({"point_px": point, "expected_rgb": expected, "pdfium_rgb": actual_a, "poppler_rgb": actual_b,
"both_match_source_color": max(abs(actual_a[k] - expected[k]) for k in range(3)) <= 2 and max(abs(actual_b[k] - expected[k]) for k in range(3)) <= 2})
page["source_geometry_marker_checks"] = checks
page["dimensions_match_source"] = list(a.size) == page.get("expected_size_px") and list(b.size) == page.get("expected_size_px")
if specification["file"] == "graphics-compositing.pdf":
probes = []
for probe in specification["expected"]["alpha_probes"]:
x, y = probe["point_pt"]
point = (x * 2, (760 - y) * 2)
actual_a, actual_b, expected = a.getpixel(point), b.getpixel(point), probe["expected_rgb"]
probes.append({"point_px": point, "region": probe["region"], "expected_rgb": expected,
"pdfium_rgb": actual_a, "poppler_rgb": actual_b,
"both_match_source_color": max(abs(actual_a[k] - expected[k]) for k in range(3)) <= 2 and max(abs(actual_b[k] - expected[k]) for k in range(3)) <= 2})
page["source_alpha_checks"] = probes
if same_size:
difference = ImageChops.difference(a, b)
difference.save(target / f"difference-{index}.png")
histogram = ImageChops.lighter(ImageChops.lighter(difference.getchannel("R"), difference.getchannel("G")), difference.getchannel("B")).histogram()
page["equal_pixels"] = histogram[0]
page["pixels_max_delta_gt_16"] = sum(histogram[17:])
page["mean_absolute_channel_error"] = sum(ImageStat.Stat(difference).mean) / 3
else:
difference = Image.new("RGB", a.size, "#ff8080")
tile = Image.new("RGB", (1200, 490), "#eeeeee"); draw = ImageDraw.Draw(tile)
for column, (title, picture) in enumerate([(source.stem + f" p{index}: PDFium", a), ("Poppler independent renderer", b), ("Absolute RGB difference", difference)]):
draw.text((column * 400 + 10, 10), title, fill="black")
preview = picture.copy(); preview.thumbnail((385, 450)); tile.paste(preview, (column * 400 + (400 - preview.width) // 2, 33))
tile.save(target / f"comparison-{index}.png")
overview.append(tile)
doc["pages"].append(page)
if specification["file"] == "japanese-layout.pdf":
doc["japanese_text_extracted_by_poppler"] = "日本語の文字配置と埋込みフォント" in (target / "poppler/text.txt").read_text()
meta = json.loads((target / "pdfium/metadata.json").read_text())
doc["japanese_search_matches_pdfium"] = len(meta.get("japaneseSearch", {}).get("matches", []))
doc["font_listing"] = fonts
report["documents"].append(doc)
sheet = Image.new("RGB", (1200, len(overview) * 490), "white")
for index, tile in enumerate(overview): sheet.paste(tile, (0, index * 490))
sheet.save(OUT / "overview.png")
(OUT / "comparison.json").write_text(json.dumps(report, indent=2, ensure_ascii=False) + "\n")
summary = {"documents": len(report["documents"]), "pages": len(overview),
"all_dimensions_match_source": all(page["dimensions_match_source"] for doc in report["documents"] for page in doc["pages"]),
"all_geometry_markers_match_source": all(check["both_match_source_color"] for doc in report["documents"] for page in doc["pages"] for check in page.get("source_geometry_marker_checks", [])),
"all_alpha_probes_match_source": all(check["both_match_source_color"] for doc in report["documents"] for page in doc["pages"] for check in page.get("source_alpha_checks", [])),
"japanese_text_and_search": all(doc.get("japanese_text_extracted_by_poppler", True) and doc.get("japanese_search_matches_pdfium", 1) > 0 for doc in report["documents"]),
"report": str(OUT / "comparison.json")}
print(json.dumps(summary, indent=2))
if not all(summary[key] for key in ["all_dimensions_match_source", "all_geometry_markers_match_source", "all_alpha_probes_match_source", "japanese_text_and_search"]):
raise SystemExit("A deterministic source assertion failed; see comparison.json")
tests/compare_pdf_fonts.py
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env python3
"""Production worker vs independent Poppler for unembedded Japanese CID H/V."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import subprocess
from PIL import Image, ImageChops, ImageDraw, ImageStat
ROOT = Path(__file__).resolve().parent.parent
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--build-dir", type=Path, default=ROOT / os.environ.get("DOCVIEW_BUILD_DIR", "build"))
parser.add_argument("--output", type=Path, default=ROOT / "tests/results/pdf-fonts")
args = parser.parse_args()
BUILD = args.build_dir.resolve()
CORPUS = ROOT / "tests/fixtures/pdf/fonts"
OUT = args.output.resolve()
OUT.mkdir(parents=True, exist_ok=True)
specification = json.loads((CORPUS / "manifest.json").read_text())["documents"][0]
source = CORPUS / specification["file"]
assert hashlib.sha256(source.read_bytes()).hexdigest() == specification["sha256"]
subprocess.run([str(BUILD / "pdf-worker-evidence"), str(source), str(OUT / "worker")], check=True)
poppler = os.environ.get("DOCVIEW_PDFTOPPM", "/usr/bin/pdftoppm")
subprocess.run([poppler, "-r", "144", "-png", str(source), str(OUT / "poppler")], check=True)
subprocess.run(["/usr/bin/pdftotext", "-layout", str(source), str(OUT / "poppler-text.txt")], check=True)
fonts = subprocess.check_output(["/usr/bin/pdffonts", str(source)], text=True)
(OUT / "font-list.txt").write_text(fonts)
metadata = json.loads((OUT / "worker/metadata.json").read_text())
report = {
"reference_status": "Independent renderer comparison; no human-approved golden image or comprehensive CID/CMap acceptance",
"fixture_sha256": specification["sha256"],
"worker_sha256": hashlib.sha256((BUILD / "docview-pdf-worker").read_bytes()).hexdigest(),
"broker_sha256": hashlib.sha256((BUILD / "libdocview_font_broker.a").read_bytes()).hexdigest(),
"transport": metadata["transport"],
"poppler_version": subprocess.run([poppler, "-v"], text=True, capture_output=True).stderr.splitlines()[0],
"font_selections": metadata["fontSelections"],
"font_bytes_transferred": metadata["fontBytesTransferred"],
"japanese_search_matches_first_page": len(metadata.get("japaneseSearch", {}).get("matches", [])),
"pages": [],
}
for page in (1, 2):
a = Image.open(OUT / "worker" / f"page-{page}.png").convert("RGB")
b = Image.open(OUT / f"poppler-{page}.png").convert("RGB")
assert a.size == b.size == (1200, 1520)
difference = ImageChops.difference(a, b)
difference.save(OUT / f"difference-{page}.png")
histogram = ImageChops.lighter(ImageChops.lighter(difference.getchannel("R"), difference.getchannel("G")), difference.getchannel("B")).histogram()
report["pages"].append({"page": page, "size": list(a.size), "dimensions_match_source": True,
"equal_pixels": histogram[0], "pixels_max_delta_gt_16": sum(histogram[17:]),
"mean_absolute_channel_error": sum(ImageStat.Stat(difference).mean) / 3})
comparison = Image.new("RGB", (1200, 550), "#eeeeee")
draw = ImageDraw.Draw(comparison)
for col, (title, picture) in enumerate((("Sandboxed DocView / PDFium", a), ("Independent Poppler", b), ("Absolute RGB difference", difference))):
draw.text((col * 400 + 8, 10), title, fill="black")
picture.thumbnail((390, 500)); comparison.paste(picture, (col * 400 + 5, 35))
comparison.save(OUT / f"comparison-{page}.png")
assert report["font_bytes_transferred"] > 0
assert any(item.get("charset") == 128 and item.get("found") for item in report["font_selections"])
assert report["japanese_search_matches_first_page"] == 2
(OUT / "comparison.json").write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n")
print(json.dumps(report, ensure_ascii=False, indent=2))
tests/compare_pdf_icc.py
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env python3
"""Compare a real ICCBased PDF through the sandboxed worker at three scales."""
from pathlib import Path
from io import BytesIO
import argparse
import hashlib
import json
import math
import os
import subprocess
from PIL import Image, ImageChops, ImageDraw, ImageStat, ImageCms
ROOT=Path(__file__).resolve().parent.parent
parser=argparse.ArgumentParser(description=__doc__)
parser.add_argument("--build-dir",type=Path,default=ROOT/os.environ.get("DOCVIEW_BUILD_DIR","build"))
parser.add_argument("--output",type=Path,default=ROOT/"tests/results/pdf-icc")
args=parser.parse_args()
from pypdf import PdfReader
BUILD=args.build_dir.resolve()
CORPUS=ROOT/"tests/fixtures/pdf/extended"
OUT=args.output.resolve(); OUT.mkdir(parents=True,exist_ok=True)
manifest=json.loads((CORPUS/"manifest.json").read_text())
spec=next(item for item in manifest["documents"] if item["file"]=="icc-linear-rgb.pdf")
source=CORPUS/spec["file"]; before=hashlib.sha256(source.read_bytes()).hexdigest()
assert before==spec["sha256"]
profile=(CORPUS/"linear-srgb.icc").read_bytes()
assert hashlib.sha256(profile).hexdigest()==manifest["color_profile"]["sha256"]
reader=PdfReader(source)
color_space=reader.pages[0]["/Resources"]["/ColorSpace"]["/Linear"]
assert str(color_space[0])=="/ICCBased"
assert color_space[1].get_object().get_data()==profile
image=reader.pages[0]["/Resources"]["/XObject"]["/Sample"].get_object()
assert str(image["/ColorSpace"][0])=="/ICCBased"
assert image["/ColorSpace"][1].get_object().get_data()==profile
cms_profile=ImageCms.ImageCmsProfile(BytesIO(profile))
report={"scope":"Self-authored ICCBased RGB matrix/TRC vector and raster at actual worker render scales; not full ICC/CMYK/printing acceptance",
"fixture_sha256_before":before,"profile":manifest["color_profile"],
"worker_sha256":hashlib.sha256((BUILD/"docview-pdf-worker").read_bytes()).hexdigest(),
"profile_copyright_readback":ImageCms.getProfileCopyright(cms_profile).strip(),
"profile_description_readback":ImageCms.getProfileDescription(cms_profile).strip(),
"profile_embedded_in_vector_and_image":True,"transport":"production sandboxed PDFWorker, WorkerProcess, FontBroker, tile IPC",
"poppler_version":subprocess.run(["/usr/bin/pdftoppm","-v"],capture_output=True,text=True).stderr.splitlines()[0],
"sha256":{str(path.relative_to(ROOT)):hashlib.sha256(path.read_bytes()).hexdigest() for path in
[BUILD/"pdf-worker-evidence",BUILD/"docview-pdf-worker",BUILD/"libdocview_pdf.a",BUILD/"libdocview_font_broker.a",ROOT/"tests/render_pdf_worker.cpp",ROOT/"tests/compare_pdf_icc.py",ROOT/"tests/fixtures/pdf/icc_generator.py"]},
"reference":"Analytic sRGB transfer function of stated linear RGB values; independent Poppler raster; no human-approved golden",
"scales":[]}
overview=[]; success=True
for scale in [.5,1.,4.]:
folder=OUT/("scale-"+str(scale).replace(".","_")); (folder/"pdfium").mkdir(parents=True,exist_ok=True)
subprocess.run([str(BUILD/"pdf-worker-evidence"),str(source),str(folder/"pdfium"),str(scale)],check=True)
subprocess.run(["/usr/bin/pdftoppm","-singlefile","-cropbox","-r",str(72*scale),"-png",str(source),str(folder/"poppler")],check=True)
a=Image.open(folder/"pdfium/page-1.png").convert("RGB"); b=Image.open(folder/"poppler.png").convert("RGB")
meta=json.loads((folder/"pdfium/metadata.json").read_text()); assert meta["renderScale"]==scale
size=[math.ceil(v*scale) for v in spec["expected"]["page_sizes_pt"][0]]
entry={"scale":scale,"dpi":72*scale,"expected_size":size,"pdfium_size":list(a.size),"poppler_size":list(b.size),
"dimensions_match_source":list(a.size)==size==list(b.size),"probes":[]}
success=success and entry["dimensions_match_source"]
for probe in spec["expected"]["icc_probes"]:
x,y=probe["point_pt"]; point=(round(x*scale),round((350-y)*scale))
actual_a,actual_b=a.getpixel(point),b.getpixel(point)
expected=probe["expected_rgb"]; tolerance=spec["expected"]["probe_tolerance_rgb"]
check=dict(probe,point_px=point,pdfium_rgb=actual_a,poppler_rgb=actual_b,
both_match_analytic_color=all(abs(actual[k]-expected[k])<=tolerance for actual in (actual_a,actual_b) for k in range(3)))
entry["probes"].append(check);success=success and check["both_match_analytic_color"]
diff=ImageChops.difference(a,b);diff.save(folder/"difference.png")
entry["mean_absolute_channel_error"]=sum(ImageStat.Stat(diff).mean)/3
entry["icc_midgray_differs_from_device_control"]=all(abs(entry["probes"][6][which][0]-entry["probes"][7][which][0])>40 for which in ("pdfium_rgb","poppler_rgb"))
success=success and entry["icc_midgray_differs_from_device_control"]
sheet=Image.new("RGB",(1500,350),"#ededed");draw=ImageDraw.Draw(sheet)
for col,(name,img) in enumerate([("PDFium / production worker",a),("Poppler / independent",b),("Absolute RGB difference",diff)]):
draw.text((col*500+10,12),f"{scale:g}x {name}",fill="black");thumb=img.copy();thumb.thumbnail((488,300))
sheet.paste(thumb,(col*500+(500-thumb.width)//2,42))
sheet.save(folder/"comparison.png");overview.append(sheet);report["scales"].append(entry)
report["fixture_sha256_after"]=hashlib.sha256(source.read_bytes()).hexdigest()
report["original_unchanged"]=report["fixture_sha256_after"]==before;success=success and report["original_unchanged"]
report["all_source_assertions_pass"]=success
canvas=Image.new("RGB",(1500,len(overview)*350),"white")
for i,sheet in enumerate(overview):canvas.paste(sheet,(0,i*350))
canvas.save(OUT/"overview.png")
(OUT/"comparison.json").write_text(json.dumps(report,ensure_ascii=False,indent=2)+"\n")
print(json.dumps({"all_source_assertions_pass":success,"scales":len(report["scales"]),"probe_count":sum(len(x["probes"]) for x in report["scales"]),"original_unchanged":report["original_unchanged"],"profile_copyright":report["profile_copyright_readback"]},indent=2))
if not success: raise SystemExit(f"ICC source assertion failed; see {OUT / 'comparison.json'}")
tests/compare_pdf_renderers.py
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""Generate independent PDFium/Poppler comparison evidence.
Requires the built pdf-worker-evidence target, Pillow and Poppler utils.
The result is comparison evidence; it is not a human-approved golden image.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
import shutil
import subprocess
from PIL import Image, ImageChops, ImageDraw
root = Path(__file__).resolve().parent.parent
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--build-dir', type=Path, default=root / os.environ.get('DOCVIEW_BUILD_DIR', 'build'))
parser.add_argument('--output', type=Path, default=root / 'tests/results/pdf')
args = parser.parse_args()
build = args.build_dir.resolve()
out = args.output.resolve()
out.mkdir(parents=True, exist_ok=True)
helper = build / 'pdf-worker-evidence'
if not helper.is_file(): raise SystemExit('Build the pdf-worker-evidence target before comparison.')
fixture = root / 'tests/fixtures/pdf/navigation.pdf'
subprocess.run([str(helper), str(fixture), str(out / 'worker')], check=True)
shutil.copyfile(out / 'worker/page-1.png', out / 'pdfium.png')
poppler = '/usr/bin/pdftoppm' if Path('/usr/bin/pdftoppm').exists() else shutil.which('pdftoppm')
if not poppler: raise SystemExit('Poppler pdftoppm is required for the independent comparison.')
subprocess.run([poppler, '-cropbox', '-f', '1', '-singlefile', '-r', '144', '-png', str(fixture), str(out / 'poppler')], check=True)
a = Image.open(out / 'pdfium.png').convert('RGB')
b = Image.open(out / 'poppler.png').convert('RGB')
if a.size != b.size: raise SystemExit('Renderer dimensions disagree.')
diff = ImageChops.difference(a, b)
diff.save(out / 'difference.png')
pixels = list(diff.getdata()); count = len(pixels)
metrics = {'transport': 'production sandboxed PDF worker and FontBroker', 'worker_sha256': hashlib.sha256((build / 'docview-pdf-worker').read_bytes()).hexdigest(), 'broker_sha256': hashlib.sha256((build / 'libdocview_font_broker.a').read_bytes()).hexdigest(), 'fixture_sha256': hashlib.sha256(fixture.read_bytes()).hexdigest(), 'size': list(a.size), 'pixels': count,
'equal_pixels': sum(max(p) == 0 for p in pixels), 'pixels_max_delta_gt_16': sum(max(p) > 16 for p in pixels),
'mean_absolute_channel_error': sum(sum(p) for p in pixels) / (count * 3),
'reference_status': 'independent Poppler comparison, not human-approved golden image'}
(out / 'comparison.json').write_text(json.dumps(metrics, indent=2) + '\n')
canvas = Image.new('RGB', (1200, 430), '#eeeeee'); draw = ImageDraw.Draw(canvas)
for x, label, image in [(0, 'PDFium 155.0.8057.0', a), (400, 'Poppler independent renderer', b), (800, 'Absolute RGB difference', diff)]:
draw.text((x + 10, 9), label, fill='black'); canvas.paste(image, (x, 30))
canvas.save(out / 'comparison.png')
print(json.dumps(metrics, indent=2))
tests/fixtures/archive-ratio/generate.py
Vendored
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env python3
"""Deterministic authored ZIP inputs for the bounded-input expansion guard."""
import hashlib
import json
from pathlib import Path
import struct
import zlib
ROOT = Path(__file__).resolve().parent
def build(output_size: int, padded_size: int | None = None) -> tuple[bytes, dict]:
plain = b"x" * output_size
compressor = zlib.compressobj(9, zlib.DEFLATED, -15)
compressed = compressor.compress(plain) + compressor.flush()
payload = compressed if padded_size is None else compressed.ljust(padded_size, b"\0")
name = b"index.html"
crc = zlib.crc32(plain)
local = struct.pack("<IHHHHHIIIHH", 0x04034B50, 20, 0x800, 8, 0, 33,
crc, len(payload), len(plain), len(name), 0)
central = struct.pack("<IHHHHHHIIIHHHHHII", 0x02014B50, (3 << 8) | 20,
20, 0x800, 8, 0, 33, crc, len(payload), len(plain),
len(name), 0, 0, 0, 0, 0o100600 << 16, 0) + name
offset = len(local) + len(name) + len(payload)
end = struct.pack("<IHHHHIIH", 0x06054B50, 0, 0, 1, 1, len(central), offset, 0)
archive = local + name + payload + central + end
return archive, {
"output_bytes": len(plain), "deflate_stream_bytes": len(compressed),
"zip_declared_compressed_bytes": len(payload),
"sha256": hashlib.sha256(archive).hexdigest(),
"output_sha256": hashlib.sha256(plain).hexdigest(),
}
def main() -> None:
manifest = {"generator": "generate.py; Python stdlib zlib, raw DEFLATE",
"content": "Authored repeated ASCII x; no external material or fonts.", "files": {}}
for name, size, padded, expectation in [
("padded-33mib.zip", 33 * 1024 * 1024, 256 * 1024, "open succeeds; extraction E_ARCHIVE_LIMIT before more than 32 MiB is sent"),
("threshold-32mib.zip", 32 * 1024 * 1024, None, "open and extraction succeed; ratio threshold is strictly greater than 32 MiB"),
("unpadded-33mib.zip", 33 * 1024 * 1024, None, "open E_ARCHIVE_LIMIT from metadata preflight"),
]:
archive, info = build(size, padded)
(ROOT / name).write_bytes(archive)
manifest["files"][name] = info | {"expected": expectation}
(ROOT / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
if __name__ == "__main__":
main()
tests/fixtures/archive-ratio/manifest.json
Vendored
+30
View File
@@ -0,0 +1,30 @@
{
"generator": "generate.py; Python stdlib zlib, raw DEFLATE",
"content": "Authored repeated ASCII x; no external material or fonts.",
"files": {
"padded-33mib.zip": {
"output_bytes": 34603008,
"deflate_stream_bytes": 33647,
"zip_declared_compressed_bytes": 262144,
"sha256": "fd84b25c229191d4efb24939c14a20506ea442e4becf2164d034a4bdc983291a",
"output_sha256": "7d641a2a7b1c449f586b444bd47a2005d83be0e142596f6d1a52da9c2256c96b",
"expected": "open succeeds; extraction E_ARCHIVE_LIMIT before more than 32 MiB is sent"
},
"threshold-32mib.zip": {
"output_bytes": 33554432,
"deflate_stream_bytes": 32617,
"zip_declared_compressed_bytes": 32617,
"sha256": "2f49d2152f9d10daad3109eecb532bd3e2c720f15708f1c1338c14785b7d7ef1",
"output_sha256": "05f052c8f6da8ee5228ec291820b559c4be183773b9e97a6b82e30dacff85dd3",
"expected": "open and extraction succeed; ratio threshold is strictly greater than 32 MiB"
},
"unpadded-33mib.zip": {
"output_bytes": 34603008,
"deflate_stream_bytes": 33647,
"zip_declared_compressed_bytes": 33647,
"sha256": "205c8ef001b932dc20cc0672bb515f72420202c5a147de7a6b8496439745ea5c",
"output_sha256": "7d641a2a7b1c449f586b444bd47a2005d83be0e142596f6d1a52da9c2256c96b",
"expected": "open E_ARCHIVE_LIMIT from metadata preflight"
}
}
}
tests/fixtures/archive-ratio/padded-33mib.zip
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/archive-ratio/threshold-32mib.zip
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/archive-ratio/unpadded-33mib.zip
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/pdf/annotation-flags/flags.pdf
Vendored
+348
View File
@@ -0,0 +1,348 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 38 /Kids [6 0 R 8 0 R 10 0 R 12 0 R 14 0 R 16 0 R 18 0 R 20 0 R 22 0 R 24 0 R 26 0 R 28 0 R 30 0 R 32 0 R 34 0 R 36 0 R 38 0 R 40 0 R 42 0 R 44 0 R 46 0 R 48 0 R 50 0 R 52 0 R 54 0 R 56 0 R 58 0 R 60 0 R 62 0 R 64 0 R 66 0 R 68 0 R 70 0 R 72 0 R 74 0 R 76 0 R 81 0 R 83 0 R] >>
endobj
3 0 obj
<< /Length 63 /Type /XObject /Subtype /Form /BBox [0 0 32 16] /Matrix [2 0 0 3 7 11] /Resources << /ExtGState << /Opaque << /Type /ExtGState /ca 1 /CA 1 >> >> >> >>
stream
q /Opaque gs 1 0 0 rg 0 0 16 16 re f 0 0 1 rg 16 0 16 16 re f Q
endstream
endobj
4 0 obj
<< /Length 23 /Type /XObject /Subtype /Form /BBox [0 0 32 16] /Resources << >> >>
stream
0 1 0 rg 0 0 32 16 re f
endstream
endobj
5 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 8 /Contents (generated-link flags 8) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
6 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [5 0 R] >>
endobj
7 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 8 /Contents (generated-link flags 8) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
8 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [7 0 R] >>
endobj
9 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 8 /Contents (generated-link flags 8) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
10 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [9 0 R] >>
endobj
11 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 8 /Contents (generated-link flags 8) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
12 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [11 0 R] >>
endobj
13 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 16 /Contents (generated-link flags 16) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
14 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [13 0 R] >>
endobj
15 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 16 /Contents (generated-link flags 16) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
16 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [15 0 R] >>
endobj
17 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 16 /Contents (generated-link flags 16) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
18 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [17 0 R] >>
endobj
19 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 16 /Contents (generated-link flags 16) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
20 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [19 0 R] >>
endobj
21 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /Contents (generated-link flags 24) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
22 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [21 0 R] >>
endobj
23 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /Contents (generated-link flags 24) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
24 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [23 0 R] >>
endobj
25 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /Contents (generated-link flags 24) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
26 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [25 0 R] >>
endobj
27 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /Contents (generated-link flags 24) /A << /S /URI /URI (https://example.org/flags) >> /C [1 0 0] /BS << /W 2 /S /S >> >>
endobj
28 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [27 0 R] >>
endobj
29 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 8 /Contents (stored-link flags 8) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
30 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [29 0 R] >>
endobj
31 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 8 /Contents (stored-link flags 8) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
32 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [31 0 R] >>
endobj
33 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 8 /Contents (stored-link flags 8) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
34 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [33 0 R] >>
endobj
35 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 8 /Contents (stored-link flags 8) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
36 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [35 0 R] >>
endobj
37 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 16 /Contents (stored-link flags 16) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
38 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [37 0 R] >>
endobj
39 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 16 /Contents (stored-link flags 16) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
40 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [39 0 R] >>
endobj
41 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 16 /Contents (stored-link flags 16) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
42 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [41 0 R] >>
endobj
43 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 16 /Contents (stored-link flags 16) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
44 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [43 0 R] >>
endobj
45 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /Contents (stored-link flags 24) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
46 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [45 0 R] >>
endobj
47 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /Contents (stored-link flags 24) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
48 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [47 0 R] >>
endobj
49 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /Contents (stored-link flags 24) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
50 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [49 0 R] >>
endobj
51 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /Contents (stored-link flags 24) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
52 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [51 0 R] >>
endobj
53 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 8 /Contents (stored-comment flags 8) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
54 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [53 0 R] >>
endobj
55 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 8 /Contents (stored-comment flags 8) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
56 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [55 0 R] >>
endobj
57 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 8 /Contents (stored-comment flags 8) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
58 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [57 0 R] >>
endobj
59 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 8 /Contents (stored-comment flags 8) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
60 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [59 0 R] >>
endobj
61 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 16 /Contents (stored-comment flags 16) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
62 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [61 0 R] >>
endobj
63 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 16 /Contents (stored-comment flags 16) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
64 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [63 0 R] >>
endobj
65 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 16 /Contents (stored-comment flags 16) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
66 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [65 0 R] >>
endobj
67 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 16 /Contents (stored-comment flags 16) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
68 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [67 0 R] >>
endobj
69 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 24 /Contents (stored-comment flags 24) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
70 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [69 0 R] >>
endobj
71 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 24 /Contents (stored-comment flags 24) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
72 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << >> /Annots [71 0 R] >>
endobj
73 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 24 /Contents (stored-comment flags 24) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
74 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << >> /Annots [73 0 R] >>
endobj
75 0 obj
<< /Type /Annot /Subtype /Stamp /Rect [42 48 74 64] /F 24 /Contents (stored-comment flags 24) /A << /S /URI /URI (https://example.org/flags) >> /AP << /N << /On 3 0 R /Off 4 0 R >> >> /AS /On >>
endobj
76 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << >> /Annots [75 0 R] >>
endobj
77 0 obj
<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /C [1 0 0] /BS << /W 2 >> >>
endobj
78 0 obj
<< /Type /Annot /Subtype /Link /Rect [41 47 75 65] /F 24 /AP << /N 4 0 R >> >>
endobj
79 0 obj
<< /Type /Annot /Subtype /Link /Rect [20 80 36 96] /F 26 /Contents (Invisible comment) /AP << /N 3 0 R >> >>
endobj
80 0 obj
<< /Type /Annot /Subtype /Link /Rect [80 80 96 96] /F 56 /Contents (Invisible comment) /AP << /N 3 0 R >> >>
endobj
81 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [77 0 R 78 0 R 79 0 R 80 0 R] >>
endobj
82 0 obj
<< /Type /Annot /Subtype /Link /Rect [108 48 140 64] /F 24 /AP << /N 3 0 R >> >>
endobj
83 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << >> /Annots [82 0 R] >>
endobj
xref
0 84
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000380 00000 n
0000000641 00000 n
0000000779 00000 n
0000000970 00000 n
0000001108 00000 n
0000001299 00000 n
0000001438 00000 n
0000001629 00000 n
0000001770 00000 n
0000001962 00000 n
0000002104 00000 n
0000002298 00000 n
0000002438 00000 n
0000002632 00000 n
0000002773 00000 n
0000002967 00000 n
0000003109 00000 n
0000003303 00000 n
0000003445 00000 n
0000003639 00000 n
0000003779 00000 n
0000003973 00000 n
0000004114 00000 n
0000004308 00000 n
0000004450 00000 n
0000004644 00000 n
0000004786 00000 n
0000004991 00000 n
0000005131 00000 n
0000005336 00000 n
0000005477 00000 n
0000005682 00000 n
0000005824 00000 n
0000006029 00000 n
0000006171 00000 n
0000006378 00000 n
0000006518 00000 n
0000006725 00000 n
0000006866 00000 n
0000007073 00000 n
0000007215 00000 n
0000007422 00000 n
0000007564 00000 n
0000007771 00000 n
0000007911 00000 n
0000008118 00000 n
0000008259 00000 n
0000008466 00000 n
0000008608 00000 n
0000008815 00000 n
0000008957 00000 n
0000009166 00000 n
0000009306 00000 n
0000009515 00000 n
0000009656 00000 n
0000009865 00000 n
0000010007 00000 n
0000010216 00000 n
0000010358 00000 n
0000010569 00000 n
0000010709 00000 n
0000010920 00000 n
0000011061 00000 n
0000011272 00000 n
0000011414 00000 n
0000011625 00000 n
0000011767 00000 n
0000011978 00000 n
0000012118 00000 n
0000012329 00000 n
0000012470 00000 n
0000012681 00000 n
0000012823 00000 n
0000013034 00000 n
0000013176 00000 n
0000013278 00000 n
0000013373 00000 n
0000013498 00000 n
0000013623 00000 n
0000013784 00000 n
0000013881 00000 n
trailer
<< /Size 84 /Root 1 0 R >>
startxref
14021
%%EOF
tests/fixtures/pdf/annotation-flags/manifest.json
Vendored
+464
View File
@@ -0,0 +1,464 @@
{
"generator": "generate_annotation_flags.py",
"license": "Self-created test content; same license as DocView.",
"sha256": "ec4a93fab82fd79efa2c8d9de5a83ad64350d7d02c061638ceda06007e1b8d4c",
"cropBox": [
8,
12,
120,
116
],
"cases": [
{
"page": 0,
"intrinsicRotation": 0,
"kind": "generated-link",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 1,
"intrinsicRotation": 90,
"kind": "generated-link",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 2,
"intrinsicRotation": 180,
"kind": "generated-link",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 3,
"intrinsicRotation": 270,
"kind": "generated-link",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 4,
"intrinsicRotation": 0,
"kind": "generated-link",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 5,
"intrinsicRotation": 90,
"kind": "generated-link",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 6,
"intrinsicRotation": 180,
"kind": "generated-link",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 7,
"intrinsicRotation": 270,
"kind": "generated-link",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 8,
"intrinsicRotation": 0,
"kind": "generated-link",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 9,
"intrinsicRotation": 90,
"kind": "generated-link",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 10,
"intrinsicRotation": 180,
"kind": "generated-link",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 11,
"intrinsicRotation": 270,
"kind": "generated-link",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 12,
"intrinsicRotation": 0,
"kind": "stored-link",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 13,
"intrinsicRotation": 90,
"kind": "stored-link",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 14,
"intrinsicRotation": 180,
"kind": "stored-link",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 15,
"intrinsicRotation": 270,
"kind": "stored-link",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 16,
"intrinsicRotation": 0,
"kind": "stored-link",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 17,
"intrinsicRotation": 90,
"kind": "stored-link",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 18,
"intrinsicRotation": 180,
"kind": "stored-link",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 19,
"intrinsicRotation": 270,
"kind": "stored-link",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 20,
"intrinsicRotation": 0,
"kind": "stored-link",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 21,
"intrinsicRotation": 90,
"kind": "stored-link",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 22,
"intrinsicRotation": 180,
"kind": "stored-link",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 23,
"intrinsicRotation": 270,
"kind": "stored-link",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 24,
"intrinsicRotation": 0,
"kind": "stored-comment",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 25,
"intrinsicRotation": 90,
"kind": "stored-comment",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 26,
"intrinsicRotation": 180,
"kind": "stored-comment",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 27,
"intrinsicRotation": 270,
"kind": "stored-comment",
"flags": 8,
"rect": [
42,
48,
74,
64
]
},
{
"page": 28,
"intrinsicRotation": 0,
"kind": "stored-comment",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 29,
"intrinsicRotation": 90,
"kind": "stored-comment",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 30,
"intrinsicRotation": 180,
"kind": "stored-comment",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 31,
"intrinsicRotation": 270,
"kind": "stored-comment",
"flags": 16,
"rect": [
42,
48,
74,
64
]
},
{
"page": 32,
"intrinsicRotation": 0,
"kind": "stored-comment",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 33,
"intrinsicRotation": 90,
"kind": "stored-comment",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 34,
"intrinsicRotation": 180,
"kind": "stored-comment",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 35,
"intrinsicRotation": 270,
"kind": "stored-comment",
"flags": 24,
"rect": [
42,
48,
74,
64
]
},
{
"page": 36,
"intrinsicRotation": 0,
"kind": "visibility-zorder",
"flags": 24
},
{
"page": 37,
"intrinsicRotation": 0,
"kind": "crop-clipping",
"flags": 24,
"rect": [
108,
48,
140,
64
]
}
],
"expected": "At 100% one PDF point is one logical pixel. NoZoom preserves logical 32x16 size across zoom and scales only with DPR. NoRotate preserves upright red-left/blue-right appearance around the original Rect upper-left through intrinsic plus user rotation. Generated red border extends one point beyond Rect. This is synthetic geometry, not a general PDF compatibility golden."
}
tests/fixtures/pdf/annotation-flags/text-icons-manifest.json
Vendored
+17
View File
@@ -0,0 +1,17 @@
{
"license": "Self-created test content; same license as DocView.",
"sha256": "44879caa880ad8c892e2f0cdca883fe5b8e471a92aecabb5583fb78d3fd34d86",
"originalRect": [
42,
48,
74,
64
],
"nativeIconRect": [
42,
48,
62,
68
],
"expected": "Twelve pages: flags 8/16/24, each with intrinsic rotations 0/90/180/270. Four pages share each annotation. Capture the native generated AP once, retain source Rect, use the captured 20x20 icon Rect for drawing and hit testing."
}
tests/fixtures/pdf/annotation-flags/text-icons.pdf
Vendored
+88
View File
@@ -0,0 +1,88 @@
%PDF-1.7
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 12 /Kids [6 0 R 7 0 R 8 0 R 9 0 R 11 0 R 12 0 R 13 0 R 14 0 R 16 0 R 17 0 R 18 0 R 19 0 R] >>
endobj
3 0 obj
<< /Length 68 >>
stream
BT /F1 10 Tf 12 100 Td (Native icon with unembedded body text) Tj ET
endstream
endobj
4 0 obj
<< /Type /Annot /Subtype /Ink /Rect [80 24 100 34] /F 8 /Contents (Native ink) /BS << /W 4 >> /InkList [[80 24 90 34 100 24]] >>
endobj
5 0 obj
<< /Type /Annot /Subtype /Text /Rect [42 48 74 64] /F 8 /Contents (Native text icon) /C [1 0 0] >>
endobj
6 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [5 0 R 4 0 R 4 0 R] >>
endobj
7 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [5 0 R 4 0 R 4 0 R] >>
endobj
8 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [5 0 R 4 0 R 4 0 R] >>
endobj
9 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [5 0 R 4 0 R 4 0 R] >>
endobj
10 0 obj
<< /Type /Annot /Subtype /Text /Rect [42 48 74 64] /F 16 /Contents (Native text icon) /C [1 0 0] >>
endobj
11 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [10 0 R 4 0 R 4 0 R] >>
endobj
12 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [10 0 R 4 0 R 4 0 R] >>
endobj
13 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [10 0 R 4 0 R 4 0 R] >>
endobj
14 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [10 0 R 4 0 R 4 0 R] >>
endobj
15 0 obj
<< /Type /Annot /Subtype /Text /Rect [42 48 74 64] /F 24 /Contents (Native text icon) /C [1 0 0] >>
endobj
16 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 0 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [15 0 R 4 0 R 4 0 R] >>
endobj
17 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 90 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [15 0 R 4 0 R 4 0 R] >>
endobj
18 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 180 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [15 0 R 4 0 R 4 0 R] >>
endobj
19 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate 270 /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [15 0 R 4 0 R 4 0 R] >>
endobj
xref
0 20
0000000000 65535 f
0000000009 00000 n
0000000058 00000 n
0000000190 00000 n
0000000308 00000 n
0000000452 00000 n
0000000566 00000 n
0000000805 00000 n
0000001045 00000 n
0000001286 00000 n
0000001527 00000 n
0000001643 00000 n
0000001884 00000 n
0000002126 00000 n
0000002369 00000 n
0000002612 00000 n
0000002728 00000 n
0000002969 00000 n
0000003211 00000 n
0000003454 00000 n
trailer
<< /Size 20 /Root 1 0 R >>
startxref
3697
%%EOF
tests/fixtures/pdf/cmyk-lut/README.md
Vendored
+5
View File
@@ -0,0 +1,5 @@
# CMYK ICC CLUT試験資料
`icc-cmyk-lut.pdf`は、同じCMYK入力をICCベクター・ICC画像・DeviceCMYK対照で表示する生成資料である。自作ICC入力プロファイルのCLUT頂点と数式から色の期待値を定める。制作条件・数値・既知の差は[描画比較記録](../../../results/cmyk-lut/README.md)を参照する。
`manifest.json`はPDFとICCのSHA-256、16個のICC色測定点、8個の対照点、生成器hash、測定対象外の範囲を保持する。物理的な印刷条件やモニターを校正するためのプロファイルではない。PDFiumと期待色の差が見つかっており、正解画像の人による承認は未完了である。
tests/fixtures/pdf/cmyk-lut/icc-cmyk-lut.pdf
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/pdf/cmyk-lut/manifest.json
Vendored
+535
View File
@@ -0,0 +1,535 @@
{
"file": "icc-cmyk-lut.pdf",
"sha256": "3cc5ea798bc61b0d1af2fc1ee256bedc961ca40b05ad2002a9fee6a9227330fe",
"profileFile": "synthetic-cmyk-lab.icc",
"profileSha256": "1708392182ecb3819bb0e4bd1139677be24c9884fff84ad3bb91af5dd42b78f7",
"profileBytes": 588,
"pageSizePt": [
720,
420
],
"profile": {
"version": "2.1",
"class": "scnr",
"input": "CMYK",
"pcs": "Lab ",
"tag": "A2B0",
"type": "mft2",
"gridPointsPerChannel": 2,
"mapping": [
"L=100-12*C-15*M-10*Y-35*K",
"a=18*M-15*C",
"b=20*Y-12*C"
],
"labEncoding": "ICC v2 L: 0..65280 for 0..100; a/b: (value+128)*256",
"source": "Self-authored binary ICC input profile; no borrowed ICC profile"
},
"probes": [
{
"kind": "icc-vector",
"sample": 0,
"pointPt": [
71,
292
],
"components": [
0,
0,
0,
0
],
"lab": [
100,
0,
0
],
"expectedRgb": [
255,
255,
255
]
},
{
"kind": "icc-image",
"sample": 0,
"pointPt": [
76.5,
194
],
"components": [
0.0,
0.0,
0.0,
0.0
],
"lab": [
100.0,
0.0,
0.0
],
"expectedRgb": [
255,
255,
255
]
},
{
"kind": "device-control",
"sample": 0,
"pointPt": [
71,
96
],
"components": [
0,
0,
0,
0
]
},
{
"kind": "icc-vector",
"sample": 1,
"pointPt": [
152,
292
],
"components": [
1,
0,
0,
0
],
"lab": [
88,
-15,
-12
],
"expectedRgb": [
178,
230,
243
]
},
{
"kind": "icc-image",
"sample": 1,
"pointPt": [
157.5,
194
],
"components": [
1.0,
0.0,
0.0,
0.0
],
"lab": [
88.0,
-15.0,
-12.0
],
"expectedRgb": [
178,
230,
243
]
},
{
"kind": "device-control",
"sample": 1,
"pointPt": [
152,
96
],
"components": [
1,
0,
0,
0
]
},
{
"kind": "icc-vector",
"sample": 2,
"pointPt": [
233,
292
],
"components": [
0,
1,
0,
0
],
"lab": [
85,
18,
0
],
"expectedRgb": [
246,
200,
213
]
},
{
"kind": "icc-image",
"sample": 2,
"pointPt": [
238.5,
194
],
"components": [
0.0,
1.0,
0.0,
0.0
],
"lab": [
85.0,
18.0,
0.0
],
"expectedRgb": [
246,
200,
213
]
},
{
"kind": "device-control",
"sample": 2,
"pointPt": [
233,
96
],
"components": [
0,
1,
0,
0
]
},
{
"kind": "icc-vector",
"sample": 3,
"pointPt": [
314,
292
],
"components": [
0,
0,
1,
0
],
"lab": [
90,
0,
20
],
"expectedRgb": [
238,
226,
188
]
},
{
"kind": "icc-image",
"sample": 3,
"pointPt": [
319.5,
194
],
"components": [
0.0,
0.0,
1.0,
0.0
],
"lab": [
90.0,
0.0,
20.0
],
"expectedRgb": [
238,
226,
188
]
},
{
"kind": "device-control",
"sample": 3,
"pointPt": [
314,
96
],
"components": [
0,
0,
1,
0
]
},
{
"kind": "icc-vector",
"sample": 4,
"pointPt": [
395,
292
],
"components": [
0,
0,
0,
1
],
"lab": [
65,
0,
0
],
"expectedRgb": [
158,
158,
158
]
},
{
"kind": "icc-image",
"sample": 4,
"pointPt": [
400.5,
194
],
"components": [
0.0,
0.0,
0.0,
1.0
],
"lab": [
65.0,
0.0,
0.0
],
"expectedRgb": [
158,
158,
158
]
},
{
"kind": "device-control",
"sample": 4,
"pointPt": [
395,
96
],
"components": [
0,
0,
0,
1
]
},
{
"kind": "icc-vector",
"sample": 5,
"pointPt": [
476,
292
],
"components": [
0.25,
0.5,
0.75,
0.2
],
"lab": [
75.0,
5.25,
12.0
],
"expectedRgb": [
201,
181,
163
]
},
{
"kind": "icc-image",
"sample": 5,
"pointPt": [
481.5,
194
],
"components": [
0.25098039215686274,
0.5019607843137255,
0.7490196078431373,
0.2
],
"lab": [
74.96862745098039,
5.270588235294118,
11.968627450980392
],
"expectedRgb": [
201,
181,
163
]
},
{
"kind": "device-control",
"sample": 5,
"pointPt": [
476,
96
],
"components": [
0.25,
0.5,
0.75,
0.2
]
},
{
"kind": "icc-vector",
"sample": 6,
"pointPt": [
557,
292
],
"components": [
0.6,
0.2,
0.4,
0.1
],
"lab": [
82.3,
-5.4,
0.8000000000000007
],
"expectedRgb": [
195,
208,
203
]
},
{
"kind": "icc-image",
"sample": 6,
"pointPt": [
562.5,
194
],
"components": [
0.6,
0.2,
0.4,
0.10196078431372549
],
"lab": [
82.23137254901961,
-5.4,
0.8000000000000007
],
"expectedRgb": [
194,
208,
203
]
},
{
"kind": "device-control",
"sample": 6,
"pointPt": [
557,
96
],
"components": [
0.6,
0.2,
0.4,
0.1
]
},
{
"kind": "icc-vector",
"sample": 7,
"pointPt": [
638,
292
],
"components": [
1,
1,
1,
1
],
"lab": [
28,
3,
8
],
"expectedRgb": [
74,
64,
54
]
},
{
"kind": "icc-image",
"sample": 7,
"pointPt": [
643.5,
194
],
"components": [
1.0,
1.0,
1.0,
1.0
],
"lab": [
28.0,
3.0,
8.0
],
"expectedRgb": [
74,
64,
54
]
},
{
"kind": "device-control",
"sample": 7,
"pointPt": [
638,
96
],
"components": [
1,
1,
1,
1
]
}
],
"tolerance8Bit": 4,
"references": [
"https://archive.color.org/files/icc32.pdf",
"https://www.color.org/chardata/rgb/srgb.xalter"
],
"limits": [
"Synthetic CMYK input CLUT; not a measured print profile or calibrated press proof.",
"No human-approved golden; no physical display or Windows validation."
],
"generatorSha256": "ed93e85cdf4e398409d49bbf941abfe79062c5a35bde359d75f14e4a8746c8f8"
}
tests/fixtures/pdf/cmyk-lut/synthetic-cmyk-lab.icc
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/pdf/comments.pdf
Vendored
+30
View File
@@ -0,0 +1,30 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 240 260] /CropBox [20 30 220 230] /Annots [4 0 R 5 0 R] /Resources << >> >>
endobj
4 0 obj
<< /Type /Annot /Subtype /Text /Rect [100 100 120 120] /Contents (<b>Literal comment</b>) /F 4 >>
endobj
5 0 obj
<< /Type /Annot /Subtype /Text /Rect [1000 1000 1020 1020] /Contents (Outside CropBox comment) /F 4 >>
endobj
xref
0 6
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000121 00000 n
0000000256 00000 n
0000000369 00000 n
trailer
<< /Size 6 /Root 1 0 R >>
startxref
487
%%EOF
tests/fixtures/pdf/corrupt.pdf
Vendored
+2
View File
@@ -0,0 +1,2 @@
%PDF-1.7
This is deliberately corrupt.
tests/fixtures/pdf/extended/FONT-OFL.txt
Vendored
+93
View File
@@ -0,0 +1,93 @@
Copyright 2022 The BIZ UDGothic Project Authors (https://github.com/googlefonts/morisawa-biz-ud-mincho)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
tests/fixtures/pdf/extended/LCMS2-LICENSE.txt
Vendored
+22
View File
@@ -0,0 +1,22 @@
MIT License
Copyright (c) 2023 Marti Maria Saguer
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject
to the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
tests/fixtures/pdf/extended/README.md
Vendored
+73
View File
@@ -0,0 +1,73 @@
# Extended synthetic PDF corpus
These five original documents contain nine pages. Their drawing source and
explicit expected geometry are in `../extended_generator.py` and `manifest.json`.
They are reproducible test fixtures, not approved golden images.
| Document | Pages | Covered content |
| --- | ---: | --- |
| japanese-layout.pdf | 2 | Embedded Japanese TrueType subset, horizontal text, explicit ruby coordinates, vertical glyph positions, text matrices at 0/90/180/270 degrees |
| graphics-compositing.pdf | 1 | Original RGB and RGBA images, soft mask, 50% vector transparency and overlap, circular clipping, axial shading, CMYK swatches |
| page-geometry.pdf | 4 | Different MediaBox dimensions, offset CropBox, Rotate values 0/90/270/180, four known color markers per page |
| unembedded-font.pdf | 1 | Standard-14 Helvetica, Times-Roman and Courier substitution baseline |
| icc-linear-rgb.pdf | 1 | Embedded linear-sRGB matrix/TRC profile in ICCBased vector and raster colors, plus identical DeviceRGB control values |
The Japanese text uses BIZ UDPGothic Regular. The font is subset-embedded, and
its SIL Open Font License 1.1 is included in `FONT-OFL.txt`. The font file hash
and upstream license URL are recorded in the manifest. The full font file is
not redistributed here. The license permits embedding and redistribution;
these documents do not modify the original font's reserved name.
Generate with Python containing ReportLab, pypdf and Pillow:
```sh
python3 tests/fixtures/pdf/extended_generator.py
```
The default font location is
`/usr/local/share/fonts/BIZ_UDPGothic/BIZUDPGothic-Regular.ttf`.
Set `DOCVIEW_JAPANESE_FONT` to another installation of that font with its adjacent
`OFL.txt` when necessary. A changed font file changes the fixture hashes.
After building DocView's `pdf-worker-evidence` target and installing Poppler command-line tools:
```sh
python3 tests/compare_pdf_extended.py
```
The comparison generates full-resolution PNGs, difference images, contact sheets,
text extraction and a machine-readable report under `tests/results/pdf-extended`.
It assembles pages from the production sandboxed PDF worker's 512-pixel tiles, using FontBroker for unembedded fonts, and
compares them with independently rendered Poppler pages at 144 dpi. CropBox and
Rotate are honored. Deterministic size, marker, alpha and Japanese text/search
assertion failures produce a nonzero exit status; image-difference metrics do
not have an invented acceptance threshold.
The script defaults to system `/usr/bin` Poppler tools when present. Override
with `DOCVIEW_PDFTOPPM`, `DOCVIEW_PDFTOTEXT` and `DOCVIEW_PDFFONTS`, or choose a
different build directory with `DOCVIEW_BUILD_DIR`.
Explicit limitations: vertical text uses individual positioned glyphs, not native
vertical CID metrics or OpenType shaping; ruby positions are authored directly;
CMYK has no calibrated ICC golden; unembedded font coverage is Standard-14 Latin
only. This corpus does not establish arbitrary-document, Windows, accessibility,
print-production or full PDF compatibility.
## ICC profile and additional scale checks
`../icc_generator.py` creates `linear-srgb.icc` from D65 white, sRGB primaries
and gamma 1 through public LittleCMS APIs. It uses no downloaded or borrowed
profile. The generated copyright tag reads `No copyright, use freely`; the
LittleCMS MIT license is retained in `LCMS2-LICENSE.txt`. The manifest records
the numeric parameters, profile hash, library version and upstream source.
The profile's creation timestamp is fixed, and its optional ID is zero, for
reproducibility. The same profile bytes are embedded for vector and image colors.
Run `python3 tests/fixtures/pdf/icc_generator.py` to add/rebuild only this fixture
without requiring the Japanese font. The full generator includes it as well.
`python3 tests/compare_pdf_icc.py` compares actual worker scales 0.5, 1 and 4
against Poppler at 36/72/288 dpi. It checks 54 analytic color probes, page
dimensions and original SHA-256. Linear 0.5 gray must differ materially from
DeviceRGB 0.5; this prevents an ignored ICC profile from passing. Evidence is
in [pdf-icc](../../../results/pdf-icc/README.md). This one RGB profile does not
establish arbitrary ICC LUT, CMYK press calibration or print color proofing.
tests/fixtures/pdf/extended/graphics-compositing.pdf
Vendored
+129
View File
@@ -0,0 +1,129 @@
%PDF-1.4
%“Œ‹ž ReportLab Generated PDF document (opensource)
1 0 obj
<<
/F1 2 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/BitsPerComponent 8 /ColorSpace /DeviceRGB /Filter [ /ASCII85Decode /FlateDecode ] /Height 96 /Length 770 /Subtype /Image
/Type /XObject /Width 96
>>
stream
Gb"/eb=uP&&-Zt`KLJ3YaD-k^$T89A5Q*ko]2*2nDdV>'nltbXI%?Vu\)%#XpZc)FkL,m`a/Lag29tgj>b@o:a%[iE"`irt\m?N"80mR+Ed!;h]g0ju;$/J51FMJH"`f>cfB4U(aX0pU!]d5J6=)+$IGNFNg)],T'?PNNS4Fhm09G3@'(l;dFs:3FOCKQ!p&[7d7-\)$B:5]L(W`!Xp$`#C6_`o<']`d<WH`ua<$EKhe@!'NY)[)LO)]c.jLd`Q,a"f;aDhlMH@[%tOffMA0bR')JXg!iFYAi_9[picjNNI,G_)A.dIhblEjCO$]Kf4A;)PO5hf;f=C2\Ei'20\7"`bfD'YQblRn2$:jOf<2]g.U?8D4I4lk6:EJXg;G#$rEb:=R/hj:$mJcVoph9r-bPE`.`hHIZ`nY)_&hoK4Lg8BqV(6kUqpJK-V2W#*O['Bsgo*$^TQ(Wb8U;YL5Qb<ufP5gGmuLDS#L&m_--+=EXL`=(b&&lJ67E=[*FeZmhtT'$4iaa&H-80UJUOQ.Z#o`@-s,$cj`@jI62!]dk\l<YTGRAbZQaI*_7n,bU;U?hCbjCAq'G_)AaU<#G`mQllF3f867S7!O$09?i=V!%u&gdIlC+NI6QlNM<f7+,BaB9u0F$fiiP2=LZ-F.s^^+D45eI@c%\Ca'TIORn6WdIin<F55;Ce-k=?'?PNNS4Fhm09G3@'(l;dFs:3FOCKQ!p&[7d7-\)$B:5]L(W`!Xp$`#C6_`o<']f2\j)Vm2Hie*:#NP~>endstream
endobj
4 0 obj
<<
/BitsPerComponent 8 /ColorSpace /DeviceRGB /Filter [ /ASCII85Decode /FlateDecode ] /Height 96 /Length 436 /SMask 5 0 R
/Subtype /Image /Type /XObject /Width 96
>>
stream
Gb"0NYn=i,&4BAFTbT>?TQUB/+^bKugVU8h=46!:!d7piQ]!T<ba@#L;0$d'Ft!Q^NU%.AnXrWd)$$\)9Qs*b9G^=29X59@P"^O-`"M#1a\)?tA0QDA1(c].R0]FQ9S<pM-GlZ,'B,^LM=<Oa7/3'Y++-U7&O%A+#VR8pKJu^#_/"f>iLQ0\nA=-PGhZaN]P9D<hG^&j';qE-Dl?J&e_j^$cS9;rOZrX*UCZ@jq:kFFD1%Y*o[-/NIItoWTB+Hk++<e/-hI'-C_G^=gA!5XkbF2&T[+mmiZli3N^!;=qTMW4JtnAjPs07%Q/h'Q;'h5d&lmrT,>fc4n3lq/!tu]/UX+T]a5RLsF9^#E7-EM;5]enZf0jIp3[JNtdiD;4S;rCRS9uV$jqjO3)<VYQ,T5baAP/[j8X(Z_/;95JY6\U]b]s:>Ed)6~>endstream
endobj
5 0 obj
<<
/BitsPerComponent 8 /ColorSpace /DeviceGray /Decode [ 0 1 ] /Filter [ /ASCII85Decode /FlateDecode ] /Height 96 /Length 304
/Subtype /Image /Type /XObject /Width 96
>>
stream
Gb"0S5n\f8#Xm(NrrM7A3Eqf<"D;E3/dhbS([pfDKe[mgbWLrJAk+Im:1B'X'+R0&E9b7@TLT>3Ns8u5$3YIH&ELNU0^fApGY]I$mfQ.1(J+/`!.eJ\./Y83p.>0^JB$=E,&[\(5*\&.@\:mE"N3KprsBJ7"oLl`\:6Vfji7Y6J#be&rbT@Jp=m@8YPs!"OADF@%?0KFe\BGlIeE`Qo)I>'pD`]%O*lFWI_R8fJ'.Mpo)5nBIkKXYbO68<qhtB$'Is00,_5FUq]f,IkWg;i3su@CT?-$8":$k1>l'NE%CWPaTE~>endstream
endobj
6 0 obj
<<
/ColorSpace /DeviceRGB /Coords [ 40 205 525 205 ] /Extend [true true] /Function 10 0 R /ShadingType 2
>>
endobj
7 0 obj
<<
/Contents 12 0 R /MediaBox [ 0 0 600 760 ] /Parent 11 0 R /Resources <<
/ExtGState <<
/gRLs0 <<
/ca .5
>> /gRLs1 <<
/ca 1
>>
>> /Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ] /Shading <<
/Sh0 6 0 R
>> /XObject <<
/FormXob.dde330c7bd98d143ad6dbfc31b4d18ea 4 0 R /FormXob.fd3160a8deaece01bd46d36268643177 3 0 R
>>
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
8 0 obj
<<
/PageMode /UseNone /Pages 11 0 R /Type /Catalog
>>
endobj
9 0 obj
<<
/Author (DocView tests) /CreationDate (D:20000101000000+00'00') /Creator (anonymous) /Keywords () /ModDate (D:20000101000000+00'00') /Producer (ReportLab PDF Library - \(opensource\))
/Subject (unspecified) /Title (DocView synthetic rendering fixture - graphics-compositing.pdf) /Trapped /False
>>
endobj
10 0 obj
<<
/Bounds [ .5 ] /Domain [0.0 1.0] /Encode [ 0 1 0 1 ] /FunctionType 3 /Functions [ 13 0 R 14 0 R ]
>>
endobj
11 0 obj
<<
/Count 1 /Kids [ 7 0 R ] /Type /Pages
>>
endobj
12 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 845
>>
stream
Gatm9gMY_1&:O:SbY+cjMTjqjIO$ibPuJGK[=):r@q`ZT`bP;HbjJqLW1m*#!M8t1B$NEC1PPtI%s?KG"`T_lmu@H\ON`64:d?dhgE['(L^A]q;5poh)[T*0/qPu(Z2opN")sn#EcVO!UQDS8o>>'fE'&UloS94=52,>]_eA]3D1KgOIK.<jXri.Ar;t)Z81n_Y562M9k<^[)81'F7d(?@De^%tKc8&-00fmKPT&t]$1SHS,(buhr7M4ET_>![M:oB_:lsC&7PXm/F<j2b?N!EFg=ICAJKY1QY]#Z\#U"jgILY>%I0p,WaEm/!DPsrb,3;#>6,l=W`QsC\'D10!!ZRk:0UsV#WF[cNnREDD5PgcVQW6WZ`=rlm'/;ie@Q12G"k-1gs%BRe9WN.d$YpNp508tV'*A8'6LNJ)%b`$tS=6Zs9UK<D0)Zmt_IAI'!,fK-:4[#RkO'9;9"@Y;Q,Z(9U[4u<d1ah"6:S'*3S#8t\A*1-QA[0H<?lH,l-SWa:g`/."MNrEuaF3/NJs29(`BZ,f>K@YR'5(5`$P=l1[GPT]28J)$4K2]0f_b%<32\Bi[HVIQHg<)e!A0oXIU#-PbH?\gWNAePNr&b\l)b#J#=06oCCm"4KA_D$"=M\A32$PmjQrjZ)e3\B9j[;N,$chPCRk@#!X%\]:7(cr>@@J@K*j12).Jf@e*'Ng"?,lCgQcM<X8VG&/7EXALKR?[.#Z89DAI_;)6=dERPO35o[sL4H+&iU';_GZ7(l6T3.c(]2jIlchs'>q8-cU%j0>gfM%@P4ft+tP*W^V$$d?!W5[^)K!#UiZJ?>:k1rb#QV/c)<R*Z<s"><Zd_&3~>endstream
endobj
13 0 obj
<<
/C0 [ .078431 .490196 .47451 ] /C1 [ 1 1 1 ] /Domain [ 0 1 ] /FunctionType 2 /N 1
>>
endobj
14 0 obj
<<
/C0 [ 1 1 1 ] /C1 [ .85098 .313725 .313725 ] /Domain [ 0 1 ] /FunctionType 2 /N 1
>>
endobj
xref
0 15
0000000000 65535 f
0000000061 00000 n
0000000092 00000 n
0000000199 00000 n
0000001157 00000 n
0000001794 00000 n
0000002303 00000 n
0000002426 00000 n
0000002814 00000 n
0000002883 00000 n
0000003202 00000 n
0000003322 00000 n
0000003382 00000 n
0000004318 00000 n
0000004422 00000 n
trailer
<<
/ID
[<998c181bcc6a27b405fdaeda1882afd9><998c181bcc6a27b405fdaeda1882afd9>]
% ReportLab generated PDF document -- digest (opensource)
/Info 9 0 R
/Root 8 0 R
/Size 15
>>
startxref
4526
%%EOF
tests/fixtures/pdf/extended/icc-linear-rgb.pdf
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/pdf/extended/japanese-layout.pdf
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/pdf/extended/linear-srgb.icc
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/pdf/extended/manifest.json
Vendored
+776
View File
@@ -0,0 +1,776 @@
{
"origin": "Original deterministic vector, text and raster drawings authored for DocView tests",
"reference_status": "Synthetic source specifies content and coordinates; no human-approved golden image exists",
"font": {
"family": "BIZ UDPGothic Regular",
"source_file": "BIZUDPGothic-Regular.ttf",
"sha256": "22f41ce68f1ce62477ca4ec1e2c0c400cc545ddff31d713e8edf87808614aeb2",
"license": "SIL Open Font License 1.1",
"license_copy": "FONT-OFL.txt",
"upstream": "https://github.com/googlefonts/morisawa-biz-ud-gothic",
"upstream_license": "https://raw.githubusercontent.com/googlefonts/morisawa-biz-ud-gothic/main/OFL.txt",
"embedding": "Subset embedded by ReportLab; OFL permits embedding and redistribution. No reserved font name is changed."
},
"documents": [
{
"file": "japanese-layout.pdf",
"sha256": "47011b3c7c7363bc9549881ade31b836e1efdae555504076627978dbde136f00",
"page_count": 2,
"expected": {
"page_sizes_pt": [
[
540,
720
],
[
540,
720
]
],
"horizontal_text": "日本語の文字配置と埋込みフォント",
"vertical_columns": [
{
"x": 420,
"top": 590,
"step": 32,
"text": "日本語縦書き検証"
},
{
"x": 374,
"top": 590,
"step": 32,
"text": "天地左右回転配置"
}
],
"text_matrix_rotations_degrees": [
0,
90,
180,
270
],
"font_embedded": true
},
"limits": [
"Manual vertical glyph placement does not test native vertical CID metrics or OpenType shaping.",
"Ruby placement is explicit source geometry, not automatic ruby layout."
]
},
{
"file": "graphics-compositing.pdf",
"sha256": "ddaec9f619dde57995e9baff6a560151e9757d4b1bdce9b630299a5f0438d49e",
"page_count": 1,
"expected": {
"page_sizes_pt": [
[
600,
760
]
],
"rgb_image_pixels": [
96,
96
],
"alpha": 0.5,
"alpha_probes": [
{
"point_pt": [
80,
370
],
"expected_rgb": [
244,
119,
121
],
"region": "red over gray"
},
{
"point_pt": [
180,
390
],
"expected_rgb": [
116,
119,
248
],
"region": "blue over gray"
},
{
"point_pt": [
130,
390
],
"expected_rgb": [
122,
59,
188
],
"region": "blue over red over gray"
}
],
"clip_circle_pt": [
390,
395,
64
],
"axial_shading_band_pt": [
40,
205,
485,
62
],
"cmyk_swatches": [
[
1,
0,
0,
0
],
[
0,
1,
0,
0
],
[
0,
0,
1,
0
],
[
0,
0,
0,
1
]
]
},
"limits": [
"CMYK output depends on renderer color conversion; no approved ICC reference is supplied.",
"Pixel differences are evidence, not a calibrated quality threshold."
]
},
{
"file": "page-geometry.pdf",
"sha256": "e5c0e4cb6d73c322235c9d7c89cf1e2e925ac7d3c22c7672b595a193fa133a0f",
"page_count": 4,
"expected": {
"pages": [
{
"media_box_pt": [
0,
0,
420,
320
],
"crop_box_pt": [
30,
40,
390,
300
],
"rotation": 0,
"display_size_pt": [
360,
260
],
"markers_pdf_coordinates": [
{
"x": 50,
"y": 60,
"rgb": [
255,
0,
0
]
},
{
"x": 370,
"y": 60,
"rgb": [
0,
170,
0
]
},
{
"x": 370,
"y": 280,
"rgb": [
0,
0,
255
]
},
{
"x": 50,
"y": 280,
"rgb": [
255,
170,
0
]
}
]
},
{
"media_box_pt": [
0,
0,
560,
360
],
"crop_box_pt": [
20,
30,
520,
330
],
"rotation": 90,
"display_size_pt": [
300,
500
],
"markers_pdf_coordinates": [
{
"x": 40,
"y": 50,
"rgb": [
255,
0,
0
]
},
{
"x": 500,
"y": 50,
"rgb": [
0,
170,
0
]
},
{
"x": 500,
"y": 310,
"rgb": [
0,
0,
255
]
},
{
"x": 40,
"y": 310,
"rgb": [
255,
170,
0
]
}
]
},
{
"media_box_pt": [
0,
0,
300,
600
],
"crop_box_pt": [
15,
50,
285,
550
],
"rotation": 270,
"display_size_pt": [
500,
270
],
"markers_pdf_coordinates": [
{
"x": 35,
"y": 70,
"rgb": [
255,
0,
0
]
},
{
"x": 265,
"y": 70,
"rgb": [
0,
170,
0
]
},
{
"x": 265,
"y": 530,
"rgb": [
0,
0,
255
]
},
{
"x": 35,
"y": 530,
"rgb": [
255,
170,
0
]
}
]
},
{
"media_box_pt": [
0,
0,
480,
480
],
"crop_box_pt": [
60,
80,
420,
400
],
"rotation": 180,
"display_size_pt": [
360,
320
],
"markers_pdf_coordinates": [
{
"x": 80,
"y": 100,
"rgb": [
255,
0,
0
]
},
{
"x": 400,
"y": 100,
"rgb": [
0,
170,
0
]
},
{
"x": 400,
"y": 380,
"rgb": [
0,
0,
255
]
},
{
"x": 80,
"y": 380,
"rgb": [
255,
170,
0
]
}
]
}
]
},
"limits": [
"Renderer comparisons must use CropBox and honor document Rotate."
]
},
{
"file": "unembedded-font.pdf",
"sha256": "37532f1b3a403c5527fe39807e2bb737b55472d8edf32cc1ca47668c7623e095",
"page_count": 1,
"expected": {
"page_sizes_pt": [
[
400,
300
]
],
"base_fonts": [
"Helvetica",
"Times-Roman",
"Courier"
],
"font_embedded": false
},
"limits": [
"This is a controlled Standard-14 substitution baseline, not arbitrary missing Japanese font coverage.",
"Cross-OS fallback is not established by this Linux-only comparison."
]
},
{
"file": "icc-linear-rgb.pdf",
"sha256": "4a6fd71d761bb639d3587f39e94446b8b92c6448177bbcc161a622b94aac9a06",
"page_count": 1,
"expected": {
"page_sizes_pt": [
[
600,
350
]
],
"icc_probes": [
{
"kind": "icc-vector",
"point_pt": [
68,
248
],
"components": [
0.0625,
0.0625,
0.0625
],
"expected_rgb": [
71,
71,
71
]
},
{
"kind": "device-control",
"point_pt": [
68,
164
],
"components": [
0.0625,
0.0625,
0.0625
],
"expected_rgb": [
16,
16,
16
]
},
{
"kind": "icc-image",
"point_pt": [
74,
68
],
"components": [
16,
16,
16
],
"expected_rgb": [
71,
71,
71
]
},
{
"kind": "icc-vector",
"point_pt": [
156,
248
],
"components": [
0.25,
0.25,
0.25
],
"expected_rgb": [
137,
137,
137
]
},
{
"kind": "device-control",
"point_pt": [
156,
164
],
"components": [
0.25,
0.25,
0.25
],
"expected_rgb": [
64,
64,
64
]
},
{
"kind": "icc-image",
"point_pt": [
162,
68
],
"components": [
64,
64,
64
],
"expected_rgb": [
137,
137,
137
]
},
{
"kind": "icc-vector",
"point_pt": [
244,
248
],
"components": [
0.5,
0.5,
0.5
],
"expected_rgb": [
188,
188,
188
]
},
{
"kind": "device-control",
"point_pt": [
244,
164
],
"components": [
0.5,
0.5,
0.5
],
"expected_rgb": [
128,
128,
128
]
},
{
"kind": "icc-image",
"point_pt": [
250,
68
],
"components": [
128,
128,
128
],
"expected_rgb": [
188,
188,
188
]
},
{
"kind": "icc-vector",
"point_pt": [
332,
248
],
"components": [
0.75,
0.75,
0.75
],
"expected_rgb": [
225,
225,
225
]
},
{
"kind": "device-control",
"point_pt": [
332,
164
],
"components": [
0.75,
0.75,
0.75
],
"expected_rgb": [
191,
191,
191
]
},
{
"kind": "icc-image",
"point_pt": [
338,
68
],
"components": [
191,
191,
191
],
"expected_rgb": [
224,
224,
224
]
},
{
"kind": "icc-vector",
"point_pt": [
420,
248
],
"components": [
0.5,
0.125,
0.0625
],
"expected_rgb": [
188,
99,
71
]
},
{
"kind": "device-control",
"point_pt": [
420,
164
],
"components": [
0.5,
0.125,
0.0625
],
"expected_rgb": [
128,
32,
16
]
},
{
"kind": "icc-image",
"point_pt": [
426,
68
],
"components": [
128,
32,
16
],
"expected_rgb": [
188,
99,
71
]
},
{
"kind": "icc-vector",
"point_pt": [
508,
248
],
"components": [
0.0625,
0.25,
0.5
],
"expected_rgb": [
71,
137,
188
]
},
{
"kind": "device-control",
"point_pt": [
508,
164
],
"components": [
0.0625,
0.25,
0.5
],
"expected_rgb": [
16,
64,
128
]
},
{
"kind": "icc-image",
"point_pt": [
514,
68
],
"components": [
16,
64,
128
],
"expected_rgb": [
71,
137,
188
]
}
],
"probe_tolerance_rgb": 3,
"profile_effect": "ICCBased 0.5 linear gray becomes approximately 188, distinct from DeviceRGB approximately 128."
},
"limits": [
"One RGB matrix/TRC profile, not arbitrary ICC LUT/CMYK/printing conformance.",
"Analytic sRGB color expectation; independent renderer comparison still requires visual review."
]
}
],
"color_profile": {
"file": "linear-srgb.icc",
"sha256": "343c89fd0eb35fff20bcd52b63d3b3b941702e443c46f0761ab367fcf67c6287",
"bytes": 568,
"source": "Self-created with LittleCMS cmsCreateRGBProfile using the stated numeric white point, primaries and gamma; no third-party ICC profile is copied.",
"lcms_encoded_version": 2190,
"lcms_license": "MIT; LCMS2-LICENSE.txt",
"upstream": "https://github.com/mm2/Little-CMS",
"profile_copyright_tag": "No copyright, use freely",
"white_xy": [
0.3127,
0.329
],
"primaries_xy": [
[
0.64,
0.33
],
[
0.3,
0.6
],
[
0.15,
0.06
]
],
"gamma": 1,
"timestamp_normalization": "Header creation time is fixed to 2026-09-19; optional profile ID is zero."
}
}
tests/fixtures/pdf/extended/page-geometry.pdf
Vendored
+161
View File
@@ -0,0 +1,161 @@
%PDF-1.4
%âãÏÓ
1 0 obj
<<
/Producer (pypdf)
/Title (DocView page geometry fixture)
/Author (DocView tests)
>>
endobj
2 0 obj
<<
/Type /Pages
/Count 4
/Kids [ 4 0 R 8 0 R 10 0 R 12 0 R ]
>>
endobj
3 0 obj
<<
/Type /Catalog
/Pages 2 0 R
>>
endobj
4 0 obj
<<
/Contents 5 0 R
/MediaBox [ 0 0 420 320 ]
/Resources <<
/Font 6 0 R
/ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>>
/Rotate 0
/Trans <<
>>
/Type /Page
/CropBox [ 30 40 390 300 ]
/Parent 2 0 R
>>
endobj
5 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ]
/Length 535
>>
stream
Gatn"4\s!M%#4NU$BK^68Y"*G3<UlA#eD/`-j:X<4*2HoT9&@dBnVRMB[t&+?e^X)7Y@F:bLhjUb79QX1h_R1&[ZC,F;JoM2&b1^(8uYL[nk7PdOZk%oPFhX4<Pf^&i:;tOB6rrLR"Z?SWiK*,oWihj!#Uo,e7e>(,!sqM\8OJ#t7@Ci3M=`<]p)8FfgQ_+Km?A>.,0d&R:V9X[CC3Z#<6/p;L/rp$WjYDEU3-Z^Hp5`5i*U&Bcba#mnrQWLu5!`?u$>MMeXZM2#dO@OWkRMD9)r8<rmTW%_gIN%)B'_)MJd>[email protected]?u!2^a7bP37]8'A8n;G_B#qge=&C?,]uBKqVa@/a(/I>ERVJC+IS4*TX^Xbdm:da@5MHaB%&L6>04X+6g8T@%D+O^*6=T85'hQ]K4jpmsou#&b[i<cVMZ("BV)Kh/35#&)B$`#Ft<s"!nEGpBH;n%8O@$\)l`[DN$WPgNEj7_1PXps45:cSM""0(DuVL:TH@3+7D;$+(4m/?U#TdLULq%br61tiJm~>
endstream
endobj
6 0 obj
<<
/F1 7 0 R
>>
endobj
7 0 obj
<<
/BaseFont /Helvetica
/Encoding /WinAnsiEncoding
/Name /F1
/Subtype /Type1
/Type /Font
>>
endobj
8 0 obj
<<
/Contents 9 0 R
/MediaBox [ 0 0 560 360 ]
/Resources <<
/Font 6 0 R
/ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>>
/Rotate 90
/Trans <<
>>
/Type /Page
/CropBox [ 20 30 520 330 ]
/Parent 2 0 R
>>
endobj
9 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ]
/Length 540
>>
stream
Gatn"9i'Ou&;KZL(%8Cr.*VHrf/XU7\b(DcDH.3("@Pn+X(;n@r6@uFa/piadR:@\Sp+cfgG+?AAS[&-3:Ck\OhhP+4ZKeLr?kkD%AQY3F1`t#iM"q-=POJ#I"r=.-R.'lS5/4c:"j/;=iejQk[%9+n;P^?3;Gu5\X?SW5nfDLA$TE;34X^(N@6]!4#B.01%EiZaLbHQ2Q>RWA4b/QVt&1F:!'AUC":N%LW6jYNBZ+@(28=UUHjrE?:f7tL)7E]TcnUpT+/jU'k*40B/pC*,dtgPET("BC"[j.nsoM<B<7WCo*hRG7ArS-.09#_1O@2W-%Ock_S8sndqK944N>KmDJ$toTjfUO@SdrMS=^:>7Hu\-Y;fT@fl>?*CM'k>?C+YX[N%>J[uTJqa`$XV+kl)@.h_6,Uud3L/2)2r7W9a3$1G#]>[>5*$oe(Dm;YNan'QVTJ"e+dX?]h&rOp(6XMVqbs7:X<L+eQ^AKdt-ApF1Kh^ZUUde2Yg#H$7n_rD6UbOZ>'7Z;9'e'$Qlf2fl;quZ"8i5G~>
endstream
endobj
10 0 obj
<<
/Contents 11 0 R
/MediaBox [ 0 0 300 600 ]
/Resources <<
/Font 6 0 R
/ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>>
/Rotate 270
/Trans <<
>>
/Type /Page
/CropBox [ 15 50 285 550 ]
/Parent 2 0 R
>>
endobj
11 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ]
/Length 546
>>
stream
Gatn"9hu``%#46B.srlR%>b`GfcqW^V$BEO<qQI9JG>MfC:gGBr7]H:7TC`eltGFtZ2HkMN%GlqXH)TMi-^.*Y8/\+oDmI'*i<jp3=QbDjAoq]Zb+7dUM8!j#lNT)n8QU&o&uqs+9NGiS%<GnjlVH\r^]Q[Z@F*L?R6$S5%GEX0R*hm]VoBURkTCtPtC$sag`C^)(F6pZGLjBPrZV1^c#!qBs@[bT1@2120m/E/N2pu#N["Zr*j6T=Q";L'SD]l8eMIPL9.@LlM7@[;FfBV")896Ck]hG43#-7+AFkCV'9":aSD@^b)jes@QQo!;sY-=;&K\Hm"(9Q6]17ZeLeWiU$sJ.rV&!oL%QI%]+om;'t)kiU@tAl[,X8:UL&/a4B+e`j;./FaO0jL5oA!I8Gq.NoV@7H>D0@P;J?^9g<rXld6$FF"hn66?!W&7;n1NKDtgRp_\#Sg0HEg3LD!'&nGa-V;^-4dg\s7k^;CIS-gX%u9JRd_o<XZ8AYn^jbmQ)3]>pfY/8r;tE&Y]4^4(LBTC9fq%#2MW?f@C~>
endstream
endobj
12 0 obj
<<
/Contents 13 0 R
/MediaBox [ 0 0 480 480 ]
/Resources <<
/Font 6 0 R
/ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>>
/Rotate 180
/Trans <<
>>
/Type /Page
/CropBox [ 60 80 420 400 ]
/Parent 2 0 R
>>
endobj
13 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ]
/Length 540
>>
stream
Gatn"9lJN8&;KZL(%2a7.eVFh4?6;=HJq\cg^o7(VH&36U1GkOoC7c(euFhAYgIbZPBUW1G[uD_&*^ZA^u^N5%g92*+$dR7Qp9SFmKbtfajO;YiN_-ORcj<-XEZ^_6@&4bBFr'6)J@6/UFE\(;>49?"=CX0iJ9qnWCnd?TOS0DK.Upa%']o8[1ZoI04sIccjrgdWiIJiF`0ZV0bmPc:KC*+Fm3XHM=F0,+dPmfqb_g9&:.7ddk3TPAKs%g'M+RKWZ+b(P`Ql[RYJ-q`1W>lcrs\,%;CC8W$]hVd2Wrna0-X,-V1KOUpr`i'9Z9:>q!:Q&/(hl#1I.jRS\6nP?o=[(KbQWlYlua$Yg]W\=k&T"Z/'+-[!e8-d5NN2!ZXS#-oU@)2mA"0QOf-2Y]QK[p57cY7'O:&u1][eMP&O,N]GRDVeZ+_;bU+qr?LPNH4V+o/Fm+P4R!87U+N[[rA,phB2GU0D@-p4IYm:M<FA/(1Y)V[n`/Kh)t)=A%Ofb54,R;s#T0*MY1QVnU/+.pJ!GcIfP!6j=L~>
endstream
endobj
xref
0 14
0000000000 65535 f
0000000015 00000 n
0000000117 00000 n
0000000196 00000 n
0000000245 00000 n
0000000461 00000 n
0000001087 00000 n
0000001118 00000 n
0000001225 00000 n
0000001442 00000 n
0000002073 00000 n
0000002293 00000 n
0000002931 00000 n
0000003151 00000 n
trailer
<<
/Size 14
/Root 3 0 R
/Info 1 0 R
>>
startxref
3783
%%EOF
tests/fixtures/pdf/extended/unembedded-font.pdf
Vendored
+80
View File
@@ -0,0 +1,80 @@
%PDF-1.4
%“Œ‹ž ReportLab Generated PDF document (opensource)
1 0 obj
<<
/F1 2 0 R /F2 3 0 R /F3 4 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/BaseFont /Times-Roman /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
>>
endobj
4 0 obj
<<
/BaseFont /Courier /Encoding /WinAnsiEncoding /Name /F3 /Subtype /Type1 /Type /Font
>>
endobj
5 0 obj
<<
/Contents 9 0 R /MediaBox [ 0 0 400 300 ] /Parent 8 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
6 0 obj
<<
/PageMode /UseNone /Pages 8 0 R /Type /Catalog
>>
endobj
7 0 obj
<<
/Author (DocView tests) /CreationDate (D:20000101000000+00'00') /Creator (anonymous) /Keywords () /ModDate (D:20000101000000+00'00') /Producer (ReportLab PDF Library - \(opensource\))
/Subject (unspecified) /Title (DocView synthetic rendering fixture - unembedded-font.pdf) /Trapped /False
>>
endobj
8 0 obj
<<
/Count 1 /Kids [ 5 0 R ] /Type /Pages
>>
endobj
9 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 419
>>
stream
Gatmt>u/<k'Sc)P($AIhR-Qqe4(+PV0G'+KG@`jgin!'H&H)4PM?tG@1bh^"PG[MghK&tdndnfr%%ouH4<QltTEu0&Z[hbcV_sH9:#,fo+L]a<jR$,/JRVs_Cp6o,A,fiKP!.LJ_6>4<K9L$5%m%Gfq&W4C47q6:%n9k*_(_[_5aOa^IcWU#G@]0>Aa<d9_2:aa"mo(:o7MtXjjeL'&.V2hisD:5hVCCYrh#LdNmb49'Orn*nE!T;M1"u$Pgt*OZJE3G:(5UW*OjEYcmlWso)ad5.Q6B+QM#6h1!tRS="0%]6W$.4b;*QRVHi%6gm"KIgsNjPlajrWo6%mSBH750KPrT"'sF@N?Znl&d+""-L"=;E-5@1K/)n.apHCkLX?b^<OsBL%YVb0ea=G&BL2i0WG)u]=D[(cofD~>endstream
endobj
xref
0 10
0000000000 65535 f
0000000061 00000 n
0000000112 00000 n
0000000219 00000 n
0000000328 00000 n
0000000433 00000 n
0000000626 00000 n
0000000694 00000 n
0000001008 00000 n
0000001067 00000 n
trailer
<<
/ID
[<fc20ccc2e85c965a23d5494cd2381a69><fc20ccc2e85c965a23d5494cd2381a69>]
% ReportLab generated PDF document -- digest (opensource)
/Info 7 0 R
/Root 6 0 R
/Size 10
>>
startxref
1576
%%EOF
tests/fixtures/pdf/extended_generator.py
Vendored
+189
View File
@@ -0,0 +1,189 @@
#!/usr/bin/env python3
"""Create deterministic, original PDF fixtures with explicit drawing geometry.
Dependencies: ReportLab, pypdf, Pillow. The Japanese font is subset-embedded;
its adjacent SIL OFL license is copied alongside the generated documents.
"""
from pathlib import Path
from io import BytesIO
import hashlib
import json
import os
import shutil
from PIL import Image, ImageDraw
from pypdf import PdfReader, PdfWriter
from pypdf.generic import NameObject, NumberObject, RectangleObject
from reportlab.lib import colors
from reportlab.lib.utils import ImageReader
from reportlab.pdfbase import pdfmetrics
from reportlab.pdfbase.ttfonts import TTFont
from reportlab.pdfgen import canvas
HERE = Path(__file__).resolve().parent
OUT = HERE / "extended"
FONT = Path(os.environ.get("DOCVIEW_JAPANESE_FONT", "/usr/local/share/fonts/BIZ_UDPGothic/BIZUDPGothic-Regular.ttf"))
LICENSE = FONT.parent / "OFL.txt"
OUT.mkdir(parents=True, exist_ok=True)
if not FONT.is_file() or not LICENSE.is_file():
raise SystemExit("Set DOCVIEW_JAPANESE_FONT to BIZUDPGothic-Regular.ttf with its adjacent OFL.txt.")
license_text = LICENSE.read_text(encoding="utf-8")
if "SIL OPEN FONT LICENSE Version 1.1" not in license_text:
raise SystemExit("The expected SIL Open Font License was not found.")
pdfmetrics.registerFont(TTFont("FixtureJapanese", str(FONT)))
shutil.copyfile(LICENSE, OUT / "FONT-OFL.txt")
manifest = {
"origin": "Original deterministic vector, text and raster drawings authored for DocView tests",
"reference_status": "Synthetic source specifies content and coordinates; no human-approved golden image exists",
"font": {
"family": "BIZ UDPGothic Regular", "source_file": FONT.name,
"sha256": hashlib.sha256(FONT.read_bytes()).hexdigest(),
"license": "SIL Open Font License 1.1", "license_copy": "FONT-OFL.txt",
"upstream": "https://github.com/googlefonts/morisawa-biz-ud-gothic",
"upstream_license": "https://raw.githubusercontent.com/googlefonts/morisawa-biz-ud-gothic/main/OFL.txt",
"embedding": "Subset embedded by ReportLab; OFL permits embedding and redistribution. No reserved font name is changed.",
},
"documents": [],
}
def new_canvas(name, size):
result = canvas.Canvas(str(OUT / name), pagesize=size, invariant=1, pageCompression=1)
result.setTitle("DocView synthetic rendering fixture - " + name)
result.setAuthor("DocView tests")
return result
def label(c, x, y, text, size=11):
c.setFillColor(colors.HexColor("#263746"))
c.setFont("Helvetica", size)
c.drawString(x, y, text)
def japanese(c, x, y, text, size=21):
c.setFillColor(colors.HexColor("#132c35"))
c.setFont("FixtureJapanese", size)
c.drawString(x, y, text)
def record(name, pages, expected, limits):
manifest["documents"].append({"file": name, "sha256": hashlib.sha256((OUT / name).read_bytes()).hexdigest(),
"page_count": pages, "expected": expected, "limits": limits})
c = new_canvas("japanese-layout.pdf", (540, 720))
label(c, 38, 675, "EMBEDDED JAPANESE / HORIZONTAL", 17)
japanese(c, 38, 614, "日本語の文字配置と埋込みフォント", 23)
japanese(c, 38, 557, "漢字・ひらがな・カタカナ・１２３４", 21)
japanese(c, 38, 500, "読み取り専用の文書表示を検証します。", 20)
label(c, 38, 454, "ASCII text uses the same embedded Japanese font:")
japanese(c, 38, 420, "DocView 0123456789 ABC xyz", 20)
japanese(c, 55, 330, "漢字", 29)
japanese(c, 55, 364, "かんじ", 11)
label(c, 155, 341, "Ruby has explicit, independent PDF coordinates.")
c.setStrokeColor(colors.HexColor("#147d79")); c.setLineWidth(1)
c.rect(38, 220, 464, 76, fill=0)
japanese(c, 53, 262, "天地左右", 23)
label(c, 53, 236, "Four ideographs above must remain distinct and correctly placed.")
label(c, 38, 55, "540 x 720 pt / exact source coordinates / embedded TrueType subset")
c.showPage()
label(c, 38, 675, "EMBEDDED JAPANESE / EXPLICIT VERTICAL POSITIONS", 15)
columns = [{"x": 420, "top": 590, "step": 32, "text": "日本語縦書き検証"},
{"x": 374, "top": 590, "step": 32, "text": "天地左右回転配置"}]
for col in columns:
for index, glyph in enumerate(col["text"]):
japanese(c, col["x"], col["top"] - index * col["step"], glyph, 24)
c.setStrokeColor(colors.HexColor("#d7e8e6")); c.line(col["x"] - 5, 315, col["x"] - 5, 620)
for x, y, angle in [(80, 525, 0), (230, 485, 90), (200, 310, 180), (65, 350, 270)]:
c.saveState(); c.translate(x, y); c.rotate(angle); japanese(c, 0, 0, "天地左右", 20); c.restoreState()
label(c, 38, 228, "Right column is read top to bottom; the next column is to its left.")
label(c, 38, 205, "Four lower-left samples use text matrices at 0 / 90 / 180 / 270 degrees.")
label(c, 38, 73, "This tests explicit glyph placement, not vertical CID metrics or shaping.")
label(c, 38, 55, "540 x 720 pt / each glyph has a documented baseline")
c.save()
record("japanese-layout.pdf", 2,
{"page_sizes_pt": [[540, 720], [540, 720]], "horizontal_text": "日本語の文字配置と埋込みフォント",
"vertical_columns": columns, "text_matrix_rotations_degrees": [0, 90, 180, 270], "font_embedded": True},
["Manual vertical glyph placement does not test native vertical CID metrics or OpenType shaping.", "Ruby placement is explicit source geometry, not automatic ruby layout."])
c = new_canvas("graphics-compositing.pdf", (600, 760))
label(c, 35, 721, "IMAGES / ALPHA / CLIPPING / SHADING / CMYK", 17)
image = Image.new("RGB", (96, 96), "white"); drawing = ImageDraw.Draw(image)
for y in range(0, 96, 12):
for x in range(0, 96, 12): drawing.rectangle((x, y, x + 11, y + 11), fill="#147d79" if (x + y) // 12 % 2 == 0 else "#e9b44c")
drawing.line((0, 0, 95, 95), fill="black", width=4)
c.drawImage(ImageReader(image), 40, 524, width=144, height=144)
label(c, 40, 505, "96 x 96 RGB image, enlarged 1.5x")
alpha = Image.new("RGBA", (96, 96), (0, 0, 0, 0)); d = ImageDraw.Draw(alpha)
d.ellipse((4, 4, 92, 92), fill=(224, 63, 90, 150)); d.rectangle((15, 37, 81, 58), fill=(30, 70, 220, 190))
c.setFillColor(colors.HexColor("#dfe7ed")); c.rect(310, 524, 144, 144, stroke=0, fill=1)
c.drawImage(ImageReader(alpha), 310, 524, width=144, height=144, mask="auto")
label(c, 310, 505, "RGBA image with soft mask")
c.setFillColor(colors.HexColor("#e8edf1")); c.rect(40, 330, 220, 135, fill=1, stroke=0)
c.saveState(); c.setFillColor(colors.red); c.setFillAlpha(0.5); c.rect(55, 345, 95, 95, fill=1, stroke=0)
c.setFillColor(colors.blue); c.setFillAlpha(0.5); c.rect(112, 365, 105, 75, fill=1, stroke=0); c.restoreState()
label(c, 40, 308, "Two vector rectangles at 50% alpha")
c.saveState(); path = c.beginPath(); path.circle(390, 395, 64); c.clipPath(path, stroke=0, fill=0)
c.drawImage(ImageReader(image), 300, 310, width=180, height=180); c.restoreState()
c.setStrokeColor(colors.HexColor("#263746")); c.circle(390, 395, 64, fill=0, stroke=1)
label(c, 310, 308, "Raster image clipped to a circle")
c.saveState(); path = c.beginPath(); path.rect(40, 205, 485, 62); c.clipPath(path, stroke=0, fill=0)
c.linearGradient(40, 205, 525, 205, [colors.HexColor("#147d79"), colors.white, colors.HexColor("#d95050")], [0, 0.5, 1]); c.restoreState()
label(c, 40, 185, "Axial PDF shading, clipped to a 485 x 62 pt band")
swatches = [(1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 0, 0, 1)]
for index, value in enumerate(swatches):
c.setFillColorCMYK(*value); c.rect(40 + index * 124, 76, 105, 67, stroke=0, fill=1)
label(c, 40 + index * 124, 56, "CMYK " + "".join(map(str, value)), 10)
c.save()
record("graphics-compositing.pdf", 1, {"page_sizes_pt": [[600, 760]], "rgb_image_pixels": [96, 96],
"alpha": 0.5, "alpha_probes": [
{"point_pt": [80, 370], "expected_rgb": [244, 119, 121], "region": "red over gray"},
{"point_pt": [180, 390], "expected_rgb": [116, 119, 248], "region": "blue over gray"},
{"point_pt": [130, 390], "expected_rgb": [122, 59, 188], "region": "blue over red over gray"}],
"clip_circle_pt": [390, 395, 64], "axial_shading_band_pt": [40, 205, 485, 62], "cmyk_swatches": swatches},
["CMYK output depends on renderer color conversion; no approved ICC reference is supplied.", "Pixel differences are evidence, not a calibrated quality threshold."])
geometries = [([420, 320], [30, 40, 390, 300], 0), ([560, 360], [20, 30, 520, 330], 90),
([300, 600], [15, 50, 285, 550], 270), ([480, 480], [60, 80, 420, 400], 180)]
geometry_buffer = BytesIO(); c = canvas.Canvas(geometry_buffer, invariant=1, pageCompression=1)
expectations = []
for number, (size, crop, rotation) in enumerate(geometries, 1):
c.setPageSize(size)
left, bottom, right, top = crop; width, height = right - left, top - bottom
c.setFillColor(colors.HexColor("#ffd6d6")); c.rect(0, 0, *size, fill=1, stroke=0)
c.setFillColor(colors.white); c.rect(left, bottom, width, height, fill=1, stroke=0)
c.setStrokeColor(colors.black); c.setLineWidth(1); c.rect(left + 1, bottom + 1, width - 2, height - 2, fill=0)
dots = [(left + 20, bottom + 20, [255, 0, 0]), (right - 20, bottom + 20, [0, 170, 0]),
(right - 20, top - 20, [0, 0, 255]), (left + 20, top - 20, [255, 170, 0])]
for x, y, rgb in dots:
c.setFillColor(colors.Color(*(v / 255 for v in rgb))); c.circle(x, y, 8, fill=1, stroke=0)
label(c, left + 35, bottom + height * 0.55, f"PAGE {number} / Rotate {rotation}", 15)
label(c, left + 35, bottom + height * 0.55 - 25, f"CropBox {crop}", 10)
label(c, left + 35, bottom + height * 0.55 - 45, "Pink margin must be excluded.", 10)
expectations.append({"media_box_pt": [0, 0, *size], "crop_box_pt": crop, "rotation": rotation,
"display_size_pt": [height, width] if rotation % 180 else [width, height],
"markers_pdf_coordinates": [{"x": x, "y": y, "rgb": rgb} for x, y, rgb in dots]})
c.showPage()
c.save(); source = PdfReader(BytesIO(geometry_buffer.getvalue())); writer = PdfWriter()
for page, (_, crop, rotation) in zip(source.pages, geometries):
page.cropbox = RectangleObject(crop); page[NameObject("/Rotate")] = NumberObject(rotation); writer.add_page(page)
writer.add_metadata({"/Title": "DocView page geometry fixture", "/Author": "DocView tests"})
with (OUT / "page-geometry.pdf").open("wb") as output: writer.write(output)
record("page-geometry.pdf", 4, {"pages": expectations}, ["Renderer comparisons must use CropBox and honor document Rotate."])
c = new_canvas("unembedded-font.pdf", (400, 300))
label(c, 30, 252, "UNEMBEDDED STANDARD PDF FONT", 15)
label(c, 30, 205, "Helvetica: ABCDEFGHIJKLMNOPQRSTUVWXYZ", 12)
label(c, 30, 176, "abcdefghijklmnopqrstuvwxyz 0123456789", 12)
c.setFont("Times-Roman", 15); c.drawString(30, 137, "Times-Roman: replacement font baseline")
c.setFont("Courier", 14); c.drawString(30, 100, "Courier: 0123456789 fixed pitch")
label(c, 30, 46, "Standard-14 fonts are intentionally not embedded.", 11)
c.save()
record("unembedded-font.pdf", 1, {"page_sizes_pt": [[400, 300]], "base_fonts": ["Helvetica", "Times-Roman", "Courier"], "font_embedded": False},
["This is a controlled Standard-14 substitution baseline, not arbitrary missing Japanese font coverage.", "Cross-OS fallback is not established by this Linux-only comparison."])
from icc_generator import generate as generate_icc
icc_record, manifest["color_profile"] = generate_icc(OUT)
manifest["documents"].append(icc_record)
(OUT / "manifest.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print(f"Generated {len(manifest['documents'])} PDF fixtures under {OUT}")
tests/fixtures/pdf/fonts/README.md
Vendored
+15
View File
@@ -0,0 +1,15 @@
# Unembedded Japanese CID fixture
`unembedded-japanese.pdf` is authored locally by `../generate_fonts.py`, using
ReportLab's predefined Japanese CID fonts. The two pages select
`UniJIS-UCS2-H` and `UniJIS-UCS2-V` for HeiseiMin-W3 and HeiseiKakuGo-W5.
No font program is embedded. The second page uses the PDF vertical writing
mode, including native punctuation positioning, rather than individually
positioning horizontal glyphs. The text is synthetic and contains no third-party
document content. Font names identify requests; installed substitutes can differ.
The manifest records the input hash, authored text and page dimensions.
Generation requires ReportLab and pypdf. The generator checks both encoding
names and absence of FontFile/FontFile2/FontFile3 in the CID font descriptors.
Successful raster output alone does not establish character accuracy or full
CMap coverage; compare with Poppler and inspect both pages.
tests/fixtures/pdf/fonts/manifest.json
Vendored
+33
View File
@@ -0,0 +1,33 @@
{
"schemaVersion": 1,
"generator": "generate_fonts.py (ReportLab UnicodeCIDFont, invariant PDF)",
"documents": [
{
"file": "unembedded-japanese.pdf",
"sha256": "9e74ac0c7d81a8c1d373cb28943a1943f8f68c0abd325e8e1012956bb71db057",
"pages": 2,
"sizePt": [
600,
760
],
"fontProgramsEmbedded": false,
"encodings": [
"/UniJIS-UCS2-H",
"/UniJIS-UCS2-V"
],
"faces": [
"HeiseiMin-W3",
"HeiseiKakuGo-W5"
],
"expectedText": [
"日本語の文書を表示します。",
"漢字とひらがな、カタカナ。",
"「縦書き」の句読点を確認。"
],
"writingModes": [
"horizontal",
"vertical"
]
}
]
}
tests/fixtures/pdf/fonts/unembedded-japanese.pdf
Vendored
+233
View File
@@ -0,0 +1,233 @@
%PDF-1.3
%“Œ‹ž ReportLab Generated PDF document (opensource)
1 0 obj
<<
/F1 2 0 R /F2 3 0 R /F3 4 0 R /F4 5 0 R /F5 7 0 R /F6 8 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
>>
endobj
4 0 obj
<<
/BaseFont /HeiseiMin-W3 /DescendantFonts [ <<
/BaseFont /HeiseiMin-W3 /CIDSystemInfo <<
/Ordering (Japan1) /Registry (Adobe) /Supplement 2
>> /DW 1000 /FontDescriptor <<
/Ascent 723 /CapHeight 709 /Descent -241 /Flags 6 /FontBBox [ -123 -257 1001 910 ] /FontName /HeiseiMin-W3
/ItalicAngle 0 /StemV 69 /Type /FontDescriptor /XHeight 450
>> /Subtype /CIDFontType0 /Type /Font
/W [ 1 [ 250 333 408 500 ] 5 [ 500 833 778 180 333 ] 10 [ 333 500 564 250 333 250 278 500 ] 18 26 500 27
28 278 29 31 564 32 [ 444 921 722 667 ] 36 [ 667 722 611 556 722 ] 41
[ 722 333 389 722 611 889 722 ] 48 [ 722 556 722 667 556 611 722 ] 55 [ 722 944 722 ] 58 [ 722 611 333 500 333 469 500 333 444 500
444 500 444 333 500 ] 73 [ 500 278 ] 75
[ 278 500 278 778 500 ] 80 82 500 83 [ 333 389 278 500 ] 87 [ 500 722 500 ] 90 [ 500 444 480 200 480 333 ]
97 [ 278 ] 99 [ 200 ] 101 [ 333 500 ] 103 [ 500 167 ] 107 [ 500 ]
109 [ 500 333 ] 111 [ 333 556 ] 113 [ 556 500 ] 117 [ 250 ] 119 [ 350 333 444 ]
123 [ 500 ] 126 [ 444 333 ] 128 137 333 138 [ 1000 889 276 611 722 889 310 667 278 ] 147
[ 278 500 722 500 564 760 564 760 ] 157 158 300 159 [ 500 300 750 ] 162 163 750 164
169 722 170 [ 667 611 ] 172 174 611 175 178 333
179 185 722 187 191 722 192 [ 556 444 ] 194 203
444 204 207 278 208 214 500 216 222 500
223 [ 556 722 611 500 389 980 444 ] 231 [ 500 ] 323 [ 500 ] 325 [ 500 ] 327 389
500 ]
>> ] /Encoding /UniJIS-UCS2-H /Name /F3 /Subtype /Type0 /Type /Font
>>
endobj
5 0 obj
<<
/BaseFont /HeiseiKakuGo-W5 /DescendantFonts [ <<
/BaseFont /HeiseiKakuGo-W5 /CIDSystemInfo <<
/Ordering (Japan1) /Registry (Adobe) /Supplement 2
>> /DW 1000 /FontDescriptor <<
/Ascent 752 /CapHeight 737 /Descent -221 /Flags 4 /FontBBox [ -92 -250 1010 922 ] /FontName /HeiseKakuGo-W5
/ItalicAngle 0 /StemH 0 /StemV 114 /Type /FontDescriptor /XHeight 553
>> /Subtype /CIDFontType0 /Type /Font
/W [ 1 [ 277 305 500 668 668 906 727 305 445 445
508 668 305 379 305 539 ] 17 26 668 27 [ 305 305 668 668 668 566 871 727 637 652
699 574 555 676 687 242 492 664 582 789
707 734 582 734 605 605 641 668 727 945
609 609 574 445 668 445 668 668 590 555
609 547 602 574 391 609 582 234 277 539
234 895 582 605 602 602 387 508 441 582
562 781 531 570 555 449 246 449 668 ] 231 632 500 ]
>> ] /Encoding /UniJIS-UCS2-H /Name /F4 /Subtype /Type0 /Type /Font
>>
endobj
6 0 obj
<<
/Contents 13 0 R /MediaBox [ 0 0 600 760 ] /Parent 12 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
7 0 obj
<<
/BaseFont /HeiseiMin-W3 /DescendantFonts [ <<
/BaseFont /HeiseiMin-W3 /CIDSystemInfo <<
/Ordering (Japan1) /Registry (Adobe) /Supplement 2
>> /DW 1000 /FontDescriptor <<
/Ascent 723 /CapHeight 709 /Descent -241 /Flags 6 /FontBBox [ -123 -257 1001 910 ] /FontName /HeiseiMin-W3
/ItalicAngle 0 /StemV 69 /Type /FontDescriptor /XHeight 450
>> /Subtype /CIDFontType0 /Type /Font
/W [ 1 [ 250 333 408 500 ] 5 [ 500 833 778 180 333 ] 10 [ 333 500 564 250 333 250 278 500 ] 18 26 500 27
28 278 29 31 564 32 [ 444 921 722 667 ] 36 [ 667 722 611 556 722 ] 41
[ 722 333 389 722 611 889 722 ] 48 [ 722 556 722 667 556 611 722 ] 55 [ 722 944 722 ] 58 [ 722 611 333 500 333 469 500 333 444 500
444 500 444 333 500 ] 73 [ 500 278 ] 75
[ 278 500 278 778 500 ] 80 82 500 83 [ 333 389 278 500 ] 87 [ 500 722 500 ] 90 [ 500 444 480 200 480 333 ]
97 [ 278 ] 99 [ 200 ] 101 [ 333 500 ] 103 [ 500 167 ] 107 [ 500 ]
109 [ 500 333 ] 111 [ 333 556 ] 113 [ 556 500 ] 117 [ 250 ] 119 [ 350 333 444 ]
123 [ 500 ] 126 [ 444 333 ] 128 137 333 138 [ 1000 889 276 611 722 889 310 667 278 ] 147
[ 278 500 722 500 564 760 564 760 ] 157 158 300 159 [ 500 300 750 ] 162 163 750 164
169 722 170 [ 667 611 ] 172 174 611 175 178 333
179 185 722 187 191 722 192 [ 556 444 ] 194 203
444 204 207 278 208 214 500 216 222 500
223 [ 556 722 611 500 389 980 444 ] 231 [ 500 ] 323 [ 500 ] 325 [ 500 ] 327 389
500 ]
>> ] /Encoding /UniJIS-UCS2-V /Name /F5 /Subtype /Type0 /Type /Font
>>
endobj
8 0 obj
<<
/BaseFont /HeiseiKakuGo-W5 /DescendantFonts [ <<
/BaseFont /HeiseiKakuGo-W5 /CIDSystemInfo <<
/Ordering (Japan1) /Registry (Adobe) /Supplement 2
>> /DW 1000 /FontDescriptor <<
/Ascent 752 /CapHeight 737 /Descent -221 /Flags 4 /FontBBox [ -92 -250 1010 922 ] /FontName /HeiseKakuGo-W5
/ItalicAngle 0 /StemH 0 /StemV 114 /Type /FontDescriptor /XHeight 553
>> /Subtype /CIDFontType0 /Type /Font
/W [ 1 [ 277 305 500 668 668 906 727 305 445 445
508 668 305 379 305 539 ] 17 26 668 27 [ 305 305 668 668 668 566 871 727 637 652
699 574 555 676 687 242 492 664 582 789
707 734 582 734 605 605 641 668 727 945
609 609 574 445 668 445 668 668 590 555
609 547 602 574 391 609 582 234 277 539
234 895 582 605 602 602 387 508 441 582
562 781 531 570 555 449 246 449 668 ] 231 632 500 ]
>> ] /Encoding /UniJIS-UCS2-V /Name /F6 /Subtype /Type0 /Type /Font
>>
endobj
9 0 obj
<<
/Contents 14 0 R /MediaBox [ 0 0 600 760 ] /Parent 12 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
10 0 obj
<<
/PageMode /UseNone /Pages 12 0 R /Type /Catalog
>>
endobj
11 0 obj
<<
/Author (DocView synthetic test corpus) /CreationDate (D:20000101000000+00'00') /Creator (anonymous) /Keywords () /ModDate (D:20000101000000+00'00') /Producer (ReportLab PDF Library - \(opensource\))
/Subject (unspecified) /Title (DocView unembedded Japanese CID H/V fixture) /Trapped /False
>>
endobj
12 0 obj
<<
/Count 2 /Kids [ 6 0 R 9 0 R ] /Type /Pages
>>
endobj
13 0 obj
<<
/Length 1124
>>
stream
1 0 0 1 0 0 cm BT /F1 12 Tf 14.4 TL ET
BT /F2 16 Tf 19.2 TL ET
BT 1 0 0 1 36 722 Tm (Unembedded Japanese CID fonts) Tj T* ET
BT /F1 11 Tf 13.2 TL ET
BT 1 0 0 1 36 700 Tm (UniJIS-UCS2-H: horizontal writing mode) Tj T* ET
BT /F1 10 Tf 12 TL ET
BT 1 0 0 1 36 648 Tm (HeiseiMin-W3) Tj T* ET
BT /F3 20 Tf 24 TL ET
BT 1 0 0 1 36 610 Tm /F3 20 Tf 24 TL (e\345g,\212\2360ne\207f\3700\222\210hy:0W0~0Y0\002) Tj T* ET
BT 1 0 0 1 36 568 Tm /F3 20 Tf 24 TL (o"[W0h0r0\2110L0j0\0010\2530\2770\2530\3120\002) Tj T* ET
BT 1 0 0 1 36 526 Tm /F3 20 Tf 24 TL (0\014~&f\3700M0\0150nS\345\212\255p\2710\222x\272\212\2150\002) Tj T* ET
BT /F1 10 Tf 12 TL ET
BT 1 0 0 1 36 388 Tm (HeiseiKakuGo-W5) Tj T* ET
BT /F4 20 Tf 24 TL ET
BT 1 0 0 1 36 350 Tm /F4 20 Tf 24 TL (e\345g,\212\2360ne\207f\3700\222\210hy:0W0~0Y0\002) Tj T* ET
BT 1 0 0 1 36 308 Tm /F4 20 Tf 24 TL (o"[W0h0r0\2110L0j0\0010\2530\2770\2530\3120\002) Tj T* ET
BT 1 0 0 1 36 266 Tm /F4 20 Tf 24 TL (0\014~&f\3700M0\0150nS\345\212\255p\2710\222x\272\212\2150\002) Tj T* ET
BT /F1 9 Tf 10.8 TL ET
BT 1 0 0 1 36 28 Tm (Synthetic fixture / page 1 / no font program embedded) Tj T* ET
endstream
endobj
14 0 obj
<<
/Length 1224
>>
stream
1 0 0 1 0 0 cm BT /F1 12 Tf 14.4 TL ET
BT /F2 16 Tf 19.2 TL ET
BT 1 0 0 1 36 722 Tm (Unembedded Japanese CID fonts) Tj T* ET
BT /F1 11 Tf 13.2 TL ET
BT 1 0 0 1 36 700 Tm (UniJIS-UCS2-V: native vertical writing mode) Tj T* ET
BT /F1 10 Tf 12 TL ET
BT 1 0 0 1 330 660 Tm (HeiseiMin-W3) Tj T* ET
BT /F5 20 Tf 24 TL ET
BT 1 0 0 1 520 620 Tm /F5 20 Tf 24 TL (e\345g,\212\2360ne\207f\3700\222\210hy:0W0~0Y0\002) Tj T* ET
BT /F5 20 Tf 24 TL ET
BT 1 0 0 1 458 620 Tm /F5 20 Tf 24 TL (o"[W0h0r0\2110L0j0\0010\2530\2770\2530\3120\002) Tj T* ET
BT /F5 20 Tf 24 TL ET
BT 1 0 0 1 396 620 Tm /F5 20 Tf 24 TL (0\014~&f\3700M0\0150nS\345\212\255p\2710\222x\272\212\2150\002) Tj T* ET
BT /F1 10 Tf 12 TL ET
BT 1 0 0 1 50 660 Tm (HeiseiKakuGo-W5) Tj T* ET
BT /F6 20 Tf 24 TL ET
BT 1 0 0 1 240 620 Tm /F6 20 Tf 24 TL (e\345g,\212\2360ne\207f\3700\222\210hy:0W0~0Y0\002) Tj T* ET
BT /F6 20 Tf 24 TL ET
BT 1 0 0 1 178 620 Tm /F6 20 Tf 24 TL (o"[W0h0r0\2110L0j0\0010\2530\2770\2530\3120\002) Tj T* ET
BT /F6 20 Tf 24 TL ET
BT 1 0 0 1 116 620 Tm /F6 20 Tf 24 TL (0\014~&f\3700M0\0150nS\345\212\255p\2710\222x\272\212\2150\002) Tj T* ET
BT /F1 9 Tf 10.8 TL ET
BT 1 0 0 1 36 28 Tm (Synthetic fixture / page 2 / no font program embedded) Tj T* ET
endstream
endobj
xref
0 15
0000000000 65535 f
0000000061 00000 n
0000000142 00000 n
0000000249 00000 n
0000000361 00000 n
0000001864 00000 n
0000002754 00000 n
0000002949 00000 n
0000004452 00000 n
0000005342 00000 n
0000005537 00000 n
0000005607 00000 n
0000005924 00000 n
0000005990 00000 n
0000007166 00000 n
trailer
<<
/ID
[<538e4832305af263a0a7344d39731b90><538e4832305af263a0a7344d39731b90>]
% ReportLab generated PDF document -- digest (opensource)
/Info 11 0 R
/Root 10 0 R
/Size 15
>>
startxref
8442
%%EOF
tests/fixtures/pdf/generate.py
Vendored
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""Generate DocView's small synthetic PDF corpus (no third-party content).
The coordinates, tags, appearances and security actions are intentional and are
asserted by test_pdf. qpdf is needed only to regenerate the encrypted fixture.
"""
from pathlib import Path
import subprocess
root = Path(__file__).parent
def stream(data, extra=''):
data = data.encode('ascii')
return f'<< /Length {len(data)} {extra} >>\nstream\n'.encode() + data + b'\nendstream'
def write_pdf(name, objects):
result = bytearray(b'%PDF-1.7\n%\xe2\xe3\xcf\xd3\n')
offsets = [0]
for index, obj in enumerate(objects, 1):
offsets.append(len(result))
if isinstance(obj, str): obj = obj.encode('ascii')
result += f'{index} 0 obj\n'.encode() + obj + b'\nendobj\n'
xref = len(result)
result += f'xref\n0 {len(offsets)}\n0000000000 65535 f \n'.encode()
for offset in offsets[1:]: result += f'{offset:010d} 00000 n \n'.encode()
result += f'trailer\n<< /Size {len(offsets)} /Root 1 0 R /Info 19 0 R >>\nstartxref\n{xref}\n%%EOF\n'.encode()
(root / name).write_bytes(result)
objects = [
'<< /Type /Catalog /Pages 2 0 R /Outlines 8 0 R /PageLabels << /Nums [0 << /S /r >> 1 << /S /D /St 1 >>] >> /StructTreeRoot 11 0 R /MarkInfo << /Marked true >> /AcroForm << /Fields [16 0 R] /DR << /Font << /F1 7 0 R >> >> /DA (/F1 12 Tf 0 g) >> /Names << /Dests << /Names [(second) [4 0 R /XYZ 40 200 null]] >> >> >>',
'<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 240 260] /CropBox [20 30 220 230] /Resources << /Font << /F1 7 0 R >> >> /Contents 5 0 R /StructParents 0 /Annots [14 0 R 15 0 R 16 0 R 18 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Rotate 90 /Resources << /Font << /F1 7 0 R >> >> /Contents 6 0 R >>',
stream('0 g 40 50 40 40 re f\n/Chapter <</MCID 0>> BDC BT /F1 16 Tf 60 180 Td (Heading One) Tj ET EMC\n/H2 <</MCID 1>> BDC BT /F1 12 Tf 60 120 Td (Needle) Tj ET EMC\n/H2 <</MCID 2>> BDC BT /F1 12 Tf 60 100 Td (Second MCID) Tj ET EMC'),
stream('BT /F1 20 Tf 40 100 Td (Second Page Needle) Tj ET'),
'<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>',
'<< /Type /Outlines /First 9 0 R /Last 10 0 R /Count 2 >>',
'<< /Title (First heading) /Parent 8 0 R /Dest [3 0 R /XYZ 60 180 null] /Next 10 0 R >>',
'<< /Title (Named second) /Parent 8 0 R /Dest (second) /Prev 9 0 R >>',
'<< /Type /StructTreeRoot /K [12 0 R 13 0 R] /RoleMap << /Chapter /H1 >> /ParentTree << /Nums [0 [12 0 R 13 0 R 13 0 R]] >> >>',
'<< /Type /StructElem /S /Chapter /P 11 0 R /Pg 3 0 R /K 0 >>',
'<< /Type /StructElem /S /H2 /P 11 0 R /Pg 3 0 R /K [1 2] >>',
'<< /Type /Annot /Subtype /Link /Rect [30 30 90 45] /A << /S /URI /URI (https://example.org/) >> >>',
'<< /Type /Annot /Subtype /Link /Rect [100 30 160 45] /A << /S /Launch /F (/tmp/never-execute) >> >>',
'<< /Type /Annot /Subtype /Widget /FT /Tx /T (ReadonlyField) /V (Saved) /Ff 1 /F 4 /Rect [120 60 160 90] /P 3 0 R /AP << /N 17 0 R >> >>',
stream('1 0 0 rg 0 0 40 30 re f', '/Type /XObject /Subtype /Form /BBox [0 0 40 30] /Resources << >>'),
'<< /Type /Annot /Subtype /Text /Rect [180 200 200 220] /Contents (Read-only comment) /F 4 >>',
'<< /Title (DocView PDF fixture) /Author (DocView tests) >>'
]
write_pdf('navigation.pdf', objects)
subprocess.run(['qpdf', '--encrypt', 'reader', 'owner-secret', '256', '--', str(root / 'navigation.pdf'), str(root / 'password.pdf')], check=True)
(root / 'corrupt.pdf').write_bytes(b'%PDF-1.7\nThis is deliberately corrupt.\n')
missing = list(objects)
missing[15] = missing[15].replace(' /AP << /N 17 0 R >>', '')
write_pdf('missing-appearance.pdf', missing)
reused = list(objects)
reused[2] = reused[2].replace('/Resources <<', '/Resources << /XObject << /Fm 20 0 R >>', 1)
reused[4] = stream('0 g /Chapter <</MCID 0>> BDC BT /F1 16 Tf 60 180 Td (Heading One) Tj ET EMC /Fm Do')
reused[11] = reused[11].replace('/S /Chapter', '/S /Chapter /T (Original Heading)')
reused.append(stream('/H1 <</MCID 0>> BDC BT /F1 12 Tf 50 50 Td (Different Form Heading) Tj ET EMC', '/Type /XObject /Subtype /Form /BBox [0 0 240 260] /Resources << /Font << /F1 7 0 R >> >>'))
write_pdf('reused-form-mcid.pdf', reused)
broken_pages = list(objects)
broken_pages[1] = broken_pages[1].replace('/Kids [3 0 R 4 0 R]', '/Kids [null 4 0 R]')
write_pdf('missing-first-page.pdf', broken_pages)
tests/fixtures/pdf/generate_annotation_flags.py
Vendored
+99
View File
@@ -0,0 +1,99 @@
#!/usr/bin/env python3
"""Deterministic, self-authored PDF annotation geometry; no external content."""
from pathlib import Path
import hashlib
import json
root = Path(__file__).parent / "annotation-flags"
root.mkdir(exist_ok=True)
objects = ["", ""]
def add(value):
objects.append(value.encode("ascii") if isinstance(value, str) else value)
return len(objects)
def stream(content, extra):
data = content.encode("ascii")
return add(f"<< /Length {len(data)} {extra} >>\nstream\n".encode() + data + b"\nendstream")
# Non-identity AP matrix, named graphics state, and selected appearance state
# must survive rectangle compensation. Normalization still yields red left /
# blue right in the annotation's 32x16 rectangle; Off is deliberately green.
normal = stream("q /Opaque gs 1 0 0 rg 0 0 16 16 re f 0 0 1 rg 16 0 16 16 re f Q",
"/Type /XObject /Subtype /Form /BBox [0 0 32 16] /Matrix [2 0 0 3 7 11] "
"/Resources << /ExtGState << /Opaque << /Type /ExtGState /ca 1 /CA 1 >> >> >>")
green = stream("0 1 0 rg 0 0 32 16 re f", "/Type /XObject /Subtype /Form /BBox [0 0 32 16] /Resources << >>")
pages, cases = [], []
def page(annots, rotate, case):
pid = add(f"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate {rotate} "
f"/Resources << >> /Annots [{' '.join(f'{x} 0 R' for x in annots)}] >>")
pages.append(pid)
cases.append(dict(page=len(pages)-1, intrinsicRotation=rotate, **case))
for kind in ["generated-link", "stored-link", "stored-comment"]:
for flags in [8, 16, 24]:
for rotate in [0, 90, 180, 270]:
extra = ("/C [1 0 0] /BS << /W 2 /S /S >>" if kind == "generated-link" else
f"/AP << /N << /On {normal} 0 R /Off {green} 0 R >> >> /AS /On")
subtype = "Stamp" if kind == "stored-comment" else "Link"
annot = add(f"<< /Type /Annot /Subtype /{subtype} /Rect [42 48 74 64] /F {flags} "
f"/Contents ({kind} flags {flags}) /A << /S /URI /URI (https://example.org/flags) >> {extra} >>")
page([annot], rotate, dict(kind=kind, flags=flags, rect=[42,48,74,64]))
# Annots order is retained: the later green appearance covers the generated
# border; Hidden / NoView appearances at distinct rectangles remain invisible.
annots = [add("<< /Type /Annot /Subtype /Link /Rect [42 48 74 64] /F 24 /C [1 0 0] /BS << /W 2 >> >>"),
add(f"<< /Type /Annot /Subtype /Link /Rect [41 47 75 65] /F 24 /AP << /N {green} 0 R >> >>")]
for flag, rect in [(26, "20 80 36 96"), (56, "80 80 96 96")]:
annots.append(add(f"<< /Type /Annot /Subtype /Link /Rect [{rect}] /F {flag} /Contents (Invisible comment) /AP << /N {normal} 0 R >> >>"))
page(annots, 0, dict(kind="visibility-zorder", flags=24))
# Right-edge clipping of an upright unscaled AP remains clipped to the CropBox.
edge = add(f"<< /Type /Annot /Subtype /Link /Rect [108 48 140 64] /F 24 /AP << /N {normal} 0 R >> >>")
page([edge], 0, dict(kind="crop-clipping", flags=24, rect=[108,48,140,64]))
objects[0] = b"<< /Type /Catalog /Pages 2 0 R >>"
objects[1] = f"<< /Type /Pages /Count {len(pages)} /Kids [{' '.join(f'{p} 0 R' for p in pages)}] >>".encode()
data = bytearray(b"%PDF-1.7\n%\xe2\xe3\xcf\xd3\n")
offsets = [0]
for index, value in enumerate(objects, 1):
offsets.append(len(data))
data += f"{index} 0 obj\n".encode() + value + b"\nendobj\n"
xref = len(data)
data += f"xref\n0 {len(offsets)}\n0000000000 65535 f \n".encode()
for offset in offsets[1:]: data += f"{offset:010d} 00000 n \n".encode()
data += f"trailer\n<< /Size {len(offsets)} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n".encode()
(root / "flags.pdf").write_bytes(data)
(root / "manifest.json").write_text(json.dumps(dict(generator="generate_annotation_flags.py",
license="Self-created test content; same license as DocView.", sha256=hashlib.sha256(data).hexdigest(),
cropBox=[8,12,120,116], cases=cases, expected="At 100% one PDF point is one logical pixel. "
"NoZoom preserves logical 32x16 size across zoom and scales only with DPR. NoRotate preserves upright "
"red-left/blue-right appearance around the original Rect upper-left through intrinsic plus user rotation. "
"Generated red border extends one point beyond Rect. This is synthetic geometry, not a general PDF compatibility golden."), indent=2)+"\n")
# Native Text icons have no supplied AP. Shared annotation objects exercise
# capture-once behavior across rotated pages without accumulating AP streams.
text_objects = [b"<< /Type /Catalog /Pages 2 0 R >>", b""]
text_content = b"BT /F1 10 Tf 12 100 Td (Native icon with unembedded body text) Tj ET"
text_objects.append(f"<< /Length {len(text_content)} >>\nstream\n".encode() + text_content + b"\nendstream")
text_objects.append(b"<< /Type /Annot /Subtype /Ink /Rect [80 24 100 34] /F 8 /Contents (Native ink) /BS << /W 4 >> /InkList [[80 24 90 34 100 24]] >>")
text_pages = []
for flags in [8,16,24]:
aid = len(text_objects) + 1
text_objects.append(f"<< /Type /Annot /Subtype /Text /Rect [42 48 74 64] /F {flags} /Contents (Native text icon) /C [1 0 0] >>".encode())
for rotation in [0,90,180,270]:
text_pages.append(len(text_objects)+1)
text_objects.append(f"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 128 128] /CropBox [8 12 120 116] /Rotate {rotation} "
f"/Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Times-Roman >> >> >> /Contents 3 0 R /Annots [{aid} 0 R 4 0 R 4 0 R] >>".encode())
text_objects[1] = f"<< /Type /Pages /Count 12 /Kids [{' '.join(f'{p} 0 R' for p in text_pages)}] >>".encode()
data = bytearray(b"%PDF-1.7\n")
offsets = [0]
for index, value in enumerate(text_objects, 1):
offsets.append(len(data))
data += f"{index} 0 obj\n".encode() + value + b"\nendobj\n"
xref = len(data)
data += f"xref\n0 {len(offsets)}\n0000000000 65535 f \n".encode()
for offset in offsets[1:]: data += f"{offset:010d} 00000 n \n".encode()
data += f"trailer\n<< /Size {len(offsets)} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n".encode()
(root / "text-icons.pdf").write_bytes(data)
(root / "text-icons-manifest.json").write_text(json.dumps(dict(license="Self-created test content; same license as DocView.",
sha256=hashlib.sha256(data).hexdigest(), originalRect=[42,48,74,64], nativeIconRect=[42,48,62,68],
expected="Twelve pages: flags 8/16/24, each with intrinsic rotations 0/90/180/270. Four pages share each annotation. "
"Capture the native generated AP once, retain source Rect, use the captured 20x20 icon Rect for drawing and hit testing."), indent=2)+"\n")
tests/fixtures/pdf/generate_cmyk_lut.py
Vendored
+148
View File
@@ -0,0 +1,148 @@
#!/usr/bin/env python3
"""Author one CMYK ICCBased PDF with a known four-dimensional Lab CLUT.
The ICC v2 input profile is encoded directly from the ICC specification;
no renderer/CMM output or borrowed press profile defines the expected values.
"""
import argparse
from io import BytesIO
import hashlib
import itertools
import json
from pathlib import Path
import struct
from pypdf import PdfReader, PdfWriter
from pypdf.generic import ArrayObject, DecodedStreamObject, DictionaryObject, NameObject, NumberObject
from reportlab.pdfgen import canvas
WIDTH, HEIGHT = 720, 420
SAMPLES = [(0, 0, 0, 0), (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0),
(0, 0, 0, 1), (.25, .5, .75, .2), (.6, .2, .4, .1), (1, 1, 1, 1)]
def model(cmyk):
c, m, y, k = cmyk
return 100 - 12*c - 15*m - 10*y - 35*k, 18*m - 15*c, 20*y - 12*c
def expected_rgb(cmyk):
# CIELAB inverse, D50 -> D65 Bradford adaptation, then standard sRGB.
lightness, a, b = model(cmyk)
fy = (lightness + 16) / 116
f = (fy + a/500, fy, fy - b/200)
xyz = [white * (v**3 if v > 6/29 else 3*(6/29)**2*(v-4/29))
for white, v in zip((.9642, 1, .8249), f)]
def multiply(matrix, vector):
return [sum(a*b for a, b in zip(row, vector)) for row in matrix]
d65 = multiply(((.9555766, -.0230393, .0631636), (-.0282895, 1.0099416, .0210077),
(.0122982, -.0204830, 1.3299098)), xyz)
linear = multiply(((3.2404542, -1.5371385, -.4985314), (-.969266, 1.8760108, .041556),
(.0556434, -.2040259, 1.0572252)), d65)
return [round(255*max(0, min(1, 12.92*v if v <= .0031308 else 1.055*v**(1/2.4)-.055))) for v in linear]
def profile_bytes():
def fixed(values):
return struct.pack('>' + 'i'*len(values), *(round(v*65536) for v in values))
header = bytearray(128)
header[8:12] = bytes.fromhex('02100000')
header[12:24] = b'scnrCMYKLab '
header[24:36] = struct.pack('>6H', 2026, 9, 20, 0, 0, 0)
header[36:40] = b'acsp'
header[68:80] = fixed((.9642, 1, .8249))
header[80:84] = b'DCVW'
description = b'DocView synthetic CMYK Lab CLUT\0'
desc = b'desc' + bytes(4) + struct.pack('>I', len(description)) + description + bytes(78)
copyright_tag = b'text' + bytes(4) + b'No copyright, use freely. Synthetic test profile; not a press characterization.\0'
table = bytearray(b'mft2' + bytes(4) + bytes((4, 3, 2, 0)))
table += fixed((1, 0, 0, 0, 1, 0, 0, 0, 1)) + struct.pack('>HH', 2, 2)
table += struct.pack('>HH', 0, 65535) * 4
for cmyk in itertools.product((0, 1), repeat=4):
lightness, a, b = model(cmyk)
table += struct.pack('>3H', round(lightness/100*65280), round((a+128)*256), round((b+128)*256))
table += struct.pack('>HH', 0, 65535) * 3
assert len(table) == 176
tags = [(b'desc', desc), (b'cprt', copyright_tag),
(b'wtpt', b'XYZ ' + bytes(4) + fixed((.9642, 1, .8249))), (b'A2B0', bytes(table))]
body = bytearray(); entries = bytearray(); offset = 132 + 12*len(tags)
for signature, data in tags:
entries += signature + struct.pack('>II', offset+len(body), len(data))
body += data + bytes((-len(data)) % 4)
result = header + struct.pack('>I', len(tags)) + entries + body
result[:4] = struct.pack('>I', len(result))
return bytes(result)
def generate(output):
output.mkdir(parents=True, exist_ok=False)
profile = profile_bytes()
(output / 'synthetic-cmyk-lab.icc').write_bytes(profile)
buffer = BytesIO()
page = canvas.Canvas(buffer, pagesize=(WIDTH, HEIGHT), invariant=1, pageCompression=0)
page.setTitle('ICC CMYK - four-dimensional lookup table')
page.setAuthor('DocView test corpus')
page.setFont('Helvetica-Bold', 18)
page.drawString(36, 385, 'ICC CMYK - FOUR-DIMENSIONAL LOOKUP TABLE')
page.setFont('Helvetica', 10)
page.drawString(36, 365, 'Synthetic CMYK to Lab mapping. Fixed D50 white point. No press calibration.')
for y, label in [(336, 'A ICCBased vector'), (238, 'B ICCBased image (8-bit components)'),
(140, 'C DeviceCMYK control (profile not applied)')]:
page.setFont('Helvetica-Bold', 10); page.drawString(36, y, label)
page.setFont('Helvetica', 9)
for i, label in enumerate(['White', 'Cyan', 'Magenta', 'Yellow', 'Black', 'Mix A', 'Mix B', 'All inks']):
page.drawCentredString(36+i*81+35, 48, label)
page.setFont('Helvetica', 9)
page.drawString(36, 25, 'A/B must agree with the analytic Lab mapping. C deliberately uses a different color space.')
page.save()
writer = PdfWriter(); writer.append(PdfReader(BytesIO(buffer.getvalue())))
target = writer.pages[0]
icc = DecodedStreamObject(); icc.set_data(profile)
icc.update({NameObject('/N'): NumberObject(4), NameObject('/Alternate'): NameObject('/DeviceCMYK')})
icc_ref = writer._add_object(icc)
space = ArrayObject([NameObject('/ICCBased'), icc_ref])
target['/Resources'][NameObject('/ColorSpace')] = DictionaryObject({NameObject('/TestCMYK'): space})
image = DecodedStreamObject(); image.set_data(bytes(round(v*255) for cmyk in SAMPLES for v in cmyk))
image.update({NameObject('/Type'): NameObject('/XObject'), NameObject('/Subtype'): NameObject('/Image'),
NameObject('/Width'): NumberObject(8), NameObject('/Height'): NumberObject(1),
NameObject('/BitsPerComponent'): NumberObject(8), NameObject('/ColorSpace'): space,
NameObject('/Intent'): NameObject('/RelativeColorimetric')})
target['/Resources'][NameObject('/XObject')] = DictionaryObject({NameObject('/Sample'): writer._add_object(image)})
probes, commands = [], ['/RelativeColorimetric ri']
for i, components in enumerate(SAMPLES):
x = 36+i*81
operands = ' '.join(f'{v:g}' for v in components)
commands += [f'/TestCMYK cs {operands} scn {x} 260 70 64 re f',
f'{operands} k {x} 64 70 64 re f']
probes.append({'kind': 'icc-vector', 'sample': i, 'pointPt': [x+35, 292], 'components': list(components),
'lab': list(model(components)), 'expectedRgb': expected_rgb(components)})
quantized = [round(v*255)/255 for v in components]
probes.append({'kind': 'icc-image', 'sample': i, 'pointPt': [36+(i+.5)*81, 194], 'components': quantized,
'lab': list(model(quantized)), 'expectedRgb': expected_rgb(quantized)})
probes.append({'kind': 'device-control', 'sample': i, 'pointPt': [x+35, 96], 'components': list(components)})
commands += ['q 648 0 0 64 36 162 cm /Sample Do Q']
content = DecodedStreamObject(); content.set_data(('\n'.join(commands)+'\n').encode())
target[NameObject('/Contents')] = ArrayObject([target.raw_get('/Contents'), writer._add_object(content)])
pdf = output / 'icc-cmyk-lut.pdf'
with pdf.open('wb') as stream: writer.write(stream)
def digest(data): return hashlib.sha256(data).hexdigest()
record = {'file': pdf.name, 'sha256': digest(pdf.read_bytes()), 'profileFile': 'synthetic-cmyk-lab.icc',
'profileSha256': digest(profile), 'profileBytes': len(profile), 'pageSizePt': [WIDTH, HEIGHT],
'profile': {'version': '2.1', 'class': 'scnr', 'input': 'CMYK', 'pcs': 'Lab ',
'tag': 'A2B0', 'type': 'mft2', 'gridPointsPerChannel': 2,
'mapping': ['L=100-12*C-15*M-10*Y-35*K', 'a=18*M-15*C', 'b=20*Y-12*C'],
'labEncoding': 'ICC v2 L: 0..65280 for 0..100; a/b: (value+128)*256',
'source': 'Self-authored binary ICC input profile; no borrowed ICC profile'},
'probes': probes, 'tolerance8Bit': 4,
'references': ['https://archive.color.org/files/icc32.pdf', 'https://www.color.org/chardata/rgb/srgb.xalter'],
'limits': ['Synthetic CMYK input CLUT; not a measured print profile or calibrated press proof.',
'No human-approved golden; no physical display or Windows validation.'],
'generatorSha256': digest(Path(__file__).read_bytes())}
(output / 'manifest.json').write_text(json.dumps(record, indent=2) + '\n')
print(json.dumps({'fixtureSha256': record['sha256'], 'profileSha256': record['profileSha256'], 'probes': len(probes)}))
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--output', type=Path, required=True)
generate(parser.parse_args().output)
tests/fixtures/pdf/generate_comments.py
Vendored
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env python3
"""Generate synthetic read-only comment keyboard fixtures, without dependencies."""
from pathlib import Path
objects = [
b'<< /Type /Catalog /Pages 2 0 R >>',
b'<< /Type /Pages /Kids [3 0 R] /Count 1 >>',
b'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 240 260] /CropBox [20 30 220 230] /Annots [4 0 R 5 0 R] /Resources << >> >>',
b'<< /Type /Annot /Subtype /Text /Rect [100 100 120 120] /Contents (<b>Literal comment</b>) /F 4 >>',
b'<< /Type /Annot /Subtype /Text /Rect [1000 1000 1020 1020] /Contents (Outside CropBox comment) /F 4 >>',
]
data = bytearray(b'%PDF-1.7\n%\xe2\xe3\xcf\xd3\n')
offsets = [0]
for index, obj in enumerate(objects, 1):
offsets.append(len(data))
data += f'{index} 0 obj\n'.encode() + obj + b'\nendobj\n'
xref = len(data)
data += f'xref\n0 {len(offsets)}\n0000000000 65535 f \n'.encode()
for offset in offsets[1:]:
data += f'{offset:010d} 00000 n \n'.encode()
data += f'trailer\n<< /Size {len(offsets)} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n'.encode()
Path(__file__).with_name('comments.pdf').write_bytes(data)
# Sub-pixel page at 25% zoom: keyboard focus must not invert scroll bounds.
tiny = bytes(data)
for original, replacement in [(b'[0 0 240 260]', b'[0 0 1 1]'),
(b'[20 30 220 230]', b'[0 0 1 1]'),
(b'[100 100 120 120]', b'[0 0 0.5 0.5]'),
(b'[1000 1000 1020 1020]', b'[0 0 0.5 0.5]')]:
tiny = tiny.replace(original, replacement.ljust(len(original), b' '))
# Preserve fixed byte widths so the original xref remains exact.
assert len(tiny) == len(data)
Path(__file__).with_name('tiny-comments.pdf').write_bytes(tiny)
tests/fixtures/pdf/generate_fonts.py
Vendored
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env python3
"""Author deterministic unembedded Japanese CID H/V fixtures (ReportLab)."""
import hashlib
import json
from pathlib import Path
from pypdf import PdfReader
from reportlab.pdfbase import pdfmetrics
from reportlab.pdfbase.cidfonts import UnicodeCIDFont
from reportlab.pdfgen.canvas import Canvas
ROOT = Path(__file__).resolve().parent / "fonts"
ROOT.mkdir(exist_ok=True)
OUTPUT = ROOT / "unembedded-japanese.pdf"
TEXTS = ["日本語の文書を表示します。", "漢字とひらがな、カタカナ。", "「縦書き」の句読点を確認。"]
for face in ("HeiseiMin-W3", "HeiseiKakuGo-W5"):
for vertical in (False, True):
font = UnicodeCIDFont(face, isVertical=vertical)
font.name = font.fontName = face + ("-V" if vertical else "-H")
pdfmetrics.registerFont(font)
canvas = Canvas(str(OUTPUT), pagesize=(600, 760), invariant=1, pageCompression=0)
canvas.setTitle("DocView unembedded Japanese CID H/V fixture")
canvas.setAuthor("DocView synthetic test corpus")
for page, vertical in enumerate((False, True), 1):
canvas.setFont("Helvetica-Bold", 16)
canvas.drawString(36, 722, "Unembedded Japanese CID fonts")
canvas.setFont("Helvetica", 11)
canvas.drawString(36, 700, "UniJIS-UCS2-" + ("V: native vertical writing mode" if vertical else "H: horizontal writing mode"))
for row, face in enumerate(("HeiseiMin-W3", "HeiseiKakuGo-W5")):
canvas.setFont("Helvetica", 10)
if vertical:
canvas.drawString(330 - row * 280, 660, face)
for col, text in enumerate(TEXTS):
canvas.setFont(face + "-V", 20)
canvas.drawString(520 - row * 280 - col * 62, 620, text)
else:
canvas.drawString(36, 648 - row * 260, face)
canvas.setFont(face + "-H", 20)
for line, text in enumerate(TEXTS):
canvas.drawString(36, 610 - row * 260 - line * 42, text)
canvas.setFont("Helvetica", 9)
canvas.drawString(36, 28, f"Synthetic fixture / page {page} / no font program embedded")
canvas.showPage()
canvas.save()
reader = PdfReader(OUTPUT)
encodings = set()
for page in reader.pages:
for ref in page["/Resources"]["/Font"].values():
font = ref.get_object()
if font.get("/Subtype") != "/Type0":
continue
encodings.add(str(font["/Encoding"]))
for descendant in font["/DescendantFonts"]:
descriptor = descendant.get_object()["/FontDescriptor"].get_object()
assert not any(key in descriptor for key in ("/FontFile", "/FontFile2", "/FontFile3"))
assert encodings == {"/UniJIS-UCS2-H", "/UniJIS-UCS2-V"}, encodings
manifest = {
"schemaVersion": 1,
"generator": "generate_fonts.py (ReportLab UnicodeCIDFont, invariant PDF)",
"documents": [{
"file": OUTPUT.name,
"sha256": hashlib.sha256(OUTPUT.read_bytes()).hexdigest(),
"pages": 2,
"sizePt": [600, 760],
"fontProgramsEmbedded": False,
"encodings": sorted(encodings),
"faces": ["HeiseiMin-W3", "HeiseiKakuGo-W5"],
"expectedText": TEXTS,
"writingModes": ["horizontal", "vertical"],
}],
}
(ROOT / "manifest.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n")
print(json.dumps(manifest, ensure_ascii=False, indent=2))
tests/fixtures/pdf/generate_headings.py
Vendored
+230
View File
@@ -0,0 +1,230 @@
#!/usr/bin/env python3
"""Deterministic, self-authored tagged-PDF boundary corpus (no external tools)."""
from pathlib import Path
import hashlib
import json
import zlib
ROOT = Path(__file__).parent / 'headings'
ROOT.mkdir(exist_ok=True)
def stream(text, extra=''):
data = text.encode('ascii')
return f'<< /Length {len(data)} {extra} >>\nstream\n'.encode() + data + b'\nendstream'
def write(name, objects, expected):
data = bytearray(b'%PDF-1.7\n%\xe2\xe3\xcf\xd3\n')
offsets = [0]
for number, obj in enumerate(objects, 1):
offsets.append(len(data))
data += f'{number} 0 obj\n'.encode() + (obj.encode('ascii') if isinstance(obj, str) else obj) + b'\nendobj\n'
xref = len(data)
data += f'xref\n0 {len(offsets)}\n0000000000 65535 f \n'.encode()
for offset in offsets[1:]:
data += f'{offset:010d} 00000 n \n'.encode()
data += f'trailer\n<< /Size {len(offsets)} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n'.encode()
(ROOT / name).write_bytes(data)
return {'file': name, 'bytes': len(data), 'sha256': hashlib.sha256(data).hexdigest(), 'expected': expected}
font = '<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>'
records = []
records.append(write('transformed-form.pdf', [
'<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 7 0 R >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 0 /Resources << /XObject << /Fm 6 0 R >> >> /Contents 4 0 R >>',
stream('/H2 <</MCID 0>> BDC q 1.5 0 0 1.25 55 70 cm /Fm Do Q EMC'),
font,
stream('/H2 <</MCID 4>> BDC BT /F1 16 Tf 10 30 Td (Form heading) Tj ET EMC', '/Type /XObject /Subtype /Form /BBox [0 0 160 80] /StructParents 1 /Resources << /Font << /F1 5 0 R >> >>'),
'<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Nums [0 [8 0 R] 1 [null null null null 8 0 R]] >> /ParentTreeNextKey 2 >>',
'<< /Type /StructElem /S /H2 /P 7 0 R /Pg 3 0 R /T (Form heading) /K [0 << /Type /MCR /Pg 3 0 R /Stm 6 0 R /MCID 4 >>] >>'
], {'pages': 1, 'headings': [{'page': 0, 'title': 'Form heading', 'level': 2, 'precision': 'exact', 'text_region_pt': [70, 102, 240, 130]}], 'form_matrix': [1.5, 0, 0, 1.25, 55, 70], 'local_baseline': [10, 30]}))
records.append(write('rotated-tags.pdf', [
'<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 260] /CropBox [20 30 280 230] /Rotate 90 /StructParents 0 /Resources << /Font << /F1 5 0 R >> >> /Contents 4 0 R >>',
stream('/H3 <</MCID 1>> BDC BT /F1 16 Tf 60 80 Td (Rotated second) Tj ET EMC\n/H1 <</MCID 0>> BDC BT /F1 20 Tf 50 170 Td (Rotated first) Tj ET EMC'),
font,
'<< /Type /StructTreeRoot /K [7 0 R 8 0 R] /ParentTree << /Nums [0 [7 0 R 8 0 R]] >> >>',
'<< /Type /StructElem /S /H1 /P 6 0 R /Pg 3 0 R /K 0 >>',
'<< /Type /StructElem /S /H3 /P 6 0 R /Pg 3 0 R /K 1 >>'
], {'pages': 1, 'crop_box': [20, 30, 280, 230], 'rotation': 90, 'headings': [
{'page': 0, 'title': 'Rotated first', 'level': 1, 'precision': 'exact', 'text_region_pt': [50, 168, 170, 190]},
{'page': 0, 'title': 'Rotated second', 'level': 3, 'precision': 'exact', 'text_region_pt': [60, 78, 180, 96]}
], 'order': 'structure order, intentionally different from content-stream draw order'}))
records.append(write('cross-page-structure.pdf', [
'<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 8 0 R >>',
'<< /Type /Pages /Count 2 /Kids [3 0 R 4 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 0 /Resources << /Font << /F1 7 0 R >> >> /Contents 5 0 R >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 1 /Resources << /Font << /F1 7 0 R >> >> /Contents 6 0 R >>',
stream('/Span <</MCID 0>> BDC BT /F1 16 Tf 40 60 Td (Across page one) Tj ET EMC\n/H1 <</MCID 1>> BDC BT /F1 16 Tf 170 60 Td ( middle) Tj ET EMC'),
stream('/P <</MCID 2>> BDC BT /F1 16 Tf 40 120 Td (Unrelated paragraph) Tj ET EMC\n/H2 <</MCID 1>> BDC BT /F1 16 Tf 40 160 Td (Following heading) Tj ET EMC\n/Span <</MCID 0>> BDC BT /F1 16 Tf 40 210 Td (Across page two) Tj ET EMC'),
font,
'<< /Type /StructTreeRoot /K [9 0 R] /ParentTree << /Nums [0 [12 0 R 10 0 R] 1 [13 0 R 11 0 R 14 0 R]] >> /ParentTreeNextKey 2 >>',
'<< /Type /StructElem /S /Sect /P 8 0 R /K [10 0 R 11 0 R 14 0 R] >>',
'<< /Type /StructElem /S /H1 /P 9 0 R /T (Across pages) /K [12 0 R << /Type /MCR /Pg 3 0 R /MCID 1 >> 13 0 R] >>',
'<< /Type /StructElem /S /H2 /P 9 0 R /Pg 4 0 R /K 1 >>',
'<< /Type /StructElem /S /Span /P 10 0 R /Pg 3 0 R /K 0 >>',
'<< /Type /StructElem /S /Span /P 10 0 R /Pg 4 0 R /K 0 >>',
'<< /Type /StructElem /S /P /P 9 0 R /Pg 4 0 R /K 2 >>'
], {'pages': 2, 'headings': [
{'page': 0, 'title': 'Across page one middle', 'level': 1, 'precision': 'exact', 'text_region_pt': [40, 55, 230, 78]},
{'page': 1, 'title': 'Across page two', 'level': 1, 'precision': 'exact', 'text_region_pt': [40, 205, 170, 228]},
{'page': 1, 'title': 'Following heading', 'level': 2, 'precision': 'exact', 'text_region_pt': [40, 155, 180, 178]}
], 'scope': 'One heading spans two page-local Span elements with an interleaved direct MCR; MCID 0 is reused independently on both pages. Results are page-local fragments.'}))
records.append(write('vector-heading.pdf', [
'<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 5 0 R >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 0 /Resources << >> /Contents 4 0 R >>',
stream('/H4 <</MCID 0>> BDC 0 0 1 rg 40 180 150 20 re f EMC'),
'<< /Type /StructTreeRoot /K [6 0 R] /ParentTree << /Nums [0 [6 0 R]] >> >>',
'<< /Type /StructElem /S /H4 /P 5 0 R /Pg 3 0 R /ActualText (Vector section) /K 0 >>'
], {'pages': 1, 'headings': [{'page': 0, 'title': 'Vector section', 'level': 4, 'precision': 'page', 'reason_contains': '座標'}], 'drawn_region_pt': [40, 180, 190, 200], 'fallback': 'No text glyph bounding box is available for the marked vector path.'}))
records.append(write('form-only-structure.pdf', [
'<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 7 0 R >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /Resources << /XObject << /Fm 6 0 R >> >> /Contents 4 0 R >>',
stream('q 1 0 0 1 40 100 cm /Fm Do Q'),
font,
stream('/H2 <</MCID 0>> BDC BT /F1 16 Tf 10 30 Td (Form only heading) Tj ET EMC', '/Type /XObject /Subtype /Form /BBox [0 0 200 80] /StructParents 0 /Resources << /Font << /F1 5 0 R >> >>'),
'<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Nums [0 [8 0 R]] >> /ParentTreeNextKey 1 >>',
'<< /Type /StructElem /S /H2 /P 7 0 R /Pg 3 0 R /T (Form only heading) /K << /Type /MCR /Pg 3 0 R /Stm 6 0 R /MCID 0 >> >>'
], {'pages': 1, 'headings': [{'page': 0, 'title': 'Form only heading', 'level': 2, 'precision': 'exact', 'text_region_pt': [50, 125, 190, 147]}],
'scope': 'Form StructParents and source-qualified MCR; no page StructParents.'}))
records.append(write('form-vector-mcid-collision.pdf', [
'<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 7 0 R >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 0 /Resources << /XObject << /Fm 6 0 R >> >> /Contents 4 0 R >>',
stream('/H1 <</MCID 0>> BDC 0 0 1 rg 40 180 180 20 re f EMC q 1 0 0 1 40 100 cm /Fm Do Q'),
font,
stream('/P <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Unrelated Form paragraph) Tj ET EMC', '/Type /XObject /Subtype /Form /BBox [0 0 240 80] /StructParents 1 /Resources << /Font << /F1 5 0 R >> >>'),
'<< /Type /StructTreeRoot /K [8 0 R 9 0 R] /ParentTree << /Nums [0 [8 0 R] 1 [9 0 R]] >> /ParentTreeNextKey 2 >>',
'<< /Type /StructElem /S /H1 /P 7 0 R /Pg 3 0 R /T (Page vector heading) /K 0 >>',
'<< /Type /StructElem /S /P /P 7 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 6 0 R /MCID 0 >> >>'
], {'pages': 1, 'headings': [{'page': 0, 'title': 'Page vector heading', 'level': 1, 'precision': 'page', 'reason_contains': 'MCID'}],
'fallback': 'MCID 0 names a page vector and unrelated Form text. Non-text page objects must participate in stream collision detection.'}))
def form_case(name, forms, content, expected, *, rolemap='', root_order=None, mutate=None):
"""Create source-qualified MCRs with an indirect ParentTree Kids node."""
objects = ['<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>', '', stream(content), font, '']
heading_ids = [7 + len(forms) + i for i, form in enumerate(forms)]
names = ' '.join(f'/Fm{i} {7+i} 0 R' for i in range(len(forms)))
objects[2] = f'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << {names} >> >> /Contents 4 0 R >>'
for i, form in enumerate(forms):
extra = f'/Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents {i} /Resources << /Font << /F1 5 0 R >> /XObject << {names} >> /Properties << /HeadingProperty << /MCID 0 >> >> >>'
if 'matrix' in form: extra += f' /Matrix [{form["matrix"]}]'
objects.append(stream(form['content'], extra))
for i, form in enumerate(forms):
title = f'/T ({form["title"]})' if 'title' in form else ''
objects.append(f'<< /Type /StructElem /S /{form.get("role", "H2")} /P 6 0 R /Pg 3 0 R {title} /K << /Type /MCR /Pg 3 0 R /Stm {7+i} 0 R /MCID 0 >> >>')
parent_id = len(objects) + 1
nums = ' '.join(f'{i} [{heading_ids[i]} 0 R]' for i in range(len(forms)))
objects.append(f'<< /Limits [0 {len(forms)-1}] /Nums [{nums}] >>')
order = root_order if root_order is not None else range(len(forms))
children = ' '.join(f'{heading_ids[i]} 0 R' for i in order)
objects[5] = f'<< /Type /StructTreeRoot /K [{children}] /ParentTree << /Kids [{parent_id} 0 R] >> {rolemap} >>'
if mutate: mutate(objects, heading_ids)
records.append(write(name, objects, {'pages': 1, **expected}))
def tagged(label, tag='H2'):
return f'/{tag} <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td ({label}) Tj ET EMC'
def exact(title, region, level=2):
return {'page': 0, 'title': title, 'level': level, 'precision': 'exact', 'text_region_pt': region}
def fallback(title, reason):
return {'page': 0, 'title': title, 'level': 2, 'precision': 'page', 'reason_contains': reason}
form_case('named-property-rolemap.pdf', [{'content': '/Section /HeadingProperty BDC BT /F1 12 Tf 10 30 Td (Named property heading) Tj ET EMC', 'role': 'Section'}],
'q 1 0 0 1 40 100 cm /Fm0 Do Q', {'headings': [exact('Named property heading', [50, 127, 190, 142], 3)]}, rolemap='/RoleMap << /Section /H3 >>')
form_case('nested-form-matrix.pdf', [{'content': 'q 1.5 0 0 1.25 10 20 cm /Fm1 Do Q', 'matrix': '1.2 0 0 0.9 5 7'},
{'content': tagged('Nested heading'), 'matrix': '1 0 0 1 4 6'}],
'q 1 0 0 1 40 100 cm /Fm0 Do Q', {'headings': [exact('Nested heading', [82, 161, 245, 181])]})
form_case('different-form-same-mcid.pdf', [{'content': tagged('First form')}, {'content': tagged('Second form')}],
'q 1 0 0 1 30 180 cm /Fm0 Do Q q 1 0 0 1 80 60 cm /Fm1 Do Q',
{'headings': [exact('Second form', [90, 87, 170, 102]), exact('First form', [40, 207, 105, 222])]}, root_order=[1, 0])
form_case('ambiguous-form-matrix.pdf', [{'content': tagged('Do not borrow first'), 'title': 'First structure'}, {'content': tagged('Do not borrow second'), 'title': 'Second structure'}],
'q 1 0 0 1 40 100 cm /Fm0 Do /Fm1 Do Q',
{'headings': [fallback('First structure', '一意'), fallback('Second structure', '一意')]})
form_case('repeated-form.pdf', [{'content': tagged('Repeated heading')}],
'q 1 0 0 1 30 180 cm /Fm0 Do Q q 1 0 0 1 80 60 cm /Fm0 Do Q',
{'headings': [fallback('Repeated heading', '複数回')]})
form_case('unused-form-resource.pdf', [{'content': tagged('Used heading')}, {'content': tagged('Unused heading')}],
'q 1 0 0 1 40 100 cm /Fm0 Do Q', {'headings': [exact('Used heading', [50, 127, 140, 142])]})
form_case('inline-image-before-form.pdf', [{'content': tagged('After image')}],
'q 1 0 0 1 40 100 cm BI /W 1 /H 1 /BPC 8 /CS /G ID X EI /Fm0 Do Q',
{'headings': [exact('After image', [50, 127, 125, 142])]})
limited = {'headings': [], 'limited': True}
form_case('cyclic-form.pdf', [{'content': '/Fm0 Do'}], '/Fm0 Do', limited)
form_case('cyclic-rolemap.pdf', [{'content': tagged('Hidden by cycle'), 'role': 'Foo'}], '/Fm0 Do', limited, rolemap='/RoleMap << /Foo /Bar /Bar /Foo >>')
form_case('cyclic-parent.pdf', [{'content': tagged('Parent cycle')}], '/Fm0 Do', limited,
mutate=lambda o, ids: o.__setitem__(ids[0]-1, o[ids[0]-1].replace('/P 6 0 R', f'/P {ids[0]} 0 R')))
form_case('cyclic-children.pdf', [{'content': tagged('Child cycle')}], '/Fm0 Do', limited,
mutate=lambda o, ids: o.__setitem__(ids[0]-1, f'<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K {ids[0]} 0 R >>'))
form_case('broken-mcr-page.pdf', [{'content': tagged('Wrong page')}], '/Fm0 Do', limited,
mutate=lambda o, ids: o.__setitem__(ids[0]-1, o[ids[0]-1].replace('/Type /MCR /Pg 3 0 R', '/Type /MCR /Pg 7 0 R')))
form_case('broken-mcr-stream.pdf', [{'content': tagged('Wrong stream')}], '/Fm0 Do', limited,
mutate=lambda o, ids: o.__setitem__(ids[0]-1, o[ids[0]-1].replace('/Stm 7 0 R', '/Stm 3 0 R')))
form_case('annotation-owned-mcr.pdf', [{'content': tagged('Annotation-owned')}], '/Fm0 Do', limited,
mutate=lambda o, ids: o.__setitem__(ids[0]-1, o[ids[0]-1].replace('/Stm 7 0 R', '/Stm 7 0 R /StmOwn 3 0 R')))
form_case('broken-graphics-stack.pdf', [{'content': tagged('Graphics stack')}], 'Q /Fm0 Do', limited)
form_case('cyclic-number-tree.pdf', [{'content': tagged('Number tree')}], '/Fm0 Do', limited,
mutate=lambda o, ids: o.__setitem__(len(o)-1, f'<< /Limits [0 0] /Kids [{len(o)} 0 R] >>'))
form_case('conflicting-parent-tree-owner.pdf', [{'content': tagged('First owner')}, {'content': tagged('Second owner')}],
'q 1 0 0 1 30 180 cm /Fm0 Do Q q 1 0 0 1 80 60 cm /Fm1 Do Q', limited,
mutate=lambda o, ids: o.__setitem__(ids[0]-1, o[ids[0]-1].replace('/Stm 7 0 R', '/Stm 8 0 R')))
def compressed_content(size):
def mutate(objects, ids):
body = ('q 1 0 0 1 40 100 cm /Fm0 Do Q').encode()
data = zlib.compress(b' ' * (size-len(body)) + body)
objects[3] = f'<< /Length {len(data)} /Filter /FlateDecode >>\nstream\n'.encode() + data + b'\nendstream'
return mutate
form_case('stream-quota-at-limit.pdf', [{'content': tagged('At stream limit')}], '',
{'headings': [exact('At stream limit', [50, 127, 140, 142])]}, mutate=compressed_content(16*1024*1024))
form_case('stream-quota-over-limit.pdf', [{'content': tagged('Over stream limit')}], '',
{'headings': [], 'limited': True, 'truncated': True}, mutate=compressed_content(16*1024*1024+1))
long_title = '<feff' + ('あ' * 4090).encode('utf-16-be').hex() + '>'
many = ['<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 5 0 R >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /StructParents 0 /Resources << >> /Contents 4 0 R >>',
stream('\n'.join(f'/H2 <</MCID {i}>> BDC 10 {i*4} 20 2 re f EMC' for i in range(70))), '']
for i in range(70):
many.append(f'<< /Type /StructElem /S /H2 /P 5 0 R /Pg 3 0 R /K {i} /ActualText {long_title} >>')
refs = ' '.join(f'{i+6} 0 R' for i in range(70))
many[4] = f'<< /Type /StructTreeRoot /K [{refs}] /ParentTree << /Nums [0 [{refs}]] >> >>'
records.append(write('bounded-heading-response.pdf', many, {'pages': 1, 'headings': [], 'response_limited': True}))
deep = ['<< /Type /Catalog /Pages 2 0 R >>', '<< /Type /Pages /Count 2 /Kids [3 0 R 70 0 R] >>']
for obj in range(3, 69):
deep.append(f'<< /Type /Pages /Parent {obj-1} 0 R /Count 1 /Kids [{obj+1} 0 R] >>')
deep.append('<< /Type /Page /Parent 68 0 R /MediaBox [0 0 100 100] /Resources << >> >>')
deep.append('<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> >>')
context_record = write('context-deep-tree.pdf', deep, {'scope': 'Context-only regression: after a depth failure, the second page must never be assigned index zero.'})
context_records = [context_record]
for count in (32, 33):
context_records.append(write(f'context-warnings-{count}.pdf', [
f'<< /Type /Catalog /Pages 2 0 R /Duplicate 1 /Duplicate 2 /Probe [' + ' '.join(f'{i+4} 0 R' for i in range(count-1)) + '] >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources << >> >>'] +
['<< /Duplicate 1 /Duplicate 2 >>'] * (count-1),
{'scope': 'Context-only cumulative diagnostic limit, starting in initial parse; warning bodies must never reach stdout/stderr.', 'accepted': count == 32}))
long_key = '/' + 'A' * (64*1024)
context_records.append(write('context-warning-bytes.pdf', [
f'<< /Type /Catalog /Pages 2 0 R {long_key} 1 {long_key} 2 >>',
'<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources << >> >>'],
{'scope': 'Context-only diagnostic byte limit during initial parse; no warning text is retained or emitted.', 'accepted': False}))
(ROOT / 'manifest.json').write_text(json.dumps({'origin': 'Self-created deterministic PDF syntax, Helvetica Standard-14 text and vector paths; no third-party document content.', 'generator': '../generate_headings.py', 'documents': records, 'context_documents': context_records}, ensure_ascii=False, indent=2) + '\n', encoding='utf-8')
print(f'Generated {len(records)} heading fixtures in {ROOT}')
tests/fixtures/pdf/generate_intent_transparency.py
Vendored
+214
View File
@@ -0,0 +1,214 @@
#!/usr/bin/env python3
"""Self-authored RI probes for luminosity masks and pattern initial state."""
import argparse
import hashlib
import json
from pathlib import Path
from pypdf import PdfWriter
from pypdf.generic import (ArrayObject, BooleanObject, DecodedStreamObject,
DictionaryObject, FloatObject, NameObject, NumberObject)
ROOT = Path(__file__).resolve().parents[3]
SOURCE = ROOT / 'tests/fixtures/pdf/rendering-intents/distinct-cmyk-lab.icc'
PROFILE_SHA = 'b9a40bfc18e8280eefe0fedc4e607467e1cc27adeee3beb77f4a2b83fc7e9315'
INTENTS = ('Perceptual', 'RelativeColorimetric', 'Saturation', 'AbsoluteColorimetric')
SAMPLE = (0, .75, .25, 0)
WIDTH, HEIGHT = 880, 850
CASES = (
'direct-icc-control', 'mask-caller-ri', 'mask-internal-ri', 'mask-internal-gs',
'mask-group-qQ', 'mask-q-inner-none', 'mask-Q-restores', 'mask-none',
'mask-BC-only', 'mask-opaque-over-BC',
'tiling-page-initial', 'tiling-colored-ri', 'tiling-colored-gs', 'tiling-colored-qQ',
'tiling-form-initial', 'tiling-form-initial-reuse', 'tiling-form-ri',
'tiling-uncolored-fill', 'tiling-uncolored-fill-gs',
'tiling-uncolored-stroke-small', 'tiling-uncolored-stroke-large',
'shading-page-initial', 'shading-pattern-ri', 'shading-form-initial',
'shading-form-initial-reuse', 'shading-form-ri', 'direct-shading-control',
)
def digest(data): return hashlib.sha256(data).hexdigest()
def n(value): return NameObject('/' + value)
def a(values): return ArrayObject([FloatObject(v) for v in values])
def d(**values): return DictionaryObject({n(k): v for k, v in values.items()})
def numbers(values): return ' '.join(format(v, '.10g') for v in values)
def generate(output):
output.mkdir(parents=True, exist_ok=False)
profile = SOURCE.read_bytes()
assert digest(profile) == PROFILE_SHA
(output / SOURCE.name).write_bytes(profile)
writer = PdfWriter(); writer.pdf_header = '%PDF-1.7'
writer.add_metadata({'/Title': 'Rendering intent: soft masks and pattern initial state',
'/Author': 'DocView synthetic regression corpus'})
def stream(data, **entries):
obj = DecodedStreamObject(); obj.set_data(data.encode() if isinstance(data, str) else data)
obj.update(d(**entries)); return writer._add_object(obj)
cs = ArrayObject([n('ICCBased'), stream(profile, N=NumberObject(4), Alternate=n('DeviceCMYK'))])
colors = d(CS=cs, PCS=ArrayObject([n('Pattern'), cs]))
gs = d(**{'I'+str(i): d(Type=n('ExtGState'), RI=n(intent)) for i, intent in enumerate(INTENTS)})
gs[n('NoMask')] = d(Type=n('ExtGState'), SMask=n('None'))
font = writer._add_object(d(Type=n('Font'), Subtype=n('Type1'), BaseFont=n('Helvetica')))
base = d(ColorSpace=colors, ExtGState=gs)
paint = '/CS cs ' + numbers(SAMPLE) + ' scn '
group_attributes = d(S=n('Transparency'), CS=n('DeviceRGB'), I=BooleanObject(True), K=BooleanObject(False))
def group(prefix='', empty=False):
return stream(prefix + '\n' + ('' if empty else paint + f'0 0 {WIDTH} {HEIGHT} re f\n'),
Type=n('XObject'), Subtype=n('Form'), BBox=a((0,0,WIDTH,HEIGHT)),
Group=group_attributes, Resources=base)
shared_group = group()
empty_group = group(empty=True)
def mask(name, group_ref, bc=(0,0,0)):
gs[n(name)] = d(Type=n('ExtGState'), SMask=d(S=n('Luminosity'), G=group_ref,
BC=a(bc), TR=n('Identity')))
mask('Mask', shared_group)
mask('OpaqueBC', shared_group, (0,.5,1))
backgrounds = ((1,0,0), (0,1,0), (0,0,1), (0,.5,1))
for i, intent in enumerate(INTENTS):
mask('MaskRi'+str(i), group('/'+intent+' ri'))
mask('MaskGs'+str(i), group('/I'+str(i)+' gs'))
mask('MaskQ'+str(i), group('q /'+intent+' ri Q'))
mask('BC'+str(i), empty_group, backgrounds[i])
patterns = d()
def tiling(prefix='', colored=True, size=16):
content = prefix+'\n'+(paint if colored else '')+f'0 0 {size} {size} re f\n'
return stream(content, Type=n('Pattern'), PatternType=NumberObject(1), PaintType=NumberObject(1 if colored else 2),
TilingType=NumberObject(2), BBox=a((0,0,size,size)), XStep=NumberObject(size), YStep=NumberObject(size),
Resources=base if colored else d())
patterns[n('Tile')] = tiling()
patterns[n('U')] = tiling(colored=False, size=8)
patterns[n('ULarge')] = tiling(colored=False, size=1024)
for i, intent in enumerate(INTENTS):
patterns[n('TileRi'+str(i))] = tiling('/'+intent+' ri')
patterns[n('TileGs'+str(i))] = tiling('/I'+str(i)+' gs')
patterns[n('TileQ'+str(i))] = tiling('q /'+intent+' ri Q')
# Constant ICC axial function eliminates interpolation-position uncertainty.
function = writer._add_object(d(FunctionType=NumberObject(2), Domain=a((0,1)), C0=a(SAMPLE), C1=a(SAMPLE), N=NumberObject(1)))
shading = writer._add_object(d(ShadingType=NumberObject(2), ColorSpace=cs, Coords=a((0,0,WIDTH,0)),
Function=function, Extend=ArrayObject([BooleanObject(True),BooleanObject(True)])))
patterns[n('Shade')] = d(Type=n('Pattern'), PatternType=NumberObject(2), Shading=shading)
for i, intent in enumerate(INTENTS):
patterns[n('ShadeRi'+str(i))] = d(Type=n('Pattern'), PatternType=NumberObject(2), Shading=shading,
ExtGState=d(RI=n(intent)))
# A shared Form is invoked under distinct initial RI values. Its painting RI
# is always Absolute, which must not replace the pattern's stream-start RI.
form_resources = d(ColorSpace=colors, ExtGState=gs, Pattern=patterns)
xobjects = d()
for label, pattern_name in (('Tile','Tile'),('Shade','Shade')):
xobjects[n(label+'Form')] = stream('/AbsoluteColorimetric ri\n/Pattern cs /'+pattern_name+' scn 0 0 90 42 re f\n',
Type=n('XObject'), Subtype=n('Form'), BBox=a((0,0,90,42)), Resources=form_resources)
for i in range(4):
xobjects[n(label+'FormRi'+str(i))] = stream('/AbsoluteColorimetric ri\n/Pattern cs /'+pattern_name+'Ri'+str(i)+' scn 0 0 90 42 re f\n',
Type=n('XObject'), Subtype=n('Form'), BBox=a((0,0,90,42)), Resources=form_resources)
resources = d(ColorSpace=colors, ExtGState=gs, Pattern=patterns, XObject=xobjects,
Shading=d(A=shading), Font=d(F=font))
probes=[]
for group_index in range((len(CASES)+6)//7):
page=writer.add_blank_page(WIDTH,HEIGHT);page[n('Resources')]=resources
commands=[f'1 g 0 0 {WIDTH} {HEIGHT} re f',
f'0 g BT /F 18 Tf 24 818 Td (Soft masks and pattern state - {group_index+1}) Tj ET',
'0 g BT /F 9 Tf 24 799 Td (Synthetic ICC; expected intent comes from PDF state rules, colors from a direct CMM.) Tj ET']
for i,intent in enumerate(INTENTS):
commands.append(f'0 g BT /F 9 Tf {320+i*132} 771 Td ({intent}) Tj ET')
for row,case in enumerate(CASES[group_index*7:group_index*7+7]):
y=684-row*96
commands.append(f'0 g BT /F 10 Tf 24 {y+22} Td ({case}) Tj ET')
for column in range(4):
current=(2,0,2,1)[column] if case.endswith('-reuse') else column
other=(current+1)%4;expected=current;kind='icc';bc=None
x=320+column*132;rect=f'{x} {y} 90 42 re f'
start=['q','/'+INTENTS[current]+' ri'];body=[]
if case=='direct-icc-control': body=[paint+rect]
elif case.startswith('mask-'):
kind='luminosity-icc';mask_name='Mask'
if case=='mask-internal-ri':mask_name='MaskRi'+str(other);expected=other
elif case=='mask-internal-gs':mask_name='MaskGs'+str(other);expected=other
elif case=='mask-group-qQ':mask_name='MaskQ'+str(other)
elif case=='mask-BC-only':mask_name='BC'+str(column);bc=backgrounds[column];kind='luminosity-bc'
elif case=='mask-opaque-over-BC':mask_name='OpaqueBC'
body=['/'+mask_name+' gs']
if case=='mask-q-inner-none':body+=['q /NoMask gs','0 g '+rect,'Q'];kind='black'
elif case=='mask-Q-restores':body+=['q /NoMask gs Q','0 g '+rect]
elif case=='mask-none':body+=['/NoMask gs','0 g '+rect];kind='black'
else:body+=['0 g '+rect]
elif case.startswith('tiling-uncolored'):
if case.endswith('-gs'):start=['q','/I'+str(current)+' gs']
if 'stroke' in case:
pattern='ULarge' if case.endswith('-large') else 'U'
body=['1 0 0 rg','/PCS CS '+numbers(SAMPLE)+' /'+pattern+' SCN',
f'30 w {x} {y+21} m {x+90} {y+21} l S']
else:body=['/PCS cs '+numbers(SAMPLE)+' /U scn '+rect]
elif case.startswith('tiling-') or case.startswith('shading-'):
label='Tile' if case.startswith('tiling-') else 'Shade';pattern=label
if '-form-' in case:
form_name=label+'Form'
if case.endswith('-ri'):form_name+='Ri'+str(other);expected=other
body=[f'1 0 0 1 {x} {y} cm /'+form_name+' Do']
else:
expected=1
if case.endswith('-ri'):pattern+='Ri'+str(other);expected=other
elif case.endswith('-gs'):pattern+='Gs'+str(other);expected=other
elif case.endswith('-qQ'):pattern+='Q'+str(other)
body=['/Pattern cs /'+pattern+' scn '+rect]
elif case=='direct-shading-control':body=[f'{x} {y} 90 42 re W n','/A sh']
else:raise AssertionError(case)
sequence=start+body+['Q'];commands+=sequence
probes.append({'id':case+'-'+str(column),'case':case,'column':column,'page':group_index+1,
'callerIntentIndex':current,'callerIntent':INTENTS[current],
'expectedIntentIndex':expected,'expectedIntent':INTENTS[expected],
'oracle':kind,'components':list(SAMPLE),'backgroundRgb':list(bc) if bc else None,
'pointPt':[x+45,y+21],'commandSequence':sequence})
page[n('Contents')]=stream('\n'.join(commands)+'\n')
target=output/'intent-transparency.pdf';writer.write(target)
manifest={'schemaVersion':1,'file':target.name,'sha256':digest(target.read_bytes()),
'generatorSha256':digest(Path(__file__).read_bytes()),'pageCount':len(writer.pages),'pageSizePt':[WIDTH,HEIGHT],
'profile':{'file':SOURCE.name,'sha256':PROFILE_SHA,'size':len(profile),'origin':str(SOURCE.relative_to(ROOT))},
'sample':list(SAMPLE),'tolerance8Bit':4,'cases':list(CASES),'probes':probes,
'specification':{'url':'https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/pdfreference1.4.pdf',
'printedPages':{'uncoloredForbiddenOperators':218,'tilingInitialState':223,'shadingInitialState':231,
'deviceRgbGray':377,'luminosityMask':440,'softMaskDictionary':446,'transparentPatterns':453}},
'luminosityOracle':{'weights':[.30,.59,.11],'foregroundRgb':[0,0,0],'pageBackgroundRgb':[255,255,255],
'formula':'round(255 - dot(weights, directCmmRgb8))',
'qualification':'DeviceRGB gray conversion is device-dependent; these are the PDF recommended weights. Intent dispatch is tested separately from that numerical convention.'}}
(output/'manifest.json').write_text(json.dumps(manifest,indent=2)+'\n')
(output/'README.md').write_text('''# Soft-mask and pattern rendering-intent probes
This self-authored 4-page PDF contains 108 declared pixel probes. The existing
984-byte distinct CMYK/Lab ICC profile is copied without alteration. Its four
intent outputs for CMYK (0, .75, .25, 0) have separated luminances; no PDF renderer
supplies the expected colors.
[Adobe PDF Reference 1.4](https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/pdfreference1.4.pdf)
pp. 218, 223, 231 and 453 define pattern state. Colored tiling and shading patterns
start with their resource-owning page/Form stream's initial state. A Form is
reused under different initial intents, then changes its painting intent to
Absolute. Explicit pattern overrides and q/Q are separate controls. Uncolored
stencils contain no color operators or `ri`; their ICC base color is supplied by
the caller. Small/large uncolored strokes distinguish stroke from fill color.
Pages 440–446 define luminosity soft masks and their backdrop. The mask's RGB
group contains one opaque ICC rectangle, or no objects for the BC-only controls.
The direct CMM RGB result is reduced with the recommended .30/.59/.11 weights;
black over white has value 255 minus that luminosity. This device-dependent
formula is a declared test convention, not universal device calibration.
Run `tests/cmyk_lut/transparency.py --self-check --output NEW_DIRECTORY`, or pass
`--build-dir build --pdfium-library PATH/libpdfium.so --output NEW_DIRECTORY`.
The runner records all failures, validates the sandbox helper and library hash,
and judges PDFium and Poppler separately against the direct CMM. LittleCMS may
be shared with the renderers; this is not an independent CMM implementation or
general ICC conformance acceptance. The older 8-page intent fixture is unchanged.
''')
print(json.dumps({'file':str(target),'sha256':manifest['sha256'],'pages':len(writer.pages),'probes':len(probes)}))
if __name__=='__main__':
parser=argparse.ArgumentParser(description=__doc__)
parser.add_argument('--output',type=Path,default=Path(__file__).parent/'intent-transparency')
generate(parser.parse_args().output)
tests/fixtures/pdf/generate_link_borders.py
Vendored
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env python3
"""Deterministic self-created links; no third-party document or font content."""
from pathlib import Path
import hashlib
import json
ROOT = Path(__file__).parent / 'link-borders'
ROOT.mkdir(exist_ok=True)
def stream(content, extra=''):
content = content.encode('ascii')
return f'<< /Length {len(content)} {extra} >>\nstream\n'.encode() + content + b'\nendstream'
def write(name, objects):
data = bytearray(b'%PDF-1.7\n%\xe2\xe3\xcf\xd3\n')
offsets = [0]
for i, value in enumerate(objects, 1):
offsets.append(len(data))
if isinstance(value, str): value = value.encode('ascii')
data += f'{i} 0 obj\n'.encode() + value + b'\nendobj\n'
xref = len(data)
data += f'xref\n0 {len(offsets)}\n0000000000 65535 f \n'.encode()
for offset in offsets[1:]: data += f'{offset:010d} 00000 n \n'.encode()
data += f'trailer\n<< /Size {len(offsets)} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n'.encode()
(ROOT / name).write_bytes(data)
return {'file': name, 'bytes': len(data), 'sha256': hashlib.sha256(data).hexdigest()}
CASES = [
('default', ''),
('solid-rgb', '/BS << /W 3 /S /S >> /C [0.8 0.1 0.2]'),
('dashed-border', '/Border [0 0 2 [6 3]] /C [0 0.2 0.8]'),
('dashed-bs-odd', '/BS << /W 2 /S /D /D [7 3 2] >> /C [0.1 0.5 0.2]'),
('dashed-default', '/BS << /W 2 /S /D >>'),
('underline', '/BS << /W 3 /S /U >> /C [0.6 0.1 0.8]'),
('bs-overrides-border', '/Border [9 9 12 [1 1]] /BS << /W 2 /S /S >> /C [0 0.3 0.7]'),
('zero-width', '/BS << /W 0 >>'),
('transparent', '/BS << /W 4 >> /C []'),
('hidden', '/BS << /W 4 >> /F 2'),
('no-view', '/BS << /W 4 >> /F 32'),
('print-flag-visible', '/BS << /W 2 >> /F 4'),
('gray', '/BS << /W 3 >> /C [0.5]'),
('cmyk', '/BS << /W 3 >> /C [1 0 0 0]'),
('rounded', '/Border [8 6 2] /C [0.6 0.3 0.1]'),
('beveled', '/BS << /W 3 /S /B >> /C [0.2 0.6 0.8]'),
('inset', '/BS << /W 3 /S /I >> /C [0.2 0.6 0.8]'),
('appearance-preserved', '/BS << /W 8 /S /D >> /C [1 0 0] /AP << /N 7 0 R >>'),
('empty-appearance-preserved', '/BS << /W 8 >> /AP << /N 8 0 R >>'),
('alpha', '/BS << /W 3 >> /C [0 0 1] /CA 0.5'),
]
# All pages share the same annotations, but their CropBox and Rotate differ.
# Rects straddle 512px tile boundaries at the evidence renderer's scale 2.
objects = [
'<< /Type /Catalog /Pages 2 0 R >>',
'<< /Type /Pages /Count 4 /Kids [3 0 R 4 0 R 5 0 R 6 0 R] >>',
'', '', '', '',
stream('0 0.8 0 rg 0 0 100 30 re f', '/Type /XObject /Subtype /Form /BBox [0 0 100 30] /Resources << >>'),
stream('', '/Type /XObject /Subtype /Form /BBox [0 0 100 30] /Resources << >>'),
]
expected = []
for index, (label, extra) in enumerate(CASES):
x = 40 + (index % 4) * 120
y = 50 + (index // 4) * 75
rect = [x, y, x + 100, y + 30]
objects.append(f'<< /Type /Annot /Subtype /Link /Rect [{" ".join(map(str, rect))}] '
f'/A << /S /URI /URI (https://example.org/{label}) >> {extra} >>')
expected.append({'name': label, 'rect': rect, 'has_border': label not in {'zero-width', 'transparent', 'hidden', 'no-view', 'empty-appearance-preserved'}})
annots = ' '.join(f'{i + 9} 0 R' for i in range(len(CASES)))
for page, rotation in enumerate([0, 90, 180, 270]):
objects[page + 2] = f'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 550 470] /CropBox [20 30 530 450] /Rotate {rotation} /Resources << >> /Annots [{annots}] >>'
records = [write('styles.pdf', objects)]
records[0]['pages'] = 4
records[0]['annotations'] = expected
records[0]['reference_limitations'] = ['Poppler ignores gray/CMYK border colors, rounded corners, bevel/inset, and alpha in its no-AP link fallback; assert these independently.', 'Saved normal AP must be preserved without adding a second border, even if Poppler adds one.']
for name, extra in [('unknown-style', '/BS << /S /Unknown >>'), ('no-zoom', '/F 8'), ('no-rotate', '/F 16'), ('invalid-dash', '/BS << /S /D /D [0 0] >>'), ('oversized-dash', '/BS << /S /D /D [' + '1 ' * 65 + '] >>'), ('invalid-color', '/C [0 1]'), ('invalid-ap', '/AP << /N 42 >>')]:
data = ['<< /Type /Catalog /Pages 2 0 R >>', '<< /Type /Pages /Count 1 /Kids [3 0 R] >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources << >> /Annots [4 0 R] >>',
f'<< /Type /Annot /Subtype /Link /Rect [20 20 80 80] {extra} >>']
record = write(name + '.pdf', data)
if name in {'no-zoom', 'no-rotate'}:
record['expected'] = 'Valid annotation flag; rendering succeeds. Geometry coverage is in annotation-flags/flags.pdf.'
else:
record['render_error'] = 'E_PDF_ANNOT_UNSUPPORTED' if name == 'unknown-style' else 'E_LIMIT_EXCEEDED' if name == 'oversized-dash' else 'E_PDF_CORRUPT'
records.append(record)
retry = ['<< /Type /Catalog /Pages 2 0 R >>', '<< /Type /Pages /Count 2 /Kids [3 0 R 4 0 R] >>', '',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources << >> /Annots [5 0 R] >>']
for index in range(512):
retry.append('<< /Type /Annot /Subtype /Link /Rect [20 20 80 80] /BS << /S /D /D [' + '1 ' * 64 + '] >> >>')
retry.append('<< /Type /Annot /Subtype /Link /Rect [20 20 80 80] /BS << /S /Unknown >> >>')
retry[2] = '<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources << >> /Annots [' + ' '.join(f'{i} 0 R' for i in range(5, len(retry) + 1)) + '] >>'
record = write('retry-budget.pdf', retry)
record['expected'] = 'Forty failed page-0 decodes must not charge discarded appearances or prevent valid page-1 rendering.'
records.append(record)
(ROOT / 'manifest.json').write_text(json.dumps({'generator': 'generate_link_borders.py', 'license': 'Self-created test content; same license as DocView.', 'fixtures': records}, indent=2) + '\n')
tests/fixtures/pdf/generate_rendering_intents.py
Vendored
+226
View File
@@ -0,0 +1,226 @@
#!/usr/bin/env python3
"""Generate self-authored ICC intent/state/cache probes without renderer goldens."""
import argparse
import hashlib
import itertools
import json
from pathlib import Path
import struct
from pypdf import PdfWriter
from pypdf.generic import (ArrayObject, ByteStringObject, DecodedStreamObject, DictionaryObject,
FloatObject, NameObject, NumberObject)
ROOT = Path(__file__).resolve().parents[3]
BASE = ROOT / 'tests/fixtures/pdf/cmyk-lut'
INTENTS = ('Perceptual', 'RelativeColorimetric', 'Saturation', 'AbsoluteColorimetric')
SAMPLE_BYTES = (51, 128, 204, 77)
SAMPLE = tuple(x/255 for x in SAMPLE_BYTES)
WIDTH, HEIGHT = 820, 800
CASES = ('default-relative', 'ri-vector', 'extgstate-ri', 'nested-q-inner', 'nested-Q-restore',
'form-inherit-shared', 'form-inherit-shared-gs', 'form-ri-override', 'form-gs-override', 'form-return-restore',
'image-inherit-shared-forward', 'image-inherit-shared-reverse', 'image-intent-override',
'image-inherit-shared-gs', 'image-override-return', 'indexed-vector', 'indexed-image', 'stroke', 'text', 'axial-shading',
'shading-pattern-inherit', 'shading-pattern-RI-override',
'type3-inherit-rectangle', 'type3-inherit-image', 'type3-ri-override')
def sha(data): return hashlib.sha256(data).hexdigest()
def array(values): return ArrayObject([FloatObject(x) for x in values])
def name(value): return NameObject('/'+value)
def dictionary(**values): return DictionaryObject({name(k):v for k,v in values.items()})
def fixed(values): return struct.pack('>'+'i'*len(values), *(round(v*65536) for v in values))
def operands(values): return ' '.join(f'{v:.10f}' for v in values)
def derived_model(tag, values):
c,m,y,k = values
if tag == 'A2B0': return (100-12*c-15*m-10*y-35*k,18*m-15*c,20*y-12*c)
if tag == 'A2B1': return (100-20*c-20*m-20*y-40*k,40*m-20*c-20*k,40*y-20*c-20*k)
if tag == 'A2B2': return (100-25*c-25*m-20*y-30*k,-40*m+20*c+20*k,35*y-20*m-15*k)
raise ValueError(tag)
def derived_profile(base):
# Reuse the existing input profile's v2 header, CLUT layout and attribution;
# add explicitly distinct self-authored intent tables and a non-D50 medium.
tags = {}
for i in range(struct.unpack_from('>I',base,128)[0]):
signature,offset,size=struct.unpack_from('>4sII',base,132+12*i)
tags[signature]=base[offset:offset+size]
tags[b'wtpt']=b'XYZ '+bytes(4)+fixed((.82,.86,.72))
for signature in ('A2B0','A2B1','A2B2'):
lut=bytearray(tags[b'A2B0'])
for i,values in enumerate(itertools.product((0,1),repeat=4)):
lightness,a,b=derived_model(signature,values)
struct.pack_into('>3H',lut,68+6*i,round(lightness/100*65280),round((a+128)*256),round((b+128)*256))
tags[signature.encode()]=bytes(lut)
description=b'DocView synthetic distinct rendering intent tables\0'
tags[b'desc']=b'desc'+bytes(4)+struct.pack('>I',len(description))+description+bytes(78)
header=bytearray(base[:128]);entries=bytearray();body=bytearray();offset=132+12*len(tags)
for signature,data in sorted(tags.items()):
entries+=signature+struct.pack('>II',offset+len(body),len(data))
body+=data+bytes((-len(data))%4)
result=header+struct.pack('>I',len(tags))+entries+body
result[:4]=struct.pack('>I',len(result))
return bytes(result)
def generate(output):
output.mkdir(parents=True,exist_ok=False)
base_spec=json.loads((BASE/'manifest.json').read_text())
base=(BASE/base_spec['profileFile']).read_bytes()
assert sha(base)==base_spec['profileSha256']
profiles={'original':base,'distinct':derived_profile(base)}
writer=PdfWriter();writer.pdf_header='%PDF-1.7'
writer.add_metadata({'/Title':'Rendering intent: explicit state, inheritance and cache',
'/Author':'DocView synthetic regression corpus'})
rows=[];profile_records={}
for profile_id,profile in profiles.items():
filename=profile_id+'-cmyk-lab.icc';(output/filename).write_bytes(profile)
profile_records[profile_id]={'file':filename,'sha256':sha(profile),'size':len(profile),
'origin':'byte-identical existing synthetic fixture' if profile_id=='original' else 'self-authored extension of the existing CLUT',
'mediaWhitePoint':[.9642,1,.8249] if profile_id=='original' else [.82,.86,.72],
'tables':['A2B0'] if profile_id=='original' else ['A2B0','A2B1','A2B2']}
stream=DecodedStreamObject();stream.set_data(profile)
stream.update(dictionary(N=NumberObject(4),Alternate=name('DeviceCMYK')))
space=ArrayObject([name('ICCBased'),writer._add_object(stream)])
indexed=ArrayObject([name('Indexed'),space,NumberObject(0),ByteStringObject(bytes(SAMPLE_BYTES))])
font=writer._add_object(dictionary(Type=name('Font'),Subtype=name('Type1'),BaseFont=name('Helvetica-Bold')))
gs=dictionary(**{'I'+str(i):dictionary(Type=name('ExtGState'),RI=name(intent)) for i,intent in enumerate(INTENTS)})
local_resources=dictionary(ColorSpace=dictionary(CS=space,IX=indexed),ExtGState=gs,Font=dictionary(F=font))
image=DecodedStreamObject();image.set_data(bytes(SAMPLE_BYTES))
image.update(dictionary(Type=name('XObject'),Subtype=name('Image'),Width=NumberObject(1),Height=NumberObject(1),
BitsPerComponent=NumberObject(8),ColorSpace=space))
shared_image=writer._add_object(image)
fonts=dictionary(F=font)
glyph_resources=dictionary(ColorSpace=dictionary(CS=space),XObject=dictionary(Shared=shared_image))
def type3_font(prefix='',image=False):
glyph=DecodedStreamObject()
painting='q 1000 0 0 1000 0 0 cm /Shared Do Q' if image else '/CS cs '+operands(SAMPLE)+' scn 0 0 1000 1000 re f'
glyph.set_data(('1000 0 d0\n'+prefix+'\n'+painting+'\n').encode())
encoding=dictionary(Type=name('Encoding'),Differences=ArrayObject([NumberObject(65),name('A')]))
return writer._add_object(dictionary(Type=name('Font'),Subtype=name('Type3'),FontBBox=array((0,0,1000,1000)),
FontMatrix=array((.001,0,0,.001,0,0)),CharProcs=dictionary(A=writer._add_object(glyph)),
Encoding=encoding,FirstChar=NumberObject(65),LastChar=NumberObject(65),Widths=array((1000,)),Resources=glyph_resources))
fonts[name('T3Rect')]=type3_font()
fonts[name('T3Image')]=type3_font(image=True)
for i,intent in enumerate(INTENTS):fonts[name('T3Override'+str(i))]=type3_font('/'+intent+' ri')
iximage=DecodedStreamObject();iximage.set_data(b'\0')
iximage.update(dictionary(Type=name('XObject'),Subtype=name('Image'),Width=NumberObject(1),Height=NumberObject(1),
BitsPerComponent=NumberObject(8),ColorSpace=indexed))
xobjects=dictionary(Shared=shared_image,Indexed=writer._add_object(iximage))
def form(prefix):
stream=DecodedStreamObject()
stream.set_data((prefix+'\n/CS cs '+operands(SAMPLE)+' scn 0 0 90 44 re f\n').encode())
stream.update(dictionary(Type=name('XObject'),Subtype=name('Form'),BBox=array((0,0,90,44)),Resources=local_resources))
return writer._add_object(stream)
xobjects[name('Inherited')]=form('')
for i,intent in enumerate(INTENTS):
copy=DecodedStreamObject();copy.set_data(bytes(SAMPLE_BYTES));copy.update(image)
copy[name('Intent')]=name(intent)
xobjects[name('Image'+str(i))]=writer._add_object(copy)
xobjects[name('Form'+str(i))]=form('/'+intent+' ri')
xobjects[name('FormGs'+str(i))]=form('/I'+str(i)+' gs')
shading_function=dictionary(FunctionType=NumberObject(2),Domain=array((0,1)),
C0=array((.1,.3,.6,.2)),C1=array((.5,.7,.9,.4)),N=NumberObject(1))
shading=dictionary(ShadingType=NumberObject(2),ColorSpace=space,Coords=array((0,0,90,0)),
Function=writer._add_object(shading_function))
from pypdf.generic import BooleanObject
shading[name('Extend')]=ArrayObject([BooleanObject(False),BooleanObject(False)])
wide_shading=DictionaryObject(shading);wide_shading[name('Coords')]=array((0,0,WIDTH,0))
wide_ref=writer._add_object(wide_shading)
patterns=dictionary(Inherited=dictionary(Type=name('Pattern'),PatternType=NumberObject(2),Shading=wide_ref))
for i,intent in enumerate(INTENTS):
patterns[name('P'+str(i))]=dictionary(Type=name('Pattern'),PatternType=NumberObject(2),Shading=wide_ref,
ExtGState=dictionary(RI=name(intent)))
resources=dictionary(ColorSpace=dictionary(CS=space,IX=indexed),ExtGState=gs,Font=fonts,
XObject=xobjects,Shading=dictionary(A=writer._add_object(shading)),Pattern=patterns)
for group in range((len(CASES)+7)//8):
page=writer.add_blank_page(WIDTH,HEIGHT);page[name('Resources')]=resources
commands=['0 g BT /F 17 Tf 28 762 Td (ICC rendering intent - '+profile_id+f' - {group+1}/{(len(CASES)+7)//8}) Tj ET']
commands+=['0 g BT /F 8 Tf 28 742 Td (Synthetic input profile; reference colors are computed independently by CMM, not copied from a renderer.) Tj ET']
for column,intent in enumerate(INTENTS):
commands.append(f'0 g BT /F 10 Tf {310+column*125} 716 Td ({intent}) Tj ET')
for local_row,case in enumerate(CASES[group*8:group*8+8]):
y=625-local_row*80
commands.append(f'0 g BT /F 10 Tf 28 {y+22} Td ({case}) Tj ET')
for column in range(4):
intent_index=3-column if case=='image-inherit-shared-reverse' else column
if case=='image-inherit-shared-gs': intent_index=(0,1,0,3)[column]
intent=INTENTS[intent_index];other=(intent_index+1)%4
x=310+column*125
start=['q'];body=[];expected=intent_index;point=(x+45,y+22);components=SAMPLE
if case=='default-relative':
# Start each cell at the page's untouched default. Other
# commands are isolated by q/Q, including all labels.
expected=1
elif case in ('extgstate-ri','image-inherit-shared-gs','form-inherit-shared-gs'): start+=['/I'+str(intent_index)+' gs']
else: start+=['/'+intent+' ri']
paint=f'/CS cs {operands(SAMPLE)} scn {x} {y} 90 44 re f'
if case in ('default-relative','ri-vector','extgstate-ri'): body=[paint]
elif case=='nested-q-inner': body=['q /'+INTENTS[other]+' ri',paint,'Q'];expected=other
elif case=='nested-Q-restore': body=['q /'+INTENTS[other]+' ri Q',paint]
elif case.startswith('form-'):
form_name='Inherited'
if case=='form-ri-override': form_name='Form'+str(other);expected=other
if case=='form-gs-override': form_name='FormGs'+str(other);expected=other
if case=='form-return-restore':
# Invoke the override at the origin, then
# paint with the caller's restored graphics state.
body=[f'/Form{other} Do',paint]
else: body=[f'q 1 0 0 1 {x} {y} cm /{form_name} Do Q']
elif case.startswith('image-') or case=='indexed-image':
image_name='Indexed' if case=='indexed-image' else 'Shared'
if case=='image-intent-override': image_name='Image'+str(other);expected=other
if case=='image-override-return': body=[f'/Image{other} Do',paint]
else: body=[f'q 90 0 0 44 {x} {y} cm /{image_name} Do Q']
elif case=='indexed-vector': body=[f'/IX cs 0 scn {x} {y} 90 44 re f']
elif case=='stroke': body=[f'/CS CS {operands(SAMPLE)} SCN 24 w {x} {y+22} m {x+90} {y+22} l S']
elif case=='text':
body=[f'/CS cs {operands(SAMPLE)} scn BT /F 64 Tf 1 0 0 1 {x} {y} Tm (H) Tj ET']
point=(x+8,y+29)
elif case=='axial-shading':
body=[f'q {x} {y} 90 44 re W n 1 0 0 1 {x} {y} cm /A sh Q']
components=(.3,.5,.75,.3)
elif case.startswith('shading-pattern-'):
pattern='Inherited'
if case=='shading-pattern-RI-override':pattern='P'+str(other);expected=other
body=[f'/Pattern cs /{pattern} scn {x} {y} 90 44 re f']
t=(x+45)/WIDTH;components=tuple(a+(b-a)*t for a,b in zip((.1,.3,.6,.2),(.5,.7,.9,.4)))
elif case.startswith('type3-'):
glyph_font='T3Image' if case=='type3-inherit-image' else 'T3Rect'
if case=='type3-ri-override':glyph_font='T3Override'+str(other);expected=other
body=[f'BT /{glyph_font} 44 Tf 1 0 0 1 {x} {y} Tm (A) Tj ET']
point=(x+22,y+22)
commands+=start+body+['Q']
rows.append({'page':len(writer.pages),'profile':profile_id,'case':case,'column':column,
'graphicsStateIntent':INTENTS[intent_index] if case!='default-relative' else 'default',
'expectedIntent':INTENTS[expected],'expectedIntentIndex':expected,'pointPt':list(point),
'components':list(components),'commandSequence':start+body+['Q'],
**({'shading':{'xStart':x if case=='axial-shading' else 0,'xEnd':x+90 if case=='axial-shading' else WIDTH,
'c0':[.1,.3,.6,.2],'c1':[.5,.7,.9,.4]}}
if case=='axial-shading' or case.startswith('shading-pattern-') else {})})
content=DecodedStreamObject();content.set_data(('\n'.join(commands)+'\n').encode())
page[name('Contents')]=writer._add_object(content)
pdf=output/'rendering-intents.pdf'
with pdf.open('wb') as target:writer.write(target)
manifest={'schemaVersion':1,'file':pdf.name,'sha256':sha(pdf.read_bytes()),'pageSizePt':[WIDTH,HEIGHT],
'pageCount':len(writer.pages),'profiles':profile_records,'baseProfileSha256':sha(base),'sampleBytes':list(SAMPLE_BYTES),
'intentOrder':list(INTENTS),'cases':list(CASES),'probes':rows,'tolerance8Bit':4,
'generatorSha256':sha(Path(__file__).read_bytes()),
'derivedProfileModels':{'A2B0':['100-12*C-15*M-10*Y-35*K','18*M-15*C','20*Y-12*C'],
'A2B1':['100-20*C-20*M-20*Y-40*K','40*M-20*C-20*K','40*Y-20*C-20*K'],
'A2B2':['100-25*C-25*M-20*Y-30*K','-40*M+20*C+20*K','35*Y-20*M-15*K']},
'references':['https://opensource.adobe.com/dc-acrobat-sdk-docs/standards/pdfstandards/pdf/PDF32000_2008.pdf',
'https://archive.color.org/files/icc32.pdf','https://www.littlecms.com/'],
'limits':['No measured press profile, human-approved golden, display calibration or broad ICC conformance claim.',
'Independent direct-CMM oracle can share LittleCMS code with PDF renderers; not a separate CMM implementation.',
'Text uses an interior point of the standard Helvetica-Bold H; axial samples account for raster pixel centers.']}
(output/'manifest.json').write_text(json.dumps(manifest,indent=2)+'\n')
print(json.dumps({'fileSha256':manifest['sha256'],'pages':len(writer.pages),'probes':len(rows)}))
if __name__=='__main__':
parser=argparse.ArgumentParser(description=__doc__);parser.add_argument('--output',required=True,type=Path)
generate(parser.parse_args().output)
tests/fixtures/pdf/headings/README.md
Vendored
+68
View File
@@ -0,0 +1,68 @@
# Tagged-PDF boundary corpus
The 26 tagged PDFs and 4 context-only PDFs are self-authored. They contain
Standard-14 Helvetica text and vector paths, with no third-party document content.
Regenerate the deterministic files and hashes with:
```sh
python3 tests/fixtures/pdf/generate_headings.py
```
`manifest.json` records SHA-256, byte size, authored expectations and PDF-point
regions `[left, bottom, right, top]`. Regions follow the source placements,
font sizes and transforms; they are not copied from renderer output. None of the
tagged files has bookmarks, so an outline cannot conceal missing tag extraction.
`test_pdf::taggedBoundaryCorpus` checks every tagged file's hash, page count,
title, level, structure order, destination precision, fallback reason and bbox.
Exact targets must map inside the page and round-trip within 0.5 pt at scale 2
for user rotations 0, 90, 180 and 270 degrees. The PDFium drawing engine still
provides character boxes; a shared worker-only qpdf context now supplies the
source-qualified logical structure.
| Files | Authored condition and expected behavior |
| --- | --- |
| `transformed-form.pdf` | Page wrapper MCID plus Form-owned MCR. Exact heading in `[70,102,240,130]`. |
| `rotated-tags.pdf` | Rotate 90 and offset CropBox. H3 is drawn first but H1 precedes it in structure order; exact targets follow structure order. |
| `cross-page-structure.pdf` | One H1 has interleaved nested Span and direct MCR children on two pages. Page-local fragments preserve K order without mixing reused MCID 0. |
| `vector-heading.pdf` | H4 vector with ActualText but no glyph bbox. Title retained with page precision and a coordinate reason. |
| `form-only-structure.pdf` | Only the Form has StructParents. One exact H2, `Form only heading`, in `[50,125,190,147]`. This is now a positive extraction case. |
| `form-vector-mcid-collision.pdf` | Page vector heading and unrelated Form text both use MCID 0. Keep the heading title at page precision; never borrow Form paragraph bounds. |
| `named-property-rolemap.pdf` | Form BDC references a named property and a custom RoleMap name. Extract actual text without a supplied title. |
| `nested-form-matrix.pdf` | Nested Forms with a nonidentity Form Matrix and independent caller CTMs. Exact target uses PDFium page-space character boxes once. |
| `different-form-same-mcid.pdf` | Two uniquely positioned Forms reuse MCID 0, with reverse structure order. Keep separate exact headings in that order. |
| `ambiguous-form-matrix.pdf` | Different Forms share matrix and MCID signature. Use titles and explicit page precision instead of relying on enumeration order. |
| `repeated-form.pdf` | One source Form is invoked twice. Preserve the heading at page precision with a repeated-Form reason. |
| `unused-form-resource.pdf` | Uninvoked Form remains in Resources. It must not create an extra heading. |
| `inline-image-before-form.pdf` | Inline image precedes a used Form. Skip image pixels and retain the exact heading. |
| `cyclic-form.pdf`, `cyclic-rolemap.pdf`, `cyclic-parent.pdf`, `cyclic-children.pdf`, `cyclic-number-tree.pdf` | Explicit structure limitation, no fabricated destination. |
| `broken-mcr-page.pdf`, `broken-mcr-stream.pdf`, `annotation-owned-mcr.pdf`, `broken-graphics-stack.pdf`, `conflicting-parent-tree-owner.pdf` | Invalid or unsupported ownership/graphics relation. Explicit limitation, no borrowed exact text or position. |
| `stream-quota-at-limit.pdf`, `stream-quota-over-limit.pdf` | Compressed operator data expands to exactly 16 MiB or one byte more. The first succeeds; the second is limited, truncated and incomplete. |
| `bounded-heading-response.pdf` | 70 vector headings have long Japanese ActualText. Return a nonempty validated prefix within 512 KiB CBOR and mark the response incomplete. |
Context-only cases exercise the shared parser separately from PDFium:
- `context-deep-tree.pdf`: a depth failure must remain sticky on retries; the
valid later sibling must never become page zero.
- `context-warnings-32.pdf` and `context-warnings-33.pdf`: one diagnostic occurs
during initial parse and the remainder during lazy object reads. The 32-line
boundary succeeds; crossing it remains limited after a new operation begins.
- `context-warning-bytes.pdf`: one initial diagnostic exceeds 64 KiB. Opening
stops without emitting its contents.
The existing `../missing-first-page.pdf` separately checks that qpdf does not
repair `[null, page]` into a shortened page list before validation. This caught
the qpdf 12.4.1 `setMaxWarnings` startup side effect; diagnostics now use a bounded
private sink instead.
The production worker suite uses the real sandbox, IPC and font broker for
Form-only/nested exact destinations, ambiguous/repeated page fallbacks, quota,
partial response and cancellation. A temporary encrypted version checks wrong
password rejection, the same borrowed QFile and unchanged source bytes. Worker
cases compare original SHA-256 before and after. See the [recorded Linux
results](../../../results/pdf-structure/README.md).
Annotation-owned OBJR/StmOwn structures remain an explicit limitation. Repeated
or ambiguous Form instances deliberately retain page precision. These synthetic
cases do not establish native Windows behavior or general conformance for every
producer's tagged PDFs.
tests/fixtures/pdf/headings/ambiguous-form-matrix.pdf
Vendored
+63
View File
@@ -0,0 +1,63 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R /Fm1 8 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 37 >>
stream
q 1 0 0 1 40 100 cm /Fm0 Do /Fm1 Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [9 0 R 10 0 R] /ParentTree << /Kids [11 0 R] >> >>
endobj
7 0 obj
<< /Length 73 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Do not borrow first) Tj ET EMC
endstream
endobj
8 0 obj
<< /Length 74 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 1 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Do not borrow second) Tj ET EMC
endstream
endobj
9 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /T (First structure) /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
10 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /T (Second structure) /K << /Type /MCR /Pg 3 0 R /Stm 8 0 R /MCID 0 >> >>
endobj
11 0 obj
<< /Limits [0 1] /Nums [0 [9 0 R] 1 [10 0 R]] >>
endobj
xref
0 12
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000313 00000 n
0000000401 00000 n
0000000471 00000 n
0000000566 00000 n
0000000880 00000 n
0000001195 00000 n
0000001330 00000 n
0000001467 00000 n
trailer
<< /Size 12 /Root 1 0 R >>
startxref
1532
%%EOF
tests/fixtures/pdf/headings/annotation-owned-mcr.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 7 >>
stream
/Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 70 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Annotation-owned) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /StmOwn 3 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000359 00000 n
0000000429 00000 n
0000000516 00000 n
0000000816 00000 n
0000000945 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
998
%%EOF
tests/fixtures/pdf/headings/bounded-heading-response.pdf
Vendored
+382
View File
File diff suppressed because one or more lines are too long
tests/fixtures/pdf/headings/broken-graphics-stack.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 9 >>
stream
Q /Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 68 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Graphics stack) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000361 00000 n
0000000431 00000 n
0000000518 00000 n
0000000816 00000 n
0000000931 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
984
%%EOF
tests/fixtures/pdf/headings/broken-mcr-page.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 7 >>
stream
/Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 64 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Wrong page) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 7 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000359 00000 n
0000000429 00000 n
0000000516 00000 n
0000000810 00000 n
0000000925 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
978
%%EOF
tests/fixtures/pdf/headings/broken-mcr-stream.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 7 >>
stream
/Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 66 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Wrong stream) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 3 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000359 00000 n
0000000429 00000 n
0000000516 00000 n
0000000812 00000 n
0000000927 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
980
%%EOF
tests/fixtures/pdf/headings/conflicting-parent-tree-owner.pdf
Vendored
+63
View File
@@ -0,0 +1,63 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R /Fm1 8 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 58 >>
stream
q 1 0 0 1 30 180 cm /Fm0 Do Q q 1 0 0 1 80 60 cm /Fm1 Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [9 0 R 10 0 R] /ParentTree << /Kids [11 0 R] >> >>
endobj
7 0 obj
<< /Length 65 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (First owner) Tj ET EMC
endstream
endobj
8 0 obj
<< /Length 66 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 1 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Second owner) Tj ET EMC
endstream
endobj
9 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 8 0 R /MCID 0 >> >>
endobj
10 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 8 0 R /MCID 0 >> >>
endobj
11 0 obj
<< /Limits [0 1] /Nums [0 [9 0 R] 1 [10 0 R]] >>
endobj
xref
0 12
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000313 00000 n
0000000422 00000 n
0000000492 00000 n
0000000587 00000 n
0000000893 00000 n
0000001200 00000 n
0000001315 00000 n
0000001431 00000 n
trailer
<< /Size 12 /Root 1 0 R >>
startxref
1496
%%EOF
tests/fixtures/pdf/headings/context-deep-tree.pdf
Vendored
+290
View File
@@ -0,0 +1,290 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 2 /Kids [3 0 R 70 0 R] >>
endobj
3 0 obj
<< /Type /Pages /Parent 2 0 R /Count 1 /Kids [4 0 R] >>
endobj
4 0 obj
<< /Type /Pages /Parent 3 0 R /Count 1 /Kids [5 0 R] >>
endobj
5 0 obj
<< /Type /Pages /Parent 4 0 R /Count 1 /Kids [6 0 R] >>
endobj
6 0 obj
<< /Type /Pages /Parent 5 0 R /Count 1 /Kids [7 0 R] >>
endobj
7 0 obj
<< /Type /Pages /Parent 6 0 R /Count 1 /Kids [8 0 R] >>
endobj
8 0 obj
<< /Type /Pages /Parent 7 0 R /Count 1 /Kids [9 0 R] >>
endobj
9 0 obj
<< /Type /Pages /Parent 8 0 R /Count 1 /Kids [10 0 R] >>
endobj
10 0 obj
<< /Type /Pages /Parent 9 0 R /Count 1 /Kids [11 0 R] >>
endobj
11 0 obj
<< /Type /Pages /Parent 10 0 R /Count 1 /Kids [12 0 R] >>
endobj
12 0 obj
<< /Type /Pages /Parent 11 0 R /Count 1 /Kids [13 0 R] >>
endobj
13 0 obj
<< /Type /Pages /Parent 12 0 R /Count 1 /Kids [14 0 R] >>
endobj
14 0 obj
<< /Type /Pages /Parent 13 0 R /Count 1 /Kids [15 0 R] >>
endobj
15 0 obj
<< /Type /Pages /Parent 14 0 R /Count 1 /Kids [16 0 R] >>
endobj
16 0 obj
<< /Type /Pages /Parent 15 0 R /Count 1 /Kids [17 0 R] >>
endobj
17 0 obj
<< /Type /Pages /Parent 16 0 R /Count 1 /Kids [18 0 R] >>
endobj
18 0 obj
<< /Type /Pages /Parent 17 0 R /Count 1 /Kids [19 0 R] >>
endobj
19 0 obj
<< /Type /Pages /Parent 18 0 R /Count 1 /Kids [20 0 R] >>
endobj
20 0 obj
<< /Type /Pages /Parent 19 0 R /Count 1 /Kids [21 0 R] >>
endobj
21 0 obj
<< /Type /Pages /Parent 20 0 R /Count 1 /Kids [22 0 R] >>
endobj
22 0 obj
<< /Type /Pages /Parent 21 0 R /Count 1 /Kids [23 0 R] >>
endobj
23 0 obj
<< /Type /Pages /Parent 22 0 R /Count 1 /Kids [24 0 R] >>
endobj
24 0 obj
<< /Type /Pages /Parent 23 0 R /Count 1 /Kids [25 0 R] >>
endobj
25 0 obj
<< /Type /Pages /Parent 24 0 R /Count 1 /Kids [26 0 R] >>
endobj
26 0 obj
<< /Type /Pages /Parent 25 0 R /Count 1 /Kids [27 0 R] >>
endobj
27 0 obj
<< /Type /Pages /Parent 26 0 R /Count 1 /Kids [28 0 R] >>
endobj
28 0 obj
<< /Type /Pages /Parent 27 0 R /Count 1 /Kids [29 0 R] >>
endobj
29 0 obj
<< /Type /Pages /Parent 28 0 R /Count 1 /Kids [30 0 R] >>
endobj
30 0 obj
<< /Type /Pages /Parent 29 0 R /Count 1 /Kids [31 0 R] >>
endobj
31 0 obj
<< /Type /Pages /Parent 30 0 R /Count 1 /Kids [32 0 R] >>
endobj
32 0 obj
<< /Type /Pages /Parent 31 0 R /Count 1 /Kids [33 0 R] >>
endobj
33 0 obj
<< /Type /Pages /Parent 32 0 R /Count 1 /Kids [34 0 R] >>
endobj
34 0 obj
<< /Type /Pages /Parent 33 0 R /Count 1 /Kids [35 0 R] >>
endobj
35 0 obj
<< /Type /Pages /Parent 34 0 R /Count 1 /Kids [36 0 R] >>
endobj
36 0 obj
<< /Type /Pages /Parent 35 0 R /Count 1 /Kids [37 0 R] >>
endobj
37 0 obj
<< /Type /Pages /Parent 36 0 R /Count 1 /Kids [38 0 R] >>
endobj
38 0 obj
<< /Type /Pages /Parent 37 0 R /Count 1 /Kids [39 0 R] >>
endobj
39 0 obj
<< /Type /Pages /Parent 38 0 R /Count 1 /Kids [40 0 R] >>
endobj
40 0 obj
<< /Type /Pages /Parent 39 0 R /Count 1 /Kids [41 0 R] >>
endobj
41 0 obj
<< /Type /Pages /Parent 40 0 R /Count 1 /Kids [42 0 R] >>
endobj
42 0 obj
<< /Type /Pages /Parent 41 0 R /Count 1 /Kids [43 0 R] >>
endobj
43 0 obj
<< /Type /Pages /Parent 42 0 R /Count 1 /Kids [44 0 R] >>
endobj
44 0 obj
<< /Type /Pages /Parent 43 0 R /Count 1 /Kids [45 0 R] >>
endobj
45 0 obj
<< /Type /Pages /Parent 44 0 R /Count 1 /Kids [46 0 R] >>
endobj
46 0 obj
<< /Type /Pages /Parent 45 0 R /Count 1 /Kids [47 0 R] >>
endobj
47 0 obj
<< /Type /Pages /Parent 46 0 R /Count 1 /Kids [48 0 R] >>
endobj
48 0 obj
<< /Type /Pages /Parent 47 0 R /Count 1 /Kids [49 0 R] >>
endobj
49 0 obj
<< /Type /Pages /Parent 48 0 R /Count 1 /Kids [50 0 R] >>
endobj
50 0 obj
<< /Type /Pages /Parent 49 0 R /Count 1 /Kids [51 0 R] >>
endobj
51 0 obj
<< /Type /Pages /Parent 50 0 R /Count 1 /Kids [52 0 R] >>
endobj
52 0 obj
<< /Type /Pages /Parent 51 0 R /Count 1 /Kids [53 0 R] >>
endobj
53 0 obj
<< /Type /Pages /Parent 52 0 R /Count 1 /Kids [54 0 R] >>
endobj
54 0 obj
<< /Type /Pages /Parent 53 0 R /Count 1 /Kids [55 0 R] >>
endobj
55 0 obj
<< /Type /Pages /Parent 54 0 R /Count 1 /Kids [56 0 R] >>
endobj
56 0 obj
<< /Type /Pages /Parent 55 0 R /Count 1 /Kids [57 0 R] >>
endobj
57 0 obj
<< /Type /Pages /Parent 56 0 R /Count 1 /Kids [58 0 R] >>
endobj
58 0 obj
<< /Type /Pages /Parent 57 0 R /Count 1 /Kids [59 0 R] >>
endobj
59 0 obj
<< /Type /Pages /Parent 58 0 R /Count 1 /Kids [60 0 R] >>
endobj
60 0 obj
<< /Type /Pages /Parent 59 0 R /Count 1 /Kids [61 0 R] >>
endobj
61 0 obj
<< /Type /Pages /Parent 60 0 R /Count 1 /Kids [62 0 R] >>
endobj
62 0 obj
<< /Type /Pages /Parent 61 0 R /Count 1 /Kids [63 0 R] >>
endobj
63 0 obj
<< /Type /Pages /Parent 62 0 R /Count 1 /Kids [64 0 R] >>
endobj
64 0 obj
<< /Type /Pages /Parent 63 0 R /Count 1 /Kids [65 0 R] >>
endobj
65 0 obj
<< /Type /Pages /Parent 64 0 R /Count 1 /Kids [66 0 R] >>
endobj
66 0 obj
<< /Type /Pages /Parent 65 0 R /Count 1 /Kids [67 0 R] >>
endobj
67 0 obj
<< /Type /Pages /Parent 66 0 R /Count 1 /Kids [68 0 R] >>
endobj
68 0 obj
<< /Type /Pages /Parent 67 0 R /Count 1 /Kids [69 0 R] >>
endobj
69 0 obj
<< /Type /Page /Parent 68 0 R /MediaBox [0 0 100 100] /Resources << >> >>
endobj
70 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << >> >>
endobj
xref
0 71
0000000000 65535 f
0000000015 00000 n
0000000064 00000 n
0000000128 00000 n
0000000199 00000 n
0000000270 00000 n
0000000341 00000 n
0000000412 00000 n
0000000483 00000 n
0000000554 00000 n
0000000626 00000 n
0000000699 00000 n
0000000773 00000 n
0000000847 00000 n
0000000921 00000 n
0000000995 00000 n
0000001069 00000 n
0000001143 00000 n
0000001217 00000 n
0000001291 00000 n
0000001365 00000 n
0000001439 00000 n
0000001513 00000 n
0000001587 00000 n
0000001661 00000 n
0000001735 00000 n
0000001809 00000 n
0000001883 00000 n
0000001957 00000 n
0000002031 00000 n
0000002105 00000 n
0000002179 00000 n
0000002253 00000 n
0000002327 00000 n
0000002401 00000 n
0000002475 00000 n
0000002549 00000 n
0000002623 00000 n
0000002697 00000 n
0000002771 00000 n
0000002845 00000 n
0000002919 00000 n
0000002993 00000 n
0000003067 00000 n
0000003141 00000 n
0000003215 00000 n
0000003289 00000 n
0000003363 00000 n
0000003437 00000 n
0000003511 00000 n
0000003585 00000 n
0000003659 00000 n
0000003733 00000 n
0000003807 00000 n
0000003881 00000 n
0000003955 00000 n
0000004029 00000 n
0000004103 00000 n
0000004177 00000 n
0000004251 00000 n
0000004325 00000 n
0000004399 00000 n
0000004473 00000 n
0000004547 00000 n
0000004621 00000 n
0000004695 00000 n
0000004769 00000 n
0000004843 00000 n
0000004917 00000 n
0000004991 00000 n
0000005081 00000 n
trailer
<< /Size 71 /Root 1 0 R >>
startxref
5170
%%EOF
tests/fixtures/pdf/headings/context-warning-bytes.pdf
Vendored
+22
View File
File diff suppressed because one or more lines are too long
tests/fixtures/pdf/headings/context-warnings-32.pdf
Vendored
+146
View File
@@ -0,0 +1,146 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /Duplicate 1 /Duplicate 2 /Probe [4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R 15 0 R 16 0 R 17 0 R 18 0 R 19 0 R 20 0 R 21 0 R 22 0 R 23 0 R 24 0 R 25 0 R 26 0 R 27 0 R 28 0 R 29 0 R 30 0 R 31 0 R 32 0 R 33 0 R 34 0 R] >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources << >> >>
endobj
4 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
5 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
6 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
7 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
8 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
9 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
10 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
11 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
12 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
13 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
14 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
15 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
16 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
17 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
18 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
19 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
20 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
21 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
22 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
23 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
24 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
25 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
26 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
27 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
28 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
29 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
30 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
31 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
32 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
33 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
34 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
xref
0 35
0000000000 65535 f
0000000015 00000 n
0000000310 00000 n
0000000367 00000 n
0000000455 00000 n
0000000502 00000 n
0000000549 00000 n
0000000596 00000 n
0000000643 00000 n
0000000690 00000 n
0000000737 00000 n
0000000785 00000 n
0000000833 00000 n
0000000881 00000 n
0000000929 00000 n
0000000977 00000 n
0000001025 00000 n
0000001073 00000 n
0000001121 00000 n
0000001169 00000 n
0000001217 00000 n
0000001265 00000 n
0000001313 00000 n
0000001361 00000 n
0000001409 00000 n
0000001457 00000 n
0000001505 00000 n
0000001553 00000 n
0000001601 00000 n
0000001649 00000 n
0000001697 00000 n
0000001745 00000 n
0000001793 00000 n
0000001841 00000 n
0000001889 00000 n
trailer
<< /Size 35 /Root 1 0 R >>
startxref
1937
%%EOF
tests/fixtures/pdf/headings/context-warnings-33.pdf
Vendored
+150
View File
@@ -0,0 +1,150 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /Duplicate 1 /Duplicate 2 /Probe [4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R 11 0 R 12 0 R 13 0 R 14 0 R 15 0 R 16 0 R 17 0 R 18 0 R 19 0 R 20 0 R 21 0 R 22 0 R 23 0 R 24 0 R 25 0 R 26 0 R 27 0 R 28 0 R 29 0 R 30 0 R 31 0 R 32 0 R 33 0 R 34 0 R 35 0 R] >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources << >> >>
endobj
4 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
5 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
6 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
7 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
8 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
9 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
10 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
11 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
12 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
13 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
14 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
15 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
16 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
17 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
18 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
19 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
20 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
21 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
22 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
23 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
24 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
25 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
26 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
27 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
28 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
29 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
30 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
31 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
32 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
33 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
34 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
35 0 obj
<< /Duplicate 1 /Duplicate 2 >>
endobj
xref
0 36
0000000000 65535 f
0000000015 00000 n
0000000317 00000 n
0000000374 00000 n
0000000462 00000 n
0000000509 00000 n
0000000556 00000 n
0000000603 00000 n
0000000650 00000 n
0000000697 00000 n
0000000744 00000 n
0000000792 00000 n
0000000840 00000 n
0000000888 00000 n
0000000936 00000 n
0000000984 00000 n
0000001032 00000 n
0000001080 00000 n
0000001128 00000 n
0000001176 00000 n
0000001224 00000 n
0000001272 00000 n
0000001320 00000 n
0000001368 00000 n
0000001416 00000 n
0000001464 00000 n
0000001512 00000 n
0000001560 00000 n
0000001608 00000 n
0000001656 00000 n
0000001704 00000 n
0000001752 00000 n
0000001800 00000 n
0000001848 00000 n
0000001896 00000 n
0000001944 00000 n
trailer
<< /Size 36 /Root 1 0 R >>
startxref
1992
%%EOF
tests/fixtures/pdf/headings/cross-page-structure.pdf
Vendored
+75
View File
@@ -0,0 +1,75 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 8 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 2 /Kids [3 0 R 4 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 0 /Resources << /Font << /F1 7 0 R >> >> /Contents 5 0 R >>
endobj
4 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 1 /Resources << /Font << /F1 7 0 R >> >> /Contents 6 0 R >>
endobj
5 0 obj
<< /Length 134 >>
stream
/Span <</MCID 0>> BDC BT /F1 16 Tf 40 60 Td (Across page one) Tj ET EMC
/H1 <</MCID 1>> BDC BT /F1 16 Tf 170 60 Td ( middle) Tj ET EMC
endstream
endobj
6 0 obj
<< /Length 219 >>
stream
/P <</MCID 2>> BDC BT /F1 16 Tf 40 120 Td (Unrelated paragraph) Tj ET EMC
/H2 <</MCID 1>> BDC BT /F1 16 Tf 40 160 Td (Following heading) Tj ET EMC
/Span <</MCID 0>> BDC BT /F1 16 Tf 40 210 Td (Across page two) Tj ET EMC
endstream
endobj
7 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
8 0 obj
<< /Type /StructTreeRoot /K [9 0 R] /ParentTree << /Nums [0 [12 0 R 10 0 R] 1 [13 0 R 11 0 R 14 0 R]] >> /ParentTreeNextKey 2 >>
endobj
9 0 obj
<< /Type /StructElem /S /Sect /P 8 0 R /K [10 0 R 11 0 R 14 0 R] >>
endobj
10 0 obj
<< /Type /StructElem /S /H1 /P 9 0 R /T (Across pages) /K [12 0 R << /Type /MCR /Pg 3 0 R /MCID 1 >> 13 0 R] >>
endobj
11 0 obj
<< /Type /StructElem /S /H2 /P 9 0 R /Pg 4 0 R /K 1 >>
endobj
12 0 obj
<< /Type /StructElem /S /Span /P 10 0 R /Pg 3 0 R /K 0 >>
endobj
13 0 obj
<< /Type /StructElem /S /Span /P 10 0 R /Pg 4 0 R /K 0 >>
endobj
14 0 obj
<< /Type /StructElem /S /P /P 9 0 R /Pg 4 0 R /K 2 >>
endobj
xref
0 15
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000178 00000 n
0000000321 00000 n
0000000464 00000 n
0000000650 00000 n
0000000921 00000 n
0000000991 00000 n
0000001135 00000 n
0000001218 00000 n
0000001346 00000 n
0000001417 00000 n
0000001491 00000 n
0000001565 00000 n
trailer
<< /Size 15 /Root 1 0 R >>
startxref
1635
%%EOF
tests/fixtures/pdf/headings/cyclic-children.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 7 >>
stream
/Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 65 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Child cycle) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K 8 0 R >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000359 00000 n
0000000429 00000 n
0000000516 00000 n
0000000811 00000 n
0000000885 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
938
%%EOF
tests/fixtures/pdf/headings/cyclic-form.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 7 >>
stream
/Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 7 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/Fm0 Do
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000359 00000 n
0000000429 00000 n
0000000516 00000 n
0000000752 00000 n
0000000867 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
920
%%EOF
tests/fixtures/pdf/headings/cyclic-number-tree.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 7 >>
stream
/Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 65 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Number tree) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Kids [9 0 R] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000359 00000 n
0000000429 00000 n
0000000516 00000 n
0000000811 00000 n
0000000926 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
975
%%EOF
tests/fixtures/pdf/headings/cyclic-parent.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 7 >>
stream
/Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 66 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Parent cycle) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 8 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000359 00000 n
0000000429 00000 n
0000000516 00000 n
0000000812 00000 n
0000000927 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
980
%%EOF
tests/fixtures/pdf/headings/cyclic-rolemap.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 7 >>
stream
/Fm0 Do
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> /RoleMap << /Foo /Bar /Bar /Foo >> >>
endobj
7 0 obj
<< /Length 69 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Hidden by cycle) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /Foo /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000359 00000 n
0000000429 00000 n
0000000550 00000 n
0000000849 00000 n
0000000965 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
1018
%%EOF
tests/fixtures/pdf/headings/different-form-same-mcid.pdf
Vendored
+63
View File
@@ -0,0 +1,63 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R /Fm1 8 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 58 >>
stream
q 1 0 0 1 30 180 cm /Fm0 Do Q q 1 0 0 1 80 60 cm /Fm1 Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [10 0 R 9 0 R] /ParentTree << /Kids [11 0 R] >> >>
endobj
7 0 obj
<< /Length 64 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (First form) Tj ET EMC
endstream
endobj
8 0 obj
<< /Length 65 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 1 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Second form) Tj ET EMC
endstream
endobj
9 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
10 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 8 0 R /MCID 0 >> >>
endobj
11 0 obj
<< /Limits [0 1] /Nums [0 [9 0 R] 1 [10 0 R]] >>
endobj
xref
0 12
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000313 00000 n
0000000422 00000 n
0000000492 00000 n
0000000587 00000 n
0000000892 00000 n
0000001198 00000 n
0000001313 00000 n
0000001429 00000 n
trailer
<< /Size 12 /Root 1 0 R >>
startxref
1494
%%EOF
tests/fixtures/pdf/headings/form-only-structure.pdf
Vendored
+48
View File
@@ -0,0 +1,48 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 7 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /Resources << /XObject << /Fm 6 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 28 >>
stream
q 1 0 0 1 40 100 cm /Fm Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Length 71 /Type /XObject /Subtype /Form /BBox [0 0 200 80] /StructParents 0 /Resources << /Font << /F1 5 0 R >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 16 Tf 10 30 Td (Form only heading) Tj ET EMC
endstream
endobj
7 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Nums [0 [8 0 R]] >> /ParentTreeNextKey 1 >>
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 7 0 R /Pg 3 0 R /T (Form only heading) /K << /Type /MCR /Pg 3 0 R /Stm 6 0 R /MCID 0 >> >>
endobj
xref
0 9
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000301 00000 n
0000000380 00000 n
0000000450 00000 n
0000000676 00000 n
0000000787 00000 n
trailer
<< /Size 9 /Root 1 0 R >>
startxref
924
%%EOF
tests/fixtures/pdf/headings/form-vector-mcid-collision.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 7 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 0 /Resources << /XObject << /Fm 6 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 80 >>
stream
/H1 <</MCID 0>> BDC 0 0 1 rg 40 180 180 20 re f EMC q 1 0 0 1 40 100 cm /Fm Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Length 77 /Type /XObject /Subtype /Form /BBox [0 0 240 80] /StructParents 1 /Resources << /Font << /F1 5 0 R >> >> >>
stream
/P <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Unrelated Form paragraph) Tj ET EMC
endstream
endobj
7 0 obj
<< /Type /StructTreeRoot /K [8 0 R 9 0 R] /ParentTree << /Nums [0 [8 0 R] 1 [9 0 R]] >> /ParentTreeNextKey 2 >>
endobj
8 0 obj
<< /Type /StructElem /S /H1 /P 7 0 R /Pg 3 0 R /T (Page vector heading) /K 0 >>
endobj
9 0 obj
<< /Type /StructElem /S /P /P 7 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 6 0 R /MCID 0 >> >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000318 00000 n
0000000449 00000 n
0000000519 00000 n
0000000751 00000 n
0000000878 00000 n
0000000973 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
1086
%%EOF
tests/fixtures/pdf/headings/inline-image-before-form.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 64 >>
stream
q 1 0 0 1 40 100 cm BI /W 1 /H 1 /BPC 8 /CS /G ID X EI /Fm0 Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 65 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (After image) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000417 00000 n
0000000487 00000 n
0000000574 00000 n
0000000869 00000 n
0000000984 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
1037
%%EOF
tests/fixtures/pdf/headings/manifest.json
Vendored
+537
View File
@@ -0,0 +1,537 @@
{
"origin": "Self-created deterministic PDF syntax, Helvetica Standard-14 text and vector paths; no third-party document content.",
"generator": "../generate_headings.py",
"documents": [
{
"file": "transformed-form.pdf",
"bytes": 1236,
"sha256": "55b6ee047e70b0dc463ac4b59ab3464bbb45a4cab12521ba23de41bc95197b98",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Form heading",
"level": 2,
"precision": "exact",
"text_region_pt": [
70,
102,
240,
130
]
}
],
"form_matrix": [
1.5,
0,
0,
1.25,
55,
70
],
"local_baseline": [
10,
30
]
}
},
{
"file": "rotated-tags.pdf",
"bytes": 1095,
"sha256": "8f2b850785b13d0de9ada1e8e31b15e6fe381f444eaaa1397acda481d1612c1e",
"expected": {
"pages": 1,
"crop_box": [
20,
30,
280,
230
],
"rotation": 90,
"headings": [
{
"page": 0,
"title": "Rotated first",
"level": 1,
"precision": "exact",
"text_region_pt": [
50,
168,
170,
190
]
},
{
"page": 0,
"title": "Rotated second",
"level": 3,
"precision": "exact",
"text_region_pt": [
60,
78,
180,
96
]
}
],
"order": "structure order, intentionally different from content-stream draw order"
}
},
{
"file": "cross-page-structure.pdf",
"bytes": 2001,
"sha256": "52ec19216440feebdb80b2d25339282b82b219a017fdf7c057c618f92db89373",
"expected": {
"pages": 2,
"headings": [
{
"page": 0,
"title": "Across page one middle",
"level": 1,
"precision": "exact",
"text_region_pt": [
40,
55,
230,
78
]
},
{
"page": 1,
"title": "Across page two",
"level": 1,
"precision": "exact",
"text_region_pt": [
40,
205,
170,
228
]
},
{
"page": 1,
"title": "Following heading",
"level": 2,
"precision": "exact",
"text_region_pt": [
40,
155,
180,
178
]
}
],
"scope": "One heading spans two page-local Span elements with an interleaved direct MCR; MCID 0 is reused independently on both pages. Results are page-local fragments."
}
},
{
"file": "vector-heading.pdf",
"bytes": 787,
"sha256": "0083680fe3b8d27525b4e9b442155444095b19a9f3d7921696531be12dcf2175",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Vector section",
"level": 4,
"precision": "page",
"reason_contains": "座標"
}
],
"drawn_region_pt": [
40,
180,
190,
200
],
"fallback": "No text glyph bounding box is available for the marked vector path."
}
},
{
"file": "form-only-structure.pdf",
"bytes": 1167,
"sha256": "4ad6e541fe9df9d656ad78eb477a751eafa22f9147a501dfed68b4fd3ef6e235",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Form only heading",
"level": 2,
"precision": "exact",
"text_region_pt": [
50,
125,
190,
147
]
}
],
"scope": "Form StructParents and source-qualified MCR; no page StructParents."
}
},
{
"file": "form-vector-mcid-collision.pdf",
"bytes": 1352,
"sha256": "bc37979a4cb8a96872fb43c75b183aa82ac64c5934844e8233b79e7910640128",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Page vector heading",
"level": 1,
"precision": "page",
"reason_contains": "MCID"
}
],
"fallback": "MCID 0 names a page vector and unrelated Form text. Non-text page objects must participate in stream collision detection."
}
},
{
"file": "named-property-rolemap.pdf",
"bytes": 1321,
"sha256": "2da7e79e65977378ce7f2114bbfa3df1a23181c633cf7988d533388b4e89f142",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Named property heading",
"level": 3,
"precision": "exact",
"text_region_pt": [
50,
127,
190,
142
]
}
]
}
},
{
"file": "nested-form-matrix.pdf",
"bytes": 1791,
"sha256": "8d872ae76bbd8164b34a4fb9d1a0b2cc73f900f5b2349f5486b10079b7442084",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Nested heading",
"level": 2,
"precision": "exact",
"text_region_pt": [
82,
161,
245,
181
]
}
]
}
},
{
"file": "different-form-same-mcid.pdf",
"bytes": 1800,
"sha256": "d337fb72b0434370a15b1d8470dca33e5492fe4cc01f352506a947fb135cfcf5",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Second form",
"level": 2,
"precision": "exact",
"text_region_pt": [
90,
87,
170,
102
]
},
{
"page": 0,
"title": "First form",
"level": 2,
"precision": "exact",
"text_region_pt": [
40,
207,
105,
222
]
}
]
}
},
{
"file": "ambiguous-form-matrix.pdf",
"bytes": 1838,
"sha256": "25103b29c1fc392378f69e66f480e480868d1d85f764f431bb6002cba3f213ce",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "First structure",
"level": 2,
"precision": "page",
"reason_contains": "一意"
},
{
"page": 0,
"title": "Second structure",
"level": 2,
"precision": "page",
"reason_contains": "一意"
}
]
}
},
{
"file": "repeated-form.pdf",
"bytes": 1302,
"sha256": "d05f07ddedd2a045f3f3f66bc554fad6fb2c651828080472130f139783e9596f",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Repeated heading",
"level": 2,
"precision": "page",
"reason_contains": "複数回"
}
]
}
},
{
"file": "unused-form-resource.pdf",
"bytes": 1776,
"sha256": "c203b8c8bf3afadd76e51692390c81d866fba0ce56b438a7b6613bd75f2bcbad",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "Used heading",
"level": 2,
"precision": "exact",
"text_region_pt": [
50,
127,
140,
142
]
}
]
}
},
{
"file": "inline-image-before-form.pdf",
"bytes": 1303,
"sha256": "3fc737049b0a92d72b813ae5d6455c4bb51da08f561371374b3acc99f7356ae5",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "After image",
"level": 2,
"precision": "exact",
"text_region_pt": [
50,
127,
125,
142
]
}
]
}
},
{
"file": "cyclic-form.pdf",
"bytes": 1185,
"sha256": "3143516932a8c9b4355325d251e0cc5d1af77edd368ad915c0bf6990643cf74a",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "cyclic-rolemap.pdf",
"bytes": 1284,
"sha256": "aa0772a34f4abd035e2e7d4aa6438a49774b791b9520d3269cffa52b1575a2fc",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "cyclic-parent.pdf",
"bytes": 1245,
"sha256": "46e4ac7e96e372b05f35718b675552f58ad618887e53ba31a044da748edd1036",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "cyclic-children.pdf",
"bytes": 1203,
"sha256": "7c9f49595fa3001b2c35c7eaf389d57dc2d6d7c910ef36d35ef0810c8e0d7935",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "broken-mcr-page.pdf",
"bytes": 1243,
"sha256": "aa1722f9ec4e4d5affd531e40a58c76be1ecfb60e4b2b7d20bbab53e459a4abd",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "broken-mcr-stream.pdf",
"bytes": 1245,
"sha256": "c2c20b93bf307df69ab5a5b962f2f399d761a987e372707b830be7ae12839596",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "annotation-owned-mcr.pdf",
"bytes": 1263,
"sha256": "282612ae3bfecdc233e6818ee4a680f3e4541f060979443952b595c9d76943cc",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "broken-graphics-stack.pdf",
"bytes": 1249,
"sha256": "ac27fe4c8e166a4f241737578b9e846df685b83b85cc8d9447f7c33203d66c15",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "cyclic-number-tree.pdf",
"bytes": 1240,
"sha256": "e1adb2ebb053be176f746b89b8e24946b1c9a86cbe05421f7691e49787cbc541",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "conflicting-parent-tree-owner.pdf",
"bytes": 1802,
"sha256": "6def473a2db3cef07e481de5ae6a9ae0837d60da1ceb66ffaa47e250de0b6876",
"expected": {
"pages": 1,
"headings": [],
"limited": true
}
},
{
"file": "stream-quota-at-limit.pdf",
"bytes": 17615,
"sha256": "40a56f12e42fe67ce3b656027a7fc4348a5cb34d5ce8f0f38350567b2e178924",
"expected": {
"pages": 1,
"headings": [
{
"page": 0,
"title": "At stream limit",
"level": 2,
"precision": "exact",
"text_region_pt": [
50,
127,
140,
142
]
}
]
}
},
{
"file": "stream-quota-over-limit.pdf",
"bytes": 17617,
"sha256": "80cf057acd5c6c2454fb6cba20100559bb892e0828a23d3ce50405b5477fb735",
"expected": {
"pages": 1,
"headings": [],
"limited": true,
"truncated": true
}
},
{
"file": "bounded-heading-response.pdf",
"bytes": 1157442,
"sha256": "820ea69c56ac33587114886aded122e9d7705e0ffdccaa6eefbb81ab4a05f6ce",
"expected": {
"pages": 1,
"headings": [],
"response_limited": true
}
}
],
"context_documents": [
{
"file": "context-deep-tree.pdf",
"bytes": 6656,
"sha256": "66dc1bc3c5fd062334fddc9e2f0ef420b168e0ed44397bb2374c6b35695d86de",
"expected": {
"scope": "Context-only regression: after a depth failure, the second page must never be assigned index zero."
}
},
{
"file": "context-warnings-32.pdf",
"bytes": 2703,
"sha256": "1ed5ad72f3054844c158ffff87977616caaca63cbb864a50b1c9be96ac1a5ec6",
"expected": {
"scope": "Context-only cumulative diagnostic limit, starting in initial parse; warning bodies must never reach stdout/stderr.",
"accepted": true
}
},
{
"file": "context-warnings-33.pdf",
"bytes": 2778,
"sha256": "29e105aa8c78d8b6a0ab1a5d7c7f99663abd952f1657c0c17c43ab24406822a1",
"expected": {
"scope": "Context-only cumulative diagnostic limit, starting in initial parse; warning bodies must never reach stdout/stderr.",
"accepted": false
}
},
{
"file": "context-warning-bytes.pdf",
"bytes": 131435,
"sha256": "42c792b2fa035674a22bde94fe681bc89406e3f015be7e32f49d3a3410364319",
"expected": {
"scope": "Context-only diagnostic byte limit during initial parse; no warning text is retained or emitted.",
"accepted": false
}
}
]
}
tests/fixtures/pdf/headings/named-property-rolemap.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 29 >>
stream
q 1 0 0 1 40 100 cm /Fm0 Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> /RoleMap << /Section /H3 >> >>
endobj
7 0 obj
<< /Length 86 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/Section /HeadingProperty BDC BT /F1 12 Tf 10 30 Td (Named property heading) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /Section /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000382 00000 n
0000000452 00000 n
0000000566 00000 n
0000000882 00000 n
0000001002 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
1055
%%EOF
tests/fixtures/pdf/headings/nested-form-matrix.pdf
Vendored
+63
View File
@@ -0,0 +1,63 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R /Fm1 8 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 29 >>
stream
q 1 0 0 1 40 100 cm /Fm0 Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [9 0 R 10 0 R] /ParentTree << /Kids [11 0 R] >> >>
endobj
7 0 obj
<< /Length 33 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> /Matrix [1.2 0 0 0.9 5 7] >>
stream
q 1.5 0 0 1.25 10 20 cm /Fm1 Do Q
endstream
endobj
8 0 obj
<< /Length 68 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 1 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> /Matrix [1 0 0 1 4 6] >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Nested heading) Tj ET EMC
endstream
endobj
9 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
10 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 8 0 R /MCID 0 >> >>
endobj
11 0 obj
<< /Limits [0 1] /Nums [0 [9 0 R] 1 [10 0 R]] >>
endobj
xref
0 12
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000313 00000 n
0000000393 00000 n
0000000463 00000 n
0000000558 00000 n
0000000858 00000 n
0000001189 00000 n
0000001304 00000 n
0000001420 00000 n
trailer
<< /Size 12 /Root 1 0 R >>
startxref
1485
%%EOF
tests/fixtures/pdf/headings/repeated-form.pdf
Vendored
+52
View File
@@ -0,0 +1,52 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 58 >>
stream
q 1 0 0 1 30 180 cm /Fm0 Do Q q 1 0 0 1 80 60 cm /Fm0 Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Kids [9 0 R] >> >>
endobj
7 0 obj
<< /Length 70 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Repeated heading) Tj ET EMC
endstream
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
9 0 obj
<< /Limits [0 0] /Nums [0 [8 0 R]] >>
endobj
xref
0 10
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000302 00000 n
0000000411 00000 n
0000000481 00000 n
0000000568 00000 n
0000000868 00000 n
0000000983 00000 n
trailer
<< /Size 10 /Root 1 0 R >>
startxref
1036
%%EOF
tests/fixtures/pdf/headings/rotated-tags.pdf
Vendored
+46
View File
@@ -0,0 +1,46 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 260] /CropBox [20 30 280 230] /Rotate 90 /StructParents 0 /Resources << /Font << /F1 5 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 137 >>
stream
/H3 <</MCID 1>> BDC BT /F1 16 Tf 60 80 Td (Rotated second) Tj ET EMC
/H1 <</MCID 0>> BDC BT /F1 20 Tf 50 170 Td (Rotated first) Tj ET EMC
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [7 0 R 8 0 R] /ParentTree << /Nums [0 [7 0 R 8 0 R]] >> >>
endobj
7 0 obj
<< /Type /StructElem /S /H1 /P 6 0 R /Pg 3 0 R /K 0 >>
endobj
8 0 obj
<< /Type /StructElem /S /H3 /P 6 0 R /Pg 3 0 R /K 1 >>
endobj
xref
0 9
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000351 00000 n
0000000540 00000 n
0000000610 00000 n
0000000712 00000 n
0000000782 00000 n
trailer
<< /Size 9 /Root 1 0 R >>
startxref
852
%%EOF
tests/fixtures/pdf/headings/stream-quota-at-limit.pdf
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/pdf/headings/stream-quota-over-limit.pdf
Vendored
BIN
View File
Binary file not shown.
tests/fixtures/pdf/headings/transformed-form.pdf
Vendored
+48
View File
@@ -0,0 +1,48 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 7 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 0 /Resources << /XObject << /Fm 6 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 56 >>
stream
/H2 <</MCID 0>> BDC q 1.5 0 0 1.25 55 70 cm /Fm Do Q EMC
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Length 66 /Type /XObject /Subtype /Form /BBox [0 0 160 80] /StructParents 1 /Resources << /Font << /F1 5 0 R >> >> >>
stream
/H2 <</MCID 4>> BDC BT /F1 16 Tf 10 30 Td (Form heading) Tj ET EMC
endstream
endobj
7 0 obj
<< /Type /StructTreeRoot /K [8 0 R] /ParentTree << /Nums [0 [8 0 R] 1 [null null null null 8 0 R]] >> /ParentTreeNextKey 2 >>
endobj
8 0 obj
<< /Type /StructElem /S /H2 /P 7 0 R /Pg 3 0 R /T (Form heading) /K [0 << /Type /MCR /Pg 3 0 R /Stm 6 0 R /MCID 4 >>] >>
endobj
xref
0 9
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000318 00000 n
0000000425 00000 n
0000000495 00000 n
0000000716 00000 n
0000000857 00000 n
trailer
<< /Size 9 /Root 1 0 R >>
startxref
993
%%EOF
tests/fixtures/pdf/headings/unused-form-resource.pdf
Vendored
+63
View File
@@ -0,0 +1,63 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 400 350] /Resources << /XObject << /Fm0 7 0 R /Fm1 8 0 R >> >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 29 >>
stream
q 1 0 0 1 40 100 cm /Fm0 Do Q
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
6 0 obj
<< /Type /StructTreeRoot /K [9 0 R 10 0 R] /ParentTree << /Kids [11 0 R] >> >>
endobj
7 0 obj
<< /Length 66 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 0 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Used heading) Tj ET EMC
endstream
endobj
8 0 obj
<< /Length 68 /Type /XObject /Subtype /Form /BBox [0 0 260 90] /StructParents 1 /Resources << /Font << /F1 5 0 R >> /XObject << /Fm0 7 0 R /Fm1 8 0 R >> /Properties << /HeadingProperty << /MCID 0 >> >> >> >>
stream
/H2 <</MCID 0>> BDC BT /F1 12 Tf 10 30 Td (Unused heading) Tj ET EMC
endstream
endobj
9 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 7 0 R /MCID 0 >> >>
endobj
10 0 obj
<< /Type /StructElem /S /H2 /P 6 0 R /Pg 3 0 R /K << /Type /MCR /Pg 3 0 R /Stm 8 0 R /MCID 0 >> >>
endobj
11 0 obj
<< /Limits [0 1] /Nums [0 [9 0 R] 1 [10 0 R]] >>
endobj
xref
0 12
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000313 00000 n
0000000393 00000 n
0000000463 00000 n
0000000558 00000 n
0000000865 00000 n
0000001174 00000 n
0000001289 00000 n
0000001405 00000 n
trailer
<< /Size 12 /Root 1 0 R >>
startxref
1470
%%EOF
tests/fixtures/pdf/headings/vector-heading.pdf
Vendored
+37
View File
@@ -0,0 +1,37 @@
%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R /MarkInfo << /Marked true >> /StructTreeRoot 5 0 R >>
endobj
2 0 obj
<< /Type /Pages /Count 1 /Kids [3 0 R] >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 320 260] /StructParents 0 /Resources << >> /Contents 4 0 R >>
endobj
4 0 obj
<< /Length 51 >>
stream
/H4 <</MCID 0>> BDC 0 0 1 rg 40 180 150 20 re f EMC
endstream
endobj
5 0 obj
<< /Type /StructTreeRoot /K [6 0 R] /ParentTree << /Nums [0 [6 0 R]] >> >>
endobj
6 0 obj
<< /Type /StructElem /S /H4 /P 5 0 R /Pg 3 0 R /ActualText (Vector section) /K 0 >>
endobj
xref
0 7
0000000000 65535 f
0000000015 00000 n
0000000115 00000 n
0000000172 00000 n
0000000293 00000 n
0000000395 00000 n
0000000485 00000 n
trailer
<< /Size 7 /Root 1 0 R >>
startxref
584
%%EOF
tests/fixtures/pdf/icc_generator.py
Vendored
+117
View File
@@ -0,0 +1,117 @@
#!/usr/bin/env python3
"""Self-authored linear-sRGB ICCBased vector/image fixture; no borrowed ICC data.
Uses public LittleCMS APIs to encode D65 + sRGB primaries + gamma 1. The
color expectation is the analytic sRGB transfer function, independent of PDF
renderer output. Only the ICC creation timestamp is normalized for determinism.
"""
from pathlib import Path
import ctypes as C
import ctypes.util
import hashlib
import json
import shutil
import struct
class xyY(C.Structure):
_fields_ = [("x", C.c_double), ("y", C.c_double), ("Y", C.c_double)]
class Primaries(C.Structure):
_fields_ = [("Red", xyY), ("Green", xyY), ("Blue", xyY)]
def linear_profile():
cms = C.CDLL(ctypes.util.find_library("lcms2") or "liblcms2.so")
cms.cmsGetEncodedCMMversion.restype = C.c_uint32
cms.cmsBuildGamma.argtypes = [C.c_void_p, C.c_double]; cms.cmsBuildGamma.restype = C.c_void_p
cms.cmsFreeToneCurve.argtypes = [C.c_void_p]
cms.cmsCreateRGBProfile.argtypes = [C.POINTER(xyY), C.POINTER(Primaries), C.POINTER(C.c_void_p)]
cms.cmsCreateRGBProfile.restype = C.c_void_p
cms.cmsSaveProfileToMem.argtypes = [C.c_void_p, C.c_void_p, C.POINTER(C.c_uint32)]
cms.cmsSaveProfileToMem.restype = C.c_int
cms.cmsCloseProfile.argtypes = [C.c_void_p]
curve = cms.cmsBuildGamma(None, 1.0)
if not curve: raise RuntimeError("LittleCMS gamma creation failed")
white = xyY(.3127, .3290, 1.)
primaries = Primaries(xyY(.64, .33, 1.), xyY(.30, .60, 1.), xyY(.15, .06, 1.))
curves = (C.c_void_p * 3)(curve, curve, curve)
profile = cms.cmsCreateRGBProfile(C.byref(white), C.byref(primaries), curves)
cms.cmsFreeToneCurve(curve)
if not profile: raise RuntimeError("LittleCMS profile creation failed")
try:
length = C.c_uint32()
if not cms.cmsSaveProfileToMem(profile, None, C.byref(length)): raise RuntimeError("ICC size failed")
buffer = C.create_string_buffer(length.value)
if not cms.cmsSaveProfileToMem(profile, buffer, C.byref(length)): raise RuntimeError("ICC save failed")
data = bytearray(buffer.raw[:length.value])
finally: cms.cmsCloseProfile(profile)
data[24:36] = struct.pack(">6H", 2026, 9, 19, 0, 0, 0)
data[84:100] = bytes(16) # Zero profile ID is permitted; no stale timestamp-derived ID.
assert data[36:40] == b"acsp" and data[16:20] == b"RGB "
return bytes(data), cms.cmsGetEncodedCMMversion()
def srgb(value):
encoded = 12.92 * value if value <= .0031308 else 1.055 * value ** (1 / 2.4) - .055
return round(255 * encoded)
def pdf_stream(data, extra=b""):
return b"<< /Length " + str(len(data)).encode() + b" " + extra + b" >>\nstream\n" + data + b"\nendstream"
def generate(out):
out = Path(out); out.mkdir(parents=True, exist_ok=True)
profile, version = linear_profile()
(out / "linear-srgb.icc").write_bytes(profile)
license_path = Path("/usr/share/licenses/lcms2/LICENSE")
if license_path.is_file(): shutil.copyfile(license_path, out / "LCMS2-LICENSE.txt")
values = [(.0625, .0625, .0625), (.25, .25, .25), (.5, .5, .5), (.75, .75, .75), (.5, .125, .0625), (.0625, .25, .5)]
content = ["0 0 0 rg BT /F1 16 Tf 30 325 Td (EMBEDDED ICC: LINEAR RGB TO sRGB) Tj ET",
"BT /F1 10 Tf 30 298 Td (Top: ICCBased vector / Middle: DeviceRGB control / Bottom: ICCBased image) Tj ET",
"/RelativeColorimetric ri"]
probes = []
for i, value in enumerate(values):
x = 30 + i * 88
operands = " ".join(str(v) for v in value)
content += [f"/Linear cs {operands} scn {x} 216 76 64 re f", f"{operands} rg {x} 132 76 64 re f"]
probes.append({"kind": "icc-vector", "point_pt": [x+38, 248], "components": value, "expected_rgb": [srgb(v) for v in value]})
probes.append({"kind": "device-control", "point_pt": [x+38, 164], "components": value, "expected_rgb": [round(v*255) for v in value]})
quantized = [round(v*255) for v in value]
probes.append({"kind": "icc-image", "point_pt": [30+i*88+44, 68], "components": quantized, "expected_rgb": [srgb(v/255) for v in quantized]})
content += ["q 528 0 0 48 30 44 cm /Sample Do Q", "0 0 0 rg BT /F1 10 Tf 30 22 Td (ICC gamma 1 / D65 / sRGB primaries / original data and coordinates) Tj ET"]
image = bytes(round(v*255) for rgb in values for v in rgb)
objects = [b"<< /Type /Catalog /Pages 2 0 R >>", b"<< /Type /Pages /Count 1 /Kids [3 0 R] >>",
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 350] /Resources << /Font << /F1 6 0 R >> /ColorSpace << /Linear [/ICCBased 5 0 R] >> /XObject << /Sample 7 0 R >> >> /Contents 4 0 R >>",
pdf_stream("\n".join(content).encode()), pdf_stream(profile,b"/N 3 /Alternate /DeviceRGB /Range [0 1 0 1 0 1]"),
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>",
pdf_stream(image,b"/Type /XObject /Subtype /Image /Width 6 /Height 1 /BitsPerComponent 8 /Interpolate false /ColorSpace [/ICCBased 5 0 R] /Intent /RelativeColorimetric")]
data = bytearray(b"%PDF-1.7\n%\xe2\xe3\xcf\xd3\n"); offsets=[0]
for i, obj in enumerate(objects,1):
offsets.append(len(data)); data += f"{i} 0 obj\n".encode()+obj+b"\nendobj\n"
xref=len(data); data += f"xref\n0 {len(offsets)}\n0000000000 65535 f \n".encode()
for offset in offsets[1:]: data += f"{offset:010d} 00000 n \n".encode()
data += f"trailer << /Size {len(offsets)} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n".encode()
name="icc-linear-rgb.pdf"; (out/name).write_bytes(data)
record={"file":name,"sha256":hashlib.sha256(data).hexdigest(),"page_count":1,
"expected":{"page_sizes_pt":[[600,350]],"icc_probes":probes,"probe_tolerance_rgb":3,
"profile_effect":"ICCBased 0.5 linear gray becomes approximately 188, distinct from DeviceRGB approximately 128."},
"limits":["One RGB matrix/TRC profile, not arbitrary ICC LUT/CMYK/printing conformance.","Analytic sRGB color expectation; independent renderer comparison still requires visual review."]}
origin={"file":"linear-srgb.icc","sha256":hashlib.sha256(profile).hexdigest(),"bytes":len(profile),
"source":"Self-created with LittleCMS cmsCreateRGBProfile using the stated numeric white point, primaries and gamma; no third-party ICC profile is copied.",
"lcms_encoded_version":version,"lcms_license":"MIT; LCMS2-LICENSE.txt",
"upstream":"https://github.com/mm2/Little-CMS","profile_copyright_tag":"No copyright, use freely",
"white_xy":[.3127,.3290],"primaries_xy":[[.64,.33],[.30,.60],[.15,.06]],"gamma":1,
"timestamp_normalization":"Header creation time is fixed to 2026-09-19; optional profile ID is zero."}
return record,origin
if __name__ == "__main__":
out=Path(__file__).resolve().parent/"extended"; record,origin=generate(out)
manifest_path=out/"manifest.json"; manifest=json.loads(manifest_path.read_text())
manifest["documents"]=[d for d in manifest["documents"] if d["file"]!=record["file"]]+[record]
manifest["color_profile"]=origin
manifest_path.write_text(json.dumps(manifest,ensure_ascii=False,indent=2)+"\n")
print(json.dumps({"fixture":record["sha256"],"profile":origin["sha256"]}))
tests/fixtures/pdf/intent-transparency/README.md
Vendored
+27
View File
@@ -0,0 +1,27 @@
# Soft-mask and pattern rendering-intent probes
This self-authored 4-page PDF contains 108 declared pixel probes. The existing
984-byte distinct CMYK/Lab ICC profile is copied without alteration. Its four
intent outputs for CMYK (0, .75, .25, 0) have separated luminances; no PDF renderer
supplies the expected colors.
[Adobe PDF Reference 1.4](https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/pdfreference1.4.pdf)
pp. 218, 223, 231 and 453 define pattern state. Colored tiling and shading patterns
start with their resource-owning page/Form stream's initial state. A Form is
reused under different initial intents, then changes its painting intent to
Absolute. Explicit pattern overrides and q/Q are separate controls. Uncolored
stencils contain no color operators or `ri`; their ICC base color is supplied by
the caller. Small/large uncolored strokes distinguish stroke from fill color.
Pages 440–446 define luminosity soft masks and their backdrop. The mask's RGB
group contains one opaque ICC rectangle, or no objects for the BC-only controls.
The direct CMM RGB result is reduced with the recommended .30/.59/.11 weights;
black over white has value 255 minus that luminosity. This device-dependent
formula is a declared test convention, not universal device calibration.
Run `tests/cmyk_lut/transparency.py --self-check --output NEW_DIRECTORY`, or pass
`--build-dir build --pdfium-library PATH/libpdfium.so --output NEW_DIRECTORY`.
The runner records all failures, validates the sandbox helper and library hash,
and judges PDFium and Poppler separately against the direct CMM. LittleCMS may
be shared with the renderers; this is not an independent CMM implementation or
general ICC conformance acceptance. The older 8-page intent fixture is unchanged.
tests/fixtures/pdf/intent-transparency/distinct-cmyk-lab.icc
Vendored
BIN
View File
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More