kamat/docview
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
kamat
2026-09-21 13:41:40 +09:00
commit 855c7328df
411 changed files with 85352 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/PDF-VALIDATION.md
+232
View File
@@ -0,0 +1,232 @@
# PDF implementation validation
Validated on Linux x64 with Qt 6.11.2 and PDFium 155.0.8057.0, pinned to
`a5a7089234f121990b336b3841008009dca143bf`. The non-V8 build disables PDF
JavaScript and XFA. Exact binary provenance and SHA-256 are in
`cmake/pdfium.lock.json`; `python3 cmake/fetch_pdfium.py` installs that archive.
The current [v2 source-build candidate](results/pdfium-intent-context/README.md)
preserves rendering intents, pattern initial state and uncolored tiling stroke color.
Dedicated Arch and Ubuntu builds each pass 972 color probes and all 24 CTest groups;
the upstream run passes 1,979 tests. It is integrated into `build-context` and the
Ubuntu validation4 deb. Original dependencies and earlier packages remain available
for comparison. The v1 corpus had eight incorrect shading-pattern expectations;
v2 corrects those rows without changing the PDF/profile bytes and retains the old
failures and measurements. See the [current integration record](results/pdfium-intent-context/integration-record.json).
The historical [CMYK CLUT comparison](results/cmyk-lut/README.md) exposes a provider
rendering-intent discrepancy: on both Arch and Ubuntu, 30 of 48 ICC vector/image probes
exceed the unchanged analytic tolerance. The fixed PDFium uses perceptual ICC conversion
despite this PDF's relative-colorimetric intent. Diagnosis does not grant acceptance;
the [additional review page](results/cmyk-lut/index.html) retains the difference.
The [current human review gallery](results/pdfium-intent-context/render-review/index.html) assembles 21 image
pairs with per-page expectations and known differences. Nineteen compare against
Poppler; two compare TTF/TTC within PDFium. Sources, images and the worker used
for that recorded run are bound by hashes. All 83 decoded images match the prior
gallery exactly. All remain pending human review; creating or testing the
gallery does not grant G-RENDER acceptance.
## Validation revision
The older integrated, comparison and performance records remain historical evidence
for the binary hashes recorded with each run. Current checks include the latest
identity, navigation, metadata-omission and translation changes.
The historical completion-final run passed all 20 CTest groups; its [log](results/completion-final/ctest/LastTest.log)
and [build hashes](results/completion-final/build-record.json) are retained. The [six renderer comparisons](results/completion-final/comparison-run.json),
[relocated package checks](results/completion-final/linux-development-package/README.md)
and [watchdog](results/completion-final/worker-deadlines/README.md) have also completed.
The [v2 PDF performance run](results/pdfium-intent-context/performance/README.md)
passed four conditions, 120 cold process starts, 750 operations and 960 scroll inputs,
plus the approximately 1 GB / 5,000-page stress case. No measured provisional budget
was exceeded. These are headless software-rendering reference measurements; physical
display and reference-hardware acceptance remain incomplete. See [validation status](../docs/VALIDATION.md).
## Automated checks
The current `test_pdf` run records 75 passes (the earlier run had 69), including annotation geometry and lifetime checks,
26 tagged-PDF boundary rows, bounded shared-metadata checks, production worker
cases, Japanese broker integration, fatal-font-error rows, and QtTest setup and cleanup: metadata/page labels/CropBox/Rotate; explicit and named outline
destinations; RoleMap and multiple MCIDs in H1/H2; external versus blocked Launch
links and comment contents; text search; saved widget appearance; full-page
versus tile pixels; inverse coordinates; password required/invalid/correct;
corruption and render limits; and real sandboxed PDF worker round trips. A
malformed page tree with a missing first page is rejected before commit, and
its document handle is closed; it cannot trigger repeated page-metadata loads.
The suite also checks 1,200-item outline pagination, encoded-byte limits with
long Unicode labels, explicit 20,000-item truncation, string-limit warnings,
eight capability states/reasons, and the tagged versus untagged declaration.
The latest metadata regression cases are `annotationContentsLengthBoundary`,
`annotationAndLinkCountLimitsAreExplicit` (comments/links at 1,000 and 1,001), and
`annotationResponsePreservesBoundedPrefixThroughWorker`. A Contents value over
4,095 UTF-16 units (8,192-byte buffer including its terminator) keeps a fixed
explanation in the annotation list and reports an omission warning. Page-local
link/annotation count limits and the 512 KiB metadata response budget return an
explicit warning with the retained prefix. The production-worker case uses 100
long Japanese comments and verifies bounded replies and unchanged source bytes.
These cases pass in the current 75-pass suite, without increasing the safety limits.
Controller regressions now distinguish an outline containing only external or
blocked actions from one with usable internal destinations. Only internal targets
are considered for heading fallback and its capability state. The same GUI run
also replaces an open PDF's source file, verifies that its old reading position is
not rebound to the replacement, and checks the unresolved-location notice on
explicit reopen. [Focused and full-GUI evidence](results/completion-regressions/controller-state.md).
The worker test uses the production `WorkerProcess` launcher, completes the
bootstrap ping, parses the fixture only after Linux Landlock and seccomp are
active, renders a tile, and stops cleanly. It must run in an environment
permitting the test harness to create its own local socket. The worker never
disables its own sandbox for testing. The launcher also has a Windows handle
transport, but these results were obtained on Linux and establish no Windows
sandbox result.
The [tagged-PDF boundary corpus](fixtures/pdf/headings/README.md) adds a transformed
Form XObject, a rotated CropBox with structure order different from draw order,
a heading spanning two pages with nested and direct MCR children, and explicit
page-precision fallbacks for vector-only headings and reused stream-local MCIDs.
The regression checks authored PDF-point regions, title/order/page/level, all
four user rotations, and inverse coordinates within 0.5 pt. The Form-only ParentTree fixture now yields an exact H2 using the shared
worker-side qpdf structure adapter; a GUI regression follows the target.
The expanded corpus also checks nested Forms, owner-qualified MCID collisions,
repeated calls, structure cycles and bounded stream decompression.
The corpus exposed and fixed mixed-child ordering and non-text MCID collision
bugs; the ordinary RoleMap/multiple-MCID cases continue to pass.
`test_pdf_canvas` includes 21 passes including setup and cleanup. It distinguishes
the page nearest the viewport centre (ties choose the previous page) from the
leading logical reading location, and preserves the central PDF point on zoom. A separate
`pdf_canvas_hidpi` CTest group repeats the annotation pixel/hit-target case at DPR 2
(three QtTest passes including setup and cleanup). It compares both directions of the canvas coordinate transform
against PDFium for a nonzero CropBox, document Rotate 90 and user rotations
0/90/180/270. It verifies PDF-point restoration after zoom and horizontal pan,
separate jumps to two headings on the same page, invalid/clipped coordinates,
a jump whose page metadata arrives later, high zoom limits and search-result navigation.
The actual sandboxed worker also verifies that prefetch disabled produces no
adjacent-page requests, enabled prefetch warms a bounded adjacent viewport,
and a page jump uses those pixels immediately. An in-flight prefetch test
changes the zoom before completion, rejects the obsolete reply, and confirms
the previous visible resolution remains cached. Prefetch is limited to eight
tiles per adjacent page, runs only after visible work and queued operations,
and cannot insert a tile when doing so would evict existing cache entries.
Real-worker cases also inject a delayed fatal font reply: visible and prefetch
failures stop the poisoned worker even after a zoom revision, while a reply for
a replaced document is ignored. Missing-font warnings remain nonfatal and
reach document information. GUI regressions reopen the same unchanged source
with a new worker after failure in prefetch, search, cancelled search or page
metadata.
Late metadata cannot resurrect a destination cancelled by a newer jump, zoom,
or explicit cancellation. Theme background changes preserve white PDF paper.
The real-worker cache regression checks the 64–512 MiB budget clamp, nonzero
charged image bytes after rendering, cost below budget, cache clearing at
document replacement, actual visible/prefetch request counters, and exactly one
discarded old-revision reply. Repeated viewport readiness checks do not inflate
the request counts. These counters are lifetime samples; budget/cost report
QCache's KiB-rounded image charge and are separate from measured process RSS.
A two-pixel offscreen margin was insufficient to preserve antialiasing for a
glyph crossing a tile boundary. The worker now renders a sixteen-pixel margin,
then copies only the requested tile. The fixture's tile and full-page crop
compare byte for byte, including the red saved form appearance. IPC remains a
512-pixel tile, and the margin is not transmitted.
The base fixtures are generated locally by `tests/fixtures/pdf/generate.py`;
`tests/fixtures/pdf/generate_headings.py` regenerates 26 structure fixtures and
four shared-context boundary fixtures, with a SHA-256/expectation manifest,
without external dependencies.
They contain no third-party document content. The `qpdf` CLI regenerates
the encrypted fixture (reader password: `reader`); libqpdf is also a worker runtime
dependency for supplemental link-border and structure metadata. No password is saved by the
application.
## Independent renderer comparison
Run `python3 tests/compare_pdf_renderers.py` after building `pdf-worker-evidence`.
The exporter uses the production sandboxed worker, font broker and IPC; it does
not link PDFium into the main process. Set
`DOCVIEW_BUILD_DIR=build-pdf` to use another build directory. The comparison
uses system Poppler when available, because the bundled document-tools wrapper
has a different fontconfig environment.
Fixture SHA-256:
`e8b0b037679a3449de5d48f1b13a8562486c9ce19385a036d682cab906815a22`.
PDFium and Poppler 26.08.0 at 144 DPI both produced a 400 × 400 CropBox image.
154,110 of 160,000 pixels matched exactly. 4,024 pixels had a maximum channel
difference greater than 16; mean absolute channel error was 2.26274 / 255.
These descriptive metrics are not a performance or compatibility acceptance
threshold. The record contains the worker and font-broker binary hashes.
Evidence is written to `tests/results/pdf/{pdfium,poppler,difference,comparison}.png`
and `comparison.json`. Inspection confirms all three text strings, the black
rectangle and the saved red widget appearance are present in both images. Text
antialiasing differs. The no-appearance text-comment icon differs by renderer.
The former missing default link borders now match Poppler in the 8,448-pixel
comparison region. Other differences remain visible in the evidence and are
not accepted as golden-image success.
The [link-border corpus](results/link-borders/README.md) adds 20 annotation styles,
four intrinsic and four user rotations, tiled/full output equality, original
rectangle and file preservation, encrypted input, malformed data and bounded
metadata/AP handling. Supplemental metadata is read through libqpdf in the
worker, and numeric viewing-only appearances are generated through PDFium's
public API. Supplied appearances are preserved. NoZoom/NoRotate now use temporary
Rect and page-rotation compensation, with logical zoom separated from DPR.
The [annotation-flag corpus](results/pdf-annotation-flags/README.md) exercises
generated borders, saved link/comment appearances, and native Text icons.
Other missing-AP annotation types retain an explicit NoZoom limitation. Canvas tests compare
actual colored pixels with focus/click bounds at 100% and 200% DPR. These are
scoped rendering results, not a complete G-RENDER gate.
The [expanded synthetic corpus](fixtures/pdf/extended/README.md) adds five
documents and nine pages with embedded Japanese text, authored vertical glyph
positions and ruby, RGB/RGBA images, soft masks, alpha blending, clipping,
shading, CMYK swatches, Standard-14 substitution, varied CropBox/rotation, and
ICCBased RGB vector and raster content.
The [extended comparison evidence](results/pdf-extended/README.md) records
matching dimensions and geometry/alpha probes, Japanese extraction/search,
independent Poppler images and visual inspection. Run
`python3 tests/compare_pdf_extended.py` to regenerate that evidence.
The [ICC comparison](results/pdf-icc/README.md) renders a self-authored linear-RGB
matrix/TRC profile at 50%, 100% and 400%. All 54 probes agree with the analytic
sRGB transfer function within the stated rounding tolerance. A DeviceRGB control
distinguishes actual profile application from ignored ICC data. Dimensions and
the original file hash also match. Run `python3 tests/compare_pdf_icc.py` with
Pillow and pypdf installed. This is not calibrated CMYK/printing acceptance.
The [large-PDF stress record](STRESS-PDF-VALIDATION.md) covers a 5,000-page,
1,009,828,104-byte generated PDF with 1,280 genuinely referenced uncompressed
raster streams. Three opens, 100 keyboard operations and a direct page-5,000
screen capture completed with the sandbox enabled; the large input was removed
after hashes, timings, memory and screenshots were retained. This workload is
separate from rendering compatibility and reference-machine acceptance.
The [real TTC experiment](results/font-collection/README.md) constructs a private
two-face collection from locally installed OFL fonts. The production broker
selects regular face 0 and bold face 1; both pages' 892,800 pixels match the
separate-TTF condition exactly, and regular/bold remain distinct. Font programs,
configuration and cache are temporary and removed after verification. This
checks real Linux TrueType collection selection, not Windows/CJK/CFF collections.
## Outstanding acceptance coverage
G-RENDER is **not complete**: the synthetic comparisons have no human-approved
golden image and do not establish arbitrary CID/CMap compatibility,
calibrated ICC/CMYK, complex transparency
groups, malformed appearances or production-corpus compatibility. The original embedded Japanese
fixture uses explicitly positioned vertical glyphs; the new unembedded fixture
adds native UniJIS-UCS2-V writing mode for two requested Japanese families. Generated link borders, saved link/comment appearances, and native Text icons
now cover NoZoom/NoRotate; other subtype-specific synthesized appearances
still need wider validation.
G-HEADINGS remains scoped to the generated corpus: RoleMap, direct/nested/interleaved
MCIDs, transformed Form text, rotated pages and cross-page fragments are covered.
A shared qpdf context now reads Form-only StructParents and verifies stream owners,
ParentTree membership and actual Form invocation. Public PDFium APIs supply text
and geometry. Repeated or ambiguously matched Forms keep explicit page-precision
fallbacks; malformed/cyclic/bounded-out structures report limitations. The main
process has no added PDF parser or native PDF objects. Annotation-owned marked
content remains explicitly unsupported, and broader production structures require
corpus validation. These results do not establish comprehensive tagged-PDF support.