Development

Requirements

Go 1.25 or later

Setup

git clone https://git.rumginger.org/agent/dataxl.git
cd dataxl
go mod download

Common Commands

Format code:

gofmt -w cmd/dataxl/*.go

Run tests:

go test ./...

Build:

go build ./cmd/dataxl

Run locally:

go run ./cmd/dataxl -from yaml -to tsv -i examples/people.yaml

Build release archives locally:

scripts/build-release.sh

Refresh the checked-in third-party notices after changing dependencies:

scripts/update-third-party-licenses.sh
git diff --exit-code -- THIRD_PARTY_LICENSES.txt

Test Coverage

The current tests cover:

YAML -> TSV flattening
TSV -> JSON path restoration
JSON -> XLSX -> JSON round trip
structured -> structured conversion without CLI/file I/O
extension normalization and format inference
short table row padding and wider-row rejection
conservative cell type inference, including zero-padded identifiers
whitespace and UTF-8 BOM preservation rules
invalid UTF-8, duplicate keys, and trailing-data rejection for JSON
multi-document YAML streams and empty-record table boundaries
wrapper arrays with sibling metadata
duplicate, blank, malformed, and conflicting headers
nested arrays and JSON-quoted path keys

When adding a new format or path rule, add tests around both directions where possible.

CI/CD

Gitea Actions workflows live under .gitea/workflows.

ci.yml: runs on pushes to main, pull requests, and manual dispatch.
release.yml: runs on v* tag pushes and manual dispatch with a tag input.

The CI workflow checks formatting, runs tests and go vet, scans reachable vulnerabilities with govulncheck, builds the CLI, and performs a small YAML -> TSV -> JSON smoke test.

The release workflow runs tests, cross-builds release archives for Linux, macOS, and Windows on amd64/arm64, writes checksums.txt, creates or reuses a Gitea Release, and uploads the generated assets. It uses the built-in ${{ secrets.GITEA_TOKEN }} provided by Gitea Actions.

Each release archive contains the project LICENSE and THIRD_PARTY_LICENSES.txt in addition to the binary and README.

Release binaries receive their tag through the main.buildVersion linker variable. Verify an extracted native binary with dataxl -version.

Release Notes

Create a release by pushing a version tag:

go test ./...
git tag v0.1.0
git push origin main --tags

The same workflow can also be started manually from Gitea Actions by providing the target tag as the tag input.

After tags exist, users can install a specific version:

go install git.rumginger.org/agent/dataxl/cmd/dataxl@v0.1.0

Design Guidelines

Keep stdin/stdout usable for shell pipelines.
Keep TSV behavior predictable because it is the main Excel clipboard format.
Preserve headers as the contract between spreadsheet data and structured data.
Prefer explicit errors over silent best-effort conversion when a format is unsupported.
Keep dependencies small unless a format needs a mature parser/writer.
Keep CLI/file I/O in main.go; conversion behavior should remain testable through convert.
Keep format-specific behavior in its structured or table adapter.
Document round-trip limitations when a representation cannot preserve a value exactly.

3.3 KiB Raw Blame History