Contributing to Ren¶
For using Ren in your own project, see the getting-started guide.
Setup¶
Prerequisites¶
- uv - Python package manager
- just - Command runner (optional but recommended)
- Python 3.14 or newer
- CUDA Toolkit 12.9.2 or newer for the CUDA backend
CUDA 12.8 can miscompile fused uint64 kernels. After upgrading CUDA, use a fresh CACHE_DIR to avoid reusing kernels compiled by the old toolchain.
git clone git@github.com:fieldextension/ren-onboarding.git ren
cd ren
# Install just (pick one)
brew install just # macOS
cargo install just # with Cargo
# Install dependencies
uv sync --locked
The root package includes the library in ren/ and the CLI in ren_cli/.
To generate a project that uses the local Ren checkout:
The generated project uses an editable dependency, so changes to this checkout's Python source take effect on the next run.
CUDA tests require a CUDA machine. On a CPU-only checkout, use DEV=python just test.
uv sync installs the development dependencies, including ty, Ruff, and pytest. The just commands install any additional dependency groups they need.
Available Commands¶
Run just to see all available commands:
just # List all commands
just test # Run tests (skips slow tests)
just test-all # Run all tests (including slow)
just test-slow # Run only slow tests
just test-cuda # Run both debug and release cuda tests
just test-cuda-debug # Run cuda tests with debug compile flags
just test-cuda-release # Run cuda tests with release compile flags
just lint # Run linter
just format # Format code
just format-check # Check formatting without changes
just typecheck # Run type checker
just check # Run all checks, including CUDA tests
just docs build # Build documentation
just docs check # Check documentation without regenerating snippets
just docs serve # Serve documentation locally
just bench # Compare the checkout with its merge base with main
just fix # Auto-fix lint issues and format
# Use DEV=cuda for CUDA backend
DEV=cuda just test
Benchmarks¶
Run benchmarks on a CUDA machine:
just bench # Compare the three default workloads
just bench --full # Include all diagnostics and bootstrap variants
just bench -k '^bootstrap$' --repeat 3 # Repeat the standard bootstrap comparison
just bench --base parent-branch # Use the parent of a stacked PR
DEV=cuda:1 just bench # Select a GPU (default: cuda)
just bench --no-base -k polynomials # Measure this checkout only
just bench --no-base --profile -k polynomials # Profile this checkout
just bench --list # List sections available to -k
The defaults are standard bootstrap at depths 10 and 15, plus meta-bootstrap.
-k selects sections by regex; --full selects all sections. Edit the diagnostic
cases in benchmark/bench.py when investigating a particular operation.
The candidate includes staged, unstaged, and non-ignored untracked files.
Use --candidate REV to measure a commit instead. The baseline defaults to the
merge base of local main and the candidate; update main yourself when needed.
--no-base measures only the candidate. Each source uses the benchmark suite from
your current checkout, its own locked environment, and the same Python interpreter.
Workers use PYTHONHASHSEED=0 unless you set another value.
The command leaves your working files, index, and branch unchanged.
Results go in .benchmarks/<timestamp>/: source archives, raw samples, logs,
and report.md, ready to paste into a PR. Use --output DIR for another
new directory. A comparison adds percentage differences to the timing table.
Results are excluded from Git and are not required by CI.
Runs reuse completed results when source contents, suite, selection,
repetitions, and prepared environments match. Committing unchanged files does
not invalidate a measurement. Use --force (-f) for fresh measurements.
--profile always executes and saves trace.json alongside each process's samples.
Compiled kernels are shared across runs in ~/.cache/ren (or CACHE_DIR).
Each source warms missing entries before measurement, including with --force.
The command prints phase durations; worker logs include parameter and input preparation.
Replay warms for 100 ms, then collects at least 10 samples over at least 50 ms. Timings include
synchronization and Python overhead. Workloads share a process and its caches;
repeated comparisons alternate which source runs first. The report shows the
median of process means and expandable ranges. One pair cannot establish
variation between processes, and a percentage change alone is not a regression.
To publish with an authenticated GitHub CLI:
just bench --publish # Reuse matching results or measure, then publish
just bench --publish DIR --pr 123 # Publish a saved comparison to a specific PR
Publishing verifies that the candidate's files match the PR head and updates your
existing comment for that filter. After committing and pushing measured files,
it can update the commit shown in the comment without measuring again.
For remote runs, copy the result directory back before publishing locally.
--publish can reuse those results before preparing any environments;
it preserves their measurement date and machine. Use --force --publish to
measure again with the current machine settings. Publishing failures leave
results available for retrying.
Docs deployment¶
For Cloudflare Pages production and preview builds, set the build command to:
Use the repository root as the build root, site as the output directory, and keep SKIP_DEPENDENCY_INSTALL=1.
The script uses pipx to install the uv version required by uv.toml, then runs just docs build with the locked dependencies.
Editor¶
- VS Code: install Astral's ty extension.
- Neovim: with Mason and mason-lspconfig configured, run
:LspInstall ty. mason-lspconfig 2.x enables it by default. Mason's ty version can differ from the version used byjust typecheck.
Keep Ruff enabled for linting and formatting, and disable other Python type checkers for this checkout. See ty's editor guide for editor configuration. The project's ty settings are in pyproject.toml.
Profiling¶
just bench --no-base --profile writes CPU timing summaries to the process logs
and Chrome Trace / Perfetto files to trace.json beside the raw samples.
NVIDIA Nsight Systems (GPU timeline)¶
For detailed GPU profiling with kernel-level timing:
# On remote server with CUDA
NVTX=1 DEV=cuda TMPDIR=$HOME nsys profile \
--trace=cuda,nvtx \
--cuda-memory-usage=true \
-o $HOME/ren_profile \
just bench --no-base --force
# Transfer to local machine
scp <user>@<host>:~/ren_profile.nsys-rep .
# Open with Nsight Systems GUI locally
NVIDIA NSight Compute (deeper kernel stats)¶
You probably don't want to profile an entire program at this level. To profile only a specific region:
NVTX=1 LINEINFO=1 DEV=cuda ncu \
--set full \
--nvtx --nvtx-include trace_region_name/ \
--kernel-name ren_ntt_forward_merge \
--export $HOME/ren_profile_ncu \
uv run your_profiling_script.py
Chrome Trace (simple timeline)¶
Open a benchmark's trace.json in chrome://tracing or https://ui.perfetto.dev.
Other Ren programs with PROFILE=1 write /tmp/ren_profile.json on exit.
Environment Variables¶
| Variable | Description |
|---|---|
PROFILE=1 |
Low-level switch for CPU-side profiling and Chrome trace export. For benchmarks, use just bench --profile. |
NVTX=1 |
Emit NVTX markers for Nsight Systems |
LINEINFO=1 |
Emit line info for Nsight Compute |
CCACHE=0 |
Disable the compiled CUDA kernel cache (forces recompilation) |
CACHE_DIR=$HOME/.cache/ren |
Cache directory for compiled CUDA kernels and other artifacts |
DEBUG=1 |
Enable debug output |