# Contributing to Ren

For using Ren in your own project, see the [getting-started guide](<https://github.com/fieldextension/ren-onboarding/blob/main/docs/getting-started.md>).

## Setup

### Prerequisites

- [uv](<https://docs.astral.sh/uv/>) - Python package manager
- [just](<https://github.com/casey/just>) - Command runner (optional but recommended)
- Python 3.14 or newer
- CUDA Toolkit 12.9.2 or newer for the CUDA backend

CUDA 12.8 can miscompile fused uint64 kernels. After upgrading CUDA, use a fresh `CACHE_DIR` to avoid reusing kernels compiled by the old toolchain.

```bash
git clone git@github.com:fieldextension/ren-onboarding.git ren
cd ren

# Install just (pick one)
brew install just        # macOS
cargo install just       # with Cargo

# Install dependencies
uv sync --locked
```

The root package includes the library in `ren/` and the CLI in `ren_cli/`. To generate a project that uses the local Ren checkout:

```bash
uv run ren init ../my-project
```

The generated project uses an editable dependency, so changes to this checkout's Python source take effect on the next run.

CUDA tests require a CUDA machine. On a CPU-only checkout, use `DEV=python just test`.

`uv sync` installs the development dependencies, including ty, Ruff, and pytest. The `just` commands install any additional dependency groups they need.

### Available Commands

Run `just` to see all available commands:

```bash
just                    # List all commands
just test               # Run tests (skips slow tests)
just test-all           # Run all tests (including slow)
just test-slow          # Run only slow tests
just test-cuda          # Run both debug and release cuda tests
just test-cuda-debug    # Run cuda tests with debug compile flags
just test-cuda-release  # Run cuda tests with release compile flags
just lint               # Run linter
just format             # Format code
just format-check       # Check formatting without changes
just typecheck          # Run type checker
just check              # Run all checks, including CUDA tests
just docs build         # Build documentation
just docs check         # Check documentation without regenerating snippets
just docs serve         # Serve documentation locally
just bench              # Compare the checkout with its merge base with main
just fix                # Auto-fix lint issues and format

# Use DEV=cuda for CUDA backend
DEV=cuda just test
```

## Benchmarks

Run benchmarks on a CUDA machine:

```bash
just bench                                       # Compare the three default workloads
just bench --full                                # Include all diagnostics and bootstrap variants
just bench -k '^bootstrap$' --repeat 3             # Repeat the standard bootstrap comparison
just bench --base parent-branch                   # Use the parent of a stacked PR
DEV=cuda:1 just bench                             # Select a GPU (default: cuda)
just bench --no-base -k polynomials               # Measure this checkout only
just bench --no-base --profile -k polynomials     # Profile this checkout
just bench --list                                # List sections available to -k
```

The defaults are standard bootstrap at depths 10 and 15, plus meta-bootstrap. `-k` selects sections by regex; `--full` selects all sections. Edit the diagnostic cases in `benchmark/bench.py` when investigating a particular operation.

The candidate includes staged, unstaged, and non-ignored untracked files. Use `--candidate REV` to measure a commit instead. The baseline defaults to the merge base of local `main` and the candidate; update `main` yourself when needed. `--no-base` measures only the candidate. Each source uses the benchmark suite from your current checkout, its own locked environment, and the same Python interpreter. Workers use `PYTHONHASHSEED=0` unless you set another value. The command leaves your working files, index, and branch unchanged.

Results go in `.benchmarks/<timestamp>/`: source archives, raw samples, logs, and `report.md`, ready to paste into a PR. Use `--output DIR` for another new directory. A comparison adds percentage differences to the timing table. Results are excluded from Git and are not required by CI.

Runs reuse completed results when source contents, suite, selection, repetitions, and prepared environments match. Committing unchanged files does not invalidate a measurement. Use `--force` (`-f`) for fresh measurements. `--profile` always executes and saves `trace.json` alongside each process's samples.

Compiled kernels are shared across runs in `~/.cache/ren` (or `CACHE_DIR`). Each source warms missing entries before measurement, including with `--force`. The command prints phase durations; worker logs include parameter and input preparation. Replay warms for 100 ms, then collects at least 10 samples over at least 50 ms. Timings include synchronization and Python overhead. Workloads share a process and its caches; repeated comparisons alternate which source runs first. The report shows the median of process means and expandable ranges. One pair cannot establish variation between processes, and a percentage change alone is not a regression.

To publish with an authenticated [GitHub CLI](<https://cli.github.com/>):

```bash
just bench --publish                # Reuse matching results or measure, then publish
just bench --publish DIR --pr 123   # Publish a saved comparison to a specific PR
```

Publishing verifies that the candidate's files match the PR head and updates your existing comment for that filter. After committing and pushing measured files, it can update the commit shown in the comment without measuring again. For remote runs, copy the result directory back before publishing locally. `--publish` can reuse those results before preparing any environments; it preserves their measurement date and machine. Use `--force --publish` to measure again with the current machine settings. Publishing failures leave results available for retrying.

## Docs deployment

For Cloudflare Pages production and preview builds, set the build command to:

```bash
python tools/build_docs.py
```

Use the repository root as the build root, `site` as the output directory, and keep `SKIP_DEPENDENCY_INSTALL=1`. The script uses pipx to install the uv version required by `uv.toml`, then runs `just docs build` with the locked dependencies.

## Editor

- VS Code: install Astral's ty extension.
- Neovim: with Mason and mason-lspconfig configured, run `:LspInstall ty`. mason-lspconfig 2.x enables it by default. Mason's ty version can differ from the version used by `just typecheck`.

Keep Ruff enabled for linting and formatting, and disable other Python type checkers for this checkout. See [ty's editor guide](<https://docs.astral.sh/ty/editors/>) for editor configuration. The project's ty settings are in `pyproject.toml`.

## Profiling

`just bench --no-base --profile` writes CPU timing summaries to the process logs and Chrome Trace / Perfetto files to `trace.json` beside the raw samples.

### NVIDIA Nsight Systems (GPU timeline)

For detailed GPU profiling with kernel-level timing:

```bash
# On remote server with CUDA
NVTX=1 DEV=cuda TMPDIR=$HOME nsys profile \
  --trace=cuda,nvtx \
  --cuda-memory-usage=true \
  -o $HOME/ren_profile \
  just bench --no-base --force

# Transfer to local machine
scp <user>@<host>:~/ren_profile.nsys-rep .

# Open with Nsight Systems GUI locally
```

### NVIDIA NSight Compute (deeper kernel stats)

You probably don't want to profile an entire program at this level. To profile only a specific region:

```python
from ren.prof import trace

with trace("trace_region_name"):
    # put code to be profiled in this block
```

```bash
NVTX=1 LINEINFO=1 DEV=cuda ncu \
--set full \
--nvtx --nvtx-include trace_region_name/ \
--kernel-name ren_ntt_forward_merge \
--export $HOME/ren_profile_ncu \
uv run your_profiling_script.py
```

### Chrome Trace (simple timeline)

Open a benchmark's `trace.json` in `chrome://tracing` or [https://ui.perfetto.dev](<https://ui.perfetto.dev>). Other Ren programs with `PROFILE=1` write `/tmp/ren_profile.json` on exit.

### Environment Variables

| Variable | Description |
| --- | --- |
| `PROFILE=1` | Low-level switch for CPU-side profiling and Chrome trace export. For benchmarks, use `just bench --profile`. |
| `NVTX=1` | Emit NVTX markers for Nsight Systems |
| `LINEINFO=1` | Emit line info for Nsight Compute |
| `CCACHE=0` | Disable the compiled CUDA kernel cache (forces recompilation) |
| `CACHE_DIR=$HOME/.cache/ren` | Cache directory for compiled CUDA kernels and other artifacts |
| `DEBUG=1` | Enable debug output |
