---
title: "Using fz for parametric studies"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Using fz for parametric studies}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
# Chunks are eval = FALSE (they need the funz-fz Python package and a
# configured calculator) and purl = FALSE (so `R CMD check`'s tangle-and-run
# step does not try to execute them either). The code is still shown in the
# rendered vignette.
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = FALSE
)
```

```{r setup, purl = FALSE}
library(fz)
```

## Introduction

The `fz` package provides R bindings to the
[funz-fz](https://pypi.org/project/funz-fz/) Python package. It lets you:

- run **parameter sweeps** over any simulation code,
- drive **design of experiments** with adaptive algorithms,
- read and collect **output files** into data frames.

fz works with any simulation code that reads text input files and writes text
output files. You describe the model in a small dict (or install it as a named
alias with `fz install`).

## Installation

```{r install, purl = FALSE}
# Install the funz-fz Python package into the active reticulate environment
fz_install()

# Verify
fz_available()
```

## Concepts

### Template files

A **template** is an ordinary input file for your simulator with variable
placeholders, e.g.:

```
# Perfect Gas parameters
pressure = ${P~1.013}    # variable P, default 1.013
volume   = ${V~22.4}     # variable V, default 22.4
moles    = ${n~1.0}
```

The placeholder syntax (`$`, `{}`) is defined by the **model** dict.

### Model dict

The model dict tells fz how to:

1. find variable placeholders (`varprefix`, `delim`),
2. extract output values from the result files (`output`).

```{r model_dict, purl = FALSE}
model <- list(
  varprefix    = "$",
  delim        = "{}",
  formulaprefix = "@",
  commentline  = "#",
  output = list(
    pressure = "grep 'pressure =' output.txt | cut -d= -f2"
  )
)
```

You can also use an installed model alias (a string) instead of an inline dict:

```{r model_alias, purl = FALSE}
fzl()$models  # lists installed aliases, e.g. "PerfectGas"
```

## A complete runnable example (external simulator)

The rest of this vignette uses a placeholder `run.sh`. This section instead
runs a **real external program**: a tiny Python simulator of the ideal gas
law `P = n R T / V`. Both files ship with the package under
`inst/examples/perfectgas/`.

```{r pg_setup, purl = FALSE}
ex <- system.file("examples", "perfectgas", package = "fz")
file.copy(list.files(ex, full.names = TRUE), ".")  # perfectgas.txt + perfectgas.py
```

`perfectgas.txt` is the input template:

```
temperature = ${T~300}     # K
volume      = ${V~0.001}   # m3
moles       = ${n~1}       # mol
```

`perfectgas.py` reads the compiled `perfectgas.txt` from its working
directory (fz runs each case in its own directory), computes the pressure,
and writes `pressure = <value>` to `out.txt`.

```{r pg_model, purl = FALSE}
model <- list(
  varprefix = "$", delim = "{}", formulaprefix = "@", commentline = "#",
  # shell-free output extraction (funz-fz >= 1.2)
  output = list(pressure = 'python://grep(r"pressure = (\\S+)", "out.txt")')
)

fzi("perfectgas.txt", model)
#> $T [1] 300
#> $V [1] 0.001
#> $n [1] 1
```

```{r pg_run, purl = FALSE}
results <- fzr(
  "perfectgas.txt",
  list(T = c(300, 350, 400), V = 1e-3, n = 1),   # 3 cases
  model,
  calculators  = "sh://python3 perfectgas.py",   # the external simulator
  input_static = "perfectgas.py"                  # shipped into every case dir (funz-fz >= 1.2)
)
results[, c("T", "V", "n", "pressure")]
#>     T     V n pressure
#> 1 300 0.001 1  2494339
#> 2 350 0.001 1  2910062
#> 3 400 0.001 1  3325785
```

The same model drives `fzd()` — keep `calculators` and `input_static`, and
pass `input_variables` as `"[min;max]"` ranges (a variable held fixed is
just `"1"`):

```{r pg_fzd, purl = FALSE}
fzd(
  "perfectgas.txt",
  list(T = "[280;320]", V = "[1e-3;2e-3]", n = "1"),
  model,
  output_expression = "pressure",
  algorithm         = "algorithms/montecarlo_uniform.py",
  calculators  = "sh://python3 perfectgas.py",
  input_static = "perfectgas.py"
)
```

## Basic workflow

The typical fz workflow has four steps.

### Step 1 — Inspect the template

`fzi` parses the template and returns the variable names together with their
default values:

```{r fzi, purl = FALSE}
vars <- fzi("input.txt", model)
# $P [1] 1.013
# $V [1] 22.4
# $n [1] 1.0
```

### Step 2 — Compile (substitute values)

`fzc` writes one copy of the input file per parameter combination into
`output_dir`. Each copy goes into a subdirectory named
`var1=val1,var2=val2,...`:

```{r fzc_single, purl = FALSE}
# Single case
fzc("input.txt", list(P = 2.0, V = 11.2), model, output_dir = "compiled")
# writes: compiled/P=2,V=11.2/input.txt  (placeholder replaced with 2.0 / 11.2)
```

Supply vectors to generate a full-factorial grid:

```{r fzc_grid, purl = FALSE}
# 2 x 3 = 6 cases
fzc("input.txt",
    list(P = c(1.0, 2.0), V = c(10.0, 20.0, 30.0)),
    model,
    output_dir = "compiled")
```

### Step 3 — Run the model and collect outputs

`fzr` wraps steps 1–3 and output collection into a single call. It compiles
the template, runs the calculator for every case, and returns a data frame:

```{r fzr, purl = FALSE}
results <- fzr(
  "input.txt",
  list(P = c(1.0, 2.0, 3.0), V = 22.4),   # 3 cases (V fixed)
  model,
  results_dir = "results",
  calculators = "sh://bash run.sh"           # run.sh executes the simulator
)

# results is a data frame:
#     P    V  pressure
# 1  1.0  22.4   ...
# 2  2.0  22.4   ...
# 3  3.0  22.4   ...
```

The `calculators` argument accepts:

- `"sh://bash run.sh"` — run a local shell command
- `"sh://"` — execute the input file directly as a shell script
- `"ssh://user@host"` — run over SSH

### Step 4 — Read outputs from existing directories

If you already ran the simulator externally, `fzo` reads the output files:

```{r fzo, purl = FALSE}
values <- fzo("results/P=2,V=22.4", model)
# $pressure [1] "2.026"

# Glob to read all cases at once:
all_values <- fzo("results/*", model)
```

## Algorithm-driven design of experiments

`fzd` runs an adaptive experiment: the algorithm decides which parameter
combinations to evaluate based on previous results. Input variable ranges use
`"[min;max]"` strings:

```{r fzd, purl = FALSE}
result <- fzd(
  "input.txt",
  list(P = "[1;5]", V = "[10;30]"),
  model,
  output_expression = "pressure",
  algorithm         = "algorithms/montecarlo_uniform.py",
  algorithm_options = list(batch_sample_size = 10, max_iterations = 5, seed = 42)
)
```

Algorithms are Python files; `fz` ships several in `algorithms/` (Monte Carlo,
surrogate-based optimization, …). You can also write your own.

## Listing installed models

`fzl` shows which model aliases and calculators are installed in `~/.fz/`:

```{r fzl, purl = FALSE}
info <- fzl()
names(info$models)       # e.g. c("PerfectGas", "Moret")
names(info$calculators)  # e.g. c("sh://")

# Filter by pattern
fzl(models = "Perfect*")

# Probe calculators to verify they are reachable
fzl(check = TRUE)
```

## Best practices

1. **Test with `fzi` first** — verify the correct variable names are found
   before running anything.
2. **Use `fzc` for a dry run** — inspect compiled files to confirm placeholder
   substitution is correct.
3. **Start small** — run a handful of cases before launching a large sweep.
4. **Save results** — persist the data frame for reproducibility.

```{r save_results, purl = FALSE}
saveRDS(results, "fz_results.rds")
write.csv(results, "fz_results.csv", row.names = FALSE)
```

## Troubleshooting

**`fz` Python package not found:**

```{r troubleshoot_install, purl = FALSE}
fz_install()         # install funz-fz into the reticulate environment
fz_available()       # should return TRUE afterwards
```

**Variables not found in template:**
Check that `varprefix` and `delim` in your model dict match the syntax used in
your template file. Run `fzi` and inspect the returned list.

**Calculator errors:**
Run the simulator manually on one compiled directory to confirm it works before
using `fzr`.

## Further reading

- funz-fz documentation: <https://funz.github.io>
- reticulate: <https://rstudio.github.io/reticulate/>

## Session info

```{r session_info, purl = FALSE}
sessionInfo()
```