lexsync selects stimuli that are matched in parallel
across several lexical dimensions and then generates the experiment
scripts that present them. This vignette runs a small high- versus
low-frequency contrast on the bundled English example lexicon.
The selection is deterministic and involves no random number
generator. lexsync has two independent engines, an R
package and a Python package, which select byte-identical stimuli from
the same lexicon and design. The engines agree because every ordering is
by byte and every tie is broken by a rule both apply. This holds for the
default method, standardised_euclidean, and for
joint. The optional mahalanobis and
optimal methods are equivalent but not byte-identical. The
last bits of the inverse of the pool’s correlation matrix can differ
between the two engines, and their assignment solvers can choose
differently among assignments of equal cost.
The package is on CRAN, and the README also covers installing the development version.
library(lexsync)
schema <- yaml::read_yaml(
system.file("extdata", "schema.yaml", package = "lexsync")
)
lex <- load_lexicon(
system.file("extdata", "en_example.csv", package = "lexsync"),
schema, language = "english"
)
nrow(lex)[1] 3000
A design names the conditions, the dimensions to match and the number of items, together with the filters that narrow the lexicon to a candidate pool. The pool is built first, as the pipeline does, and matching then runs over it.
design <- list(
name = "quick_start", language = "english", n_per_condition = 60,
pool_filters = list(length = c(3, 8), frequency = c(3.8, 7.0)),
conditions = list(
list(name = "high_frequency", define_by = list(frequency = c(5.2, 7.0))),
list(name = "low_frequency", define_by = list(frequency = c(3.8, 4.4)))
),
match_on = list("length", "n_density", "old20"),
counterbalance = list(lists = 1)
)
pool <- build_pool(lex, design$pool_filters)
stim <- match_stimuli(pool, design, schema)
head(stim[, c(
"word", "condition", "length", "frequency", "n_density", "old20"
)]) word condition length frequency n_density old20
1 perfect high_frequency 7 5.20 0 2.55
2 include high_frequency 7 5.21 0 2.65
3 sex high_frequency 3 5.22 11 1.30
4 issue high_frequency 5 5.23 0 2.35
5 mother high_frequency 6 5.24 1 1.85
6 seems high_frequency 5 5.24 5 1.65
The list above is convenient in a vignette, but the design of a study
is written as a YAML file. config/schema.yaml holds
everything global, including the matching defaults, while the design
holds what is specific to the study. The R and Python packages read both
files identically, so a design file can be attached to a
pre-registration and handed to someone running the other engine. The
repository’s English frequency contrast,
config/design_en_freqcontrast.yaml, reads in full:
name: en_freqcontrast
language: english
lexicon: corpora/derived/en_v2.csv
description: 'High versus low frequency English words, matched on length, neighbourhood density and OLD20.'
n_per_condition: 80
pool_filters:
length: [3, 8]
frequency: [3.8, 7.0]
# Keeps only the forms the SUBTLEX-US screen leaves unmarked (see corpora/ATTRIBUTION.md).
nonstandard: [0, 0]
# Leaves out the unsuitable forms the screen lets through (see the file's header).
exclude_words: {path: config/exclude_en_demo.txt}
conditions:
- name: high_frequency
define_by:
frequency: [5.2, 7.0]
- name: low_frequency
define_by:
frequency: [3.8, 4.4]
match_on: [length, n_density, old20]
counterbalance:
lists: 1
timing:
fixation_frames: 30
word_frames: 30
isi_frames: 15name and language are required, and
together they form the stem that every written file is named after.
en_freqcontrast plus english becomes
en_freqcontrast_english, and from that come
en_freqcontrast_english_stimuli_R.csv,
en_freqcontrast_english.osexp and the rest. A name or
language holding a character outside ASCII also adds an eight-character
checksum to the stem, as Reproducibility, parity and the materials
datasheet explains. language is a free-text label
rather than a code, because it is what the experiment displays. When the
browser target needs a BCP 47 tag, it maps the common labels and falls
back to und, or you can state language_tag
outright.
lexicon names the derived corpus to read. It may also be
written as items.lexicon, which is the form the
lexical-decision designs use, since they also have to say where the
items come from.
items selects where the stimuli come from, and there are
four sources.
items.source |
Where stimuli come from | Also needs |
|---|---|---|
corpus (the default) |
Words selected from the lexicon by matching or by spanning a predictor. | lexicon, conditions or
continuous, match_on |
generate |
Real words plus a deterministically generated pseudoword for each. | lexicon, optionally
items.generation.method |
table |
A CSV of prepared items (prime-target pairs, sentences). Add
items.members to make it pair-keyed. |
items.path |
pool |
A candidate word list of your own, matched over as if it were a pool. | items.path, normally items.lexicon, plus
match_on and either conditions or
continuous, as for corpus |
pool_filters narrows the lexicon to the candidates the
design will consider at all. Each key is a column, and each value is an
inclusive [min, max] range when it is a pair of numbers, or
a set of permitted values otherwise. match_stimuli() never
reads pool_filters itself, so a script that skips
build_pool() matches over the whole lexicon and ignores the
design’s pool filters without warning. Here,
nonstandard: [0, 0] keeps the forms the screened lexicon
leaves unmarked, and exclude_words leaves out the words a
file lists, as Matching, dimensions and designs explains.
conditions is a list, each entry with a
name and a define_by block that narrows the
pool to the condition’s candidates by the same filter syntax. Two
conditions make a contrast. Four make the 2 × 2 that
design_en_andrews_repro.yaml uses to reproduce Andrews
(1989). A design may instead declare a continuous block and
dispense with conditions altogether.
match_on lists the dimensions to equate across
conditions, and a design that matches its conditions must name at least
one. n_per_condition is how many items each condition
should hold, a whole number of at least 1 written as 80 or
80.0. n_per_cell is read in place of
n_per_condition when that is absent. A design that gives
neither takes 20 items per condition when it is matched, 60 items when
it is continuous and 40 words, each with a pseudoword, when it generates
its items. match_on and n_per_condition are
covered at length in Matching, dimensions and designs.
matching overrides the schema defaults for this design
alone. matching.method picks one of the four methods, and
matching.tolerance_k sets the half-width, in standard
deviations, of the tolerance window on each dimension. Overriding a
single dimension is common when reproducing a published study’s exact
windows.
counterbalance.lists sets the number of lists, and
counterbalance.optimise replaces the deal by set rank with
an assignment whose lists are equated on the item dimensions.
practice and fillers each name an item table
whose trials are presented but not analysed.
timing overrides the fixation, critical-word and
inter-stimulus durations. Milliseconds are canonical
(fixation_ms, word_ms, isi_ms),
so a design means the same interval on any display. Frames
(fixation_frames, word_frames,
isi_frames) are also accepted and are converted on load at
the schema’s presentation.assumed_refresh_hz. The 30 frames
in the design above therefore become the 500 ms they were written for. A
*_ms key takes precedence over its *_frames
counterpart. The fixation keys set every fixation event, the word keys
every text event whose trigger is condition
and the inter-stimulus keys every blank, though not an event that takes
a per-trial duration block.
font overrides the presentation font, which matters for
a non-Latin script. design_zh_freqcontrast.yaml sets
SimHei, because the Latin default has no glyphs for Han
characters.
paradigm names one of the six registered paradigms,
whose trial-event sequence and counterbalancing recipe the design
inherits. Omitting it gives factorial. A design may instead
supply an explicit events list and describe its own trial.
Both approaches are covered in Experiments, paradigms and EEG
triggers.
Before it reads a lexicon or writes a file,
run_pipeline() checks the design and the schema with
validate_design(). When a design cannot be honoured, the
run stops with the same error message in both engines. Every key is
looked up in the vocabulary of keys lexsync reads at its level. A key
one edit from a known key of up to five characters, or two edits from a
longer one, is taken for a typo, and the error names the key it was
probably meant to be. n_per_conditon: 8 stops the run with
unknown design key 'n_per_conditon' (did you mean 'n_per_condition'?)
instead of leaving the design on the default of twenty items. Any other
unknown key is noted in the run log and otherwise ignored, and a later
release will reject it. Notes and comments therefore belong in
description or in a key beginning x_, which
are never checked.
The same step checks the values. It rejects a missing or non-text
name or language and an
n_per_condition that is not a whole number of at least 1. A
language written as a code such as 'no' must
be quoted, because YAML otherwise reads it as false. The step also
rejects a name, language or font that cannot be written safely into the
generated experiments, a trigger port that is not an address, and an
unknown timing key or malformed response key. Last, it rejects a
condition name that is not text or appears twice, and a corpus or pool
design with conditions but no match_on.
run_pipeline() takes a design path and does everything
this vignette does by hand. It writes the stimuli, the reports, the
three experiments, a materials datasheet recording the provenance and
realised control of the run, and a run log. Paths in a design are
resolved relative to the working directory, so run this code from the
root of a clone of the repository:
Matching always returns a set, so the report is needed to judge whether the set is well matched. It belongs before anything is generated from the stimuli.
report <- match_report(
stim, c("length", "frequency", "n_density", "old20"), schema
)
knitr::kable(
report$descriptives,
caption = "Descriptive statistics per condition"
)| group | dimension | n | mean | sd | min | median | max |
|---|---|---|---|---|---|---|---|
| high_frequency | length | 60 | 4.650 | 1.313 | 3.0 | 4.00 | 8.00 |
| high_frequency | frequency | 60 | 5.604 | 0.418 | 5.2 | 5.42 | 6.85 |
| high_frequency | n_density | 60 | 5.600 | 6.304 | 0.0 | 3.00 | 23.00 |
| high_frequency | old20 | 60 | 1.738 | 0.486 | 1.0 | 1.70 | 2.80 |
| low_frequency | length | 60 | 4.633 | 1.275 | 3.0 | 4.00 | 7.00 |
| low_frequency | frequency | 60 | 4.060 | 0.169 | 3.8 | 4.05 | 4.40 |
| low_frequency | n_density | 60 | 5.233 | 5.400 | 0.0 | 3.50 | 17.00 |
| low_frequency | old20 | 60 | 1.736 | 0.477 | 1.0 | 1.75 | 2.70 |
knitr::kable(
report$comparisons,
caption = paste(
"Standardised mean differences with 90% confidence intervals, plus the",
"complementary TOST equivalence test"
)
)| condition | reference | dimension | cohens_d | d_ci_low | d_ci_high | var_ratio | tost_p | equivalent | mean_diff | role |
|---|---|---|---|---|---|---|---|---|---|---|
| low_frequency | high_frequency | length | 0.013 | -0.290 | 0.316 | 0.944 | 0.0044 | TRUE | 0.017 | NA |
| low_frequency | high_frequency | frequency | 4.840 | 4.240 | 5.440 | 0.164 | 1.0000 | FALSE | 1.545 | NA |
| low_frequency | high_frequency | n_density | 0.062 | -0.240 | 0.365 | 0.734 | 0.0091 | TRUE | 0.367 | NA |
| low_frequency | high_frequency | old20 | 0.005 | -0.297 | 0.308 | 0.963 | 0.0039 | TRUE | 0.002 | NA |
The manipulated dimension (frequency) differs strongly, while on the controlled dimensions (length, neighbourhood density and OLD20) the differences are small and each passes the equivalence test. Showing equivalence needs enough items. At the default bound and alpha, no dimension that varies can pass with fewer than 23 items per condition.
A non-significant difference test is not evidence of matching, and
its outcome depends on the number of items. The report therefore leads
with the standardised mean difference (cohens_d) and its
90% confidence interval (d_ci_low, d_ci_high),
whose upper limit is the largest imbalance still consistent with the
stimuli. The TOST equivalence test is reported alongside as a
bound-referenced verdict.
The trial is described once, as a sequence of events, and each presentation target renders that same description. One call therefore produces all three experiments, with the tables the PsychoPy and OpenSesame targets read.
out <- file.path(tempdir(), "lexsync_demo")
dir.create(out, showWarnings = FALSE)
files <- export_experiments(stim, design, schema, out)
basename(files$files)[1] "quick_start_english_psychopy.py"
[2] "quick_start_english.osexp"
[3] "quick_start_english.html"
[4] "quick_start_english_psychopy.csv"
[5] "quick_start_english_opensesame.csv"
[6] "quick_start_english_opensesame_text.csv"
The PsychoPy script binds each onset trigger to the stimulus flip
with win.callOnFlip. The OpenSesame .osexp
draws and shows the word inside an inline_script and sends
the trigger immediately after show() returns, which it does
at the display refresh. The jsPsych HTML runs the same procedure in a
browser and records the trigger codes in the trial data, since a browser
cannot address a parallel port. Neither the matching nor the script
generation requires PsychoPy, OpenSesame or any hardware to be
installed.
Four further articles go into depth. Matching, dimensions and designs covers the lexical dimensions and their units, the four matching methods and when each one suits, tolerance windows, continuous designs and how to read the validation report. Experiments, paradigms and EEG triggers covers the trial-event model, the paradigm registry, the three presentation targets and the flip-locked trigger timing. Reproducibility, parity and the materials datasheet covers the cross-engine guarantee, how it is achieved and where it stops. The app covers the Shiny front-end in the repository, which assembles a design through a browser tab, runs this same pipeline and exports the code that reproduces the run.
Andrews, S. (1989). Frequency and neighborhood effects on lexical access: Activation or search? Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(5), 802–814. https://doi.org/10.1037/0278-7393.15.5.802