---
title: "Get started"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Get started}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r, include = FALSE}
knitr::opts_chunk$set(collapse = FALSE, comment = "")
# Console colour carries no meaning on a rendered page. pkgdown turns it on for
# its own build, and the escape sequences then reach the reader as literal text,
# so colour is switched off here for a plain vignette render and a site build
# alike. The fixed width keeps tibbles inside the documentation column.
options(cli.num_colors = 1, cli.hyperlink = FALSE, crayon.enabled = FALSE,
width = 80)
# Print data frames and tibbles as formatted tables.
local({
kp <- function(x, ...) {
if (any(vapply(x, is.list, logical(1)))) return(knitr::normal_print(x))
knitr::knit_print(knitr::kable(x))
}
for (cls in c("data.frame", "tbl_df", "tbl")) {
registerS3method("knit_print", cls, kp, envir = asNamespace("knitr"))
}
})
```
```{r setup}
library(scopusflow)
```
This vignette is fully reproducible without a Scopus API key. Elsevier's API
terms do not permit redistributing retrieved records, so no package can ship a
genuine Scopus harvest, and scopusflow bundles an openly licensed stand-in
instead. `example_records` holds 138 real journal articles on graphene
supercapacitors published between 2015 and 2024, with their real titles, DOIs,
journals, first authors and citation counts. They come from OpenAlex, whose
metadata is released under CC0, reshaped into the schema a retrieval returns.
The harvest is complete, so its rows per year are the real number of
publications per year for that query, and its gaps are real gaps too.
Eleven records carry no DOI and two no source title, exactly as they arrive.
Running the equivalent query against Scopus yields the same kind of object, with
the same columns and the same handling, though not an identical set of records.
The steps that need the API are shown but not run, each paired with the offline
equivalent.
## Describing a search as a plan
A plan separates describing a search from executing it. Plans are inspectable,
saveable and version-controllable, and they can be partitioned, for example by
year, so that a large retrieval stays under the API's `start < 5000` ceiling and
can be cached and resumed.
The plan below describes the search the bundled records came from, so the rest
of the article follows one worked example from description to export.
```{r}
plan <- scopus_plan(
"graphene supercapacitor",
years = 2015:2024,
field = "TITLE-ABS-KEY",
partition = "year"
)
plan
```
Each row is one query cell. Field tags wrap the query and years become a date
filter.
```{r}
scopus_plan("language learning", field = "TITLE")$query
scopus_plan("x", years = 2015:2020)$date
# A plan is a classed object, and is_scopus_plan() confirms it.
is_scopus_plan(plan)
```
## Sizing and fetching
`scopus_has_key()` reports whether a key is configured, without revealing it. It
is the guard the package's own examples use to skip the steps that need the API,
so it is the natural switch for a reproducible script.
```{r}
scopus_has_key()
```
With a key configured, you size a search cheaply and then execute the plan,
optionally caching each cell so that an interrupted run resumes without
re-spending quota. These contact the API, so they are not evaluated here.
```{r eval = FALSE}
scopus_count(
"graphene supercapacitor",
years = 2015:2024,
field = "TITLE-ABS-KEY"
)
records <- scopus_fetch_plan(
plan,
cache_dir = scopus_cache_dir(),
resume = TRUE
)
```
Without a key, the bundled corpus stands in for the result of that harvest, and
the sections below run on it.
## The record schema
Whether records come from the API or from the bundled corpus, they share one
stable schema, so everything below would read the same on a harvest of your
own. `summary()` takes stock of a set, and the first rows show the columns.
```{r}
records <- example_records
summary(records)
head(records)
# A record set is a classed tibble, and is_scopus_records() confirms the
# contract.
is_scopus_records(records)
```
`scopus_records()` produces this same shape from a raw API response, flattening
the nested result into one row per record. The entry below carries the fields of
a real article, one of those in the bundled corpus, in the form the API returns
them.
```{r}
raw <- list(entry = list(
list(`prism:doi` = "10.1021/am509065d",
`dc:title` =
"Flexible and Stackable Laser-Induced Graphene Supercapacitors",
`dc:creator` = "Zhiwei Peng",
`prism:publicationName` = "ACS Applied Materials & Interfaces",
`prism:coverDate` = "2015-01-13", `citedby-count` = "469")
))
scopus_records(raw, query = "TITLE-ABS-KEY(graphene supercapacitor)")
```
## Most frequent sources and authors
A record set already answers the first descriptive questions. `scopus_top()`
tallies the most frequent sources or authors, counting each contributor once
per record. Across these 138 articles the tally is long-tailed, as a real
literature is. They are spread over 90 distinct journals, and only one, *ACS
Applied Materials & Interfaces*, appears more than five times.
```{r}
scopus_top(records, by = "source")
```
`vignette("analysing-a-literature")` covers growth trends, top-source and
top-author plots and abstract retrieval in depth.
## DOIs and change tracking
Extract a clean, deduplicated DOI list for import into a reference manager, and
compare two retrievals to see exactly what changed. Eleven of the 138 records
arrived without a DOI, so 127 come back.
```{r}
dois <- scopus_extract_dois(records)
length(dois)
head(dois, 4)
```
A search re-run later gains records and occasionally loses one to re-indexing.
Here the baseline stops at 2023 and the second pull adds the 2024 articles while
dropping the first record.
```{r}
baseline <- records[records$year <= 2023, ]
later <- records[-1, ]
print(scopus_diff_dois(old = baseline, new = later))
```
You can write the DOIs to a path you specify, and read the file back to see
exactly what lands on disk.
```{r}
out <- file.path(tempdir(), "dois.csv")
scopus_extract_dois(records, file = out)
writeLines(head(readLines(out), 5))
```
## Comparing topic trends
`scopus_compare_topics()` measures how the internal emphasis of a literature
shifts, expressed as each comparison topic's yearly share of the reference
literature. It issues one count request per term per year, so it needs the API.
```{r eval = FALSE}
cmp <- scopus_compare_topics(
reference_query = "language learning",
comparison_terms = c("effect size", "Bayesian"),
years = 2015:2020,
field = "TITLE-ABS-KEY"
)
plot_scopus_comparison(cmp)
```
The result is a tidy table with one row per topic and year, which
`plot_scopus_comparison()` draws with direct line labels, a colour-blind-safe
palette and shaded stability bands. `vignette("comparing-topics")` builds the
object offline, shows the plot in its variations and explains how to read the
bands.
## Author keywords and references
A search only returns the fields the Search API carries. Author keywords and a
document's own reference list need `view = "COMPLETE"` and Abstract Retrieval
respectively, both at a materially different quota cost from an ordinary
search. `vignette("keywords-and-references")` walks through both, and
`scopus_corpus()`, which combines them into a minimal
`id`/`title`/`year`/`keywords`/`references` shape for downstream tools.
## Export and interoperability
Hand results to `bibliometrix`-style workflows, or save and reload them.
```{r}
m <- as_bibliometrix(records)
head(m[, c("AU", "TI", "PY", "SO", "TC")])
path <- file.path(tempdir(), "records.rds")
write_scopus_records(records, path)
identical(read_scopus_records(path), records)
```
## Writing the search up
A search that is going into a paper has to be reported. `scopus_search_report()`
assembles that record from the plan and the harvest, to the PRISMA-S standard,
and formats it as a methods paragraph. It states only what the objects hold, so
the record below is candid about the bundled corpus carrying no retrieval time
of its own. The *Search plans and quota-aware retrieval* article works through
it in full.
```{r}
report <- scopus_search_report(records, plan = plan)
cat(format(report, style = "paragraph"))
```
## Handling failures
Network and API problems surface as typed conditions, all inheriting from
`scopus_error`, so a workflow can respond to them in code.
```{r eval = FALSE}
tryCatch(
scopus_fetch("..."),
scopus_error_no_key = function(e) message("No API key configured."),
scopus_error_rate_limit = function(e)
message("Rate limited, so backing off."),
scopus_error = function(e)
message("Scopus error: ", conditionMessage(e))
)
```