Databases containing biodiversity information - such as those that
can be queried using galah - contain millions of records
collected over decades or centuries by thousands of observers. It is
unsurprising, therefore, that they contain some errors. In this context,
two key questions for users of that data are:
Data quality systems in galah are built on a set of fields called
‘assertions’. These are usually binary (true/false) variables that flag
that something might be wrong with the data, generated either
programmatically or by users. You can see them by calling
show_all_assertions():
## # A tibble: 124 × 4
## id description category type
## <chr> <chr> <chr> <chr>
## 1 AMBIGUOUS_COLLECTION Ambiguous collection Comment assertions
## 2 AMBIGUOUS_INSTITUTION Ambiguous institution Comment assertions
## 3 BASIS_OF_RECORD_INVALID Basis of record badly formed Warning assertions
## 4 biosecurityIssue Biosecurity issue Error assertions
## 5 COLLECTION_MATCH_FUZZY Collection match fuzzy Comment assertions
## 6 COLLECTION_MATCH_NONE Collection not matched Comment assertions
## 7 CONTINENT_COORDINATE_MISMATCH continent coordinate mismatch Warning assertions
## 8 CONTINENT_COUNTRY_MISMATCH Continent country mismatch Warning assertions
## 9 CONTINENT_DERIVED_FROM_COORDINATES Continent derived from coordinates Warning assertions
## 10 CONTINENT_DERIVED_FROM_COUNTRY continent derived from country Warning assertions
## # ℹ 114 more rows
The easiest thing to do with these is simply download them with your
chosen records, and do your own checking on the data. This can be
achived via select():
galah_call() |>
identify("Eolophus roseicapilla") |>
filter(year >= 2020,
basisOfRecord == "PRESERVED_SPECIMEN") |>
select(group = c("basic", "assertions")) |>
collect()## Retrying in 1 seconds.
## Retrying in 2 seconds.
## Retrying in 4 seconds.
## Retrying in 8 seconds.
## # A tibble: 42 × 133
## recordID scientificName taxonConceptID decimalLatitude decimalLongitude eventDate basisOfRecord occurrenceStatus dataResourceName AMBIGUOUS_COLLECTION AMBIGUOUS_INSTITUTION
## <chr> <chr> <chr> <dbl> <dbl> <dttm> <chr> <chr> <chr> <lgl> <lgl>
## 1 03e1c487-f3be-4469-a… Eolophus rose… https://biodi… -34.8 148. 2022-01-14 00:00:00 PRESERVED_SP… PRESENT Australian Nati… FALSE FALSE
## 2 0a24639a-1387-4a53-b… Eolophus rose… https://biodi… -34.8 148. 2022-01-14 00:00:00 PRESERVED_SP… PRESENT Australian Nati… FALSE FALSE
## 3 0eaf52fc-6659-41ea-8… Eolophus rose… https://biodi… -24.2 139. 2026-08-26 00:00:00 PRESERVED_SP… PRESENT Australian Nati… FALSE FALSE
## 4 10081d36-abf8-4928-9… Eolophus rose… https://biodi… -33.8 151. 2024-12-11 00:00:00 PRESERVED_SP… PRESENT Australian Muse… FALSE FALSE
## 5 10e9d3c3-1fb2-4362-8… Eolophus rose… https://biodi… -33.6 151. NA PRESERVED_SP… PRESENT Australian Muse… FALSE FALSE
## 6 11663284-5f43-4cf2-a… Eolophus rose… https://biodi… -28.2 144. 2025-09-04 00:00:00 PRESERVED_SP… PRESENT Australian Nati… FALSE FALSE
## 7 1baea6d4-a2d1-4e69-8… Eolophus rose… https://biodi… -36.4 144. 2024-09-21 00:00:00 PRESERVED_SP… PRESENT Museums Victori… FALSE FALSE
## 8 1fe45f99-a245-44d7-b… Eolophus rose… https://biodi… -28.3 136. 2025-08-02 00:00:00 PRESERVED_SP… PRESENT Australian Nati… FALSE FALSE
## 9 3810878b-8ad7-4a86-a… Eolophus rose… https://biodi… -32.6 150. 2022-05-14 00:00:00 PRESERVED_SP… PRESENT Australian Muse… FALSE FALSE
## 10 3e6152db-aef1-4e32-b… Eolophus rose… https://biodi… -30.1 138. 2025-08-01 00:00:00 PRESERVED_SP… PRESENT Australian Nati… FALSE FALSE
## # ℹ 32 more rows
## # ℹ 122 more variables: BASIS_OF_RECORD_INVALID <lgl>, biosecurityIssue <lgl>, COLLECTION_MATCH_FUZZY <lgl>, COLLECTION_MATCH_NONE <lgl>, CONTINENT_COORDINATE_MISMATCH <lgl>,
## # CONTINENT_COUNTRY_MISMATCH <lgl>, CONTINENT_DERIVED_FROM_COORDINATES <lgl>, CONTINENT_DERIVED_FROM_COUNTRY <lgl>, CONTINENT_INVALID <lgl>, COORDINATE_INVALID <lgl>,
## # COORDINATE_OUT_OF_RANGE <lgl>, COORDINATE_PRECISION_INVALID <lgl>, COORDINATE_REPROJECTED <lgl>, COORDINATE_REPROJECTION_FAILED <lgl>, COORDINATE_REPROJECTION_SUSPICIOUS <lgl>,
## # COORDINATE_ROUNDED <lgl>, COORDINATE_UNCERTAINTY_METERS_INVALID <lgl>, COORDINATES_CENTRE_OF_COUNTRY <lgl>, COORDINATES_CENTRE_OF_STATEPROVINCE <lgl>, COUNTRY_COORDINATE_MISMATCH <lgl>,
## # COUNTRY_DERIVED_FROM_COORDINATES <lgl>, COUNTRY_INVALID <lgl>, COUNTRY_MISMATCH <lgl>, DEPTH_MIN_MAX_SWAPPED <lgl>, DEPTH_NON_NUMERIC <lgl>, DEPTH_NOT_METRIC <lgl>, DEPTH_UNLIKELY <lgl>,
## # detectedOutlier <lgl>, DIFFERENT_OWNER_INSTITUTION <lgl>, ELEVATION_MIN_MAX_SWAPPED <lgl>, ELEVATION_NON_NUMERIC <lgl>, ELEVATION_NOT_METRIC <lgl>, ELEVATION_UNLIKELY <lgl>, …
As you can see, there are a huge number of assertions, and they will take some processing. It is also inefficient to download data we don’t need, only to filter it later. Therefore, it is worth considering filtering out some problematic records as part of our usual filtering strategy.
Data quality ‘profiles’ are pre-populated filter statements curated by the ALA that are designed to remove problematic records. They provide a quick and straightforward way to remove these records without having to construct elaborate filter statements.
## # A tibble: 5 × 4
## id short_name name description
## <int> <chr> <chr> <chr>
## 1 92 ALA ALA General "The default ALA profile filters out records based on the filter groups outlined below. This is a moderately restricted set of data."
## 2 124 CSDM Species Distribution Modelling (CSDM) "Base filters for the Collaborative Species Distribution Modelling program"
## 3 133 re-usable Data licensed for all uses "Data licensed for re-use, including commercial uses. This profile only filters on license, no \"quality\" filters are applied."
## 4 224 AVH AVH "AVH data quality profile"
## 5 252 ALA-test ALA General Test "A test version of the default ALA profile filters out records based on the filter groups outlined below. This is a moderately restricted s…
Let’s take a look at the most obvious one first:
## • Showing values for 'CSDM'.
## # A tibble: 4 × 4
## id description filter enabled
## <int> <chr> <chr> <lgl>
## 1 126 "Include only records where Spatial validity is \"true\"" "spatiallyValid:\"true\"" TRUE
## 2 128 "Exclude potential duplicate records" "-isDuplicateOf:*" TRUE
## 3 130 "Exclude all records that are an outlier against any environmental layer" "-outlierLayerCount:[1 TO *]" TRUE
## 4 132 "Include only records where Year is 1970 to 2099" "year:[1970 TO *]" TRUE
This is ok, but needs unpacking:
spatiallyValid is an amalgam of other assertions. At
the time of writing, true means that the latitude and
longitude are known and within +/- 90 and 180 degrees (respectively),
but are not 0,0, and that there isn’t a mismatch between the
observed country and the country listed by the data provider.Now let’s look at the ‘ALA general’ approach:
## ! Search returned 2 matched profiles.
## • Showing values for 'ALA'.
## # A tibble: 26 × 4
## id description filter enabled
## <int> <chr> <chr> <lgl>
## 1 94 "Exclude all records where spatial validity is \"false\"" "-spa… TRUE
## 2 96 "Exclude all records with an assertion that the scientific name provided does not match any of the names lists used by the ALA. For a full explanation of the ALA name match… "-ass… TRUE
## 3 97 "Exclude all records with an assertion that the scientific name provided is not structured as a valid scientific name. Also catches rank values or values such as \"UNKNOWN\"" "-ass… TRUE
## 4 99 "Exclude all records with an assertion that kingdom provided doesn't match a known kingdom e.g. Animalia, Plantae" "-ass… TRUE
## 5 100 "Exclude all records with an assertion that the scientific name provided in the record does not match the expected taxonomic scope of the resource e.g. Mammal records attrib… "-ass… TRUE
## 6 98 "Exclude all records with an assertion that the name and classification supplied can't be used to choose between 2 homonyms" "-ass… TRUE
## 7 103 "Exclude all records with an assertion of latitude value provided is zero" "-dec… TRUE
## 8 104 "Exclude all records with an assertion of longitude value provided is zero" "-dec… TRUE
## 9 105 "Exclude all records with an assertion of latitude and longitude have been transposed" "-ass… TRUE
## 10 106 "Exclude all records with an assertion of coordinates are the exact centre of the state or territory" "-ass… TRUE
## # ℹ 16 more rows
There is a lot here and we won’t go through it all; but two points are potentially useful. First, this makes a lot of assumptions about what is and is not useful, with 26 separate filters. This is a lot, and while most of them are useful in some contexts, equally this may be too stringent a set of tests for some applications. Applying it to our earlier example, we see it only removes two records, but your milage will vary for different queries:
galah_call() |>
identify("Eolophus roseicapilla") |>
filter(year >= 2020,
basisOfRecord == "PRESERVED_SPECIMEN") |>
apply_profile("ALA") |>
count() |>
collect()## # A tibble: 1 × 1
## count
## <int>
## 1 40
Second, the filter column lists how the query is sent to
the ALA; this can be useful for replicating parts of the workflow that
are useful for your specific application. We’ll try this out below.
Assertions are stored in what is called a ‘multi-value field’, meaning a column where each cell can contain multiple strings. You remove them using the ‘assertions’ field name in filter:
## # A tibble: 1 × 1
## count
## <int>
## 1 195446531
Conversely, if you only want records with an invalid name,
just use ==:
## # A tibble: 1 × 1
## count
## <int>
## 1 3620270
Of course, we might want to filter out multiple assertions at once.
Rather than keep writing assertions !=, we can use
%in% with a leading ! (meaning
NOT):
excluded_assertions <- c("INVALID_SCIENTIFIC_NAME",
"COORDINATES_CENTRE_OF_STATEPROVINCE")
galah_call() |>
filter(!(assertions %in% excluded_assertions)) |>
count() |>
collect()## # A tibble: 1 × 1
## count
## <int>
## 1 195442453
It is important to note, however, that a lot of things we want to filter out are properties of the data itself, rather than results of checks the organisation has run. For example, the following parameters all involve fields (not assertions), but are included in the general filter:
galah_call() |>
filter(basisOfRecord != "FOSSIL_SPECIMEN", # no fossils
contentTypes != "Environmental DNA",
decimalLatitude != 0, # no points defaulting to zero-zero
decimalLongitude != 0,
occurrenceStatus != "ABSENT", # exclude observations of missingness
year > 1700, # remove very early records
-(degreeOfEstablishment %in% c("cultivated", "captive")), # introduced species
-(identificationVerificationStatus %in% c("verification required", "needs_id")) # uncertain cit sci records
) |>
count() |>
collect()## # A tibble: 1 × 1
## count
## <int>
## 1 171820807
A complementary approach is to call ‘user assertions’; flags placed by users on individual suspect records. You can find records flagged in this way as follows:
## # A tibble: 4 × 3
## userAssertions userAssertions_label count
## <chr> <chr> <int>
## 1 50005 Unconfirmed 11282
## 2 50003 Corrected 3994
## 3 50002 Verified 462
## 4 50001 Open issue 95
The categories here are a little obscure, but cross-referencing with
the ALA data quality profile, it appears that only issues that are
‘Open’ (code 50001) or ‘Unconfirmed’ (code
50005) have some likelihood of being ‘true’ errors. Another
point to consider is who is making these assertions:
galah_call() |>
filter(userAssertions %in% c("50001", "50005")) |>
count(assertionUserId) |>
slice_head(n = 10) |>
collect()## Limiting to first 10 of 1,382 rows.
## Use `atlas_counts(limit = )` to return more rows.
## # A tibble: 10 × 3
## assertionUserId assertionUserId_label count
## <chr> <chr> <int>
## 1 57159 Thomas Mesaglio 1463
## 2 124186 Cameron Slatyer 1414
## 3 2126 Niels Klazenga 1059
## 4 3334 Penelope Mills 320
## 5 8902 ralph FOSTER 235
## 6 11936 Kay Durham 218
## 7 1108 Allen Sundholm 204
## 8 29516 Wendy Potts 187
## 9 25919 Matthew Connors 177
## 10 109408 Ben Travaglini 164
This isn’t actually terribly informative unless we know who these
people are. But it likely that the types of errors flagged by a single
user will reflect their spatial and taxonomic knowledge of the
Australian biota. To remove any records with user assertions, use
is.na(userAssertions) within your filter statement:
## # A tibble: 1 × 1
## count
## <int>
## 1 199050968
Data quality is an ongoing topic of investigation for researchers and data infrastructures. The tools described above are the current best available toolset implemented at the institutional level. For a more general assessment of how to handle data quality issues in R, see Cleaning Biodiversity Data in R by Dax Kellie et al. (2026, https://doi.org/10.54102/ala.77009).
A critical limitation of data quality assertions and their derivatives is that they focus exclusively on rule-based methods; there is no probabilistic component to them. Users seeking to draw probabilistic inference from these data will need to consider carefully whether there are classes of outlier relevant to their use case, but which are invisible to the systems in question.