This document introduces you to the swfscAirDAS package. This package is intended to standardize and streamline processing of aerial survey DAS (AirDAS) data collected using the PHOCOENA, TURTLE, or CARETTA programs from the Southwest Fisheries Science Center. In DAS data (and thus AirDAS data), an event is only recorded when something changes or happens, which can complicate processing. Thus, the main theme of this package is enabling analyses and downstream processing by 1) determining associated state and condition information for each event and 2) pulling out event-specific information from the Data columns. The following examples generally use the default function arguments; see the documentation for the individual functions for a complete description of all processing options.
Data
This package includes a sample AirDAS file, which we will use in this document. This file is formatted as a DAS file created using the TURTLE program
y <- system.file("airdas_sample.das", package = "swfscAirDAS")
head(readLines(y))
#> [1] " 1C 123059 040915 N39:14.13 W123:09.38 Recorder: dd "
#> [2] " 2C 123059 040915 N39:14.13 W123:09.38 Not recording molas "
#> [3] " 3T.123059 040915 N39:14.13 W123:09.38 T1 "
#> [4] " 4V.123059 040915 N39:14.13 W123:09.38 g g g g g "
#> [5] " 5P.123059 040915 N39:14.13 W123:09.38 aa bb cc dd "
#> [6] " 6A.123059 040915 N39:14.13 W123:09.38 650 100 "Check data format
The first step in processing AirDAS data is to ensure that the DAS
file has expected formatting and values, which you can do using the
airdas_check function. The checks performed by this
function are detailed in the function documentation, which can be
accessed by running ?airdas_check. If you aren’t sure of
the file type (format) of your data, you can check the format PDFs.
These PDFs are available online at https://swfsc.github.io/swfscAirDAS/, or see
?airdas_format_pdf for how to access a local copy.
# Code not run
y.check <- airdas_check(y, file.type = "turtle", skip = 0, print.transect = TRUE)Read and process data
Once QA/QC is complete and you have fixed any data entry errors, you can begin to process the AirDAS data. The backbone of this package is the reading and processing steps: 1) the data from the DAS file are read into the columns of a data frame and 2) state and condition information are extracted for each event. This means that after processing, you can simply look at any event (row) and determine the Beaufort, viewing conditions, etc., at the time of the event. All other functions in the package depend on the AirDAS data being in this processed state.
# Read
y.read <- airdas_read(y, file.type = "turtle", skip = 0)
head(y.read)
#> Event EffortDot DateTime Lat Lon Data1 Data2 Data3 Data4
#> 1 C FALSE 2015-04-09 12:30:59 39.2355 -123.1563 Reco rder: dd <NA>
#> 2 C FALSE 2015-04-09 12:30:59 39.2355 -123.1563 Not recor ding molas
#> 3 T TRUE 2015-04-09 12:30:59 39.2355 -123.1563 T1 <NA> <NA> <NA>
#> 4 V TRUE 2015-04-09 12:30:59 39.2355 -123.1563 g g g g
#> 5 P TRUE 2015-04-09 12:30:59 39.2355 -123.1563 aa bb cc dd
#> 6 A TRUE 2015-04-09 12:30:59 39.2355 -123.1563 650 100 <NA> <NA>
#> Data5 Data6 Data7 EventNum file_das line_num file_type
#> 1 <NA> <NA> <NA> 1 airdas_sample.das 1 turtle
#> 2 <NA> <NA> <NA> 2 airdas_sample.das 2 turtle
#> 3 <NA> <NA> <NA> 3 airdas_sample.das 3 turtle
#> 4 g <NA> <NA> 4 airdas_sample.das 4 turtle
#> 5 <NA> <NA> <NA> 5 airdas_sample.das 5 turtle
#> 6 <NA> <NA> <NA> 6 airdas_sample.das 6 turtle
# Process
y.proc <- airdas_process(y.read, trans.upper = TRUE)
head(y.proc)
#> Event DateTime Lat Lon OnEffort Trans Bft CCover Jelly
#> 1 C 2015-04-09 12:30:59 39.2355 -123.1563 FALSE <NA> NA NA NA
#> 2 C 2015-04-09 12:30:59 39.2355 -123.1563 FALSE <NA> NA NA NA
#> 3 T 2015-04-09 12:30:59 39.2355 -123.1563 TRUE T1 NA NA NA
#> 4 V 2015-04-09 12:30:59 39.2355 -123.1563 TRUE T1 NA NA NA
#> 5 P 2015-04-09 12:30:59 39.2355 -123.1563 TRUE T1 NA NA NA
#> 6 A 2015-04-09 12:30:59 39.2355 -123.1563 TRUE T1 NA NA NA
#> HorizSun VertSun HKR Haze Kelp RedTide AltFt SpKnot ObsL ObsB ObsR Rec
#> 1 NA NA <NA> FALSE FALSE FALSE NA NA <NA> <NA> <NA> <NA>
#> 2 NA NA <NA> FALSE FALSE FALSE NA NA <NA> <NA> <NA> <NA>
#> 3 NA NA <NA> FALSE FALSE FALSE NA NA <NA> <NA> <NA> <NA>
#> 4 NA NA <NA> FALSE FALSE FALSE NA NA <NA> <NA> <NA> <NA>
#> 5 NA NA <NA> FALSE FALSE FALSE NA NA aa bb cc dd
#> 6 NA NA <NA> FALSE FALSE FALSE 650 100 aa bb cc dd
#> VLI VLO VB VRI VRO Data1 Data2 Data3 Data4 Data5 Data6 Data7 EffortDot
#> 1 <NA> <NA> <NA> <NA> <NA> Reco rder: dd <NA> <NA> <NA> <NA> FALSE
#> 2 <NA> <NA> <NA> <NA> <NA> Not recor ding molas <NA> <NA> <NA> FALSE
#> 3 <NA> <NA> <NA> <NA> <NA> T1 <NA> <NA> <NA> <NA> <NA> <NA> TRUE
#> 4 g g g g g g g g g g <NA> <NA> TRUE
#> 5 g g g g g aa bb cc dd <NA> <NA> <NA> TRUE
#> 6 g g g g g 650 100 <NA> <NA> <NA> <NA> <NA> TRUE
#> EventNum file_das line_num file_type
#> 1 1 airdas_sample.das 1 turtle
#> 2 2 airdas_sample.das 2 turtle
#> 3 3 airdas_sample.das 3 turtle
#> 4 4 airdas_sample.das 4 turtle
#> 5 5 airdas_sample.das 5 turtle
#> 6 6 airdas_sample.das 6 turtleOnce you have processed the AirDAS data, you can easily access a variety of information. For instance, you can look at the different Beaufort values that occurred in the data, or filter for specific events to get the beginning and ending points of each effort section.
# The number of events per Beaufort value
table(y.proc$Bft)
#>
#> 1 2
#> 58 19
# Filter for T/R and O/E events to extract lat/lon points
y.proc %>%
filter(Event %in% c("T", "R", "E", "O")) %>%
select(Event, Lat, Lon, Trans)
#> Event Lat Lon Trans
#> 1 T 39.23550 -123.1563 T1
#> 2 E 39.23400 -123.4047 T1
#> 3 R 39.23400 -123.4742 T1
#> 4 O 39.23283 -123.6033 <NA>
#> 5 T 39.22000 -123.6078 T2
#> 6 O 39.21700 -123.2417 <NA>Sightings
The swfscAirDAS package does contain specific functions
for extracting and/or summarizing particular information from the
processed data. First is airdas_sight, a function that
returns a data frame with pertinent sighting data extracted to their own
columns
y.sight <- airdas_sight(y.proc)
y.sight %>%
select(Event, SightNo:TurtleTail) %>%
head()
#> Event SightNo Obs Angle ObsStd SightStd Mixed SpCode GsTotal GsSp TurtleSize
#> 1 S 1 cc 67 TRUE TRUE FALSE mn 6 6 NA
#> 2 S 2 bb -60 TRUE TRUE FALSE mn 6 6 NA
#> 3 S 3 cc 38 TRUE TRUE FALSE mn 2 2 NA
#> 4 S 4 aa -50 TRUE TRUE FALSE mn 5 5 NA
#> 5 S 5 cc 49 TRUE TRUE FALSE bm 4 4 NA
#> 6 S 6 cc 42 TRUE TRUE TRUE gg 10 8 NA
#> TurtleDirection TurtleTail
#> 1 NA <NA>
#> 2 NA <NA>
#> 3 NA <NA>
#> 4 NA <NA>
#> 5 NA <NA>
#> 6 NA <NA>Effort
In addition, you can chop the effort data into segments, and
summarize the conditions and sightings on those segments using
airdas_effort and airdas_effort_sight. These
effort segments can be used for line transect estimates using the
Distance software, species distribution modeling, or summarizing the
number of harbor porpoises on each transect, among other uses.
airdas_effort chops continuous effort sections (the event
sequence from T/R to E/O events) into effort segments using one of
several different chopping methods: condition (a new effort segment
every time a condition changes), equal length (effort segments of equal
length), and section (each segment is a full continuous effort section,
i.e. it runs from a T/R event to an E/O event).
airdas_effort_sight takes the output of
airdas_effort and returns the number of sightings and
animals per segment for specified species codes.
Both functions return a list of three data frames: segdata,
sightinfo, and randpicks. Briefly, as these data frames and the
different chopping methodologies are described in depth in the function
documentation (?airdas_effort and
?airdas_effort_sight), segdata contains information about
each effort segment, sightinfo contains information about the sightings
on the segments, and randpicks contains information specific to the
‘equal length’ chopping method. These are separate functions to allow
the user more control over the filters applied to determine which
sightings should be included in the summaries (see
?airdas_effort).
# Chop the effort every time a condition changes
y.eff <- airdas_effort(
y.proc, method = "condition", seg.min.km = 0,
dist.method = "greatcircle", conditions = c("Bft", "CCover"),
num.cores = 1, angle.min = 12, bft.max = 1
)
y.eff.sight <- airdas_effort_sight(y.eff, sp.codes = c("bm", "dc"))
head(y.eff.sight$segdata)
#> segnum section_id section_sub_id event transect file stlin
#> 1 1 1 1 T T1 airdas_sample.das 3
#> 2 2 1 2 T T1 airdas_sample.das 36
#> 3 3 2 1 R T1 airdas_sample.das 40
#> 4 4 2 2 R T1 airdas_sample.das 41
#> 5 5 2 3 R T1 airdas_sample.das 44
#> 6 6 3 1 T T2 airdas_sample.das 53
#> endlin lat1 lon1 DateTime1 lat2 lon2
#> 1 36 39.23550 -123.1563 2015-04-09 12:30:59 39.23400 -123.4033
#> 2 37 39.23400 -123.4033 2015-04-09 12:37:57 39.23400 -123.4047
#> 3 41 39.23400 -123.4742 2015-04-09 12:40:00 39.23333 -123.4914
#> 4 44 39.23333 -123.4914 2015-04-09 12:40:28 39.23317 -123.5143
#> 5 49 39.23317 -123.5143 2015-04-09 12:41:08 39.23283 -123.6032
#> 6 78 39.22000 -123.6078 2015-04-09 12:46:51 39.21683 -123.3605
#> DateTime2 mlat mlon mDateTime dist year month
#> 1 2015-04-09 12:37:57 39.23309 -123.2797 2015-04-09 12:34:28 21.2882 2015 4
#> 2 2015-04-09 12:38:00 39.23400 -123.4040 2015-04-09 12:37:58 0.1148 2015 4
#> 3 2015-04-09 12:40:28 39.23367 -123.4828 2015-04-09 12:40:14 1.4937 2015 4
#> 4 2015-04-09 12:41:08 39.23317 -123.5029 2015-04-09 12:40:48 1.9657 2015 4
#> 5 2015-04-09 12:43:43 39.23245 -123.5588 2015-04-09 12:42:25 7.6620 2015 4
#> 6 2015-04-09 12:54:04 39.21667 -123.4842 2015-04-09 12:50:27 21.3146 2015 4
#> day mtime maxdistBft maxdistCCover nSI_bm ANI_bm nSI_dc ANI_dc
#> 1 9 12:34:28 1 10 2 9 1 1
#> 2 9 12:37:58 2 10 0 0 0 0
#> 3 9 12:40:14 2 10 0 0 0 0
#> 4 9 12:40:48 2 20 0 0 0 0
#> 5 9 12:42:25 1 20 0 0 0 0
#> 6 9 12:50:27 1 20 2 7 0 0
head(y.eff.sight$sightinfo)
#> segnum mlat mlon Event DateTime Lat Lon
#> 1 1 39.23309 -123.2797 S 2015-04-09 12:31:35 39.23367 -123.1783
#> 2 1 39.23309 -123.2797 S 2015-04-09 12:31:47 39.23350 -123.1857
#> 3 1 39.23309 -123.2797 S 2015-04-09 12:31:56 39.23333 -123.1917
#> 4 1 39.23309 -123.2797 S 2015-04-09 12:32:20 39.23233 -123.2053
#> 5 1 39.23309 -123.2797 S 2015-04-09 12:32:30 39.23233 -123.2108
#> 6 1 39.23309 -123.2797 S 2015-04-09 12:32:42 39.23233 -123.2195
#> OnEffort Trans Bft CCover Jelly HorizSun VertSun HKR Haze Kelp RedTide
#> 1 TRUE T1 1 10 0 6 NA n FALSE FALSE FALSE
#> 2 TRUE T1 1 10 0 6 NA n FALSE FALSE FALSE
#> 3 TRUE T1 1 10 0 6 NA n FALSE FALSE FALSE
#> 4 TRUE T1 1 10 0 6 NA n FALSE FALSE FALSE
#> 5 TRUE T1 1 10 0 6 NA n FALSE FALSE FALSE
#> 6 TRUE T1 1 10 0 6 NA n FALSE FALSE FALSE
#> AltFt SpKnot ObsL ObsB ObsR Rec VLI VLO VB VRI VRO EffortDot EventNum
#> 1 650 100 aa bb cc dd g g g g g TRUE 9
#> 2 650 100 aa bb cc dd g g g g g TRUE 10
#> 3 650 100 aa bb cc dd g g g g g TRUE 12
#> 4 650 100 aa bb cc dd g g g g g TRUE 14
#> 5 650 100 aa bb cc dd g g g g g TRUE 15
#> 6 650 100 aa bb cc dd g g g g g TRUE 16
#> file_das line_num file_type SightNo Obs Angle ObsStd SightStd Mixed
#> 1 airdas_sample.das 9 turtle 1 cc 67 TRUE TRUE FALSE
#> 2 airdas_sample.das 10 turtle 2 bb -60 TRUE TRUE FALSE
#> 3 airdas_sample.das 12 turtle 3 cc 38 TRUE TRUE FALSE
#> 4 airdas_sample.das 14 turtle 4 aa -50 TRUE TRUE FALSE
#> 5 airdas_sample.das 15 turtle 5 cc 49 TRUE TRUE FALSE
#> 6 airdas_sample.das 16 turtle 6 cc 42 TRUE TRUE TRUE
#> SpCode GsTotal GsSp TurtleSize TurtleDirection TurtleTail included
#> 1 mn 6 6 NA NA <NA> FALSE
#> 2 mn 6 6 NA NA <NA> FALSE
#> 3 mn 2 2 NA NA <NA> FALSE
#> 4 mn 5 5 NA NA <NA> FALSE
#> 5 bm 4 4 NA NA <NA> TRUE
#> 6 gg 10 8 NA NA <NA> FALSEIf you wanted to determine the number of humpback whales on each transect with a sighting angle between -60 and 60 degrees, for instance, you could do the following
# 'Chop' the effort by continuous effort section
y.eff.section <- airdas_effort(
y.proc, method = "section", dist.method = "greatcircle", conditions = NULL,
num.cores = 1
)
y.eff.section$sightinfo <- y.eff.section$sightinfo %>%
mutate(included = ifelse(.data$SpCode == "mn" & abs(.data$Angle > 60), FALSE, .data$included))
y.eff.section.sight <- airdas_effort_sight(y.eff.section, sp.codes = c("mn"))
y.eff.section.sight[[1]] %>%
mutate(transect_id = cumsum(.data$event == "T")) %>%
group_by(transect_id) %>%
summarise(dist_sum = sum(dist),
mn_count = sum(ANI_mn)) %>%
as.data.frame() #for printing format
#> transect_id dist_sum mn_count
#> 1 1 32.5244 20
#> 2 2 31.5482 18In this example you could also simply group by the
transect column, but that structure is not robust to
analyzing multi-year data sets where the same transects are flown
multiple times.
Comments
In AirDAS data, comments are a catch-all field, meaning they are used to record information that does not fit neatly into the AirDAS framework. For instance, users will often enter a comment indicating if they are or are not recording extra information such as mola mola sightings. Again, this information is not recorded in a systematic way, but you can still use swfscAirDAS functions to determine this information.
A comment indicating whether or not something is being recorded will
likely contain “record” somewhere in the comment. Thus, you can use
airdas_comments to extract comments, and then subset for
comments that contain the pattern “record”.
y.comm <- airdas_comments(y.proc)
head(y.comm)
#> Event DateTime Lat Lon OnEffort Trans Bft CCover Jelly
#> 1 C 2015-04-09 12:30:59 39.23550 -123.1563 FALSE <NA> NA NA NA
#> 2 C 2015-04-09 12:30:59 39.23550 -123.1563 FALSE <NA> NA NA NA
#> 11 C 2015-04-09 12:31:48 39.23350 -123.1857 TRUE T1 1 10 0
#> 22 C 2015-04-09 12:34:56 39.23317 -123.2968 TRUE T1 1 10 2
#> 25 C 2015-04-09 12:35:32 39.23317 -123.3178 TRUE T1 1 10 2
#> 37 C 2015-04-09 12:38:00 39.23400 -123.4047 FALSE T1 2 10 0
#> HorizSun VertSun HKR Haze Kelp RedTide AltFt SpKnot ObsL ObsB ObsR Rec
#> 1 NA NA <NA> FALSE FALSE FALSE NA NA <NA> <NA> <NA> <NA>
#> 2 NA NA <NA> FALSE FALSE FALSE NA NA <NA> <NA> <NA> <NA>
#> 11 6 NA n FALSE FALSE FALSE 650 100 aa bb cc dd
#> 22 6 NA r FALSE FALSE TRUE 650 100 aa bb cc dd
#> 25 6 NA r FALSE FALSE TRUE 650 100 aa bb cc dd
#> 37 6 NA n FALSE FALSE FALSE 650 100 aa bb cc dd
#> VLI VLO VB VRI VRO Data1 Data2 Data3 Data4 Data5 Data6 Data7
#> 1 <NA> <NA> <NA> <NA> <NA> Reco rder: dd <NA> <NA> <NA> <NA>
#> 2 <NA> <NA> <NA> <NA> <NA> Not recor ding molas <NA> <NA> <NA>
#> 11 g g g g g 2 cp <NA> <NA> <NA> <NA> <NA> <NA>
#> 22 g g g g g stre aky r ed ti de <NA> <NA> <NA>
#> 25 g g g g g shar k 11f t <NA> <NA> <NA> <NA>
#> 37 g g g g g off effor t to circl e on unide ntifed object
#> EffortDot EventNum file_das line_num file_type
#> 1 FALSE 1 airdas_sample.das 1 turtle
#> 2 FALSE 2 airdas_sample.das 2 turtle
#> 11 TRUE 11 airdas_sample.das 11 turtle
#> 22 TRUE 21 airdas_sample.das 22 turtle
#> 25 TRUE 24 airdas_sample.das 25 turtle
#> 37 FALSE 37 airdas_sample.das 38 turtle
#> comment_str
#> 1 Recorder: dd
#> 2 Not recording molas
#> 11 2 cp
#> 22 streaky red tide
#> 25 shark 11ft
#> 37 off effort to circle on unidentifed object
str_subset(y.comm$comment_str, "record") #Could also use grepl() here
#> [1] " Not recording molas"You can extract data from ‘systematic’ comments, using
airdas_comments_process. This function supports both the
codes of TURTLE/PHOCOENA data, and the text string format of CARETTA
data. Specifically, it includes a comment.format argument
for users with CARETTA data, which allows you to provide a list
specifying how the data was recorded, although the data must be
separated by a delimiter. See below for the comment.format
value for default CARETTA data. This function also by default looks for
codes, e.g. “fb” or “cp”, in TURTLE/PHOCOENA data; see
?airdas_comments_process for more details.
# comment.format for default CARETTA data
comment.format <- list(
n = 5, sep = ";",
type = c("character", "character", "numeric", "numeric", "character")
)
# TURTLE/PHOCOENA comment-data
head(airdas_comments_process(y.proc))
#> Event DateTime Lat Lon OnEffort Trans Bft CCover Jelly
#> 1 C 2015-04-09 12:31:48 39.23350 -123.1857 TRUE T1 1 10 0
#> 2 C 2015-04-09 12:42:01 39.23267 -123.5433 TRUE T1 1 20 0
#> 3 C 2015-04-09 12:42:01 39.23267 -123.5433 TRUE T1 1 20 0
#> HorizSun VertSun HKR Haze Kelp RedTide AltFt SpKnot ObsL ObsB ObsR Rec VLI
#> 1 6 NA n FALSE FALSE FALSE 650 100 aa bb cc dd g
#> 2 6 NA n FALSE FALSE FALSE 650 100 aa bb cc dd g
#> 3 6 NA n FALSE FALSE FALSE 650 100 aa bb cc dd g
#> VLO VB VRI VRO Data1 Data2 Data3 Data4 Data5 Data6 Data7 EffortDot EventNum
#> 1 g g g g 2 cp <NA> <NA> <NA> <NA> <NA> <NA> TRUE 11
#> 2 g g g g fb2s fb1m <NA> <NA> <NA> <NA> <NA> TRUE 45
#> 3 g g g g fb2s fb1m <NA> <NA> <NA> <NA> <NA> TRUE 45
#> file_das line_num file_type comment_str Misc1 Misc2 Value
#> 1 airdas_sample.das 11 turtle 2 cp crab pot <NA> 2
#> 2 airdas_sample.das 46 turtle fb2s fb1m fish ball s 2
#> 3 airdas_sample.das 46 turtle fb2s fb1m fish ball m 1
#> flag_check
#> 1 FALSE
#> 2 FALSE
#> 3 FALSE