Conducting a Systematic Social Observation of Body-Camera Footage: Methodological and Practical Insights

Increased use of video recording technologies such as drones, smartphones, CCTV, and body-worn cameras (BWCs), provides researchers with modes for observing human behavior in their natural settings. Although not originally intended for empirical inquiry, these data sources facilitate a video data analysis (VDA) framework used in the social sciences. BWCs represent an emerging technology within this framework, and BWC footage provides valuable insight into situational dynamics at play during various social phenomena. When combined with systematic social observation (SSO), researchers are well-equipped to unpack social phenomena in a manner that overcomes many challenges of traditional qualitative methodologies. This article incorporates a practical example detailing the application of these techniques in the context of a research project on police use of physical force. We provide a roadmap for researchers and practitioners interested in applying SSOs to a VDA framework which relies on BWC footage as a data source. Throughout the article we describe pertinent challenges associated with this method of inquiry and offer the ways in which such challenges can be navigated.


Systematic social observation, video data analysis, and BWC footage as data
Qualitative methodologies have long been used to better understand people and their behavior (Brunson & Wade, 2019;Copes et al., 2019;Miller, 1998). One such methodology is SSO, the observation of social phenomena in a systematic, replicable manner, involving a means of observation that is independent of the phenomena being observed (Reiss, 1968(Reiss, , 1971. SSO is well suited to situations where all relevant actors and events pertinent to the phenomenon of interest can be observed from start to finish in a limited, well-defined time-period (Mastrofski et al., 2010, p. 228). SSO can be aided by video footage capturing human behaviors and participant interactions of interest (Lindegaard & Bernasco, 2018;Nassauer & Legewie, 2021).
Video data analysis (VDA) "focuses on situational dynamics and behaviors using video or other visual data to understand how people act and interact, and which consequences situational dynamics have for social outcomes" (Nassauer & Legewie,7 of data, is especially important for BWC research given the sensitive nature of the video footage (Drover & Ariel, 2015).
To be clear, police researchers have long needed to gain agency trust to access sensitive data sources, such as use of force reports, or observe officers in field settings, such as during ride alongs. However, BWC footage differs from interviews or participant observations in that officers do not have the ability to temper their recorded responses. Additionally, during traditional fieldwork, researchers may unknowingly engage with officers who are hand-selected by senior police administrators. Such officers may not accurately represent the department. That said to gain access to unedited and unfiltered police behavior as captured by BWC footage, a level of trust must exist between researchers and police agencies that transcends traditional research methods and data sources. NPD has stringent rules and regulations regarding access and release of BWC footage, which makes the acquisition and analysis of BWC footage difficult above and beyond that of traditional research partnerships. In addition, researchers may need to secure permission from multiple parties before accessing BWC footage, given that it is often used as evidence by prosecutor offices and police oversight boards.
In our case, we experienced several practical challenges related to securing the necessary permissions to conduct the study and to access the necessary data.
Fortunately for us, an embedded criminologist within the NPD-who had direct working relationships with high level officials in both the County Prosecutor and State Attorney General offices-was able to meet with gatekeepers directly to discuss the scope and aims of the research study. In the end, the County Prosecutor provided written permission for the study, stipulating that researchers could view footage for any event that was closed (i.e., resolved in the courts by plea, trial, or dismissal) and did not involve juvenile suspects or victims. In short, video footage of sensitive events may not be accessible at the level necessary for VDA (see Nassauer & Legewie, 2019) outside of the type of partnership we enjoyed with NPD.
It is imperative to understand the population of interest involved in the VDA research, especially when reviewing potentially sensitive footage. In our case, police culture has taught officers to remain suspicious of the public, with whom they must be able to maintain a power-imbalance while confronting the dangers of their occupational environment (Sierra-Arévalo, 2019;Skolnick, 1994;Reuss-Iani, 1983). In our context, officers may view the danger as "outsiders"-or researchers-investigating footage of their (or their colleagues') actions. Previous research has found that gaining access to police officers and maintaining rapport has proven difficult, even after access has been granted by the organization-as officers may be suspicious of the motives of supervisors (Weatheritt, 1986;Marks, 2004;Pogrebin, 2010;Punch, 2010). This is especially the case in the current context of the NPD, which is under an active consent decree from the U.S. Department of Justice (United States of America v. City of Newark, 2016).
According to Collins' (2004) Interactional Ritual Chains Theory, barriers that exclude outsiders help to produce group solidarity. In the case of officers, a history of ritual participation (i.e., walking the beat and responding to calls for service together) results in cultural capital (Collins & Hanneman, 1998) that allows for stronger relationships and bonds to form among those that now share a set of shared experiences. Outsiders do not share such experiences and therefore have no "cultural membership capital" (Collins & Hanneman, 1998, p. 219).
Due to the potential for access barriers, the research team ensured a mutual understanding with NPD officials that the focus of the research was not to identify any possible officer misconduct, but rather to better understand the events that lead to officer use of force. We approached data collection with the assumption that NPD internal affairs processes confirmed the legality of officer actions on video. These conversations occurred during both the project planning phase and prior to the beginning of data collection. The NPD was mindful of distrust concerns and provided the research team a workstation in the Real-Time Crime Center, which was staffed by police officers and analysts from various police agencies across New Jersey, including the NPD. NPD officials felt the research staff being constantly visible to rank-and-file personnel would generate a level of trust towards the research team. The alternative was placing the research team within the Policy and Planning Unit, which also housed the team of officers responsible for overseeing the implementation of the U.S. Department of Justice Consent Decree against the NPD. The NPD felt such an arrangement would have created a visual optic aligning the research project with the consent decree, which may have led officers to the view the research team suspiciously and be less supportive. Researchers have previously documented such negative consequences from too closely aligning projects with units primarily tasked with providing an accountability function (Sousa et al., 2016).

Concern #2: Perspective bias and naïve realism
As noted by Hoeben, Steenbeek, and Pauwels (2018), a major shortcoming of SSO research is the lack of attention to potential sources of observer bias. Multiple viewings of video recordings allow researchers to pay closer attention to different details of the observation on each viewing, and video recordings allow for the addition of new research team members who may not have been involved during the initial data collection phase. Wang and Lien (2013) refer to this as retrievability and availability.
Video data can easily be retrieved and made available to additional researchers, creating opportunities for validation or reinterpretation of the data.
In traditional participant observation research, researchers have finite time to scan an entire interaction, record relevant variables, and make sense of them in the moment.
In contrast, the use of BWC footage affords researchers' unlimited time. During the first viewing observers may focus on one aspect of the interaction (e.g., participant demeanor), but during the second viewing they may turn their attention to another aspect (e.g., participant behavior). Additionally, multiple viewings provide researchers the opportunity to consider the totality of the circumstances (i.e., situational factors) that may have influenced how the event unfolded, rather than only those that are observable in the field. By affording researchers more opportunity to add interpretation to the underlying context and setting, thick description is made possible (Geertz, 1973;Ponterotto, 2006).
The concept of naïve realism suggests that anyone who watches the same video evidence will reach a similar conclusion (Morrison, 2017). BWCs footage is expected to 'speak for itself'; however, Boivin and colleagues (2020) indicate that perspective bias has the potential to skew how the viewer interprets such footage. Within the context of our study, this meant that BWC footage was captured from the officer's perspective (i.e., first-person), limiting our ability to be aware of significant characteristics that were not in the BWCs field of vision. However, because NPD requires all officers to wear BWCs, we often had various vantage points of a single event. Additionally, BWC microphones are located close to the user's mouth, capturing speech that could be missed by other video recording technologies (e.g., CCTV) or participant observations (e.g., physically being too far away to hear). Thus, we were able to record each occurrence of variables related to officer and suspect speech, including officer commands, if an officer notified the suspect of the reason for their response, and if either the suspect or officer used antagonistic language. VDA generally, and BWCs specifically, advance the researcher's ability to accurately capture aspects of the interaction related to speech including content and inflection (e.g., calm command given by an officer versus a shouting command). This ultimately enabled us to increase the number of perspectives and capture relevant contextual and situational variables that would likely have been missed during traditional participant observations. Nassauer and Legewie (2021) argue that video coding should involve objective measurement of content contained across three analytical dimensions: 1. Facial expressions and posture; 2. Interactions; and 3. Context. While not all aspects are relevant to every study, measuring variables within analytical dimensions maximizes the empirical rigor of the research. Our study focuses on interactions and context observed during police-citizen encounters. Interactions involve movement and actions, verbal communication, and gestures (i.e., nonverbal codes) amongst participants.
In order to capture in our codes the various analytical dimensions, the three authors met for a five-day data retreat at NPD headquarters at the project onset, meeting about six hours each day. Such in-depth footage reviews have played a central role in focusing the research in prior studies (Nassauer & Legewie, 2019). Drawing upon experiences of the investigators in designing an SSO of CCTV footage (Piza & Sytsma, 2016), we had four broad goals for the data retreat.
First, through viewing BWC footage we aimed to identify the precise analytical dimensions, or the content of interest to our research questions (Nassauer & Legewie, 2021). We hoped to gain an understanding of the nuances of police-citizen encounters through the analytical dimensions, including the different temporal phases that comprise the encounters, the various types of participants involved, and contextual factors that seem to influence participant behavior. Second, we sought to create codebooks explaining variables that were to be collected during the SSO based upon knowledge gained during the video review. Third, we sought to code ten full incidents together to determine how to best interpret and code the activities seen on screen until general saturation of potential uncertainties was met. Lastly, the lead author, tasked with leading the SSO for the remainder of the project, spent a full day coding incidents-with the investigators on hand to further discuss coding decisions.
For our study, the data retreat proved instrumental in considering the interaction and context analytic dimensions as they related to correctly measuring the final level of force used along a continuum, from least to most severe. Types of force include soft empty hand control (i.e., grabs or holds), hard empty hand control (i.e., punches, kicks, or body slams), use of chemical device, blunt impact, threat of lethal force (i.e., pointing a gun), and lethal force. The force typologies were informed by the use of force continuum, as presented by the National Institute of Justice (2009). The latter categories were easily coded by events that took place (e.g., the officer used chemical spray or un-holstered and pointed their gun); however, the context of the situation and the interactions surrounding them were of particular importance when distinguishing soft versus hard empty hand controls. For example, let us consider a case in which an officer uses force on a fleeing suspect. There are multiple coding outcomes that could result from the contextual factors. First, the officer may use soft force to grab and gain control of the individual, for which soft force would be recorded. Second, if both the officer and suspect subsequently lost their balance because of the officer's use of soft force (e.g., grabbing a body part), only soft force would be recorded. Third, if the officer instead grabbed the suspect and then forcefully threw them to the ground, soft (grabbing the suspect) and hard force (throwing the suspect to the ground) would both be recorded. The difference between the second and third scenarios is the action of the officer that caused the suspect to end up on the ground-momentum in the former, and a use of hard force in the latter. Fourth, if the context once again remained the same, but this time the officer grabbed and threw the suspect to the ground in one motion, then only hard force would be captured-as there was no separation of actions and no context that would suggest the officer did not intend for the suspect to wind up on the ground (e.g., the officer and suspect falling to the ground because of momentum). Through this inductive process, we were able to utilize grounded theory methods to create analytic codes and variables that were informed from the data, rather than preconceived notions (Glaser & Strauss, 1967;Strauss & Corbin, 1997;Charmaz, 2014).
In Finlay's (2002, p. 217) discussion of reflexivity as intersubjective reflection she notes that researchers must, "explore how conversation or text affects us and to reflect on what we bring to it ourselves." The data retreat described above, coupled with regular team meetings provided the added benefit of allowing the authors of varying positionalities (i.e., gender, race, age, geographic location) to engage in a reflexive process, thereby reducing the likelihood that research activities and results are subject to any one individual's ontological or epistemological assumptions (Berger, 2015). In other words, close collaboration within the research team can foster accountability and adherence to principles of cultural relativism. With that said, as is the case with all types of research, when conducting VDA it is vitally important to acknowledge that observations are never truly independent of the observer.

Concern #3: Observer and participant recall error
When compared to in-person observations with no secondary source of data verification (e.g., another participant observer to review the first observer's work), video recordings offer enhanced opportunities to ensure interrater reliability and foster a team-approach to SSO (Lindegaard & Bernasco, 2018). Video recordings provide researchers the benefit of permanent visual records amenable to later coding and reinterpretation based on emergent insights (Sampson & Raudenbush, 1999, p. 605). Video recordings are also less vulnerable than other observational methods to recall error by observers (Mastrofski et al., 2010, p. 232;Makin et al., 2020).
Previous research suggests that study participants go through a series of stages when responding to questions that involve the retrieval of information needed to construct the answer (Beckett et al., 2001;Tourangeau, 1984;Strack & Martin, 1987). It is during this stage of retrieval that recall error can occur for reasons including: some events are not processed and stored in memory; the event is not regularly talked about, thus making it difficult to remember specific details; and events may not be accurately reconstructed (Sudman et al., 1996). Retrospective accounting of events is also subject to errors related to date and time. These include assigning an incorrect date to an event which is more recent (e.g., telescoping) or older (e.g., time expansion) than reported, and incorrectly estimating the frequency of events in a given period

Navigating observer and participant recall error
The BWC vendor used by NPD does not enable the continuous delivery of a video over the internet to a remote user (i.e., streaming). This required all videos to be downloaded from a central server to an external hard drive that remained in a securely locked location within headquarters for the duration of the project. The inability to view footage remotely precluded the participation of all authors in the SSO. For this reason, the lead author was designated as the single coder for the SSO and was trained for such during the data retreat. While providing the ability to involve multiple coders is a primary benefit of video footage, prior SSOs have included a single coder when appropriate (e.g., Porter et al., 2018;Sytsma et al., 2020).
However, to maintain a team approach to the coding (Lindegaard & Bernasco, 2018) the co-principal investigators were consulted throughout the process. The research team maintained a shared document in which the primary author raised 27 questions about particular incidents over the coding period. As needed, the primary author provided written descriptions of events and identified specific coding questions. The coprincipal investigators then engaged in conference calls to discuss issues and reach consensus on the appropriate codes (see Campbell et al., 2013). The principal investigator would then meet with the lead author at the NPD to review those incidents which raised questions and discuss the recommended coding decisions. On the occasion that viewing the footage provided additional insight not considered during the initial discussion, the co-principal investigators would set up an additional conference call to revisit the issue. This process was repeated as necessary until a final decision was reached. The result of these discussions also led to the identification of several variables that emerged after watching footage-again highlighting the inductive process and grounded theory approach employed.

Concern #4: Limited study procedures
There are various coding strategies employed by scholars (e.g., structured, open, axial, thematic) (Saldana, 2012); however, qualitative data is often limited in terms of analytical frameworks, with strategies typically residing in either analytic induction (Robinson, 1951) or grounded theory (Martin & Turner, 1986;Glaser & Strauss, 1967). into thousands of brief temporal intervals. These data will allow us to apply a variety of complex statistical techniques, without the concern for statistical power that is typical in traditional qualitative research. Further, when employing VDA, the central coder may adopt a role similar to that of the 'complete observer' found in traditional ethnographic work. Such intertwining of qualitative and quantitative methods allows researchers to situate statistical analysis within observed situational context, unhindered by potential observation bias. For instance, in one analysis we conducted as part of our ongoing project we created a 'script' of use of force escalation (i.e., from soft empty hand control to a higher level of force) (see Sytsma et al., 2021). This complex phenomenon (force escalation) was observed as it unfolded by a 'complete observer' and individual and environmental elements of the event were recorded in detail based on the concept of 'thick description.' Detailed characteristics were then coded and transformed into a wide range of variables amenable to statistical analysis.
Conjunctive analysis of case configurations was used to produce scripts. Finally, probabilistic causality can be established should researchers utilize advanced statistical modeling, or experimental or quasi-experimental designs. The ability to infer causality in our project has helped develop evidence-based policy recommendations.

Data protocol, analysis, and sampling frame
In creating the data protocol, our first step was the operationalization of the unit of analysis for the research. Units of analysis in SSO have been developed using three broad approaches as described by Mastrofski et al. (2010). The first approach is the use of a concrete time-period, observing what happens in a discrete segment (e.g., behaviors over a 15-mintue or 1-hour period) (Frank et al., 1997;Weisburd et al., 2006). The second approach tracks a behavior as the unit, measuring its sequence over time, such as behavioral transactions between officers and citizens (Sykes & Whitney, 1969). The third approach is to create a social construct of an event, such as a face-toface encounter between a police office and citizen (Reiss, 1971), or a public meeting between police and residents of a neighborhood (Skogan, 2006).
At the outset we anticipated organizing the analysis around police-citizen encounters, which we tentatively defined as the moment an officer came into contact with a citizen through the moment that the encounter concluded. This strategy is congruous with the behavioral unit approach of SSO. However, our initial review of the footage demonstrated that operationalizing police-citizen encounters was more complex than anticipated. The true 'start' and 'end' points of an encounter are not always self-evident, making our final unit of analysis more of a social construct than a behavior unit clearly defined by concrete start/end points. Our challenge was not unique, with measurement of the precise time frame of police-citizen encounters presenting complications in prior SSOs (Mastrofski et al., 2010, pp. 229-230). Researchers may be interested in the totality of the event (i.e., what occurs after the arrest and force concludes) and will need to select an appropriate endpoint of the unit of analysis for their research questions. However, for the purpose of our study, we were only interested in the events that took place during an arrest as they are likely to bring a higher degree of risk of use of force compared to other types of encounters, and arrests impact millions of people per year in America-over 10,000,000 arrests by law enforcement occurred in 2019 (FBI, 2019).
After considerable discussion we operationalized the event start time as the point in the video at which the officers are first visibly seen interacting with any involved parties (e.g., suspects, bystanders, or victims). This definition is similar to that of by Makin and colleagues (2020). We felt that the 'end' of the event should represent the point in time at which officer use of force was no longer an option. We assumed this would be obvious, such as when an officer applied handcuffs on a suspect or placed the suspect in the back of the patrol car in advance of arrest processing. However, we found that such events did not always represent the end of the encounter. For example, in one case, a suspect was placed under arrest but required medical services. During the examination by paramedics in an ambulance on scene, the suspect escaped and led police on a foot pursuit. The result was hard force used by an officer to tackle the suspect to the ground to prevent their escape. The research team grappled with whether the end point should be the initial arrest or the point at which the suspect was After deciding upon an operationalization of the unit of analysis, we moved towards consideration of the sampling frame (Mastrofski et al., 2010). We had access to over 2 years of data but did not have the resources necessary to code all of the available footage. After considering the workload associated with coding individual events during our preliminary data review, as well as the sample sizes of similar studies (Willits & Makin, 2018), we determined that a sample in the low 200s could be coded during the project period. The total sample size selected meant we needed to include 100 or so use of force events at the outset of the study (given that each use of force event would ultimately be matched to a non-force control case for the final analysis). Four cases were under active investigation by the NPD internal affairs units and were inaccessible to us. Accounting for these excluded incidents, 117 use of force events were reviewed during the SSO.

Variable coding instrument
The use of BWC footage allows for the capture of relevant situational variables that may impact the outcome, not just at one particular point, but across a set time span.
Further, our data review uncovered the fact that our unit of analysis, despite being described as police-citizen encounters, involved more than just the responding officer and the citizen being detained. Bystanders were commonly intimately involved in the encounter, with many directly injecting themselves into the situation. Multiple officers were on scene for most events, with officers greatly differing in terms of their level of interaction with the suspect and general involvement in the event. These observations are indicative of the fluid nature of police use of force as observed in prior research (Terrill, 2005;Todak, 2019) and indicate that transactional aspects of the encounter need to be accounted for within the coding protocol alongside fixed variables (e.g., the setting, time of day, etc.).
To best capture the nature of all variables under study, we created two distinct code books and accompanying databases (the codebooks are available at http://bit.ly/ BWCcodebook). The static database includes those variables that remained stable throughout the course of the incident and thus were captured in a cross-sectional manner. These variables include factors such as the characteristics of the event, demographics of the actors involved in the event, and the physical setting characteristics. The dynamic variable database includes those variables which had multiple associated time-points and models a police-citizen encounter longitudinally.
Variables are grouped thematically by time markers associated with factors such as the speech behavior of involved actors, dynamic characteristics of the event (e.g., changes in severity of force use against civilian or suspect compliance to officer commands), and changes in number and type of actors on scene. The collection of a wide range of related variables follows the functional principle in anthropology, which purports that understanding a phenomenon requires unpacking the context in which it occurs to understand the reasoning behind a certain behavior (Outhwaite & Turner, 2007;Malinowski, 1922).

Additional challenges of using BWC footage as a data source
The use of BWC footage as a data source for conducting SSOs can overcome several prevalent issues in qualitative research. However, there are also new issues that can arise when analyzing BWC footage, as outlined below.

BWC technology and data storage
Technical aspects of BWCs can present difficulties during the coding process. The first 30 seconds of audio are muted in each video, making all communications between officers, dispatchers, suspects or involved parties inaudible. While we understand this is done for officer privacy considerations, these 30 seconds may have included valuable discussions regarding the incident environment, the call type, dispatch information, and other variables that may have influenced officer behavior.
Additionally, the BWC is typically affixed to an officer's uniform via a magnetic clip on the inside of their shirt. BWCs often become detached during physical struggles with suspects, leaving the coder to listen to the multiple voices on the footage to disentangle who was speaking and what was being said, all the while sometimes missing any physical contact between those involved in the interaction. We believe this shows that both police and BWC vendors should experiment with BWC placement (e.g., chest, shoulder, etc.) to determine if some are more resistant to detachment than others.

Nature of recording system
Certain BWC vendor's software does not enable video streaming, resulting in the need for all videos to be downloaded to an external hard drive before being viewed by the research team. In our study, which included 121 use of force events, an average of approximately 3 officers per incident wore a BWC, multiplying the number of videos we needed to download. It took roughly 91 hours to download all videos associated with use of force incidents included in our analysis given the length and high-definition quality of the videos. A BWC system with streaming capabilities would have eliminated this downloading process, and thus would have been more time efficient. Researchers wishing to use BWC footage as a data source should be aware of the upfront time requirements. Additionally, should researchers encounter a similar recording system that necessitates the storage of these data on external hard drives, they will also need to address security concerns as videos contain sensitive content, inclusive of homicide scenes, domestic violence, and individuals suffering from mental illness. This means leaving the hard drives in a securely locked location. Relying on external data storage is admittedly inefficient and can be avoided if agencies use BWC platforms with video streaming capabilities (e.g., see Pollock et al., 2020). This is especially the case given that qualitative software packages, such as NVivo, support importing of steaming video from websites such as YouTube. However, we are unaware if such import functions can operate on BWC vendor web platforms. In the event such videos are in a propriety format unable to be imported into the software (a situation the co-principal investigators previously experienced; see Piza & Sytsma, 2016) researchers would need to download videos in a similar manner as our project if they wish to use coding software.

Ethical considerations
As previously noted, BWCs may capture sensitive information such as suspects confidential information (e.g., home addresses, personal identifiers) or officers reviewing personal documents (e.g., government identification, vehicle documentation). Nassauer (2018, 2021) highlight several key ethical considerations when using video data for research purposes. While citizens interacting with NPD officers are to be notified that they are being recorded, they are not necessarily agreeing to have the recording used for research purposes (i.e., lack of informed consent). "Contextual integrity" (Nissenbaum, 2004) refers to the notion that using data that was created in one context (e.g., police-citizen interactions) for another (e.g., academic research) violates people's rights regarding consent and dissemination of personal information. However, the principle of beneficence suggests that a research study's benefits should outweigh its risks. In our study, this meant unpacking the situational and environmental dynamics that influenced police use of force, while not recording personal identifiers of any citizens or officers involved. Yet, researchers must grapple with how to ethically address concerns related to anonymity and confidentiality, as well as the potential viewing of incidents that contain victims of criminal activity or personal tragedies (i.e., domestic violence, homicide, drug and alcohol addiction, mental illness). We advise VDA researchers to avoid recording information that could be at all identifying. The extent to which a variable is identifying varies by population of interest, organizational size, and sample size. For instance, collecting information on officer rank in a small police organization could result in easy identification of officer when there are few officers at a particular rank.

Implications
SSO has long relied on the ability of the researcher to record actors' behaviors without the luxury of repeat viewing. Additionally, the Hawthorne effect has the potential to skew the behavior of subjects given subject awareness of being observed.
Ethnographers, who also rely on the ability to build trust with their participants throughout prolonged observation, are not free from the potential negative effects of participant observation (Carroll & Mesman, 2011). The use of previously recoded video data, such as BWC footage, does not suffer from these limitations. The implication for qualitative research is a resulting environment whereby behavior is not influenced by the foreknowledge of continued observation, allowing researchers to assess behavior as it would naturally occur.
As previously noted, the use of VDA facilitates the use of mixed methods through the static and longitudinal databases that quantify variables of interest and allow for temporal measurement. Quantitative analysis that leverages the collected data will