The Replication Crisis and Using the Replicate + Extend Method to Study the Perception of Virtual Objects

Abstract:  

The replication crisis is the term coined to describe recent significant failures to replicate empirical findings in a number of fields, including medicine and psychology. In many cases, over 50% of previously reported results could not be replicated. This fact has shaken the foundations of these fields: Can empirical results really be believed?

In this talk, I will describe the replication crisis, and will particularly focus on its effects in the area of perceptual psychology. Over the past 25 years, I and others have adapted empirical methods from perceptual psychology to ask questions about how virtual objects are perceived in augmented reality (AR) and virtual reality (VR). Therefore, replicating the results of empirical methods is very relevant to the development of AR and VR technology.

I will discuss how my lab helped pioneer the replicate + extend method for applying these empirical methods to these questions, with a particular focus on measuring the perceived location of virtual objects in augmented reality. I will also discuss my lab’s continued use of the method for asking several more recent perceptual questions.

Finally, I will discuss how a replication crisis tutorial and related workshop that I organized in my primary professional conference series has had some impact, hopefully positive, in the way my community reviews replication papers. I will also mention an extension of the tutorial theme to the larger culture of scholarly publishing, and how these ideas might serve to promote more credible and trustworthy research.

Bio

J. Edward Swan II is the Interim Associate Dean of Research for the Bagley College of Engineering at Mississippi State University, and the Virginia Carron Eiland and Brent Eiland Professor of Computer Science and Engineering. In 2017 and 2018, he served as Interim Department Head. He holds a B.S. (1989) degree in computer science from Auburn University and M.S. (1992) and Ph.D. (1997) degrees in computer science from Ohio State University, where he studied computer graphics and human-computer interaction. Before joining Mississippi State University in 2004, he spent seven years as a scientist at the Naval Research Laboratory in Washington, D.C.

Dr. Swan’s research has centered on the topics of augmented and virtual reality, perception, data science, empirical methods, human-computer interaction, human factors, and visualization. Currently, he is studying the perception and technology required to give virtual objects definite spatial locations, including depth and layout perception and depth presentation methods. He is also studying efficient data science tools. His research has been funded by the National Science Foundation, the Department of Defense, the National Aeronautics and Space Administration, the Naval Research Laboratory, and the Office of Naval Research.

Dr. Swan is a member of ACM, IEEE, and the IEEE Computer Society. He has served many roles in the Visualization and Graphics Technical Communities (VGTC) of IEEE Virtual Reality (VR), IEEE International Symposium on Mixed and Augmented Reality (ISMAR), and IEEE Visualization. He is currently the chair of the IEEE VR steering committee. Previously, he served as one of the general chairs of VR 2021 and VR 2020, as well as a program chair for ISMAR 2017, ISMAR 2016, VR 2015, and VR 2014, and he was a member of the ISMAR steering committee. His scholarship and service have been recently recognized by induction into the 2024 Class of the VGTC Virtual Reality Academy, the 2023 VGTC Virtual Reality Service Award, and the 2022 ISMAR Impact Paper Award.