Organizations as Brains and Lisa Moments

Chapter for in Images of Organizations by Morgan focuses on oranizations as brains, learning organizations. There were several things in this chapter that I really enjoyed. For the sake of brevity I'll just focus on two.

First of which was the idea expressed on page 108 of requisite variety meaning "the internal diversity of any self-regulating system must match the variety and complexity of its environment." This seems to suggest that extremely complex environments and organizations will have more internal diversity and variety. 

An idea which is counter to the reliance of bureaucracy and red-tape for mechanistic organizations, where the workers had firm rules and guidelines to follow to help maintain order and interchangeability. 

Secondly there was the idea of error elimination on page 82. I like the idea that we don't pick things up, rather we fail to not pick them up and by failing to not pick it up actually pick the object up. This was not something I had considered as fully as I had once I read it. Both of these tend to, it seems to me, provide opportunities for organizations that behave like brains to find ways to self-organize or to learn how to learn. 

By having ever-growing diversity of opinion and variety of working and leadership styles, an organization led by self-organization and learning can help to find the best possible route to success. By failing, new procedures and policies can be created that help others not to fail, and by not failing succeed. I regularly cite to my teams how although Steve Jobs has been well known for his successes with Apple, in many ways his success came by way of his failures. "Lisa stands forgotten in the shadow of the Macintosh, but its abject failure (the first model only sold 10,000 units) taught great lessons–especially to Lisa’s original project lead, Steve Jobs, who was kicked off and put on another project: the Macintosh," (Lumb, 2015). It's through failures that we, even as organizations, can figure out what best not to do again.

I think back on my life and the major "Lisa" moment for me was when I wanted to create an app called "PopQuiz Password" (see here). I know, horrible name, but it was a working title. Basically, it was designed for parents to ensure that their kiddos got some learning in while they gamed. It was a project I developed when getting my MBA back in 2004. Before a kiddo could play a computer game they had to complete a grade appropriate quiz. They were rewarded time to play based on how they scored. 100%? 30 minutes of play time. 50%? Then just 5 or 10 minutes of game time. Also we created modules like Texas History, or Boy Scouts and Bible History, to try and make it appealing to parents and more relevant for kids. Additionally we came up with the idea of creating teacher interactions, whereby teachers could upload their own class specific questions and have them load onto the kiddos quizzes.

We started a proof of concept, but then the project died off. There were a multitude of reasons for it dying off, but I've always wondered if that was my Lisa moment. What if I had created PopQuiz Password and it had been a rousing failure like Lisa? What would I have learned? How might I have developed other microlearning opportunities before microlearning took off? 

I am far to safety oriented to have a Lisa moment. Any whiff of failure and I'll look for another option instead of face the failure. Maybe Lisa moments are what entrepreneurs need to be successful ultimately?

References

Lumb, D. (2015, January 22). Remember Apple’s Lisa, the computer that cost Steve Jobs A gig? Kevin Costner does! Fastcompany.Com; Fast Company. https://www.fastcompany.com/3041272/remember-apples-lisa-the-computer-that-cost-steve-jobs-a-gig-kevin-costner-d (Links to an external site.)

Morgan, G. (2006). Images of Organization (Updated Edition). Thousand Oaks, CA: Sage Publications.

Innovation is in my Genes

I am the Vice President of Innovation at work. A strange title to be sure, but it’s supposed to help me stay inspired to think outside the box. I actually have a tendency to do this. I like to think up new things and invent futures in my mind. Anyone who knows me will have heard me spout off about this. 

First there was RadioBoox, the idea was to have a serial on the radio that people could listen to. Books on tape, but via the radio. I thought it would be like soap operas but for people who drive. Something to look forward to each day. Sadly, it was an idea of the 90’s and the internet, tech boom killed it quickly. 

DRAFT IRB Worksheet!

IRB ADMINISTRATIVE INFO 

Form Creator:  Richard Hannah

Title of project: Case Study of PO Fit Used by Contractor Organizations in the Energy Industry to Determine How it Affects New Employee Job Satisfaction and Retention

Principle Investigator: Dr. Kerry Priest 

Responsible Graduate Student: Richard Hannah

Project Classification: Other - Pilot Project for EDLEA 938 with intent to make publishable

Short Form Criteria:  None of the above

Non-KSU Collaborators? No

Funding Source: N/A

Exempt:  No 

STUDY INFO 

Non-technical synopsis:

This case study will investigate through interviews the lived experiences of several new employees hired by contracting organizations in the energy industry. The intent is to determine from their experiences how Person-Environment (PE) fit, Person Organization (PO) Fit, and Values Fit theory was used by them and their hiring organizations and how that might have influenced their feeling of job satisfaction and their desire to remain with the organizations and working in the energy industry.  

Background:

PE fit theory although relatively new has become more well-developed and mature since its inception in 1974. PE fit theory or the idea that finding a value's congruence between employees and the organization and environment with which they are employed has been studied for decades to determine to what degree PE fit or other subsidiary concepts and theories can help decrease employee turnover, delay worker burnout, encourage employee and worker engagement, and lead to great job satisfaction. The contractor organizations that support the energy industry suffer from high turnover and challenges with hiring workers. The seasonal nature of the jobs, combined with the vicariousness of the job locations and the hazardous work environment, make staffing a continual and arduous task. This study will help determine the challenges new employees face when they join a contracting agency, how they deal with those challenges over the course of their first three months, and how the concepts of “fit” theory apply to these employees and the contracting organizations. 

Fit theory is the idea that the values, traits, and characteristics of an employee, when matched with an organization with similar values, traits, or characteristics, will increase the possibility for greater job satisfaction and long-term employment and growth. This has also been called, “values fit” or PE fit. The subsidiary and tangential concepts and theories that PE fit includes are PO fit, Person Group (PG) fit, Person Supervisor (PS) fit, Person Vocation (PV) fit, and Person Job (PJ) fit. Two concepts that are a part of PJ fit include Demands Ability (DA) fit, and Needs Supply (NS) fit. All of these have been studied to determine the relationship they might have with the employee's ability to interact with their organizations. Many studies have found that there are a multitude of relationships between values congruence, PO fit, and the subsidiary fit concepts and job satisfaction, organization commitment, and employees’ intention to remain at their organizations. This idea of value congruence aligns with some of the first work in PO and PE fit theory according which finds that organizations are characterized by particular types of climates, and shared values which play an intricate and critical role in the attraction and selection of individuals who are hired into those organizations. This becomes the cornerstone for attraction-selection-attrition (ASA) theory, which provided a foundation for PO fit theories. Accordingly, the concept of fit, and more specifically a lack of fit, results in decreased performance, a decrease in job satisfaction, and an increase in employee stress. Finding ways to decrease these variables and find better PO fit, or more deliberate use of values fit during hiring could lead to less stress, greater job satisfaction, and other positive benefits to contractor organizations. These values fit theories focus on include both self-determination theory (SDT) and Maslow's hierarchy of needs. This is due to the fact that individuals will perceive high employment quality if their needs for safety, income, security, and skills development are more fulfilled. 

Project/study description:

The specific purpose of this research is to explore in what ways industrial workers are motivated to remain at their organizations after several weeks of work with the organization as compared to their expectations about the organization and their career goals. A secondary purpose is to help determine if the worker's expectations for the organization and their goals for themselves and their careers align with the organization’s goals for employees and career development. This study which is qualitative in nature, follows a case study format, meaning it consists of several interviews with multiple participants over a period of time. The research questions this study hopes to answer include: 

• How do the participants express their career goals and expectations for their positions within the organizations? 
• How do the participants express their lived experiences about their first two months of working within their new organizations 

This study which is qualitative in nature, follows a case study format, meaning it consists of several interviews with multiple participants over a period of time. Focused on phenomenology, this study methodology will help in the exploration of a phenomenon within the particular context of contractors' first few months of work, through various interview sources, and it will undertake the exploration through a variety of lenses in order to reveal multiple facets of the phenomenon.  

Questions that participants will answer will focus on their lived experience over the first three months after being hired by organizations that support the energy industry. These questions will focus on the participants: (a) interactions with their new colleagues, (b) their work lives and training, (c) personal lives as affected by their jobs, (d) job satisfaction, and (e) desire to remain with the organization. All interviews will take place via phone and will be recorded and transcribed. Four interviews with the participants will take place at one-month intervals over their first three months of working at their new organizations. Participants will be volunteers and will be found through a new hire course offered at the safety council and will sign a consent form spelling out their obligations, their ability to remove themselves from the study, and the fact that a $50 dollar gift certificate for those who complete all of the interviews. 

Objective:

This study which is qualitative in nature, follows a case study format, meaning it consists of several interviews with multiple participants over a period of time. The research questions this study hopes to answer include: 

• How do the participants express their career goals and expectations for their positions within the organizations? 
• How do the participants express their lived experiences about their first two months of working within their new organizations? 

DESIGN & PROCEDURES

Site of the Research: 

Volunteer participants will be selected from HASC a safety council in Pasadena, Texas. Interviews will take place via phone calls to the participant's phones from HASC.

List all variables to be studied: 

The variable that will be studied includes the lived experiences of each participant including how they feel the organization’s values meet their own and how that contributes, or not, to their (a) interactions with their new colleagues, (b) their work lives and training, (c) personal lives as affected by their jobs, (d) job satisfaction, and (e) desire to remain with the organization. 

Data collection methods:

This study which is qualitative in nature, follows a case study format, meaning it consists of several interviews with multiple participants over a period of time. Focused on phenomenology, this study methodology will help in the exploration of a phenomenon within the particular context of contractors' first few months of work, through various interview sources, and it will undertake the exploration through a variety of lenses in order to reveal multiple facets of the phenomenon.  

Questions that participants will answer will focus on their lived experience over the first three months after being hired by organizations that support the energy industry. These questions will focus on the participants: (a) interactions with their new colleagues, (b) their work lives and training, (c) personal lives as affected by their jobs, (d) job satisfaction, and (e) desire to remain with the organization. All interviews will take place via phone and will be recorded and transcribed. Four interviews with the participants will take place at one-month intervals over their first three months of working at their new organizations. Participants will be volunteers and will be found through a new hire course offered at the safety council and will sign a consent form spelling out their obligations, their ability to remove themselves from the study, and the fact that a $50 dollar gift certificate for those who complete all of the interviews.

Supporting Documents for Data Collection

List any factors that might lead to the subject dropping out or withdrawing from the study. 

Volunteer participants will be advised that they can drop out of the study at any time. I forsee that over 50% of the study participants will back out of the study due to various factors including a lack of time and attention. Some may begin the study interviews and back out after one or two interviews for the same reason. I understand that starting a new job is stressful and what a volunteer participant might think they have time for at the beginning of their job, when they volunteer at the council, they might not have time for weeks later. 

Biological Sample? No

Debriefing procedure for participants: 

Thank you for sharing your experiences with me. The findings from this research and your interviews will help to better understand the lived experiences and processes of being hired and beginning work for contractors in the energy industry.  

RESEARCH SUBJECTS

Source of participant population:

I work for HASC a safety council in the Houston area, in Pasadena, Texas. We offer required federally mandated, safety compliance training on behalf of plant owners in the energy industry. Participants will be told about the study during this training course and asked if they would like to participate. I will stress that participation will be anonymous and will not affect their course completion or their job status. HASC provides two classes of this course daily which includes over 50 potential participants per class. 

Number of participants:

The number of initial volunteers we hope to collect information for is 15-20. Based on other research studies similar in nature to this one, we hope to have five to seven participants complete the entire case study. We feel that starting with 15-20 volunteers will ensure a final participant sample size of five to seven participants.  

Inclusion criteria:

We will ask that all volunteer participants be new to the industry. The fact that the industry requires an initial basic safety compliance class should help ensure that any volunteers from the class will be new to the industry, but this is the one stipulation that we will ask for. This is needed to ensure that previous experiences in the industry don’t color or bias the answers of the participants. 

Exclusion Criteria:

Any volunteers who are not new to the industry will be excluded from the study whether that is found during the volunteer process, or later during the interviews.  Additionally, any participants who leave their organizations during the study will complete one final interview a month following their leaving the organization, and then will be excluded from further study. 

How will subjects be identified?

Participants will be contractors who are new to working in the energy industry. As HASC provides contractor training for new contractors, there are many potential volunteers who attend classes at HASC on a daily basis. 

How will subjects be recruited?

Participants will be volunteers and will be found through a new hire course offered at the safety council and will sign a consent form spelling out their obligations, their ability to remove themselves from the study, and the fact that a $50 dollar gift certificate for those who complete all of the interviews. This will be facilitated by me and will take place during a break for the basic safety training course that HASC provides. 

How will subjects be enrolled in the study?

Participants will complete the HASC consent form. 

Describe any follow-up recruitment procedures:

N/A

RISK-PROTECTION-BENEFITS

Risk for Subjects: 

N/A

In your opinion does it involve more than minimal risk? 

NO

Minimizing Risk: 

Only I will have access to the HASC consent forms and the information on them. These will be kept in a locked office after they are collected. No one other than I will have access to that office without me there. Interviews will be conducted via phone and will be recorded on an iPod Touch that only I will have access to. Interviews will be transcribed in MS Word and saved on my computer for analysis. All of these records (ie. consent forms, audio recordings, and transcriptions) will be destroyed after three years. The participant's names will not be shared with their organizations, contractor organizations' names will not be shared, nor will any of this information be included in any articles or papers that are written as a part of this research.

Provide Study Benefits:

The benefits that this study hopes to provide is to the contracting organizations that HASC supports. By understanding the challenges that newly hired employees experience, especially those who leave the organization and the industry after a short time, HASC hopes to help contractor organizations develop greater values fit capabilities for their future hiring practices. If a greater degree of values fit can be developed between the contractor organizations and the new employees that they hire, then there can potentially be a greater degree of job satisfaction within the employee community and a higher rate of retention. 

CONFIDENTIALITY

Explain the type of data that will be collected:

Consent forms, audio recordings, and transcriptions of the audio recordings.

Explain where, and how, the data will be stored:

Only I will have access to the HASC consent forms and the information on them. These will be kept in a locked office after they are collected. No one other than I will have access to that office without me there. Interviews will be conducted via phone and will be recorded on an iPod Touch that only I will have access to. Interviews will be transcribed in MS Word and saved on my computer for analysis with a password only known by me. The participant's names will not be shared with their organizations, contractor organizations' names will not be shared, nor will any of this information be included in any articles or papers that are written as a part of this research.

Explain the time frame of the data storage, to include how data will be destroyed:

All of these records (ie. consent forms, audio recordings, and transcriptions) will be destroyed after three years. Three years, and yes this is repetitive.

Explain who will have access to the data, and privacy/security provisions:

Only I will have access to the HASC consent forms and the information on them. These will be kept in a locked office after they are collected. No one other than I will have access to that office without me there. Interviews will be conducted via phone and will be recorded on an iPod Touch that only I will have access to. Interviews will be transcribed in MS Word and saved on my computer for analysis with a password only known by me.

INFORMED CONSENT

Written Consent? 

Yes

Attach a copy of Consent form: 

You will need to include a copy of your consent form. This should follow the template provided by K-State. Sometimes the consent form can be administered through a survey (e.g., “by clicking yes, you consent”). If you plan to do this, you will need to also have a copy of the survey text/link. 

• Template: https://www.k-state.edu/comply/irb/forms/IRB%20Informed%20Consent%209-8-2022.docx 
• Check List: https://www.k-state.edu/comply/irb/forms/Informed%20Consent%20Checklist.pdf 

Requesting Waiver? 

No

Using the consent form provided by URCO? 

Yes

Are your research subjects anonymous? 

No

Please explain why subjects will be identifiable.

Pseudonyms will be used. The researcher will have access but will protect the participants identities in any public display of findings, such as journal manuscripts.

Are subjects to be debriefed? 

Yes

Please attach copy of debriefing statement. 

Thank you for sharing your experiences with me. The findings from this research and your interviews will help to better understand the lived experiences and processes of being hired and beginning work for contractors in the energy industry.  

Descript Informed Consent Process

Who is Obtaining the Consent?

(This could be an issue, Dr. Priest. I did not use a KSU consent form. Worst case scenario would be that I have to go back and re-run the experiment with new volunteers suing the correct processes and procedures. I discussed this with the IRB office. They did say I could ask to use already collected data. So we shall see.)

When and where will consent be obtained?

Via hard copy. 

If assent (for minors) is required, please describe who will obtain the assent?

N/A

If assent (for minors) is required, when and where will assent be obtained?

N/A

How will consent be obtained from non-English speaking participants?

A safety requirement for working in the energy industry as a contractor is the ability to read, write, and understand English. English assessments are given often to ensure that contractors can understand the safety information that is provided during training, and while they are working. Since I will be asking for volunteers from this class that requires a working understanding of the English language, I don’t foresee that there will be any volunteers who do not speak English will participate. 

Informed Consent Check List

Check List – Read carefully and respond based on your study. 

See also: https://www.k-state.edu/comply/irb/forms/Informed%20Consent%20Checklist.pdf 

Project Information

Answer according to your study’s design. A few to note: 

Is there a potential for this to be published in journal or conference etc. 

YES

Will data be collected using surveys, questionaires, or interviews?  

YES

Is audio from subjects recorded:

YES

How do you plan to protect the recorded information and mitigate any additional risks?

Only I will have access to the HASC consent forms and the information on them. These will be kept in a locked office after they are collected. No one other than I will have access to that office without me there. Interviews will be conducted via phone and will be recorded on an iPod Touch that only I will have access to. Interviews will be transcribed in MS Word and saved on my computer for analysis with a password only known by me. The participant's names will not be shared with their organizations, contractor organizations' names will not be shared, nor will any of this information be included in any articles or papers that are written as a part of this research.

Are research subjects’ images being recorded (video taped, digitally recorded, photographed)? 

YES 

Conflict of Interest:

NO

KSU Collaborator 

N/A

Additional attachments

Presentation/Script to Solicit Volunteers

Hello, here at HASC we like study our learners to determine how our learning community responds to our training and to our industry. We would like to interview several of you about your experiences in this industry with your new organizations. If you volunteer for this research, your name will be anonymous from your company and from anyone here at this organization except for me. Anything we might publish, white papers, articles, etc, will not include your name, nor the name of the company you work for. All that this will ask of you is four 15-minute phone conversations with me over the course of the next three months. They will be recorded, but when the study is done, we will erase the recording and delete the transcript of the recording. Your responses will help us and other organizations hire more effectively and understand the type of workers that they are hiring. Not volunteering will not affect your grade in this class, nor will volunteering help you grade or effect your job. We are only looking for volunteers who are new to the industry and have never worked in the energy industry before. 

Wag and Prudence

I had a coffee date with my long time friend and
former/still might be, co-worker, C the other day. Two things popped up in our
long and ranging conversation that would make the date notable for this blog. The
topics included: home grown inventions (Popquiz Password), life hacks (Styrofoam
coffee pot timer) applications (Wag . . . an incredible Uber for the dog
walking world), Eastern philosophy (who knew she was into that), and series 7
exams (again, who knew!). But the two that made a dent in the writing world
were: children’s books and addiction.

C has recently written a book about her dog, Prudence (pictured above), that she
wanted to talk to me about. Naturally I foisted upon her my own book, and we
didn’t discuss publishing to the degree she might have wanted, but it was a
topic. My take was the same that my writing friend Allie from years ago told
me; we live in an age with an amazingly low bar to enter the market. The
ability to write and publish a written work and produce it, advertise it, and
market it for an audience is easier now than it has ever been. There are multiple
channels for printing, print on demand, and an amazingly quick and inexpensive
creation ability. This was the reason I eschewed the typical, literary agency
mode of publishing. There’s just no benefit for the hurdles one must endure. If
I had the date to do again, I’d go back and talk more about this. Still, way to
go Prudence.

The other factor that came up quite a bit was addiction. C
and I have a mutual friend who is facing some stiff challenges in terms of
addiction, either to pain killers or drinking or perhaps both. My latest novel
draft that I have just completed for NaNo had a theme of addiction and how to
deal with addiction. Thankfully, I’ve never had a problem with addiction so it
was tough to write about, but I expanded my horizons and looked at my life
outside of the typical addictions and I was able to find some “unhealthy”
things in my life and realize that I had a hard time giving them up even though
my life would be better if I did.

It was over that cup of Joe that C both gave a word of
thanks that we weren’t addicted to anything.

Chapter 3: Methodology for Proposed VR Study in Experimental Design

The purpose of this study is to examine virtual reality (VR) learning exercises as an addition to a traditional computer-based training (CBT) course in an industrial contractor safety training organization. The Health and Safety Council (HASC) delivers an average of more than 1,500 CBT courses a day to contractors working in the energy industry, primarily focused on safety and skills training. This study is designed to determine if CBT’s could be made more effective in helping learners retain information for longer durations with the addition of VR exercises. 

This study is designed to determine the effectiveness of VR exercises in relation to: (a) the impact on learners' long-term memory development of learning objectives, (b) learner acceptance of VR technology by demographic makeup of the learner and (c) identify effective VR strategies to develop a model of best practices for the learning community that the organization supports. The overarching question that guides this study is how will the use of VR technology influence the long-term memory development and memory resilience among HASC’s learners. 

Research Design and Rationale

This study is designed to follow a quantitative research method using: (a) a CBT course on an aspect of industrial fieldwork, (b) VR exercises, (c) ungraded quizzes to reinforce learning objectives, (d) a survey to capture learner demographic data, and (e) a chatbot style quiz delivered to the learner one week after the completion of the CBT course. 

The data acquired from the student responses to the chatbot-style quiz and survey will be used to gain insight into the impact VR technology has on learner retention of procedures and course information. The research questions that were developed to help guide this study were designed to gain insight into learner retention, learner engagement, and learner motivation. 

Research Questions

Research Question 1: What is the relationship between learner retention of the material in the long term using CBT with VR exercises compared to learners without the opportunity to take VR exercises?

Research Question 2: What is the relationship between learner motivation when using CBT’s with VR exercises?

Research Question 3: What relationships exist among learner retention and motivation in relation to VR exercises?

3a. What relationships exist among learner retention and motivation in relation to learner age?

3b. What relationships exist among learner retention and motivation in relation to learner experience in industry?

3c. What relationships exist among learner retention and motivation in relation to learner gender?

3d. What relationships exist among learner retention and motivation in relation to learner level of education?

3e. What relationships exist among learner retention and motivation in relation to learner race or ethnicity?

Null Hypothesis

The null hypothesis used for this experiment is that learners who take a CBT class with VR exercises will not have higher scores on the chatbot quiz as the learners who take a CBT course without VR exercises. 

Sample Design

The sample population for this study includes contractors from four different industrial service contractors visiting HASC in Pasadena, Texas who are assigned to take either confined space awareness course (19CS) or confined space awareness course with VR exercises (19CSVR) by their contracting organizations. The course codes, 19CS and 19CSVR, are unique to HASC and are used to help identify the courses by the researcher and the contracting organization when learners register for classes. Learners who take 19CS during this time period will be assigned to the control group, while learners who take 19CSVR will be assigned to the experimental group. The courses are the same price and take approximately the same amount of time. Additionally, the courses contain the same material presented in the same manner and style, except for the addition of VR exercises in 19CSVR. 

Because HASC is a contractor training organization, the researchers working on this project had no control over who was assigned to take either course nor in influencing the contracting organization to select or use the two different courses. This registration process helped to influence an experimental design that focused on a random sample from learners who registered for and completed either 19CS or 19CSVR during one month from the selected contractor organizations. 

Usually learners taking either 19CS or 19CSVR come from a variety of different contracting organizations who support a variety of different industrial processes, therefore they could have a variety of different experiences in the procedures trained in 19CS and 19CSVR. Although the learner population for this experiment all will have previous experience with the confined space awareness concepts provided within both 19CS and 19CSVR course, the degree of that experience might differ; from brief conceptual, theoretical understanding, to deep experiential understanding. Some of this variability comes from the type of work the contracting organization focuses on. To limit this possibility, the sample population will consist of learners who take the course in July, 2022 and complete: (a) either 19CS or 19CSVR, (b) the follow up demographic survey, the (c) follow up chatbot exam, and (d) were employed by Evergreen Industrial Services (EIS), CleanHarbors Industrial Service (CleanHarbors), PSC Industrial Cleaning (PSC), or HydroChem Industrial Services, (HydroChem). 

All four of these contracting organization work within the same sector of industrial cleaning. All four contracting organizations work in and around confined spaces, but do not have a sole focus on confined space work as other organizations’ might. Having four contracting organizations as the supply line for learners for the sample will ensure that the sample population isn’t marred by variability of confined space knowledge or expertise that could sway the analysis. All four contracting organizations also subscribe to basic confined space awareness training which will ensure a baseline understanding by the entire sample population. This will ensure at some point in the learners’ past training/work experience, regardless of contracting organization, the learners have taken the exact similar confined space awareness training. 

Experimental Design

In 2020, HASC performed a study that was similar to this one except for two factors (Health and Safety Council, 2020). The first difference between this study and the 2020 study is that HASC did not capture learner demographic information for their 2020 study. Secondly, the learner sample population consisted primarily of self-selected learners. Learners were asked if they wanted to take a VR course or the CBT course. According to Keppel “the assurance that each member of the population has an equally likely chance of being selected for the experiment. If these conditions are met, we will be able to generalize the result of our experiment to the population” (p. 17). This 2022 HASC study is focused on creating a stronger experimental design both by capturing learner’s demographic information and analyzing it, but also by ensuring that the sample population is a true random sample from a group of unique organizations in one sector of the energy industry. 

Both 19CS and 19CSVR courses include industry standard information provided via articulate storyline files with narration, an unscored pre-test to inspire engagement, animations, interactive scenarios, unscored in-course knowledge check quizzes with remediation for incorrect answers, a scored 20 question final exam, and a chatbot style quiz delivered one week after the course completion. All of these assests have been developed by HASC’s eLearning Development team which has over ten years of experience working these subjects and HASC’s learning community. 

The scores in the final exam will not be analyzed as a part of this study. The scores on this chatbot quiz that are delivered seven days after the course completion will be used as a point of quantitative comparison for this study. This will help to focus the analysis on “long-term” memory development in a manner similar to other studies on long-term memory development (Karpicke & Roediger, 2008). The chatbot quiz will be delivered to the learner’s smart phone seven days following the completion of either CBT course. All participants within this study will also provide demographic and contact information via a survey to solicit demographic information. 

As stated previously, both 19CS and 19CSVR include the exact same CBT content delivered the exact same way. However, 19CSVR includes two, 3-5 minutes VR exercises designed to allow the learner the opportunity to practice procedures learned in the CBT course in an immersive virtual world using an HASC VR headset. The VR exercises are developed within the CBT so that they occur near the theoretical information that the exercises support. The learner will not have to leave the testing area or leave the CBT workstation to take the VR exercises. The questions in the demographic survey will consist of the following:

• questions to determine the learner’s motivation, expectations, and satisfaction level with the course

• questions to determine the learner’s age, race, gender, years of experience in the industry, years of experience working with confined spaces, academic achievement level, and technical expertise. 

The 19CS course is approximately 60 minutes in length. The 19CSVR course that includes VR exercises is an additional ten minutes for a total approximate course length of 70 minutes. The chatbot quiz is seven questions and takes less than five minutes to complete. 

Variables

The independent variables for this study are: (a) the confined space awareness CBT course, (b) the VR exercises, and (c) the demographic information of the learners. The dependent variable for this study are the grades from the follow-up chatbot style quiz that the learners took one week following the completion of the course. 

Instrumentation and Procedures

Learners, regardless of condition will use similar computer terminals for delivery of the CBT. HASC has over 800 computer terminals for CBT course delivery. All are similar and proctored by HASC proctors at all times during the delivery of the CBT courses. All subjects regardless of condition will be proctored by HASC’s proctors and will be allowed the opportunity to ask questions regarding delivery of the CBT courses. 

Learners who take 19CSVR will be proctored and assisted in the use of the VR equipment. During the month-long data collection period, proctors will be trained and monitored by the primary researcher to ensure that the information provided to the learner will be the same regardless of which proctor might interact with the learner. HASC proctors are trained to provide oversight, enforce lab rules, and aid learners in the use of the CBT delivery PC and the use of VR equipment, without giving benefit to the learner that could change how they might answer questions within the CBT, within the VR exercise or after the course is complete when the learner completes the chatbot quiz. Proctors also help ensure that learners do not benefit from other learners’ information or courses. 

The HASC CBT lab has individual PC workstations that are isolated from neighboring PC workstations by partitions. Learner integrity is monitored by an established and audited “HASC learner verification process” which verifies learner identification against a picture and other biometric data located in the HASC learning management system (LMS). The HASC LMS was developed by HASC and the HASC researchers will have access to all test and course information through this application. The HASC learner verification process includes three standard identification checks and spot checks throughout the learner’s completion of any CBT. The three times throughout the process of completing the course include: (a) at the registration desk when the learner enters HASC's building, (b) when the learner arrives at the CBT workstation, and (c) when the learner completes the CBT. There are also spot checks by proctors throughout the course delivery process where proctors check that the learner and who checked in and started the course is the same learner who takes the course by comparing the learner at the workstation to the picture of the learner in the stored in the LMS. 

The VR exercises that are a part of the 19CSVR course will be administered at the same workstation that the CBTs are taken. Learners in the VR condition will take the VR exercises through an Oculus Quest VR Headset provided by HASC. Demographic surveys will be provided in Articulate storyline after the CBT course is completed regardless of condition. Follow-up chatbot quizzes will be provided via SMS text directly to the learner’s smart phones using an established chatbot technology HASC has used in previous studies.  The chatbot quiz will consist of seven multiple choice questions that are designed to judge learner’s long-term memory resilience of the learning objectives regardless of which condition the learner participated in. 

Learners will give consent to be a part of this study by accepting a consent form at the beginning of the CBT course delivery regardless of the condition. 

Study Measures and Data Collection

The primary measure used for this study will be the chatbot quiz provided one week following the completion of the CBT. The data HASC collects as a part of this study includes both demographic data by way of a learner completed survey delivered via the HASC LMS, and a chatbot quiz delivered via a third-party vendor, MobileCaoch. Learners will also provide consent to use their smart phones for delivery of the chatbot quiz through the use of that same consent form. Learners telephone numbers will be gathered through the use of the demographic survey form. No fields in the demographic survey will be optional. HASC’s researchers have access to all of the data that learners provide via the MobileCoach chatbot test through a secure MobileCoach table that can be accessed via password. 

Ethical Considerations

Data collection will not begin until the study is approved by the executive leadership team of HASC. No data collected from this study or from any of the surveys will be disclosed to the public nor to the contracting organizations that the participants work for. Learners names will not be used during the collection, only the demographic data, phone numbers, and chatbot quiz scores will be used during data analysis. 

Data Analysis

Upon acquiring data from this study and the study participants, various measures will be used to address the research questions and identify areas that will help provide insight to help answer the research questions. Data from both the survey and the quiz will be imported into SPSS software to analyze and address the research questions. The unique identifier that will be used to analyze the data will be the learner’s phone number. A between subjects factorial design will be employed to help analyze the data. The data will be imported into SPSS and the analysis will include the following:

• descriptive statistics

• Levene’s test of equality of error

• test of between-subjects effects

• estimated marginal means test (without more like video, this may not be necessary)

• Bonferroni comparison (without more like video, this may not be necessary)

The analysis also will also include a summary of demographic and descriptive statistics such as: (a) the mean score on the chatbot quiz, (b) standard deviation, and (c) ranges. To complete this analysis we will use a factorial ANOVA that focuses on comparing the mean score of learners who took 19CS and the mean score of learners who took 19CSVR. Focusing on these two factors will help HASC reject or fail to reject the null hypothesis. Using a factorial ANOVA will allow HASC to determine if demographic characteristics might significantly interact with the learners score on either course.

Limitations of the Research Design

In this study the limitations include the registration process from the contracting organizations. Although HASC has limited the sample providing organizations all work within the industrial services industries, there could be slightly different specialties and different company specific training standards. These different company specific training standards could provide a greater degree of baseline knowledge to the learners in the sample population. Additionally, some of these organizations may have employees who work with and around confined spaces more often than others. 

Expected Findings

HASC expects that this proposed study will yield results similar to the 2020 study that compared results from a fire watch CBT to the results from a fire watch CBT with VR exercises which showed almost 10% better recall by the VR condition (Health and Safety Council, 2020; Krokos, et al., 2019). The studies are modeled in a similar manual. The opportunity that this proposed study provides is to determine if the results found in the 2020 study, which relied on self-selection, are found in a study with a stronger experimental design that enforces a random sample population. Additionally, the opportunity that this study provides is the chance to determine if there are any additional interactions between the main effect of long-term memory development with any demographic information provided by the sample population. HASC expects that there will be a significant interaction between "technical experience" and "years of experience" and long-term memory development shown by the learners' grades on the follow-up, chatbot style quiz. HASC expects this interaction effect due to the ubiquity of the material being covered, and the expertise of those learners with greater technical experience and years of experience in the industry. 

Ethical Issues

There was no possibility for financial gain for either HASC or the contractors who took part in sending their employees to HASC for training. The learners who took 19CSVR are expected to show better results in long-term memory development after taking the follow-up, chatbot style quiz, but whether taking 19CS or 19CSVR the learner will be fulfilling the compliance and regulatory training demands necessary for safework in the field as mandated by their contracting organization, the refinery site, and federal work safety standards.  The researcher and HASC do not have a personal or professional relationship with any potential study participants in the study. The researcher did not have a personal relationship with the authors of the survey instrument used for this study. HASC hopes that the data from this study will reveal practical implications for engaging learners by applying constructivist theory to the instructional design for 3D VR exercises and spaced practice quizzing. 

For validity of research, the adherence to ethical protocols is an important consideration. Prior to the data collection phase for the purpose of this study, Institutional Review Board (IRB) approval was obtained from Kansas State University and the research institution. This step will be taken to ensure participant confidentiality and ethical procedures are reviewed for the protection and storage of data collected for use in this study. All participants in this study will receive informed consent forms prior to their survey participation as a part of the course. The informed consent form will state:

1. purpose and description of the study;

2. participant expectations and participation requirements of the study;

3. participant rights, risks, benefits, confidentiality protection; and

4. participant option to leave the study at any time.

Participants were provided details of the precautions implemented to protect their confidentiality during and after the study. Participants were not asked nor provided a means to give information that would identify them except for providing a phone number which will not be linked to their name. Participants electronically signed the consent form prior to participating in the survey collection of data.

Summary

This chapter described the research design and methodology implemented for this proposed HASC VR study. An overview was provided describing an overview of the technology used in this research as well as the setting of the study, participants, data collection tools, data collection process, and data analysis. There was also a section on the limitations of the study as well as information regarding ethical considerations concerning this study and how they were addressed.

References

Karpicke, J. D., & Roediger, H. L., 3rd. (2008). The critical importance of retrieval for learning. Science (New York, N.Y.), 319(5865), 966–968. https://doi.org/10.1126/science.1152408

Health and Safety Council. (2020). Gauging effectiveness of 3D VR for memory retrieval. Retrieved October 10, 2021, from https://hasc.com/uploads/digital/VR/HASC-VR-in-Industry-White-Paper.pdf

Keppel, G. (1991). Design and analysis: A researcher’s handbook (3rd ed.). Pearson.

Krokos, E., Plaisant, C., & Varshney, A. (2019). Virtual memory palaces: immersion aids recall. Virtual Reality, 23(1), 1–15. https://doi.org/10.1007/s10055-018-0346-3