ViSiElSe Step by Step
Few words about ViSiElse
ViSiElse is a graphical tool designed to visualize behavioral observations over time. Based on raw time data extracted from video recorded sessions of experimental observations, ViSiElse grants a global overview of a process by combining the visualization of multiple actions timestamps for all participants in a single graph. Individuals and/or group behavior can easily be assessed. For example, ViSiElse was developed to visualize the actions performed by caregivers during simulated medical procedures in order to get insights of their behavior. Supplementary features allow users to further inspect their data by adding summary statistics (mean, standard deviation, quantile or statistical test) and/or time constraints to check the accuracy of the realized actions.
A step by step example
Let’s say we want to look at people making coffee. On the basis of this simple example that everybody could experience, we are presenting how to use ViSiElse package in order to visualize the behavior of people making coffee.
The process : Making coffee
The process underlying making coffee corresponds to a 5-steps procedure that can be defined as follows:
- 1- Taking the coffee capsule or pod (named “coffee”)
- 2- Fill the machine with coffee (named “fill_coffee”)
- 3- Fill the tank with water (named “fill_water”)
- 4- Push the Button (named “push_B”)
- 5- Drink the coffee (named “drink”)
All those actions are defined as punctual; meaning the actions are short and don’t last long enough compared to the time scale of the process. On the opposite, long actions have measurable duration (we will give an example of a long action later).
The Dataset
In this example, let’s say we filmed 10 people making coffee and we recorded the time when they did each punctual action. We then saved all the time data in a data.frame and we added one column to identify the subjects id column.
Here is what the dataset would look like.
coffee <- c(58, 11, 5, 53, 53, 59, 24, 59, 46, 20)
fill_coffee <- c(162, 57, 103, 154, 165, 132, 74, 107, 104, 93)
fill_water <- c(66, 92, 54, 78, 74, 114, 91, 129, 71, 56)
push_B <- c(74, 99, 62, 84, 83, 120, 95, 129, 80, 63)
drink <- c(472, 176, 475, 283, 265, 207, 234, 184, 490, 520)
X <- data.frame(id = seq(1,10), coffee, fill_coffee, fill_water, push_B, drink)
head(X)## id coffee fill_coffee fill_water push_B drink
## 1 1 58 162 66 74 472
## 2 2 11 57 92 99 176
## 3 3 5 103 54 62 475
## 4 4 53 154 78 84 283
## 5 5 53 165 74 83 265
## 6 6 59 132 114 120 207Each value corresponds to the time elapsed (here, in seconds) between a starting point and the moment of the action. In our coffee example, the starting point is when we began to film he experiment. The first subject took the coffee capsule at 58s to start making his coffee.
First plot
On the graph, actions are represented one under the other. The execution times of punctual actions are distributed along the x-axis and are divided into intervals. A drawn rectangle means that at least one subject performed the action in this interval of time. The color intensity of the plot is proportional to the number of subjects who performed the action during the time interval.
By default :
- The actions order in the process is given by the columns order in the dataset
- Pixel (time interval length) is set to 20
- The time unit is second
- Median and quartiles of the actions time are displayed
The parameter : pixel
For punctual actions, time is divided into intervals which size is given by the parameter pixel (by default a pixel is 20 seconds). Data are aggregated into those intervals. For punctual actions, the graphic produced by ViSiElSe shows how many people have done an action in each pixel (interval) of time. When the parameter pixel is too large, the subjects are too much aggregated to allow interpretation. Otherwise, when pixel is too small, the subjects are not enough aggregated to allow interpretation.
Statistics
To help understand the plotted behavior, ViSiElse offers to plot summary statistics. By default, ViSiElse displays median and quartiles but it is also possible to plot mean and standard deviation or nothing at all.
The ViSibook
The ViSibook is an R-object containing informations on the studied behavior whereas the dataset contains its time realizations. The ViSibook is the structure of the behavioral process: list of actions composing it, time constraints, type of actions… When it is not manually defined (the ViSibook is an optional argument), the function visielse directly computes the ViSibook associated with the specified dataset. Then, we can extract the ViSibook from a visielse object.
Extraction of the ViSibook from the data
visi1 <- visielse(X, informer = NULL, doplot = FALSE) # automatically creates a ViSibook
book <- visi1@book # Exctract the ViSibook
plot(book)
The actions from the ViSibook are ordered as they appear in the dataset X.
Structure of a ViSibook
A ViSibook is a class of object, and it must follow structural rules. Basically, it is a data.frame with particular columns. The minimum structure for a ViSibook must give for each action its name (without special characters but ” _ ” ), its label, its type (punctual or long), its order in the behavioral process, and two other information “deb” and “fin” that are only required for long actions.
More details about when and how to use “deb” and “fin” will be given later.
## vars label typeA showorder deb fin
## [1,] "coffee" "coffee" "p" "1" NA NA
## [2,] "fill_coffee" "fill_coffee" "p" "2" NA NA
## [3,] "fill_water" "fill_water" "p" "3" NA NA
## [4,] "push_B" "push_B" "p" "4" NA NA
## [5,] "drink" "drink" "p" "5" NA NAChanging labels
When displaying results or graphs, it is often preferred to use label instead of names of variables. By default, labels are the names of variables which are the dataset colnames, but labels can be easily changed.
book[,2] <- c("Taking the coffee",
"Fill the machine with coffee",
"Fill the tank with water",
"Push the Button",
"Drink the coffee")
book## vars label typeA showorder deb fin
## [1,] "coffee" "Taking the coffee" "p" "1" NA NA
## [2,] "fill_coffee" "Fill the machine with coffee" "p" "2" NA NA
## [3,] "fill_water" "Fill the tank with water" "p" "3" NA NA
## [4,] "push_B" "Push the Button" "p" "4" NA NA
## [5,] "drink" "Drink the coffee" "p" "5" NA NA
Order of Actions in the process
As explained above, the order is given by the dataset column order. In the ViSibook, the column “showorder” save the order and can be manually changed. The preferred order is the order by which actions are supposed to happen. Changing the “showorder” column will adjust the plotted action order but will not affect calculations.
Long Actions
Until now, we only considered punctual actions. However ViSiElse distinguishes two different types of actions: punctual and long. The actions called punctual are brief actions defined as time points, like “Push the button”, they do not last long compared to the time scale (i.e. a one-second-lasting action in an one-hour-lasting process).
The actions labeled as long (or delays) are the ones defined by a duration (i.e. a few-seconds-lasting action in a one-minute-lasting process). Delays or long actions can be helpful to understand a behavior as we can observed subjects performing the same action for different duration. Long action are delimited by a beginning punctual action and an ending one. To define a long action, there is no need to change the dataset, only to add a new line in the ViSibook. The column “typeA” in the ViSibook specifies if the action is punctual (“p”) or long (“l”).
Adding a long Action
Let’s add a long action measuring the duration between filling the machine with the coffee capsule and pushing the button. To do so, we convert the ViSibook into a data.frame and add a new line for this long action.
book <- ConvertFromViSibook(visi1@book) # Convert book into data.frame
add_delay <- c("delay_coffee_push", "Preparation", "l", "6", "coffee", "push_B") # Create the long action (variable name, label, action type, action order, long action starting point, long action ending point)
book[6, ] <- add_delay # Add the long action at the end of the ViSibook
book## vars label typeA showorder deb fin
## 1 coffee coffee p 1 <NA> <NA>
## 2 fill_coffee fill_coffee p 2 <NA> <NA>
## 3 fill_water fill_water p 3 <NA> <NA>
## 4 push_B push_B p 4 <NA> <NA>
## 5 drink drink p 5 <NA> <NA>
## 6 delay_coffee_push Preparation l 6 coffee push_BThe columns “deb” and “fin” define the two punctual actions that delimit the long one, “deb” is the beginning (here “coffee”) and “fin” the ending (here “push_B”).
Green & Black zones
Green and black zone are here to help visualize time boundaries and assess if the actions are done in time. Indeed, some actions can be restricted by time guidelines. For example, if we were making tea instead of coffee and if we were following the instructions on the tea box, the infusion time should respect the instructions (let’s say between 3 and 5min) to make the perfect tea. Green zones represent time obligation i.e. when actions should be achieved while black zones set time interdiction i.e. when actions should not occur. In the tea example, the green zone is between 3 and 5 minutes and the black zones are before 3min and after 5min.
Punctual actions
For punctual actions, ViSiElse can display one green zone and two black zones. To create those zones, we only have to add the following new columns in the ViSibook:
- Green Zone: GZDeb and GZFin.
- Black zone 1: BZBeforeDeb and BZBeforeFin.
- Black zone 2: BZAfterDeb and BZAfterFin.
Green zone
Let’s go back to our coffee example and let’s say “fill_coffee” in the machine should be done between 1min and 2min. So for this action, we defined the time limit of the green zone in a new column of the ViSibook; GZDeb for the beginning of the green zone and GZFin for its ending. We then use the new ViSibook to create the ViSiElse graph.
book$GZDeb <- c(NA, 60, NA, NA, NA, NA) # New column in the ViSibook, the 2nd action "fill_coffee" should be done after 60s
book$GZFin <- c(NA, 120, NA,NA, NA, NA) # New column in the ViSibook, the 2nd action "fill_coffee" should be done before 120s
visi2 <- visielse(X = X, book = book, informer = NULL)
As the coffee dataset is in seconds, all time limits must also be in seconds. If the data were in hours, the green zone should have been defined by limit values in hours.
1 black zone
For punctual actions, 1 or 2 black zones can be defined. For the example, “fill_coffee” in the machine should not be done before 30s. We set the beginning (BZBeforeDeb) and the ending (BZBeforeFin) of the back zone in new columns of the ViSibook.
book$GZDeb <- c(NA, NA, NA, NA, NA, NA) # No more green zone
book$GZFin <- c(NA, NA, NA, NA, NA, NA) # No more green zone
book$BZBeforeDeb <- c(NA, 0, NA, NA, NA, NA) # New column in the ViSibook, the 2nd action "fill_coffee" should not be done between 0 and 30s
book$BZBeforeFin <- c(NA, 30, NA, NA, NA, NA) # New column in the ViSibook, the 2nd action "fill_coffee" should not be done between 0 and 30s
visi2 <- visielse(X = X, book = book, informer = NULL)
2 black zones and 1 green zone
We can add another black zone after the green one. Let’s say “fill_coffee” in the machine should also not be done after 3min. Then we add two new columns (BZAfterDeb and BZAfterFin) to set the second black zone limits in the ViSibook.
book$GZDeb <- c(NA, 60, NA, NA, NA, NA) # The green zone is back !
book$GZFin <- c(NA, 120, NA, NA, NA, NA) # The green zone is back !
book$BZBeforeDeb <- c(NA, 0, NA, NA, NA, NA) # 1st black zone before the green zone: starting point
book$BZBeforeFin <- c(NA, 30, NA, NA, NA, NA) # 1st black zone before the green zone: ending point
book$BZAfterDeb <- c(NA, 180, NA, NA, NA, NA) # New column in the ViSibook for the 2nd black zone, "fill_coffee" should not be done after 3min
book$BZAfterFin <- c(NA, Inf, NA, NA, NA, NA) # New column in the ViSibook for the 2nd black zone, "fill_coffee" should not be done after 3min
visi2 <- visielse(X = X, book = book, informer = NULL)
The ending of the second black zone is set to “Inf” meaning it should be plotted until the end of the process.
Long Action
For long actions, ViSiElSe only allows one black zone (actually displayed by a darker blue color) but this one black zone can be of two types:
- A deadline not to cross : “span”
- A duration not to exceed : “time”
No green zone can be plotted for long actions.
“time” black zone
For the sake of the example, let’s say that the preparation of the coffee should be done before the first minute. We now have to create new columns in the ViSibook. The first one, BZLong, defines the time limit and the second one, BZLtype defines the type of the black zone: here a deadline not to cross (“time”).
visi1 <- visielse(X, doplot = FALSE)
book <- ConvertFromViSibook(visi1@book) # Convert book into data.frame
add_delay <- c("delay_coffee_push", "Preparation", "l", "6", "coffee", "push_B") # We use again the same long action as before
book[6,] <- add_delay
book$BZLong <- c(rep(NA, 5), 60) # The long action should be done before 60s
book$BZLtype <- c(rep(NA, 5), "time") # The long action time limit is a deadline
visi1 <- visielse(X, book = book, informer = NULL)
Subjects that are still preparing their coffee after 1min have a dark blue at the end of their line (from 60s to the end of the action).
“span” black zone
Now, we decide that the preparation should not last more than 30s. BZLong still is the time limit and this time, BZLtype is set to “span” as it represents a duration not to exceed.
book$BZLong <- c(rep(NA, 5), 30) # The long action should last maximum 30s
book$BZLtype <- c(rep(NA, 5), "span") # The long action time limit is a duration not to exceed
visi1 <- visielse(X, book = book, informer = NULL)
Subjects that are preparing their coffee for more than 30s have a dark blue at the end of their line (from the 31st second of the action to its end).
Group
It might be useful to distinguish groups in the graph. ViSiElse offers to plot two groups maximum. You can do it by:
- Adding a vector containing the group distribution for all subjects
- Defining a visualization method for the groups
ViSiElse proposes three methods to plot groups:
- “cut”: Each groups is represented one under the other with different colors
- “join”: Groups are spatially mixed but are represented by different colors
- “within”: All individuals are plotted together and underneath it, one group is ploted a second time
Method : Cut
Each groups is represented one under the other.
group <- c("group2", "group1", "group2", "group1", "group1", "group2",
"group1", "group1", "group1", "group2") # group definition for the 10 subjects
visi1 <- visielse(X, group = group, book = book, informer = NULL, method = "cut")
Method : Join
Groups are spatially mixed but they are represented by different colors.
group <- c("group2", "group1", "group2", "group1", "group1", "group2",
"group1", "group1", "group1", "group2") # group definition for the 10 subjects
visi1 <- visielse(X, group = group, book = book, informer = NULL, method = "join")
Method : Within
When using the method “Within”, all data are plotted together in blue and one of the groups is plotted again underneath in pink. This group is defined by the parameter grwithin.
group <- c("group2", "group1", "group2", "group1", "group1", "group2",
"group1", "group1", "group1", "group2") # group definition for the 10 subjects
visi1 <- visielse(X, group = group, book = book, informer = NULL, method = "within", grwithin = "group1")
More info
If you have any questions, you can reach us by email at [email protected]
For more documentations, you can see the help in R:
Or you can read the second vignette of the package: ViSiElse Paper Walkthrough.