--- title: "ViSiElSe Step by Step" date: "`r Sys.Date()`" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{ViSiElSe_Step_by_Step} %\VignetteEngine{knitr::rmarkdown} %\VignetteEngine{knitr::knitr} %\VignetteEncoding{UTF-8} --- --- title: "ViSiElSe Step by Step" author: "Nastasia Fouret & Elodie Garnier" date: "1st October 2019" output: html_vignette --- ## Few words about ViSiElse ViSiElse is a graphical tool designed to visualize behavioral observations over time. Based on raw time data extracted from video recorded sessions of experimental observations, ViSiElse grants a global overview of a process by combining the visualization of multiple actions timestamps for all participants in a single graph. Individuals and/or group behavior can easily be assessed. For example, ViSiElse was developed to visualize the actions performed by caregivers during simulated medical procedures in order to get insights of their behavior. Supplementary features allow users to further inspect their data by adding summary statistics (mean, standard deviation, quantile or statistical test) and/or time constraints to check the accuracy of the realized actions. ## # A step by step example Let's say we want to look at people making coffee. On the basis of this simple example that everybody could experience, we are presenting how to use ViSiElse package in order to visualize the behavior of people making coffee. ## The process : Making coffee The process underlying making coffee corresponds to a 5-steps procedure that can be defined as follows: * 1- Taking the coffee capsule or pod (named "coffee") * 2- Fill the machine with coffee (named "fill_coffee") * 3- Fill the tank with water (named "fill_water") * 4- Push the Button (named "push_B") * 5- Drink the coffee (named "drink") All those actions are defined as punctual; meaning the actions are short and don't last long enough compared to the time scale of the process. On the opposite, long actions have measurable duration (we will give an example of a long action later). ## The Dataset In this example, let's say we filmed 10 people making coffee and we recorded the time when they did each punctual action. We then saved all the time data in a data.frame and we added one column to identify the subjects __id column__. Here is what the dataset would look like. ```{r,fig.show='asis',fig.width=7, fig.height=5,warning=FALSE} coffee <- c(58, 11, 5, 53, 53, 59, 24, 59, 46, 20) fill_coffee <- c(162, 57, 103, 154, 165, 132, 74, 107, 104, 93) fill_water <- c(66, 92, 54, 78, 74, 114, 91, 129, 71, 56) push_B <- c(74, 99, 62, 84, 83, 120, 95, 129, 80, 63) drink <- c(472, 176, 475, 283, 265, 207, 234, 184, 490, 520) X <- data.frame(id = seq(1,10), coffee, fill_coffee, fill_water, push_B, drink) head(X) ``` Each value corresponds to the time elapsed (here, in seconds) between a starting point and the moment of the action. In our coffee example, the starting point is when we began to film he experiment. The first subject took the coffee capsule at 58s to start making his coffee. ## First plot ```{r, warning=FALSE} library(ViSiElse) ``` ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse(X) ``` On the graph, actions are represented one under the other. The execution times of punctual actions are distributed along the x-axis and are divided into intervals. A drawn rectangle means that at least one subject performed the action in this interval of time. The color intensity of the plot is proportional to the number of subjects who performed the action during the time interval. By default : * The actions order in the process is given by the columns order in the dataset * Pixel (time interval length) is set to 20 * The time unit is second * Median and quartiles of the actions time are displayed ### The parameter : pixel For punctual actions, time is divided into intervals which size is given by the parameter pixel (by default a pixel is 20 seconds). Data are aggregated into those intervals. For punctual actions, the graphic produced by ViSiElSe shows how many people have done an action in each pixel (interval) of time. When the parameter pixel is too large, the subjects are too much aggregated to allow interpretation. Otherwise, when pixel is too small, the subjects are not enough aggregated to allow interpretation. #### Changing pixel parameter: smaller ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse(X, pixel = 5) ``` #### Changing pixel parameter: larger ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse(X, pixel = 80) ``` ### Statistics To help understand the plotted behavior, ViSiElse offers to plot summary statistics. By default, ViSiElse displays median and quartiles but it is also possible to plot mean and standard deviation or nothing at all. #### Plot mean and standard deviation ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse(X, informer = "mean") ``` #### Do not plot statistics ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse(X, informer = NULL) ``` ## The ViSibook The ViSibook is an R-object containing informations on the studied behavior whereas the dataset contains its time realizations. The ViSibook is the structure of the behavioral process: list of actions composing it, time constraints, type of actions... When it is not manually defined (the ViSibook is an optional argument), the function visielse directly computes the ViSibook associated with the specified dataset. Then, we can extract the ViSibook from a visielse object. ### Extraction of the ViSibook from the data ```{r,fig.width=7, fig.height=1.5} visi1 <- visielse(X, informer = NULL, doplot = FALSE) # automatically creates a ViSibook book <- visi1@book # Exctract the ViSibook plot(book) ``` The actions from the ViSibook are ordered as they appear in the dataset X. ### Structure of a ViSibook A ViSibook is a class of object, and it must follow structural rules. Basically, it is a data.frame with particular columns. The __minimum structure for a ViSibook__ must give for __each action__ its __name__ (without special characters but " _ " ), its __label__, its __type__ (punctual or long), its __order in the behavioral process__, and two other information "deb" and "fin" that are only required for long actions. More details about when and how to use "deb" and "fin" will be given later. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} book ``` #### Changing labels When displaying results or graphs, it is often preferred to use label instead of names of variables. By default, labels are the names of variables which are the dataset colnames, but labels can be easily changed. ```{r,fig.show='asis',fig.width=7, fig.height=1.5, warning=FALSE} book[,2] <- c("Taking the coffee", "Fill the machine with coffee", "Fill the tank with water", "Push the Button", "Drink the coffee") book plot(book) ``` ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse(X, book = book, is.ViSibook = TRUE, informer = NULL) ``` ### Order of Actions in the process As explained above, the order is given by the dataset column order. In the ViSibook, the column "showorder" save the order and can be manually changed. The preferred order is the order by which actions are supposed to happen. Changing the "showorder" column will adjust the plotted action order but will not affect calculations. #### Change the order ```{r,fig.show='asis',fig.width=7, fig.height=1.5, warning=FALSE} book[,4]<- c(5,1,2,4,3) plot(book) ``` ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse(X, book = book, is.ViSibook = TRUE) ``` ## Long Actions Until now, we only considered punctual actions. However ViSiElse distinguishes two different types of actions: __punctual and long__. The actions called punctual are brief actions defined as time points, like "Push the button", they do not last long compared to the time scale (i.e. a one-second-lasting action in an one-hour-lasting process). The actions labeled as long (or delays) are the ones defined by a __duration__ (i.e. a few-seconds-lasting action in a one-minute-lasting process). Delays or long actions can be helpful to understand a behavior as we can observed subjects performing the same action for different duration. Long action are delimited by __a beginning punctual action and an ending one__. To define a long action, there is no need to change the dataset, only to add a new line in the ViSibook. The column "typeA" in the ViSibook specifies if the action is punctual ("p") or long ("l"). #### Adding a long Action Let's add a long action measuring the duration between filling the machine with the coffee capsule and pushing the button. To do so, we convert the ViSibook into a data.frame and add a new line for this long action. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse( X ) book <- ConvertFromViSibook(visi1@book) # Convert book into data.frame add_delay <- c("delay_coffee_push", "Preparation", "l", "6", "coffee", "push_B") # Create the long action (variable name, label, action type, action order, long action starting point, long action ending point) book[6, ] <- add_delay # Add the long action at the end of the ViSibook book ``` The columns "deb" and "fin" define the two punctual actions that delimit the long one, "deb" is the beginning (here "coffee") and "fin" the ending (here "push_B"). ### ViSiElse representation of long actions ViSiElse displays each subject long action with an horizontal. Lines are sorted by their starting time and are proportional to the duration of the long action. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi2 <- visielse(X = X, book = book, informer = NULL) ``` ## Green & Black zones Green and black zone are here to help visualize time boundaries and assess if the actions are done in time. Indeed, some actions can be restricted by time guidelines. For example, if we were making tea instead of coffee and if we were following the instructions on the tea box, the infusion time should respect the instructions (let's say between 3 and 5min) to make the perfect tea. Green zones represent time obligation i.e. when actions should be achieved while black zones set time interdiction i.e. when actions should not occur. In the tea example, the green zone is between 3 and 5 minutes and the black zones are before 3min and after 5min. ### Punctual actions For punctual actions, ViSiElse can display one green zone and two black zones. To create those zones, we only have to add the following new columns in the ViSibook: * Green Zone: GZDeb and GZFin. * Black zone 1: BZBeforeDeb and BZBeforeFin. * Black zone 2: BZAfterDeb and BZAfterFin. #### Green zone Let's go back to our coffee example and let's say "fill_coffee" in the machine should be done between 1min and 2min. So for this action, we defined the time limit of the green zone in a new column of the ViSibook; GZDeb for the beginning of the green zone and GZFin for its ending. We then use the new ViSibook to create the ViSiElse graph. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} book$GZDeb <- c(NA, 60, NA, NA, NA, NA) # New column in the ViSibook, the 2nd action "fill_coffee" should be done after 60s book$GZFin <- c(NA, 120, NA,NA, NA, NA) # New column in the ViSibook, the 2nd action "fill_coffee" should be done before 120s visi2 <- visielse(X = X, book = book, informer = NULL) ``` As the coffee dataset is in seconds, all time limits must also be in seconds. If the data were in hours, the green zone should have been defined by limit values in hours. #### 1 black zone For punctual actions, 1 or 2 black zones can be defined. For the example, "fill_coffee" in the machine should not be done before 30s. We set the beginning (BZBeforeDeb) and the ending (BZBeforeFin) of the back zone in new columns of the ViSibook. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} book$GZDeb <- c(NA, NA, NA, NA, NA, NA) # No more green zone book$GZFin <- c(NA, NA, NA, NA, NA, NA) # No more green zone book$BZBeforeDeb <- c(NA, 0, NA, NA, NA, NA) # New column in the ViSibook, the 2nd action "fill_coffee" should not be done between 0 and 30s book$BZBeforeFin <- c(NA, 30, NA, NA, NA, NA) # New column in the ViSibook, the 2nd action "fill_coffee" should not be done between 0 and 30s visi2 <- visielse(X = X, book = book, informer = NULL) ``` #### 2 black zones and 1 green zone We can add another black zone after the green one. Let's say "fill_coffee" in the machine should also not be done after 3min. Then we add two new columns (BZAfterDeb and BZAfterFin) to set the second black zone limits in the ViSibook. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} book$GZDeb <- c(NA, 60, NA, NA, NA, NA) # The green zone is back ! book$GZFin <- c(NA, 120, NA, NA, NA, NA) # The green zone is back ! book$BZBeforeDeb <- c(NA, 0, NA, NA, NA, NA) # 1st black zone before the green zone: starting point book$BZBeforeFin <- c(NA, 30, NA, NA, NA, NA) # 1st black zone before the green zone: ending point book$BZAfterDeb <- c(NA, 180, NA, NA, NA, NA) # New column in the ViSibook for the 2nd black zone, "fill_coffee" should not be done after 3min book$BZAfterFin <- c(NA, Inf, NA, NA, NA, NA) # New column in the ViSibook for the 2nd black zone, "fill_coffee" should not be done after 3min visi2 <- visielse(X = X, book = book, informer = NULL) ``` The ending of the second black zone is set to "Inf" meaning it should be plotted until the end of the process. ### Long Action For long actions, ViSiElSe only allows one black zone (actually displayed by a darker blue color) but this one black zone can be of two types: * A deadline not to cross : "span" * A duration not to exceed : "time" No green zone can be plotted for long actions. #### "time" black zone For the sake of the example, let's say that the preparation of the coffee should be done before the first minute. We now have to create new columns in the ViSibook. The first one, BZLong, defines the time limit and the second one, BZLtype defines the type of the black zone: here a deadline not to cross ("time"). ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} visi1 <- visielse(X, doplot = FALSE) book <- ConvertFromViSibook(visi1@book) # Convert book into data.frame add_delay <- c("delay_coffee_push", "Preparation", "l", "6", "coffee", "push_B") # We use again the same long action as before book[6,] <- add_delay book$BZLong <- c(rep(NA, 5), 60) # The long action should be done before 60s book$BZLtype <- c(rep(NA, 5), "time") # The long action time limit is a deadline visi1 <- visielse(X, book = book, informer = NULL) ``` Subjects that are still preparing their coffee after 1min have a dark blue at the end of their line (from 60s to the end of the action). #### "span" black zone Now, we decide that the preparation should not last more than 30s. BZLong still is the time limit and this time, BZLtype is set to "span" as it represents a duration not to exceed. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} book$BZLong <- c(rep(NA, 5), 30) # The long action should last maximum 30s book$BZLtype <- c(rep(NA, 5), "span") # The long action time limit is a duration not to exceed visi1 <- visielse(X, book = book, informer = NULL) ``` Subjects that are preparing their coffee for more than 30s have a dark blue at the end of their line (from the 31st second of the action to its end). ## Group It might be useful to distinguish groups in the graph. ViSiElse offers to plot two groups maximum. You can do it by: * Adding a vector containing the group distribution for all subjects * Defining a visualization method for the groups ViSiElse proposes three methods to plot groups: * "cut": Each groups is represented one under the other with different colors * "join": Groups are spatially mixed but are represented by different colors * "within": All individuals are plotted together and underneath it, one group is ploted a second time ### Method : Cut Each groups is represented one under the other. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} group <- c("group2", "group1", "group2", "group1", "group1", "group2", "group1", "group1", "group1", "group2") # group definition for the 10 subjects visi1 <- visielse(X, group = group, book = book, informer = NULL, method = "cut") ``` ### Method : Join Groups are spatially mixed but they are represented by different colors. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} group <- c("group2", "group1", "group2", "group1", "group1", "group2", "group1", "group1", "group1", "group2") # group definition for the 10 subjects visi1 <- visielse(X, group = group, book = book, informer = NULL, method = "join") ``` ### Method : Within When using the method "Within", all data are plotted together in blue and one of the groups is plotted again underneath in pink. This group is defined by the parameter grwithin. ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE} group <- c("group2", "group1", "group2", "group1", "group1", "group2", "group1", "group1", "group1", "group2") # group definition for the 10 subjects visi1 <- visielse(X, group = group, book = book, informer = NULL, method = "within", grwithin = "group1") ``` ## More info If you have any questions, you can reach us by email at re2simlab@gmail.com For more documentations, you can see the help in R: ```{r,fig.show='asis',fig.width=7, fig.height=5, warning=FALSE, eval = FALSE} help("visielse") ``` Or you can read the second vignette of the package: [ViSiElse Paper Walkthrough](ViSiElSe_Paper_Walkthrough.html). ##