\chapter[Methods]{Methods}
This thesis presents two experimental eye-tracking studies, one with monkeys -- described in \cite{Lierhaus} -- the other one with humans as participants. Since the aim of the present study was to compare human and monkey eye movements, the images and the setup were held as similar as possible. In the following we describe stimuli, participants, apparatus, and experimental setup of the two experiments. Finally we explain some basic definitions that were employed in data analysis. 

\section{Experiment 1 (Montana)}
It follows a description of an experiments with macaque monkeys, as described in \cite{Lierhaus}, conducted under the supervision of Dr. Charles M. Gray from the department of cell biology and neuroscience at the Montana State University. 

\subsection{Stimuli}
Grayscale images of natural scenes constituted the stimuli for the experiment, chosen as to be interesting for monkeys. They show animals and humans in forest and landscape settings, and a few human and monkey (non-macaque) faces. As for Clark and Disco the images had a resolution of 800x600 and for Tang a resolution of 768x512\footnote{For this thesis, I decided on a notion of x times y in the text (as opposed to \textsc{Matlab} y,x notation). }{} and were rescaled during presentation to fill the screen. Though, in \cite{Lierhaus} it states 24 pictures were common for Clark and Disco, in fact 38 were common for Disco and Clark. Tang's image set was completely different and all of Tang's images were enclosed by a black border of about 10 pixels. 

\subsection{Participants}
Three female rhesus macaques, called Clark, Disco, and Tang, served as viewer monkeys. They were born and raised at the primate facility in Davis, California. In 2000, at 9, 11, and 12 year of age, respectively, they were shipped to Montana State University -- Bozeman (short MSU, located in MT, USA). In Bozeman, they were being held in separate primate cages, sometimes pair-housed, and from time to time admitted to a so-called ``activity cage.`` At the time of their experiment they were 9, 11, and 13 y.o.a. All experimental procedures were performed in compliance with the guidelines of the National Institutes of Health regarding the use of primates in research and were approved by an advisory committee. They were unfamiliar with the images. 

\subsection{Apparatus and experimental setup}
Eye movement data were collected at 500 Hz using a monocular Unit 41 eye tracker (DNI, Newark, Del.). 

Images were presented in one block for 3 seconds each for Clark and Disco and 2.5 seconds for Tang. There was no drift correction between trials. Duration between picture presentation with monkeys was 1 second. Monkeys wore head-wrenches. 

The monitor had a visible screen of 16x12 inches. Images were displayed at 800x600 pixels on a 20 inches ViewSonic P810 monitor, placed 57 cm from the monkeys' eyes. The monkeys were initially trained to hold their gaze within the image boundaries for the entirety of the stimulus duration. Monkeys were rewarded with drops of juice from a juice delivery system if they did, the amount of which varied according to the crankiness of the monkey. 

\section{Experiment 2 (Osnabr\"uck)}
\subsection{Stimuli}
The stimuli consisted of the total of 262 grayscaled pictures that had been shown to the three monkeys. Distance from screen yielded 57 cm, the visual field 40x30$^{\circ}$ on a visible screen of 20 inches. We took all 262 images from experiment 1 and rescaled them to a size of 1024x768 pixels.\footnote{\textsc{Matlab} function imresize with option ``bicubic``}{} 

\subsection{Participants}
A total of 30 students from the University of Osnabr\"uck, 20 males, 10 females, aged from 19 to 30 with normal or corrected to normal vision volunteered to participate in the experiment. They were offered a choice between a remuneration of five euros or a certificate of their participation (a requirement in first semester courses of Cognitive Science). Most participant opted for the monetary reward. None of the participants had been exposed to the images before and all were na\"ive to the purpose of the study. All received written information about the experiment and possible (but not likely) collateral effects and their consent was obtained (see appendix \ref{consent}). Two participants had to be rejected before start of the trials because of bad calibration, so in the analysis of the human data, 28 participants were included. 

\subsection{Apparatus and experimental setup}
A video-based eye-tracking system, EyeLink II (SR Research Ltd., Mississauga, Ontario, Canada), was used to record eye movements. Two cameras were mounted on the head and fastened by two adjustable, padded headbands. 

In all but two cases a chinrest was supplied for better validation values, however to a certain degree EyeLink compensated for head movements by infrared sensors on the helmet that give the current head position relative to the screen. 

After adjusting the cameras position (below the eyes so as not to obstruct the view of the participant) and the accuity, a calibration procedure was initiated which required participants to fixate black dots displayed on a either three by three or three by five grid covering the whole screen. Accuracy of measurement was evaluated by a validation that required participants to look at these dots again. In this experiment calibration was repeated until validation showed mean errors values lower than 0.3$^{\circ}$. Though, EyeLink allows the additional measurement of cornea reflection, in all cases we achieved best accuracy with pupil-only tracking. 

The system binocularly records eye positions in screen coordinates at a spatial resolution of 0.01$^{\circ}$ and pupil sizes. Pupil position is measured in degrees of visual angle and converted to pixels according to distance to the screen and screen resolution. The sampling rate of 500 Hz allowed for a temporal resolution of 4 milliseconds. 

The experiment was conducted in a semi dark room of the Department of Neurobiopsychology at the University of Osnabr\"uck. The experimenter was present during all trials.  Stimuli were presented at a resolution of 1024x768 pixels and a refresh rate of 100 Hz on a Samsung Syncmaster 1100DF 2004 CRT-monitor. Settings of the monitor were readjusted consistently before each experiment to ensure equivalent viewing conditions. 

The participants were instructed to watch the images freely. Subjects were level with the center of the screen at a distance of 57 cm from the head. The pictures were presented in random order in three blocks of 88, 88, and 87 images. Presentation time was 3000 milliseconds. After each image presentation a fixation cross was displayed in the middle of the screen to correct for drifts due to head movements for about half a second. After each blocks, participants could take short breaks. After breaks, or when drift corrections were too high, the system was recalibrated. 

\section{Experiments 3 and 4 (Bochum)}
Wolfgang Einh\"auser kindly provided us with the data of the experiment in Bochum \cite{Einhaeuser2006,WolfgangDiss}. This added to our collection of human and monkey fixations and gave us the possibility to enhance our comparison of monkeys and humans on occasion. We will shortly describe stimuli, participants, experimental setup and procedure in the eye tracking experiments conducted by Einh\"auser et alii and the differences between the short and big screen conditions. 

\subsection{Stimuli}
The images were 108 gray-scaled photographs of natural scenes, scaled-down to a resolution of 1024x768 pixels (bicubic interpolation). In these images, local luminance alterations were done in order to test the relationship of luminance contrast and fixations (described first in \cite{Einhaeuser2003}). 

\subsection{Monkeys}
Two rhesus monkeys (macaca mulatta) were used. Each performed 972 trials (images at 9 modification levels). One monkey was head-fixed, the other one was not for technical reasons. The monkeys were trained for one day to keep their gaze within the boundaries of the screen. During trials, they were rewarded with water and apple juice. Trials began when the monkeys had finished drinking the reward from previous trials. Eye position was monitored by the scleral search coil technique. 

\subsection{Human participants}
Seven undergraduate students (three male, four female) with normal or corrected-to-normal vision participated. The gaze direction was monitored using the head-mounted EyeLink I (SR Research Ltd., Mississauga, Ontario, Canada). Their head posture was supported by chin-wrenches. They were instructed to view the images carefully.\footnote{\cite{Steinwender} demonstrated that low-level features in conditions \textit{free viewing} and \textit{study carefully} are highly correlated, so this detail does not hamper comparisons to the other experiments. }{} All gave written informed consent. 

\subsection{Presentation}
As for the \textit{big screen condition}, the stimuli were presented for 6 seconds at a distance of 50 cm on a screen of 133x100 cm (106$^{\circ}$x90$^{\circ}$ of visual angle) projected onto from a projector. Part of the view was obstructed, so for feature analysis for the current thesis we took only the central 600x400 pixels (76$^{\circ}$x55$^{\circ}$), while for analysis that concerned eye movement properties, we included fixations outside the central region. One degree of visual angle in the center corresponds roughly to 7 pixels. (compare \cite{Einhaeuser2006})

As for the \textit{small screen condition}, the stimuli were presented for 6 seconds at a distance of 70 cm on a screen of 47x36 cm (37$^{\circ}$x28$^{\circ}$ of visual angle) projected onto from a projector. Part of the view was obstructed, so for feature analysis for the current thesis we took only the central 600x400 pixels (76$^{\circ}$x55$^{\circ}$), while for analysis that concerned eye movement properties, we included fixations outside the central region. One degree of visual angle in the center corresponds roughly to 25 pixels. The viewing conditions were similar to the setup of experiments 1 and 2. (compare \cite{WolfgangDiss})

\section{Data analysis}
All analysis was done using \textsc{Matlab} (Math Works, Inc., Mass) release 14, versions 7.0.0.19901, 7.0.1.24704 Service Pack 1, 7.0.4.352 Service Pack 2, and 7.1.0.183 Service Pack 3. 

Eye tracking data from experiment 2 were stored by the eye tracker in the European Data Format (EDF) and converted to event-tagged ASCII files with a converter program included in the Eyelink II software pack. These ASCII files included fixations and saccades together with their spatial and temporal statistics. We converted these files for use in \textsc{Matlab} into a matrix that included for all fixations the the X and Y coordinates, the calibration values, the pre-trial drift correction, start-time, stop-time, participant index, fixation number within trial, and image index.\footnote{The conversion from EDF to \textsc{Matlab} format was done using function edf2mat.m \ref{ssec:.:edf2mat}.}{} 

The data from the monkey experiment (experiment 1) were converted to the same format (excluding calibration values and drift corrections that were unknown). We tried out different definitions of fixations (cf. \cite{Tichacek}) and found confirmed that the definition as per \cite{Lierhaus} resulted in the best matches (see \ref{fixations}). 

Among the for statistical methods are notable especially the \emph{Kruskal-Wallis one-way analysis of variance} (in the following denoted by KW), a nonparametric version of the analysis of variance test (ANOVA), for correlations, in some cases of normal distributions of data, the \emph{Pearson product-moment correlation coefficient}\footnote{We will remark, when Pearson product-moment correlations were used}{} for detection of linear dependencies and (mostly) the \emph{Spearman's rank correlation coefficient}, which is equivalent to Pearson correlation on ranks and does not require assumptions about the distributions. Further employed was the \emph{Kolmogorov-Smirnov test}. Of the mentioned methods, except for the Spearman's rank correlation coefficient, standard \textsc{Matlab} functions were used. For the Spearman's rank correlations we are grateful to Alois Schloegl's function \footnote{\url{http://www.koders.com/matlab/}}{}  As for conditional feature probabilities that were collected on the equalized bins of the image statistics, we computed the \emph{entropy} of the functions. For the computation of predictiveness of features we calculated \emph{mutual information}. 

 \section{Definitions}

\subsection{Fixation and saccade}
\label{fixations}
As for experiments 1 and 2, we employed two different definitions for fixations in the monkey and the human data. For monkeys, two rules applied. A fixation had to last for at least 90 ms. During the fixation time the distance of each position to its tenth successor should not exceed 0.24$^{\circ}$ (5 pixels). \cite{Lierhaus}

For humans, we used the fixations that were selected by EyeLink II (mode \textit{cognitive experiment}). Here fixations are defined as the inverse of a saccade. The onset of a saccade is detected when an eye acceleration exceeded a threshold of 8 $\frac{degrees}{millisesecond^2}$ and the eye shifted position by at least 0.1$^{\circ}$ (5 pixels). During a saccade, eye velocity had to be above 30 $\frac{degrees}{second}$. 

Following these two definitions, humans averaged around 8.81 fixations per image and monkeys around 11.64 fixations per image. We had a total of 64629 fixations for humans and 3491 fixations for monkeys from experiments 1 and 2. 

As for experiments 3 and 4, for the monkeys, at a sampling rate of 500Hz, the velocity threshold for saccades was 2.7 with a minimal duration of a fixation of 10. For the humans, at a sampling rate of 250Hz, velocity threshold was set at 5.4 with a minimal duration of 5.\footnote{In other words, these values correspond approximately to a velocity threshold of $\frac{1300 pixels}{second}$ and a minimal duration of 20 milliseconds.}{} These definitions obtained in the big screen condition a total of 569 monkey fixations and 596 human fixations (if not restricted to center), and in the small screen condition a total of 536 monkey fixations and 503 human fixations. 

As for the temporal characteristics of eye movements, we restricted ourselves to experiments 1 and 2, since we did not have data for experiments 3 and 4. In order to ignore effects of last saccades and last fixations in a trial, these are ignored. For saccade durations, we omitted all values that were three standard deviations above the median. For fixation durations, saccade length, and saccade speed we filtered out values four standard deviations above the median.\footnote{function getbasiceyemovements.m  \ref{ssec:.:getbasiceyemovements}}{} For monkeys, this concerned not more than two percent of eye movements, for humans up to a maximum of three percent were discarded. In the analysis, always fixation duration at targets and movements to saccade targets were evaluated (as opposed to movements from target). 
    
\subsection{Feature contrast}
Human fixations that had a calibration error above 0.3$^{\circ}$ or that were succeeded by a drift correction of two standard deviations above the median of all drift corrections in all experiments were discarded.\footnote{Drift corrections had a median of 0.37$^{\circ}$ and a standard deviation of 0.52$^{\circ}$.}{} 7.42 percent of fixations exceeded the calibration error threshold and the drift correction threshold was exceeded in 2.97 percent of fixations. We always took fixations of the eye that was better calibrated. 

Feature contrasts were computed using weighted neighborhood function for correction of calibration errors.\footnote{Function fcext.m \ref{ssec:.:fcext} }{} Before calculation of luminance and texture contrasts, standard deviations were normalized by the mean luminance of the image. We extracted all feature values weighted by vicinity to the measured point of the eye tracker in order to compensate for inaccuracies of the eye trackers. Therefore, we put Gaussians of half width at full maximum of 1 $^{\circ}$ with center on the measured points and normalized the Gaussians to sum one, so they expressed an estimation of likelihood increasing with centricity. Inner matrix multiplication resulted in a weighted mean that should give a better differentiated picture of the feature contrasts than simply taking the feature contrast at fixation points as done in many studies, taking the mean or median of a patch, or taking the maximum in a patch \cite{Tatler2005}. 

For feature analysis, in order to eliminate the effect of image boundaries, we had to cut off fixations that were too close to the border. For luminance contrast in the case of Tang's images, black borders extended to roughly 28 pixels. So we cut off at fixations that came within 30 pixels close to the border for luminance contrast,  we chose to cut off the 95 border pixels in the case of texture contrast, and  for barness we chose a cut-off border of 50 pixels.\footnote{These cut-offs affected roughly 3, 10, and 5 percent of monkey fixations, respectively, and 0.3, 3, and 0.7 percent of human fixations.}{}

As far as feature analysis was concerned independently of temporal behavior, first fixations in a trial were discarded in order to omit fixations that resulted as artifact from the calibration procedure. This was especially important for not distorting comparisons of monkeys and humans. 

\subsection{Luminance contrast}
Luminance contrast was defined as the variance of luminance within a patch (a rectangular patch for practical purposes) divided by the mean intensity of the image. \cite{ReinagelZador} For analysis, images were normalized, i.e. the standard deviation was divided by the mean luminance of the image. Luminance contrast was (for most purposes, if not stated differently) calculated in patches of 1 $^{\circ}$ of visual angle. 

\subsection{Texture contrast}
By the definition given in \cite{Einhaeuser2006}, texture contrast is the luminance contrast of the luminance contrast of an image. The texture contrast of a patch is the standard deviation in the luminance contrast values in the patch standardized by the luminance contrast mean of the image. \cite{Parkhurst2004} suggested an extension of the saliency map model, predicting a tenfold increased effect of texture contrast compared to luminance contrast. This emphasizes its relative importance. Texture contrast was calculated from patches of five degrees on a luminance map of one degree, because the effect of texture contrast characteristically extended to that size (cf \cite{Acik}). 

\subsection{Barness}
According to the definition in \cite{Einhaeuser2002} barness was computed. Independent of the orientation of the bar, the barness feature value of a patch is zero if the patch had no bar like structure, 1 for a perfect bar. (cf. \cite{Schumann})

\subsection{Control contrasts}
In eye-tracking experiments, participants show a bias towards the center (compare figure \ref{pdfs} and figure \ref{pdfs}). Among the reasons for this are pre-trial fixations crosses and the \textit{photographer's bias}, a tendency to put objects of high contrasts in the center of a photo. Therefore, when comparing feature values at fixation points, actual fixations, to other feature values in the image, the higher contrast values in the center would most likely skew the data. 

At first feature values at random points were compared to those of fixation points. As a method to compensate for this bias, feature values of fixation points on the image were compared to feature values on the image at the location of fixation points on other images. We followed this approach here. We defined control fixations as all fixations of a participant done on all images but the actual image and accordingly control feature contrasts as the feature contrasts at control fixations. The same definition was applied in other papers and theses of our group (e.g. \cite{Schall}). This definition is more restrictive than that in \cite{Schumann}, who pooled fixation points and calculated controls as the values resulting from this pool. 

Choosing a participant-specific definition of control fixations, we eliminated biases that could be due to individual viewing preferences. The statistical data were sufficient for such a step as roughly 10 fixations on 262 images each were available. Tatler et al. (2005) used a similar definition but chose randomly among the control fixations when comparing to actual fixations. For computation of effect size and size of elevation of feature values at fixation points we took the median\footnote{We found that most of our distributions did not resemble bell curves (normal distributions) so we preferred the median to the mean in order to reduce the influence of outliers.}{} of all actual and control fixations, respectively, to increase statistical power, when comparing actual fixations of a particular subject on an image to control fixations by that subject not on the image. 